函数调用#

您可以使用函数调用（也称为工具调用）将 NIM 连接到外部工具和服务。通过提供可用函数的列表，NIM 可以选择输出相关函数的函数参数，您可以执行这些参数以使用相关的外部信息来增强提示。

函数调用通过 tool_choice、tools 和 parallel_tool_calls 参数进行控制。只有以下模型支持函数调用，并且只有这些模型的一个子集支持并行工具调用。

支持的模型#

模型

支持并行工具调用

最新支持的 NIM 版本

Llama-3.1-8B-Instruct

否

1.2

Llama-3.1-70B-Instruct

否

1.2

Llama-3.1-405B-Instruct

否

1.2

Mistral NeMo 12B Instruct

是

1.1.2

Mistral 7B Instruct v0.3

是

1.1.2

参数#

要使用函数调用，请修改 tool_choice、tools 和 parallel_tool_calls 参数。

参数

描述

tool_choice

指定模型应如何选择工具。有四个选项："none"、"auto"、"required" 或命名工具选择。需要同时设置 tools。

tools

定义模型可以调用的函数的工具对象列表。需要同时设置 tool_choice。

parallel_tool_calls

布尔值（True 或 False），指定是否并行进行工具调用。默认为 False。需要模型支持此功能。

tool_choice 选项#

  • "none"：禁用工具的使用。

  • "auto"：允许模型决定是否使用工具以及使用哪些工具。

  • "required"：强制模型使用工具，但模型选择使用哪个工具。

  • 命名工具选择：强制模型使用特定的工具。它必须采用以下格式

    {
  "type": "function",
  "function": {
    "name": "name of the tool goes here"
  }
}

注意：只有在同时设置 tools 时，才能设置 tool_choice，反之亦然。这些参数协同工作，以定义和控制模型响应中工具的使用。有关这些参数及其用法的更多信息，请参阅 OpenAI API 文档。

示例工作流程#

这些示例展示了使用 NIM 进行函数调用的各种方法

  1. 基本函数调用：演示如何将单个函数与自动工具选择一起使用。

  2. 多个工具：展示如何提供多个工具，包括一个没有参数的工具。

  3. 强制工具使用：说明如何强制模型使用特定的工具。

  4. 并行工具调用：举例说明如何将并行工具调用与支持它的模型一起使用。

1. 基本函数调用#

此示例展示了如何将单个函数与自动工具选择一起使用。

from openai import OpenAI

client = OpenAI(base_url="http://0.0.0.0:8000/v1", api_key="not-used")
MODEL_NAME = "meta/llama-3.1-70b-instruct"

# Define available function
weather_tool = {
    "type": "function",
    "function": {
        "name": "get_current_weather",
        "description": "Get the current weather",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "The city and state, e.g. San Francisco, CA"
                },
                "format": {
                    "type": "string",
                    "enum": ["celsius", "fahrenheit"],
                    "description": "The temperature unit to use. Infer this from the user's location."
                }
            },
            "required": ["location", "format"]
        }
    }
}

messages = [
    {"role": "user", "content": "Is it hot in Pittsburgh, PA right now?"}
]

chat_response = client.chat.completions.create(
    model=MODEL_NAME,
    messages=messages,
    tools=[weather_tool],
    tool_choice="auto",
    stream=False
)

assistant_message = chat_response.choices[0].message
messages.append(assistant_message)

print(assistant_message)
# Example output:
# ChatCompletionMessage(content=None, role='assistant', function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_abc123', function=Function(arguments='{"location": "Pittsburgh, PA", "format": "fahrenheit"}', name='get_current_weather'), type='function')])

# Simulate external function call
tool_call_result = 88
tool_call_id = assistant_message.tool_calls[0].id
tool_function_name = assistant_message.tool_calls[0].function.name
messages.append({"role": "tool", "content": str(tool_call_result), "tool_call_id": tool_call_id, "name": tool_function_name})

chat_response = client.chat.completions.create(
    model=MODEL_NAME,
    messages=messages,
    tools=[weather_tool],
    tool_choice="auto",
    stream=False
)

assistant_message = chat_response.choices[0].message
print(assistant_message)
# Example output:
# ChatCompletionMessage(content='Based on the current temperature of 88°F (31°C) in Pittsburgh, PA, it is indeed quite hot right now. This temperature is generally considered warm to hot, especially if accompanied by high humidity, which is common in Pittsburgh during summer months.', role='assistant', function_call=None, tool_calls=None)

2. 多个工具#

您还可以为 tools 定义多个工具，包括没有参数的工具，例如下面的 time_tool。

weather_tool = {
    # ... (same as in the previous example)
}

time_tool = {
    "type": "function",
    "function": {
        "name": "get_current_time_nyc",
        "description": "Get the current time in NYC.",
        "parameters": {}
    }
}

messages = [
    {"role": "user", "content": "What's the current time in New York?"}
]

chat_response = client.chat.completions.create(
    model="meta/llama-3.1-70b-instruct",
    messages=messages,
    tools=[weather_tool, time_tool],
    tool_choice="auto",
    stream=False
)

assistant_message = chat_response.choices[0].message
print(assistant_message)
# Example output:
# ChatCompletionMessage(content=None, role='assistant', function_call=None, tool_calls=[
#     ChatCompletionMessageToolCall(id='call_ghi789', function=Function(arguments='{}', name='get_current_time_nyc'), type='function')
# ])

# Process tool calls and generate final response as in the previous example

3. 命名工具使用#

此示例强制模型使用特定的工具。

chat_response = client.chat.completions.create(
    model="meta/llama-3.1-70b-instruct",
    messages=[{"role": "user", "content": "What's the weather in New York City like?"}],
    tools=[weather_tool],
    tool_choice={
        "type": "function",
        "function": {
            "name": "get_current_weather"
        }
    },
    stream=False
)

assistant_message = chat_response.choices[0].message
print(assistant_message)
# Example output:
# ChatCompletionMessage(content=None, role='assistant', function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_jkl012', function=Function(arguments='{"location": "New York, NY", "format": "fahrenheit"}', name='get_current_weather'), type='function')])

# Process tool call and generate final response as in the previous examples

4. 并行工具调用#

某些模型能够在一条消息中响应多个工具调用。此示例演示了使用支持并行工具调用的模型进行并行工具调用。

chat_response = client.chat.completions.create(
    model="mistralai/Mistral-7B-Instruct-v0.3",
    messages=[{"role": "user", "content": "What's the weather and time in New York?"}],
    tools=[weather_tool, time_tool],
    tool_choice="auto",
    parallel_tool_calls=True,
    stream=False
)

assistant_message = chat_response.choices[0].message
print(assistant_message)
# Example output:
# ChatCompletionMessage(content=None, role='assistant', function_call=None, tool_calls=[
#     ChatCompletionMessageToolCall(id='call_mno345', function=Function(arguments='{"location": "New York, NY", "format": "fahrenheit"}', name='get_current_weather'), type='function'),
#     ChatCompletionMessageToolCall(id='call_pqr678', function=Function(arguments='{}', name='get_current_time'), type='function')
# ])

# Process multiple tool calls in parallel
tool_results = []
for tool_call in assistant_message.tool_calls:
    if tool_call.function.name == "get_current_weather":
        # Simulate weather API call
        weather_result = "75°F"
        tool_results.append({"role": "tool", "content": weather_result, "tool_call_id": tool_call.id, "name": tool_call.function.name})
    elif tool_call.function.name == "get_current_time":
        # Simulate time API call
        time_result = "2:30 PM EDT"
        tool_results.append({"role": "tool", "content": time_result, "tool_call_id": tool_call.id, "name": tool_call.function.name})

# Add tool results to messages
messages.extend(tool_results)

# Generate final response based on all tool call results
# Note that not all models support parallel tool calls
chat_response = client.chat.completions.create(
    model="mistralai/Mistral-7B-Instruct-v0.3",
    messages=messages,
    tools=[weather_tool, time_tool],
    tool_choice="auto",
    stream=False
)

final_response = chat_response.choices[0].message
print(final_response)
# Example output:
# ChatCompletionMessage(content="In New York, the current weather is 75°F (23.9°C), which is quite pleasant. It's not too hot or cold. The current time in New York is 2:30 PM EDT (Eastern Daylight Time). It's mid-afternoon there right now.", role='assistant', function_call=None, tool_calls=None)
在本页中