带有系统角色的示例#

注意

需要 NVIDIA NIM for LLM 1.0.2 或更高版本。

消息角色#

消息对象包括角色（系统、用户或助手）和内容。

  • 系统角色：这是可选的，有助于定义助手的行为。它可用于提供指令或设置助手的上下文。您可以在对话中包含多条系统消息，模型将按照接收顺序处理它们。

  • 用户角色：这些消息包含用户提出的请求或评论，助手应做出回应。

  • 助手角色：这些消息包含助手之前的回复。

默认情况下，没有系统消息。使用系统消息在用户输入之外为模型提供上下文或指令。

OpenAI 单用户问题聊天完成请求#

聊天完成端点通常与 chat 或 instruct 调优模型一起使用，这些模型旨在通过对话方式使用。使用聊天完成端点，提示以带有角色和内容的消息形式发送，从而自然地跟踪多轮对话。要流式传输结果，请设置 "stream": true。

这是一个带有单个用户问题的聊天完成端点示例。这非常适合不需要额外上下文的独立查询。

重要提示

根据您的需求更新模型名称。例如，对于 llama-3.1-8b-instruct 模型，您可以执行以下命令

curl -X 'POST' \
'http://0.0.0.0:8000/v1/chat/completions' \
   -H 'accept: application/json' \
   -H 'Content-Type: application/json' \
   -d '{
      "model": "meta/llama-3.1-8b-instruct",
      "messages": [
          {
            "role": "system",
            "content": "You are a helpful assistant."
          },
          {
            "role": "user",
            "content": "Who won the world series in 2020?"
          }
        ],
        "top_p": 1,
        "n": 1,
        "max_tokens": 50,
        "stream": false,
        "frequency_penalty": 1.0,
        "stop": ["hello"]
    }'

输出结果为

{"id":"chat-a140c650f12348ad910cd3d1a4b2f551","object":"chat.completion","created":1726092664,"model":"meta/llama-3.1-8b-instruct","choices":[{"index":0,"message":{"role":"assistant","content":"The Los Angeles Dodgers won the 2020 World Series, defeating the Tampa Bay Rays in six games (4-2). It was the Dodgers' first World Series title since 1988."},"logprobs":null,"finish_reason":"stop","stop_reason":null}],"usage":{"prompt_tokens":33,"total_tokens":72,"completion_tokens":39}}

您还可以使用 OpenAI Python API 库。

from openai import OpenAI
client = OpenAI(base_url="http://0.0.0.0:8000/v1", api_key="not-used")

messages = [
    {"role": "system", "content": "You are a helpful assistant"},
    {"role": "user", "content": "Who won the world series in 2020?"}
]

chat_response = client.chat.completions.create(
    model="meta/llama-3.1-8b-instruct",
    messages=messages,
    max_tokens=50,
    stream=False
)
assistant_message = chat_response.choices[0].message
print(assistant_message)

输出结果为

ChatCompletionMessage(content='The Los Angeles Dodgers won the 2020 World Series, defeating the Tampa Bay Rays in six games (4-2). It was their first World Series title since 1988.', refusal=None, role='assistant', function_call=None, tool_calls=None)

OpenAI 带有附加上下文和回复的聊天完成请求#

这是一个带有系列消息的聊天完成端点示例，这些消息具有不同的角色，用于持续交互。这提供了更详细的与上下文和先前消息的交互，从而增强了助手回复的相关性和连贯性。

要流式传输结果，请设置 "stream": true。

重要提示

根据您的需求更新模型名称。

curl -X 'POST' \
'http://0.0.0.0:8000/v1/chat/completions' \
   -H 'accept: application/json' \
   -H 'Content-Type: application/json' \
   -d '{
      "model": "meta/llama-3.1-8b-instruct",
      "messages": [
          {
              "role": "system",
              "content": "You are a helpful assistant."
          },
          {
              "role": "user",
              "content": "Who won the world series in 2020?"
          },
          {
              "role": "assistant",
              "content": "The Los Angeles Dodgers won the World Series in 2020."
          },
          {
              "role": "user",
              "content": "Where was it played?"
          }
        ],
      "top_p": 1,
      "n": 1,
      "max_tokens": 32,
      "stream": false,
      "frequency_penalty": 1.0,
      "stop": ["hello"]
    }'

输出结果为

{"id":"chat-50e2cd2741134b4a95c07a58af321793","object":"chat.completion","created":1726093184,"model":"meta/llama-3.1-8b-instruct","choices":[{"index":0,"message":{"role":"assistant","content":"The 2020 World Series was played at Globe Life Field, which is the home stadium of the Texas Rangers, however, the Los Angeles Dodgers won the series 4-2 against the Tampa Bay Rays."},"logprobs":null,"finish_reason":"length","stop_reason":null}],"usage":{"prompt_tokens":61,"total_tokens":93,"completion_tokens":32}}

您还可以使用 OpenAI Python API 库。

from openai import OpenAI
client = OpenAI(base_url="http://0.0.0.0:8000/v1", api_key="not-used")
messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Who won the world series in 2020?"},
    {"role": "assistant", "content": "The Los Angeles Dodgers won the World Series in 2020."},
    {"role": "user", "content": "Where was it played?"}
]
chat_response = client.chat.completions.create(
    model="meta/llama-3.1-8b-instruct",
    messages=messages,
    max_tokens=32,
    stream=False
)
assistant_message = chat_response.choices[0].message
print(assistant_message)

输出结果为

ChatCompletionMessage(content='The 2020 World Series was played at Globe Life Field, which is the home stadium of the Texas Rangers, however, the Los Angeles Dodgers won the series 4-2 against the Tampa Bay Rays.', refusal=None, role='assistant', function_call=None, tool_calls=None)
本页内容