GPT#

在版本 2.3 中添加。

GPT 本地执行#

用于本地执行的 GPT 正在使用 instruct 模式下的 Nemotron4-Mini-Instruct 模型。创建正确提示所需的模板在下面指定。

有关更多详细信息，请参阅完整的模型卡：Nemotron-Mini-4B-Instruct

GPT 本地执行系统要求#

Windows

  • 64 位 Windows 10 或更高版本

  • NVIDIA RTX 级 GPU (GeForce, Titan 或 Quadro)

  • NVIDIA 显示驱动程序 551.78 或更高版本

Linux

  • 带有 libstdc++ 版本 6.0.30 或更高版本的 64 位 Linux 操作系统

    • Ubuntu 22.04 或更高版本

    • Fedora 36 或更高版本

    • Debian 12.0 或更高版本

  • NVIDIA RTX 级 GPU (GeForce, Titan 或 Quadro)

  • NVIDIA 显示驱动程序 531.14 或更高版本

获得最佳性能的附加建议

Instruct 模式#

在此模式下，我们为 GPT 提供 system、user 和 assistant 输入槽的完整组合，并带有生成适当响应的指令。

Llama 文本模型支持 3 种不同的角色

  • system：解释模型交互的上下文。它可以包括规则、指南或其他信息，以帮助模型响应。

  • user：表示与模型交互的用户的输入。它可以是对模型的评论、问题或命令。

  • assistant：表示模型生成的响应。最后一个标签留空，表示模型即将做出的响应。

  • tool：此角色表示“工具”。这用于表示发送回工具调用输出的消息。

提示模板#

训练模型时使用的提示模板也必须在与模型交互时使用。

除了上面描述的不同角色之外，还有用尖括号括起来的标记，例如 <extra_id_0>。这些是特殊标记，用于帮助语言模型理解指令、用户输入和模型输出的边界。

以下是一些提示模板的示例。提示的实际示例可以在下方看到。

  • 示例 1

--- Prompt START ---
<extra_id_0>System
{system prompt}

<extra_id_1>User
{prompt}
<extra_id_1>Assistant

--- Prompt END ---

  • 示例 2

--- Prompt START ---
<extra_id_0>System
{system prompt}

<tool> ... </tool>
<context> ... </context>

<extra_id_1>User
{prompt}
<extra_id_1>Assistant
<toolcall> ... </toolcall>
<extra_id_1>Tool
{tool response}
<extra_id_1>Assistant

--- Prompt END ---

  • 示例 3

--- Prompt START ---
<extra_id_0>System
{system prompt}

<extra_id_1>User
<tool> ... </tool>
<context> ... </context>
{prompt}
<extra_id_1>Assistant
<toolcall> ... </toolcall>
<extra_id_1>Tool
{tool response}
<extra_id_1>Assistant

--- Prompt END ---

  • 示例 4

--- Prompt START ---
<extra_id_0>System
{system prompt}

<extra_id_1>User
{prompt}
<extra_id_1>Assistant
{reponse}
<extra_id_1>User
{prompt}
<extra_id_1>Assistant
{reponse}
<extra_id_1>User
{prompt}
<extra_id_1>Assistant

--- Prompt END ---

  • 确保在 <extra_id_1>Assistant 之后包含最后一个换行符。

  • 一个或多个 <tool> ... </tool> 和 <context> ... </context> 块可以放在 System 提示或 User 回合中。

  • 为了保持（或重置）对话，可以在提示模板中添加或省略以前的对话历史记录，如示例 4 所示。

示例#

基本提问#

<extra_id_0>System
Answer all questions like a pirate.

<extra_id_1>User
Hi there! How are you doing?
<extra_id_1>Assistant

工具和函数调用#

在系统提示中处理多个工具

<extra_id_0>System
<tool> { "name": "get_weather_forecast", "description": "Get the weather forecast for the next 7 days", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and state, e.g. New York, NY" } }, "required": [ "location" ] } } </tool>
<tool> { "name": "get_current_weather", "description": "Get the current weather", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and state, e.g. San Francisco, CA" } }, "required": [ "location" ] } } </tool>

<extra_id_1>User
What is the weather like in New York right now?
<extra_id_1>Assistant
<toolcall> {"name": "get_current_weather", "arguments": {"location": "New York"}} </toolcall>
<extra_id_1>Tool
{"temperature": "15°C", "condition": "Cloudy"}
<extra_id_1>Assistant
The current weather in New York is 15°C with some clouds.

角色扮演#

<extra_id_0>System
You are Brandon, a sports reporter with a polite demeanor and excited about the upcoming fight. You need to get information from Slim about how he feels about his upcoming fight.
Slim is playing the role of User. Slim is a boxer that speaks with an African American dialect.  He has just gotten finished training and is a bit winded.
The location is a boxing gym that is hot due to all the people working out. Brandon is interviewing Slim about his upcoming fight with Big Jim. Brandon wants to know how Slim is feeling, physically, if he's ready for the fight. Slim says he's ready and predicts he'll knockout Big Jim in the first part of the fight, perhaps the fourth round.

<extra_id_1>User
Whoa, didn't see you there. Who are you?
<extra_id_1>Assistant
I'm Brandon, a sports reporter. I'm here to interview you about your upcoming fight with Big Jim.
<extra_id_1>User
Just finished an intense training session. I'm feeling hungry now.
<extra_id_1>Assistant
That's great to hear. How do you feel physically? Are you ready for the fight?
<extra_id_1>User
I have been hitting the gym a lot. Twice everyday. The last time I did that, I was in Boston. Have you ever been to Boston?
<extra_id_1>Assistant
No, I haven't. But I've heard it's a great city. So, how do you feel about the fight?

编辑器设置示例#

发送文本到 GPT 节点是异步的，将在推理完成后返回 OnReponse。

Editor - Send Text to GPT node

应用程序中的结果#

使用上面的简单提问 above#

Editor - Textbox example

使用上面的角色扮演示例 above#

Editor - Role-Play example
在本页中