openbuddy 对于 llama3.1 8b 模型进行了少量的中文训练实现了不错的中文理解处理，以下是使用社区提供的gguf 格式，制作一个ollama 新模型
模型制作
- 下载模型
下载gguf 文件，推荐使用加速工具 - Modelfile
参考了llama3.1 的
FROM /home/models/openbuddy-llama3.1-8b-v22.1-131k-q4_k_m.gguf
TEMPLATE """{{ if .Messages }}
{{- if or .System .Tools }}<|start_header_id|>system<|end_header_id|>
{{- if .System }}
{{ .System }}
{{- end }}
{{- if .Tools }}
You are a helpful assistant with tool calling capabilities. When you receive a tool call response, use the output to format an answer to the orginal use question.
{{- end }}<|eot_id|>
{{- end }}
{{- range $i, $_ := .Messages }}
{{- $last := eq (len (slice $.Messages $i)) 1 }}
{{- if eq .Role "user" }}<|start_header_id|>user<|end_header_id|>
{{- if and $.Tools $last }}
Given the following functions, please respond with a JSON for a function call with its proper arguments that best answers the given prompt.
Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}. Do not use variables.
{{ $.Tools }}
{{- end }}
{{ .Content }}<|eot_id|>{{ if $last }}<|start_header_id|>assistant<|end_header_id|>
{{ end }}
{{- else if eq .Role "assistant" }}<|start_header_id|>assistant<|end_header_id|>
{{- if .ToolCalls }}
{{- range .ToolCalls }}{"name": "{{ .Function.Name }}", "parameters": {{ .Function.Arguments }}}{{ end }}
{{- else }}
{{ .Content }}{{ if not $last }}<|eot_id|>{{ end }}
{{- end }}
{{- else if eq .Role "tool" }}<|start_header_id|>ipython<|end_header_id|>
{{ .Content }}<|eot_id|>{{ if $last }}<|start_header_id|>assistant<|end_header_id|>
{{ end }}
{{- end }}
{{- end }}
{{- else }}
{{- if .System }}<|start_header_id|>system<|end_header_id|>
{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>
{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>
{{ end }}{{ .Response }}{{ if .Response }}<|eot_id|>{{ end }}"""
PARAMETER stop "<|start_header_id|>"
PARAMETER stop "<|end_header_id|>"
PARAMETER stop "<|eot_id|>"
- 构建
ollama create openbuddy-llama3.1 -f Modelfile
使用
测试的函数调用
- appv3.py
import openai
import json
def add(a, b):
return a + b
def sub(a, b):
return a - b
def send_email(userid, mailContent):
return f"send email to {userid} with content: {mailContent}"
def add_role(userid, roleid):
return f"add role {roleid} to user {userid}"
openai.api_key = "demo"
openai.base_url = "http://localhost:11434/v1/"
funcs = {
"add": add,
"sub": sub,
"send_email":send_email,
"add_role":add_role
}
tools = [{
"type": "function",
"function": {
"name": "add",
"description": "Add two numbers together",
"parameters": {
"type": "object",
"properties": {
"a": {"type": "number", "description": "First number"},
"b": {"type": "number", "description": "Second number"}
},
"required": ["a", "b"]
}
}
},
{
"type": "function",
"function": {
"name": "sub",
"description": "计算两个数的差",
"parameters": {
"type": "object",
"properties": {
"a": {"type": "number", "description": "First number"},
"b": {"type": "number", "description": "Second number"}
},
"required": ["a", "b"]
}
}
},
{
"type": "function",
"function": {
"name": "add_role",
"description": "给用户添加权限",
"parameters": {
"type": "object",
"properties": {
"userid": {"type": "string", "description": "用户id"},
"roleid": {"type": "string", "description": "角色id"}
},
"required": ["userid", "roleid"]
}
}
},
{
"type": "function",
"function": {
"name": "send_email",
"description": "给用户发送邮件",
"parameters": {
"type": "object",
"properties": {
"userid": {"type": "string", "description": "用户id"},
"mailContent": {"type": "string", "description": "邮件内容"}
},
"required": ["userid", "mailContent"]
}
}
}
]
response = openai.chat.completions.create(
stream=False,
model="rongfengliang/openbuddy-llama3.1",
messages=[
{"role": "system", "content": "你是一个人工智能助手，你可以帮助用户处理数据，发送邮件等，但是你必须使用中文"},
{"role": "user", "content": "给用户001发送数据处理已经完成的邮件通知同时添加角色0002"},
],
tools=tools,
tool_choice="auto",
)
message = response.choices[0].message
print(message)
if message.tool_calls:
result = response.choices[0].message.tool_calls[0].function
print(result)
func = funcs.get(result.name)
params = json.loads(result.arguments)
print(func(**params))
else:
print(message.content)
- 效果
说明
目前基于openbuddy支持中文的进行函数调用比原生的是好一些，原生的有时有出现莫名其妙的中文信息
参考资料
https://openbuddy.ai/
https://github.com/OpenBuddy/OpenBuddy
https://huggingface.co/OpenBuddy/openbuddy-llama3.1-8b-v22.1-131k
https://huggingface.co/sunnyyy/openbuddy-llama3.1-8b-v22.1-131k-Q4_K_M-GGUF