下载SakuraLLM模型

鉴于显存为6G，下载20241012-Qwen2.5-1.5B-v1.0模型，去https://hf-mirror.com/SakuraLLM/Sakura-1.5B-Qwen2.5-v1.0-GGUF/tree/main下载gguf文件

编译llama.cpp

下载llama.cpp代码包

cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release

将build/bin/llama-server拷贝到下一节下载好后的Sakura_Launcher_GUI/llama目录下

运行Sakura启动器

下载PiDanShouRouZhouXD/Sakura_Launcher_GUI代码包，python main.py，程序运行起来之后，设置SakuraLLM模型的位置，启动翻译服务，即可访问http://localhost:8080，问一句，它翻译一句

批量化翻译

import openai

client=openai.OpenAI(base_url='http://127.0.0.1:8080/v1',api_key='1')
messages=[{'role':'system','content':'请将以下日文翻译成中文'}]

with open('test.txt',encoding='utf8') as f:
    contents=f.readlines()
print(contents)
for i in range(len(contents)):
    print(contents[i].strip())
    messages.append({'role':'user','content':contents[i].strip()})
    completion=client.chat.completions.create(model='sakura-1.5b-qwen2.5-v1.0-fp16',messages=messages)# 服务启动时会打印模型名称，那个即为model的值
    print(completion.choices[0].message.content)
    messages.append({'role':'assistant','content':completion.choices[0].message.content})

参考链接：
SakuraLLM/SakuraLLM

Sakura模型本地部署教程

PiDanShouRouZhouXD/Sakura_Launcher_GUI

ggerganov/llama.cpp

Build llama.cpp locally

LLaMA.cpp HTTP Server

创建于2501152208，修改于2501152208