首页 > 其他分享 >在项目里快速运行史上最强开源大模型「Llama 3.1」

在项目里快速运行史上最强开源大模型「Llama 3.1」

时间：2024-08-13 19:06:17浏览次数：5  
标签：chat assistant system content llama3.1 开源 role Llama 3.1

7月份，Meta正式发布迄今为止最强大的开源模型——Llama 3.1，包含了405B、70B和8B三个版本的模型。

 

作为开发者，如何在项目里快速运行LLaMa3.1大模型？

今天，我们结合英智未来自己的LLM推理API平台，给大家介绍一套免费实践流程。

免费使用地址：https://cognihub.baystoneai.com

方式一：不使用流式传输，直接让大模型返回最终结果请求：

# sh
export API_BASE_URL="https://api.baystoneai.com/cognihub/service/66b97dd5fb19cc7397384f42"export API_KEY="Sy0FY5eH1jOaKtcfVvIxN1zsgRWED4IgfoFA"
curl -X POST $API_BASE_URL/v1/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer $API_KEY" -d '{    "model": "llama3.1",    "stream": false,    "messages": [        {"role": "system", "content": "You are a helpful assistant."},        {"role": "user", "content": "英智人工智能智汇API是什么？"}    ]}'

应答：

{  "id": "chatcmpl-754",  "object": "chat.completion",  "created": 1723433933,  "model": "llama3.1",  "system_fingerprint": "fp_ollama",  "choices": [    {      "index": 0,      "message": {        "role": "assistant",        "content": "英智人工智能智汇API（Application Programming Interface）是指一种开放的开发接口，让第三方服务或应用能够访问和调用英智的人工智能技术。通过这些API，开发者可以整合英智的人工智能功能，如自然语言处理、图像识别和情绪分析，来提高自己的产品或服务。\n\n例如，一家移动应用开发商可能会使用英智的语音识别API，让用户能够通过语音说话给应用进行交互。或者，一家教育平台可能会利用英智的情绪分析API，对学生的情绪变化进行实时监测和反馈等应用。\n\n通过API接口，第三方开发者可以使用英智的人工智能能力，为更加强大和有趣的应用或服务提供支持。这也促进了应用和服务之间的快速创新与集成。"      },      "finish_reason": "stop"    }  ],  "usage": {    "prompt_tokens": 31,    "completion_tokens": 186,    "total_tokens": 217  }}
方式二：设置stream（流式传输）请求： 
# sh
export API_BASE_URL="https://api.baystoneai.com/cognihub/service/66b97dd5fb19cc7397384f42"export API_KEY="Sy0FY5eH1jOaKtcfVvIxN1zsgRWED4IgfoFA"
curl -X POST $API_BASE_URL/v1/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer $API_KEY" -d '{    "model": "llama3.1",    "stream": true,    "messages": [        {"role": "system", "content": "You are a helpful assistant."},        {"role": "user", "content": "英智人工智能智汇API是什么？"}    ]}'
应答：
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"英"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"智"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"人"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"工"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"智能"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"智"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"汇"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"API"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"是一"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"种"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"可以"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"为"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"网站"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"、"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"应用"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"程序"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"和"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"其他"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"设备"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"提供"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"话"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"语"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"建议"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"、"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"文"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"本"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"分析"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"等"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"服务"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"的"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434273,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"软件"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434273,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"接"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434273,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"口"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434273,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"。"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434273,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":"stop"}]}
data: [DONE]

下面，给大家介绍一下「英智LLM推理API平台」，以及平台具有的优势。

 

英智LLM推理API平台

英智LLM推理API是一种基于人工智能大语言模型的推理API服务，它可以帮助开发者简单灵活地使用市面上的所有人工智能大语言模型，为企业定制专属的智能化解决方案。目前已支持包括Llama 3.1、Mistral Large 2、Qwen 2等在内的主流开源大模型的最新版本，同时兼容OpenAI API。

 

 

平台优势

  • 免费注册：申请服务的免费套餐后可获取专属您的“API_KEY”，即可接入本服务来免费使用API。

  • 在线使用：支持在线使用开源大模型。

  • 兼容能力强：提供API调用方式，与OpenAI API接口兼容(只要支持OpenAI API的应用，都可以使用)。

  • 推理速度快：基于NVIDIA高性能GPU算力，推理速度更快。

  • 自然语言处理：通过对大量文本数据进行分析和理解，实现对自然语言的智能处理，如情感分析、文本分类、关键词提取等。

  • 语义理解：通过对自然语言的深入理解，实现对用户意图的准确识别，从而提供更精准的信息服务。

  • 智能问答：通过对大量知识库的学习和理解，实现对用户提问的智能回答，提高用户满意度。

  • 文本生成：通过对大量文本数据的学习，实现对特定主题或场景的文本生成，如新闻稿、产品描述等。

  • 语音识别与合成：通过对语音数据的分析，实现对语音的识别和合成，提高人机交互的自然度和便捷性。

  • 图像识别与处理：通过对图像数据的分析和理解，实现对图像内容的识别和处理，如人脸识别、物体检测等。


免费使用，请访问：https://cognihub.baystoneai.com

加“英智未来”好友，免费体验更多AI产品

 

标签：chat,assistant,system,content,llama3.1,开源,role,Llama,3.1
From： https://www.cnblogs.com/ai2nv/p/18357542

相关文章

  • 本地通过python运行AI大语言模型LLaMa2
    什么是Llama2Llama全称largelanguagemodel, MetaAI公司2023年发布的开源AI大型语言模型，参数7B~65B。最新版本为Llama3。更为详细的介绍可参考，LLaMA-维基百科，自由的百科全书(wikipedia.org)。什么是llama.cpp?一个开源c++库，用c++重写了LLaMa的推理代码，可用于加载......
  • 了解VSCode：一款功能强大的开源代码编辑器
    VisualStudioCode（简称VSCode）是由微软开发的一款免费、开源的源代码编辑器。它以其强大的功能、丰富的插件生态系统、跨平台兼容性以及出色的用户体验，成为了广大开发者的首选工具。以下是对VSCode的详细介绍，涵盖其特点、功能、安装与配置、以及扩展生态等方面。一、VSCode的......
  • 一些知名开源软件的编程语言比例
    记于2024年8月，数据来自github仓库。 github仓库的编程语言比例图：下面按主编程语言进行分类展示。 C++openjdkJava74.6%C++13.8%C7.2%Assembly2.8%Objective-C0.4%HTML0.3%Other0.9%MySQL-serverC++81.2%C9.5%Java2.4%C#1.9%CMake0.7%BitBake0......
  • 开源的 P2P 跨平台传文件应用「GitHub 热点速览」
    就在上周，发完那篇文章之后不久，我就有幸获得了GitHubModels服务公测的访问权限，所以就体验了一下Playground聊天功能。起初，我以为这是“微软菩萨”降临，但玩了一圈下来，发现实际效果并没有那么惊艳，还没上周热门的开源项目有趣。例如，基于WebRTC的文件传输平台ShareDrop，只需......
  • 【GeoScene】三、GeoScene Enterprise 3.1 在银河麒麟V10上安装部署
    前言：哈哈哈，想到什么就写什么了，没有顺序可言系统要求：性能越高越好......
  • 抖音矩阵系统源码搭建，矩阵系统贴牌，矩阵工具开源
    在当今的社交媒体时代，抖音的影响力日益增强。对于许多开发者和企业来说，搭建一个抖音矩阵系统源码具有重要的战略意义。本文将为您详细介绍抖音矩阵系统源码搭建的全过程。今天，抖去推矩阵系统通过为商家提供矩阵管理、内容创作、视频生产、数据统计、等一站式SaaS解决方案。解......
  • 为了给Javaer落地DDD，我们不得不写开源组件
    本文上回书接《这是DDD建模最难的部分（其实很简单）》，欢迎关注我的同名公众号。https://mp.weixin.qq.com/s/HZKMLF0_I10iczzp2mAR-w 故事背景2013年中，我们的Java后端团队为了落地DDD，全面引入了dotnet技术栈，具体过程和成果，可以看我的B站频道《Java8到.NET8，团队升级报告-第......
  • 无缝融入，即刻智能[一]：Dify-LLM大模型平台，零编码集成嵌入第三方系统，42K+星标见证专属智
    无缝融入，即刻智能[一]：Dify-LLM大模型平台，零编码集成嵌入第三方系统，42K+星标见证专属智能方案1.Dify简介1.1功能情况Dify，一款引领未来的开源大语言模型（LLM）应用开发平台，革新性地融合了后端即服务(BackendasaService,BaaS）与LLMOps的精髓，为开发者铺设了一条从创意原型到高效生......
  • C#窗体自定义快捷操作键的实现 - 开源研究系列文章
          这次想到应用程序的窗体的快捷操作键的使用的问题。      上次发布过一个快捷键的例子（https://www.cnblogs.com/lzhdim/p/18342051），区别在于它是操作系统全局注册的热键，如果其它应用程序注册了对应的热键，那就会失效。此例子是对某个窗体里的按键的操作进行的......
  • 实用好软-----开源 很好用的计算机工具箱
    特点：软件版本：2.0软件特色：独立任务管理器，‘可以修复大部分的系统功能。默认以管理员模式运行。支持VMWARE、VBOX等虚拟机运行内置注册表。MBR防护支持系统：XP-11界面：   注意：0.由于本程序由易语言编写，某些杀毒软件（不包括火绒安全）对我的软件产品存在误报,我保证我的软件产......

赞助商

阅读排行

网站主页关于我们联系我们网站地图

本网站内容转载自其他媒体，侵权联系[admin##ips99.com]。

Copyright © 2020-2023 IPS99 版权所有 IPS99