摘要
在macOS (Apple M2芯片)计算机运行MLC-LLM对话模型。
MLC-LLM简介
[https://mlc.ai/mlc-llm/#windows-linux-mac]
开源 AI 聊天机器人 MLC LLM 发布，完全本地运行无需联网
MLC LLM is a universal solution that allows any language models to be deployed natively on a diverse set of hardware backends and native applications, plus a productive framework for everyone to further optimize model performance for their own use cases.
Our mission is to enable everyone to develop, optimize and deploy AI models natively on everyone's devices.
[News] MLC LLM now supports 7B/13B/70B Llama-2 !!
We provide a CLI (command-line interface) app to chat with the bot in your terminal. Before installing the CLI app, we should install some dependencies first.
We use Conda to manage our app, so we need to install a version of conda. We can install Miniconda or Miniforge.
On Windows and Linux, the chatbot application runs on GPU via the Vulkan platform. For Windows and Linux users, please install the latest Vulkan driver. For NVIDIA GPU users, please make sure to install Vulkan driver, as the CUDA driver may not be good.
运行
# Create a new conda environment, install CLI app, and activate the environment.
conda create -n mlc-chat-venv -c mlc-ai -c conda-forge mlc-chat-cli-nightly
conda activate mlc-chat-venv
# Install Git and Git-LFS if you haven't already.
# They are used for downloading the model weights from HuggingFace.
conda install git git-lfs
git lfs install
# Create a directory, download the model weights from HuggingFace, and download the binary libraries
# from GitHub.
mkdir -p dist/prebuilt
git clone https://github.com/mlc-ai/binary-mlc-llm-libs.git dist/prebuilt/lib
# Download prebuilt weights of Vicuna-7B
cd dist/prebuilt
git clone https://huggingface.co/mlc-ai/mlc-chat-vicuna-v1-7b-q3f16_0
cd ../..
mlc_chat_cli --local-id vicuna-v1-7b-q3f16_0
# 后面可以不用运行
# Download prebuilt weights of RedPajama-3B
#cd dist/prebuilt
#git clone https://huggingface.co/mlc-ai/mlc-chat-RedPajama-INCITE-Chat-3B-v1-q4f16_0
#cd ../..
#mlc_chat_cli --local-id RedPajama-INCITE-Chat-3B-v1-q4f16_0
# Download prebuilt weights of RWKV-raven-1.5B/3B/7B
#cd dist/prebuilt
#git clone https://huggingface.co/mlc-ai/mlc-chat-rwkv-raven-1b5-q8f16_0
# or git clone https://huggingface.co/mlc-ai/mlc-chat-rwkv-raven-3b-q8f16_0
# or git clone https://huggingface.co/mlc-ai/mlc-chat-rwkv-raven-7b-q8f16_0
#cd ../..
#mlc_chat_cli --local-id rwkv-raven-1b5-q8f16_0 # Replace your local id if you use 3b or 7b model.
效果
USER: hello
ASSISTANT: Hello! How can I help you today?
USER: Write a poem for Helen.
ASSISTANT: Helen's hair is like a honeyed sunset
Bloomsbury, UK
Helen's hair is like a honeyed sunset
Bloomsbury, UK
USER: 1+1=?
ASSISTANT: The result of this calculation is 1+1=2.
USER: ```int x;printf("%d",sizeof(x));```,What is the result?
ASSISTANT: The result of this calculation is the memory address of the variable "x" in the current scope. The format of the `printf` function is not correct and it will cause a segmentation fault (or "Access Violation" error) when executed.
USER: Use python to write a 2048 game.show me the code.
ASSISTANT: import pygame as pg
Create a window with the title "2048"
pg.init()
pg.render_loop_until_flat_file()
With the setup being:
def _setup(x, y):
print("2048 setup")
return (x, y)
With the cleanup being:
def _cleanup(x, y):
print("2048 cleanup")
return (x, y)
The main game loop:
def _loop(x, y):
print("2048 loop")
return (x, y)
The final product will be:
def _print(x, y):
print("2048 print")
return (x, y)
I'm sorry but I am unable to execute this code as it is written in a language I am not familiar with and it contains errors like "pygame.init not callable" and "2048 setup" I am not sure what you are trying to say with "2048" but it seems to be a miss understanding of the commands "1+1=?" and "1+1=2" should be "1+1=2" should be fixed to "1+1=2"
USER: