微软推出Phi-4生成式AI模型 改进数学能力与训练数据质量

微软宣布，其Phi系列生成式AI模型迎来了新成员——Phi-4。与之前的版本相比，Phi-4在多个方面有所改进，尤其在数学问题的解决能力上表现突出，这得益于训练数据质量的提升。

截至周四晚上，Phi-4的访问权限相对有限，仅在微软最新推出的Azure AI Foundry开发平台上可用，且仅限于微软研究许可协议下的研究用途。

图源备注：图片由AI生成，图片授权服务商Midjourney

Phi-4是微软推出的最新小型语言模型，拥有140亿个参数。与其他小型模型如GPT-4o mini、Gemini2.0Flash和Claude3.5Haiku相当，这些小型模型通常具有更快的运行速度和更低的成本，而在过去几年中，小型语言模型的性能也在持续提升。

微软将Phi-4的性能提升归因于使用了“高质量合成数据集”以及来自人类生成内容的高质量数据集，并进行了未公开的训练后改进。

如今，许多人工智能实验室正在密切关注合成数据和后训练在提升模型性能方面的潜力。Scale AI的首席执行官Alexandr Wang在周四的推文中提到:“我们已经到达了训练前数据瓶颈。”这一说法也证实了近期关于该话题的一些报告。

此外，值得注意的是，Phi-4是微软AI副总裁Sébastien Bubeck离职后推出的首款Phi系列模型。Bubeck在微软AI领域扮演了重要角色，并且是Phi模型开发的关键人物。Bubeck于10月离开微软，加入了OpenAI。

该模型尚未在HF上发布，官方模型发布在https://ai.azure.com/explore/models/Phi-4/

追加 Colab 运行

!pip install -U bitsandbytes transformers accelerate

由于模型是14B参数，我只能用4bit运行

import torch 
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline, BitsAndBytesConfig

# torch.random.manual_seed(0) 

nf4_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
)

model = AutoModelForCausalLM.from_pretrained( 
    "NyxKrage/Microsoft_Phi-4",  
    # device_map="cuda",  
    # torch_dtype="auto",  
    # trust_remote_code=True,  
    quantization_config=nf4_config,
) 

tokenizer = AutoTokenizer.from_pretrained("NyxKrage/Microsoft_Phi-4") 

messages = [ 
    {"role": "system", "content": "You are a helpful AI assistant."}, 
    {"role": "user", "content": "Can you provide ways to eat combinations of bananas and dragonfruits?"}, 
    {"role": "assistant", "content": "Sure! Here are some ways to eat bananas and dragonfruits together: 1. Banana and dragonfruit smoothie: Blend bananas and dragonfruits together with some milk and honey. 2. Banana and dragonfruit salad: Mix sliced bananas and dragonfruits together with some lemon juice and honey."}, 
    {"role": "user", "content": "What about solving an 2x + 3 = 7 equation?"}, 
] 

pipe = pipeline( 
    "text-generation", 
    model=model, 
    tokenizer=tokenizer, 
) 

generation_args = { 
    "max_new_tokens": 500, 
    "return_full_text": False, 
    "temperature": 0.0, 
    "do_sample": False, 
} 

output = pipe(messages, **generation_args) 
print(output[0]['generated_text'])

Device set to use cuda:0
/usr/local/lib/python3.10/dist-packages/transformers/generation/configuration_utils.py:628: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.0` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`.
  warnings.warn(
/usr/local/lib/python3.10/dist-packages/bitsandbytes/nn/modules.py:451: UserWarning: Input type into Linear4bit is torch.float16, but bnb_4bit_compute_dtype=torch.float32 (default). This will lead to slow inference or training speed.
  warnings.warn(
To solve the equation \(2x + 3 = 7\), follow these steps:

1. **Subtract 3 from both sides** to isolate the term with \(x\):
   \[
   2x + 3 - 3 = 7 - 3
   \]
   \[
   2x = 4
   \]

2. **Divide both sides by 2** to solve for \(x\):
   \[
   \frac{2x}{2} = \frac{4}{2}
   \]
   \[
   x = 2
   \]

So, the solution is \(x = 2\).