加载模型及量化frommodelscopeimportAutoTokenizer,AutoModeltokenizer=AutoTokenizer.from_pretrained(model_dir,trust_remote_code=True)model=AutoModelForCausalLM.from_pretrained(model_dir,trust_remote_code=True).quantize(4).half().cuda()构造数据集