首页 > 其他分享 >【Coursera GenAI with LLM】 Week 2 PEFT Class Notes

【Coursera GenAI with LLM】 Week 2 PEFT Class Notes

时间:2024-03-14 11:13:29浏览次数:24  
标签:Week prompt parameters Notes Coursera weights LLM model LoRA

With PEFT, we only train on small portion of parameters!

What's using memory while training model?

  • Trainable weights
  • Optimizer states
  • Gradients
  • Forward Activations
  • Temporary memory

PEFT Trade-offs

  • Parameter Efficiency
  • Memory Efficiency
  • Model Performance
  • Training Speed
  • Inference Costs

PEFT Methods

  • Selective: select subset of initial LLM parameters to fine-tune
  • Re-parameterize: re-parameterize model weights using a low-rank representation. ex. LoRA
  • Additive: add trainable layers or parameters to model while keeping all of the original LLM weights frozen
    1. Adapter methods: add new trainable layers to the architecture of the model, typically inside the encoder or decoder components after the attention or feed-forward layers.
    2. Soft prompt methods: keep the model architecture fixed and frozen, and focus on manipulating the input to achieve better performance

Re-cap of how Transformer works

  1. The input prompt is turned into tokens
  2. Tokens converted to embedding vectors and passed into the encoder and/or decoder parts of the transformer.
  3. In Encoder and Decoder, there are two kinds of neural networks: self-attention and feedforward networks.
  4. The weights of these networks are learned during pre-training.
  5. During full fine-tuning, every parameter in these layers is updated.

Or, step 5, we can get LoRA going!

LoRA (Low-Rank Adaptation of LLM): LoRA is a strategy that reduces the number of parameters to be trained during fine-tuning by freezing all of the original model parameters and then injecting a pair of rank decomposition matrices alongside the original weights. Then you can get a LoRA fine-tuned LLM for a specific task

You can use a single GPU instead of multiple of them, if you are using LoRA.

You can switch out the matrices for different tasks, those matrices are typically very small:

It's not the case that bigger matrices, better performance. Ranks in the range of 4-32 can provide you with a good trade-off between reducing trainable parameters and preserving performance.

Prompt Tuning: different from prompt engineering, you add additional trainable tokens (soft prompts) to your prompt and leave it up to the supervised learning process to determine their optimal values

Soft prompts: weights of the model are frozen, but the embedding vectors of the soft prompt gets updated over time to optimize the model's completion of the prompt.

Bigger the model, more effective prompt tuning is:

标签:Week,prompt,parameters,Notes,Coursera,weights,LLM,model,LoRA
From: https://www.cnblogs.com/miramira/p/18071182

相关文章

  • 【Coursera GenAI with LLM】 Week 2 Fine-tuning LLMs with instruction Class Notes
    GenAIProjectLifecycle:Afterpickingpre-trainedmodels,wecanfine-tune!In-contextlearning(ICL):zero/one/fewshotinference.Includingafewmodelsinthepromptformodeltolearnandgenerateabettercomplement(akaoutput).Itsdrawbacks......
  • 3月11号-完成第一阶段coursera前三期课程
    前三期课程主要包含了互联网发展的历史,从二战英国,波兰对抗德国的密码战所催生的对于计算机的启蒙发明创造,之后计算机从机械结构衍生到电子管结构,中心从欧洲转移到了美国。不得不说,计算机和物理学科以及其他学科紧密相连。无论是二战时构建原始的计算机还是之后美国大学促进互联网......
  • Pacing guide is based on five 50 minute class sessions per week
    Pacingguideisbasedonfive50minuteclasssessionsperweekcorecontent     corecontent     capstone     explorations     optionalcontent     WEEK1 Session1Session2Session......
  • Papers in week 1
    文章总结(week1)2024.3.4~2024.3.10DeepRitzMethodforEllipticalMultipleEigenvalueProblemsIF=2.5,JournalofScientificComputingDOI:10.1007/s10915-023-02443-8文章研究了用神经网络求解椭圆型多重特征值问题。基于椭圆特征值问题的惩罚变分形式,提出了......
  • 蓝桥杯算法集训 - Week1:二分、前缀和、差分算法
    蓝桥杯算法集训-Week1本系列随笔用于整理AcWing题单——《蓝桥杯集训·每日一题2024》的系列题型及其对应的算法模板。一、二分查找二分算法原理复习参考:二分查找-Hello算法Ⅰ、二分模板boolcheck(intx){/*...*/}//检查x是否满足某种性质//区间[l,r]被划分......
  • SMU Winter 2024 div2 ptlks的周报Week 5(3.4-3.10)
    维护子树的全部子树的权值和时,需要用到树的DFS序列,树的每个子树都对应DFS序列中的连续一段黄金树影题意:给定一棵树及每个节点的权值,给定一组操作,输入1ax,表示节点a权值加上x;输入2a,表示询问节点a的子树权值和(包含a)。考虑到树的DFS序列,则问题转变为对某个序列维护区间和以......
  • Weekly Contest 387
    ProblemADistributeElementsIntoTwoArraysI思路按照题意模拟即可.代码classSolution{publicint[]resultArray(int[]nums){intn=nums.length;int[]ans=newint[n];int[]arr1=newint[n];int[]arr2=newint[......
  • NewStarCTF 2023 公开赛道 做题随笔(WEEK1|MISC部分)
    第一题下载打开得到TXT文件好的看样子应该是base32,复制到base在线转换看看得到这玩意 base58转换得到 出了flag  第二题 下载得到一张二维码用隐写软件试试得到一张这个以为是摩斯密码,试试得到有个这玩意,嘶,好像不是试试LSB 得到flag 第三题......
  • Week 2 Problems
    T1代换式、替换式求代换式\((P\rightarrow(P\rightarrowQ))[P/P\rightarrowR]\)求替换式\((P\lorR\rightarrowP\lorR\landS)[(P\lorR)/(P\landR)]\)已知\(P,Q,R,S\)是命题逻辑合式公式,\(P\)是\(Q\)的子公式,\(R\)不是\(Q\)的子公式,用\(Q^1\equivQ[P/R]\)和「替......
  • NewStar Week2-3部分pwn wp
    stack_migrationchecksec开启了NX保护,但是没有PIE和Canary代码审计可以看到有两个read和一个printf。第一个read没什么用我们看第二个。因为v2距离rbp有0x50个字节,而read只能读入0x60个字节,意味着我们剩余的字节数只有0x10,没法构造完整的ROP链,那么我们就只能利用栈迁移来变......