我在读博期间研究了关于非线性优化算法中的Barzilai-Borwein(BB)步长。正如我的导师安老师所言：梯度下降法是优化算法的基石(只考虑一般可微的目标函数)。在该算法中，迭代点每次沿着当前的负梯度方向"走一步"，一直迭代下去，逼近最优解(当然是局部的)。一个自然地问题是这个"走一步"的步长该如何选取？神奇的BB步长提供了一个方案(包括了长短两个步长)。该步长的提出者Jonathan Barzilai 是一位富有激情的学者，他直到生命的最后仍然在进行科学研究(因身患癌症，于2022年5月去世)。下图是Barzilai教授在2019在网上回答问题的截图。 (During my doctoral studies, I studied the Barzilai-Borwein (BB) step size in nonlinear optimization algorithms. As my mentor, Mr. An, said: Gradient descent is the cornerstone of optimization algorithms (only considering general differentiable objective functions). In this algorithm, the iteration point "takes a step" in the direction of the current negative gradient each time, and iterates continuously to approach the optimal solution (of course, local). A natural question is how to choose the step size of this "step"? The magical BB step size provides a solution (including long and short step sizes). Jonathan Barzilai, the proposer of this step size, was a passionate scholar who continued to conduct scientific research until the end of his life(he passed away in May 2022, after a battle with cancer). The picture below is a screenshot of Prof. Barzilai answering questions online in 2019.
做为一个初入科研领域的学生，我想以此博文来纪念这位对科学研究做出卓越贡献的学者。(As a student just entering the field of scientific research, I would like to use this blog post to commemorate this scholar who has made outstanding contributions to scientific research.)
Barzilai 教授出生于英国托管时期的以色列特拉维夫。他是一名才华横溢的学生，在以色列海法的以色列理工学院 (Technion) 获得了应用数学学士、硕士和博士学位。(Prof.Barzilai was born in Tel Aviv, Israel during the British mandate. A brilliant student, he received his bachelor's, master's and doctoral degrees in applied mathematics from the Israel Institute of Technology (Technion) in Haifa, Israel.)
\(\qquad\qquad\qquad\quad\quad\quad\quad\quad\)
随后，他在以色列军队服役六年，担任情报官，研究应用数学算法，之后获得数学优化研究生学位。在攻读博士学位期间，他是特拉华大学的客座研究员，毕业后，他进入德克萨斯大学奥斯汀分校和约克大学担任博士后。1984 年，他首次受聘于达尔豪西大学管理学院。在此期间，他还被达尔豪西大学数学系兼职。(He then spent six years in the Israeli military where he worked on applied mathematical algorithms as an intelligence officer before completing graduate degrees in mathematical optimization. During his doctorate, he was a visiting researcher at the University of Delaware and upon completion, he took up postdoctoral positions at the University of Texas in Austin and York University in Ontario. His first faculty appointment was at the Faculty of Management at Dalhousie University in 1984. During this time, he was also cross appointed at the Department of Mathematics at Dal.)
他与 Jonathan Borwein 一起开发了 Barzilai-Borwein 梯度法，被广泛认为是对非线性优化的重大贡献。他对优化研究的热情一直持续到生命的最后一天，在此期间，他在算法方面取得了重大突破，对人工智能训练等应用具有重要意义。(Along with Jonathan Borwein, he developed the Barzilai-Borwein gradient method, widely recognized as a very significant contribution to non-linear optimization. His passion for optimization research continued to his very last days, and throughout that time he made significant breakthroughs in algorithms with implications for such applications as AI training.)
1987 年，他加入新斯科舍技术大学 (TUNS)，担任计算机科学学院院长。1997 年，TUNS 与 Dal 合并，他转到工业工程系。2017 年退休后，他一直活跃于研究领域，直到生命的最后一刻。他的另一个爱好是测量理论和决策理论与分析的相关领域。(In 1987, he joined the Technical University of Nova Scotia (TUNS) as the Director of the School of Computer Science. When TUNS merged with Dal in 1997, he moved to the Department of Industrial Engineering. After his retirement in 2017, he continued to stay active in research until the very end. His other passion was the related areas of measurement theory and decision theory and analysis.)
Barzilai 博士发表了关于测量和决策理论的主要论文，并开发了一种用于单个决策者或群体进行测量、评估和决策的方法——偏好函数建模 (PFM)，该方法基于对层次分析法、效用理论、决策理论、测量理论和相关领域的数学基础的二十多年研究。他对这些领域的现有理论提出了挑战，最近完成了一本名为《纯经济学》的书（即将在他死后出版），书中他指出了这些学科的主要缺陷并提出了纠正措施。(Dr. Barzilai published major papers on measurement and decision theory and developed a methodology, Preference Function Modelling (PFM), for measurement, evaluation, and decision making by a single decision maker or a group, based on more than twenty years of research into the mathematical foundations of the Analytic Hierarchy Process, utility theory, decision theory, measurement theory, and related fields. He challenged existing theories in these domains and very recently completed a book entitled "Pure Economics" (soon to be published posthumously) in which he points out the major flaws in these disciplines and offers corrections.)
Barzilai 博士对自己和他人都抱有很高的道德和智力标准。他挑战传统，并准备为了捍卫自己的信念而变得不受欢迎。他是一位具有极强幽默感的老师、向导和导师，为那些有幸认识他的人提供宝贵的建议和见解。(Dr. Barzilai held high ethical and intellectual standards, both for himself and others. He challenged convention and was ready to be unpopular in defense of his convictions. He was a teacher, guide, and mentor with a great sense of humour, offering invaluable advice and insights to those who were fortunate to know him.)
参考文献(Reference)
- Jonathan Barzilai, Jonathan Michael Borwein, Two-Point Step Size Gradient Methods, IMA Journal of Numerical Analysis 8(1):141-148, 1988