Advances in Optimization Techniques for Large Language Models

信息管理与工程学院308室
上海财经大学（第三教学楼西侧）

上海市杨浦区武东路100号

Jiaxiang Li is currently a postdoc fellow at the ECE department, University of Minnesota. He obtained his phd in applied math at UC Davis in 2023. His research focus is numerical optimization and its applications for problems arised in machine learning and operation research.

AI systems is becoming ubiquitous as we've observed the success of products like ChatGPT, GPT4, Claude, Llama etc. However, these products, particularly the large language models (LLMs) they are based on, face significant challenges, one of which is the increasing difficulty to aligning with human behavior due to the sheer size of the models and datasets. This talk will explore recent advancements in pretraining and fine-tuning LLMs to enhance their performance across various dimensions. We will introduce an approach that employs inverse reinforcement learning and bi-level optimization to align LLMs with human behavior more effectively. Additionally, we will examine the application of zeroth-order optimization techniques for efficient fine-tuning of LLMs. The overarching goal of this talk is to demonstrate the critical role of sophisticated optimization modeling and algorithm design in advancing the capabilities of AI systems.