AdaMeZO: Adam-Styled Zeroth-Order Optimizer for LLM Fine-tuning Without Memorizing the Moments

Published in under review, 2025

Zhijie Cai *, Haolong Chen *, Guangxu Zhu