AdaMeZO: Adam-Styled Zeroth-Order Optimizer for LLM Fine-tuning Without Memorizing the Moments

Published in ICML 2026, 2025

Zhijie Cai *, Haolong Chen *, Guangxu Zhu