李 “‘다음은 北’ 이상한 소리하는 사람 있어…무슨 득 있나”
Muon outperforms every optimizer we tested (AdamW, SOAP, MAGMA). Multi-epoch training matters. And following work by Kotha et al. , scaling to large parameter counts works if you pair it with aggressive regularization -- weight decay up to 16x standard, plus dropout. The baseline sits at ~2.4x data efficiency against modded-nanogpt.，这一点在PDF资料中也有详细论述
Essential digital access to quality FT journalism on any device. Pay a year upfront and save 20%.。业内人士推荐PDF资料作为进阶阅读
更多精彩内容，关注钛媒体微信号（ID：taimeiti），或者下载钛媒体App