{"id":"arxiv/2610.35209","url":"https://arxiv.org/abs/2610.35209","tags":["cs.LG","math.OC"],"score":0,"title":"Understanding Practical Optimization in Machine Learning","domain":"cs.LG","source":"arxiv","authors":["Egor Shulgin"],"reviews":0,"subject":"cs.LG","summary":"Modern machine learning relies on practical optimization methods that often differ from the algorithms studied in classical analyses: they train submodels, clip or normalize gradients, use layer-wise non-Euclidean directions, or approximate costly inner computations. This thesis analyzes the objective, assumptions, update, and computation defined by these procedures, rather than those of simplified proxies. For submodel training, we show that Independent Subnetwork Training can converge to a shifted fixed point because sketching before differentiation biases the update for the original…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-10-26","reactions":{}}