{"id":"arxiv/2609.31983","url":"https://arxiv.org/abs/2609.31983","tags":["cs.LG","math.OC"],"score":0,"title":"Understanding the Subspace Stabilization of the Hessian and Gradient Covariance Matrix","domain":"cs.LG","source":"arxiv","authors":["Fangshuo Liao","Anastasios Kyrillidis"],"reviews":0,"subject":"cs.LG","summary":"The phenomenon of the top subspace stabilization of the Hessian matrix is an surprising and critical aspect in study of the second-order information of neural network training. Prior work argues that the top subspace of the Hessian stabilizes by measuring the overlap between the top subspaces of the step-wise Hessian, and explains this stabilization with diminishing parameter change in the late phase of training. In this paper, we define a new instability metric for the subspace evolution, and use it to detect subspace stabilization that is independent of the magnitude of parameter change.…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-09-25","reactions":{}}