2106.07683
Extracting Global Dynamics of Loss Landscape in Deep Learning Models
Mohammed Eslami, Hamed Eramian, Marcio Gameiro, Konstantin Mischaikow, William Kalies
correctmedium confidence
- Category
- Not specified
- Journal tier
- Specialist/Solid
- Processed
- Sep 28, 2025, 12:56 AM
- arXiv Links
- Abstract ↗PDF ↗
Audit review
The paper explicitly states the core claims: (i) under a multivalued outer approximation F of a continuous map f on a polytopal decomposition, recurrent dynamics of f occurs within recurrent strongly connected components (SCCs) of F and dynamics outside those components is nonrecurrent; (ii) minimal elements in the Morse graph M(F) correspond to attracting blocks, and when an order retraction σ: SC(F) → M(F) exists, it organizes basins and separatrices; (iii) the sets A(M) generated from recurrent components yield a finite sublattice contained in the lattice of attracting blocks; and (iv) when F is built with a variance parameter μ ensuring an outer approximation, μ is a lower bound on the probability that M(F) is a correct poset representation of the dynamics. These are stated in the paper’s Conley–Morse graphs section and its discussion of order retractions and lattices, with algorithmic and lattice-theoretic support cited to prior works . The construction of F0 and F and the outer-approximation framework are defined in the text and supplement ; the surrogate-model context and workflow are also described . The candidate solution reconstructs these points with more explicit proofs (e.g., localization of recurrence via Poincaré returns to cells, fiber invariance under σ, and the interior property of trapping blocks). Its arguments are standard in computational Conley theory and consistent with the paper’s claims and cited foundations. Minor caveats (orientation of the SC(F) order, existence of σ, resolution/no spurious merges) are acknowledged in the paper and are handled in the solution’s remarks, so both are substantively correct and aligned.
Referee report (LaTeX)
\textbf{Recommendation:} minor revisions
\textbf{Journal Tier:} specialist/solid
\textbf{Justification:}
The paper correctly applies established computational Conley theory to neural network training dynamics and articulates a coherent pipeline from outer approximations to Morse graphs and order retractions. Claims are accurate under standard assumptions and supported by prior literature. Minor clarifications on order orientation, explicit conditions for order retractions, and discretization fidelity would enhance rigor and readability, but the core contributions are sound.