Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond

The paper states the chain-rule decomposition for ϕ(x)=F(x,y*(x)) (its Eq. (14)) and then derives the implicit-gradient formula (its Eq. (33)) by differentiating the LL stationarity condition ∂y f(x,y*(x))=0, assuming the LL Hessian ∂²yy f is invertible and working under the lower-level singleton (LLS) setting; see their discussion that IGBR requires second-order differentiability and an invertible Hessian and relies on uniqueness of the LL solution set . The candidate solution gives the same formula, using the Implicit Function Theorem to justify a C^1 local selection and explicitly identifying it with the best response under LLS and nondegenerate KKT, then applying the chain rule. Both arguments are essentially the same (implicit differentiation of the LL KKT with invertible ∂²yy f); the model provides a more explicit IFT-based justification of smoothness and selection. The paper’s presentation is correct but high-level; the model fills in standard regularity details.