Long Expressive Memory for Sequence Modeling

The paper’s Proposition 4.4 states that LEM (3) can approximate any finite-horizon Lipschitz discrete-time system (8) under boundedness assumptions, with proof deferred to SM§D.4. The construction there freezes the gates by setting Δt=1 and taking large gate biases (so ˆσ(b∞)≈1), realizes each step as a two-layer tanh MLP, uses block structure so only the state coordinates feed the next step, and controls error growth via Lipschitz bounds, culminating in an explicit LEM parameterization that achieves ||o_n−ω_n||≤ε for 1≤n≤N . The candidate solution follows the same blueprint: freeze the time gates near 1, approximate the composite map with a two-layer tanh network on a compact set, enforce block-sparsity so outputs do not feed back, and bound the accumulated error with a discrete Grönwall/Lipschitz argument. The only technical difference is that the candidate explicitly quantifies the gate-induced error (ε_gate) instead of the paper’s notational shortcut of setting the gate deviation δ→0 for simplicity, but both yield the same conclusion. Hence both are correct and essentially the same proof at a conceptual level.