Deep Reinforcement Learning for Online Control of Stochastic Partial Differential Equations

The paper formulates SPDE control as an MDP and defines the “squashing” objective J with ū(t) = E[(1/2π)∫ u dx], but only sketches the MDP mapping and reward choice without rigorous derivations or assumptions. It also adopts a specific finite-difference/Crank–Nicolson discretization without formal transition-kernel characterization. The candidate solution supplies the missing analysis: (a) a correct variance identity for E[∫(u−ū)^2], (b) a concrete MDP construction via Fourier–Galerkin and Euler–Maruyama with explicit Gaussian kernels and reward equal to −J_N, and (c) a sound argument that improved return strictly reduces the expected spatial L^2 variance at some time. Minor slips (a sign omission in one explanatory sentence and the sign of the state-mean variance adjustment) do not affect correctness. Overall, the paper’s argument is conceptually right but incomplete, while the model’s solution is correct on the posed tasks. See the paper’s definition of the reward and cost (their Eqs. (2)–(3)) and “squashing” objective for stochastic Burgers (their Eq. (5) and surrounding text) for alignment .