The differentiability of Monte Carlo PDE solvers stems from the fact that their estimators are themselves smooth functions of the parameters, allowing unbiased gradient estimates to be obtained directly via reparameterization tricks or likelihood ratio methods, without needing to backtrack through the intermediate states of a large-scale linear system.