Hard Constraint Projection Boosts PINN Accuracy by an Order of Magnitude: Evidence from the Burgers Equation

Physics-informed neural networks (PINNs) have long been expected to transform fluid simulation—promising to complete simulations at a fraction of the computational cost of traditional numerical methods while improving the interpretability and robustness of purely data-driven models. But PINNs have a persistent pain point: physical constraints appear only as soft penalty terms in the loss function, and the model is not guaranteed to actually satisfy the governing equations. A new work on the one-dimensional Burgers equation extends Hard Constraint Projection (HCP) from linear PDEs to nonlinear PDEs, offering a compelling solution.

From Soft to Hard Constraints: Transferring the Idea

The HCP method was first proposed by Chen et al. in the Journal of Computational Physics, originally for linear partial differential equations. Its core idea is: rather than letting the network "try its best" to satisfy the equation, add a non-learnable projection layer at the network's final layer that projects the prediction directly onto the hyperplane where the exact solution of the discretized equation lies, thereby enforcing the constraint strictly.

The key contribution of this study is extending that idea to the nonlinear setting. The authors chose the one-dimensional Burgers equation as the test case—a classic equation with a nonlinear convective term and a viscous dissipation term, often used as a touchstone for numerical methods in fluids and widely recognized as a difficult case for PINNs.

Discretization and Matrix Decomposition

The approach first applies a local discretization to the Burgers equation (using a finite-difference stencil), then splits the discretized equation into a constraint matrix and a prediction matrix. Using linear algebra tools, a projection matrix is constructed from the constraint matrix. The PINN predicts the velocity field, and the final non-learnable layer projects the velocity prediction onto the hyperplane of exact solutions of the discretized Burgers equation. The whole process amounts to turning the physics from a "training objective" into "part of the network architecture."

Mathematical meaning of hyperplane projection: In linear algebra, the solution set of a system of linear equations forms an affine subspace (i.e., a hyperplane). For a discretized linear PDE, its exact solution lies on the affine subspace defined by the constraint matrix. The role of the projection matrix is to map any vector to the nearest point on that subspace, which is mathematically equivalent to solving a least-squares problem. In HCP, this projection matrix is constructed from the pseudoinverse of the constraint matrix, so that the network output, after projection, strictly satisfies the discretized equation. For nonlinear PDEs, the solution set is no longer a hyperplane, but through local linearization or iterative projection, a similar constraint mechanism can still be constructed—this is the key technical difficulty in generalizing the method.

Finite-difference stencils and discretization of the Burgers equation: A finite-difference stencil approximates derivatives by a linear combination of function values at neighboring grid points. For the one-dimensional Burgers equation, one typically uses forward Euler or Crank-Nicolson for the time derivative, central or upwind differencing for the first spatial derivative (the nonlinear convective term), and central differencing for the second derivative (the viscous term). After discretization, the equation forms an algebraic equation at each grid point, and all equations combine into a matrix form. The constraint matrix encodes the linear relationships among the unknowns in these discretized equations (after local linearization), while the prediction matrix corresponds to the mapping from network output to the discretized solution. Through matrix decomposition (such as QR decomposition or singular value decomposition), one can extract the null space or range of the constraint matrix, thereby constructing the projection matrix.

Experimental Results: Advantages Emerge Only in Late Training

The most noteworthy finding is the difference in training dynamics. According to the paper, HCP-PINN performs similarly to an equivalent ordinary PINN in the early stages of training, with the two loss curves almost overlapping. But when the ordinary PINN's loss enters a plateau and no longer decreases noticeably, the HCP-PINN's loss continues to fall.

This phenomenon is intuitive: once a soft-constraint PINN reaches a certain balance between data fitting and physical residual, it becomes difficult to further reduce the error; a hard constraint structurally excludes the solution space that violates the equation, allowing the network to focus on optimizing within the feasible solution set.

Generalization Ability and Accuracy

When tested on previously unseen uniform grid points covering the entire solution domain, HCP-PINN demonstrated higher accuracy, with its data error converging to a level about an order of magnitude lower than a comparable PINN. In addition, adding the HCP layer improved the model's adherence to the governing equations.

The difference between data error and equation residual: Data error measures the discrepancy between the network prediction and a reference solution (such as a high-accuracy numerical solution), usually expressed as relative L2 error; equation residual measures the imbalance when the predicted solution is substituted into the governing equation. A soft-constraint PINN is trained by minimizing a weighted sum of residuals, but the final residual is not necessarily zero; a hard-constraint PINN forces the residual to zero (in the discrete sense) through projection, hence better adherence to the equation. It is worth noting that forcing zero residual does not automatically guarantee small data error, because the discretized equation may have multiple solutions or the projection may introduce bias, but experiments show that HCP-PINN performs better on both.

Robustness to Collocation Point Selection

The paper also points out that HCP-PINN is more robust to the choice of physical collocation points and less dependent on their number. This is significant for practical applications: the performance of traditional PINNs is often quite sensitive to how collocation points are arranged and how many are used, making tuning costly, whereas hard constraints alleviate this pain point to some extent.

Collocation points and the collocation method: Collocation points are the sampling points used to evaluate the physical residual during PINN training. The loss function of a traditional PINN includes the equation residual computed at collocation points, so the distribution and number of collocation points directly affect training performance. If the collocation points are sparse or poorly distributed, the residual constraint may be insufficient to guide the network to converge to the correct solution. HCP-PINN, through structural hard constraints, makes the network output automatically satisfy the discretized equation after projection, thereby reducing dependence on collocation point density. Note, however, that collocation points are still used to train the network to fit the data; removing them entirely may impair data-fitting ability, so HCP does not eliminate collocation points but rather reduces sensitivity to them.

Significance and Limitations

This work extends the applicability of HCP from linear PDEs to nonlinear PDEs, validating the feasibility of "structure-level physical constraints" for more complex equations. For simulation tasks that require strict satisfaction of conservation laws or governing equations, this approach offers a technical path distinct from increasing penalty weights or increasing collocation point density.

It should be viewed objectively: the paper describes itself as "preliminary results," and the experiments are concentrated on the relatively simple setting of the one-dimensional Burgers equation. The construction of the projection matrix depends on the specific discretization scheme, and whether the approach is equally effective when extended to higher dimensions, complex boundary conditions, or strongly nonlinear systems remains to be further verified. In addition, the extra computational overhead introduced by the projection layer and its stability under longer-time integration are questions that future research needs to answer.

Overall, HCP-PINN demonstrates a pragmatic engineering approach: rather than hoping the loss function will teach the network to obey physics, it writes physics directly into the network architecture. For scenarios where PINN accuracy has stalled, this may be a direction worth trying.

One-line Summary

Extending hard constraint projection to nonlinear PDEs makes PINNs structurally satisfy the equation exactly, improving accuracy by an order of magnitude.

Paragraph Summary

This study extends the Hard Constraint Projection (HCP) method from linear partial differential equations to the nonlinear one-dimensional Burgers equation. HCP adds a non-learnable projection layer at the end of the network that projects the predicted solution onto the hyperplane where the exact solution of the discretized equation lies, turning physical constraints from soft penalties into hard structural constraints. Experiments show that HCP-PINN performs similarly to an ordinary PINN in early training, but its loss continues to decrease after the ordinary PINN's loss plateaus; on unseen grids, its data error is about an order of magnitude lower, it adheres better to the governing equation, and it is more robust to the choice of collocation points. The authors note that these are preliminary results, and generalization to higher dimensions, complex boundaries, and strongly nonlinear systems still requires verification.