Neural Networks Taxonomy - 24. Physics-Informed and Scientific ML Networks

24. Physics-Informed & Scientific ML Networks

Scientific machine learning bakes known physics into the network. A physics-informed neural network (PINN), in Fig 26, represents the solution of a differential equation as a network and, using automatic differentiation to obtain its derivatives, adds the equation's residual to the loss alongside any data — so training respects the governing law even where measurements are scarce.

Origins and rise

Raissi, Perdikaris & Karniadakis introduced PINNs for both forward and inverse PDE problems.^[325] Operator-learning methods generalise further: DeepONet learns mappings between whole function spaces,^[326] and the Fourier Neural Operator learns resolution-independent solution operators in the spectral domain.^[327] The continuous-depth view connects them to neural ODEs.

Classical PINNs and variants

The original physics-informed neural network (PINN) stays close to the diagram in Fig 26: a single network whose loss blends data fit with the differential-equation residual.^[325] fPINNs extend the same residual idea to fractional derivatives,^[328] hp-VPINNs recast the loss in variational form with domain decomposition for stiffer problems,^[329] and B-PINNs turn the PINN Bayesian, placing a prior over the network and returning calibrated uncertainty in the solution.

Operator-learning networks

Where a PINN solves one instance of a PDE, operator learning learns the mapping from coefficient or boundary function to solution, generalising across the whole family. DeepONet branched into a branch net (encoding the input function) and a trunk net (encoding the query point)^[326] while the Fourier Neural Operator (FNO) parameterised the kernel in frequency space with a global FFT, making it fast and resolution-invariant.^[327] The Graph Neural Operator generalises FNO to irregular meshes, and MP-PDE casts the operator as message passing on a mesh graph, linking the approach to graph networks.^[330]

SciML frameworks

Building PINNs and operator networks by hand is laborious, so the community shipped general-purpose solvers. NeuralPDE (part of the SciML Julia ecosystem) and DeepXDE provide high-level APIs for declaring equations, boundary conditions, and architectures, and then dispatching training across backends — turning scientific machine learning from a research exercise into an engineering workflow.

Applications

• Solving and inverting PDEs in fluids, heat, and electromagnetics.
• Fast surrogates for expensive simulations.
• Data assimilation from sparse sensors.

Strengths and limitations