beignet: A new neural architecture reduces error by up to 17 times in solving physical equations

Des chercheurs de Caltech et Google DeepMind présentent beignet, une architecture neuronale utilisant Fourier pour résoudre plus précisément des équations physiques.

Reproducing the evolution of a wave, the propagation of a chemical reaction, or the emergence of a mathematical singularity requires solving equations whose behavior can change abruptly at very small scales. Physics-informed neural networks offer an alternative way to approach these calculations, but still struggle to reach the precision of certain classic numerical methods. A team of researchers from the California Institute of Technology and Google DeepMind proposes a new architecture called beignet, for Bandlimited Embedding with Interpolated Grid Network. Its principle consists of replacing the randomly selected frequency representations usually used with a pyramid of Fourier frequencies organized across multiple resolutions. In four differential equation evaluations, this approach reduces the relative error by a factor of approximately 6 to 17 compared to PirateNet, with fewer parameters. Neural Networks Facing the Equations of Physics

Partial differential equations describe many natural phenomena: heat transport, fluid flow, wave propagation, or the evolution of chemical reactions. Solving them involves determining how a quantity varies in space and time, starting from initial conditions and physical constraints. When solutions exhibit rapid oscillations, very sharp interfaces, or large gradients, obtaining an accurate numerical result becomes particularly difficult.

Physics-Informed Neural Networks (PINNs) approach the problem by representing the sought-after solution with a neural network. Rather than learning solely from solution examples, the model is trained to directly respect the physical equation. Its predictions are evaluated at different points in space, then corrected in order to reduce the gap between their derivatives and the imposed mathematical relationships.

However, this method encounters a problem known as spectral bias. Neural networks generally learn slow variations more easily than high-frequency details. They can therefore correctly reproduce the general shape of a phenomenon while missing its finest structures. Enlarging the network or increasing the frequencies used to represent the signal is not always sufficient: computation becomes more expensive, and training can lose stability. A Fourier Pyramid Rather Than an Ever-Larger Network

Beignet primarily modifies the way coordinates are represented before being processed by the neural network. Usual methods often use sinusoidal functions whose frequencies are randomly selected. The researchers replace this distribution with a pyramid of trainable Fourier grids, ranging from the coarsest to the most detailed representations.

Each level of the pyramid represents a range of spatial frequencies. To calculate the value of a physical field at a given position, beignet interpolates the information contained in the different grids, gathers it, and then passes it to a small neural network responsible for producing the result. This organization allows adjusting the amount of information dedicated to large structures and fine details without necessarily increasing the size of the main network.

The advantage also lies in controlling the maximum frequency represented. Some equations require covering a broad spectrum to capture sudden changes, while others benefit from a representation focused on lower frequencies. Too wide a range can amplify errors during derivative calculations. With beignet, researchers can explicitly choose grid resolution and allocate model capacity according to the characteristics of each problem. The Fourier Transform Speedup for Derivative Calculations

The other distinctive feature of beignet concerns the calculation of spatial derivatives, which are essential for verifying that a solution respects a physical equation. In traditional PINNs, these derivatives are often obtained through automatic differentiation applied to the network. When equations require higher-order derivatives, the amount of computation and memory used can increase rapidly.

Beignet leverages the fast Fourier transform (FFT) to efficiently calculate the derivatives of its frequency representations on structured grids. The results are then combined with the derivatives of the small neural network using the chain rule. The architecture thus avoids placing all the workload on a large-scale network.

For time-dependent problems, the authors add multiple representations corresponding to different time steps, then interpolate between them. This solution makes it possible to retain Fourier's spatial advantages without artificially imposing periodic evolution in time. The researchers also introduce slight random grid shifts during training to limit errors that might remain invisible between grid points. Up to 17 Times Less Error across Four Equations

Performance was evaluated on four problems from the JAX-PI suite: Allen–Cahn, Korteweg–de Vries, Ginzburg–Landau, and Gray–Scott. These equations present different difficulties, ranging from wave propagation to the evolution of reaction-diffusion systems producing complex patterns. The authors compare their architecture to JAX-PI and PirateNet, two existing physics-informed neural network approaches.

On Allen–Cahn, beignet achieves an L2 relative error of 1.88 × 10⁻⁶, compared to 1.21 × 10⁻⁵ for PirateNet, representing an error roughly 6.4 times lower. On Korteweg–de Vries, it drops from 2.59 × 10⁻⁴ to 3.71 × 10⁻⁵, a reduction close to a factor of 7. Gains increase on the two reaction-diffusion problems: about 10.6 times for Ginzburg–Landau and 16.8 times for Gray–Scott, on the main fields compared.

This improvement does not require more parameters than PirateNet. The beignet configurations use approximately 412,000 parameters for the first two problems, compared to 1.92 million for PirateNet. For Ginzburg–Landau and Gray–Scott, the results rely on calculation windows each containing about 856,000 parameters, compared to 1.92 million for competing configurations. The authors thus report better accuracy