Why physics needs neural operators, not just bigger language models

Language models learn patterns in text. Physical systems add a harder requirement: the input and output may live at many resolutions, evolve across space and time, and obey constraints that a plausible-looking prediction can still violate. On Latent Space, Caltech professor Anima Anandkumar argued that AI for science needs models built around those properties rather than treating the physical world as another fixed-size data format.

Anandkumar's thesis is that the field has foundation models for language, while physics still lacks a comparable model that can work across phenomena and scales. She is a professor of mathematics and computer science at Caltech and discussed work on weather, plasma, formal verification, and inverse design in the episode's AI-for-science interview. 1

The distinction matters because ideas are only the first step in science. A language model can generate hypotheses, but a scientific model must also produce predictions that researchers can test, preserve physical constraints, and remain useful when the data changes. Anandkumar connected that problem to TorchLean, a framework for writing PyTorch-like neural networks inside the Lean proof assistant. The goal is to formally check properties such as robustness, numerical precision, and compliance with physical laws in settings where a neural network may enter a control loop. 1

A standard neural network expects fixed-size inputs and outputs. Physical phenomena are multi-scale. A weather field may be observed at one resolution and later need a finer prediction; a simulation may need to combine measurements with constraints at another scale. Anandkumar described neural operators as models that learn mappings between continuous functions, which lets the input and output resolution vary.

Fourier neural operators provide one way to make that mapping efficient. Fourier transforms can represent non-local relationships with quasi-linear complexity, while the network's nonlinear layers preserve more expressive behavior than a purely linear Fourier representation. Anandkumar contrasted that with the cost of transformer attention at very high physical resolution, where all-to-all connections become impractical. 1

The architectural point is easy to miss. The Fourier transform is not a claim that every physical process is fundamentally a linear frequency signal. It is a useful coordinate system for capturing broad dependencies efficiently, with learned nonlinear transformations around it. A storm system can connect distant locations; a model that only sees each local patch in isolation will struggle to represent that relationship.

Anandkumar said her group began with weather because the ERA5 reanalysis dataset was available. The team trained a model to predict global weather variables autoregressively at six-hour steps. She described ForecastNet's progression from short-term prediction toward a foundation-model ambition that covers short-term weather, longer-range climate behavior, and the geometry of a spherical Earth. 1

Weather also makes the limits visible. Local forecasts become chaotic over longer horizons, so climate questions require many trajectories and probabilistic summaries rather than one deterministic path. Anandkumar said the long-run validation problem remains open: researchers want the speed of AI simulation and confidence that long rollouts preserve the relevant physical behavior.

The payoff can still be large. She said AI weather models have been adopted by weather agencies and can give smaller agencies access to capabilities once concentrated in large institutions. That is a claim about access and speed, not a promise that AI has replaced physical modeling. The training data itself incorporates historical observations and physics-based reanalysis, and the model still needs checks for extreme events and higher-resolution extrapolation.

The same pattern appears in fusion. Anandkumar discussed a neural-operator model of plasma evolution in a tokamak, developed with the UK Atomic Energy Agency, that she said runs about a million times faster than traditional simulations. The goal is a digital twin of the plasma: a fast model that can support later work on controlling disruptions while preserving the relevant physics. 1

She also described inverse design. A forward model predicts what a design will do; an inverse model searches for a design that produces a desired outcome. The conversation mentioned electromagnetic circuits, materials, heat sinks, fluid flow, inverse lithography, quantum dots, and nonlinear photonics. Simulation inside the design loop lets a model propose shapes and then test whether those shapes work.

That is where formal verification becomes more than an abstract safety feature. A model that influences a reactor or a high-energy experiment needs bounds on perturbations, finite-precision effects, and physical constraints. TorchLean addresses the formal side; neural operators address the multi-scale simulation side. Both remain constrained by computation and by the quality of the assumptions entering the model.

Anandkumar was explicit that a model cannot transfer to completely unseen physics with no data. Scientific discovery is inherently extrapolative: a new discovery has no prior dataset in the ordinary sense. Physical structure and conservation laws can narrow the search, but they do not remove the need for evidence.

Her broader ambition is a multi-physics foundation model that handles coupled phenomena and different geometries, then supports inverse design as well as simulation. The episode supplies a research direction rather than a finished general model. The current achievements are narrower and more useful to state precisely: neural operators can make some physical simulations faster, flexible across resolution, and compatible with learned patterns plus physical constraints; long-horizon validation and high-resolution extrapolation remain active problems.

The original episode is available on Latent Space.