UT Austin researchers introduce Panda: The basic model of nonlinear dynamics of 20,000 Chaos Odes discovered through evolutionary search

Chaotic systems, such as fluid dynamics or brain activity, are highly sensitive to initial conditions, making long-term prediction difficult. Even in modeling these systems, it may grow rapidly, limiting the effectiveness of many scientific machine learning (SCIML) methods. Traditional prediction methods rely on models in specific time series or in broad datasets that lack truly dynamic structures. However, recent work has shown that by learning the numerical rules that manage these systems, local prediction models can be made to predict chaotic systems more accurately. The real challenge is to implement cross-domain generalization, i.e. to create models that can adapt to and predict new, previously invisible dynamic systems. This will require the integration of prior knowledge with local adaptability. Nevertheless, the need for task-specific data limits the current method and often ignores critical dynamical system properties such as Ergodicity, channel coupling, and conservative quantities.

Machine learning of dynamic systems (MLDS) takes advantage of the unique properties of systems such as inductance deviation. These include fixed relationships between system variables and invariant statistical measures, such as strange attractors or conservative quantities. MLDS models use these properties to build more accurate and generalizable models, sometimes combining probabilistic or latent variable techniques. Although data sets of dynamic systems have been curated and new systems are often generated by tuning parameters or using symbolic methods, these methods often fail to ensure diversified or stable dynamics. Structural stability is a challenge – small changes may not produce new behaviors, while large behaviors may lead to trivial dynamics. The basic model is designed to solve this problem by enabling transfer learning and zero-photo inference. Nevertheless, most current models are similar to standard time series models or are limited in generating meaningful dynamic changes. Some progress has been made through technologies such as embedded space or symbolic discovery, but richer and more diverse sampling of dynamic behavior remains an open challenge.

Researchers at Auden College in Austin have introduced Pandas (the fill of nonlinear dynamics), a proven model trained on synthetic data from chaotic systems generated by 20,000 algorithms. These systems are created using evolutionary algorithms based on known chaotic ODEs. Although trained on low-dimensional ODE only, Panda shows strong zero shooting predictions on real-world nonlinear systems, including fluid dynamics and electrophysiology, and unexpectedly generalizes it to PDES. The model combines innovations such as masking preprocessing, channel attention, and kernelized patches to capture dynamic structures. Neural scaling methods have also emerged, linking pandas’ predictive performance with the diversity of training systems.

Using a genetic algorithm, the researchers generated 20,000 new chaotic systems that evolved from a curated set of 135 known chaotic odes. These systems are mutations and recombined through a skewed product approach, retaining true chaotic behavior only through rigorous testing. Enhancements such as time-delay embedding and affine transformation extend the dataset while preserving its dynamics. Hold a separate set of 9300 invisible systems for zero shot tests. The model Panda is built on PatchTST and has features such as channel attention, temporal channel attention layer, and dynamic embedding using polynomial and Fourier functions, inspired by Koopman operator theory.

Pandas exhibit strong zero-fire prediction capabilities on invisible nonlinear dynamic systems, surpassing models such as timing-SFT in various indicators and prediction ranges. Training is only received on 3D systems, and due to channel concerns, it is summarized as higher-dimensional training. Although PDEs have never been encountered in training, Panda succeeded in real-world experimental data and chaotic PDEs, such as Kuramoto-Sivashinsky and Von Kármán Vortex Street. Architectural ablation confirms the importance of channel attention and dynamic embedding. The model exhibits neural scaling with increased dynamic system diversity and forms an interpretable attention pattern indicating resonance and attractor-sensitive structures. This suggests a broad generalization of complex dynamic behaviors by pandas.

In short, Pandas is a validated model designed to discover generalizable patterns in dynamic systems. Pandas trained large, diverse synthetic chaos systems, showing strong zero-shooting predictions on invisible real-world data and even partial differential equations, although only trained in low-dimensional odes. Its performance is improved through system diversity, revealing the law of neural scaling. The model also shows nonlinear resonances that occur in the attention pattern. Despite focusing on low-dimensional dynamics, this approach can be extended to higher-dimensional systems by leveraging sparse interactions. Future directions include improving the rollout performance prediction of chaotic behavior.

View paper. All credits for this study are to the researchers on the project. Also, please stay tuned for us twitter And don’t forget to join us 95k+ ml reddit And subscribe Our newsletter.

Sana Hassan, a consulting intern at Marktechpost and a dual-degree student at IIT Madras, is passionate about applying technology and AI to address real-world challenges. He is very interested in solving practical problems, and he brings a new perspective to the intersection of AI and real-life solutions.