Abstract: Parametric PDEs arise in key applications ranging from parameter optimization, inverse state estimation, to uncertainty quantification. Accurately solving these tasks requires an efficient treatment of the resulting sets of parametric PDE solutions that are generated when parameters vary in a certain range. These solution sets are difficult to handle since their are embedded in infinite dimensional spaces, and present a complex structure. They need to be approximated with numerically efficient reduction techniques, usually called Model Order Reduction methods. The techniques need to be adapted both to the nature of the PDE, and to the given application task. In this course, we will give an overview of linear and nonlinear model order reduction methods when applied to forward and inverse problems. We will particularly emphasize on the role played by nonlinear approximation and geometrical PDE properties to address classical bottlenecks.

Tentative schedule: (4×1.5 = 6 hours)

1. Elements of approximation theory:
   • Definition of linear, and nonlinear approximation
   • The enconder-decoder framework
   • Notions of optimal approximation performance
   • Neural networks: approximation properties and open questions
2. Linear and Nonlinear Schemes for Model Order Reduction:
   • Parametric PDEs
   • Linear MOR schemes
   • Nonlinear schemes
3. Linear and Nonlinear Schemes for Inverse State and Parameter Estimation:
   • Optimal reconstruction algorithms
   • Implementable linear and nonlinear Schemes: constructions and results on optimality

This course will be followed by a computer session. Tentative prerequisites: Python, NumPy, PyTorch.

Abstract: Operators are mappings between infinite-dimensional spaces, which arise in the context of differential equations. Learning operators is challenging due to the inherent infinite-dimensional context. In this course, we present different architectures for learning operators from data. These include operator networks such as DeepONets and Neural operators such as Fourier Neural Operators (FNOs) and their variants. We will present theoretical results that show that these architectures learn operators arising from PDEs. A large number of numerical examples will be provided to illustrate them.

This course will be followed by a computer session.

Abstract: High-fidelity numerical simulation of physical systems modeled by time-dependent partial differential equations (PDEs) has been at the center of many technological advances in the last century. However, for engineering applications such as design, control, optimization, data assimilation, and uncertainty quantification, which require repeated model evaluation over a potentially large number of parameters, or initial conditions, these simulations remain prohibitively expensive, even with state-of-art PDE solvers. The necessity of reducing the overall cost for such downstream applications has led to the development of surrogate models, which captures the core behavior of the target system but at a fraction of the cost. In this context, new advances in machine learning provide a new path for developing surrogates models, particularly when the PDEs are not known and the system is advection-dominated. In a nutshell, we seek to find a data-driven latent representation of the state of the system, and then learn the latent-space dynamics. This allows us to compress the information, and evolve in compressed form, therefore, accelerating the models. In this series of lectures, I will present recent advances in two fronts: deterministic and probabilistic modeling latent representations. In particular, I will introduce the notions of hyper-networks, a neural network that outputs another neural network, and diffusion models, a framework that allows us to represent probability distributions of trajectories directly. I will provide the foundation for such methodologies, how they can be adapted to scientific computing, and which physical properties they need to satisfy. Finally, I will provide several examples of applications to scientific computing.

Tentative schedule:

1. Intro to time domain problems:
   • time dependent PDEs
   • advection dominated problems
   • Kolmogorov n-widths
   • Failure of linear projections
2. Introduction to HyperNetworks:
   • Radiance-networks
   • Hypernetworks
   • Hyper encoders
   • Latent Space trajectories
   • Smoothness constraints
3. Intro to SDEs and Diffusion models:
   • Diffusion processes
   • Denoising and backwards evolution
   • Score-based diffusion models
4. Sampling with Constraints:
   • Tweedies’ formula
   • Linear constraints and probabilistic approximations
   • Non-linear constraints and extreme events

This course will be followed by a computer session. Tentative prerequisites: Python, JAX, Flax.

Abstract: In this talk, I will give an overview of recent successes (and some failures) of combining modern, high order discretization schemes with machine learning submodels and their applications for large scale computations. The primary focus will be on supervised learning strategies, where a multivariate, non-linear function approximation of given data sets is found through a high-dimensional, non-convex optimization problem that is efficiently solved on modern GPUs. This approach can thus for example be employed in cases where current submodels in the discretization schemes currently rely on heuristic data. A prime of example of this is shock detection and shock capturing for high order methods, where essentially all known approaches require some expert user knowledge as guiding input. As an illustrative example, I will show how modern, multiscale neural network architectures originally designed for image segmentation can ameliorate this problem and provide parameter free and grid independent shock front detection on a subelement level. With this information, we can then inform a high order artificial viscosity operator for inner-element shock capturing. In the second part of my talk, I will present data-driven approaches to LES modeling for implicitly filtered high order discretizations. Whereas supervised learning of the Reynolds force tensor based on non-local data can provide highly accurate results that provide higher a priori correlation than any existing closures, a posteriori stability remains an issue. I will give reasons for this and introduce reinforcement learning as an alternative optimization approach. Our experiments with this method suggest that is it much better suited to account for the uncertainties introduced by the numerical scheme and its induced filter form on the modeling task. For this coupled RL-DG framework, I will present discretization-aware model approaches for the LES equations and discuss the future potential of these solver-in-the-loop optimizations.

Tentative schedule:

1. Deep Neural Networks and Supervised Learning for augmenting PDEs:
   • Basic and Advanced Architectures for Supervised Learning
   • Incorporating constraints
   • Examples: Combining CFD and ML for solving real-world problems
2. Data-Driven Models for Large Eddy Simulation of Turbulence:
   • Large Eddy Simulation, turbulence and numerics
   • Subgrid scale modeling with ML
   • Model-Data Consistency
3. Deep Reinforcement Learning for turbulence:
   • Introduction to RL
   • Optimizable PDE solvers on HPC systems
   • Deep RL for turbulence modelling

This course will be followed by a practical session, where we will investigate model training for canonical turbulent flows through supervised learning and explore methods to incorporate physical constraints into the ML-based models. Tentative prerequisites: Python, NumPy, TensorFlow.

Full version in pdf

Andrea Beck’s research works