├── resources ├── learning-similarity-metrics-divider.jpeg ├── physics-based-deep-learning-overview.jpg └── physics-based-deep-learning-teaser1.jpg └── README.md /resources/learning-similarity-metrics-divider.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tum-pbs/Physics-Based-Deep-Learning/HEAD/resources/learning-similarity-metrics-divider.jpeg -------------------------------------------------------------------------------- /resources/physics-based-deep-learning-overview.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tum-pbs/Physics-Based-Deep-Learning/HEAD/resources/physics-based-deep-learning-overview.jpg -------------------------------------------------------------------------------- /resources/physics-based-deep-learning-teaser1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tum-pbs/Physics-Based-Deep-Learning/HEAD/resources/physics-based-deep-learning-teaser1.jpg -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Physics-Based Deep Learning 2 | 3 | The following collection of materials targets _"Physics-Based Deep Learning"_ 4 | (PBDL), i.e., the field of methods with combinations of physical modeling and 5 | deep learning (DL) techniques. Here, DL will typically refer to methods based 6 | on artificial neural networks. The general direction of PBDL represents a very 7 | active and quickly growing field of research. 8 | 9 | ![An overview of categories of physics-based deep learning methods](resources/physics-based-deep-learning-overview.jpg) 10 | 11 | Within this area, we can distinguish a variety of different physics-based 12 | approaches, from targeting designs, constraints, combined methods, and 13 | optimizations to applications. More specifically, all approaches either target 14 | _forward_ simulations (predicting state or temporal evolution) or _inverse_ 15 | problems (e.g., obtaining a parametrization for a physical system from 16 | observations). 17 | Apart from forward or inverse, the type of integration between learning 18 | and physics gives a means for categorizing different methods: 19 | 20 | - _Data-driven_: the data is produced by a physical system (real or simulated), 21 | but no further interaction exists. 22 | 23 | - _Loss-terms_: the physical dynamics (or parts thereof) are encoded in the 24 | loss function, typically in the form of differentiable operations. The 25 | learning process can repeatedly evaluate the loss, and usually receives 26 | gradients from a PDE-based formulation. 27 | 28 | - _Interleaved_: the full physical simulation is interleaved and combined with 29 | an output from a deep neural network; this requires a fully differentiable 30 | simulator and represents the tightest coupling between the physical system and 31 | the learning process. Interleaved approaches are especially important for 32 | temporal evolutions, where they can yield an estimate of future behavior of the 33 | dynamics. 34 | 35 | Thus, methods can be roughly categorized in terms of forward versus inverse 36 | solve, and how tightly the physical model is integrated into the 37 | optimization loop that trains the deep neural network. Here, especially approaches 38 | that leverage _differentiable physics_ allow for a tighter and tighter integration 39 | of deep learning and numerical simulations. 40 | 41 | This repository collects links to works on _deep learning algorithms for physics 42 | problems_, with a particular emphasis on _fluid flow_, i.e., Navier-Stokes related 43 | problems. It primarily collects links to the work of the I15 lab at TUM, as 44 | well as miscellaneous works from other groups. This is by no means a complete 45 | list, so let us know if you come across additional papers in this area. We 46 | intentionally also focus on works from the _deep learning_ field, not machine 47 | learning in general. 48 | 49 | ![An example flow result from tempoGAN](resources/physics-based-deep-learning-teaser1.jpg) 50 | 51 | 52 | ## I15 Physics-based Deep Learning Links 53 | 54 | Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers , 55 | Project+Code: 56 | 57 | Numerical investigation of minimum drag profiles in laminar flow using deep learning surrogates , 58 | PDF: 59 | 60 | Purely data-driven medium-range weather forecasting achieves comparable skill to physical models at similar resolution , 61 | PDF: 62 | 63 | Data-driven Regularization via Racecar Training for Generalizing Neural Networks , 64 | PDF: 65 | 66 | Latent Space Subdivision: Stable and Controllable Time Predictions for Fluid Flow , 67 | PDF: 68 | 69 | WeatherBench: A benchmark dataset for data-driven weather forecasting , 70 | Project: 71 | 72 | Learning Similarity Metrics for Numerical Simulations (LSiM) , 73 | Project+Code: 74 | 75 | Learning to Control PDEs with Differentiable Physics , 76 | Project+Code: 77 | 78 | Lagrangian Fluid Simulation with Continuous Convolutions , 79 | PDF: 80 | 81 | Tranquil-Clouds: Neural Networks for Learning Temporally Coherent Features in Point Clouds , 82 | Project+Code: 83 | 84 | ScalarFlow: A Large-Scale Volumetric Data Set of Real-world Scalar Transport Flows for Computer Animation and Machine Learning , 85 | Project+Code: 86 | 87 | tempoGAN: A Temporally Coherent, Volumetric GAN for Super-resolution Fluid Flow , 88 | Project+Code: 89 | 90 | Deep Fluids: A Generative Network for Parameterized Fluid Simulations , 91 | Project+Code: 92 | 93 | Latent-space Physics: Towards Learning the Temporal Evolution of Fluid Flow , 94 | Project+Code: 95 | 96 | A Multi-Pass GAN for Fluid Flow Super-Resolution , 97 | PDF: 98 | 99 | A Study of Deep Learning Methods for Reynolds-Averaged Navier-Stokes Simulations , 100 | Project+Code: 101 | 102 | Data-Driven Synthesis of Smoke Flows with CNN-based Feature Descriptors , 103 | Project+Code: 104 | 105 | Liquid Splash Modeling with Neural Networks , 106 | Project+Code: 107 | 108 | Generating Liquid Simulations with Deformation-aware Neural Networks , 109 | Project+Code: 110 | 111 | 112 | ## Additional Links for Fluids 113 | 114 | Super-resolution and denoising of fluid flow using physics-informed convolutional neural networks without high-resolution labels , 115 | PDF: 116 | 117 | A Point-Cloud Deep Learning Framework for Prediction of Fluid Flow Fields on Irregular Geometries , 118 | PDF: 119 | 120 | Learning Mesh-Based Simulations with Graph Networks , 121 | PDF: 122 | 123 | Using Machine Learning to Augment Coarse-Grid Computational Fluid Dynamics Simulations , 124 | PDF: 125 | 126 | Learning to swim in potential flow , 127 | PDF: 128 | 129 | Enhanced data efficiency using deep neural networks and Gaussian processes for aerodynamic design optimization , 130 | PDF: 131 | 132 | Learned discretizations for passive scalar advection in a 2-D turbulent flow , 133 | PDF: 134 | 135 | Combining Differentiable PDE Solvers and Graph Neural Networks for Fluid Flow Prediction , 136 | PDF: 137 | 138 | CFDNet: A deep learning-based accelerator for fluid simulations , 139 | PDF: 140 | 141 | Controlling Rayleigh-Benard convection via Reinforcement Learning , 142 | PDF: 143 | 144 | Embedding Hard Physical Constraints in Neural Network Coarse-Graining of 3D Turbulence , 145 | PDF: 146 | 147 | Learning to Simulate Complex Physics with Graph Networks , 148 | PDF: 149 | 150 | DPM: A deep learning PDE augmentation method (with application to large-eddy simulation) , 151 | PDF: 152 | 153 | Towards Physics-informed Deep Learning for Turbulent Flow Prediction , 154 | PDF: 155 | 156 | DeepFlow: History Matching in the Space of Deep Generative Models , 157 | PDF: 158 | 159 | Deep learning observables in computational fluid dynamics , 160 | PDF: 161 | 162 | Compressed convolutional LSTM: An efficient deep learning framework to model high fidelity 3D turbulence , 163 | PDF: 164 | 165 | Deep neural networks for data-driven LES closure models , 166 | PDF: 167 | 168 | Dynamic Upsampling of Smoke through Dictionary-based Learning , 169 | PDF: 170 | 171 | Deep unsupervised learning of turbulence for inflow generation at various Reynolds numbers , 172 | PDF: 173 | 174 | Computing interface curvature from volume fractions: A machine learning approach , 175 | PDF: 176 | 177 | Deep Neural Networks for Data-Driven Turbulence Models , 178 | PDF: 179 | 180 | Deep Dynamical Modeling and Control of Unsteady Fluid Flows , 181 | PDF: 182 | 183 | Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids , 184 | Project+Code: 185 | 186 | Application of Convolutional Neural Network to Predict Airfoil Lift Coefficient , 187 | PDF: 188 | 189 | Prediction of laminar vortex shedding over a cylinder using deep learning , 190 | PDF: 191 | 192 | Lat-Net: Compressing Lattice Boltzmann Flow Simulations using Deep Neural Networks , 193 | PDF: 194 | 195 | Reasoning About Liquids via Closed-Loop Simulation , 196 | PDF: 197 | 198 | Prediction model of velocity field around circular cylinder over various Reynolds numbers by fusion convolutional neural networks based on pressure on the cylinder , 199 | PDF: 200 | 201 | Accelerating Eulerian Fluid Simulation With Convolutional Networks , 202 | Project+Code: 203 | 204 | Reynolds averaged turbulence modelling using deep neural networks with embedded invariance , 205 | PDF: 206 | 207 | ![Image divider for general PDE section](resources/learning-similarity-metrics-divider.jpeg) 208 | 209 | 210 | ## Additional Links for General PDEs 211 | 212 | Aphynity: Augmenting physical models with deep networks for complex dynamics forecasting , 213 | PDF: 214 | 215 | Hierarchical Deep Learning of Multiscale Differential Equation Time-Steppers , 216 | PDF: 217 | 218 | Learning Compositional Koopman Operators for Model-Based Control , 219 | Project: 220 | 221 | Universal Differential Equations for Scientific Machine Learning , 222 | PDF: 223 | 224 | Understanding and mitigating gradient pathologies in physics-informed neural networks , 225 | PDF: 226 | 227 | Variational Physics-Informed Neural Networks For Solving Partial Differential Equations , 228 | PDF: 229 | 230 | Poisson CNN: Convolutional Neural Networks for the Solution of the Poisson Equation with Varying Meshes and Dirichlet Boundary Conditions , 231 | PDF: 232 | 233 | IDENT: Identifying Differential Equations with Numerical Time evolution , 234 | PDF: 235 | 236 | PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network , 237 | PDF: 238 | 239 | Data-driven discretization: a method for systematic coarse graining of partial differential equations , 240 | PDF: 241 | 242 | Solving high-dimensional partial differential equations using deep learning , 243 | PDF: 244 | 245 | Neural Ordinary Differential Equations , 246 | PDF: 247 | 248 | Deep Learning the Physics of Transport Phenomena , 249 | PDF: 250 | 251 | DGM: A deep learning algorithm for solving partial differential equations , 252 | PDF: 253 | 254 | Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations , 255 | PDF: 256 | 257 | Data-assisted reduced-order modeling of extreme events in complex dynamical systems , 258 | Project+Code: 259 | 260 | PDE-Net: Learning PDEs from Data , 261 | Project+Code: 262 | 263 | Learning Deep Neural Network Representations for Koopman Operators of Nonlinear Dynamical Systems , 264 | PDF: 265 | 266 | 267 | ## Additional Links for Other Physics Problems and Physics-related Problems 268 | 269 | Deep Energy-based Modeling of Discrete-Time Physics , 270 | PDF: 271 | 272 | NeuralSim: Augmenting Differentiable Simulators with Neural Networks , 273 | PDF: 274 | 275 | Fourier Neural Operator for Parametric Partial Differential Equations , 276 | PDF: 277 | 278 | Learning Composable Energy Surrogates for PDE Order Reduction , 279 | PDF: 280 | 281 | Transformers for Modeling Physical Systems , 282 | PDF: 283 | 284 | Reinforcement Learning for Molecular Design Guided by Quantum Mechanics , 285 | PDF: 286 | 287 | Scalable Differentiable Physics for Learning and Control , 288 | PDF: 289 | 290 | Cloth in the Wind: A Case Study of Physical Measurement through Simulation , 291 | PDF: 292 | 293 | Learning to Slide Unknown Objects with Differentiable Physics Simulations , 294 | PDF: 295 | 296 | Physics-aware Difference Graph Networks for Sparsely-Observed Dynamics , 297 | Project: 298 | 299 | Differentiable Molecular Simulations for Control and Learning , 300 | PDF: 301 | 302 | Incorporating Symmetry into Deep Dynamics Models for Improved Generalization , 303 | PDF: 304 | 305 | Learning to Measure the Static Friction Coefficient in Cloth Contact , 306 | PDF: 307 | 308 | Learning to Simulate Complex Physics with Graph Networks , 309 | PDF: 310 | 311 | Hamiltonian Neural Networks , 312 | PDF: 313 | 314 | Interactive Differentiable Simulation , 315 | PDF: 316 | 317 | DiffTaichi: Differentiable Programming for Physical Simulation , 318 | PDF: 319 | 320 | COPHY: Counterfactual Learning of Physical Dynamics , 321 | Project: 322 | 323 | Modeling Expectation Violation in Intuitive Physics with Coarse Probabilistic Object Representations , 324 | Project: 325 | 326 | End-to-End Differentiable Physics for Learning and Control , 327 | Project+Code: 328 | 329 | Stochastic seismic waveform inversion using generative adversarial networks as a geological prior , 330 | PDF: 331 | 332 | Learning to Optimize Multigrid PDE Solvers , 333 | PDF: 334 | 335 | Latent-space Dynamics for Reduced Deformable Simulation , 336 | Project+Code: 337 | 338 | Learning-Based Animation of Clothing for Virtual Try-On , 339 | PDF: 340 | 341 | Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning , 342 | PDF: 343 | 344 | Flexible Neural Representation for Physics Prediction , 345 | Project+Code: 346 | 347 | Robust Reference Frame Extraction from Unsteady 2D Vector Fields with Convolutional Neural Networks , 348 | PDF: 349 | 350 | Physics-as-Inverse-Graphics: Joint Unsupervised Learning of Objects and Physics from Video , 351 | PDF: 352 | 353 | Unsupervised Intuitive Physics from Past Experiences , 354 | PDF: 355 | 356 | Reasoning About Physical Interactions with Object-Oriented Prediction and Planning , 357 | PDF: 358 | 359 | Neural Material: Learning Elastic Constitutive Material and Damping Models from Sparse Data , 360 | PDF: 361 | 362 | Discovering physical concepts with neural networks , 363 | PDF: 364 | 365 | Fluid directed rigid body control using deep reinforcement learning , 366 | Project: 367 | 368 | DeepMimic, Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills , 369 | PDF: 370 | 371 | Unsupervised Intuitive Physics from Visual Observations , 372 | PDF: 373 | 374 | Graph networks as learnable physics engines for inference and control , 375 | PDF: 376 | 377 | DeepWarp: DNN-based Nonlinear Deformation , 378 | PDF: 379 | 380 | A proposal on machine learning via dynamical systems , 381 | Journal: 382 | 383 | Interaction Networks for Learning about Objects, Relations and Physics , 384 | PDF: 385 | 386 | 387 | 388 | ## Surveys and Overview Articles 389 | 390 | Integrating Physics-Based Modeling with Machine Learning: A Survey , 391 | PDF: 392 | 393 | Integrating Machine Learning with Physics-Based Modeling , 394 | PDF: 395 | 396 | A review on Deep Reinforcement Learning for Fluid Mechanics , 397 | PDF: 398 | 399 | Machine Learning for Fluid Mechanics , 400 | PDF: 401 | 402 | 403 | 404 | ## Simulation and Deep Learning Frameworks 405 | 406 | phiflow: 407 | 408 | diff-taichi: 409 | 410 | jax-md: 411 | 412 | 413 | # Concluding Remarks 414 | 415 | Physics-based deep learning is a very dynamic field. Please let us know if we've overlooked 416 | papers that you think should be included by sending a mail to _i15ge at cs.tum.de_, 417 | and feel free to check out our homepage at . 418 | 419 | --------------------------------------------------------------------------------