Skip to main content
SpringerLink
Log in

An efficient training method to learn a model of turbulence

  • Regular Article
  • Published:
The European Physical Journal Plus Aims and scope Submit manuscript

Abstract

Turbulence is the paradigm of complex systems, notably for its non-equilibrium multi-scale character. Since it is rarely possible to have access to direct simulations of turbulence, the development of efficient models is crucial. The use of machine learning tools has recently received ample attention, yet learning such a complex system is a challenging task. In the present work, we explore the limitations of machine learning in describing high-order statistics, which are essential for understanding turbulence structure. We will focus on shell models of turbulence. They have a manageable dimension and display the same characteristics as Navier–Stokes turbulence, such as Kolmogorov scaling and intermittency. We aim to develop a robust model using machine learning strategies, focusing on analysing recurrent and multi-layer networks. We have found that the standard protocol for training a recurrent neural network is not satisfactory for the description of rare events. The standard training approach creates a discrepancy between what the model learns and what the model has to be able to do. We propose a new training method called Forecast Training. Our extensive statistical analysis shows that this method significantly improves performance. We can train a model that gives excellent agreement with the actual truth, even with limited training data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Algorithm 2
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data Availability Statement

All data are available upon request. The source code can be found at https://github.com/raudh1/ShellModels.

Notes

  1. Bias terms are omitted.

References

  1. B. Stephen, Pope, Turbulent flows (Cambridge University Press, Cambridge, 2000)

    Google Scholar 

  2. U. Frisch, Turbulence: The Legacy of A (N. Kolmogorov. Cambridge University Press, Cambridge, 1995). https://doi.org/10.1017/CBO9781139170666

    Book  Google Scholar 

  3. A.S. Monin, A.M. Yaglom, Statistical fluid mechanics: mechanics of turbulence. Courier Corporation, (2013)

  4. A.N. Kolmogorov, The local structure of turbulence in incompressible viscous fluid for very large reynolds numbers. Proc. R. Soc. London Ser. A Math. Phys. Sci. 434(1890), 9–13 (1991). https://doi.org/10.1098/rspa.1991.0075

    Article  ADS  MathSciNet  Google Scholar 

  5. R. Benzi, G. Paladin, G. Parisi, A. Vulpiani, On the multifractal nature of fully developed turbulence and chaotic systems. J. Phys. A Math. Gen. 17(18), 3521 (1984). https://doi.org/10.1088/0305-4470/17/18/021

    Article  ADS  MathSciNet  Google Scholar 

  6. L.D. Landau, E.M. Lifshitz, Fluid Mechanics: Landau and Lifshitz: Course of Theoretical Physics, Volume 6, volume 6. Elsevier, (2013)

  7. T. Bohr, M.H. Jensen, G. Paladin, and A. Vulpiani. Dynamical Systems Approach to Turbulence. Cambridge Nonlinear Science Series. Cambridge University Press, (1998). https://doi.org/10.1017/CBO9780511599972

  8. S. Bengio, O. Vinyals, N. Jaitly, N. Shazeer. Scheduled sampling for sequence prediction with recurrent neural networks. Advances in neural information processing systems, 28, (2015)

  9. S.L. Brunton, B.R. Noack, P. Koumoutsakos, Machine learning for fluid mechanics. Ann. Rev. Fluid Mech. 52, 477–508 (2020). https://doi.org/10.1146/annurev-fluid-010719-060214

    Article  ADS  MathSciNet  Google Scholar 

  10. S. Cheng, R. Morel, E. Allys, B. Ménard, S. Mallat, Scattering spectra models for physics. arXiv preprint arXiv:2306.17210, (2023)

  11. F. Borra, M. Baldovin, Using machine-learning modeling to understand macroscopic dynamics in a system of coupled maps. Chaos Interd. J. Nonlinear Sci. (2021). https://doi.org/10.1063/5.0036809

    Article  Google Scholar 

  12. J. Pathak, B. Hunt, M. Girvan, L. Zhixin, E. Ott, Model-free prediction of large spatiotemporally chaotic systems from data: A reservoir computing approach. Phys. Rev. Lett. 120(2), 024102 (2018). https://doi.org/10.1103/PhysRevLett.120.024102

    Article  ADS  Google Scholar 

  13. M. Buzzicotti, F. Bonaccorso, P. Clark Di Leoni, L. Biferale, Reconstruction of turbulent data with deep generative models for semantic inpainting from turb-rot database. Phys. Rev. Fluids 6(5), 050503 (2021). https://doi.org/10.1103/PhysRevFluids.6.050503

    Article  ADS  Google Scholar 

  14. P.C.D. Leoni, A. Mazzino, L. Biferale, Synchronization to big data: Nudging the navier-stokes equations for data assimilation of turbulent flows. Phys. Rev. X 10(1), 011023 (2020). https://doi.org/10.1103/PhysRevX.10.011023

    Article  Google Scholar 

  15. L. Biferale, Shell models of energy cascade in turbulence. Ann. Rev. Fluid Mech. 35, 441–468 (2003). https://doi.org/10.1146/annurev.fluid.35.101101.161122

    Article  ADS  MathSciNet  Google Scholar 

  16. P.D. Ditlevsen, Turbulence and Shell Models (Cambridge University Press, Cambridge, 2010)

    Book  Google Scholar 

  17. M.A. Bucci, O. Semeraro, A. Allauzen, S. Chibbaro, L. Mathelin, Curriculum learning for data-driven modeling of dynamical systems. Eur. Phys. J. E 46(3), 12 (2023). https://doi.org/10.1140/epje/s10189-023-00269-8

    Article  Google Scholar 

  18. V. Lvov, E. Podivilov, A. Pomyalov, I. Procaccia, D. Vandembroucq, Improved shell model of turbulence. Phys. Rev. E 58, 03 (1998). https://doi.org/10.1103/PhysRevE.58.1811

    Article  MathSciNet  Google Scholar 

  19. M. Yamada, K. Ohkitani, Lyapunov spectrum of a chaotic model of three-dimensional turbulence. J. Phys. Soc. Jpn. 56(12), 4210–4213 (1987). https://doi.org/10.1143/JPSJ.56.4210

    Article  ADS  Google Scholar 

  20. F. Cecconi, A. Vulpiani, Approximation of chaotic systems in terms of markovian processes. Phys. Lett. A 201(4), 326–332 (1995). https://doi.org/10.1016/0375-9601(95)00286-C

    Article  ADS  MathSciNet  Google Scholar 

  21. M.H. Jensen, G. Paladin, A. Vulpiani, Intermittency in a cascade model for three-dimensional turbulence. Phys. Rev. A 43, 798–805 (1991). https://doi.org/10.1103/PhysRevA.43.798

    Article  ADS  Google Scholar 

  22. D. Pisarenko, L. Biferale, D. Courvoisier, U. Frisch, M. Vergassola, Further results on multifractality in shell models. Phys. Fluids A Fluid Dyn. 5(10), 2533–2538 (1993). https://doi.org/10.1063/1.858766

    Article  ADS  Google Scholar 

  23. J.L. McClelland, D.E. Rumelhart, PDP Research. Group, et al, Parallel Distributed Processing, Volume 2: Explorations in the Microstructure of Cognition: Psychological and Biological Models, volume 2. MIT press, (1987)

  24. S. Hochreiter, J. Schmidhuber, Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997). https://doi.org/10.1162/neco.1997.9.8.1735

    Article  Google Scholar 

  25. A.J. Robinson, F. Fallside, The utility driven dynamic error propagation network. Technical Report CUED/F-INFENG/TR.1, Engineering Department, Cambridge University, Cambridge, UK, (1987)

  26. R.J. Williams, D. Zipser, A learning algorithm for continually running fully recurrent neural networks. Neural Comput. 1(2), 270–280 (1989). https://doi.org/10.1162/neco.1989.1.2.270

    Article  Google Scholar 

  27. Y. Bengio, P. Simard, P. Frasconi, Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Netw. 5(2), 157–166 (1994). https://doi.org/10.1109/72.279181

    Article  Google Scholar 

  28. R. Pascanu, T. Mikolov, Y. Bengio, On the difficulty of training recurrent neural networks. In Proceedings of the 30th International Conference on Machine Learning, ICML 2013, Atlanta, GA, USA, 16-21 June 2013, volume 28 of JMLR Proceedings, pages 1310–1318. JMLR.org, (2013). URL: http://jmlr.org/proceedings/papers/v28/pascanu13.html

  29. S. Wiseman, A.M. Rush, Sequence-to-sequence learning as beam-search optimization. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1296–1306, Austin, Texas, November 2016. Association for Computational Linguistics. URL: https://aclanthology.org/D16-1137, https://doi.org/10.18653/v1/D16-1137

  30. H. Daumé, J. Langford, D. Marcu, Search-based structured prediction. Mach. Learn. 75(3), 297–325 (2009). https://doi.org/10.1007/s10994-009-5106-x

    Article  Google Scholar 

  31. R. Leblond, J.B. Alayrac, A. Osokin, S. Lacoste-Julien, SEARNN: Training RNNs with global-local losses. In International Conference on Learning Representations, (2018). URL: https://openreview.net/forum?id=HkUR_y-RZ

  32. M. Sangiorgio, F. Dercole, Robustness of lstm neural networks for multi-step forecasting of chaotic time series. Chaos Solitons Fract. 139, 110045 (2020). https://doi.org/10.1016/j.chaos.2020.110045

    Article  MathSciNet  Google Scholar 

  33. C. Tallec, Y. Ollivier, Can recurrent neural networks warp time? arXiv:1804.11188 (2018)

  34. P.N. Brown, G.D. Byrne, A.C. Hindmarsh, Vode: a variable-coefficient ode solver. SIAM J. Sci. Stat. Comput. 10, 1038–1051 (1989). https://doi.org/10.1137/0910062

    Article  MathSciNet  Google Scholar 

  35. Z.S. She, E. Leveque, Universal scaling laws in fully developed turbulence. Phys. Rev. Lett. 72, 336–339 (1994). https://doi.org/10.1103/PhysRevLett.72.336

    Article  ADS  Google Scholar 

  36. P. Castiglione, M. Falcioni, A. Lesne, A. Vulpiani, Chaos and Coarse Graining in Statistical Mechanics (Cambridge University Press, Cambridge, 2008)

    Book  Google Scholar 

  37. B. Mehlig, Machine Learning with Neural Networks: An Introduction for Scientists and Engineers (Cambridge University Press, Cambridge, 2021)

    Book  Google Scholar 

  38. T. Pfaff, M. Fortunato, A. Sanchez-Gonzalez, P.W. Battaglia, Learning mesh-based simulation with graph networks. arXiv:2010.03409, (2020)

  39. A. Sanchez-Gonzalez, N. Heess, J.T. Springenberg, J. Merel, M. Riedmiller, R. Hadsell, P. Battaglia, Graph networks as learnable physics engines for inference and control. In International Conference on Machine Learning, pages 4470–4479. PMLR, (2018)

  40. J. Su, J. Kempe, D. Fielding, N. Tsilivis, M. Cranmer, S. Ho, Adversarial noise injection for learned turbulence simulations. In Machine Learning for Physical Sciences Workshop, Advances in Neural Information Processing Systems, (2022)

Download references

Funding

This study was funded by ANR (Agence Nationale de la Recherche) within the framework of the SPEED (Simulating Physical PDEs Efficiently with Deep Learning) project ANR-20-CE23-0025-01.

Author information

Authors and Affiliations

  1. Institut Jean Le Rond d’Alembert, Sorbonne Université, Paris, France

    Daniele Noto

  2. CNRS, UMR 9015, LISN, Université Paris-Saclay, 91405, Orsay Cedex, France

    Daniele Noto & Sergio Chibbaro

  3. LAMSADE, Université Paris Dauphine, Paris, France

    Alexandre Allauzen

Authors
  1. Daniele Noto
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alexandre Allauzen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sergio Chibbaro
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daniele Noto.

Additional information

Guest editors: S. Chibbaro, M.-C. Firpo, G. Napoli and S. Randoux.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Noto, D., Allauzen, A. & Chibbaro, S. An efficient training method to learn a model of turbulence. Eur. Phys. J. Plus 139, 298 (2024). https://doi.org/10.1140/epjp/s13360-024-05056-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1140/epjp/s13360-024-05056-8

Search

Navigation