Skip to main content

Advertisement

SpringerLink
Log in

Estimating predictability limit from processes with characteristic timescale, Part I: AR(1) process

  • Research
  • Published:
Theoretical and Applied Climatology Aims and scope Submit manuscript

Abstract

Inferring intrinsic predictability (IP) or predictability limit (PL) from time series plays a crucial role in understanding complex systems and guiding predictions. Though PL is often considered to depend on the characteristic timescale (CT) of an underlying process, the quantitative relation between IP, PL and CT has not been well studied. As the simplest process with an adjustable CT, the Auto-Regression of order one, i.e. AR(1), is taken as a representative process to explore this quantitative relation, then this relation is leveraged to estimate PL. Our results show that directly estimating the PL highly relies on the CT of a specific AR(1) process, and the uncertainties and bias of PL estimations dramatically increase with the enhanced CT, which indicates that more data points and computational cost are required for reliably estimating PL from the process with a large CT value, and it is unrealizable to directly estimate PL from most of real-world series with limited length. To solve this problem, an IP metric, i.e. the time series predictability defined by the weighted permutation entropy (WPE), is proposed to indirectly estimate PL reliably with much lower uncertainties without biases for short series. The findings in this study can greatly improve the accuracy of PL estimation and in-depth understandings on the predictability studies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data availability

No external data are required for this work, as the figures produced are simulations of the proposed systems. Thus, the required data are contained in the work.

References

  • Bhaduri M (2020) On modifications to the Poisson-triggered hidden Markov paradigm through partitioned empirical recurrence rates ratios and its applications to natural hazards monitoring. Sci Rep 10:15889

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  • Bhaduri M, Ho C (2019) On a temporal investigation of hurricane strength and frequency. J Environ Model Assess 24:495–507

    Article  Google Scholar 

  • Boffetta G, Cencini M, Falcioni M, Vulpiani A (2002) Predictability: a way to characterize complexity. Phys Rep 356:367–474

    Article  ADS  MathSciNet  Google Scholar 

  • Ding RQ, Li JP (2007) Nonlinear finite-time Lyapunov exponent and predictability. Phys Lett A 364:396–400

    Article  ADS  CAS  Google Scholar 

  • Domeisen DI, Badin G, Koszalka IM (2018) How predictable are the Arctic and North Atlantic Oscillations? Exploring the variability and predictability of the Northern Hemisphere. J Climate 31:997–1014

    Article  ADS  Google Scholar 

  • Fadlallah B, Chen B, Keil A, Príncipe J (2013) Weighted-permutation entropy: A complexity measure for time series incorporating amplitude information. Phys Rev E 87:022911

    Article  ADS  Google Scholar 

  • Franzke CL, Barbosa S, Blender R, Fredriksen HB, Laepple T, Lambert F et al (2020) The structure of climate variability across scales. Rev Geophys 58:e2019RG000657

    Article  ADS  Google Scholar 

  • Fu S, Huang Y, Feng T, Nian D, Fu ZT (2019) Regional contrasting DTR’s predictability over China. Phys A 521:282–292

    Article  Google Scholar 

  • Garland J, James R, Bradley E (2014) Model-free quantification of time-series predictability. Phys Rev E 90:052910

    Article  ADS  Google Scholar 

  • Gong HH, Fu ZT (2022) Beyond linear correlation: Strong nonlinear structures in diurnal temperature range variability over southern China. Chaos Soliton Fract 164:112737

    Article  Google Scholar 

  • He WP, Xie X, Mei Y, Wan SQ, Zhao SS (2021) Decreasing predictability as a precursor indicator for abrupt climate change. Clim Dynam 56:3899–3908

    Article  ADS  Google Scholar 

  • Ho C, Bhaduri M (2015) On a novel approach to forecast sparse rare events: applications to Parkfield earthquake prediction. Nat Hazards 78:669–679

    Article  Google Scholar 

  • Ho C, Bhaduri M (2017) A quantitative insight into the dependence dynamics of the Kilauea and Mauna Loa volcanoes. Hawaii Math Geosci 49:893–911

    Article  MathSciNet  Google Scholar 

  • Höll M, Kantz H (2015) The relationship between the detrended fluctuation analysis and the autocorrelation function of a signal. Eur Phys J B 88:1–7

    Google Scholar 

  • Hou ZL, Li JP, Ding RQ, Karamperidou C, Duan WS, Liu T, Feng J (2018) Asymmetry of the predictability limit of the warm ENSO phase. Geophys Res Lett 45:7646–7653

    Article  ADS  Google Scholar 

  • Huang Y, Fu ZT (2019) Enhanced time series predictability with well-defined structures. Theor Appl Climatol 138:373–385

    Article  ADS  Google Scholar 

  • Itoh K, Nakada T (2013) Human brain detects short-time nonlinear predictability in the temporal fine structure of deterministic chaotic sounds. Phys Rev E 87:042916

    Article  ADS  Google Scholar 

  • Király A, Jánosi IM (2002) Stochastic modeling of daily temperature fluctuations. Phys Rev E 65:051102

    Article  ADS  Google Scholar 

  • Li JP, Ding RQ (2011) Temporal-spatial distribution of atmospheric predictability limit by local dynamical analogs. Mon Weather Rev 139:3265–3283

    Article  ADS  Google Scholar 

  • Li JY, Thompson DW (2021) Widespread changes in surface temperature persistence under climate change. Nature 599:425–430

    Article  ADS  CAS  PubMed  Google Scholar 

  • Li X, Ding RQ, Li JP (2020) Quantitative comparison of predictabilities of warm and cold events using the backward nonlinear local Lyapunov exponent method. Adv Atmos Sci 37:951–958

    Article  Google Scholar 

  • Lorenz EN (1963) Deterministic nonperiodic flow. J Atmos Sci 20:130–141

    Article  ADS  MathSciNet  Google Scholar 

  • Lorenz EN (1969) Atmospheric predictability as revealed by naturally occurring analogues. J Atmos Sci 26:636–646

    Article  ADS  Google Scholar 

  • Lorenz EN (1996) Predictability: a problem partly solved. In: proc. seminar on predictability vol I, reading. United Kingdom, ECMWF, 1–18

  • Meyer PG, Kantz H (2019) Inferring characteristic timescales from the effect of autoregressive dynamics on detrended fluctuation analysis. New J Phys 21:033022

    Article  MathSciNet  Google Scholar 

  • Nian D, Yuan NM, Ying KR, Liu G, Fu ZT, Qi Y, Franzke CL (2020) Identifying the sources of seasonal predictability based on climate memory analysis and variance decomposition. Clim Dynam 55:3239–3252

    Article  ADS  Google Scholar 

  • Önskog T, Franzke CL, Hannachi A (2018) Predictability and non-Gaussian characteristics of the North Atlantic Oscillation. J Climate 31:537–554

    Article  ADS  Google Scholar 

  • Patzelt F, Bouchaud JP (2018) Universal scaling and nonlinearity of aggregate price impact in financial markets. Phys Rev E 97:012304

    Article  ADS  CAS  PubMed  Google Scholar 

  • Pennekamp F, Iles AC, Garland J, Brennan G, Brose U, Gaedke U (2019) The intrinsic predictability of ecological time series and its potential to guide forecasting. Ecol Monogr 89:e01359

    Article  Google Scholar 

  • Riedl M, Müller A, Wessel N (2013) Practical considerations of permutation entropy. Eur Phys J 222:249–262

    Google Scholar 

  • Roe G (2009) Feedbacks, timescales, and seeing red. Annu Rev Earth Pl Sc 37:93–115

    Article  ADS  MathSciNet  CAS  Google Scholar 

  • Scaife AA, Smith D (2018) A signal-to-noise paradox in climate science. Npj Clim Atmos Sci 1:1–8

    Article  Google Scholar 

  • Shi H, Jin FF, Wills RC, Jacox MG, Amaya DJ, Black BA et al (2022) Global decline in ocean memory over the 21st century. Sci Adv 8:3468

    Article  Google Scholar 

  • Song C, Qu Z, Blumm N, Barabási AL (2010) Limits of predictability in human mobility. Science 327:1018–1021

    Article  ADS  MathSciNet  CAS  PubMed  Google Scholar 

  • Toth Z (1991) Estimation of atmospheric predictability by circulation analogs. Mon Wea Rev 119:65–72

    Article  ADS  Google Scholar 

  • Wolf A, Swift JB, Swinney HL, Vastano JA (1985) Determining Lyapunov exponents from a time series. Physica D 16:285–317

    Article  ADS  MathSciNet  Google Scholar 

  • Yoden S, Nomura M (1993) Finite-time Lyapunov stability analysis and its application to atmospheric predictability. J Atmos Sci 50:1531–1543

    Article  ADS  MathSciNet  Google Scholar 

  • Yuan NM, Lu ZH (2020) Warming reduces predictability. Nat. Clim Change 10:13–14

    ADS  CAS  Google Scholar 

  • Yuan NM, Fu ZT, Liu SD (2013) Long-term memory in climate variability: A new look based on fractional integral techniques. J Geophys Res-Atmos 118:12962

    Article  ADS  Google Scholar 

  • Yuan NM, Huang Y, Duan JP, Zhu C, Xoplaki E, Luterbacher J (2019) On climate prediction: how much can we expect from climate memory? Clim Dynam 52:855–864

    Article  ADS  Google Scholar 

  • Zhang YW, Fan JF, Marzocchi W, Shapira A, Hofstetter R, Havlin S, Ashkenazy Y (2020) Scaling laws in earthquake memory for interevent times and distances. Phys Rev Res 2:013264

    Article  CAS  Google Scholar 

  • Ziehmann C, Smith LA, Kurths J (2000) Localized Lyapunov exponents and the prediction of predictability. Phys Lett A 271:237–251

    Article  ADS  MathSciNet  CAS  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Nos. 41975059 and 42175065).

Author information

Authors and Affiliations

  1. Laboratory for Climate and Ocean-Atmosphere Studies, Department of Atmospheric and Oceanic Sciences, School of Physics, Peking University, Beijing, China

    Huanhuan Gong, Yu Huang & Zuntao Fu

  2. Earth System Modelling, School of Engineering & Design, Technical University of Munich, Munchen, Germany

    Yu Huang

Authors
  1. Huanhuan Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yu Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zuntao Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

H.H. Gong: Investigation, Software, Validation, Visualization, Writing-original draft. Y. Huang: Validation, Writing-original draft. Z.T. Fu: Supervision, Funding acquisition, Writing-review & editing.

Corresponding author

Correspondence to Zuntao Fu.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Keypoints

1. Well-defined monotonic relation is quantified between predictability limit (PL), intrinsic predictability (IP) and characteristic timescale (CT) of AR(1) process.

2. Directly estimating PL of AR(1) process is biased with large uncertainties due to finite data length.

3. A novel way by leveraging IP is proposed to unbiasedly estimate PL for AR(1) process with lower uncertainties.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gong, H., Huang, Y. & Fu, Z. Estimating predictability limit from processes with characteristic timescale, Part I: AR(1) process. Theor Appl Climatol (2024). https://doi.org/10.1007/s00704-024-04917-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00704-024-04917-7

Keywords

Search

Navigation