Chaotic time series. Part II. System Identification and Prediction

“Chaotic time series. Part II. System Identification and Prediction”

Authors: Bjørn Lillekjendlie, Dimitris Kugiumtzis and Nils Christophersen,
Affiliation: SINTEF and University of Oslo
Reference: 1994, Vol 15, No 4, pp. 225-245.

Keywords: Nonlinear systems, chaos, prediction, time series, forecasting

Abstract: This paper is the second in a series of two, and describes the current state of the art in modeling and prediction of chaotic time series. Sample data from deterministic non-linear systems may look stochastic when analysed with linear methods. However, the deterministic structure may be uncovered and non-linear models constructed that allow improved prediction. We give the background for such methods from a geometrical point of view, and briefly describe the following types of methods: global polynomials, local polynomials, multilayer perceptrons and semi-local methods including radial basis functions. Some illustrative examples from known chaotic systems are presented, emphasising the increase in prediction error with time. We compare some of the algorithms with respect to prediction accuracy and storage requirements, and list applications of these methods to real data from widely different areas.

PDF

PDF (2723 Kb) DOI: 10.4173/mic.1994.4.2

DOI forward links to this article:

[1] G.P. Pavlos, M.A. Athanasiu, G.C. Anagnostopoulos, A.G. Rigas and E.T. Sarris (2004), doi:10.1016/j.pss.2003.06.009

[2] L. Barbosa, L. Duarte, C. Linhares and L. da Mota (2006), doi:10.1103/PhysRevE.74.026702

[3] C.C. Holmes and B.K. Mallick (2000), doi:10.1109/72.822507

[4] H. Carli, L.G.S. Duarte and L.A.C.P. da Mota (2013), doi:10.1016/j.cpc.2013.12.001

[5] D. Kugiumtzis (1997), doi:10.1016/S0167-2789(96)00292-8

[6] Balazs Feil, Balazs Balasko and Janos Abonyi (2006), doi:10.1007/s00500-006-0111-5

[7] D. Kugiumtzis, O.C. Lingjærde and N. Christophersen (1998), doi:10.1016/S0167-2789(97)00171-1

[8] Rainer Hegger, Holger Kantz and Thomas Schreiber (1999), doi:10.1063/1.166424

[9] K. K. Phoon, M. N. Islam, C. Y. Liaw and S. Y. Liong (2002), doi:10.1061/(ASCE)1084-0699(2002)7:2(116)

[10] Masoud Mirmomeni, Caro Lucas, Babak Nadjar Araabi, Behzad Moshiri and Mohammad Reza Bidar (2011), doi:10.1007/s11207-011-9810-x

[11] A. C. ILIOPOULOS and G. P. PAVLOS (2010), doi:10.1142/S0218127410026939

[12] J. McNAMES, J. A. K. SUYKENS and J. VANDEWALLE (1999), doi:10.1142/S0218127499001048

[13] Lisa Borland (1996), doi:10.1016/S0167-2789(96)00143-1

[14] D Kugiumtzis (1996), doi:10.1016/0167-2789(96)00054-1

[15] Francesco Lisi and Vigilio Villi (2001), doi:10.1111/j.1752-1688.2001.tb00967.x

[16] Ali Gholipour, Caro Lucas, Babak N. Araabi, Masoud Mirmomeni and Masoud Shafiee (2007), doi:10.1007/s00521-006-0062-x

[17] Skander Soltani (2002), doi:10.1016/S0925-2312(01)00648-8

[18] Masoud Mirmomeni, E. Kamaliha, Masoud Shafiee and Caro Lucas (2009), doi:10.1186/BF03352959

[19] J. Abonyi, B. Feil, S. Nemeth, P. Arva and R. Babuska (2004), doi:10.1109/ICSMC.2004.1400684

[20] Runjie Liu, Zhan Hou and Jinyuan Shen (2009), doi:10.1109/IWCFTA.2009.65

[21] G. de. A. Barreto and A.F.R. Araujo (2001), doi:10.1109/IJCNN.2001.938498

[22] M. Ataei, A. Khaki-Sedigh and B. Lohmann (2003), doi:10.1016/S1474-6670(17)34757-2

[23] Shu-Fai Wong and Kwan-Yee Kenneth Wong (2004), doi:10.1007/978-3-540-28648-6_117

[24] Pilar Gomez-Gil, Angel Garcia-Pedrero and Juan Manuel Ramirez-Cortes (2010), doi:10.1007/978-3-642-15534-5_16

[25] D. Kugiumtzis (2002), doi:10.1007/978-1-4615-0931-8_5

[26] D. Guégan and L. Mercier (1998), doi:10.1007/978-1-4612-1768-8_25

[27] Javad Sharifi and Nafiseh Saeednia (2019), doi:10.1007/s40998-019-00227-1

[28] Kamalanand Krishnamurthy, Sujatha C. Manoharan and Ramakrishnan Swaminathan (2019), doi:10.1007/s12652-019-01525-6

[29] Ay e and Fatih ÇEMREK (2019), doi:10.17093/alphanumeric.629722

[30] D. Guégan and L. Mercier (2005), doi:10.1080/13518470110074846

[31] K. Kiran, K. C. Ajithprasad, V. M. Ananda Kumar and K. P. Harikrishnan (2021), doi:10.1007/s11207-021-01761-0

[32] M. Ataei, A. Khaki-Sedigh, B. Lohmann and C. Lucas (2003), doi:10.23919/ECC.2003.7086516

References:
[1] Abarbanel, Henry D.I., Reggie Brown, James B. Kadtke, (1989). Prediction and system identification in chaotic nonlinear systems: Time series with broadband spectra, Physics Letters A. 138(8):401-108, Jul 1989 doi:10.1016/0375-9601(89)90839-6
[2] Abarbanel, Henry D.I. , Reggie Brown, James B. Kadtke, (1990). Prediction in chaotic nonlinear systems: Methods for time series with broadband Fourier spectra, Physics Rewiev A, 4.4:1782-1807, Feb 1990 doi:10.1103/PhysRevA.41.1782
[3] Albus, James, (1975). Data storage in the cerebellar model articulation controller, cmac. Journal of Dynamic Systems, Measurement, and Control. Transactions of the ASME, pages 228-233. Sep 1975.
[4] Albus, James, (1975). A new approach to manipulator control: The cerebellar model articulation controller, CMAC. Journal of Dynamic Systems; Measurement, and Control, Transactions of the ASME, pages 220-227. Sep 1975.
[5] Barnsley, Michael (1988). Fractals everywhere, Academic Press, Inc., New York.
[6] Broomhead, D.S. D. Lowe (1988). Multivariable functional interpolation and adaptive networks, Complex Systems, 2:321-355, 1988.
[7] Caddell, David T. (1991). Neural networks and the mathematics of chaos - an investigation of these methodologies as accurate predictors of corporate bankruptcy, In Proc. of The First International Conference on Artificial Intelligence Applications on Wall Street, New York, Oct 1991.
[8] Carlin, Mats. (1991). Neural nets for empirical modelling, Master´s thesis, Norwegian Institute of Technology.NTH. 1991.
[9] Casdagli, Martin. (1989). Nonlinear prediction of chaotic time series, Physica D, 35:335-356, 1989 doi:10.1016/0167-2789(89)90074-2
[10] Casdagli, Martin. (1991). Chaos and deterministic versus stochastic and non-linear modelling, J. R. Statist. Soc. B, 5.2:303-328, 1991.
[11] Casti, J.L. (1977). Dynamical systems and their applications - Linear theory, Academic Press, New York, 1977.
[12] Cortini, M. C.C. Barton. (1993). Nonlinear forecasting analyses of inflation-deflation patterns of an active caldera, campi flegrei, italy. Geology. 21:239-242, Mar 1993.
[13] Cortini, M., L. Cilento, A. Runo. (1991). Vertical ground movements in the campi flegrei caldera as a chaotic dynamic phenomenon, Journal of Vulcanology and Geothermal Research, 48:199-222, 1991 doi:10.1016/0377-0273(91)90036-Y
[14] Cybenko, G. (1989). Approximation by superpositions of a sigmoidal function, Math. Control, Signals Sys., 2:303-314, 1989 doi:10.1007/BF02551274
[15] Deco, Gustavo Jürgen Ebmeyer. (1993). Coarse coding resource-allocating networks, Neural Computation, .1:105-114. Jan 1993 doi:10.1162/neco.1993.5.1.105
[16] Deller, James R. (1989). Set membership identification in digital signal processing, IEEE ASAP Magazine, 4:4-20, 1989.
[17] Deppisch, J., H.U. Bauser, T. Geisel. (1991). Hierarchical training of neural networks and prediction of chaotic time series, Physics Letters, A(158):57-63, 1991 doi:10.1016/0375-9601(91)90340-E
[18] Efron, Bradley. (1982). The Jackknife, the Bootstrap and Other Resampling Plans, Society of industrial and Applied Mathematics, 1982.
[19] Elsner, J.B. (1992). Prediciting time series using a neural network as a method of distinguishing chaos from noise, J. Phys. A: Math. Gen, .25:843-850, 1992.
[20] Elsner, J.B. A.A. Tsonis. (1992). Nonlinear prediciting, chaos and noise, Bulletin American Meteorological Society, 73:49-60, jan 1992.
[21] Farmer, J.D. J.J. Sidorowitch. (1987). Predicting chaotic time series, Physical Review Letters, 5.8:845-848, Aug 1987 doi:10.1103/PhysRevLett.59.845
[22] Franke, Richard. (1982). Scattered data interpolation: Tests of some methods, Mathematics of Computation, 3.157:181-200, Jan 1982 doi:10.2307/2007474
[23] Giona, M., F. Lentini, V. Cimagalli. (1991). Functional reconstruction and local prediction of chaotic time series, Physical Review A, 4.6:3496-3502, Sep 1991 doi:10.1103/PhysRevA.44.3496
[24] Grassberer, Peter, Thomas Schreiber, Carsten Schaffrath. (1991). Non-linear time series analysis, Int. J. on Bifurcations and Chaos, 1:521, 1991 doi:10.1142/S0218127491000403
[25] Hartman, Eric, James D. Keeler. (1991). Predicting the future: Advantages of semilocal units, Neural Computation, .3, 1991.
[26] Henon, M. (1976). A two dimensional mapping with a strange attractor, Comm. Math. Ph.ys., 50:69-77, 1976.
[27] Hunter, Jr., Norman F. (1992). Nonlinear prediction of speach signals, In Martin Casdagli and Stephen Eubanks. editors, Nonlinear Modelling and Forecasting, pages 467-492. Addison- Wesley, 1992.
[28] Kennel, M.B. S. Isabelle. (1992). Method to distinguish possible chaos from colored noise and to determine embedding parameters, Physical Review A. 4.6:3111-3118, Sep 1992 doi:10.1103/PhysRevA.46.3111
[29] Kirkpatrick, S., CD. Gelatt Jr., M.P. Vecchi. (1983). Optimization by simulated annealing, Science, 22.41598:671-680, May 1983 doi:10.1126/science.220.4598.671
[30] Kugiumtzis, D., B. Lillekjendlie, N. Christophersen. (1994). Chaotic time series, part I - estimating, Modelling, Identification and Control, 1994, Vol 15, No. 4, pp. 205-224 doi:10.4173/mic.1994.4.1
[31] Lapedes, Alan Robert Farber. (1987). How neural nets work, In D.Z. Anderson, editor, Neural Information Processing Systems, pages 442-456. American Institute of Physics, New York. 1987.
[32] Lee, Sukhan Rhee M. Kil. (1988). Multilayer feedforward potential function network, In IEEE International Conference on Neural Networks, pages 1-161 - I-171, San Diego, 1988.
[33] Lee, Sukhan Rhee M. Kil. (1991). A Gaussian potential function network with hierarchically self-organizing learning, Neural Networks, 4:207-224, 1991 doi:10.1016/0893-6080(91)90005-P
[34] Lichtenberg, A.J. M.A. Lieberman. (1992). Regular and chaotic dynamics, Springer-Verlag, 2nd edition. 1992.
[35] Unsay, Paul S. (1991). An efficient method of forecasting chaotic time series using linear interpolation, Physics Letters A. 15.6,7:353-356, Mar 1991.
[36] Ljung, Lennart. (1991). Issues in system identification, IEEE Control Systems, pages 25-29, Jan 1991.
[37] Lorenz, E.N. (1963). Deterministic non-periodic flows, Jon. of Atmospheric Science, 20:130-141, 1963.
[38] Lorenz, E.N. (1969). Atmospheric predictability as revealed by naturally occuring analogies, Journ. of Atmospheric Science, 26:636, 1969.
[39] Lowe, D., A.R. Webb. (1991). Time series prediction by adaptive networks: a - dynamical system perspective, IEE Proceedings-F, 13.1:17-24, Feb 1991.
[40] Mackey, M. L. Glass. (1977). Oscillation and chaos in physological control systems, Science, 19.287, 1977.
[41] MacQueen, J. (1967). Some methods for classification and analyses of multivariate observations, In L.M. LeCam and J. Neyman, editors, Proc. of the 5th Berkeley Symp. on Mathematics, Statistics and Probability, 1967.
[42] Mead, W.C., R.D. Jones, Y.C. Lee, C.W. Barnes, G.W. Flake, L.A. Lee, M.K. O´Rourke. (1991). Using cnls-net to predict the mackey-glass chaotic time series, In Proc. IEEE Int. Joint Conf. on Neural Networks.LICNN, pages 11485-11490, New York. 1991. IEEE.
[43] Mead, W.C., R.D. Jones, Y.C. Lee, C.W. Barnes, G.W. Flake, L.A. Lee, M.K. O´Rourke. (1992). Prediction of chaotic time series using cnls-net, example: The mackey-glass equation, In M. Casdagli and S. Eubank, editors, Nonlinear Modelling and Forecasting, pages 3-24. Addison-Wesley, 1992.
[44] Mees, A.I. (1992). Tesselation and clynamical systems, In M. Casclagli and S. Eubank. editors, Nonlinear Modelling and Forecasting, pages 3-24. Addison-Wesley, 1992.
[45] Michelli, C.A. (1986). Interpoaltion of scattered data: distance matrixes and conditionally positive definite functions, Contructive Approximation, 2:11, 1986 doi:10.1007/BF01893414
[46] Moody, John. (1989). Fast learning in multi-resolution hierarchies, In D.S. Touretzky, editor, Advances in Neural information Processing Systems 1. Morgan Kauffman, 1989.
[47] Moody, John, Cristian J. Darken. (1988). Learning with localized receptive fields, In Touretzky et.al., editor, Proceedings of the 1988 Connectionist Models Summer School, pages 133-143. Morgan-Kaufman, 1988.
[48] Moody, John Cristian J. Darken. (1989). Fast learning in networks of locally-tuned processing units, Neural Computation, 1:281 - 294, 1989 doi:10.1162/neco.1989.1.2.281
[49] Park, Jooyoung Irwin W. Sandberg. (1993). Approximation and radial-basis-function networks, Neural Computation, .2:305-316, Mar 1993 doi:10.1162/neco.1993.5.2.305
[50] Pawelzik, K. H.G. Schuster. (1991). Unstable periodic orbits and prediction, Physical Rev. A, 4.4:1808-1812, 1991 doi:10.1103/PhysRevA.43.1808
[51] Platt, John. (1991). A resource-allocating network for function interpolation, Neural Computation, .2:213-225, 1991 doi:10.1162/neco.1991.3.2.213
[52] Powell, M.J.D. (1987). Radial basis functions for mulivariable interpolation: A review, In J. C. Mason and Ni. G. Cox, editors, Algorithms for Approximation. Clarendon Press, London, 1987.
[53] Preparata, F.R., M. I. Shamos. (1985). Computational Geometry: An Introduction, Springer Verlag, 1985.
[54] Press, William H., Brian P. Flannery, Saul A. Teukolsky, William T. Vetterling. (1988). Numerical Recipes in, C. Cambridge University Press, Cambridge, 1988.
[55] Priestley, M.B. (1981). Spectral Analysis and Time Series, Academic Press, Inc., London, 1981.
[56] Rögvaldsson, Thorsteinn S. (1993). Brownian motion updating of multi-layered perceptrons, In Stan Gielen and Bert Kappen, editors, Proc. Int. Conf. on Artificial Neural Networks, pages 527-532, Sep 1993.
[57] Rössler, O.E. (1976). An equation for continuous chaos, Physics Letter, A(57):397, 1976 doi:10.1016/0375-9601(76)90101-8
[58] Rumelhart, David E., James L. McLelland the PDP Research Group. (1986). Parallel Distributed Processing, Vol. 1-2. The MIT Press, 1986.
[59] Sanger, Terence D. (1990). Neural Computation, .
[60] Sanger, Terence D. (1990). Basis-function trees for approximation in high-dimensional spaces, In Proceedings of the 1990 Coanectionist Models Summer School. Morgan-KauflThan.
[61] Sauer, Tim, James A. Yorke, Martin Casdagli. (1991). Embedology, Journal of Statistical Physics, 6.3/4:579-615, 1991 doi:10.1007/BF01053745
[62] Scott, David W. (1992). Multivariate Density Estimation, John Wiley and Sons, Inc., New York.
[63] Söderström, Torsten Petre Stoica. (1989). System Identification, Prentice-Hall, New York.
[64] Stokbro, K., D. K. Umberger. (1992). Forecasting with weighted maps, In M. Casdagli and S. Eubank, editors, Nonlinear Modelling and Forecasting, pages 73-94. Addison-Wesley.
[65] Stokbro, K., D.K. Umberger, J.A. Hertz. (1990). Exploiting neurons with localized receptive fields to learn chaos, Complex. systems, 4:603.
[66] Stone, M. (1977). Cross-validation, a review, Math. Operationforsch. Statist. Set. Statist., 9:127-139.
[67] Sugihara, George Robert M. May. (1990). Nonlinear forecasting as a way of distinguishing chaos from measurement error in time series, Nature, 344:734-741, Apr 1990 doi:10.1038/344734a0
[68] Takens, Floris. (1981). Detecting strange attractors in turbulence, In D.A. Rand and L.S. Yound, editors, Dynamical Systems and Turbulence, pages 366-381. Springer Verlag, Berlin, 1981 doi:10.1007/BFb0091924
[69] Fernando, Manoel Tenorio Wei-Tsih Lee. (1989). Self organizing neural networks for the identification problem, In D. S. Touretzky, editor, Avances in Neural Information Processing Systems I. Morgan Kaufman Publishers, 1989.
[70] Tong, H. (1990). Nonlinear Time Series: A Dynamical System Approach, Oxford University Press.
[71] Townshend, Brent. (1992). Nonlinear prediction of speach signals, In Martin Casdagli and Stephen Eubanks, editors, Nonlinear Modelling and Forecasting, pages 433-453. Addison-Wesley.
[72] Weigend, A., B. Huberman, D.E. Rumelhart. (1990). Prediciting the future: a connectionist approach, Int. J. Neural Systems, 1:193-209, 1990 doi:10.1142/S0129065790000102
[73] Weigend, A., B. Huberman. D.E. Rumelhart. (1992). Prediciting sunspots and excange rates with connectionist networks, In M. Casdagli and S. Eubank, editors, Nonlinear Modelling and Forecasting, pages 395-432. Addison-Wesley, 1992.
[74] Welstead, S.T. (1991). Multilayer feedforward networks can learn strange attractors, In Proc. Int. Joint Conference of Neural Networks, pages II139-II144, New York, Jul 1991. IEEE.
[75] West, B.J. H.J. Mackey. (1992). Forecasting chaos: A review, Journal of Scientific and Industrial Reseacrh, 51:634-643, 1992.
[76] Wolpert, D.M. R.C. Miall. (1990). Detecting chaos with neural network, Proc. R. Soc. London, .242:82-86 doi:10.1098/rspb.1990.0107

BibTeX:
@article{MIC-1994-4-2,
  title={{Chaotic time series. Part II. System Identification and Prediction}},
  author={Lillekjendlie, Bjørn and Kugiumtzis, Dimitris and Christophersen, Nils},
  journal={Modeling, Identification and Control},
  volume={15},
  number={4},
  pages={225--245},
  year={1994},
  doi={10.4173/mic.1994.4.2},
  publisher={Norwegian Society of Automatic Control}
};

Article

“Chaotic time series. Part II. System Identification and Prediction”