Constrained and regularized system identification

“Constrained and regularized system identification”

Authors: Tor A. Johansen,
Affiliation: NTNU, Department of Engineering Cybernetics
Reference: 1998, Vol 19, No 2, pp. 109-116.

Keywords: Regularization, Optimization, Parameter Estimation, Nonlinear Systems

Abstract: Prior knowledge can be introduced into system identification problems in terms of constraints on the parameter space, or regularizing penalty functions in a prediction error criterion. The contribution of this work is mainly an extension of the well known FPE (Final Production Error) statistic to the case when the system identification problem is constrained and contains a regularization penalty. The FPECR statistic (Final Production Error with Constraints and Regularization) is of potential interest as a criterion for selection of both regularization parameters and structural parameters such as order.

PDF

PDF (1243 Kb) DOI: 10.4173/mic.1998.2.4

DOI forward links to this article:

[1] Ronald K. Pearson (2006), doi:10.1016/j.compchemeng.2006.05.028

[2] T.A. Johansen, K.J. Hunt and H. Fritz (1998), doi:10.1109/37.664655

[3] Jie Chen, Cédric Richard and José Carlos M. Bermudez (2016), doi:10.1016/j.sigpro.2016.03.017

[4] Wenyuan Wang and Haiquan Zhao (2017), doi:10.1016/j.sigpro.2017.10.006

[5] Ling Zhang, Youqing Wang and Xifa Sun (2018), doi:10.1109/ICSP.2018.8652465

[6] M. Khosravi and R. S. Smith (2023), doi:10.1109/TAC.2023.3243099

References:
[1] AKAIKE, H. (1969). Fitting autoregressive models for prediction, Ann. Inst. Stat. Math. 21, 243-247 doi:10.1007/BF02532251
[2] AKAIKE, H. (1974). A new look at the statistical model identification, IEEE Trans. Automatic Control 19, 716-723 doi:10.1109/TAC.1974.1100705
[3] BAI, E.W. SASTRY, S.S. (1986). Parameter identification using prior information, Int. J. Control 44, 455-473 doi:10.1080/00207178608933612
[4] CARLSTEIN, E. (1992). Resampling techniques for stationary time-series: Some recent developments, In: New Direction in Time Series Analysis, Part I.D. Brillinger et.al, Ed., pp. 75-85. Springer-Verlag, New York, NY.
[5] CRAVEN, P. WAHBA, G. (1979). Smoothing noisy data with spline functions, Estimating the correct degree of smoothing by the method of generalized cross-validation. Numerical Math. 31,317-403.
[6] DAYAL, B.S. MACGREGOR, J.F. (1996). Identification of finite impulse response models: Methods and robustness issues, Industrial and Engineering Chemistry Research 35, 4078-4090 doi:10.1021/ie960180e
[7] DE MOOR, B., GEVERS, M. GOODWIN, G.C. (1994). l2-overbiased, l2-underbiased and l2-unbiased estimation of transfer functions, Automatica 30,893-898 doi:10.1016/0005-1098(94)90179-1
[8] ESKINAT, E. (1995). System identification using constrained estimation, In: Proc. European Control Conference, Rome, pp. 856-861.
[9] ESKINAT, E., JOHNSON, S. LUYBEN, W. (1993). Use of auxiliary information in system identification, Ind. Eng. Chem. Research 32,1981-1992 doi:10.1021/ie00021a021
[10] FOSS, B.A. JOHANSEN, T.A. (1997). Identification and convexity in optimizing control, Preprints IFAC Symposium on System Identification, Kitakyushu. Japan. pp. 691-696.
[11] GAWTHROP, P.J., JONES, R.W. MACKENZIE, S.A. (1992). Identification of partially-known systems, Automatica 28, 831-836 doi:10.1016/0005-1098(92)90046-I
[12] JOHANSEN, T.A. (1996). Identification of non-linear systems using empirical data and prior knowledge - An optimization approach, Automatica 32, 337-356 doi:10.1016/0005-1098(95)00146-8
[13] JOHANSEN, T.A. (1996). Robust identification of Takagi-Sugeno-Kang fuzzy models using regularization, In: Proc. IEEE Conf. Fuzzy Systems, New Orleans, pp. 180-186.
[14] JOHANSEN, T. A. (1997). On Tikhonov regularization, bias and variance in nonlinear system identification, Automatica 33, 441-446 doi:10.1016/S0005-1098(96)00168-9
[15] JOHANSEN, T.A. FOSS, B.A. (1997). ORBIT-operating regime based modeling and identification toolkit, Preprints IFAC Symposium on System Identification, Kitakyushu, Japan, pp. 961-968.
[16] KARNY, M., HALOUSKOVA, A. NEDOMA, P. (1995). Recursive approximation by ARX model: A tool for grey-box modelling, Int. J. Adaptive Control and Signal Processing 9,525-546 doi:10.1002/acs.4480090606
[17] KRAMER, M.A., THOMPSON, M.L. PHAGAT, P.M. (1992). Embedding theoretical models in neural networks, In: Proceedings American Control Conference, Chicago, IL., pp. 475-479.
[18] KUNISCH, K. SACHS, E.W. (1992). Reduced SQP methods for parameter identification problems, SIAM J. Numerical Analysis 29, 1793-1820 doi:10.1137/0729100
[19] LARSEN, J. HANSEN, L.K. (1994). Generalization performance of regularized neural network models, In: Proc. IEEE Workshop on Neural Networks for Signal Processing, Ermioni, Greece.
[20] MOONS, C. DE MOOR, B. (1995). Parameter identification of induction motor drives, Automatica 31, 1137-1147 doi:10.1016/0005-1098(95)00016-P
[21] PETERKA, V. (1981). Bayesian system identification, Automatica 17, 41-53 doi:10.1016/0005-1098(81)90083-2
[22] RISSANEN, J. (1978). Modeling by shortest data description, Automatica 14, 465-471 doi:10.1016/0005-1098(78)90005-5
[23] SJÖBERG, J., HJALMARSSON, H. LJUNG, L. (1994). Neural networks in system identification, In: Preprints 10th IFAC Symp. System Identification. Copenhagen. Vol. 2. pp. 49-72.
[24] SJÖBERG, J., MCKELVEY, T. LJUNG, L. (1993). On the use of regularization in system identification, In: Preprints 12th IFAC World Congress, Sydney. Vol. 7, pp. 381-386.
[25] SÖDERSTRÖM, T. STOICA, P. (1988). System Identification, Prentice Hall, Englewood Cliffs, NJ.
[26] STOICA, P., EYKHOFF, P., JANSSEN, P. SÖDERSTRÖM, T. (1986). Model-structure selection by cross-validation, Int. J. Control 43, 1841-1878 doi:10.1080/00207178608933575
[27] STONE, M. (1974). Cross-validatory choice and assessment of statistical predictions, J. Royal Statistical Soc. B 36, 111-133.
[28] TAKAGI, T. SUGENO, M. (1985). Fuzzy identification of systems and its application to modeling and control, IEEE Trans. Systems, Man, and Cybernetics 15, 116-132.
[29] THOMPSON, M.L. KRAMER, M.A. (1994). Modeling chemical processes using prior knowledge and neural networks, AIChE J. 40, 1328-1340 doi:10.1002/aic.690400806
[30] TIKHONOV, A.N. ARSENIN, V.Y. (1977). Solutions of ill-posed Problems, Winston, Washington DC.
[31] TULLEKEN, H.J.A.F. (1993). Grey-box modelling and identification using physical knowledge and Bayesian techniques, Automatica 29, 285-308 doi:10.1016/0005-1098(93)90124-C
[32] XIN, J., OHMORI, H. SANO, A. (1995). Minimum MSE based regularization for system identification in the presence of input and output noise, In: Proc. 34th IEEE Conf. Decision and Control, New Orleans, pp. 1807-1814.

BibTeX:
@article{MIC-1998-2-4,
  title={{Constrained and regularized system identification}},
  author={Johansen, Tor A.},
  journal={Modeling, Identification and Control},
  volume={19},
  number={2},
  pages={109--116},
  year={1998},
  doi={10.4173/mic.1998.2.4},
  publisher={Norwegian Society of Automatic Control}
};

Article

“Constrained and regularized system identification”