Tuning complexity in regularized kernel-based regression and linear system identification: The robustness of the marginal likelihood estimator

Kernel-based regularization approaches have been successfully applied in the last years for regression purposes. Recently, these machine learning techniques have been also introduced in linear system identification, by interpreting impulse response estimation as a function learning problem. The adopted estimator solves a regularized least squares problem which admits also a Bayesian interpretation where the impulse response is modeled as a zero-mean Gaussian vector. A possible choice for the covariance is the so called stable spline kernel. It includes information on smoothness and exponential stability, containing just two unknown parameters which can be tuned via marginal likelihood (ML) optimization. Experimental evidence has shown that this new approach may outperform traditional system identification approaches, such as PEM and subspace techniques. The aim of this work is to provide new insights on the stable spline estimator equipped with ML estimation of hyperparameters. To this...