Automated model maintenance enabled by unsupervised model diagnostic measures

• Verena Haunschmid
• Ramin Nikzad-Langerodi
• Edwin Lughofer
• Thomas Natschläger

A crucial part of multivariate calibration methods is the ongoing determination of the predictive ability of calibrated models. Models can become unreliable over time due to variations of the analyzed medium or changes to the measurement hardware [3]. It is desired to achieve an efficient and automated approach for model maintenance. Therefor it is necessary to detect automatically and timely that a calibration model is unreliable. This has been a problem of interest for a long time and lead to the development of a wide range of such measures [3]. Unfortunately, model diagnostic measures like Q measures and Hotellings T2 have not yet shown useful results on our real life NIR spectroscopic data provided by an industrial partner. Other approaches of interest are outlier detection methods like the Monte Carlo outlier map presented in [1]. The method identifies outliers by studying the distribution of prediction errors when training multiple models using Monte Carlo cross validation using Cook’s Distance. It yielded useful results in our experiments and can be used for detecting outliers during the calibration phase but not afterwards when it would be required, since reference values are needed.