Improving offline deep learning models by censored online data and quadratic programming

• Sabrina Luftensteiner

Digitalization and Industy 4.0 are causing a continuous increase regarding the usage of sensors and its produced data, especially in industrial environments. The generated data is processed using machine learning algorithms to create models for predictions, etc. Since traditional offline learning is often insufficient, due to time-consuming and computational reasons, the method of online learning is increasingly being used. Online learning enables the adaptation of the model without having to train a new model.

This thesis covers the continued and adaptive training of existing offline models used for regression problems in an online learning setting, whereat the main focus lies on the integration of solutions for catastrophic forgetting and censored data. Catastrophic forgetting of already gained knowledge occurs in online learning settings and is provoked by the learning of new knowledge. Different approaches regarding the solution are presented and cover the integration of a representative memory as well as different approaches regarding the loss computation and optimizer usage. Censored data are often recorded during the production processes due to, e.g., physical limits of sensors and affect the model negatively during training as it is trained with wrong data. As a solution for this problem, two approaches regarding quadratic and one for linear programming problems are explained.

As a result of this thesis, a configurable framework is presented including all of the mentioned methods. The experiment section demonstrates the usage of this framework and presents and discusses under which circumstances the presented methods improve existing models in an industrial setting by means of three regression problems.