Smart test generation and efficient test execution
Aims and Research Topics
Software development is characterized by the constant evolution of large, complex software systems, often in the form of highly customizable and highly configurable products or product lines. The approach to ensure the quality of such systems is the application of static and dynamic test methods. With these approaches new functionality can be explored and validated and undesirable side effects on the existing implementation can be detected. As complexity increases, so does the scope and complexity of the necessary test infrastructure and test cases. The application of generative test approaches and the investigation of the automatic reusability of test cases for different system configurations should enable a more effective use of available test resources.
Goals and Innovation
Based on the use cases and challenges of the project partners in dealing with the complexity and variety of their systems, specific measures are identified and solutions in the following areas are researched and implemented.
Static and dynamic analysis for test case generation
Testing requires deeper domain knowledge. By integrating static and dynamic analyses of software systems (e.g. code analysis, symbolic execution, architecture documentation, user profiling) relevant knowledge can be extracted and used in testing or for test case generation.
Regression testing should ideally run all tests with all possible configurations. The enormous number of possible configurations and variants that result from the combination of different configuration settings and product properties usually make complete testing impossible. Therefore an intelligent test strategy and test case selection is required.
At the level of system testing, a high level of coverage is often required in terms of tested requirements (requirements-based testing) or model elements (model-based testing). System configurations and product variants introduce an additional factor that must be taken into account when calculating coverage ratios and result variables. However, it is still not possible to say with certainty how good the tests are and how well the system is tested. This requires additional methods and metrics.
In order to efficiently design tests in the environment of increasingly complex software systems and the necessary variety of variants, new methods from research in the industrial environment must be applied and answers must be found to the following questions:
- How can test cases and the test infrastructure be reused?
- How can data collection from operation and automatic test executions support testing?
- If and how can AI approaches be used for testing in an industrial environment?
In cooperation of all project partners the described approaches will be developed and validated in an industrial environment:
- SCCH - Software Competence Center
- ENGEL AUSTRIA GmbH
- Palfinger EMEA GmbH
- PKE Holding AG
- Windhager Central Heating Technology GmbH
- JKU, Institute for Software Systems Engineering
48 months (January 1, 2019 - December 31, 2022)
This project is subsidized in the frame of COMET – Competence Centers for Excellent Technologies by BMVIT, BMK, State of Upper Austria and its scientific partners. The COMET program is handled by FFG.