A lightweight approach for estimating probability in risk-based software testing

Authors Rudolf Ramler
Michael Felderer
Matthias Leitner
Editors Jürgen Großmann
Michael Felderer
Fredrik Seehusen
Title A lightweight approach for estimating probability in risk-based software testing
Booktitle Risk Assessment and Risk-Driven Quality Assurance - RISK 2016, Revised Selected Papers
Type in book
Publisher Springer
Series Lecture Notes in Computer Science
Volume 10224
ISBN 978-3-319-57857-6
DOI 10.1007/978-3-319-57858-3_9
Month April
Year 2017
Pages 115-128
SCCH ID# 17027

Using risk information in testing is requested in many testing strategies and recommended by international standards. The resulting, widespread awareness creates an increasing demand for concrete implementation guidelines and for methodological support on risk-based testing. In practice, however, many companies still perform risk-based testing in an informal way, based only on expert opinion or intuition. In this paper we address the task of quantifying risks by proposing a lightweight approach for estimating risk probabilities. The approach follows the “yesterday’s weather” principle used for planning in Extreme Programming. Probability estimates are based on the number of defects in the previous version. This simple heuristic can easily be implemented as part of risk-based testing without specific prerequisites. It suits the need of small and medium enterprises as well as agile environments which have neither time nor resources for establishing elaborated approaches and procedures for data collection and analysis. To investigate the feasibility of the approach we used historical defect data from a popular open-source application. Our estimates for three consecutive versions achieved an accuracy of 73% to 78% and showed a low number of critical overestimates (<4%) and few underestimates (<1%). For practical risk-based testing such estimates provide a reliable quantitative basis that can be easily augmented with the expert knowledge of human decision-makers. Furthermore, these results also define a baseline for future research on improving probability estimation approaches.