Live replay of screen videos: Automatically executing real applications as shown in recordings

Autoren Rudolf Ramler
Marko Gattringer
Josef Pichler
Editoren
Titel Live replay of screen videos: Automatically executing real applications as shown in recordings
Buchtitel Proceedings of the 2020 IEEE 27th International Conference on Software Analysis, Evolution, and Reengineering (SANER'20)
Typ in Konferenzband
Verlag IEEE
ISBN 978-1-7281-5143-4
Monat February
Jahr 2020
Seiten 664-665
SCCH ID# 20001
Abstract

Screencasts and videos with screen recordings are becoming an increasingly popular source of information for users to understand and learn about software applications. However, searching for answers to specific questions in screen videos is notoriously difficult due to the effort for locating specific events of interest and reproducing the application's state up to this event. To increase the efficiency when working with screen videos, we propose a solution for replaying recorded sequences shown in videos directly on live applications. In this paper, we describe the analysis of screen videos to automatically identify and extract user interactions and the construction of visual scripts, which are used to run the application in sync with replaying the video. Currently, a first prototype has been developed to demonstrate the technical feasibility of the approach. The paper provides an overview of the implemented solution concept and discusses technical challenges, open issues, as well as future application scenarios.