Live replay of screen videos: Automatically executing real applications as shown in recordings

R. Ramler, M. Gattringer, J. Pichler. Live replay of screen videos: Automatically executing real applications as shown in recordings. pages 664-665, DOI https://doi.org/10.1109/SANER48275.2020.9054833, 2, 2020.

Autoren
  • Rudolf Ramler
  • Marko Gattringer
  • Josef Pichler
BuchProceedings of the 2020 IEEE 27th International Conference on Software Analysis, Evolution, and Reengineering (SANER'20)
TypIn Konferenzband
VerlagIEEE
DOIhttps://doi.org/10.1109/SANER48275.2020.9054833
ISBN978-1-7281-5143-4
Monat2
Jahr2020
Seiten664-665
Abstract

Screencasts and videos with screen recordings are becoming an increasingly popular source of information for users to understand and learn about software applications. However, searching for answers to specific questions in screen videos is notoriously difficult due to the effort for locating specific events of interest and reproducing the application's state up to this event. To increase the efficiency when working with screen videos, we propose a solution for replaying recorded sequences shown in videos directly on live applications. In this paper, we describe the analysis of screen videos to automatically identify and extract user interactions and the construction of visual scripts, which are used to run the application in sync with replaying the video. Currently, a first prototype has been developed to demonstrate the technical feasibility of the approach. The paper provides an overview of the implemented solution concept and discusses technical challenges, open issues, as well as future application scenarios.