Deep learning based trajectory prediction and anomaly detection in crowds

L. Zhao. Deep learning based trajectory prediction and anomaly detection in crowds. 6, 2018.

  • Lingcong Zhao

Recently, with the growth of crowd phenomena, Crowd analysis in video has been a nucleus of research interest over the last decade. An increasing number of works have been reported in this field, covering a wide range of aspects including crowd counting, trajectory prediction, crowd motion understanding, anomaly detection in crowds and so on.

Traditionally, a lot of works are based on handcrafted features such as optic flow. However, they are facing numerous challenges such as the complexity of human motion, dynamic range of environment, occlusion, clutter and so on. As many other computer vision areas, deep learning-based approaches have achieved the state-of-art performance in crowd analysis field. In this work, we explored 2 crucial branches in this field - trajectory prediction and anomaly detection in crowds, with focus on the state-of-art deep learning approaches.

Firstly, we concluded the main challenges and constraints facing in crowd trajectory prediction and anomaly detection field, and then conclusively demonstrated how classical approaches in literature dealt with them. Secondly, for trajectory prediction part, we improved the state-of-the-art approach - Social LSTM, and compared its results with another newer approach - Social attention. Moreover, we extended the application of the two state-of-arts approaches to group sport (football match), aiming at rooting out the limitations of them and more importantly, discovering crucial changes needed to switch from normal crowd analysis to strategic crowd analysis. With regards to anomaly detection in crowds, we proposed 3 new approaches based on deep learning models and validated their effectiveness on a diversity of crowd scenes including different abnormal behaviors, various environment, distinct crowd density. One of our approaches achieved state-of-art performance on CAVIAR anomaly dataset. Finally, future work and trends are discussed to inspire future research.