Model-based approach

This ``hard to describe" but ``easy to verify" property of unusual events suggests an intuitive two-step solution for their detection. In the first step, one extracts image features from the video, typically achieved by detecting and tracking moving objects [14]. From tracked objects trajectory, speed, and possibly the shape descriptor of the moving objects can be computed [6]. In the second step the extracted features are used to develop models for the ``normal'' activities, either by hand or by applying supervised machine learning techniques [7]. A common choice is to use Hidden Markov Models [1,9,10] or other graphical models [11] which quantize image features into a set of discrete states and model how states change in time. In order to detect unusual events the video is matched against a set of normal models and segments which do not fit the models is considered unusual.

Figure 1: Snapshots from the videos used for experiments.

(a) Nursing home	(b) Road	(c) Poker game

This model-based approach can be quite effective in situations where ``normal" activity is well-defined and constrained. However in a typical real-life video, like those used in our experiments, the number of different ``normal'' activity types observed can easily surpass the number of unusual types. Hence, defining and modeling what is the ``normal'' activity in an unconstrained environment can be more difficult than defining what is unusual. If the goal is to detect what unusual events in a long video, the model-based approach is often over-kill.