Video-prototype co-occurrence

A video segment $v_j$ can be represented by the occurrence of prototype features in that video segment. We define a co-occurrence matrix $C \in R^{K \times N}$ between the video segments and prototype features in the following way: $C(i,j)=1$ iff segment $v_j$ contains and image frame whose image feature is classified as prototype $p_i$. Figure 4 shows an example for the roadway sequence. $C$ contains all necessary information about what happened in the video. Each row $i$ of $C$ reveals which video segment a prototype $p_i$ occurred in, and each column $j$ of $C$ indicates which prototype features occurred in the video segment $v_j$.

Figure 4: (left) Video-prototype co-occurrence matrix C contains all necessary video information. (right) Reorganization of rows and columns of C reveals the underlying correspondence between thevideo segments and prototype features, making it easy tofind both unusual events and important features.