Clustering Textures with EHG Algorithm for Modelling Video

IJCSEC Front Page

In this paper we present a novel approach for common recognition of group activities for video surveillance applications. We propose a Energetic-based approach for detecting abnormal events in surveillance video. It requires the appropriate definition of similarity between events. Human pose estimation via motion tracking systems can be considered as a regression problem within a discriminative framework. We defined the overfitting problem was handled by Hidden Markov Model based similarity. We propose in this paper a multi model-based similarity measure. In this measure, the Hidden Markov Model training and distance measuring are based on multiple samples. The novel Energetic Hierarchical Group (EHG) method acquired the multiple training data. By iteratively reclassifying and retraining the data groups at different clustering levels, the initial training and clustering errors due to overfitting will be sequentially corrected in later steps. Experimental results on real surveillance video show an improvement of the proposed method over a stand column method that uses single sample- based similarity measure and spectral clustering.
Keywords:pose estimation, group event detection, clustering, group representative, surveillance, motion tracking systems.
Identifying human behavior or human interactions has attracted increasing the research interests [1-6]. The following events are group events.

  • people fighting
  • people walking together
  • people being followed
  • group conversations in a party
  • terrorist launching attacks in groups
In this paper we propose a multi model-based similarity measure to hold back the overfitting problem, where Hidden Markov Model representation is based on several similar samples. The acquisition of these several training data is by hierarchically collect and iteratively retraining the whole dataset, which is summarized as Energetic Hierarchical Group (EHG) algorithm. This algorithm can animatedly correct initial overfitting errors as the numbers of training samples increase (i.e. data clusters become bigger )
In addition, it is not sensitive to the absolute values of similarity, because simple comparison operation instead of eigenvalue decomposition is needed in the proposed approach.
In real videos, the suspicious events are rare, difficult to describe, hard to predict and can be subtle. However, based on the assumption that an abnormal event is associated with the distinctness of the activity. (e.g., a running person where everybody walks is interpreted as abnormal as well as a walking person where the rest run) and a normal event indicates the commonality. (e.g., a path that most people walk on)In this paper, we address the following issues for cluster incident discovery.
1.1 Cluster incident discovery with supple or unreliable number of group members
Most previous cluster event detection researches [1-2] use a Hidden Markov Model or its variation to model the human interactions. Some people try to recognize human interactions based on a content-independent semantic set [3-4]. However, most of these works are designed to recoggnize group activities with a fixed number of group members, where the input feature vector length is fixed.
They cannot handle cases where the number of group members is supple or even unreliable, which is often the case in our daily life (e.g., people may leave or join a group activity). In this case, the input feature vector length may vary with different number of group members.


  1. [1] Iveel Jargalsaikhan, Cem Direkoglu, Noel E. O’Connor1, Alan F. Smeaton, Kathy Clawson, Hao Li,Marcos Nieto, Aitor Rodriguez, Pedro Sanchez, Karina Villarroel Paniza, AnaMart´ınez Llorens ,”Interactive Surveillance Event Detection” , SAVASA Project@ TRECVID 2012.
  2. [2] Weiyao Lin, Ming-Ting Sun, Radha Poovendran and Zhengyou Zhang, “Group Event Detection with a Varying Number of Group Members for Video Surveillance” IEEE Trans. Circuits System and Video Technology”, VOL. 20, NO. 8, AUGUST 2010
  3. [3] Wanqing Li, Member, Zhengyou Zhang, Fellow and Zicheng Liu, Senior Member, IEEE,” Expandable Data-Driven Graphical Modeling of Human Actions Based on Salient Postures”, IEEE Trans. Circuits System and Video Technology, VOL. 18, NO. 11, NOVEMBER 2008.
  4. [4] Weiyao Lin, Ming-Ting Sun, Fellow, IEEE, Radha Poovendran, Senior Member, IEEE, and Zhengyou Zhang,” Activity Recognition Using a Combination of Category Components and Local Models for Video Surveillance”, IEEE Trans. Circuits System and Video Technology, VOL. 18, NO. 8, AUGUST 2008.
  5. [5] Jack M. Wang, David J. Fleet and Aaron Hertzmann, Member, IEEE,” Gaussian Process Dynamical Models for Human Motion”, IEEE Trans. Pattern analysis and machine intelligence, VOL. 30, NO. 2, FEBRUARY 2008.
  6. [6] Adeel Mumtaz, Emanuele Coviello, Gert. R. G. Lanckriet, Antoni B. Chan,” Clustering Dynamic Textures with the Hierarchical EM Algorithm for Modeling Video”, JOURNAL OF LATEX CLASS FILES, VOL. 6, NO. 1, JANUARY 2007.
  7. [7] Christ, R.E.R.,” Gaussian Hierarchical Bayesian Clustering Algorithm”, Seventh International Conference on Intelligent Systems Design and Applications, ISDA 2007.
  8. [8] Zicheng Liu, Senior Member, IEEE, Michael Cohen, Senior Member, IEEE, Deepti Bhatnagar, Ross Cutler, and Zhengyou Zhang,”Head-Size Equalization for Improved Visual Perception in Video Conferencing” , IEEE Trans. Multimedia, VOL. 9, NO. 7, NOVEMBER 2007.
  9. [9] Dong Zhang, Student Member, Daniel Gatica-Perez, Samy Bengio, and Iain McCowan “Modeling Individual and Group Actions in Meetings with Layered HMMs”, IEEE Trans. Multimedia, VOL. 8, NO. 3, JUNE 2006.
  10. [10] R. Urtasun, S. Epfl, D. J. Fleet, and P. Fua, “3D people tracking with Gaussian process dynamical models,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognition., 2006, vol. 1, pp. 238–245.
  11. [11] Dong Zhang, Student Member, Daniel Gatica-Perez, Samy Bengio, and Iain McCowan “Modeling Individual and Group Actions in Meetings with Layered HMMs”, IEEE Trans. Multimedia, VOL. 8, NO. 3, JUNE 2006.
  12. [12] Ying-li Tian, Arun Hampapur, Lisa Brown, Rogerio Feris, Max Lu, Andrew Senior, Chiao-fe Shu, and Yun Zhai,” Event detection, query, and retrieval for video surveillance”, Book chapter for Artificial Intelligence for Maximizing Content Based Image Retrieval.
  13. [13] Chris Whiten, Robert Lagani`ere, Ehsan Fazl-Ersi, Feng Shi, “Interactive Surveillance Event Detection”, VIVA Research Lab University of Ottawa, Ottawa, Canada.