By: Ryan Wendel
It is an ongoing analysis in which videos are
analyzed frame by frame
Most of the video recognition is pulled from
3-D graphic engines
“HAA” stands for Human Activity Analysis
Surveillance systems
Patient monitoring systems
Human-computer interfaces
We are going to take a look at methodologies
that have been developed for simple human
actions.
And high-level activities.
Gestures
Actions
Interactions
Group activities
Basic movements of a persons body parts.
For example:
Raising an arm
Lifting a leg
A Single persons activities which could entail
multiple gestures.
For example:
Walking
Waving
Shaking body
Interactions that involve two or more
people / items.
For Example:
Two people fighting
Activities performed by multiple people.
For example:
A group running
A group walking
A group fighting
Can be separated into two sections
◦ Single-layered approaches: An approach that deals
with recognizing human activities based on a video
feed (frame by frame.)
◦ Hierarchical approaches: An approach aimed at
describing the high level approach to HAA by
showing high level activities in simpler terms.
Main objective is to analyze simple sequences
of movements of humans
Can be categorized into two different
categories
◦ Space-time approach: takes an input video as a 3D volume
◦ Sequential approach: takes an input video and
interprets it as a sequence of observations
Divided into three different subsections
based on features
◦ Space-time volume
◦ Space-time Trajectories
◦ Space-time features
Captures a group of human activities by
analyzing volumes of a video (frame by
frame.)
Also uses types of recognition using spacetime volumes to measure similarities between
two volumes
Uses stick figure modeling to extract joint
positions of a person at each frame by frame
Does not extract features frame by frame
Extracts features when there is a appearance
or shape change in 3-D Space-time volume
Space-Time Volume
◦ Hard to differentiate between multiple people in the
same scene.
Space-Time Trajectories
◦ 3-D body-part detection and tracking is still an
unsolved problem, and it requires a strong lowlevel component that can estimate 3-D join
location.
Space-Time features
◦ Not suitable for modeling complex activities
Divided into two different subsections based
on features
◦ Exemplar-based
◦ State model-based
Review
◦ Sequential approach: takes an input video and
interprets it as a sequence of observations
Exemplar-based
◦ Shows human activities with a set of sample
sequences of action executions
Sequential set of sequences that represent a
human activity as a model composed of a set
of states.
Exemplar-based is more flexible in terms of
comparing multiple sample sequences
Where as State Model-based can handle a
probabilistic analysis of an activity better.
Sequential approach is able to handle and
detect more complex activities performed
Whereas the Space-time approach handles
simpler less complex activities.
Both methods are based off of some type of a
sequences of images
Allows the recognition of high-level activities
based on the recognition results of other
simpler activities
Advantages of the Hierarchical Approach
◦ Has the ability to recognize high-level activities
with a more in depth structure
◦ Amount of data required to recognize an activity is
significantly less then single-layered approach
◦ Easier to incorporate human knowledge
Statistical approach
Syntactic approach
Description-based approach
Statistical approaches use the state-based
models to recognize activities
If you use multiple layers of a state-based
model you can use these separate models to
recognize activities with sequential structures
Human activities are recognized as a string of
symbols
Human activities are shown as a set of
production rules generating a string of
actions
Human activities that use recognition with
complex spatio-temporal structures
◦ A spatio-temporal structure is a detector used for
recognizing human actions
Uses Context-free grammars (CFGs) to
represent activities
◦ CFGs are used to recognize high-level activities
◦ The detection extracts space-time points and local
periodic motions to obtain a sparse distribution of
interest points in a video
Probability theory
Fuzzy logic
Bayesian network:
◦ Used for recognition of an activity, based on the
activities temporal structure representation
◦ Uses a large network with over 10,000 nodes
A group of persons marching
◦ The images are recognized as an overall motion of
an entire group
A group of people fighting
◦ Multiple videos are used to recognize the activity
that a “group is fighting”
Recognition of interactions between humans
and objects requires multiple components
involved.
A lot of human-object interaction ignores
interaction between object recognition and
motion estimation
You can also factor in object dependencies,
motions, and human activities to determine
activities involved
J.K. Aggarwal and M.S. Ryoo. 2011. Human activity analysis: A review.
ACM Comput. Surv. 43, 3, Article 16 (April 2011), 43 pages.
DOI=10.1145/1922649.1922653
http://doi.acm.org/10.1145/1922649.1922653
Christopher O. Jaynes. 1996. Computer vision and artificial intelligence.
Crossroads 3, 1 (September 1996), 7-10. DOI=10.1145/332148.332152
http://doi.acm.org/10.1145/332148.332152
Zhu Li, Yun Fu, Thomas Huang, and Shuicheng Yan. 2008. Real-time
human action recognition by luminance field trajectory analysis. In
Proceedings of the 16th ACM international conference on Multimedia
(MM '08). ACM, New York, NY, USA, 671-676.
DOI=10.1145/1459359.1459456
http://doi.acm.org/10.1145/1459359.1459456
Paul Scovanner, Saad Ali, and Mubarak Shah. 2007. A 3-dimensional sift
descriptor and its application to action recognition. In Proceedings of the
15th international conference on Multimedia (MULTIMEDIA '07). ACM,
New York, NY, USA, 357-360. DOI=10.1145/1291233.1291311
http://doi.acm.org/10.1145/1291233.1291311
0
