Learning Activities from Video
In this project we have devoloped a method for learning about human activities from video.
The idea is to look for repeating patterns of changing spatial relationships between tracked entities
in the scene. Having discovered these patterns or events in this way, we look at the roles objects play in these events
and thereby produce a taxonomy of objects.
We demonstrate our framework on a kitchen set up, where the scene consists of hands simulating the preparation of
various dishes, lasting around 10 minutes. We obtain a event taxonomy and a object taxonomy as shown.
Data Set Sample
Download Video 
- Muralikrishna Sridhar, Anthony Cohn, Hogg David C, Learning Functional Object-Categories from a Relational
Spatio-Temporal Representation ,ECAI 08. Download
- Activities are represented in a activity graph that captures spatio-temporal interactions between objects.
- Events are learned by mining frequent subgraphs from the activity graph. An Event taxonomy capturing "part of" relationships is induced by relating events.
- An object taxonomy is learned by hierarchical clustering of objects, which are represented in terms of the role they play in the learned events.
|