Advances in Segmentation for Video Understanding
BY DR. Jason Corso, SUNY, Buffalo, NY
Monday, December 3, 2012 – 2:00 pm
Location: Babbio Center, Room 110, Stevens Institute of Technology
The use of video segmentation as an early processing step in video understanding lags behind the use of image segmentation for image understanding, despite many available video segmentation methods. The reasons for this are likely due to a general lack of critical analysis to help us understand which methods work well in which scenarios, and the simple fact that videos are an order of magnitude bigger than images. In this talk, I will cover recent advances in my group that address both of these reasons. First, I will discuss LIBSVX, a suite of five supervoxel methods coupled with a set of spatiotemporal metrics to evaluate the segmentation methods. Our evaluation summarily arrives at the conclusion that hierarchical segmentation methods, which reevaluate similarity at multiple scales within the hierarchy, perform best overall.
Despite the hierarchical methods performing best, the state of the art hierarchical algorithms have two significant limitations: they require the full video to be loaded into memory, limiting the videos they can process, and the hierarchical output they produce often overwhelms the user with too much data; it is not always clear which level of the hierarchy should be used. The second part of the talk will discuss how we systematically address these two limitations. We have proposed an approximation framework for streaming hierarchical video segmentation motivated by data stream algorithms: each video frame is processed only once and does not change the segmentation of previous frames. We also propose a new criterion for flattening the hierarchy based on the notion of uniform motion entropy; select segments throughout the hierarchy so as to balance the amount of motion entropy within the selected segments. Time permitting, I will present an example of how video understanding can benefit from using the segmentation: for the video label propagation problem, our supervoxel-based propagation method is significantly more capable than the best of the state of the art pixel-based methods.
Corso is an assistant professor in the Computer Science and Engineering Department of SUNY at Buffalo. He received his Ph.D. in Computer Science at The Johns Hopkins University in 2005. He received the M.S.E Degree from The Johns Hopkins University and the B.S. Degree with honors from Loyola College In Maryland, both in Computer Science. From 2005-2007, Corso was a post-doctoral research fellow in neuro-imaging and statistics at the University of California, Los Angeles. He is the recipient of the Army Research Office Young Investigator Award 2010, NSF CAREER award 2009, SUNY Buffalo Young Investigator Award 2011, a member of the 2009 DARPA Computer Science Study Group, and a recipient of the Link Foundation Fellowship in advanced Simulation and Training 2004. He holds the Associate Editor position of Computer Methods and Programs in Biomedicine since 2009. Corso has authored more than sixty peer-reviewed journal and conference papers, three book chapters, and six reports on topics of his research interest including computer vision, medical imaging, robotics, computational biomedicine, machine intelligence, statistical learning, perceptual interfaces and smart environments.