My research focus is joint space-time segmentation and analysis of video sequences. Traditional video processing methods use two image frames at a time to analyze such
dynamics as motion, occlusions, etc. We explore new framework that is based on joint treatment of many image frames (e.g., 20-30). A form of joint space-time processing, this framework is essentially
three-dimensional (3-D) since its domain is the x-y-t space of image sequences. It is expected to result in more reliable video segmentation, detection of occlusion effects and identification of various dynamic
events. To date, we have developed a video segmentation method within this framework that is based on an active-surface model and level-set solution. Our formulation is a form of volume competition. Applied to both synthetic and natural image sequences this method results in “object tunnels” in the x-y-t space.
Analyzing walls of these object tunnels, we were able to identify certain occlusion events and measure time instants of object occlusions, disappearance, entry, etc. However, without explicit modeling of background occlusion and exposure regions, they get included into object or background tunnel, thus creating errors in segmentation. To solve that problem, we developed multiphase formulation with explicit modeling of background occlusion and exposed volumes and parametric modeling of object and background motion trajectories (which we estimate jointly with segmentation). Finally, we expanded our model to include object occlusion and exposed volume detection.