Motion and Semantic Segmentation

    Advised by: Prof. Chieh-Chih (Bob) Wang
    Date: July 2013 - July 2014

    In robotics applications involving data from a moving camera—such as navigation, tracking, and localization—it is inevitable to encounter multiple moving objects. As a result, motion segmentation becomes a critical task to enhance the performance of these operations. Our goal is to partition regions in an RGB-D image that share the same estimated motion in 6 degrees of freedom (6 DoF). Additionally, we incorporated the concept of physical support to improve the accuracy and understanding of the motion segments.

Motion and Semantic Segmentation in Crowded Urban Areas with a Moving RGB-D Camera

We propose a novel and robust motion segmentation algorithm leveraging RGB-D video captured with a structured light camera. Using a modified Random Sample Consensus (RANSAC) algorithm, we estimate motion by exploiting the coherence of color and spatial relationships in the scene. This approach inherits the computational efficiency and probabilistic robustness of the RANSAC paradigm. After aligning two point clouds using the estimated transformation, we utilize the distribution of inliers and outliers as prior knowledge for graph-cut-based segmentation.

To further enhance the understanding of motion segments, we integrate the concept of physical support relationships, allowing us to capture meaningful interactions between motion segments. Additionally, we provide a comprehensive annotated RGB-D dataset captured in a crowded urban environment to validate our method. Comparisons with existing motion segmentation approaches demonstrate the superior performance of our method in highly dynamic scenes, particularly in accurately handling challenging scenarios such as occlusions and complex motions.

Ching-Hsiang Hsu, "Graph-cut based Motion Segmentation with Physical Support Relationships in Crowded Urban Areas from a Moving RGB-D Camera," National Taiwan University Thesis Press, July, 2014.

Interactive Image Segmentation

    Date: Apr. 2013 - June 2013

In the motion segmentation section, two key techniques used for segmentation are graph-cut optimization and Simple Linear Iterative Clustering (SLIC). I initially experimented with these methods in interactive image segmentation. The process involves using a red pen to annotate regions of interest and a blue pen for regions of non-interest. By applying the max-flow algorithm, the regions are effectively separated based on the annotations. This approach is straightforward to implement and delivers impressive performance.

We keep moving forward, opening new doors, and doing new things, because we're curious and curiosity keeps leading us down new paths.

- Walt Disney