Research

Industry partners

Some of my research have been sponsored by or done in collaboration with the following industry partners.

       

Robust fitting

For several years, I have been working on robust geometric optimisation (a.k.a. “robust model fitting” or “robust fitting”) in computer vision. Roughly speaking, robust fitting is the fundamental problem of estimating the parameters of a geometric model from noisy and outlier contaminated data. Examples include estimating lines in images, planes in 3D point clouds, and 2D transformation function between two images.

Why is this an important problem?
Outliers are subsets of the data that are not explainable by the model of interest, and they invariably exist due to the imperfectness in data acquisition systems and/or preprocessing algorithms. Without enabling computer vision algorithms to be robust against outliers, the algorithms will almost certainly fail in real-life settings. Robust fitting is also the basis of many practical applications, as described in the rest of this page.

What have I done in this area?
My team and I have investigated various aspects of robust fitting in the context of computer vision, including

Some of my research findings in this area have been collected in the following book.

3D reconstruction

Under construction.

Space surveillance (joint work with Inovor and DST Group)

Under construction.

Vision for mining (joint work with Maptek)

Under construction.

Point cloud registration (joint work with Maptek)

I developed an interactive semi-automatic point cloud registration system. Guided by the system, the user’s role is simply to identify and search for overlapping regions across the point clouds. Specifically, given two point sets, the user clicks on a point in one set, then simply hovers the mouse on the other set to find a matching point. Each mouse position gives rise to a translation, and our system instantly optimises the rotation that aligns the point clouds.

Given a candidate 3D translation, only the 3D rotation needs to be estimated to align two point sets. Jointly with my colleagues, I proposed a fast rotation search algorithm that delivers globally optimal results in real time. Our method conducts branch-and-bound optimisation with a novel bounding function whose evaluation amounts to simple sorted array operations. This provides smooth and accurate feedback to the user’s search for overlapping regions. Our system is intuitive and helps to accelerate the registration of multiple scans. See Chin et al. and Parral Bustos et al. for details.

Our system is now part of Maptek I-Site Studio, the world's premier point cloud processing software for the mining industry.

Video stabilisation

Conceptually, video stabilization is achieved by estimating the camera trajectory throughout the video and then smoothing the trajectory. In practice, the pipeline invariably leads to estimating update transforms that adjust each frame of the video such that the overall sequence appears to be stabilized. Therefore, we argue that estimating good update transforms is more critical to success than accurately modeling and characterizing the motion of the camera. Based on this observation, we propose the usage of homography fields for video stabilization. A homography field is a spatially varying warp that is regularized to be as projective as possible, so as to enable accurate warping while adhering closely to the underlying geometric constraints. We show that homography fields are powerful enough to meet the various warping needs of video stabilization, not just in the core step of stabilization, but also in video inpainting. This enables relatively simple algorithms to be used for motion modeling and smoothing. Here is a sample result:

Compare with YouTube stabiliser's result:

More results of our method and coresponding results from YouTube stabiliser. See Liu and Chin, DICTA15 for more details of our method.

Augmented reality

To photo-realistically retexture a planar or non-rigidly deforming surface in an image, the 3D representation of the shape of the surface must first be accurately estimated. Robust fitting algorithms can be used to estimate the 3D representation of rigid and non-rigid surfaces (see Tran et al., ECCV12).

In the case of a rigid scene with multiple planar surfaces, multiple structure fitting algorithms can be used to simultaneously estimate the 3D representation of the planar surfaces for photo-realistic editing (see Pham et al., TPAMI14 and Pham et al., TIP14 for details).