Monocular Visual-Inertial Sensing of Unknown Rotating Objects: Observability Analyses and Case Study for Metric 3D Reconstructing of Space Debris

The fusion of a monocular camera and an inertial sensor (i.e., IMU) can help the autonomous navigation ability of a mobile robot, as well as obtain the environmental information. However, it is difficult to adapt the fusion for dynamic objects. Although several recent studies have tracked or even retrieved simple structural information about dynamic objects, there is no reported visual-inertial sensing research for unknown rotating objects, especially for those that are not surrounded by any static environmental visual feature. In fact, such rotating objects indeed represent a class of real and challenging scenarios, e.g., space debris. This letter discusses the monocular visual-inertial sensing of unknown rotating objects. Our first contribution is the comprehensive nonlinear observability analyses. We theoretically prove that the monocular visual-inertial sensing possesses favourable observability properties with respect to rotating objects, and we derive all observable and unobservable modes for the cases with unknown gravity and without gravity. The second contribution of this letter is a validation case study, i.e., the metric 3D reconstruction of space debris by means of a micro space robot. We propose a practical inertial sensor-assisted Structure from Motion strategy that can perceive rotating space debris. The experiment demonstrates that by using only the onboard monocular camera and IMU, the robot can obtain sufficient information about the space debris, such as all rotation parameters and even a dense point cloud model with absolute metric scale information. This case study confirms our theoretical results.