Monocular Dense Reconstruction

Monocular dense reconstruction is the problem of estimating dense and accurate 3D maps from a single moving camera.

REMODE: Probabilistic, Monocular Dense Reconstruction in Real Time

REMODE (REgularized, probabilistic, MOnocular, DEnse reconstruction) is a novel method to estimate dense and accurate depth maps from a single moving camera. A probabilistic depth measurement is carried out in real time on a per-pixel basis and the computed uncertainty is used to reject erroneous estimations and provide live feedback on the reconstruction progress. REMODE uses a novel approach to depth map computation that combines Bayesian estimation and recent development on convex optimization for image processing. In the paper, we demonstrate that our method outperforms state-of-the-art techniques in terms of accuracy, while exhibiting high efficiency in memory usage and computing power. Our CUDA-based implementation runs at 50Hz on a laptop computer and is released as open-source software (code here).



M. Pizzoli, C. Forster, D. Scaramuzza

REMODE: Probabilistic, Monocular Dense Reconstruction in Real Time

IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, 2014.

PDF YouTube Code

In this video, we demonstrate the use of REMODE on our micro quadrotor, which is used as an autonomous flying 3D scanner . Our quadrotor flies autonomously over an area of interest using only a single onboard, downward-looking camera and an inertial measurement unit (IMU). All flight-relevant computation is performed onboard, on a smartphone processor. No pilot, GPS, or motion-capture system is used. Since the drone currently does not have a GPU, images down to a laptop computer, where the dense 3D model of the environment is computed online. Our algorithms are so efficient that we can create these models in real-time, thus showing the progress of the dense 3D model in real time. Thus, our system has high potential for applications where detailed information about an unknown environment is time critical, such as during search-and-rescue missions.



M. Faessler, F. Fontana, C. Forster, E. Mueggler, M. Pizzoli, D. Scaramuzza

Autonomous, Vision-based Flight and Live Dense 3D Mapping with a Quadrotor Micro Aerial Vehicle

Journal of Field Robotics, 2016.

PDF YouTube1 YouTube2 YouTube3 YouTube4 Software

Active, Dense Reconstruction

The estimation of the depth uncertainty makes REMODE extremely attractive for motion planning and active-vision problems. In this work, we investigate the following problem: Given the image of a scene, what is the trajectory that a robot-mounted camera should follow to allow optimal dense 3D reconstruction? The solution we propose is based on maximizing the information gain over a set of candidate trajectories. In order to estimate the information that we expect from a camera pose, we introduce a novel formulation of the measurement uncertainty that accounts for the scene appearance (i.e., texture in the reference view), the scene depth, and the vehicle pose. We successfully demonstrate our approach in the case of realtime, monocular reconstruction from a small quadrotor and validate the effectiveness of our solution in both synthetic and real experiments. This is the first work on active, monocular dense reconstruction, which chooses motion trajectories that minimize perceptual ambiguities inferred by the texture in the scene.


C. Forster, M. Pizzoli, D. Scaramuzza

Appearance-based Active, Monocular, Dense Reconstruction for Micro Aerial Vehicles

Robotics: Science and Systems, Berkely, 2014.


This video shows results from a modification of the SVO algorithm that generalizes to a set of four rigidly attached cameras (not necessarily overlapping). Simultaneously, we run a CPU implementation of the REMODE algorithm on the four cameras. Everything runs in real-time on a laptop computer at 50 Hz with input from the 4 cameras. Parking garage dataset courtesy of NVIDIA.



C. Forster, M. Pizzoli, D. Scaramuzza

SVO: Fast Semi-Direct Monocular Visual Odometry

IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, 2014.

PDF YouTube Software

Continuous, Monocular Elevation Mapping applied to Autonomous Landing

In this work, we propose a resource-efficient system for real-time 3D terrain reconstruction and landing-spot detection for micro aerial vehicles. The system runs on an on-board smartphone processor (i.e., Odroid) using only input from a single downward-looking camera and an IMU. We generate a 2D elevation map that is probabilistic, of fixed size, and robot-centric; thus, always covering the area immediately underneath the robot. The elevation map is continuously updated at a rate of 1 Hz with depth maps that are triangulated from multiple views using recursive Bayesian estimation. To highlight the usefulness of the proposed mapping framework for autonomous navigation of micro aerial vehicles, we successfully demonstrate fully autonomous landing including landing-spot detection in real-world experiments.



C. Forster, M. Faessler, F. Fontana, M. Werlberger, D. Scaramuzza

Continuous On-Board Monocular-Vision-based Elevation Mapping Applied to Autonomous Landing of Micro Aerial Vehicles

IEEE International Conference on Robotics and Automation (ICRA), Seattle, 2015.

PDF YouTube

Air-Ground Localization and Map Augmentation

In this work, we propose a new method for the localization of a small quadrotor with respect to a ground robot. We solve the problem of registering the 3D maps computed by the two robots using different sensors, namely, a dense 3D reconstruction from the quadrotor single camera is aligned with the map computed from the depth sensor on the ground robot. Once aligned, the dense reconstruction from the quadrotor is used to augment the map computed by the ground robot, by augmenting it with the information conveyed by the aerial views. The overall approach is novel as it builds on recent developments in live dense reconstruction from moving cameras (i.e., REMODE) to address the problem of air-ground localization.



C. Forster, M. Pizzoli, D. Scaramuzza

Air-Ground Localization and Map Augmentation Using Monocular Dense Reconstruction

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Tokyo, 2013.