ESIM: an Open Event Camera Simulator

Event cameras are revolutionary sensors that work radically differently from standard cameras. Instead of capturing intensity images at a fixed rate, event cameras measure changes of intensity asynchronously, in the form of a stream of events, which encode per-pixel brightness changes. In the last few years, their outstanding properties (asynchronous sensing, no motion blur, high dynamic range) have led to exciting vision applications, with very low-latency and high robustness. However, these sensors are still scarce and expensive to get, slowing down progress.

To address this issue, we present ESIM: an efficient event camera simulator implemented in C++ and available open source. ESIM can simulate arbitrary camera motion in 3D scenes, while providing events, standard images, inertial measurements, with full ground truth information including camera pose, velocity, as well as depth and optical flow maps.

Features

In addition to simulating event cameras accurately, ESIM can also simulate standard cameras (including accurate motion blur) and an inertial measurement unit (IMU). Our open-source code comes with multiple rendering engines, among which a fast, custom renderer based on OpenGL that can work in real time, and a photorealistic renderer based on the UnrealCV project.

How does ESIM work?

Event cameras are challenging to simulate because they work fundamentally differently from standard cameras. Simulating events (which correspond to pixel-wise brightness changes) would require, in principle, access to the continuous brightness signal at every pixel of the sensor. In practice, the brightness signal must be reconstructed from a discrete number of samples (obtained by rendering images along the trajectory). Unlike previous simulators which rendered images at a high framerate, ESIM renders images only when they are necessary, through an adaptive sampling scheme. This is made possible by a tight integration between the rendering engine and the event simulator. Further details are available in our paper.

Where do I get ESIM?

ESIM is available open source here.

References