ESIM: an Open Event Camera Simulator

Event cameras are revolutionary sensors that work radically differently from standard cameras. Instead of capturing intensity images at a fixed rate, event cameras measure changes of intensity asynchronously, in the form of a stream of events, which encode per-pixel brightness changes. In the last few years, their outstanding properties (asynchronous sensing, no motion blur, high dynamic range) have led to exciting vision applications, with very low-latency and high robustness. However, these sensors are still scarce and expensive to get, slowing down progress.

To address this issue, we present ESIM: an efficient event camera simulator implemented in C++ and available open source. ESIM can simulate arbitrary camera motion in 3D scenes, while providing events, standard images, inertial measurements, with full ground truth information including camera pose, velocity, as well as depth and optical flow maps.


In addition to simulating event cameras accurately, ESIM can also simulate standard cameras (including accurate motion blur) and an inertial measurement unit (IMU). Our open-source code comes with multiple rendering engines, among which a fast, custom renderer based on OpenGL that can work in real time, and a photorealistic renderer based on the UnrealCV project.

 Example outputs
Example outputs from ESIM. From left to right: preview of the scene, depth map, optical flow, events, 3D event point cloud with camera trajectory.

How does ESIM work?

Event cameras are challenging to simulate because they work fundamentally differently from standard cameras. Simulating events (which correspond to pixel-wise brightness changes) would require, in principle, access to the continuous brightness signal at every pixel of the sensor. In practice, the brightness signal must be reconstructed from a discrete number of samples (obtained by rendering images along the trajectory). Unlike previous simulators which rendered images at a high framerate, ESIM renders images only when they are necessary, through an adaptive sampling scheme. This is made possible by a tight integration between the rendering engine and the event simulator. Further details are available in our paper.

ESIM architecture
ESIM relies on a tight coupling between the event simulator and the rendering engine to generate events accurately.

Where do I get ESIM?

ESIM is available open source here.



H. Rebecq, D. Gehrig, D. Scaramuzza

ESIM: an Open Event Camera Simulator

Conference on Robot Learning (CoRL), Zurich, 2018.

PDF YouTube Project page