DSEC: A Stereo Event Camera Dataset for Driving Scenarios + CVPRW 2021 Competition



Description

Once an academic venture, autonomous driving has received unparalleled corporate funding in the last decade. Still, operating conditions of current autonomous cars are mostly restricted to ideal scenarios. This means that driving in challenging illumination conditions such as night, sunrise, and sunset remains an open problem. In these cases, standard cameras are being pushed to their limits in terms of low light and high dynamic range performance. To address these challenges, we propose, DSEC, a new dataset that contains such demanding illumination conditions and provides a rich set of sensory data. DSEC offers data from a wide-baseline stereo setup of two color frame cameras and two high-resolution monochrome event cameras. In addition, we collect lidar data and RTK GPS measurements, both hardware synchronized with all camera data. One of the distinctive features of this dataset is the inclusion of high-resolution event cameras. Event cameras have received increasing attention for their high temporal resolution and high dynamic range performance. However, due to their novelty, event camera datasets in driving scenarios are rare. This work presents the first high resolution, large scale stereo dataset with event cameras. The dataset contains 53 sequences collected by driving in a variety of illumination conditions and provides ground truth disparity for the development and evaluation of event-based stereo algorithms.

Citing

If you use this work in your research, please cite the following paper:

DSEC: A Stereo Event Camera Dataset for Driving Scenarios

M. Gehrig, W. Aarents, D. Gehrig, D. Scaramuzza

DSEC: A Stereo Event Camera Dataset for Driving Scenarios

IEEE Robotics and Automation Letters (RA-L), 2021.

PDF Code


BibTeX:
@Article{Gehrig21ral,
  author  = {Mathias Gehrig and Willem Aarents and Daniel Gehirg and Davide Scaramuzza},
  title   = {DSEC: A Stereo Event Camera Dataset for Driving Scenarios},
  journal = {IEEE Robotics and Automation Letters},
  year    = {2021},
  doi     = {10.1109/LRA.2021.3068942}
}


Competition

We are participating in the CVPR 2021 Workshop on Event-Based Vision!

Task

The goal is to estimate dense disparity from two event cameras in a stereo setup.
Details:
  • The predictions will be evaluated on the left event camera at specified timestamps on the test set.
  • Usage of global shutter (RGB) cameras is not allowed for this competition.
  • The relevant metric is the average absolute error of the disparity.
  • The evaluation will be performed on all pixels on which groundtruth disparity is available.

Deadline

The deadline for the submission is 6th of June, 2021 (11:59 PM Pacific Time).

Submission Format

The submission will be evaluated by an evaluation server and contain the disparity maps at specified timestamps on the test set. The exact submission format will be announced as soon as the evaluation server is online.

Technical Report

The participants should prepare a short (1 or 2 pages) technical report that will be available to the public upon the participants' consent.

Winner

The winner of the competition will be invited to present at the CVPR 2021 Workshop on Event-Based Vision.

Copyright

This dataset is provided to you under the Creative Commons Attribution-ShareAlike 4.0 International public license (CC BY-SA 4.0). This means that you must attribute the work in the manner specified by the authors and must distribute your contributions under the same license. Using the material for commercial purposes is allowed.

Download

We provide the option to download the dataset in larger zip files or more fine-grained in separated files per sequence.

Training Data

train_events.zip (125 GB)
train_images.zip (216 GB)
train_disparity.zip (12 GB)
train_calibration.zip

Test Data

test_events.zip (27 GB)
test_images.zip (43 GB)
test_calibration.zip

The test set does not contain groundtruth because it will be used for the upcoming benchmark.

Individual Files

Training Sequences

interlaken_00_c
Disparity Data
disparity_event.zip (39 MB)
disparity_image.zip (107 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (816 MB)
events_right.zip (769 MB)
image_timestamps.txt
images_rectified_left.zip (1.2 GB)
images_rectified_right.zip (1.2 GB)
interlaken_00_d
Disparity Data
disparity_event.zip (145 MB)
disparity_image.zip (388 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (3.1 GB)
events_right.zip (3.0 GB)
image_timestamps.txt
images_rectified_left.zip (4.3 GB)
images_rectified_right.zip (4.2 GB)
interlaken_00_e
Disparity Data
disparity_event.zip (121 MB)
disparity_image.zip (322 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (2.4 GB)
events_right.zip (2.4 GB)
image_timestamps.txt
images_rectified_left.zip (4.0 GB)
images_rectified_right.zip (3.9 GB)
interlaken_00_f
Disparity Data
disparity_event.zip (75 MB)
disparity_image.zip (204 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (1.2 GB)
events_right.zip (1.2 GB)
image_timestamps.txt
images_rectified_left.zip (2.7 GB)
images_rectified_right.zip (2.7 GB)
interlaken_00_g
Disparity Data
disparity_event.zip (87 MB)
disparity_image.zip (235 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (1.2 GB)
events_right.zip (1.2 GB)
image_timestamps.txt
images_rectified_left.zip (2.3 GB)
images_rectified_right.zip (2.2 GB)
thun_00_a
Disparity Data
disparity_event.zip (17 MB)
disparity_image.zip (45 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (285 MB)
events_right.zip (261 MB)
image_timestamps.txt
images_rectified_left.zip (470 MB)
images_rectified_right.zip (457 MB)
zurich_city_00_a
Disparity Data
disparity_event.zip (62 MB)
disparity_image.zip (177 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (997 MB)
events_right.zip (861 MB)
image_timestamps.txt
images_rectified_left.zip (1.8 GB)
images_rectified_right.zip (1.8 GB)
zurich_city_00_b
Disparity Data
disparity_event.zip (94 MB)
disparity_image.zip (284 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (1.2 GB)
events_right.zip (1.0 GB)
image_timestamps.txt
images_rectified_left.zip (2.9 GB)
images_rectified_right.zip (2.9 GB)
zurich_city_01_a
Disparity Data
disparity_event.zip (48 MB)
disparity_image.zip (137 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (678 MB)
events_right.zip (547 MB)
image_timestamps.txt
images_rectified_left.zip (1.3 GB)
images_rectified_right.zip (1.3 GB)
zurich_city_01_b
Disparity Data
disparity_event.zip (77 MB)
disparity_image.zip (214 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (1.5 GB)
events_right.zip (1.3 GB)
image_timestamps.txt
images_rectified_left.zip (2.7 GB)
images_rectified_right.zip (2.8 GB)
zurich_city_01_c
Disparity Data
disparity_event.zip (50 MB)
disparity_image.zip (137 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (1.3 GB)
events_right.zip (1.1 GB)
image_timestamps.txt
images_rectified_left.zip (2.0 GB)
images_rectified_right.zip (2.0 GB)
zurich_city_01_d
Disparity Data
disparity_event.zip (42 MB)
disparity_image.zip (118 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (950 MB)
events_right.zip (829 MB)
image_timestamps.txt
images_rectified_left.zip (1.6 GB)
images_rectified_right.zip (1.6 GB)
zurich_city_01_e
Disparity Data
disparity_event.zip (109 MB)
disparity_image.zip (304 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (2.0 GB)
events_right.zip (1.7 GB)
image_timestamps.txt
images_rectified_left.zip (4.1 GB)
images_rectified_right.zip (4.2 GB)
zurich_city_01_f
Disparity Data
disparity_event.zip (95 MB)
disparity_image.zip (263 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (2.0 GB)
events_right.zip (1.7 GB)
image_timestamps.txt
images_rectified_left.zip (3.4 GB)
images_rectified_right.zip (3.4 GB)
zurich_city_02_a
Disparity Data
disparity_event.zip (15 MB)
disparity_image.zip (43 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (340 MB)
events_right.zip (269 MB)
image_timestamps.txt
images_rectified_left.zip (501 MB)
images_rectified_right.zip (508 MB)
zurich_city_02_b
Disparity Data
disparity_event.zip (64 MB)
disparity_image.zip (185 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (1.8 GB)
events_right.zip (1.5 GB)
image_timestamps.txt
images_rectified_left.zip (2.8 GB)
images_rectified_right.zip (2.9 GB)
zurich_city_02_c
Disparity Data
disparity_event.zip (123 MB)
disparity_image.zip (342 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (4.9 GB)
events_right.zip (4.3 GB)
image_timestamps.txt
images_rectified_left.zip (6.5 GB)
images_rectified_right.zip (6.6 GB)
zurich_city_02_d
Disparity Data
disparity_event.zip (72 MB)
disparity_image.zip (204 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (1.7 GB)
events_right.zip (1.4 GB)
image_timestamps.txt
images_rectified_left.zip (4.3 GB)
images_rectified_right.zip (4.4 GB)
zurich_city_02_e
Disparity Data
disparity_event.zip (77 MB)
disparity_image.zip (211 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (3.2 GB)
events_right.zip (2.9 GB)
image_timestamps.txt
images_rectified_left.zip (4.3 GB)
images_rectified_right.zip (4.4 GB)
zurich_city_03_a
Disparity Data
disparity_event.zip (40 MB)
disparity_image.zip (106 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (1.6 GB)
events_right.zip (1.6 GB)
image_timestamps.txt
images_rectified_left.zip (1.8 GB)
images_rectified_right.zip (1.8 GB)
zurich_city_04_a
Disparity Data
disparity_event.zip (57 MB)
disparity_image.zip (167 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (726 MB)
events_right.zip (593 MB)
image_timestamps.txt
images_rectified_left.zip (1.4 GB)
images_rectified_right.zip (1.4 GB)
zurich_city_04_b
Disparity Data
disparity_event.zip (22 MB)
disparity_image.zip (64 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (264 MB)
events_right.zip (252 MB)
image_timestamps.txt
images_rectified_left.zip (543 MB)
images_rectified_right.zip (537 MB)
zurich_city_04_c
Disparity Data
disparity_event.zip (81 MB)
disparity_image.zip (237 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (925 MB)
events_right.zip (765 MB)
image_timestamps.txt
images_rectified_left.zip (2.2 GB)
images_rectified_right.zip (2.2 GB)
zurich_city_04_d
Disparity Data
disparity_event.zip (61 MB)
disparity_image.zip (173 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (772 MB)
events_right.zip (679 MB)
image_timestamps.txt
images_rectified_left.zip (1.7 GB)
images_rectified_right.zip (1.7 GB)
zurich_city_04_e
Disparity Data
disparity_event.zip (18 MB)
disparity_image.zip (53 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (261 MB)
events_right.zip (227 MB)
image_timestamps.txt
images_rectified_left.zip (522 MB)
images_rectified_right.zip (514 MB)
zurich_city_04_f
Disparity Data
disparity_event.zip (52 MB)
disparity_image.zip (150 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (799 MB)
events_right.zip (620 MB)
image_timestamps.txt
images_rectified_left.zip (1.6 GB)
images_rectified_right.zip (1.6 GB)
zurich_city_05_a
Disparity Data
disparity_event.zip (150 MB)
disparity_image.zip (427 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (1.8 GB)
events_right.zip (1.6 GB)
image_timestamps.txt
images_rectified_left.zip (3.4 GB)
images_rectified_right.zip (3.3 GB)
zurich_city_05_b
Disparity Data
disparity_event.zip (125 MB)
disparity_image.zip (353 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (1.8 GB)
events_right.zip (1.6 GB)
image_timestamps.txt
images_rectified_left.zip (3.1 GB)
images_rectified_right.zip (3.0 GB)
zurich_city_06_a
Disparity Data
disparity_event.zip (129 MB)
disparity_image.zip (354 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (2.5 GB)
events_right.zip (2.1 GB)
image_timestamps.txt
images_rectified_left.zip (3.3 GB)
images_rectified_right.zip (3.3 GB)
zurich_city_07_a
Disparity Data
disparity_event.zip (101 MB)
disparity_image.zip (295 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (1.7 GB)
events_right.zip (1.5 GB)
image_timestamps.txt
images_rectified_left.zip (2.9 GB)
images_rectified_right.zip (2.9 GB)
zurich_city_08_a
Disparity Data
disparity_event.zip (69 MB)
disparity_image.zip (200 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (978 MB)
events_right.zip (806 MB)
image_timestamps.txt
images_rectified_left.zip (1.4 GB)
images_rectified_right.zip (1.4 GB)
zurich_city_09_a
Disparity Data
disparity_event.zip (69 MB)
disparity_image.zip (174 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (2.8 GB)
events_right.zip (2.5 GB)
image_timestamps.txt
images_rectified_left.zip (3.4 GB)
images_rectified_right.zip (3.4 GB)
zurich_city_09_b
Disparity Data
disparity_event.zip (12 MB)
disparity_image.zip (32 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (368 MB)
events_right.zip (299 MB)
image_timestamps.txt
images_rectified_left.zip (743 MB)
images_rectified_right.zip (745 MB)
zurich_city_09_c
Disparity Data
disparity_event.zip (52 MB)
disparity_image.zip (143 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (1.3 GB)
events_right.zip (1.1 GB)
image_timestamps.txt
images_rectified_left.zip (2.7 GB)
images_rectified_right.zip (2.7 GB)
zurich_city_09_d
Disparity Data
disparity_event.zip (57 MB)
disparity_image.zip (155 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (1.7 GB)
events_right.zip (1.5 GB)
image_timestamps.txt
images_rectified_left.zip (3.4 GB)
images_rectified_right.zip (3.4 GB)
zurich_city_09_e
Disparity Data
disparity_event.zip (26 MB)
disparity_image.zip (69 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (844 MB)
events_right.zip (751 MB)
image_timestamps.txt
images_rectified_left.zip (1.7 GB)
images_rectified_right.zip (1.7 GB)
zurich_city_10_a
Disparity Data
disparity_event.zip (106 MB)
disparity_image.zip (270 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (3.7 GB)
events_right.zip (3.4 GB)
image_timestamps.txt
images_rectified_left.zip (5.2 GB)
images_rectified_right.zip (5.2 GB)
zurich_city_10_b
Disparity Data
disparity_event.zip (102 MB)
disparity_image.zip (273 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (4.5 GB)
events_right.zip (4.2 GB)
image_timestamps.txt
images_rectified_left.zip (5.1 GB)
images_rectified_right.zip (5.1 GB)
zurich_city_11_a
Disparity Data
disparity_event.zip (38 MB)
disparity_image.zip (117 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (320 MB)
events_right.zip (269 MB)
image_timestamps.txt
images_rectified_left.zip (860 MB)
images_rectified_right.zip (839 MB)
zurich_city_11_b
Disparity Data
disparity_event.zip (164 MB)
disparity_image.zip (476 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (1.7 GB)
events_right.zip (1.5 GB)
image_timestamps.txt
images_rectified_left.zip (3.7 GB)
images_rectified_right.zip (3.5 GB)
zurich_city_11_c
Disparity Data
disparity_event.zip (159 MB)
disparity_image.zip (452 MB)
disparity_timestamps.txt
Calibration
Event Data Image Data
calibration.zip events_left.zip (1.8 GB)
events_right.zip (1.6 GB)
image_timestamps.txt
images_rectified_left.zip (3.6 GB)
images_rectified_right.zip (3.5 GB)

Test Sequences


interlaken_00_a
Calibration
Event Data Image Data
calibration.zip events_left.zip (2.2 GB)
events_right.zip (2.1 GB)
image_timestamps.txt
images_rectified_left.zip (2.5 GB)
images_rectified_right.zip (2.4 GB)
interlaken_00_b
Calibration
Event Data Image Data
calibration.zip events_left.zip (2.6 GB)
events_right.zip (2.4 GB)
image_timestamps.txt
images_rectified_left.zip (3.4 GB)
images_rectified_right.zip (3.3 GB)
interlaken_01_a
Calibration
Event Data Image Data
calibration.zip events_left.zip (2.0 GB)
events_right.zip (1.7 GB)
image_timestamps.txt
images_rectified_left.zip (3.5 GB)
images_rectified_right.zip (3.4 GB)
thun_01_a
Calibration
Event Data Image Data
calibration.zip events_left.zip (255 MB)
events_right.zip (235 MB)
image_timestamps.txt
images_rectified_left.zip (395 MB)
images_rectified_right.zip (385 MB)
thun_01_b
Calibration
Event Data Image Data
calibration.zip events_left.zip (1.4 GB)
events_right.zip (1.4 GB)
image_timestamps.txt
images_rectified_left.zip (2.1 GB)
images_rectified_right.zip (2.1 GB)
zurich_city_12_a
Calibration
Event Data Image Data
calibration.zip events_left.zip (1.3 GB)
events_right.zip (1.2 GB)
image_timestamps.txt
images_rectified_left.zip (1.4 GB)
images_rectified_right.zip (1.4 GB)
zurich_city_13_a
Calibration
Event Data Image Data
calibration.zip events_left.zip (346 MB)
events_right.zip (297 MB)
image_timestamps.txt
images_rectified_left.zip (716 MB)
images_rectified_right.zip (717 MB)
zurich_city_13_b
Calibration
Event Data Image Data
calibration.zip events_left.zip (360 MB)
events_right.zip (310 MB)
image_timestamps.txt
images_rectified_left.zip (654 MB)
images_rectified_right.zip (655 MB)
zurich_city_14_a
Calibration
Event Data Image Data
calibration.zip events_left.zip (422 MB)
events_right.zip (357 MB)
image_timestamps.txt
images_rectified_left.zip (902 MB)
images_rectified_right.zip (907 MB)
zurich_city_14_b
Calibration
Event Data Image Data
calibration.zip events_left.zip (417 MB)
events_right.zip (321 MB)
image_timestamps.txt
images_rectified_left.zip (1.1 GB)
images_rectified_right.zip (1.1 GB)
zurich_city_14_c
Calibration
Event Data Image Data
calibration.zip events_left.zip (1.0 GB)
events_right.zip (825 MB)
image_timestamps.txt
images_rectified_left.zip (2.3 GB)
images_rectified_right.zip (2.3 GB)
zurich_city_15_a
Calibration
Event Data Image Data
calibration.zip events_left.zip (1.5 GB)
events_right.zip (1.4 GB)
image_timestamps.txt
images_rectified_left.zip (2.4 GB)
images_rectified_right.zip (2.3 GB)