Fall 2018 - Vision Algorithms for Mobile Robotics

UZH-BMINF020 / ETH-151-0632-00L

The course is open to all the students of the University of Zurich and ETH. Students should register through their own institutions.

Goal of the Course

For a robot to be autonomous, it has to perceive and understand the world around it. This course introduces you to the key computer vision algorithms used in mobile robotics, such as feature extraction, multiple view geometry, dense reconstruction, tracking, image retrieval, event-based vision, and visual-inertial odometry (the algorithms behind Apple ARKit, Google ARCore, Microsoft Hololens, Magic Leap and the Mars rovers). Basics knowledge of algebra, geomertry, and matrix calculus are required.

Time and location

Lectures: every Thursday from 10:15 to 12:00 in ETH LFW C5, Universitätstrasse 2, 8092 Zurich.

Exercises: Thursdays, roughly every two weeks, from 13:15 to 15:00 in ETH HG E 1.1, Rämistrasse 101, 8092 Zurich.

Please check out the course agenda below for the exact schedule.

Course Program, Slides, and Additional Reading Material

Official course program (please notice that this is a tentative schedule and that the effective content of the lecture can change from week to week.


Lecture and Exercise description

Slides, exercises, and additional reading material

20.09.2018 Lecture 01 - Introduction to Computer Vision and Visual Odometry Slides (updated 26.09.2018, see changes here)
Visual odometry tutorial Part I
Visual odometry tutorial Part II
SLAM survey paper
27.09.2018 Lecture 02 - Image Formation 1: perspective projection and camera models
Exercise 01 - Augmented reality wireframe cube
Slides (updated 26.09.2018)
Introduction to Matlab
Exercise 01
Exercise 01 Solution
04.10.2018 Lecture 03 - Image Formation 2: camera calibration algorithms
Exercise 02 - PnP problem
Slides (updated 11.10.2018, see changes here)
Additional reading on P3P and PnP problems
Exercise 02
Exercise 02 Solution
11.10.2018 Lecture 04 - Filtering & Edge detection Slides (updated 14.01.2019), see changes here)
18.10.2018 Lecture 05 - Point Feature Detectors, Part 1
Exercise 03 - Harris detector + descriptor + matching
Slides (updated 18.10.2018)
Exercise 03
Exercise 03 Solution
25.10.2018 Lecture 06 - Point Feature Detectors, Part 2
Exercise 04 - SIFT detector + descriptor + matching
Slides (updated 06.11.2018, see changes here)
Additional reading on feature detection
Exercise 04
Exercise 04 Solution
01.11.2018 Lecture 07 - Multiple-view geometry
Exercise 05 - Stereo vision: rectification, epipolar matching, disparity, triangulation
Slides (updated 27.01.2019, see changes here)
Additional reading on stereo image rectification
Exercise 05
Exercise 05 Solution
08.11.2018 Lecture 08 - Multiple-view geometry 2
Exercise 06 - Eight-Point Algorithm
Slides (updated 09.11.2018, see changes here)
Additional reading on 2-view geometry
Exercise 06 ( updated 09.11.2018 )
Exercise 06 Solution
15.11.2018 Lecture 09 - Multiple-view geometry 3 (Part 1)
Exercise 07 - P3P algorithm and RANSAC
Slides (last update 21.11.2018)
Additional reading on open-source VO algorithms
Exercise 07
Exercise 07 Solution
22.11.2018 Lecture 10 - Multiple-view geometry 3 (Part 2)
Exercise session: Intermediate VO Integration
Same slides as Lecture 09
29.11.2018 Lecture 11 - Optical Flow and Tracking (Lucas-Kanade)
Exercise 08 - Lucas-Kanade tracker
Slides (last update 28.11.2018)
Additional reading on open-source VO algorithms
Exercise 08
Exercise 08 Solution
06.12.2018 Lecture 12 - Place recognition and 3D Reconstruction
Exercise session: Deep Learning Tutorial
Slides on place recognition (last update 08.12.2018, see changes here)
Slides on 3D Reconstruction (last update 13.12.2018), see changes here)
Deep Learning Slides (last update 06.12.2018)

Additional reading on Bag-of-Words-based place recognition

Additional reading on dense 3D reconstruction

Optional exercise: place recognition
Optional exercise Solution
13.12.2018 Lecture 13 - Visual inertial fusion
Exercise 09 - Bundle Adjustment
Slides (last update 13.12.2018, see changes here)
Advanced Slides for the interested reader
Additional reading on visual-inertial fusion

Exercise 09
Exercise 09 Solution
20.12.2018 Lecture 14 - Event based vision

After the lecture, we will Scaramuzza's lab. Departure from lecture room at 12:00 via tram 10)
Exercise session: final VO integration (this exercise will take place in a different location, close to Scaramuzza's lab)
Slides (last update 20.12.2018, see changes here)
Additional reading on event-based vision


Oral Exams

10.01.2019 Oral exams for UZH students only.
Exam location: Scaramuzza's office, Andreasstrasse 15, 2.10, 8050 Zurich.
21.01.2019 to 01.02.2019 Oral exams for ETH students (the dates are handled by ETH Exam Center and are usually communicated before Christmas).
Exam location: Scaramuzza's office, Andreasstrasse 15, 2.10, 8050 Zurich.

Oral Exam Questions (udpated 21.12.2017)

The oral exam will last 30 minutes and will consist of one application question followed by two theoretical questions. This document contains a "non exhaustive" list of possible application questions and an "exhaustive" list of all the topics that you should learn about the course, which will be subject of discussion in the theoretical part.


You will be required to bring your own laptop to the exercise session. You will need to have Matlab already pre-installed in your machine for the exercise.

  • ETH students: Download from here
  • UZH students: Download from here; Info on how to setup the license can be found there.
Please install all the toolboxes included in the license.

Grading and optional Mini Project

The final grade is based on the oral exam (30 minutes, exam date for UZH: Jan. 10, 2019; exam date for ETH students will be between January 7 and 29, 2019 (the effective ETH dates will be communicated by ETH before Christmas). Mini projects are optional. Depending on the result of the mini project (the Project Specifications will be communicated on this page later during the course), the student will be rewarded with a grade increase of up to 0.5 of the final grade. However, notice that the mini project can be quite time consuming. Mini project specifications and files can be found in the table below. The deadline for the mini-project is Sunday, 06.01.2019, 23:59:59, and it can be submitted via e-mail to the assistants (detailed instructions in Project Specifications, to appear).

Project Specification vo_project_statement.pdf
FAQ Frequently Asked Questions
Parking garage dataset (easy) parking.zip (208.3 MB)
KITTI 00 dataset (hard) kitti00.zip (2.3 GB)
Malaga 07 dataset (hard) malaga-urban-dataset-extract-07.zip (2.4 GB)
Matlab script to load datasets main.m

Recommended Textbooks

(All available in the NEBIS catalogue)
  • Robotics, Vision and Control: Fundamental Algorithms, 2nd Ed., by Peter Corke 2017. The PDF of the book can be freely downloaded (only with ETH VPN) from the author's webpage.
  • Computer Vision: Algorithms and Applications, by Richard Szeliski, Springer, 2010. The PDF of the book can be freely downloaded from the author's webpage.
  • An Invitation to 3D Vision, by Y. Ma, S. Soatto, J. Kosecka, S.S. Sastry.
  • Multiple view Geometry, by R. Hartley and A. Zisserman.
  • Chapter 4 of "Autonomous Mobile Robots", by R. Siegwart, I.R. Nourbakhsh, D. Scaramuzza. PDF

Spring 2015 - Autonomous Mobile Robots

The course is currently open to all the students of the University of Zurich and ETH (Bachelor's and Master's). Lectures take place every Monday (from 16.02.2014 to 30.05.2014) from 14:15 to 16:00 in the ETH main building (HG) in room E 1.2. Exercise take place almost every second Tuesday from 10:15 to 12:00 in the ETH main building in room G1.

The course is also given as an MOOC (Massive Open Online Course) under edX.

Course Program

Official course webpage.

Recommended Textbook

R. Siegwart, I.R. Nourbakhsh, and D. Scaramuzza

Introduction to autonomous mobile robots 2nd Edition (hardback)

A Bradford Book, The MIT Press, ISBN: 978-0-262-01535-6, February, 2011

The book can be bought during the first lecture or on Amazon.

MIT Website Book Website Buy

Archived slides, videos, and lecture recordings

Since 2007, Prof. Davide Scaramuzza has been teaching this course at ETH Zurich and since 2012 the course has been shared also with University of Zurich. The lectures are based on Prof. Scaramuzza's book Autonomous Mobile Robots, MIT Press. Recordings of previous lectures (until 2012) can be watched or downloaded, only by ETH students, here.

You can download all the lecture slides and videos of past lectures (updated in 2010) from the following links: