Teaching



Fall 2017 - Vision Algorithms for Mobile Robotics

UZH-BMINF020 / ETH-151-0632-00L



The course is open to all the students of the University of Zurich and ETH. Students should register through their own institutions.



Goal of the Course


For a robot to be autonomous, it has to perceive and understand the world around it. This course introduces you to the key computer vision algorithms used in mobile robotics, such as feature extraction, multiple view geometry, dense reconstruction, tracking, image retrieval, event-based vision, and visual-inertial odometry (the algorithms behind Google Tango, Apple ARKit, Google ARCore, Microsoft Hololens, Magic Leap and the Mars rovers). Basics knowledge of algebra, geomertry, and matrix calculus are required.



Time and location


Lectures: every Thursday from 10:15 to 12:00 in ETH LFW C5, Universitätstrasse 2, 8092 Zurich.

Exercises: Thursdays, roughly every two weeks, from 13:15 to 15:00 in ETH HG E 1.1, Rämistrasse 101, 8092 Zurich.

Please check out the course agenda below for the exact schedule.



Course Program, Slides, and Add-on Material


Official course program (please notice that this is a tentative schedule and that the effective content of the lecture can change from week to week.


Lecture Date Lecture and Exercise Title Slides and add-on material
21.09.2017 Lecture 01 - Introduction to Computer Vision and Visual Odometry Slides (last update 22.09.2016)
Visual odometry tutorial Part I
Visual odometry tutorial Part II
28.09.2017 Lecture 02 - Image Formation 1: perspective projection and camera models Slides (last update 06.10.2016)
05.10.2017 Lecture 03 - Image Formation 2: camera calibration algorithms
Exercise 01 - Augmented reality wireframe cube
Slides (last update 18.10.2016)
Additional reading on P3P and PnP problems
Exercise 01 (last update 06.10.2016)
Introduction to Matlab
12.10.2017 Lecture 04 - Filtering & Edge detection
Exercise 02 - PnP problem
Slides (last update 13.10.2016)
Exercise 02 (last update 13.10.2016)
19.10.2017 Lecture 05 - Point Feature Detectors, Part 1
Exercise 03 - Harris detector + descriptor + matching
Slides (last update 20.10.2016)
Exercise 03 (last update 25.10.2016)
26.10.2017 Lecture 06 - Point Feature Detectors, Part 2 Slides (last update 27.10.2016)
Additional reading on feature detection
02.11.2017 Lecture 07 - Multiple-view geometry 1
Exercise 04 - Stereo vision: rectification, epipolar matching, disparity, triangulation
Slides (last update 10.11.2016)
Additional reading on stereo image rectification
Exercise 04 (last update 02.11.2016)
09.11.2017 Lecture 08 - Multiple-view geometry 2
Exercise 05 - Two-view Geometry
Slides (last update 16.11.2016)
Additional reading on 2-view geometry
Exercise 05 (last update 10.11.2016)
16.11.2017 Lecture 09 - Multiple-view geometry 3
Exercise 06 - P3P algorithm and RANSAC
Slides (last update 16.11.2016)
Additional reading on open-source VO algorithms
Exercise 06 (last update 16.11.2016)
23.11.2017 Lecture 10 - Dense 3D Reconstruction
Exercise 07 - Intermediate VO Integration
Slides (last update 30.11.2016)
Additional reading on dense 3D reconstruction
Find the VO project downloads below
30.11.2017 Lecture 11 - Optical Flow and Tracking (Lucas-Kanade)
Exercise 08 - Lucas-Kanade tracker
Slides (last update 30.11.2016)
Additional reading on Lucas-Kanade
Exercise 08 (last update 01.12.2016)
Exercise 08 Solutions (last update 15.12.2016)
07.12.2017 Lecture 12 - Place recognition
Exercise 09 - Recognition with Bag of Words
Slides (last update 07.12.2016)
Additional reading on Bag-of-Words-based place recognition
Exercise 09 (last update 08.12.2016)
Exercise 09 Solutions (last update 15.12.2016)
14.12.2017 Lecture 13 - Visual inertial fusion
Exercise 10 - Bundle Adjustment
Slides (last update 15.12.2016)
Advanced Slides for intrerested reader
Additional reading on visual-inertial fusion
Exercise 10 (last update 15.12.2016)
Exercise 10 Solutions (last update 25.12.2016)
21.12.2017 Lecture 14 - Event based vision + Scaramuzza's lab visit with live demos
Exercise 11 - final VO integration
Slides (last update 15.12.2016)
Additional reading on event-based vision



Oral Exam Questions (last udpate 01.01.2017)


The oral exam will last 30 minutes and will consist of one application question followed by two theoretical questions. This document contains a "non exhaustive" list of possible application questions and an "exhaustive" list of all the topics that you should learn about the course, which will be subject of discussion in the theoretical part.




Grading and optional Mini Project (last udpate 17.09.2017)


The final grade is based on the oral exam (30 minutes, exam date for UZH: Jan. 18). Exam date for ETH students will be between January and February 2018, dates communicated by ETH). Mini projects are optional and up to the students. In case a mini project is handed in and this runs properly producing a reasonable result, the student will be rewarded with an up 0.5 grade increase on the final grade. However, notice that the mini project can be quite time consuming. Mini project specification and files can be found in the table below. The deadline for the project is Sunday, 07.01.2018, 23:59:59, and it can be submitted via e-mail to the assistants (add links to external resources if the file size limit is exceeded).


DescriptionLink(size)
Project specification VO_project_statement_02.pdf (999.7 kB, last updated 01.12.2016)
FAQ Frequently Asked Questions
Solutions to all exercises all_solns.zip (40.3 kB, last updated 24.11.2016)
Parking garage dataset (easy) parking.zip (208.3 MB)
KITTI 00 dataset (hard) kitti00.zip (2.3 GB)
Malaga 07 dataset (hard) malaga-urban-dataset-extract-07.zip (2.4 GB)
Matlab script to load datasets main.m (2.6 kB)


Recommended Textbooks

(All available in the NEBIS catalogue)
  • Robotics, Vision and Control: Fundamental Algorithms, by Peter Corke 2011. The PDF of the book can be freely downloaded (only with ETH VPN) from the author's webpage.
  • Computer Vision: Algorithms and Applications, by Richard Szeliski, Springer, 2010. The PDF of the book can be freely downloaded from the author's webpage.
  • An Invitation to 3D Vision, by Y. Ma, S. Soatto, J. Kosecka, S.S. Sastry.
  • Multiple view Geometry, by R. Hartley and A. Zisserman.

Spring 2015 - Autonomous Mobile Robots


The course is currently open to all the students of the University of Zurich and ETH (Bachelor's and Master's). Lectures take place every Monday (from 16.02.2014 to 30.05.2014) from 14:15 to 16:00 in the ETH main building (HG) in room E 1.2. Exercise take place almost every second Tuesday from 10:15 to 12:00 in the ETH main building in room G1.


The course is also given as an MOOC (Massive Open Online Course) under edX.


Course Program

Official course webpage.


Recommended Textbook

R. Siegwart, I.R. Nourbakhsh, and D. Scaramuzza

Introduction to autonomous mobile robots 2nd Edition (hardback)

A Bradford Book, The MIT Press, ISBN: 978-0-262-01535-6, February, 2011

The book can be bought during the first lecture or on Amazon.

MIT Website Book Website Buy



Archived slides, videos, and lecture recordings

Since 2007, Prof. Davide Scaramuzza has been teaching this course at ETH Zurich and since 2012 the course has been shared also with University of Zurich. The lectures are based on Prof. Scaramuzza's book Autonomous Mobile Robots, MIT Press. Recordings of previous lectures (until 2012) can be watched or downloaded, only by ETH students, here.


You can download all the lecture slides and videos of past lectures (updated in 2010) from the following links: