Teaching
Fall 2020 - Vision Algorithms for Mobile Robotics
UZH-DINF2039 / ETH-151-0632-00L
Goal of the Course
For a robot to be autonomous, it has to perceive and understand the world around it. This course introduces you to the key computer vision algorithms used in mobile robotics, such as image formation, filtering, feature extraction, multiple view geometry, dense reconstruction, tracking, image retrieval, event-based vision, visual-inertial odometry, Simultaneous Localization And Mapping (SLAM), and some basics of deep learning (the algorithms behind Apple ARKit, Google Visual Positioning Service, Microsoft Hololens, Magic Leap, Oculus Quest and Oculus Insight, the Mars Curiosity rover), and the Mars helicopter. Basic knowledge of algebra, geomertry, and matrix calculus are required.
Time and location
Since 05.11.2020, all lectures and exercises have been taking place on ZOOM. The links have been communicated by email.
Lectures: every Thursday from 10:15 to 12:00.
Exercises: every Thursday from 14:15 to 16:00.
Please check out the course agenda below for the exact schedule.
Instructors
Lecture Instructor: Prof. Dr. Davide Scaramuzza. Exercise Instructors: Daniel Gehrig and Manasi Muglikar. Office hours of Prof. Scaramuzza: Every Thursday from 16:00 to 18:00 on ZOOM (please book your slot by email by using VAMR" as email subject and he will reply with the ZOOM link)
Video Recordings
All video recordings of the lectures and exercises are available in this folder for your convenience. Alterantivey, they can be accessed from the url reported in the course program below. The password has been communicated by email to all the enrolled students.
Course Program, Slides, and Additional Reading Material
Please notice that this is a tentative schedule and that the effective content of the lecture may change from week to week.
Date |
Lecture and Exercise description |
Slides, exercises, and additional reading material |
| 17.09.2020 | Lecture 01 - Introduction to Computer Vision and Visual Odometry No Exercise today. Lecture 01 Video Recording | Lecture Slides Visual odometry tutorial Part I Visual odometry tutorial Part II SLAM survey paper |
| 24.09.2020 | Lecture 02 - Image Formation: perspective projection and camera models Exercise 01- Augmented reality wireframe cube Lecture 02 Video Recording Exercise 01 Video Recording | Lecture Slides (updated 18.01.2021, see changes here) Introduction to Matlab Exercise 01 (updated 01.10.2020) Solutions 01 (updated 01.10.2020) |
| 01.10.2020 | Lecture 03 - Camera Calibration Exercise 02 - PnP problem Lecture 03 Video Recording Exercise 02 Video Recording | Lecture Slides (updated 01.10.2020, see changes here) Additional reading on P3P and PnP problems Exercise 02 (updated 07.10.2020) Solutions 02 |
| 08.10.2020 | Lecture 04 - Filtering & Edge detection No Exercise today. Lecture 04 Video Recording | Lecture Slides |
| 15.10.2020 | Lecture 05 - Point Feature Detectors, Part 1 Exercise 03 - Harris detector + descriptor + matching Lecture 05 Video Recording Exercise 03 Video Recording | Lecture Slides Exercise 03 Solution 03 |
| 22.10.2020 | Lecture 06 - Point Feature Detectors, Part 2 Exercise 04 - SIFT detector + descriptor + matching Lecture 06 Video Recording Exercise 04 Video Recording | Lecture Slides (updated 28.10.2020, see changes here) Additional reading on feature detection Exercise 04 Solution 04 |
| 29.10.2020 | Lecture 07 - Multiple-view geometry 1 Exercise 05 - Stereo vision: rectification, epipolar matching, disparity, triangulation Lecture 07 Video Recording Exercise 05 Video Recording | Lecture Slides (updated 04.01.2021, see changes here) Additional reading on stereo image rectification Exercise 05 Solution 05 |
| 05.11.2020 | Lecture 08 - Multiple-view geometry 2 Exercise 06 - Eight-Point Algorithm Lecture 08 Video Recording Exercise 06 Video Recording | Lecture Slides (updated 14.11.2020, see changes here) Additional reading on 2-view geometry Exercise 06 Solution 06 |
| 12.11.2020 | Lecture 09 - Multiple-view geometry 3 Exercise 07 - P3P algorithm and RANSAC Lecture 09 Video Recording Exercise 07 Video Recording | Lecture Slides (updated 12.11.2020, see changes here) Additional reading on RANSAC Exercise 07 Solutions 07 |
| 19.11.2020 | Lecture 10 - Multiple-view geometry 4 Exercise session: Intermediate VO Integration Lecture 10 Video Recording Exercise VO integration - Video Recording | Lecture Slides (updated 03.01.2021, see changes here) Additional reading on VO- accuracy evaluation |
| 26.11.2020 | Lecture 11 1st hour: Optical Flow and KLT Tracking 2nd hour: Seminar by Christian Forster, Facebook Zurich, on the Oculus Quest VIO/SLAM 1st hour of the afternoon: KLT tracking continued. 2nd hour of the afternoon: Exercise 08 - Lucas-Kanade tracker Lecture 11 Part 1 + Facebook seminar - Video Recording Lecture 11 Part 2 - Video Recording Exercise 08 Video Recording | Lecture Slides (updated 26.11.2020, see changes here) Additional reading on KLT Exercise 08 Solutions 08 |
| 03.12.2020 | Lecture 12a (1st hour) - Place recognition Lecture 12b (2nd hour) - Dense 3D Reconstruction and Place recognition Lecture 12c (3rd and 4th hour, replaces exercise) - Deep Learning Tutorial Optional Exercise on Place Recogntion Lecture 12ab - Video Recording Lecture 12c - Video Recording | Lecture Slides on Place Recognition Lecture Slides on Dense 3D Reconstruction Lecture Slides on Deep Learning Optional Exercise on Place Recogntion Additional reading on dense 3D reconstruction Additional reading on Bag-of-Words-based place recognition |
| 10.12.2020 | Lecture 13 - Visual inertial fusion Exercise 09 - Bundle Adjustment Lecture 13 - Video Recording Exercise 09 - Video Recording |
Lecture Slides (updated 18.01.2021, see changes here)
Additional reading on VIO Exercise 09 Solutions 09 |
| 17.12.2020 | Lecture 14 - Event based vision Exercise session: Final VO Integration Lecture 14 - Video Recording Exercise on Exam and final VO integration - Video Recording |
Lecture Slides
Additional reading on event-based vision |
Date |
Oral Exams |
| 21.01.2021 to 22.01.2021 | Oral exams for UZH students only. Exam location: Online or physical in Scaramuzza's office, Andreasstrasse 15, 2.10, 8050 Zurich. |
| 27.01.2021 to 12.02.2021 | Oral exams for ETH students (the dates are handled by ETH Exam Center and are usually communicated by ETH to you via email before Christmas). Exam location: Online or physical in Scaramuzza's office, Andreasstrasse 15, 2.10, 8050 Zurich. |
Oral Exam and Exam Questions Examples (udpated 20.12.2019)
The oral exam will last 30 minutes and will consist of one application question followed by two theoretical questions. This document contains a non-exhaustive list of possible application questions and an exhaustive list of all the topics that you should learn about the course, which will be subject of discussion in the theoretical part.
Exercises
You will be required to bring your own laptop to the exercise session. You will need to have Matlab already pre-installed in your machine for the exercise.
- ETH students: Download from here
- UZH students: Download from here; Info on how to setup the license can be found there.
Optional Mini Project and Grading
Students taking the final exam have the opportunity to submit a mini project to increase their final grade. Notice that the mini projects are optional, that is, not mandatory. Depending on the result of the mini project (see Project Specifications from last year below (to be updated)), the student will be rewarded with a grade increase of up to 0.5 on the top of the grade of the oral exam. However, bear in mind that the mini project can be quite time consuming. Mini project specifications and files can be found in the table below. The deadline for the mini-project is Sunday, 10.01.2021, 23:59:59, and it can be submitted via e-mail to the assistants.
| Description | Link (size) |
| Project Specification | vo_project_statement.pdf |
| FAQ | Frequently Asked Questions |
| Parking garage dataset (easy) | parking.zip (208.3 MB) |
| KITTI 00 dataset (hard) | kitti00.zip (2.3 GB) |
| Malaga 07 dataset (hard) | malaga-urban-dataset-extract-07.zip (2.4 GB) |
| Matlab script to load datasets | main.m |
Recommended Textbooks
(All available in the NEBIS catalogue)Unfortunately, a single textbook that covers all the material seen in class does not exist yet. However, the topics covered in class can be found in the books listed below. At the end of each lecture's slide deck, you will find a list of chapters and books where the topics can be found.
- Computer Vision: Algorithms and Applications, by Richard Szeliski, Springer, 1st Edition, 2010. The PDF of the book can be freely downloaded from the author's webpage. Although the author is working on a 2nd edition, this is still under progress. Thus, in the lecture nodes we will reference chapters of the 1st edition.
- Robotics, Vision and Control: Fundamental Algorithms, 2nd Ed., by Peter Corke 2017. The PDF of the book can be freely downloaded (only with ETH VPN) from the author's webpage.
- An Invitation to 3D Vision, by Y. Ma, S. Soatto, J. Kosecka, S.S. Sastry.
- Multiple view Geometry, by R. Hartley and A. Zisserman.
- Chapter 4 of "Autonomous Mobile Robots", by R. Siegwart, I.R. Nourbakhsh, D. Scaramuzza. PDF
Spring 2015 - Autonomous Mobile Robots
The course is currently open to all the students of the University of Zurich and ETH (Bachelor's and Master's). Lectures take place every Monday (from 16.02.2014 to 30.05.2014) from 14:15 to 16:00 in the ETH main building (HG) in room E 1.2. Exercise take place almost every second Tuesday from 10:15 to 12:00 in the ETH main building in room G1.
The course is also given as an MOOC (Massive Open Online Course) under edX.
Course Program
Archived slides, videos, and lecture recordings can be found here.
Recommended Textbook
Introduction to autonomous mobile robots 2nd Edition (hardback)
A Bradford Book, The MIT Press, ISBN: 978-0-262-01535-6, February, 2011
The book can be bought during the first lecture or on Amazon.