Student Projects


How to apply

To apply, please send your CV, your Ms and Bs transcripts by email to all the contacts indicated below the project description. Do not apply on SiROP . Since Prof. Davide Scaramuzza is affiliated with ETH, there is no organizational overhead for ETH students. Custom projects are occasionally available. If you would like to do a project with us but could not find an advertized project that suits you, please contact Prof. Davide Scaramuzza directly to ask for a tailored project (sdavide at ifi.uzh.ch).


Upon successful completion of a project in our lab, students may also have the opportunity to get an internship at one of our numerous industrial and academic partners worldwide (e.g., NASA/JPL, University of Pennsylvania, UCLA, MIT, Stanford, ...).



3D reconstruction with event cameras - Available

Description: Event cameras are bio-inspired sensors that offer several advantages, such as low latency, high-speed and high dynamic range, to tackle challenging scenarios in computer vision. Research on structure from motion and multi-view stereo with images has produced many compelling results, in particular accurate camera tracking and sparse reconstruction. Active sensors with standard cameras like Kinect have been used for dense scene reconstructions. Accurate and efficient reconstructions using event-camera setups is still an unexplored topic. This project will focus on solving the problem of 3D reconstruction using active perception with event cameras​ .

Goal: The goal is to develop a system for accurate mapping of complex and arbitrary scenes using depth acquired by an event camera setup. We seek a highly motivated student with the following minimum qualifications: - Excellent coding skills in Python and C++ - At least one course in computer vision (multiple view geometry) - Strong work ethic - Excellent communication and teamwork skills Preferred qualifications: - Experience with machine learning Contact for more details.

Contact Details: Manasi Muglikar, muglikar (at) ifi (dot) uzh (dot) ch

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Event-based depth estimation​ - Available

Description: Depth estimation plays an important role in many computer vision and robotics applications, such as augmented reality, navigation, or industrial inspection. Structured light (SL) systems estimate depth by actively projecting a known pattern on the scene and observing with a camera how light interacts (i.e., deforms and reflects) with the surfaces of the objects. This project will focus on event-based depth estimation using structured light systems. The resulting approach would make structured light systems suitable for generating high-speed scans.

Goal: The goal is to develop a system for 3D depth maps with event cameras. Preferred candidate should have knowledge of computer vision and strong programming skills in Python, C++.

Contact Details: Manasi Muglikar, muglikar (at) ifi (dot) uzh (dot) ch

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Visual Odometry with new Unprecedented Event Camera - Available

Goal: In this project, you will explore and implement new algorithms for visual odometry (VO) with a new prototype event camera with unprecedented performance. This new and unexplored sensor has a high potential to improve upon existing works in the field of VO. Together with us, you will also collaborate closely with our industry partner. You should have prior programming experience and completed at least one course in computer vision.

Contact Details: Nico Messikommer [nmessi (at) ifi (dot) uzh (dot) ch], Daniel Gehrig [dgehrig (at) ifi (dot) uzh (dot) ch]

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Robust state estimation and visual tracking - Available

Description: Visual Odometry (VO) and Simultaneous Localization and Mapping (SLAM) have gained a lots attention from industry in recent years. Several obstacles, such as robustness, accuracy, etc., are still in the way when the VO, Visual SLAM (V-SLAM) technologies are applied in real industry applications. Joint with an industry company, we investigate the robustness of an on-line tracking approach on body-worn devices with fast movements such as running, jumping, roping and turns. The final application is supposed to be run with multiple maps and under dynamic lighting conditions ranging from direct sun-light to almost no-light, e.g. in a dark room. Requirements: Understanding of VIO and SLAM, Computer Vision and Programming experience in C++.

Goal: Development and performance evaluation of a VIO / V-SLAM algorithm for state estimation and tracking. The obstacles and difficulties in challenging scenarios as fast movements and tracking in dynamic lighting conditions will be identified and solutions will be proposed and evaluated accordingly.

Contact Details: Giovanni Cioffi (cioffi@ifi.uzh.ch)

Thesis Type: Master Thesis

See project on SiROP

Event-based Feature Tracking on an Embedded Platform - Available

Description: Event cameras such as the Dynamic Vision Sensor (DVS) are recent sensors with large potential for high-speed and high dynamic range robotic applications, such as fast obstacle avoidance. In particular, event cameras can be used to track features or objects in the blind time between two frames which makes it possible to react quickly to changes in the scene.

Goal: In this project we want to deploy an event-based feature tracking algorithm on a resource constrained platform such as a drone. Applicants should have a strong background in C++ programming and low-level vision. Experience with embedded programming is a plus.

Contact Details: Daniel Gehrig (dgehrig (at) ifi.uzh.ch), Elia Kaufmann (ekaufmann (at) ifi (dot) uzh (dot) ch), Mathias Gehrig (mgehrig (at) ifi (dot) uzh (dot) ch)

Thesis Type: Semester Project

See project on SiROP

Building and controlling a negative thrust quadcopter - Available

Description: Quadcopter platforms have been gaining popularity in recent years due to their maneuverability and uncomplicated design. Recent advances in hardware components, such as motors and electronic speed controllers (ESCs) unlock different possible extensions of the classical quadcopter design in order to gain new capabilities. In this project, we aim to modify the current design of our platform to build a quadcopter that is able to generate thrust in both positive and negative directions by changing the rotation direction of the motors on the fly. This will provide the quadcopter with new ways of performing complex maneuvers in ways unseen until now.

Goal: The student will first modify our current drone design to include motors and ESCs that support changes in directions. Then, the student will design a control architecture, based on the existing ones, that take into account the new input space. As a proof of concept, the student would test this new drone design by performing highly agile maneuvers and compare them with state-of-the-art platforms and algorithms.

Contact Details: Please send your CV and transcripts (bachelor and master) to Angel Romero (roagui AT ifi DOT uzh DOT ch) and Leonard Bauersfeld (bauersfeld AT ifi DOT uzh DOT ch)

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Designing an Event Camera for Learning - Available

Description: Event cameras such as the Dynamic Vision Sensor (DVS) are recent sensors with a lot of potential for high-speed and high-dynamic-range robotic applications.They have been successfully applied in many applications, such as high speed video and high speed visual odometry. Recently, many new event cameras have been commercialized with higher and higher spatial resolutions and high temporal resolution. However, these developments steadily increase the the computational requirements for downstream algorithms, increasing the necessary bandwidth and reducing the time available to process events. In this work we want to find out how important these design parameters are for deep learning applications. Applicants should have experience in coding image processing algorithms in C++ and experience with learning frameworks in python such as tensorflow or pytorch.

Goal: The goal of this project is to find out how important the design parameters of event cameras, such as spatial and temporal resolution, are for deep learning applications.

Contact Details: Daniel Gehrig (dgehrig (at) ifi (dot) uzh (dot) ch), Antonio Loquercio (antonilo (at) ifi (dot) uzh (dot) ch)

Thesis Type: Semester Project / Internship / Master Thesis

See project on SiROP

Flying Drones on Mars: How Gravity Affects Self-Motion Estimation in Human Drone Pilots - Available

Description: Drones offer enormous potential for Mars exploration in search of habitable environments above and underground. Human-in-the-loop drone operations will provide remote sensing and delivery capabilities beyond the state-of-the-art of fully autonomous systems. However, it is currently unknown how altered gravity conditions (such as microgravity) affect the vision-based self-motion estimation of drone operators under conditions of uncertainty. In this project, we experimentally investigate how gravity affects decision-making in human subjects. The student will develop an experimental paradigm to test the effects of gravity of self-motion estimation using Unity3D and Python, collect behavioral responses and flight data from human drone pilots, and statistically analyze these data. Requirements: Programming experience in Python/Matlab, Unity3D, a background in statistical analysis/data science methods, and a strong interest in human subjects research and human-machine interaction.

Goal: The goal is to understand how gravity affects vision-based self-motion perception in human drone pilots.

Contact Details: Please send your CV and transcripts (bachelor and master) to Christian Pfeiffer (cpfeiffe AT ifi DOT uzh DOT ch) and Manasi Muglikar (muglikar AT ifi DOT uzh DOT ch).

Thesis Type: Semester Project / Internship / Master Thesis

See project on SiROP

Neural Network Representation for Vision-Based MPC Control - Available

Description: Model predictive control (MPC) is a versatile optimization-based control method that allows the incorporation of constraints directly into the control problem. The advantages of MPC can be seen in its ability to accurately control dynamic systems that include large time delays and high-order dynamics. Recent advances in compute hardware allow running MPC even on compute-constrained quadrotors. While model-predictive control can deal with complex systems and constraints, it still assumes the existence of a reference trajectory. With this project, we aim to guide the MPC to a feasible reference trajectory by using a neural network that directly predicts from camera images an expressive intermediate representation. Such tight coupling of perception and control would allow pushing the speed limits of autonomous flight through cluttered environments. Requirements: Machine learning experience (TensorFlow and/or PyTorch), Experience in MPC preferable but not strictly required, Programming experience in C++ and Python

Goal: Evaluate different intermediate representations for autonomous flight. Implement the learned perception system in simulation and integrate the predictions into an existing MPC pipeline. If possible, deploy on a real system.

Contact Details: Leonard Bauersfeld (bauersfeld AT ifi DOT uzh DOT ch), Elia Kaufmann (ekaufmann (at) ifi (dot) uzh (dot) ch)

Thesis Type: Master Thesis

See project on SiROP

Learned Low-Level Controller - Available

Description: Typical control pipelines of drones consist of a high- and a low-level controller, where the outer loop sends high-level commands such as desired velocities (VEL). Alternatively, the outer controller can send collective thrust and body rate (CTBR) commands to the low-level controller. The latter then computes the motor commands based on the current state of the drone and the reference signal provided by the outer loop. It is well-known that collective thrust and bodyrate commands are more suitable for agile flight. In this project we investigate whether the advantage the CTBR control strategy can be offset using a learned low-level controller which takes velocity commands as an input. Requirements: Machine learning experience (TensorFlow and/or PyTorch), Programming experience in C++ and Python

Goal: Develop and deploy (simulation and, optionally, real world) a neural network controller that controls the drone using only linear-velocity commands as an input. This controller should be suitable for agile flight.

Contact Details: Leonard Bauersfeld (bauersfeld AT ifi DOT uzh DOT ch), Elia Kaufmann (ekaufmann (at) ifi (dot) uzh (dot) ch)

Thesis Type: Master Thesis

See project on SiROP

End-to-End Learned Vision-Based Navigation - Available

Description: Humans can pilot drones at speeds over 15 m/s through narrow racecourses while only relying on onboard vision. Although humans get better over time, a skilled human pilot will be able to fly through a new course the first time he sees it. For a drone to be able to do the same thing, It a) needs to identify a gate autonomously b) fly through the detected gate. For a) existing approaches can be used that reliably detect a gate. Therefore, the focus of this project is to accomplish item b) using a neural network that operates on the gate detections as an input. The network should not need to be trained on a specific track but rather generalize to new, unseen track-layouts just like the human counterparts. Requirements: Machine learning experience (TensorFlow and/or PyTorch), Programming experience in C++ and Python

Goal: Develop and deploy (simulation and, optionally, real world) a neural network controller that flies a drone through a sequence of drone-racing gates.

Contact Details: Leonard Bauersfeld (bauersfeld AT ifi DOT uzh DOT ch), Elia Kaufmann (ekaufmann (at) ifi (dot) uzh (dot) ch)

Thesis Type: Master Thesis

See project on SiROP

Asynchronous Processing for Event-based Deep Learning - Available

Description: Event cameras such as the Dynamic Vision Sensor (DVS) are recent sensors with large potential for high-speed and high dynamic range robotic applications. Since their output is sparse traditional algorithms, which are designed for dense inputs such as frames, are not well suited. The goal of this project is explore ways to adapt existing deep learning algorithms to handle sparse asynchronous data from events. Applicants should have experience in C++ and python deep learning frameworks (tensorflow or pytorch), and have a strong background in computer vision.

Goal: The goal of this project is explore ways to adapt existing deep learning algorithms to handle sparse asynchronous data from events.

Contact Details: Daniel Gehrig (dgehrig at ifi.uzh.ch)

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Study on the effects of camera resolution in Visual Odometry - Available

Description: Visual Odometry (VO) algorithms have gone beyond academic research and are now widely used in the real world. Robotics and AR/VR applications, among many others, rely on VO to estimate the ego motion of the camera. Hardware and software co-design is key to develop accurate and robust algorithms. In this project, we will investigate how design choices at the hardware level affect the VO performance. In particular, we will study how the camera resolution affects the accuracy and robustness of some of the state-of-art VO pipelines. We believe that the results of this project will help academic research and companies in the hardware and software co-design of VO solutions and expand the use of VO algorithms in commercial products.

Goal: Get familiar with VO pipelines and simulation tools. Generate a high-resolution dataset including different camera motions. Benchmark some of the state-of-the-art VO pipelines on this dataset as well as real-world ones. We look for students with strong programming (C++ preferred) and computer vision (ideally have taken Prof. Scaramuzza's class) background.

Contact Details: Giovanni Cioffi (cioffi@ifi.uzh.ch)

Thesis Type: Master Thesis

See project on SiROP

Tracking with Spiking Neural Networks and Event Cameras - Available

Description: This project aims at developing a camera tracking approach with sparse input (events from event cameras) and sparse computation (spiking neural networks). Conventional approaches are built on visual inertial odometry using image and imu data. Ideally, this approach would process image data at high frequency for maximum accuracy. However, this is not attainable on resource constraint devices such as mobile phones or wearables. Event data, in combination with spiking neural networks, can overcome this trade-off by leveraging sparse computation by design. To achieve this goal, we will investigate ego-motion tracking for rotational motion and subsequently investigate 6DoF ego-motion tracking. This project will be done in collaboration with Synsense ( https://www.synsense-neuromorphic.com ) and benefit from their experience as well as our own prior work in this research space.

Contact Details: Mathias Gehrig, mgehrig (at) ifi.uzh.ch

Thesis Type: Master Thesis

See project on SiROP

Computational Photography and Videography - Available

Description: Computational Photography is a hot topic in computer vision because it finds widespread applications in mobile devices. Traditionally, the problem has been studied using frames from a single camera. Today, mobile devices feature multiple cameras and sensors that can be combined to push the frontier in computational photography and videography. In previous work (https://youtu.be/eomALySSGVU), we have successfully reconstructed high-speed, HDR video from events. In this project, we aim for combining information from a standard and event camera to exploit their complementary nature. Applications range from high-speed, HDR video to deblurring and beyond. Contact us for more details.

Contact Details: Mathias Gehrig (mgehrig at ifi.uzh.ch); Daniel Gehrig (dgehrig at ifi.uzh.ch)

Thesis Type: Master Thesis

See project on SiROP

Deep learning based motion estimation from events - Available

Description: Optical flow estimation is the mainstay of dynamic scene understanding in robotics and computer vision. It finds application in SLAM, dynamic obstacle detection, computational photography, and beyond. However, extracting the optical flow from frames is hard due to the discrete nature of frame-based acquisition. Instead, events from an event camera indirectly provide information about optical flow in continuous time. Hence, the intuition is that event cameras are the ideal sensors for optical flow estimation. In this project, you will dig deep into optical flow estimation from events. We will make use of recent innovations in neural network architectures and insights of event camera models to push the state-of-the-art in the field. Contact us for more details.

Goal: The goal of this project is to develop a deep learning based method for dense optical flow estimation from events. Strong background in computer vision and machine learning required.

Contact Details: Mathias Gehrig, mgehrig (at) ifi (dot) uzh (dot) ch

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Deep Learning for Vision-Based State Estimation in Drone Racing - Available

Description: Drone pilots use eye movement to extract relevant visual information from a first-person video stream. How eye movements affect drone state estimation and piloting behaviour is poorly understood. The goal of this study is to investigate the relationship of eye gaze, optical flow, and drone state using deep learning and statistical modeling. The student will be provided with a large dataset of eyetracking and optical flow data of human pilots in a drone race. The student will use statistical methods (e.g., general linear mixed models), machine learning (e.g., LSTM, deep learning), and data visualization techniques to clarify the relationship between optical flow, eye gaze, and piloting behavior in various drone racing maneuvers. Requirements: Strong programming skills in Python or Matlab. Background in machine learning and statistics. Previous experience with optical flow and eyetracking is a plus.

Contact Details: Please send your CV and transcripts (bachelor and master) to Christian Pfeiffer (cpfeiffe AT ifi DOT uzh DOT ch) and Yunlong Song (song AT ifi DOT uzh DOT ch).

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Continuous-time Online Visual-Inertial Odometry for Fast Flights - Available

Description: The continuous time (CT) trajectory representation in visual inertial odometry (VIO) has the advantage of facilitating the fusion of the asynchronous, and potentially shifted, camera and IMU measurements. This is beneficial in the case of a hardware synchronized sensor suite is not available. CT-VIO introduces a prior encoding the smoothness of the trajectory to be estimated. This prior can help the pose estimations of fast flying drones whose trajectories are expected to be smooth. Temporal basis functions, e.g. B-splines, are the most common choice for CT-VIO / CT-SLAM. Recent works have proposed algorithms to speed up the computation of the spline derivatives. Also, other efficient spline functions, like Hermite spline, exist. In this project, we will start with studying different spline functions in terms of efficiency. We will select the best candidate and use it to develop an efficient CT-VIO algorithm which runs online on resource-constrained quadrotors.

Goal: Develop an efficient CT-VIO algorithm capable to run online on our quadrotors (target platform is the Nvidia Jetson TX2). Benchmark the proposed algorithm against existing state-of-the-art VIO algorithms. We look for students with strong computer vision and programming background (C++ preferred). This work involves a final demonstration of the proposed CT-VIO algorithm in a closed-loop controller to track fast drone trajectories.

Contact Details: Giovanni Cioffi (cioffi@ifi.uzh.ch)

Thesis Type: Master Thesis

See project on SiROP

Efficient Learning-aided Visual Inertial Odometry - Available

Description: Recent works have shown that deep learning (DL) techniques are beneficial for visual inertial odometry (VIO). Different ways to include DL in VIO have been proposed: end-to-end learning from images to poses, replacing one/more block/-s of a standard VIO pipeline with learning-based solutions, and include learning in a model-based VIO block. The project will start with a study of the current literature on learning-based VIO/SLAM algorithms and an evaluation of how/where/when DL is beneficial for VIO/SLAM. We will use the results of this evaluation to enhance a current state-of-the-art VIO pipeline with DL, focusing our attention on algorithm efficiency at inference time. The developed learning-aided VIO pipeline will be compared to existing state-of-the-art model-based algorithms, with focus on robustness, and deployed on embedded platforms (Nvidia Jetson TX2 or Xavier).

Goal: Enhance standard VIO algorithms with DL techniques to improve robustness. Benchmark the proposed algorithm against existing state-of-the-art standard VIO algorithms. Deploy the proposed algorithm on embedded platforms. We look for students with strong computer vision background and familiar with common software tools used in DL (for example, PyTorch or TensorFlow).

Contact Details: Giovanni Cioffi (cioffi@ifi.uzh.ch)

Thesis Type: Master Thesis

See project on SiROP

Vision-based Dynamic Obstacle Avoidance - Available

Description: Dynamic obstacle avoidance is a grand challenge in vision-based drone navigation. The classical mapping-planning-control pipeline might have difficulties when facing dynamic objects. Learning-based systems, such as end-to-end neural network policies, are gaining popularity in robotics for dynamic objects, due to their powerful performance and versatility in handling high-dimensional state representations. Particularly, deep reinforcement learning allows for optimizing neural network policies via trial-and-error, forgoing the need for demonstrations.

Goal: The goal is to develop an autonomous vision-based navigation system that can avoid dynamic obstacles using deep reinforcement learning. Applicants should have strong experience in C++ and python programming. Reinforcement learning and robotics background are required.

Contact Details: Yunlong Song (song (at) ifi (dot) uzh (dot) ch), Antonio Loquercio (loquercio (at) ifi (dot) uzh (dot) ch)

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Eyetracking Toolbox for Drone Racing Research - Available

Description: Eyetracking is one of the most important tools in human-machine interaction research. In our lab, we use eyetracking to study visual attention in drone racing pilots and to develop assistive technology for drone pilots. State-of-the-art eyetracker software lacks the ability to automatically calibrate the eye tracker, compute and visualize data quality metrics, extract features of interest, process data in parallel, and stream data in real-time to ROS software. The goal of this project is thus to extend the pupil-labs codebase (https://github.com/pupil-labs/pupil) with a custom toolbox that solves the aforementioned tasks. The student will implement the toolbox in Python and C++, compare the performances to state-of-the-art eyetracker software, and develop a demonstrator for real-time applications for drone racing research. Requirements: Strong programming skills in Python, C++. Previous experience with ROS, and eyetracking is a plus.

Contact Details: Please send your CV and transcripts (bachelor and master) to Christian Pfeiffer (cpfeiffe AT ifi DOT uzh DOT ch) and Manasi Muglikar (muglikar AT ifi DOT uzh DOT ch).

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Data-driven Keypoint Extractor for Event Data - Available

Description: Neuromorphic cameras exhibit several amazing properties such as robustness to HDR scenes, high-temporal resolution, and low power consumption. Thanks to these characteristics, event cameras are applied for camera pose estimation for fast motions in challenging scenes. A common technique for camera pose estimation is the extraction and tracking of keypoints on the camera plane. In the case of event cameras, most existing keypoint extraction methods are handcrafted manually. As a new promising direction, this project tackles the keypoint extraction in a data-driven fashion based on recent advances in frame-based keypoint extractors.

Goal: The project aims to develop a data-driven keypoint extractor, which computes interest points in event data. Based on the current advances of learned keypoint extractors for traditional frames, the approach will leverage neural network architectures to extract and describe keypoints in an event stream. The student should have prior programming experience in a deep learning framework and completed at least one course in computer vision.

Contact Details: Contact Details: Nico Messikommer [nmessi (at) ifi (dot) uzh (dot) ch], Mathias Gehrig [mgehrig (at) ifi (dot) uzh (dot) ch]

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Domain Transfer between Events and Frames - Available

Description: During the last years, a vast collection of frame-based datasets was collected for countless tasks. In comparison, event-based datasets represent only a tiny fraction of the available datasets. Thus, it is highly promising to use labelled frame datasets to train event-based networks as current data-driven approaches heavily rely on labelled data.

Goal: In this project, the student extends current advances from the UDA literature for traditional frames to event data in order to transfer multiple tasks from frames to events. The approach should be validated on several tasks (segmentation, object detection, etc.) in challenging environments (night, high-dynamic scenes) to highlight the benefits of event cameras. As several deep learning methods are used as tools for the task transfer, a strong background in deep learning is required. If you are interested, we are happy to provide more details.

Contact Details: Nico Messikommer [nmessi (at) ifi (dot) uzh (dot) ch], Daniel Gehrig [dgehrig (at) ifi (dot) uzh (dot) ch]

Thesis Type: Semester Project / Master Thesis

See project on SiROP