Student Projects
How to apply
To apply, please send your CV, your Ms and Bs transcripts by email to all the contacts indicated below the project description. Do not apply on SiROP . Since Prof. Davide Scaramuzza is affiliated with ETH, there is no organizational overhead for ETH students. Custom projects are occasionally available. If you would like to do a project with us but could not find an advertized project that suits you, please contact Prof. Davide Scaramuzza directly to ask for a tailored project (sdavide at ifi.uzh.ch).
Upon successful completion of a project in our lab, students may also have the opportunity to get an internship at one of our numerous industrial and academic partners worldwide (e.g., NASA/JPL, University of Pennsylvania, UCLA, MIT, Stanford, ...).
State Estimation for Drone Racing - Available
Description: In drone racing, human pilots navigate quadrotor drones as fast as possible through a sequence of gates. To achieve the performances of world-class pilots with an autonomous system, the main challenge is to estimate the drone’s state from onboard sensors accurately. This project aims to implement a state estimation pipeline for autonomous racing on real hardware.
Goal: This project has three goals: First, extend an existing communication protocol (ROS) with sensor readings from the low-level flight controller (Betaflight). Second, integrate the sensor readings in an existing state estimation pipeline and validate its performance against ground truth data (real-world motion capture). Third, demonstrate state estimation for closed-loop control using an available flight stack.
Contact Details: Please send your CV and transcripts (bachelor and master) to Christian Pfeiffer (cpfeiffe AT ifi DOT uzh DOT ch) and Giovanni Cioffi (cioffi AT ifi DOT uzh DOT ch).
Thesis Type: Semester Project
Developing a Custom Toolbox for Eye Tracking in Drone Racing Pilots - Available
Description: Are you interested in human-machine interaction research? Do you want to contribute to the development of assistive technology for drone pilots? If so, then this student project is perfect for you! State-of-the-art eyetracker software lacks several features, including automatic calibration of the eye tracker, computation and visualization of data quality metrics, feature extraction, parallel processing, and real-time data streaming to ROS software. As a student, you will work on extending the pupil-labs codebase, a well-established open-source eyetracker software. You will develop a custom toolbox that solves the challenges mentioned above using Python and C++ programming languages. Additionally, you will compare the performances of your toolbox with the state-of-the-art eyetracker software. You will work with our research team and gain valuable experience in conducting research in the area of human-machine interaction.
Goal: Your primary responsibilities will include: (i) Implementing the custom toolbox in Python and C++ programming languages; (ii) Comparing the performances of the toolbox with state-of-the-art eyetracker software; (iii) Developing a demonstrator for real-time applications for drone racing research. **Requirements:** The ideal candidate should possess strong programming skills in Python and C++. Previous experience with ROS and eyetracking is a plus. We are looking for someone who is passionate about human-machine interaction research and willing to learn new skills. This project will provide you with a unique opportunity to gain hands-on experience in eyetracking technology and develop a toolbox that will make a significant contribution to drone racing research. You will also have the chance to work in a collaborative and supportive environment, learn from experienced researchers, and enhance your programming skills. If you are interested in this project and meet the requirements, please submit your application today. We look forward to hearing from you!
Contact Details: Please send your CV and transcripts (bachelor and master) to Christian Pfeiffer (cpfeiffe AT ifi DOT uzh DOT ch) and Manasi Muglikar (muglikar AT ifi DOT uzh DOT ch).
Thesis Type: Semester Project / Master Thesis
A Deep Learning Investigation of Eye Movements and Drone State Estimation in Racing - Available
Description: Drones have become increasingly popular in recent years, and drone racing has become a popular sport. However, little is known about how drone pilots use their eyes to extract relevant visual information from the video stream in order to control the drone. This project aims to investigate the relationship between eye gaze, optical flow, and drone state estimation and piloting behavior using statistical modeling and deep learning techniques. The successful student will gain experience in statistical modeling, machine learning, and data visualization, and will have the opportunity to make a significant contribution to the field of drone racing.
Goal: The goal of this project is to investigate how eye movements affect drone state estimation and piloting behavior by analyzing a large dataset of eyetracking and optical flow data from human pilots in a drone race. The student will use statistical methods, such as general linear mixed models, machine learning techniques, including LSTM and deep learning, and data visualization techniques to clarify the relationship between optical flow, eye gaze, and piloting behavior in various drone racing maneuvers. **Requirements:** The successful student will possess strong programming skills in Python and have a background in machine learning and statistics. Previous experience with optical flow and eyetracking is a plus, but not required. Additionally, the student should be able to work independently and have strong communication skills.
Contact Details: Please send your CV and transcripts (bachelor and master) to Christian Pfeiffer (cpfeiffe AT ifi DOT uzh DOT ch) and Mathias Gehrig (mgehrig AT ifi DOT uzh DOT ch).
Thesis Type: Semester Project / Master Thesis
Learning features for efficient deep reinforcement learning - Available
Description: The study of end-to-end deep learning in computer vision has mainly focused on developing useful object representations for image classification, object detection, or semantic segmentation. Recent work has shown that it is possible to learn temporally and geometrically aligned keypoints given only videos, and the object keypoints learned via unsupervised learning manners can be useful for efficient control and reinforcement learning.
Goal: The goal of this project is to find out if it is possible to learn useful features or intermediate representation s for controlling mobile robots in high-speed. For example, can we use the Transporter (a neural network architecture) for finding useful features in an autonomous car racing environment? if so, can we use these features for discovering an optimal control policy via deep reinforcement learning? **Required skills:** Python/C++ reinforcement learning, and deep learning skills.
Contact Details: Yunlong Song (song@ifi.uzh.ch)
Thesis Type: Semester Project / Master Thesis
Multi-agent Drone Racing via Self-play and Reinforcement Learning - Available
Description: Drone racing requires human pilots to not only complete a given race track in minimum-time, but also to compete with other pilots through strategic blocking, or to overtake opponents during extreme maneuvers. Single-player RL allows autonomous agents to achieve near-time-optimal performance in time trial racing. While being highly competitive in this setting, such training strategy can not generalize to the multi-agent scenario. An important step towards artificial general intelligence (AGI) is versatility -- the capability of discovering novel skills via self-play and self-supervised autocurriculum. In this project, we tackle multi-agent drone racing via self-play and reinforcement learning.
Goal: Create a multi-agent drone racing system that can discover novel racing skills and compete against each other. Applicants should have strong experience in C++ and python programming. Reinforcement learning and robotics background are required.
Contact Details: Yunlong Song (song (at) ifi (dot) uzh (dot) ch), Drew Hanover (hanover (at) ifi (dot) uzh (dot) ch).
Thesis Type: Master Thesis
Vision-based Dynamic Obstacle Avoidance - Available
Description: Dynamic obstacle avoidance is a grand challenge in vision-based drone navigation. The classical mapping-planning-control pipeline might have difficulties when facing dynamic objects. Learning-based systems, such as end-to-end neural network policies, are gaining popularity in robotics for dynamic objects, due to their powerful performance and versatility in handling high-dimensional state representations. Particularly, deep reinforcement learning allows for optimizing neural network policies via trial-and-error, forgoing the need for demonstrations.
Goal: The goal is to develop an autonomous vision-based navigation system that can avoid dynamic obstacles using deep reinforcement learning. Applicants should have strong experience in C++ and python programming. Reinforcement learning and robotics background are required.
Contact Details: Yunlong Song (song (at) ifi (dot) uzh (dot) ch)
Thesis Type: Semester Project / Master Thesis
Bayesian Optimization for Racing Aerial Vehicle MPC Tuning - Available
Description: In recent years, model predictive control, one of the most popular methods for controlling constrained systems, has benefitted from the advancements of learning methods. Many applications showed the potential of the cross fertilization between the two fields, i.e., autonomous drone racing, autonomous car racing, etc. Most of the research efforts have been dedicated to learn and improve the model dynamics, however, the controller tuning, which has a crucial importance, have not been studied much.
Goal: The objective of this project is to implement an auto-tuning learning-based algorithm for Model Predictive Contouring Control in a racing drone setting. The controller will learn how to tune the controller weights by using specialized Bayesian optimization algorithms [1] that can explore the large dimensional controller parameters space. The learning algorithm will be first tested in simulation and then validated with hardware experiments on a racing aerial vehicle. Your project would include: - A literature research on the current state of the art about Bayesian optimization [1] for controller tuning and on MPCC literature [2] - Implementation of the state-of-the-art algorithms identified in the previous point - Development of a tailored automatic controller parameters adaptation based on Bayesian optimization - Simulation of the developed algorithms on a racing drone - Test of the algorithms on a real racing drone The thesis will be in collaboration between UZH Robotics and Perception group and ETH IDSC Intelligent Control Systems group. [1] Fröhlich, Lukas P., Melanie N. Zeilinger, and Edgar D. Klenske. "Cautious Bayesian optimization for efficient and scalable policy search." Learning for Dynamics and Control. PMLR, 2021. [2] A. Romero, S. Sun, P. Foehn, and D. Scaramuzza, “Model predictive contouring control for time-optimal quadrotor flight,” IEEE Trans. Robot., doi: 10.1109/TRO.2022.3173711.
Contact Details: Angel Romero Aguilar roagui@ifi.uzh.ch, Andrea Carron, carrona@ethz.ch, Kim Wabersich wkim@ethz.ch
Thesis Type: Master Thesis
Adversarial Robustness in Event-Based Neural Networks - Available
Description: The robustness and reliability of neural networks are of utmost importance in several computer vision applications, especially in automotive applications where real-time predictions are crucial for safe and efficient operation. In this context, event-based cameras, due to their unique property of capturing changes in the scene, have shown impressive performance in low-latency prediction tasks such as object detection, tracking, and optical flow prediction. However, in order to be widely adopted in the real world, the robustness and reliability of such event-based networks have to be properly studied and verified. Until now, however, these aspects have been overlooked in the event-based literature. We look for students with strong programming (Pyhton/Matlab) and computer vision background. Additionally, knowledge in machine learning frameworks (pytorch, tensorflow) is required.
Goal: The project will focus on studying various neural network architectures for event-based inference datasets and evaluate their performance in the presence of adversarial attacks.
Contact Details: Marco Cannici (cannici AT ifi DOT uzh DOT ch)
Thesis Type: Semester Project / Master Thesis
Generating High-Speed Video with Event Cameras - Available
Description: Event cameras have shown amazing capabilities in slowing down video as was shown in our previous work, TimeLens (https://www.youtube.com/watch?v=dVLyia-ezvo). This is because, compared to standard cameras, event cameras only capture a highly compressed representation of the visual signal, and do this with high dynamic range and very low latency. It is this signal that can be decoded into intermediate frames. In this project we want to push the limits of what is possible using such a method and explore new extensions.
Goal: In this project we want to explore new extensions of video frame interpolation using an event camera.
Contact Details: Daniel Gehrig (dgehrig (at) ifi.uzh.ch), Mathias Gehrig (mgehrig (at) ifi (dot) uzh (dot) ch)
Thesis Type: Semester Project / Master Thesis
Learning an Event Camera - Available
Description: Event cameras such as the Dynamic Vision Sensor (DVS) are recent sensors with a lot of potential for high-speed and high dynamic range robotic applications. They have been successfully applied in many applications, such as high speed video and high speed visual odometry. In spite of this success, the exact operating principle of event cameras, that is, how events are generated from a given visual signal and how noise is generated, is not well understood. In his work we want to explore new techniques for modelling the generation of events in an event camera, which would have wide implications for existing techniques. Applicants should have a background in C++ programming and low-level vision. In addition, familiarity with learning frameworks such as pytorch or tensorflow are required.
Goal: The goal of this project is to explore new techniques for modelling an event camera.
Contact Details: Daniel Gehrig (dgehrig (at) ifi (dot) uzh (dot) ch), Mathias Gehrig (mgehrig (at) ifi (dot) uzh (dot) ch)
Thesis Type: Semester Project / Internship / Master Thesis
Asynchronous Processing for Event-based Deep Learning - Available
Description: Event cameras such as the Dynamic Vision Sensor (DVS) are recent sensors with large potential for high-speed and high dynamic range robotic applications. Since their output is sparse traditional algorithms, which are designed for dense inputs such as frames, are not well suited. The goal of this project is explore ways to adapt existing deep learning algorithms to handle sparse asynchronous data from events. Applicants should have experience in C++ and python deep learning frameworks (tensorflow or pytorch), and have a strong background in computer vision.
Goal: The goal of this project is explore ways to adapt existing deep learning algorithms to handle sparse asynchronous data from events.
Contact Details: Daniel Gehrig (dgehrig at ifi.uzh.ch)
Thesis Type: Semester Project / Master Thesis
Data-driven Event Generation from Images - Available
Description: Many frame-based datasets exist with labels created by tedious manual labor. In contrast, only a few event-based datasets are available since event cameras are relatively new sensors. Thus, in the scope of this project, we want to leverage frame-based datasets by generating synthetic events based on frames. Compared to existing work, we want to generate synthetic events in a data-driven manner to reduce the simulation-to-reality gap.
Goal: In this project, the student applies current state-of-the-art deep learning concepts for image generation to create artificial events from standard frames. Since multiple state-of-the-art deep learning methods will be explored in the scope of this project, a good background in deep learning is required. If you are interested, we are happy to provide more details.
Contact Details: Nico Messikommer [nmessi (at) ifi (dot) uzh (dot) ch], Mathias Gehrig [mgehrig (at) ifi (dot) uzh (dot) ch]
Thesis Type: Semester Project / Master Thesis
Neural-based scene reconstruction and synthesis using event cameras - Available
Description: Purely learning-based methods leveraging implicit scene representations have shown impressive results in the reconstruction and synthesis of complex scenes from just a few images, largely surpassing those of traditional methods such as Structure-from-motion, photogrammetry, and image-based rendering. Due to their recent introduction, their advantages over traditional methods are still being explored in the field of computer vision. In particular, their use in conjunction with event-based cameras, bio-inspired sensors with improved latency, temporal resolution, and dynamic range, is still under-explored.
Goal: The project will focus on exploring the use of event-based cameras in neural-based scene reconstruction and synthesis, extending available approaches to event-based data. We look for students with strong programming (Pyhton/Matlab) and computer vision background. Additionally, knowledge of machine learning frameworks (pytorch, tensorflow) is required.
Contact Details: Marco Cannici (cannici AT ifi DOT uzh DOT ch)
Thesis Type: Semester Project / Master Thesis
Data-driven Keypoint Extractor for Event Data - Available
Description: Neuromorphic cameras exhibit several amazing properties such as robustness to HDR scenes, high-temporal resolution, and low power consumption. Thanks to these characteristics, event cameras are applied for camera pose estimation for fast motions in challenging scenes. A common technique for camera pose estimation is the extraction and tracking of keypoints on the camera plane. In the case of event cameras, most existing keypoint extraction methods are handcrafted manually. As a new promising direction, this project tackles the keypoint extraction in a data-driven fashion based on recent advances in frame-based keypoint extractors.
Goal: The project aims to develop a data-driven keypoint extractor, which computes interest points in event data. Based on a previous student project (submitted to CVPR23), the approach will leverage neural network architectures to extract and describe keypoints in an event stream. The student should have prior programming experience in a deep learning framework and completed at least one course in computer vision.
Contact Details: Nico Messikommer [nmessi (at) ifi (dot) uzh (dot) ch], Mathias Gehrig [mgehrig (at) ifi (dot) uzh (dot) ch]
Thesis Type: Semester Project / Master Thesis
Learned Low-Level Controller - Available
Description: Typical control pipelines of drones consist of a high- and a low-level controller, where the outer loop sends high-level commands such as desired velocities (VEL). Alternatively, the outer controller can send collective thrust and body rate (CTBR) commands to the low-level controller. The latter then computes the motor commands based on the current state of the drone and the reference signal provided by the outer loop. It is well-known that collective thrust and bodyrate commands are more suitable for agile flight. In this project we investigate whether the advantage the CTBR control strategy can be offset using a learned low-level controller which takes velocity commands as an input. Requirements: Machine learning experience (TensorFlow and/or PyTorch), Programming experience in C++ and Python
Goal: Develop and deploy (simulation and, optionally, real world) a neural network controller that controls the drone using only linear-velocity commands as an input. This controller should be suitable for agile flight.
Contact Details: Leonard Bauersfeld (bauersfeld AT ifi DOT uzh DOT ch), Drew Hanover ( hanover (at) ifi (dot) uzh (dot) ch)
Thesis Type: Master Thesis
End-to-End Learned Vision-Based Navigation - Available
Description: Humans can pilot drones at speeds over 15 m/s through narrow racecourses while only relying on onboard vision. Although humans get better over time, a skilled human pilot will be able to fly through a new course the first time he sees it. For a drone to be able to do the same thing, It a) needs to identify a gate autonomously b) fly through the detected gate. For a) existing approaches can be used that reliably detect a gate. Therefore, the focus of this project is to accomplish item b) using a neural network that operates on the gate detections as an input. The network should not need to be trained on a specific track but rather generalize to new, unseen track-layouts just like the human counterparts. Requirements: Machine learning experience (TensorFlow and/or PyTorch), Programming experience in C++ and Python
Goal: Develop and deploy (simulation and, optionally, real world) a neural network controller that flies a drone through a sequence of drone-racing gates.
Contact Details: Leonard Bauersfeld (bauersfeld AT ifi DOT uzh DOT ch), Yunlong Song (song (at) ifi (dot) uzh (dot) ch)
Thesis Type: Master Thesis
Localization techniques for drone racing - Available
Description: For fast and agile flight, most approaches require precise knowledge of the metric state. In contrast to a classical SLAM setting, drone racing offers additional features. In this project, we want to evaluate and compare different strategies for localization in this drone-racing scenario. The following classes of methods could be investigated: - classic feature-based SLAM - learned features with classic SLAM pipeline - learning-based localization - filtering based approaches Requirements: - Machine learning experience (TensorFlow and/or PyTorch) - Programming experience in C++ and Python
Goal: The goal of the project is to gain a detailed understanding of which method is best suited for - real-time localization and - offline postprocessing of the data.
Contact Details: Leonard Bauersfeld (bauersfeld AT ifi DOT uzh DOT ch), Giovanni Cioffi ( cioffi (at) ifi (dot) uzh (dot) ch)
Thesis Type: Master Thesis
Computational Photography and Videography - Available
Description: Computational Photography is a hot topic in computer vision because it finds widespread applications in mobile devices. Traditionally, the problem has been studied using frames from a single camera. Today, mobile devices feature multiple cameras and sensors that can be combined to push the frontier in computational photography and videography. In previous work (https://youtu.be/eomALySSGVU), we have successfully reconstructed high-speed, HDR video from events. In this project, we aim for combining information from a standard and event camera to exploit their complementary nature. Applications range from high-speed, HDR video to deblurring and beyond. Contact us for more details.
Contact Details: Mathias Gehrig (mgehrig at ifi.uzh.ch); Daniel Gehrig (dgehrig at ifi.uzh.ch)
Thesis Type: Master Thesis
Deep learning based motion estimation from events - Available
Description: Optical flow estimation is the mainstay of dynamic scene understanding in robotics and computer vision. It finds application in SLAM, dynamic obstacle detection, computational photography, and beyond. However, extracting the optical flow from frames is hard due to the discrete nature of frame-based acquisition. Instead, events from an event camera indirectly provide information about optical flow in continuous time. Hence, the intuition is that event cameras are the ideal sensors for optical flow estimation. In this project, you will dig deep into optical flow estimation from events. We will make use of recent innovations in neural network architectures and insights of event camera models to push the state-of-the-art in the field. Contact us for more details.
Goal: The goal of this project is to develop a deep learning based method for dense optical flow estimation from events. Strong background in computer vision and machine learning required.
Contact Details: Mathias Gehrig, mgehrig (at) ifi (dot) uzh (dot) ch
Thesis Type: Semester Project / Master Thesis
3D reconstruction with event cameras - Available
Description: Event cameras are bio-inspired sensors that offer several advantages, such as low latency, high-speed and high dynamic range, to tackle challenging scenarios in computer vision. Research on structure from motion and multi-view stereo with images has produced many compelling results, in particular accurate camera tracking and sparse reconstruction. Active sensors with standard cameras like Kinect have been used for dense scene reconstructions. Accurate and efficient reconstructions using event-camera setups is still an unexplored topic. This project will focus on solving the problem of 3D reconstruction using active perception with event cameras .
Goal: The goal is to develop a system for accurate mapping of complex and arbitrary scenes using depth acquired by an event camera setup. We seek a highly motivated student with the following minimum qualifications: - Excellent coding skills in Python and C++ - At least one course in computer vision (multiple view geometry) - Strong work ethic - Excellent communication and teamwork skills Preferred qualifications: - Experience with machine learning Contact for more details.
Contact Details: Manasi Muglikar, muglikar (at) ifi (dot) uzh (dot) ch
Thesis Type: Semester Project / Master Thesis
Event-based depth estimation - Available
Description: Depth estimation plays an important role in many computer vision and robotics applications, such as augmented reality, navigation, or industrial inspection. Structured light (SL) systems estimate depth by actively projecting a known pattern on the scene and observing with a camera how light interacts (i.e., deforms and reflects) with the surfaces of the objects. This project will focus on event-based depth estimation using structured light systems. The resulting approach would make structured light systems suitable for generating high-speed scans.
Goal: The goal is to develop a system for 3D depth maps with event cameras. Preferred candidate should have knowledge of computer vision and strong programming skills in Python, C++.
Contact Details: Manasi Muglikar, muglikar (at) ifi (dot) uzh (dot) ch
Thesis Type: Semester Project / Master Thesis
Learning to calibrate an event camera - Available
Description: Camera calibration is an important prerequisite for 3D computer vision tasks. Calibration techniques currently used for event cameras require a special calibration target with blinking pattern. This project focuses on developing a toolkit to calibrate an event camera using deep learning methods. The project will build on state of the art deep learning techniques for events and evaluate on camera calibration task.
Goal: The goal of this project is to develop and evaluate deep learning tools for event camera for the task of calibration.
Contact Details: Manasi Muglikar, muglikar (at) ifi (dot) uzh (dot) ch , Mathias Gehrig, mgehrig (at) ifi (dot) uzh (dot) ch
Thesis Type: Semester Project / Bachelor Thesis / Master Thesis
Model-based Reinforcement Learning for Autonomous Drone Racing - Available
Goal: The objective of this project is to modify an existing model-free Reinforcement Learning pipeline for drones to a model-based Reinforcement Learning pipeline. The goal is to investigate potential performance improvements of the reinforcement learning algorithm by incorporating a model of the drone's dynamics, which will allow the algorithm to make more informed decisions. This will result in faster learning and better generalization, leading to better performance in real-world scenarios. To accomplish this goal, the student will need to research and implement various model-based reinforcement learning algorithms and evaluate their performance in a simulation environment for drone navigation. The student will also need to fine-tune the parameters of the algorithm to achieve optimal performance. The final product will be a pipeline that can be used to train a drone to navigate in a variety of environments with improved efficiency and accuracy. Applicants should have a strong background in both model-free and model-based reinforcement learning techniques, programming in C++ and Python, and a good understanding of nonlinear dynamic systems. Additional experience in signal processing, machine learning, as well as being comfortable operating in a hands-on environment is highly desired.
Contact Details: Please send your CV and transcripts (bachelor and master), and any projects you have worked on that you find interesting to Angel Romero (roagui AT ifi DOT uzh DOT ch) and Yunlong Song (song AT ifi DOT uzh DOT ch)
Thesis Type: Semester Project / Master Thesis
Top Secret (Hardware/Networking) - Available
Description: Potential students should be highly capable in C++ programming and developing in linux environments. Experience with wireless networks, python, ROS, and common machine learning libraries such as pytorch, pandas, etc is mandatory. Extra points are given to applicants with demonstrated experience building their own robotics and or machine learning projects completely from scratch in their free time (i.e. not course projects). Additional ways to stand out are demonstrated industrial experience where you made a significant impact such as cost savings, dramatically speeding up existing software, developing bespoke software architectures from scratch, or improving the design of hardware in the domains of power/mechanical efficiency, aerodynamics, etc. Expect your weekends to be occupied and come prepared to build cutting edge robotic systems which work in the real world.
Goal: Contact Drew Hanover and Giovanni Cioffi for further details
Contact Details: Drew Hanover [hanover (at) ifi (dot) uzh (dot) ch], Giovanni Cioffi [cioffi (at) ifi (dot) uzh (dot) ch]
Thesis Type: Master Thesis
Development of Off-Board Vision System Software for Autonomous Drones - Available
Goal: Design and develop the software for real-time image transmission from the drone camera to the receiver. Implement and test the software to ensure its functionality and performance. Evaluate the system's performance, specially latency, under different conditions. Demonstrate closed-loop capabilities of the platform by using an offboard Visual Odometry pipeline. The project will require the use of image processing libraries, wireless communication protocols, and programming languages such as Python and C++.
Contact Details: Please send your CV and transcripts (bachelor and master), and any projects you have worked on that you find interesting to Angel Romero (roagui AT ifi DOT uzh DOT ch) and Alex Barden (barden AT ifi DOT uzh DOT ch)
Thesis Type: Semester Project / Master Thesis
Top Secret (SLAM, VIO, Computer Vision) - Available
Description: Potential students should be very familiar with state of the art VIO and SLAM algorithms such as SVO, ORB-SLAM etc. Experience with multi-robot SLAM is highly desired. Applicants should have experience with the following tools: C++, python, linux development, machine learning libraries, ROS, Kalibr etc. Ways to stand out include running your own VIO/SLAM/computer vision projects at home. Hands-on experience with real world sensors such as the Realsense T265 or D435 would be a huge bonus. Additional ways to stand out would be industrial experience in computer vision or self driving car companies, or successful semester projects which led to publication in top robotics or computer vision conferences/journals. Expect your weekends to be occupied and come prepared to build cutting edge robotic systems which work in the real world.
Goal: Contact Drew Hanover and Giovanni Cioffi for further details
Contact Details: Drew Hanover [hanover (at) ifi (dot) uzh (dot) ch], Giovanni Cioffi [cioffi (at) ifi (dot) uzh (dot) ch]
Thesis Type: Master Thesis
Game Theoretic Planning for Navigation in Crowded Areas - Available
Description: Humans are astonishingly good at navigating in complex and crowded environments. In the train station, we manage to navigate with minimal collisions AND minimal explicit communication - but how do we do this so effectively? This project will examine learning based game theoretic methods for navigating in complex environments across a multitude of dynamic systems including aerial vehicles (drones) and legged robots.
Goal: Investigate the potential of learning based game theoretic planners for dynamic robotic systems
Contact Details: Drew Hanover [hanover (at) ifi (dot) uzh (dot) ch]
Thesis Type: Semester Project / Master Thesis
Domain Transfer between Events and Frames - Available
Description: During the last years, a vast collection of frame-based datasets was collected for countless tasks. In comparison, event-based datasets represent only a tiny fraction of the available datasets. Thus, it is highly promising to use labelled frame datasets to train event-based networks as current data-driven approaches heavily rely on labelled data.
Goal: In this project, the student extends upon a previous student project (published at ECCV22) and current advances from the UDA literature in order to transfer multiple tasks from frames to events. The approach should be validated on several tasks (segmentation, object detection, etc.) in challenging environments (night, high-dynamic scenes) to highlight the benefits of event cameras. As several deep learning methods are used as tools for the task transfer, a strong background in deep learning is required. If you are interested, we are happy to provide more details.
Contact Details: Nico Messikommer [nmessi (at) ifi (dot) uzh (dot) ch], Mathias Gehrig [mgehrig (at) ifi (dot) uzh (dot) ch]
Thesis Type: Semester Project / Master Thesis
Study on the effects of camera resolution in Visual Odometry - Available
Description: Visual Odometry (VO) algorithms have gone beyond academic research and are now widely used in the real world. Robotics and AR/VR applications, among many others, rely on VO to estimate the ego motion of the camera. Hardware and software co-design is key to develop accurate and robust algorithms. In this project, we will investigate how design choices at the hardware level affect the VO performance. In particular, we will study how the camera resolution affects the accuracy and robustness of some of the state-of-art VO pipelines. We believe that the results of this project will help academic research and companies in the hardware and software co-design of VO solutions and expand the use of VO algorithms in commercial products.
Goal: Get familiar with VO pipelines and simulation tools. Generate a high-resolution dataset including different camera motions. Benchmark some of the state-of-the-art VO pipelines on this dataset as well as real-world ones. We look for students with strong programming (C++ preferred) and computer vision (ideally have taken Prof. Scaramuzza's class) background.
Contact Details: Giovanni Cioffi (cioffi@ifi.uzh.ch), Manasi Muglikar (muglikar@ifi.uzh.ch)
Thesis Type: Master Thesis
Efficient Learning-aided Visual Inertial Odometry - Available
Description: Recent works have shown that deep learning (DL) techniques are beneficial for visual inertial odometry (VIO). Different ways to include DL in VIO have been proposed: end-to-end learning from images to poses, replacing one/more block/-s of a standard VIO pipeline with learning-based solutions, and include learning in a model-based VIO block. The project will start with a study of the current literature on learning-based VIO/SLAM algorithms and an evaluation of how/where/when DL is beneficial for VIO/SLAM. We will use the results of this evaluation to enhance a current state-of-the-art VIO pipeline with DL, focusing our attention on algorithm efficiency at inference time. The developed learning-aided VIO pipeline will be compared to existing state-of-the-art model-based algorithms, with focus on robustness, and deployed on embedded platforms (Nvidia Jetson TX2 or Xavier).
Goal: Enhance standard VIO algorithms with DL techniques to improve robustness. Benchmark the proposed algorithm against existing state-of-the-art standard VIO algorithms. Deploy the proposed algorithm on embedded platforms. We look for students with strong computer vision background and familiar with common software tools used in DL (for example, PyTorch or TensorFlow).
Contact Details: Giovanni Cioffi (cioffi@ifi.uzh.ch), Manasi Muglikar (muglikar@ifi.uzh.ch)
Thesis Type: Master Thesis
Developing Smart Vision Assistive Technology - Available
Description: More than 200 million people are estimated to have moderate or severe vision impairment in 2020. Their lack of autonomy limits the completion of many daily living activities. In this project, we will focus on applying robotics techniques, such as state estimation and path planning, to help visually impaired people to navigate unknown and unstructured environments. Image credits: Katzschmann et al. 2018.
Goal: This project aims to develop a navigation solution for visually impaired people. The student will work on state estimation, such as VIO and SLAM, and path planning algorithms from the robotic domain and make them suitable for the navigation of visually impaired people. A successful navigation system will be deployed during the 2024 edition of the Cybathlon Competition, https://cybathlon.ethz.ch/en. The project will be carried out at RPG in collaboration with the Autonomous System Lab, ETH Zurich.
Contact Details: Giovanni Cioffi (cioffi@ifi.uzh.ch)
Thesis Type: Semester Project / Master Thesis
Efficient Processing of Event Data for Deep Learning - Available
Description: Event Cameras show great potential for high-speed and low-latency computer vision and robotics applications. In recent years, we have particularly focused on designing novel data-driven approaches to maximize performance and minimize latency. Still, efficient and effective feature extraction from event data remains challenging. This project investigates novel approaches proposed in the machine-learning and signal-processing community to maximize the benefits of event cameras in time-critical scenarios. Applications of this project range include optical flow prediction, object detection, and tracking to closed-loop control with event-based vision in the loop. Contact us for more details.
Goal: This project aims to develop a new differentiable module for efficient and effective low-level event data processing. To achieve this goal, we will investigate new techniques recently proposed in the machine learning and signal processing literature. We will integrate the proposed method with state-of-the-art deep learning models to enable low-latency inference in applications ranging from optical flow prediction, object detection/tracking, and closed-loop drone navigation with vision in the loop.
Contact Details: Mathias Gehrig (mgehrig at ifi.uzh.ch)
Thesis Type: Master Thesis
Event camera-based navigation for planetary landing in future Mars and Moon missions - Available
Description: Event-based cameras have remarkable advantages in challenging robotics conditions involving high-dynamic range and very fast motion. These are exactly the conditions a spacecraft encounters during descent on celestial bodies such as Mars or the Moon, where abrupt changes in illumination and fast dynamics relative to the ground can affect vision-based navigation systems relying on standard cameras. In this work, we want to design novel spacecraft navigation methods for descent and landing phases. The project is in collaboration with European Space Agency at the European Space Research and Technology Centre (ESTEC) in Noordwijk (NL). We look for students with strong programming (Pyhton/Matlab) and computer vision background. Additionally, knowledge in machine learning frameworks (pytorch, tensorflow) is required.
Goal: Help build the next generation of high-speed event camera-based spacecraft navigation in challenging illumination conditions.
Contact Details: Marco Cannici (cannici AT ifi DOT uzh DOT ch), Daniel Gehrig (dgehrig AT ifi DOT uzh DOT ch)
Thesis Type: Semester Project