Student Projects


How to apply

To apply, please send your CV, your Ms and Bs transcripts by email to all the contacts indicated below the project description. Do not apply on SiROP . Since Prof. Davide Scaramuzza is affiliated with ETH, there is no organizational overhead for ETH students. Custom projects are occasionally available. If you would like to do a project with us but could not find an advertized project that suits you, please contact Prof. Davide Scaramuzza directly to ask for a tailored project (sdavide at ifi.uzh.ch).


Upon successful completion of a project in our lab, students may also have the opportunity to get an internship at one of our numerous industrial and academic partners worldwide (e.g., NASA/JPL, University of Pennsylvania, UCLA, MIT, Stanford, ...).



Optimization for Spiking Neural Networks - Available

Description: Spiking neural networks (SNNs) are neural networks that process information with timing of events/spikes rather than numerical values. Together with event-cameras, SNNs show promise to both lower latency and computational burden compared to artificial neural networks. In recent years, researchers have proposed several methods to estimate gradients of SNN parameters in a supervised learning context. In practice, many of these approaches rely on assumptions that might not hold in all scenarios. Requirements: - Background in machine learning; especially deep learning - Good programming skills; experience in CUDA is a plus.

Goal: In this project we explore state-of-the-art optimization methods for SNNs and their suitability to solve the temporal credit-assignment problem. As a first step, an in-depth evaluation of a selection of algorithms is required. Based on the acquired insights, the prospective student can propose improvements and implement their own method.

Contact Details: Mathias Gehrig, mgehrig (at) ifi (dot) uzh (dot) ch

Thesis Type: Master Thesis

See project on SiROP

Learning features for efficient deep reinforcement learning - Available

Description: The study of end-to-end deep learning in computer vision has mainly focused on developing useful object representations for image classification, object detection, or semantic segmentation. Recent work has shown that it is possible to learn temporally and geometrically aligned keypoints given only videos, and the object keypoints learned via unsupervised learning manners can be useful for efficient control and reinforcement learning.

Goal: The goal of this project is to find out if it is possible to learn useful features or intermediate representation s for controlling mobile robots in high-speed. For example, can we use the Transporter (a neural network architecture) for finding useful features in an autonomous car racing environment? if so, can we use these features for discovering an optimal control policy via deep reinforcement learning? **Required skills:** Python/C++ reinforcement learning, and deep learning skills.

Contact Details: Yunlong Song (song@ifi.uzh.ch) and Titus Cieslewski ( titus at ifi.uzh.ch )

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Reinforcement learning without exploration - Available

Description: Learning from finite data sets is useful for many real-world applications, where data collection may be difficult and costly. Batch reinforcement learning is the study of algorithms that can learn from a finite batch of data, without directly interacting with the environment, however, it suffers from the so called "extrapolation error". The recent study achieved great progress in off-policy reinforcement learning without exploration where the algorithm can learn from arbitrary, fixed batch data in the continuous control domain.

Goal: The goal of this project is to study state-of-the-art batch reinforcement learning algorithms and apply these algorithms to quadrotor control. Potentially, we want to combine it with an event camera. **Required skills:** Python/C++, reinforcement learning, and deep learning skills.

Contact Details: song@ifi.uzh.ch

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Building a simulator for quadrotors - Available

Description: Building a simulator that combines the photorealistic image rendering engine with the ROS framework could greatly help the robotics research community to develop algorithms with the simulator. For example, both two popular open-source simulators, CARLA and AirSim, are supporting ROS.

Goal: The goal of this project is to develop a quadrotor simulator that combines our current photorealistic image rendering engine with our existing sensor simulators, e.g., the UAV dynamics simulator and the event camera simulator, that was developed with ROS. You will first familiarize yourself with our existing simulator that already has all the basic components and is ready to run. Then, you have to figure out the most efficient way of organizing the code such that it runs fast. For example, how to retrieve images from the rendering engine and save it? You will be required to write C++ code and compile the simulator in ROS.

Contact Details: Yunlong Song (song AT ifi DOT uzh DOT ch), Attach CV and transcripts. Programming skills: C/C++ and ROS.

Thesis Type: Semester Project / Bachelor Thesis / Master Thesis

See project on SiROP

Designing a New Event Camera with Events and Images - Available

Description: Event cameras such as the Dynamic Vision Sensor (DVS) are recent sensors with a lot of potential for high-speed and high dynamic range robotic applications. They have been successfully applied in many applications, such as high speed video and high speed visual odometry. Due to their high speed and

Goal: The goal of this project is to design a new event camera that combines events and standard images.

Contact Details: Daniel Gehrig (dgehrig (at) ifi (dot) uzh (dot) ch), Mathias Gehrig (mgehrig (at) ifi (dot) uzh (dot) ch)

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Event-based Vision for Autonomous Driving - Available

Description: Billions of dollars are spent each year to bring autonomous vehicles closer to reality. One of the remaining challenges is the design of reliable algorithms that work in a diverse set of environments and scenarios. At the core of this problem is the choice of sensor setup. Ideally, there is a certain redundancy in the setup while each sensor should also excel at a certain task. Sampling based sensors (e.g. LIDAR, standard cameras, etc.) are today's essential building blocks of autonomous vehicles. However, they typically oversample far-away structure (e.g. building 200 meters away) and undersample close structure (e.g. fast bike crossing in front of the car). Thus, they enforce a trade-off between sampling frequency and computational budget. Unlike sampling-based sensors, event cameras capture change in their field-of-view with precise timing and do not record redundant information. As a result, they are well suited for highly dynamic scenarios such as driving on roads. There are also other benefits such as very high dynamic range, unmatched by standard cameras.

Goal: Event-based vision is a fast growing field in need of high-quality datasets. In this project, we explore the utility of event cameras in an autonomous car scenario. In order to achieve this, a high-quality driving dataset will be created that incorporates not only common sensors such as standard cameras, GPS, IMU and possibly LIDAR but also state-of-the-art event cameras. You will be collaborating with a research division of Volkswagen to join expertise in event-based vision and autonomous driving for high-quality results. We seek a highly motivated student with the following minimum qualifications: - Experience with programming microcontrollers or motivation to acquire it quickly - Good coding skills in Python and C++ - At least one course in computer vision - Strong work ethic Preferred qualifications: - Background in robotics and experience with ROS - Experience with deep learning - Experience with event-based vision

Contact Details: Mathias Gehrig (mgehrig at ifi.uzh.ch); Daniel Gehrig (dgehrig at ifi.uzh.ch) Please add CV + transcripts (Bachelor and Master)

Thesis Type: Master Thesis

See project on SiROP

Designing an Event Camera for Learning - Available

Description: Event cameras such as the Dynamic Vision Sensor (DVS) are recent sensors with a lot of potential for high-speed and high-dynamic-range robotic applications.They have been successfully applied in many applications, such as high speed video and high speed visual odometry. Recently, many new event cameras have been commercialized with higher and higher spatial resolutions and high temporal resolution. However, these developments steadily increase the the computational requirements for downstream algorithms, increasing the necessary bandwidth and reducing the time available to process events. In this work we want to find out how important these design parameters are for deep learning applications. Applicants should have experience in coding image processing algorithms in C++ and experience with learning frameworks in python such as tensorflow or pytorch.

Goal: The goal of this project is to find out how important the design parameters of event cameras, such as spatial and temporal resolution, are for deep learning applications.

Contact Details: Daniel Gehrig (dgehrig (at) ifi (dot) uzh (dot) ch), Antonio Loquercio (antonilo (at) ifi (dot) uzh (dot) ch)

Thesis Type: Semester Project / Internship / Master Thesis

See project on SiROP

Learning 3D Reconstruction using an Event Camera - Available

Description: Event cameras such as the Dynamic Vision Sensor (DVS) are recent sensors with large potential for high-speed and high dynamic range robotic applications. In particular, they have been used to generate high speed video and for high speed visual odometry. In this project we want to explore the possibility using an event camera to do asynchronous 3D reconstruction with very high temporal resolution. These properties are critical in applications such as fast obstacle avoidance and fast mapping. Applicants should have a background in C++ programming and low-level vision. In addition, familiarity with learning frameworks such as pytorch or tensorflow are required.

Goal: The goal of this project is to explore a learning-based 3D reconstruction method with an event camera.

Contact Details: Daniel Gehrig (dgehrig (at) ifi (dot) uzh (dot) ch), Mathias Gehrig (mgehrig (at) ifi (dot) uzh (dot) ch)

Thesis Type: Collaboration / Master Thesis

See project on SiROP

Learning an Event Camera - Available

Description: Event cameras such as the Dynamic Vision Sensor (DVS) are recent sensors with a lot of potential for high-speed and high dynamic range robotic applications. They have been successfully applied in many applications, such as high speed video and high speed visual odometry. In spite of this success, the exact operating principle of event cameras, that is, how events are generated from a given visual signal and how noise is generated, is not well understood. In his work we want to explore new techniques for modelling the generation of events in an event camera, which would have wide implications for existing techniques. Applicants should have a background in C++ programming and low-level vision. In addition, familiarity with learning frameworks such as pytorch or tensorflow are required.

Goal: The goal of this project is to explore new techniques for modelling an event camera.

Contact Details: Daniel Gehrig (dgehrig (at) ifi (dot) uzh (dot) ch), Mathias Gehrig (mgehrig (at) ifi (dot) uzh (dot) ch)

Thesis Type: Semester Project / Internship / Master Thesis

See project on SiROP

Asynchronous Processing for Event-based Deep Learning - Available

Description: Event cameras such as the Dynamic Vision Sensor (DVS) are recent sensors with large potential for high-speed and high dynamic range robotic applications. Since their output is sparse traditional algorithms, which are designed for dense inputs such as frames, are not well suited. The goal of this project is explore ways to adapt existing deep learning algorithms to handle sparse asynchronous data from events. Applicants should have experience in C++ and python deep learning frameworks (tensorflow or pytorch), and have a strong background in computer vision.

Goal: The goal of this project is explore ways to adapt existing deep learning algorithms to handle sparse asynchronous data from events.

Contact Details: Daniel Gehrig (dgehrig at ifi.uzh.ch)

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Pushing hard cases in tag detection with a CNN - Available

Description: Visual Tags such as April or Aruco tags are nowadays detected with a handcrafted algorithm. This algorithm has its limitations in special cases, such as when the tag is far away from the camera, when the tag is partially occluded or when a camera with high distortion is used.

Goal: In this project, you will train a CNN to handle these special cases. We will first brainstorm a meaningful architecture that will allow a CNN to complement classical tag detection in the most effective way. You will then figure out the most effective way to create meaningful training data (hybrid of synthetic and real data?). Finally, you will use that data to train the desired detector.

Contact Details: Titus Cieslewski ( titus at ifi.uzh.ch ), APPLY VIA EMAIL, ATTACH CV AND TRANSCRIPT! Required skills: Linux, Python, ability to read C++ code. Desirable skill: Tensorflow or similar.

Thesis Type: Semester Project / Bachelor Thesis / Master Thesis

See project on SiROP

Teach and Aggressive Repeat - Available

Description: When we think of robot path planning, we often think of fitting optimal trajectories into dense 3D maps. This requires high quality 3D maps in the first place, which are often hard to obtain. An alternative approach, called Teach and Repeat, is to retrace previously traversed paths. Teach and Repeat maps are easier to create, as no globally consistent pose estimate is required. They can also be very compact, as the environment only needs to be sampled at sparse, visually salient locations. In this project, you will do T&R with a twist: Try to fly the repeat as fast as possible.

Goal: Start by building a basic teach and repeat based on existing components. Then, start increasing the repeat speed. Find out what the limitations are. Perceptual limitations like motion blur? If so, can this be solved with event cameras ( https://goo.gl/itzpJN ) ? Or is it avoiding collisions, as potentially tight maneuvers from the slow teach phase cannot be repeated at high velocities? You will most likely start with deployment on a real quadrotor very soon.

Contact Details: Titus Cieslewski ( titus at ifi.uzh.ch ), APPLY VIA EMAIL, ATTACH CV AND TRANSCRIPT (also Bachelor)! Required skills: Linux, C++, ROS. Students who took the Vision Algorithms for Mobile Robots class are at an advantage.

Thesis Type: Master Thesis

See project on SiROP

MPC for high speed trajectory tracking - Available

Description: Many algorithms exist for model predictive control for trajectory tracking for quadrotors and equally many implementation advantages and disadvantages can be listed. This thesis should find the main influence factors on high speed/high precision trajectory tracking such as: modell accuracy, aerodynamic forces modelling, execution speed, underlying low-level controllers, sampling times and sampling strategies, noise sensitivity or even come up with a novel implementation.

Goal: The end-goal of the thesis should be a comparison of the influence factors and based on that a recommendation or even implementation of an improved solution.

Contact Details: Philipp Föhn (foehn at ifi.uzh.ch)

Thesis Type: Master Thesis

See project on SiROP

Generation of Fast or Time-Optimal Tracjectories for Quadrotor Flight - Available

Description: With the rise of complex control and planning methods, quadrotors are capable of executing astonishing maneuvers. While generating trajectories between two known poses or states is relatively simple, planning through multiple waypoints is rather complicated. The master class of this problem is the task of flying as fast as possible through multiple gates, as done in drone racing. While humans can perform such fast racing maneuvers at extreme speeds of more than 100 km/h, algorithms struggle with even planning such trajectories. Within this thesis, we want to research methods to generate such fast trajectories and work towards a time-optimal planner. This requires some prior knowledge in at least some of the topics including: planning for robots, optimization techniques, model predictive control, RRT, and quadrotors or UAVs in general. The tasks will reach from problem analysis, approximation, and solution concepts to implementation and testing in simulation with existing software tools.

Goal: The goal would be to analyse the planning problem, develop approximation techniques and solve it as time-optimal as possible during thesis.

Contact Details: Philipp Föhn (foehn at ifi.uzh.ch)

Thesis Type: Master Thesis

See project on SiROP

Decentralized Visual Map Building - Available

Description: In State-of-the-Art decentralized mapping methods, optimization (correcting odometry drift) is typically done using pose graph optimization due to the fact that a pose graph is a very compact representation. Unfortunately, this compression in data results in limitations in precision and robustness. Bundle Adjustment is a map optimization method for visual maps which is much more precise and robust, but also much more data intensive.

Goal: In this work, you will figure out a way to achieve the superior precision of Bundle Adjustment while minimizing the amount of data that needs to be exchanged between robots in a decentralized setting.

Contact Details: Titus Cieslewski ( titus at ifi.uzh.ch ), APPLY VIA EMAIL, ATTACH CV AND TRANSCRIPT! Required skills: Matlab or C++, with a preference for the latter. Desirable: Background in optimization (Nonlinear least squares, Gauss-Newton or similar)

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Learning minimal representations of places - Available

Description: Place recognition and 6DoF localization has a wide range of applications, whether in robot autonomy, VR/AR or navigation interfaces. Given sensor readings (we focus on images), the goal is to establish position and orientation of a robot/device with respect to a previously recorded map. Recently, this is generally solved with a mixture of machine learning and geometry (NetVLAD, SuperPoint, LF-NET, PoseNet). Our focus in particular will be to solve this problem with a minimal representation.

Goal: Given query agent A and map agent B, have B establish a pose of A within its map, with minimal data transmission from A to B. We have a couple of ideas on how to solve this (see our most recent publication on this: https://arxiv.org/abs/1811.10681 ), but you are encouraged to bring your own ideas to the table.

Contact Details: Titus Cieslewski ( titus at ifi.uzh.ch ), APPLY VIA EMAIL, ATTACH CV AND TRANSCRIPT (also Bachelor)! Preferred skills: Linux, Python, “Vision Algorithms for Mobile Robots” class or equivalent, TensorFlow/PyTorch or equivalent.

Thesis Type: Master Thesis

See project on SiROP

High-Performance Simulation of Spiking Neural Network on GPUs - Available

Description: One major complication in research of biologically-inspired spiking neural Networks (SNNs) is simulation performance on conventional hardware (CPU/GPU). Computation in SNNs is dominated by operations on sparse tensors but usually this potential benefit is ignored to save development time. However, the exploitation of sparsity could be beneficial to scale simulation of SNNs to larger datasets. Requirements: - Experience with deep learning frameworks (e.g. TensorFlow or PyTorch) - Excellent programming skills and experience in CUDA

Goal: In this project, you will leverage sparse computation to develop high-performance simulations of SNNs that can be used for optimization. This will help to scale experiments and drastically improve results obtained by SNNs.

Contact Details: Mathias Gehrig, mgehrig (at) ifi (dot) uzh (dot) ch

Thesis Type: Semester Project

See project on SiROP