Student Projects


How to apply

To apply, please send your CV, your Ms and Bs transcripts by email to all the contacts indicated below the project description. Do not apply on SiROP . Since Prof. Davide Scaramuzza is affiliated with ETH, there is no organizational overhead for ETH students. Custom projects are occasionally available. If you would like to do a project with us but could not find an advertized project that suits you, please contact Prof. Davide Scaramuzza directly to ask for a tailored project (sdavide at ifi.uzh.ch).


Upon successful completion of a project in our lab, students may also have the opportunity to get an internship at one of our numerous industrial and academic partners worldwide (e.g., NASA/JPL, University of Pennsylvania, UCLA, MIT, Stanford, ...).



MPC for high speed trajectory tracking - Available

Description: Many algorithms exist for model predictive control for trajectory tracking for quadrotors and equally many implementation advantages and disadvantages can be listed. This thesis should find the main influence factors on high speed/high precision trajectory tracking such as: modell accuracy, aerodynamic forces modelling, execution speed, underlying low-level controllers, sampling times and sampling strategies, noise sensitivity or even come up with a novel implementation.

Goal: The end-goal of the thesis should be a comparison of the influence factors and based on that a recommendation or even implementation of an improved solution.

Contact Details: Philipp Föhn (foehn at ifi.uzh.ch)

Thesis Type: Master Thesis

See project on SiROP

Generation of Fast or Time-Optimal Tracjectories for Quadrotor Flight - Available

Description: With the rise of complex control and planning methods, quadrotors are capable of executing astonishing maneuvers. While generating trajectories between two known poses or states is relatively simple, planning through multiple waypoints is rather complicated. The master class of this problem is the task of flying as fast as possible through multiple gates, as done in drone racing. While humans can perform such fast racing maneuvers at extreme speeds of more than 100 km/h, algorithms struggle with even planning such trajectories. Within this thesis, we want to research methods to generate such fast trajectories and work towards a time-optimal planner. This requires some prior knowledge in at least some of the topics including: planning for robots, optimization techniques, model predictive control, RRT, and quadrotors or UAVs in general. The tasks will reach from problem analysis, approximation, and solution concepts to implementation and testing in simulation with existing software tools.

Goal: The goal would be to analyse the planning problem, develop approximation techniques and solve it as time-optimal as possible during thesis.

Contact Details: Philipp Föhn (foehn at ifi.uzh.ch)

Thesis Type: Master Thesis

See project on SiROP

Decentralized Visual Map Building - Available

Description: In State-of-the-Art decentralized mapping methods, optimization (correcting odometry drift) is typically done using pose graph optimization due to the fact that a pose graph is a very compact representation. Unfortunately, this compression in data results in limitations in precision and robustness. Bundle Adjustment is a map optimization method for visual maps which is much more precise and robust, but also much more data intensive.

Goal: In this work, you will figure out a way to achieve the superior precision of Bundle Adjustment while minimizing the amount of data that needs to be exchanged between robots in a decentralized setting.

Contact Details: Titus Cieslewski ( titus at ifi.uzh.ch ), APPLY VIA EMAIL, ATTACH CV AND TRANSCRIPT! Required skills: Matlab or C++, with a preference for the latter. Desirable: Background in optimization (Nonlinear least squares, Gauss-Newton or similar)

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Learning minimal representations of places - Available

Description: Place recognition and 6DoF localization has a wide range of applications, whether in robot autonomy, VR/AR or navigation interfaces. Given sensor readings (we focus on images), the goal is to establish position and orientation of a robot/device with respect to a previously recorded map. Recently, this is generally solved with a mixture of machine learning and geometry (NetVLAD, SuperPoint, LF-NET, PoseNet). Our focus in particular will be to solve this problem with a minimal representation.

Goal: Given query agent A and map agent B, have B establish a pose of A within its map, with minimal data transmission from A to B. We have a couple of ideas on how to solve this (see our most recent publication on this: https://arxiv.org/abs/1811.10681 ), but you are encouraged to bring your own ideas to the table.

Contact Details: Titus Cieslewski ( titus at ifi.uzh.ch ), APPLY VIA EMAIL, ATTACH CV AND TRANSCRIPT (also Bachelor)! Preferred skills: Linux, Python, “Vision Algorithms for Mobile Robots” class or equivalent, TensorFlow/PyTorch or equivalent.

Thesis Type: Master Thesis

See project on SiROP

Optimization for Spiking Neural Networks - Available

Description: Spiking neural networks (SNNs) are neural networks that process information with timing of events/spikes rather than numerical values. Together with event-cameras, SNNs show promise to both lower latency and computational burden compared to artificial neural networks. In recent years, researchers have proposed several methods to estimate gradients of SNN parameters in a supervised learning context. In practice, many of these approaches rely on assumptions that might not hold in all scenarios. Requirements: - Background in machine learning; especially deep learning - Good programming skills; experience in CUDA is a plus.

Goal: In this project we explore state-of-the-art optimization methods for SNNs and their suitability to solve the temporal credit-assignment problem. As a first step, an in-depth evaluation of a selection of algorithms is required. Based on the acquired insights, the prospective student can propose improvements and implement their own method.

Contact Details: Mathias Gehrig, mgehrig (at) ifi (dot) uzh (dot) ch

Thesis Type: Master Thesis

See project on SiROP

High-Performance Simulation of Spiking Neural Network on GPUs - Available

Description: One major complication in research of biologically-inspired spiking neural Networks (SNNs) is simulation performance on conventional hardware (CPU/GPU). Computation in SNNs is dominated by operations on sparse tensors but usually this potential benefit is ignored to save development time. However, the exploitation of sparsity could be beneficial to scale simulation of SNNs to larger datasets. Requirements: - Experience with deep learning frameworks (e.g. TensorFlow or PyTorch) - Excellent programming skills and experience in CUDA

Goal: In this project, you will leverage sparse computation to develop high-performance simulations of SNNs that can be used for optimization. This will help to scale experiments and drastically improve results obtained by SNNs.

Contact Details: Mathias Gehrig, mgehrig (at) ifi (dot) uzh (dot) ch

Thesis Type: Semester Project

See project on SiROP

Reinforcement learning for car racing in GTS - Available

Description: The Gran Turismo SPORT (GTS) for the Sony Playstation 4 is a modern car racing game which can simulate accurate physics of the car racing environment, including detailed models of various race cars and racing tracks. Data-driven approaches (e.g. reinforcement learning) have shown to be a promising method for solving such a continuous control problem where the dynamics of the environment are hard to model. We trained a neural network policy end-to-end using policy search methods for car racing in GTS. The policy has shown results comparable with human players and exceeded the performance of model-based agents. However, the model-free agent requires a large number of sampling steps in the environment, and thus, suffers from high sample complexities. -- Programming experience in C++/Python and TensorFlow/Pytorch. Reinforcement learning, deep learning or machine learning experience.

Goal: The goal of this master thesis is to first investigate reinforcement learning methods (imitation learning, model-free, model-based, hybrid methods, etc) for car racing, and then, train an agent that can drive faster than human experts. Ultimately, we aim to race the car with other players on the same track such that it also learn how to avoid dynamic obstacles.

Contact Details: Yunlong Song (song@ifi.uzh.ch), Elia Kaufmann (ekaufmann@ifi.uzh.ch)

Thesis Type: Master Thesis

See project on SiROP

Learning-Guided MPC Flight - Available

Description: Model predictive control (MPC) is a versatile optimization-based control method that allows to incorporate constraints directly into the control problem. The advantages of MPC can be seen in its ability to accurately control dynamical systems that include large time delays and high-order dynamics. Recent advances in compute hardware allow to run MPC even on compute-constrained quadrotors. While model-predictive control can deal with complex systems and constraints, it still assumes the existence of a reference trajectory. With this project we aim to guide the MPC to a feasible reference trajectory by using a neural network that directly predicts from camera images an expressive intermediate representation. Such tight coupling of perception and control would allow to push the speed limits of autonomous flight through cluttered environments. Requirements: - Machine learning experience (TensorFlow and/or PyTorch) - Experience in MPC preferable but not strictly required - Programming experience in C++ and Python

Goal: Evaluate different intermediate representations for autonomous flight. Implement the learned perception system in simulation and integrate the predictions into an existing MPC pipeline. If possible, deploy on a real system.

Contact Details: Elia Kaufmann (ekaufmann@ifi.uzh.ch) Philipp Föhn (foehn@ifi.uzh.ch)

Thesis Type: Master Thesis

See project on SiROP

Probabilistic System Identification of a Quadrotor Platform - Available

Description: Most planning & control algorithms used on quadrotors make use of a nominal model of the platform dynamics to compute feasible trajectories or generate control commands. Such models are derived using first principles and typically cannot fully capture the true dynamics of the system, leading to sub-optimal performance. One appealing approach to overcome this limitation is to use Gaussian Processes for system modeling. Gaussian Process regression has been widely used in supervised machine learning due to its flexibility and inherent ability to describe uncertainty in the prediction. This work investigates the usage of Gaussian Processes for uncertainty-aware system identification of a quadrotor platform. Requirements: - Machine learning experience preferable but not strictly required - Programming experience in C++ and Python

Goal: Implement an uncertainty-aware model of the quadrotor dynamics, train and evaluate the model on simulated and real data.

Contact Details: Elia Kaufmann (ekaufmann@ifi.uzh.ch)

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Why so complicated? Global localization without fuss - Available

Description: The current state-of-the-art for global vision-based localization, NetVLAD, has a relatively sophisticated network architecture. We would like to question whether this complexity is really necessary. While there are comparisons against simpler architectures, we wonder: what if some simple trick slipped under the radar?

Goal: Achieve comparable or better performance than NetVLAD, with a simpler network architecture.

Contact Details: Titus Cieslewski ( titus at ifi.uzh.ch ), ATTACH CV AND TRANSCRIPT (also Bachelor)! Preferred skills: Linux, Python, some Computer Vision background, TensorFlow/PyTorch or equivalent. This project will be co-supervised by Dimche Kostadinov.

Thesis Type: Master Thesis

See project on SiROP

Data-Driven Visual Inertial Odometry for Quadrotor Flight - Available

Description: Classical VIO pipelines use geometric information to infer the ego-motion of the camera and couple this information with measurements from the IMU. While these pipelines have shown very good performance in controlled, structured environments, their performance decreases when applied in low-texture or dynamic environments or when applied to high-speed motion. Recent works propose the usage of data-driven approaches for camera ego-motion estimation. While such approaches could potentially learn a VIO pipeline end-to-end, their generalizability is not good enough for real-world deployment. This work investigates the usage of a hybrid VIO pipeline featuring a learned visual frontend. Requirements: - Background in computer vision and machine learning - Deep learning experience preferable but not strictly required - Programming experience in C++ and Python

Goal: Based on results from a previous student project, the goal is to deploy a hybrid VIO pipeline on a quadrotor equipped with a GPU (Jetson TX2).

Contact Details: Elia Kaufmann (ekaufmann@ifi.uzh.ch); Philipp Foehn (foehn@ifi.uzh.ch)

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Learning to Deblur Images with Events - Available

Description: Images suffer from motion blur due to long exposure in poor light condition or rapid motion. Unlike conventional cameras, event-cameras do not suffer from motion blur. This is due to the fact that event-cameras provide events together with the exact time when they were triggered. In this project, we will make use of hybrid sensors which provide both conventional images and events such that we can leverage the advantages of both. Requirements: - Background in computer vision and machine learning - Deep learning experience preferable but not strictly required - Programming experience in C++ and Python

Goal: The goal is to develop an algorithm capable producing a blur-free image from the captured, blurry image, and events within the exposure time. To this end, synthetic data can be generated by our simulation framework which is able to generate both synthetic event data and motion blurred images. This data can be used by machine learning algorithms designed to solve the task at hand. At the end of the project, the algorithm will be adapted to perform optimally with real-world data.

Contact Details: Mathias Gehrig (mgehrig at ifi.uzh.ch); Daniel Gehrig (dgehrig at ifi.uzh.ch)

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Simulation to Real World Transfer - Available

Description: Recent techniques based on machine learning enabled robotics system to perform many difficult tasks, such as manipulation or navigation. Those techniques are usually very data-intensive, and require simulators to generate enough training data. However, a system only trained in simulation (usually) fails when deployed in the real world. In this project, we will develop techniques to maximally transfer knowledge from simulation to the real world, and apply them to real robotics systems.

Goal: The project aims to develop techniques based on machine learning to have maximal knowledge transfer between simulated and real world on a navigation task.

Contact Details: **Antonio Loquercio** loquercio@ifi.uzh.ch

Thesis Type: Semester Project / Bachelor Thesis / Master Thesis

See project on SiROP

Unsupervised Obstacle Detection Learning - Available

Description: Supervised learning is the gold standard algorithm to solve computer vision tasks like classification, detection or segmentation. However, for several interesting tasks (e.g. moving object detection, depth estimation, etc.) collecting the large annotated datasets required by the aforementioned algorithms is a very tedious and costly process. In this project, we aim to build a self-supervised depth estimation and segmentation algorithm by embedding classic computer vision principles (e.g. brightness constancy) into a neural network. **Requirements**: Computer vision knowledge; programming experience with python. Machine learning knowledge is a plus but it is not required.

Goal: The goal of this project consists of building a perception system which can learn to detect obstacles without any ground truth annotations.

Contact Details: Antonio Loquercio, _loquercio@ifi.uzh.ch_

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Target following on nano-scale UAV - Available

Description: Autonomous Unmanned Aerial Vehicles (UAVs) have numerous applications due to their agility and flexibility. However, navigation algorithms are computationally demanding, and it is challenging to run them on-board of nano-scale UAVs (i.e., few centimeters of diameter). This project focuses on the object tracking, (i.e., target following) on such nano-UAVs. To do this, we will first train a Convolutional Neural Network (CNN) with data collected in simulation, and then run the aforementioned network on a parallel ultra-low-power (PULP) processor, enabling flight with on-board sensing and computing only. **Requirements**: Knowledge of python, cpp and embedded programming. Machine learning knowledge is a plus but it is not strictly required.

Contact Details: Antonio Loquercio, _loquercio@ifi.uzh.ch_

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Ground Segmentation for Landing - Available

Description: In order for UAVs to fly and especially land autonomously we need evaluate the safety of landing spots. Having semantic information about the scenery, eg. road, forest, roof, etc, enables the UAV to pick a safe landing spot.

Goal: The goal of this project is to use Machine Learning to segment the grounds scenery into a set of classes using either a monocular or stereo camera setup. Part of the project is the evaluation of existing data-sets for training, finding the optimal algorithm, as well as creating an efficient representation of the segmented areas, so that it can be integrated into a complete system.

Contact Details: - Kevin Kleber: kevinkleber@ifi.uzh.ch - Kunal Shrivastava: shrivastava@ifi.uzh.ch - Antonio Loquercio: loquercio@ifi.uzh.ch

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Stereo matching using CUDA - Available

Description: Dense stereo matching is crucial for creating depth maps. However, it is very computationally expensive on CPUs. As a result the update rate is low, which makes it unusable for many tasks, eg. avoidance in dynamic environments. Especially on drones the computation power is limited. Through the introduction of the Nvidia Jetson series computers we got access to lightweight embedded GPUs. Running the stereo matching on a GPU can potentially make it significantly faster.

Goal: Create a dense stereo matching pipeline utilizing the CUDA capabilities of Nvidia Jetson devices.

Contact Details: - Kevin Kleber: kevinkleber@ifi.uzh.ch - Kunal Shrivastava: shrivastava@ifi.uzh.ch - Philipp Foehn: foehn@ifi.uzh.ch

Thesis Type: Master Thesis

See project on SiROP

VIO in dynamic environments - Available

Description: Visual-Inertial Odometry is a great solution for drone-navigation in GPS-denied environments. Its ability to provide centimeter-level precision in local navigation makes it a suitable choice in many commercial applications like last centimeter drone delivery. Conventional VIO algorithms work well in static environments. However, when the environment is dynamic, i.e most of the visual features come from a moving environment, for instance a moving platform, the VIO does not perform reliably. This problem can be attributed to the unreliable initialisation phase of the VIO pipeline, which is the most critical phase. Most initialisation algorithms are based on structure-from-motion, which assumes that the environment is static. In such a scenario the initialisation algorithm needs to be adapted to take into account the motion of the features.

Goal: The goal of this project is to develop a specialized initialization algorithm for VIO that can be used in dynamic environments.

Contact Details: - Kevin Kleber: kevinkleber@ifi.uzh.ch - Kunal Shrivastava: shrivastava@ifi.uzh.ch

Thesis Type: Master Thesis

See project on SiROP

Pushing hard cases in tag detection with a CNN - Available

Description: Visual Tags such as April or Aruco tags are nowadays detected with a handcrafted algorithm. This algorithm has its limitations in special cases, such as when the tag is far away from the camera, when the tag is partially occluded or when a camera with high distortion is used.

Goal: In this project, you will train a CNN to handle these special cases. We will first brainstorm a meaningful architecture that will allow a CNN to complement classical tag detection in the most effective way. You will then figure out the most effective way to create meaningful training data (hybrid of synthetic and real data?). Finally, you will use that data to train the desired detector.

Contact Details: Titus Cieslewski ( titus at ifi.uzh.ch ), APPLY VIA EMAIL, ATTACH CV AND TRANSCRIPT! Required skills: Linux, Python, ability to read C++ code. Desirable skill: Tensorflow or similar.

Thesis Type: Semester Project / Bachelor Thesis / Master Thesis

See project on SiROP

Teach and Aggressive Repeat - Available

Description: When we think of robot path planning, we often think of fitting optimal trajectories into dense 3D maps. This requires high quality 3D maps in the first place, which are often hard to obtain. An alternative approach, called Teach and Repeat, is to retrace previously traversed paths. Teach and Repeat maps are easier to create, as no globally consistent pose estimate is required. They can also be very compact, as the environment only needs to be sampled at sparse, visually salient locations. In this project, you will do T&R with a twist: Try to fly the repeat as fast as possible.

Goal: Start by building a basic teach and repeat based on existing components. Then, start increasing the repeat speed. Find out what the limitations are. Perceptual limitations like motion blur? If so, can this be solved with event cameras ( https://goo.gl/itzpJN ) ? Or is it avoiding collisions, as potentially tight maneuvers from the slow teach phase cannot be repeated at high velocities? You will most likely start with deployment on a real quadrotor very soon.

Contact Details: Titus Cieslewski ( titus at ifi.uzh.ch ), APPLY VIA EMAIL, ATTACH CV AND TRANSCRIPT (also Bachelor)! Required skills: Linux, C++, ROS. Students who took the Vision Algorithms for Mobile Robots class are at an advantage.

Thesis Type: Master Thesis

See project on SiROP