Student Projects


How to apply

To apply, please send your CV, your Ms and Bs transcripts by email to all the contacts indicated below the project description. Do not apply on SiROP . Since Prof. Davide Scaramuzza is affiliated with ETH, there is no organizational overhead for ETH students. Custom projects are occasionally available. If you would like to do a project with us but could not find an advertized project that suits you, please contact Prof. Davide Scaramuzza directly to ask for a tailored project (sdavide at ifi.uzh.ch).


Upon successful completion of a project in our lab, students may also have the opportunity to get an internship at one of our numerous industrial and academic partners worldwide (e.g., NASA/JPL, University of Pennsylvania, UCLA, MIT, Stanford, ...).



Online trajectory re-planning using Gaussian Processes - Available

Description: Online re-planning may be necessary to guarantee safe autonomous navigation of quadrotors in unknown environments. Gaussian processes (GPs), which are machine learning models used in tasks such as regression and classification, have been proposed in robotics to solve the planning problem. GPs are an interesting tool for motion planning since they give a probabilistic perspective on the problem by formulating it as probabilistic inference. Such inference can be performed quickly, which allows to efficiently and incrementally re-plan the current trajectory. In this project, we will use GPs for planning trajectories for landing of a quadrotor platform in scenarios where safety is a key requirement (e.g., in proximity to humans, on hazardous surfaces) and investigate the advantages and disadvantages of such models compared to polynomial trajectories. Requirements: - Experience in Machine learning preferable - Experience in motion planning in robotics preferable - Programming experience in C++ and Python.

Goal: The final goal of this thesis is to benchmark GPs motion planning against polynomial trajectories in simulation. If promising results are achieved, we will deploy the algorithm developed in this thesis on a real system.

Contact Details: Giovanni Cioffi (cioffi@ifi.uzh.ch)

Thesis Type: Semester Project

See project on SiROP

Line tracking using event cameras - Available

Description: Event cameras are recent sensors with large potential for high speed and high dynamic range robotic applications. Recent works have shown promising results in using event cameras on board resource constrained platforms such as quadrotors to perform tasks where the standard cameras usually fail, for example due to fast motion and challenging illumination conditions. In this thesis, we will use state-of-the-art methods in event based vision and/or adapt standard computer vision algorithms to develop a reliable line tracking system. A successful thesis will lead to the deployment of the developed algorithm on a real quadrotor platform for power-line inspection. Requirements: - Experience with event cameras preferable but not required - Passionate about robotics - Programming experience in C++ and Python.

Goal: In this project we will develop a light-weight event-based line tracking algorithm and deploy on a real quadrotor.

Contact Details: Giovanni Cioffi (cioffi@ifi.uzh.ch) and Javier Hidalgo-Carrió (jhidalgocarrio@ifi.uzh.ch)

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Active Perception for Motion Planning - Available

Description: Fast local re-planning is necessary for safe autonomous navigation in the presence of unknown and/or moving obstacles. Recent works have raised interest for active perception methods in motion planning. Such methods include perception awareness in the planning problem. However, how to represent perception awareness is an open research problem. It depends on the representation of the environment. Key requirements for a suitable environment representation are that it can be incrementally updated on-line, light-weight, and it contains confidence measurements on the occupancy.

Goal: In this thesis, we will evaluate commonly used mapping methods (e.g., voxel map, point-clouds, ...) for the local re-planning problem. The final goal is to based on such evaluation a recommendation or even implementation of a novel mapping solution. Requirements: - Experience with mapping in robotics preferable but not required - Programming experience in C++ and Python.

Contact Details: Giovanni Cioffi (cioffi@ifi.uzh.ch)

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Bringing Thermal Cameras into Robotics - Available

Description: Thermographic cameras can capture detailed images regardless of ambient lighting conditions.They use an infrared (IR) sensing technology to map heat variations within the sensor’s range and field-of-view, providing movement detection and hot-spot mapping even in total darkness. Visible range covers wavelengths of approximately 400 – 700 nanometres (nm) in length. However, thermographic cameras generally sample thermal radiation from within the longwave infrared range(approximately 7,000 – 14,000 nm) with a great potential in robotics. Thermography images are useful to identify week points on the power line, along the cable and on the isolators or containers. However, current lightweight thermal cameras are unexplored, with limited in pixel resolution (32x32 pixels) unable to deliver exceptional sensitivity, resolution and image quality for meaningful applications. This work aims to expand the frontiers of computer vision by using thermographic cameras and investigate their application in robotics i.e. perception, state estimation and path planning. The project will combine traditional computer vision techniques together with deep-learning approaches to bring thermography images into the field of robotics. Requirements: Background in computer vision and machine learning - Deep learning experience preferable – Excellent programming experience in C++ and Python

Goal: Perception, state estimation or path planning using thermographic cameras.

Contact Details: Javier Hidalgo-Carrió (jhidalgocarrio@ifi.uzh.ch) and Giovanni Cioffi (cioffi@ifi.uzh.ch)

Thesis Type: Semester Project / Master Thesis

See project on SiROP

MPC for high speed trajectory tracking - Available

Description: Many algorithms exist for model predictive control for trajectory tracking for quadrotors and equally many implementation advantages and disadvantages can be listed. This thesis should find the main influence factors on high speed/high precision trajectory tracking such as: modell accuracy, aerodynamic forces modelling, execution speed, underlying low-level controllers, sampling times and sampling strategies, noise sensitivity or even come up with a novel implementation.

Goal: The end-goal of the thesis should be a comparison of the influence factors and based on that a recommendation or even implementation of an improved solution.

Contact Details: Philipp Föhn (foehn at ifi.uzh.ch)

Thesis Type: Master Thesis

See project on SiROP

Learning features for efficient deep reinforcement learning - Available

Description: The study of end-to-end deep learning in computer vision has mainly focused on developing useful object representations for image classification, object detection, or semantic segmentation. Recent work has shown that it is possible to learn temporally and geometrically aligned keypoints given only videos, and the object keypoints learned via unsupervised learning manners can be useful for efficient control and reinforcement learning.

Goal: The goal of this project is to find out if it is possible to learn useful features or intermediate representation s for controlling mobile robots in high-speed. For example, can we use the Transporter (a neural network architecture) for finding useful features in an autonomous car racing environment? if so, can we use these features for discovering an optimal control policy via deep reinforcement learning? **Required skills:** Python/C++ reinforcement learning, and deep learning skills.

Contact Details: Yunlong Song (song@ifi.uzh.ch)

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Perception Aware Model Predictive Control for Autonomous Power Line Tracking - Available

Description: Classical power line inspection and maintenance are dangerous, costly and time consuming. Drones could mitigate the risk for humans and minimize the cost for the direct benefit of the power line infrastructure. Coupling perception and path planning with control has become increasingly popular in aerial vehicles. This project will continue to investigate vision-based navigation tightly couple approaches of perception and control to satisfy the drone dynamics and compute feasible trajectories with respect to the input saturation. This involves to further development the research on perception aware Model Predictive Control (MPC) for quadrotors, solving the challenging aspects of a power line inspection scenario. A perception aware MPC approach would ideally improve the aerial vehicle's behavior during an approaching maneuver.

Goal: Pursue research on a unified control and planning approach that integrates the action and perception objectives to satisfy this functionality. The functionality will accomplish to the challenging environmental conditions and the morphology of the target which imposes several points of interest for the optimizer.

Contact Details: Javier Hidalgo-Carrió (jhidalgocarrio@ifi.uzh.ch) and Yunlong Song (song@ifi.uzh.ch)

Thesis Type: Bachelor Thesis / Master Thesis

See project on SiROP

Power-Line Dataset for Autonomous Drone Inspection - Available

Description: Classical power line inspection and maintenance are dangerous, costly and time consuming. Drones could mitigate the risk for humans and minimize the cost for direct benefit of the infrastructure. Several sensing capabilities has been already tested (i.e. RGB, LiDAR) which gives the drone the abilities to operate in unstructured environments. Sensor fusion is a popular technique to get the best of each sensor for autonomous navigation. Benchmark of perception strategies is a key part for solid and robust algorithm development before final deployment on the system. However, the lack of relevant and accurate data for multiple sensors makes difficult the Verification and Validation (V&V) process of perception algorithms. The goals of this project is to deliver the first multi sensor power-line inspection dataset for drones with alternative sensory data and ground truth. Requirements: Background in robotics and autonomous systems – Drone navigation preferable – Excellent programming in C++ and Python – Knowledge of ROS and robotic middle-ware - Passionate about robotics and engineering in general. - Linux

Goal: Release an open-access dataset for the evaluation of perception pipelines for autonomous drones. The goal is to establish a solid benchmark for autonomous drone inspection of power-lines . The following sensors are consider to be part of the dataset: - Absolute depth information - RGB images - Event-based camera - Thermography - Inertial Sensory information - Ground truth positioning

Contact Details: Javier Hidalgo-Carrió (jhidalgocarrio@ifi.uzh.ch) and Giovanni Cioffi (cioffi@ifi.uzh.ch)

Thesis Type: Bachelor Thesis / Master Thesis

See project on SiROP

Generation of Fast or Time-Optimal Tracjectories for Quadrotor Flight - Available

Description: With the rise of complex control and planning methods, quadrotors are capable of executing astonishing maneuvers. While generating trajectories between two known poses or states is relatively simple, planning through multiple waypoints is rather complicated. The master class of this problem is the task of flying as fast as possible through multiple gates, as done in drone racing. While humans can perform such fast racing maneuvers at extreme speeds of more than 100 km/h, algorithms struggle with even planning such trajectories. Within this thesis, we want to research methods to generate such fast trajectories and work towards a time-optimal planner. This requires some prior knowledge in at least some of the topics including: planning for robots, optimization techniques, model predictive control, RRT, and quadrotors or UAVs in general. The tasks will reach from problem analysis, approximation, and solution concepts to implementation and testing in simulation with existing software tools.

Goal: The goal would be to analyse the planning problem, develop approximation techniques and solve it as time-optimal as possible during thesis.

Contact Details: Philipp Föhn (foehn at ifi.uzh.ch)

Thesis Type: Master Thesis

See project on SiROP

High Performance Computation for Non-Linear Estimation - Available

Description: Visual-inertial odometry (VIO) has matured and became the go-to solution for mobile robot state estimation. There exists a variety of VIO pipelines, reaching from computationally-efficient filter-based approaches to more accurate optimization-based sliding window estimators. However, such sliding window estimators are often relatively slow and introduce significant latency. We are developing a novel optimization-based backend with focus on low latency estimation. This thesis will focus on improving the computational efficiency of our new backend by analyzing the existing framework, reporting bottlenecks, and implementing new optimization strategies or exploiting existing acceleration libraries (such as BLAS, LAPACK or higher level libraries). There are multiple target platforms, including small ARM single-board computers on drones, on which the final solution will be demonstrated. Strong c++ knowledge is needed and experience in numerical optimization is welcome. While there is a high focus on code development, there are options for theoretical contributions as well.

Goal: The goal of this thesis is to analyze and profile an existing optimization backend for visual-inertial state estimation and propose, implement, and benchmark improvements for low-latency execution on handheld devices and drones. These improvements can be code optmiziations, combination with existing libraries for sparse and accelerated linear algebra, or theoretical advances.

Contact Details: Philipp Föhn (foehn at ifi.uzh.ch)

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Embedded systems development with NVIDIA Jetson TX2 for fast drone flying - Available

Description: The TX2 is a powerful computational unit with 2 Denver 64-bit CPUs + Quad-Core A57 Complex, NVIDIA Pascal™ Architecture GPU. We use image processing and IMU data in order to deploy our machine learning algorithms in real life experiments. This makes our robots fly autonomously without any help of external communication. In a first iteration, the objective is to have a fully functional connector board to the TX2 including power management, USB OTG, USB 3.0, 2 UARTs, (1 Serial port) Ethernet and a CSI (camera connector). In a second iteration we are redesigning the hardware, integrating our own Obvio-board (time synchronized IMU and RGB) and our own flight controller (integration of an ARM STM32 MyC).

Goal: Test and verify the existing prototype in collaboration with our lab engineers. Create a second iteration with the integration of a microcontroller in order to integrate time synchronization with an IMU and a camera and integrate our custom built flight controller using D-Shot.Applicants should have a solid understanding of Linux device tree for embedded ARM core and some experience using pcb software (kiCAD/Eagle) as well as solid knowledge on communication protocols (UART, USB, SPI, Ethernet).

Contact Details: Manuel Sutter (Systems Engineer BSc) msutter(at)ifi(dot)uzh(dot)ch

Thesis Type: Semester Project / Collaboration / Internship / Bachelor Thesis / Master Thesis

See project on SiROP

Probabilistic System Identification of a Quadrotor Platform - Available

Description: Most planning & control algorithms used on quadrotors make use of a nominal model of the platform dynamics to compute feasible trajectories or generate control commands. Such models are derived using first principles and typically cannot fully capture the true dynamics of the system, leading to sub-optimal performance. One appealing approach to overcome this limitation is to use Gaussian Processes for system modeling. Gaussian Process regression has been widely used in supervised machine learning due to its flexibility and inherent ability to describe uncertainty in the prediction. This work investigates the usage of Gaussian Processes for uncertainty-aware system identification of a quadrotor platform. Requirements: - Machine learning experience preferable but not strictly required - Programming experience in C++ and Python

Goal: Implement an uncertainty-aware model of the quadrotor dynamics, train and evaluate the model on simulated and real data.

Contact Details: Elia Kaufmann (ekaufmann@ifi.uzh.ch)

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Learning-Guided MPC Flight - Available

Description: Model predictive control (MPC) is a versatile optimization-based control method that allows to incorporate constraints directly into the control problem. The advantages of MPC can be seen in its ability to accurately control dynamical systems that include large time delays and high-order dynamics. Recent advances in compute hardware allow to run MPC even on compute-constrained quadrotors. While model-predictive control can deal with complex systems and constraints, it still assumes the existence of a reference trajectory. With this project we aim to guide the MPC to a feasible reference trajectory by using a neural network that directly predicts from camera images an expressive intermediate representation. Such tight coupling of perception and control would allow to push the speed limits of autonomous flight through cluttered environments. Requirements: - Machine learning experience (TensorFlow and/or PyTorch) - Experience in MPC preferable but not strictly required - Programming experience in C++ and Python

Goal: Evaluate different intermediate representations for autonomous flight. Implement the learned perception system in simulation and integrate the predictions into an existing MPC pipeline. If possible, deploy on a real system.

Contact Details: Elia Kaufmann (ekaufmann@ifi.uzh.ch) Philipp Föhn (foehn@ifi.uzh.ch)

Thesis Type: Master Thesis

See project on SiROP

Event-based Feature Tracking on an Embedded Platform - Available

Description: Event cameras such as the Dynamic Vision Sensor (DVS) are recent sensors with large potential for high-speed and high dynamic range robotic applications, such as fast obstacle avoidance. In particular, event cameras can be used to track features or objects in the blind time between two frames which makes it possible to react quickly to changes in the scene.

Goal: In this project we want to deploy an event-based feature tracking algorithm on a resource constrained platform such as a drone. Applicants should have a strong background in C++ programming and low-level vision. Experience with embedded programming is a plus.

Contact Details: Daniel Gehrig (dgehrig (at) ifi.uzh.ch), Elia Kaufmann (ekaufmann (at) ifi (dot) uzh (dot) ch), Mathias Gehrig (mgehrig (at) ifi (dot) uzh (dot) ch)

Thesis Type: Semester Project

See project on SiROP

3rd Person View Imitation Learning - Available

Description: Manually programming robots to carry out specific tasks is a difficult and time consuming process. A possible solution to this problem is to use _imitation learning_, in which a robot aims to imitate a teacher, e.g., a human, that knows how to perform the task. Usually, the teacher and the learner share the same point of view on the problem. However, this last assumption might not be necessary. As humans, for example, we learn to cook by looking at others cooking. During this project, we will explore the possibility of repeating such a kind of 3rd person view _imitation learning_ with flying robots on a navigation task.

Goal: The project aims to develop machine learning based techniques that will enable a drone to learn flying by looking at an other robot flying.

Contact Details: **Antonio Loquercio**: loquercio@ifi.uzh.ch

Thesis Type: Semester Project / Bachelor Thesis / Master Thesis

See project on SiROP

Safe Simulation to Real World Transfer - Available

Description: Recent techniques based on machine learning enabled robotics system to perform many difficult tasks, such as manipulation or navigation. Those techniques are usually very data-intensive, and require simulators to generate enough training data. In this project, we will develop techniques to formalize the notion of simulation to reality transfer in a robotics setting. In particular, we are interested in finding performance guarantees to bound the performance drop usually encountered when deploying a policy trained in simulation on a physical platform.

Goal: The project aims to develop techniques based on machine learning to have maximal knowledge transfer between simulated and real world on a general robotic task.

Contact Details: **Antonio Loquercio** loquercio@ifi.uzh.ch

Thesis Type: Semester Project / Bachelor Thesis / Master Thesis

See project on SiROP

Safe Unsupervised Learning for Drone Perception and Control - Available

Description: Supervised learning is the gold standard algorithm to solve computer vision tasks like classification, detection or segmentation. However, for several interesting tasks (e.g. control of a drone, etc.) collecting a large annotated datasets is a very tedious and costly process. In this project, we aim to build a system to let a drone learn how to fly aggressively in a complex environment by letting the drone interact with its surroundings in a safe way. **Requirements**: Computer vision knowledge; programming experience with python. Machine learning knowledge is a plus but it is not required.

Goal: The goal of this project consists of building a system which can control a drone in complex environments.

Contact Details: Antonio Loquercio, _loquercio@ifi.uzh.ch_

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Target following on nano-scale UAV - Available

Description: Autonomous Unmanned Aerial Vehicles (UAVs) have numerous applications due to their agility and flexibility. However, navigation algorithms are computationally demanding, and it is challenging to run them on-board of nano-scale UAVs (i.e., few centimeters of diameter). This project focuses on the object tracking, (i.e., target following) on such nano-UAVs. To do this, we will first train a Convolutional Neural Network (CNN) with data collected in simulation, and then run the aforementioned network on a parallel ultra-low-power (PULP) processor, enabling flight with on-board sensing and computing only. **Requirements**: Knowledge of python, cpp and embedded programming. Machine learning knowledge is a plus but it is not strictly required.

Contact Details: Antonio Loquercio, _loquercio@ifi.uzh.ch_

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Video Reconstruction from Events - Available

Description: Event cameras have a number of advantages over standard frame-based cameras. Two of them are the high-dynamic range and high temporal resolution. In previous work, we have successfully reconstructed images from a stream of events (see: https://youtu.be/eomALySSGVU). Now, we want to take a step further and improve this pipeline to improve the overall quality of the reconstruction. Applications range from computational photography/videography to calibration of event cameras. This project requires previous experience in machine learning as well at least one course in computer vision. During the project, you will have the opportunities to design novel deep learning architectures tailored to event-based vision and image reconstruction. Contact us for more details.

Goal: The goal of this project is to extract high-dynamic range, high-frame rate video from a stream of events.

Contact Details: Mathias Gehrig (mgehrig at ifi.uzh.ch)

Thesis Type: Master Thesis

See project on SiROP

Computational Photography and Videography - Available

Description: Computational Photography is a hot topic in computer vision because it finds widespread applications in mobile devices. Traditionally, the problem has been studied using frames from a single camera. Today, mobile devices feature multiple cameras and sensors that can be combined to push the frontier in computational photography and videography. In previous work (https://youtu.be/eomALySSGVU), we have successfully reconstructed high-speed, HDR video from events. In this project, we aim for combining information from a standard and event camera to exploit their complementary nature. Applications range from high-speed, HDR video to deblurring and beyond. Contact us for more details.

Contact Details: Mathias Gehrig (mgehrig at ifi.uzh.ch); Daniel Gehrig (dgehrig at ifi.uzh.ch)

Thesis Type: Master Thesis

See project on SiROP

Optimization for Spiking Neural Networks - Available

Description: Spiking neural networks (SNNs) are closely inspired by the extremely efficient computation of brains. Unlike artificial neural networks, it processes information using accurate timing of events/spikes. Together with event-cameras, SNNs show promise to both lower latency and computational burden compared to artificial neural networks. In recent years, researchers have proposed several methods to estimate gradients of SNN parameters in a supervised learning context. In practice, many of these approaches rely on assumptions that lead to unknown consequences in the learning process. Requirements: - Background in machine learning; especially deep learning - Good programming skills; experience in CUDA is a plus.

Goal: In this project we aim to establish a principled framework for gradient-based optimization for spiking neural networks. As a first step, we evaluate recently proposed methods on real-world relevant tasks. Next, we extend previous work to take into previously ignored properties of spiking networks. Finally, the new approach will be compared to previous methods for validation. If progress allows, we will apply this approach to robotics and computer vision problems to demonstrate real-world applicability.

Contact Details: Mathias Gehrig, mgehrig (at) ifi (dot) uzh (dot) ch; Daniel Gehrig, dgehrig (at) ifi (dot) uzh (dot) ch

Thesis Type: Semester Project / Master Thesis

See project on SiROP