Student Projects


How to apply

To apply, please send your CV, your Ms and Bs transcripts by email to all the contacts indicated below the project description. Do not apply on SiROP . Since Prof. Davide Scaramuzza is affiliated with ETH, there is no organizational overhead for ETH students. Custom projects are occasionally available. If you would like to do a project with us but could not find an advertized project that suits you, please contact Prof. Davide Scaramuzza directly to ask for a tailored project (sdavide at ifi.uzh.ch).


Upon successful completion of a project in our lab, students may also have the opportunity to get an internship at one of our numerous industrial and academic partners worldwide (e.g., NASA/JPL, University of Pennsylvania, UCLA, MIT, Stanford, ...).



Reinforcement Learning for Drone Racing - Available

Description: In drone racing, human pilots navigate quadrotor drones as quickly as possible through a sequence of gates arranged in a 3D track. Inspired by the impressive flight performance of human pilots, the goal of this project is to train a deep sensorimotor policy that can complete a given track as fast as possible. To this end, the policy directly predicts low-level control commands from noisy odometry data. Provided with an in-house drone simulator, the student investigates state-of-the-art reinforcement learning algorithms and reward designs for the task of drone racing. The ultimate goal is to outperform human pilots on a simulated track. Applicants should have strong experience in C++ and python programming. Reinforcement learning and robotics background are required.

Goal: Find the fastest possible trajectory through a drone racing track using reinforcement learning. Investigate different reward formulations for the task of drone racing. Compare the resulting trajectory with other trajectory planning methods, e.g., model-based path planning algorithms or optimization-based algorithms.

Contact Details: Yunlong Song (song (at) ifi (dot) uzh (dot) ch), Elia Kaufmann (ekaufmann (at) ifi (dot) uzh (dot) ch)

Thesis Type: Semester Project

See project on SiROP

3rd Person View Imitation Learning - Available

Description: Manually programming robots to carry out specific tasks is a difficult and time consuming process. A possible solution to this problem is to use _imitation learning_, in which a robot aims to imitate a teacher, e.g., a human, that knows how to perform the task. Usually, the teacher and the learner share the same point of view on the problem. However, this last assumption might not be necessary. As humans, for example, we learn to cook by looking at others cooking. During this project, we will explore the possibility of repeating such a kind of 3rd person view _imitation learning_ with flying robots on a navigation task.

Goal: The project aims to develop machine learning based techniques that will enable a drone to learn flying by looking at an other robot flying.

Contact Details: **Antonio Loquercio**: loquercio@ifi.uzh.ch

Thesis Type: Semester Project / Bachelor Thesis / Master Thesis

See project on SiROP

Safe Reinforcement Learning for Robotics - Available

Description: Reinforcement Learning (RL) has recently emerged has a technique to let robots learn by their own experience. Current methods for RL are very data-intensive, and require a robot to fail many times before actually accomplishing their goal. However some systems, such as flying robots, require to respect safety constraints during learning and/or deployment. While maximizing performance, those methods usually aim to minimize the number of system failures and overall risk.

Goal: During this project, we will develop machine learning based techniques to let a (real) drone learn to fly nimbly through gaps and gates, while minimizing the risk of critical failures and collisions.

Contact Details: **Antonio Loquercio** loquercio@ifi.uzh.ch

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Safe Simulation to Real World Transfer - Available

Description: Recent techniques based on machine learning enabled robotics system to perform many difficult tasks, such as manipulation or navigation. Those techniques are usually very data-intensive, and require simulators to generate enough training data. In this project, we will develop techniques to formalize the notion of simulation to reality transfer in a robotics setting. In particular, we are interested in finding performance guarantees to bound the performance drop usually encountered when deploying a policy trained in simulation on a physical platform.

Goal: The project aims to develop techniques based on machine learning to have maximal knowledge transfer between simulated and real world on a general robotic task.

Contact Details: **Antonio Loquercio** loquercio@ifi.uzh.ch

Thesis Type: Semester Project / Bachelor Thesis / Master Thesis

See project on SiROP

3D reconstruction with event cameras - Available

Description: Event cameras are bio-inspired sensors that offer several advantages, such as low latency, high-speed and high dynamic range, to tackle challenging scenarios in computer vision. Research on structure from motion and multi-view stereo with images has produced many compelling results, in particular accurate camera tracking and sparse reconstruction. Active sensors with standard cameras like Kinect have been used for dense scene reconstructions. Accurate and efficient reconstructions using event-camera setups is still an unexplored topic. This project will focus on solving the problem of 3D reconstruction using active perception with event cameras​ .

Goal: The goal is to develop a system for accurate mapping of complex and arbitrary scenes using depth acquired by an event camera setup. Preferred candidate should have knowledge of computer vision, specifically 3D reconstruction and SLAM.

Contact Details: Manasi Muglikar, muglikar (at) ifi (dot) uzh (dot) ch, Javier Hidalgo-Carrió (jhidalgocarrio@ifi.uzh.ch)

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Event-based depth estimation​ - Available

Description: Depth estimation plays an important role in many computer vision and robotics applications, such as augmented reality, navigation, or industrial inspection. Structured light (SL) systems estimate depth by actively projecting a known pattern on the scene and observing with a camera how light interacts (i.e., deforms and reflects) with the surfaces of the objects. This project will focus on event-based depth estimation using structured light systems. The resulting approach would make structured light systems suitable for generating high-speed scans.

Goal: The goal is to develop a system for 3D depth maps with event cameras. Preferred candidate should have knowledge of computer vision

Contact Details: Manasi Muglikar, muglikar (at) ifi (dot) uzh (dot) ch

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Event camera calibration - Available

Description: Camera calibration is an important prerequisite for 3D computer vision tasks. Calibration techniques currently used for event cameras require a special calibration target with blinking pattern. This project focuses on developing a toolkit to calibrate an event camera using a simple calibration target. The project will build on existing techniques for calibration and provide a well-packaged solution for the event-camera calibration task.

Goal: The goal of this project is to develop and evaluate calibration toolbox for event camera.

Contact Details: Manasi Muglikar, muglikar (at) ifi (dot) uzh (dot) ch , Mathias Gehrig, mgehrig (at) ifi (dot) uzh (dot) ch

Thesis Type: Semester Project / Bachelor Thesis / Master Thesis

See project on SiROP

Learning features for efficient deep reinforcement learning - Available

Description: The study of end-to-end deep learning in computer vision has mainly focused on developing useful object representations for image classification, object detection, or semantic segmentation. Recent work has shown that it is possible to learn temporally and geometrically aligned keypoints given only videos, and the object keypoints learned via unsupervised learning manners can be useful for efficient control and reinforcement learning.

Goal: The goal of this project is to find out if it is possible to learn useful features or intermediate representation s for controlling mobile robots in high-speed. For example, can we use the Transporter (a neural network architecture) for finding useful features in an autonomous car racing environment? if so, can we use these features for discovering an optimal control policy via deep reinforcement learning? **Required skills:** Python/C++ reinforcement learning, and deep learning skills.

Contact Details: Yunlong Song (song@ifi.uzh.ch)

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Brain-Body-Drone Interface - Available

Description: Brain- and body-computer interfaces not only allow motion-impaired individuals to use machines - such as drones - but can increase the bandwidth of information transmission between abled-body users and machines, thereby improving performance in challenging tasks such as drone racing. This project aims at collecting a multimodal dataset (i.e., eye movements, electrical brain signals, manual control inputs, drone state) from drone racing pilots. These data will be used for training machine learning classifiers to predict future drone states, trajectories, and control commands. Successful classifiers will be evaluated in real-time using a high-quality drone racing simulator and real-world drone racing. The student will learn about experiment design and multimodal data collection (including eye-tracking and electroencephalography) in human subjects, perform classifier selection, and real-time evaluation. Requirements: Excellent knowledge in Python, Pytorch, Matlab; Interest in human subjects research; Experience with CPP, brain-computer interfaces, eye-tracking is a plus but not strictly necessary.

Goal: The goal of this project is to collect a multimodal dataset to develop brain- and body-machine interfaces for drone racing.

Contact Details: Christian Pfeiffer (cpfeiffe@ifi.uzh.ch)

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Visual-inertial odometry and Event dataset for Drone Racing - Available

Description: Vision-based autonomous drone racing requires Visual-Inertial Odometry (VIO) algorithms that perform well on large optical flow and motion blur. Existing benchmark datasets often fail to address fast and aggressive trajectories as observed in drone racing or lack high-resolution ground-truth poses. This project aims at collecting an extension of the UZH-FPV Drone Racing dataset [https://fpv.ifi.uzh.ch/], previously used for international VIO competitions. The student will learn about VIO and event data collection in a large-scale position tracking arena, multi-camera-IMU calibration and time-synchronization of multimodal data, and will compare state-of-the-art VIO algorithms on this new dataset. Requirements: Experience with Linux, ROS, Python; CPP knowledge is a plus; Prior experience in quadrotor flight is a plus but not strictly necessary.

Goal: The goal of this project is to collect a visual-inertial and event dataset for drone racing and curate the data to meet the standards of international VIO competitions.

Contact Details: Christian Pfeiffer (cpfeiffe@ifi.uzh.ch), Giovanni Cioffi (cioffi@ifi.uzh.ch)

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Rotor-Failure Recovery MPC - Available

Description: A multitude of advanced control techniques for quadrotors have matured over the recent years, with the most promising one being Model Predictive Control. While drones nowadays display high robustness and impressive flight capabilities, they’re not resilient to all possible failures. Especially in the case where a motor failure occurs most controllers struggle to keep the drone stable. Since the quadrotor is already an underactuated vehicle, the loss of one rotor also implies the loss of control over one degree-of-freedom, significantly changing the system dynamics. However, MPC provides some elegant methods to catch such failures. This thesis will investigate the possibilities in controlling a vehicle under rotor failure, design a control system within an existing flight software stack and use our latest drone hardware. The requirements are a basic knowledge of control systems, preliminary experience in optimal control, such as MPC, and being familiar with C++ programming.

Goal: The goal is to develop a solution using Model Predictive Control to catch a rotor failure, demonstrate it on an existing real quadrotor platform, and compare it to other existing approaches.

Contact Details: Philipp Föhn (foehn at ifi.uzh.ch), Sihao Sun (sun at ifi.uzh.ch)

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Probabilistic System Identification of a Quadrotor Platform - Available

Description: Most planning & control algorithms used on quadrotors make use of a nominal model of the platform dynamics to compute feasible trajectories or generate control commands. Such models are derived using first principles and typically cannot fully capture the true dynamics of the system, leading to sub-optimal performance. One appealing approach to overcome this limitation is to use Gaussian Processes for system modeling. Gaussian Process regression has been widely used in supervised machine learning due to its flexibility and inherent ability to describe uncertainty in the prediction. This work investigates the usage of Gaussian Processes for uncertainty-aware system identification of a quadrotor platform. Requirements: - Machine learning experience preferable but not strictly required - Programming experience in C++ and Python

Goal: Implement an uncertainty-aware model of the quadrotor dynamics, train and evaluate the model on simulated and real data.

Contact Details: Elia Kaufmann (ekaufmann@ifi.uzh.ch)

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Learning-Guided MPC Flight - Available

Description: Model predictive control (MPC) is a versatile optimization-based control method that allows to incorporate constraints directly into the control problem. The advantages of MPC can be seen in its ability to accurately control dynamical systems that include large time delays and high-order dynamics. Recent advances in compute hardware allow to run MPC even on compute-constrained quadrotors. While model-predictive control can deal with complex systems and constraints, it still assumes the existence of a reference trajectory. With this project we aim to guide the MPC to a feasible reference trajectory by using a neural network that directly predicts from camera images an expressive intermediate representation. Such tight coupling of perception and control would allow to push the speed limits of autonomous flight through cluttered environments. Requirements: - Machine learning experience (TensorFlow and/or PyTorch) - Experience in MPC preferable but not strictly required - Programming experience in C++ and Python

Goal: Evaluate different intermediate representations for autonomous flight. Implement the learned perception system in simulation and integrate the predictions into an existing MPC pipeline. If possible, deploy on a real system.

Contact Details: Elia Kaufmann (ekaufmann@ifi.uzh.ch) Philipp Föhn (foehn@ifi.uzh.ch)

Thesis Type: Master Thesis

See project on SiROP

Vision for human-piloted Drone Racing - Available

Description: Human drone pilots use a single First-Person-View camera to gather information about the drone’s pose and the immediate visual environment. This project aims to identify which type(s) of visual information are necessary and minimally sufficient for performing a drone racing task successfully. The student will investigate the effects of active vs restrained eye movements, large vs narrow camera field of view, and central vs peripheral vision on the flight performance of professional pilots using a high-quality drone racing simulator. The student will learn about experiment design and data collection in human subjects (i.e., eye tracking, control commands, drone state, video frames) and statistical analyses of these behavioral and physiological time-series data. Requirements: Strong Python or Matlab skills; Interest in human-subjects research; Pytorch, Eye tracking, and drone flight experience is a plus but not strictly necessary.

Goal: The goal is to conduct a human-subjects experiment to identify the effects of active vs. restrained eye movements, large vs. narrow camera field of view, and central vs. peripheral vision on flight performance in drone racing.

Contact Details: Christian Pfeiffer (cpfeiffe@ifi.uzh.ch)

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Events and Lidar For Autonomous Driving - Available

Description: Billions of dollars are spent each year to bring autonomous vehicles closer to reality. One of the remaining challenges is the design of reliable algorithms that work in a diverse set of environments and scenarios. At the core of this problem is the choice of sensor setup. Ideally, there is a certain redundancy in the setup while each sensor should also excel at a certain task. Sampling-based sensors (e.g. LIDAR, standard cameras, etc.) are today's essential building blocks of autonomous vehicles. However, they typically oversample far-away structure (e.g. building 200 meters away) and undersample close structure (e.g. fast bike crossing in front of the car). Thus, they enforce a trade-off between sampling frequency and computational budget. Unlike sampling-based sensors, event cameras capture change in their field-of-view with precise timing and do not record redundant information. As a result, they are well suited for highly dynamic scenarios such as driving on roads. We are currently building a large-scale dataset with high-resolution event cameras and 128-beam Lidar that is targeting object detection and tracking. This project builds on top of already existing hardware that we have built and tested in the last year. In this project, it will be extended with state-of-the-art sensors.

Goal: In this project, we explore the utility of event cameras in an autonomous car scenario. In order to achieve this, a high-quality driving dataset with state-of-the-art Lidar and event cameras will be created. Depending on the progress, the prospective student will work on building novel 3D object detection pipelines on the dataset. We seek a highly motivated student with the following minimum qualifications: - Experience with programming microcontrollers or motivation to acquire it quickly - Excellent coding skills in Python and C++ - At least one course in computer vision (multiple view geometry) - Strong work ethic - Excellent communication and teamwork skills Preferred qualifications: - Background in robotics and experience with ROS - Experience with machine learning - Experience with event-based vision Contact us for more details.

Contact Details: Mathias Gehrig (mgehrig at ifi.uzh.ch); Daniel Gehrig (dgehrig at ifi.uzh.ch) Please add CV + transcripts (Bachelor and Master)

Thesis Type: Semester Project / Internship / Master Thesis

See project on SiROP

MPC for high speed trajectory tracking - Available

Description: Many algorithms exist for model predictive control for trajectory tracking for quadrotors and equally many implementation advantages and disadvantages can be listed. This thesis should find the main influence factors on high speed/high precision trajectory tracking such as: modell accuracy, aerodynamic forces modelling, execution speed, underlying low-level controllers, sampling times and sampling strategies, noise sensitivity or even come up with a novel implementation.

Goal: The end-goal of the thesis should be a comparison of the influence factors and based on that a recommendation or even implementation of an improved solution.

Contact Details: Philipp Föhn (foehn at ifi.uzh.ch)

Thesis Type: Master Thesis

See project on SiROP

Generation of Fast or Time-Optimal Tracjectories for Quadrotor Flight - Available

Description: With the rise of complex control and planning methods, quadrotors are capable of executing astonishing maneuvers. While generating trajectories between two known poses or states is relatively simple, planning through multiple waypoints is rather complicated. The master class of this problem is the task of flying as fast as possible through multiple gates, as done in drone racing. While humans can perform such fast racing maneuvers at extreme speeds of more than 100 km/h, algorithms struggle with even planning such trajectories. Within this thesis, we want to research methods to generate such fast trajectories and work towards a time-optimal planner. This requires some prior knowledge in at least some of the topics including: planning for robots, optimization techniques, model predictive control, RRT, and quadrotors or UAVs in general. The tasks will reach from problem analysis, approximation, and solution concepts to implementation and testing in simulation with existing software tools.

Goal: The goal would be to analyse the planning problem, develop approximation techniques and solve it as time-optimal as possible during thesis.

Contact Details: Philipp Föhn (foehn at ifi.uzh.ch)

Thesis Type: Master Thesis

See project on SiROP

High Performance Computation for Non-Linear Estimation - Available

Description: Visual-inertial odometry (VIO) has matured and became the go-to solution for mobile robot state estimation. There exists a variety of VIO pipelines, reaching from computationally-efficient filter-based approaches to more accurate optimization-based sliding window estimators. However, such sliding window estimators are often relatively slow and introduce significant latency. We are developing a novel optimization-based backend with focus on low latency estimation. This thesis will focus on improving the computational efficiency of our new backend by analyzing the existing framework, reporting bottlenecks, and implementing new optimization strategies or exploiting existing acceleration libraries (such as BLAS, LAPACK or higher level libraries). There are multiple target platforms, including small ARM single-board computers on drones, on which the final solution will be demonstrated. Strong c++ knowledge is needed and experience in numerical optimization is welcome. While there is a high focus on code development, there are options for theoretical contributions as well.

Goal: The goal of this thesis is to analyze and profile an existing optimization backend for visual-inertial state estimation and propose, implement, and benchmark improvements for low-latency execution on handheld devices and drones. These improvements can be code optmiziations, combination with existing libraries for sparse and accelerated linear algebra, or theoretical advances.

Contact Details: Philipp Föhn (foehn at ifi.uzh.ch)

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Deep learning based motion estimation from events - Available

Description: Optical flow estimation is the mainstay of dynamic scene understanding in robotics and computer vision. It finds application in SLAM, dynamic obstacle detection, computational photography, and beyond. However, extracting the optical flow from frames is hard due to the discrete nature of frame-based acquisition. Instead, events from an event camera indirectly provide information about optical flow in continuous time. Hence, the intuition is that event cameras are the ideal sensors for optical flow estimation. In this project, you will dig deep into optical flow estimation from events. We will make use of recent innovations in neural network architectures and insights of event camera models to push the state-of-the-art in the field. Contact us for more details.

Goal: The goal of this project is to develop a deep learning based method for dense optical flow estimation from events. Strong background in computer vision and machine learning required.

Contact Details: Mathias Gehrig, mgehrig (at) ifi (dot) uzh (dot) ch

Thesis Type: Master Thesis

See project on SiROP

Asynchronous Processing for Event-based Deep Learning - Available

Description: Event cameras such as the Dynamic Vision Sensor (DVS) are recent sensors with large potential for high-speed and high dynamic range robotic applications. Since their output is sparse traditional algorithms, which are designed for dense inputs such as frames, are not well suited. The goal of this project is explore ways to adapt existing deep learning algorithms to handle sparse asynchronous data from events. Applicants should have experience in C++ and python deep learning frameworks (tensorflow or pytorch), and have a strong background in computer vision.

Goal: The goal of this project is explore ways to adapt existing deep learning algorithms to handle sparse asynchronous data from events.

Contact Details: Daniel Gehrig (dgehrig at ifi.uzh.ch)

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Computational Photography and Videography - Available

Description: Computational Photography is a hot topic in computer vision because it finds widespread applications in mobile devices. Traditionally, the problem has been studied using frames from a single camera. Today, mobile devices feature multiple cameras and sensors that can be combined to push the frontier in computational photography and videography. In previous work (https://youtu.be/eomALySSGVU), we have successfully reconstructed high-speed, HDR video from events. In this project, we aim for combining information from a standard and event camera to exploit their complementary nature. Applications range from high-speed, HDR video to deblurring and beyond. Contact us for more details.

Contact Details: Mathias Gehrig (mgehrig at ifi.uzh.ch); Daniel Gehrig (dgehrig at ifi.uzh.ch)

Thesis Type: Master Thesis

See project on SiROP

Designing an Event Camera for Learning - Available

Description: Event cameras such as the Dynamic Vision Sensor (DVS) are recent sensors with a lot of potential for high-speed and high-dynamic-range robotic applications.They have been successfully applied in many applications, such as high speed video and high speed visual odometry. Recently, many new event cameras have been commercialized with higher and higher spatial resolutions and high temporal resolution. However, these developments steadily increase the the computational requirements for downstream algorithms, increasing the necessary bandwidth and reducing the time available to process events. In this work we want to find out how important these design parameters are for deep learning applications. Applicants should have experience in coding image processing algorithms in C++ and experience with learning frameworks in python such as tensorflow or pytorch.

Goal: The goal of this project is to find out how important the design parameters of event cameras, such as spatial and temporal resolution, are for deep learning applications.

Contact Details: Daniel Gehrig (dgehrig (at) ifi (dot) uzh (dot) ch), Antonio Loquercio (antonilo (at) ifi (dot) uzh (dot) ch)

Thesis Type: Semester Project / Internship / Master Thesis

See project on SiROP

Designing a New Event Camera with Events and Images - Available

Description: Event cameras such as the Dynamic Vision Sensor (DVS) are recent sensors with a lot of potential for high-speed and high dynamic range robotic applications. They have been successfully applied in many applications, such as high speed video and high speed visual odometry. Due to their high speed and

Goal: The goal of this project is to design a new event camera that combines events and standard images.

Contact Details: Daniel Gehrig (dgehrig (at) ifi (dot) uzh (dot) ch), Mathias Gehrig (mgehrig (at) ifi (dot) uzh (dot) ch)

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Online trajectory re-planning using Gaussian Processes - Available

Description: Online re-planning may be necessary to guarantee safe autonomous navigation of quadrotors in unknown environments. Gaussian processes (GPs), which are machine learning models used in tasks such as regression and classification, have been proposed in robotics to solve the planning problem. GPs are an interesting tool for motion planning since they give a probabilistic perspective on the problem by formulating it as probabilistic inference. Such inference can be performed quickly, which allows to efficiently and incrementally re-plan the current trajectory. In this project, we will use GPs for planning trajectories for landing of a quadrotor platform in scenarios where safety is a key requirement (e.g., in proximity to humans, on hazardous surfaces) and investigate the advantages and disadvantages of such models compared to polynomial trajectories. Requirements: - Experience in Machine learning preferable - Experience in motion planning in robotics preferable - Programming experience in C++ and Python.

Goal: The final goal of this thesis is to benchmark GPs motion planning against polynomial trajectories in simulation. If promising results are achieved, we will deploy the algorithm developed in this thesis on a real system.

Contact Details: Giovanni Cioffi (cioffi@ifi.uzh.ch)

Thesis Type: Semester Project

See project on SiROP

Line tracking using event cameras - Available

Description: Event cameras are recent sensors with large potential for high speed and high dynamic range robotic applications. Recent works have shown promising results in using event cameras on board resource constrained platforms such as quadrotors to perform tasks where the standard cameras usually fail, for example due to fast motion and challenging illumination conditions. In this thesis, we will use state-of-the-art methods in event based vision and/or adapt standard computer vision algorithms to develop a reliable line tracking system. A successful thesis will lead to the deployment of the developed algorithm on a real quadrotor platform for power-line inspection. Requirements: - Experience with event cameras preferable but not required - Passionate about robotics - Programming experience in C++ and Python.

Goal: In this project we will develop a light-weight event-based line tracking algorithm and deploy on a real quadrotor.

Contact Details: Giovanni Cioffi (cioffi@ifi.uzh.ch) and Javier Hidalgo-Carrió (jhidalgocarrio@ifi.uzh.ch)

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Active Perception for Motion Planning - Available

Description: Fast local re-planning is necessary for safe autonomous navigation in the presence of unknown and/or moving obstacles. Recent works have raised interest for active perception methods in motion planning. Such methods include perception awareness in the planning problem. However, how to represent perception awareness is an open research problem. It depends on the representation of the environment. Key requirements for a suitable environment representation are that it can be incrementally updated on-line, light-weight, and it contains confidence measurements on the occupancy.

Goal: In this thesis, we will evaluate commonly used mapping methods (e.g., voxel map, point-clouds, ...) for the local re-planning problem. The final goal is to based on such evaluation a recommendation or even implementation of a novel mapping solution. Requirements: - Experience with mapping in robotics preferable but not required - Programming experience in C++ and Python.

Contact Details: Giovanni Cioffi (cioffi@ifi.uzh.ch)

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Bringing Thermal Cameras into Robotics - Available

Description: Thermographic cameras can capture detailed images regardless of ambient lighting conditions.They use an infrared (IR) sensing technology to map heat variations within the sensor’s range and field-of-view, providing movement detection and hot-spot mapping even in total darkness. Visible range covers wavelengths of approximately 400 – 700 nanometres (nm) in length. However, thermographic cameras generally sample thermal radiation from within the longwave infrared range(approximately 7,000 – 14,000 nm) with a great potential in robotics. Thermography images are useful to identify week points on the power line, along the cable and on the isolators or containers. However, current lightweight thermal cameras are unexplored, with limited in pixel resolution (32x32 pixels) unable to deliver exceptional sensitivity, resolution and image quality for meaningful applications. This work aims to expand the frontiers of computer vision by using thermographic cameras and investigate their application in robotics i.e. perception, state estimation and path planning. The project will combine traditional computer vision techniques together with deep-learning approaches to bring thermography images into the field of robotics. Requirements: Background in computer vision and machine learning - Deep learning experience preferable – Excellent programming experience in C++ and Python

Goal: Perception, state estimation or path planning using thermographic cameras.

Contact Details: Javier Hidalgo-Carrió (jhidalgocarrio@ifi.uzh.ch) and Giovanni Cioffi (cioffi@ifi.uzh.ch)

Thesis Type: Semester Project / Master Thesis

See project on SiROP

Perception Aware Model Predictive Control for Autonomous Power Line Tracking - Available

Description: Classical power line inspection and maintenance are dangerous, costly and time consuming. Drones could mitigate the risk for humans and minimize the cost for the direct benefit of the power line infrastructure. Coupling perception and path planning with control has become increasingly popular in aerial vehicles. This project will continue to investigate vision-based navigation tightly couple approaches of perception and control to satisfy the drone dynamics and compute feasible trajectories with respect to the input saturation. This involves to further development the research on perception aware Model Predictive Control (MPC) for quadrotors, solving the challenging aspects of a power line inspection scenario. A perception aware MPC approach would ideally improve the aerial vehicle's behavior during an approaching maneuver.

Goal: Pursue research on a unified control and planning approach that integrates the action and perception objectives to satisfy this functionality. The functionality will accomplish to the challenging environmental conditions and the morphology of the target which imposes several points of interest for the optimizer.

Contact Details: Javier Hidalgo-Carrió (jhidalgocarrio@ifi.uzh.ch) and Yunlong Song (song@ifi.uzh.ch)

Thesis Type: Bachelor Thesis / Master Thesis

See project on SiROP

Power-Line Dataset for Autonomous Drone Inspection - Available

Description: Classical power line inspection and maintenance are dangerous, costly and time consuming. Drones could mitigate the risk for humans and minimize the cost for direct benefit of the infrastructure. Several sensing capabilities has been already tested (i.e. RGB, LiDAR) which gives the drone the abilities to operate in unstructured environments. Sensor fusion is a popular technique to get the best of each sensor for autonomous navigation. Benchmark of perception strategies is a key part for solid and robust algorithm development before final deployment on the system. However, the lack of relevant and accurate data for multiple sensors makes difficult the Verification and Validation (V&V) process of perception algorithms. The goals of this project is to deliver the first multi sensor power-line inspection dataset for drones with alternative sensory data and ground truth. Requirements: Background in robotics and autonomous systems – Drone navigation preferable – Excellent programming in C++ and Python – Knowledge of ROS and robotic middle-ware - Passionate about robotics and engineering in general. - Linux

Goal: Release an open-access dataset for the evaluation of perception pipelines for autonomous drones. The goal is to establish a solid benchmark for autonomous drone inspection of power-lines . The following sensors are consider to be part of the dataset: - Absolute depth information - RGB images - Event-based camera - Thermography - Inertial Sensory information - Ground truth positioning

Contact Details: Javier Hidalgo-Carrió (jhidalgocarrio@ifi.uzh.ch) and Giovanni Cioffi (cioffi@ifi.uzh.ch)

Thesis Type: Bachelor Thesis / Master Thesis

See project on SiROP

Embedded systems development with NVIDIA Jetson TX2 for fast drone flying - Available

Description: The TX2 is a powerful computational unit with 2 Denver 64-bit CPUs + Quad-Core A57 Complex, NVIDIA Pascal™ Architecture GPU. We use image processing and IMU data in order to deploy our machine learning algorithms in real life experiments. This makes our robots fly autonomously without any help of external communication. In a first iteration, the objective is to have a fully functional connector board to the TX2 including power management, USB OTG, USB 3.0, 2 UARTs, (1 Serial port) Ethernet and a CSI (camera connector). In a second iteration we are redesigning the hardware, integrating our own Obvio-board (time synchronized IMU and RGB) and our own flight controller (integration of an ARM STM32 MyC).

Goal: Test and verify the existing prototype in collaboration with our lab engineers. Create a second iteration with the integration of a microcontroller in order to integrate time synchronization with an IMU and a camera and integrate our custom built flight controller using D-Shot.Applicants should have a solid understanding of Linux device tree for embedded ARM core and some experience using pcb software (kiCAD/Eagle) as well as solid knowledge on communication protocols (UART, USB, SPI, Ethernet).

Contact Details: Manuel Sutter (Systems Engineer BSc) msutter(at)ifi(dot)uzh(dot)ch

Thesis Type: Semester Project / Collaboration / Internship / Bachelor Thesis / Master Thesis

See project on SiROP