Student Projects
How to apply
To apply, please send your CV, your Ms and Bs transcripts by email to all the contacts indicated below the project description. Do not apply on SiROP . Since Prof. Davide Scaramuzza is affiliated with ETH, there is no organizational overhead for ETH students. Custom projects are occasionally available. If you would like to do a project with us but could not find an advertized project that suits you, please contact Prof. Davide Scaramuzza directly to ask for a tailored project (sdavide at ifi.uzh.ch).
Upon successful completion of a project in our lab, students may also have the opportunity to get an internship at one of our numerous industrial and academic partners worldwide (e.g., NASA/JPL, University of Pennsylvania, UCLA, MIT, Stanford, ...).
Deep Learning for Mitral Valve Segmentation from Echocardiography - Available
Description: Mitral valve repair is one of the most challenging cardiac procedures and strongly depends on the surgeon’s experience and ability to interpret echocardiography. Automating the segmentation of cardiac structures and the mitral valve from ultrasound is a key step toward robust, AI-assisted preoperative planning. This project evaluates state-of-the-art deep learning segmentation methods on publicly available echocardiography datasets (e.g., CAMUS, EchoNet-Dynamic, MVSeg2023). Applicants should possess strong Python programming skills and practical experience with deep learning frameworks such as PyTorch or TensorFlow, alongside a solid understanding of computer vision or medical image analysis principles. While not mandatory, prior exposure to nnU-Net, MONAI, or specific cardiac imaging techniques would be considered a strong asset for this project.
Goal: The focus is on building a robust pipeline for cardiac chamber and mitral valve segmentation, and assessing its feasibility as the core of a future clinical decision-support tool. If time permits, the approach can be explored on additional clinical ultrasound data from Clinique du Coeur Lausanne.
Contact Details: Interested candidates should send their CV, transcripts (bachelor and master), and descriptions of relevant projects to Victoria Catalán Pastor (pastor@ifi.uzh.ch), Dr. Isabel Lavanchy (isabel.lavanchy@uzh.ch), and Prof. Davide Scaramuzza (sdavide@ifi.uzh.ch).
Thesis Type: Semester Project / Master Thesis
Combinatorial Optimization for Perception-Aware Flight - Available
Description: This master's thesis/project offers an exciting opportunity to work on real-world vision-based drone navigation in challenging environments (e.g., at night, in deserts, on mountains, or on other planets). The goal is to develop algorithms that enable quadrotors to fly autonomously in sparse feature environments towards a target. Therefore, we aim to plan a path via combinatorial/mixed-integer optimization that incorporates knowledge of simultaneously mapped features. This approach pioneers new possibilities for the future of autonomous robots and builds on the most recent research results from our laboratory. This project is ideal for motivated master’s students interested in robotics, computer vision, optimization (nonlinear/quadratic mixed-integer), and real-world deployment. It offers a rare chance to contribute to a high-impact area, with strong potential for further academic or industrial opportunities.
Goal: The goal is to investigate how to incorporate vision-based localization into an optimization problem and how to approximate it to make it tractable.
Contact Details: Interested candidates should send their CV, transcripts (bachelor and master), and descriptions of relevant projects to Rudolf Reiter (rreiter AT ifi DOT uzh DOT ch)
Thesis Type: Semester Project / Master Thesis
Learning Perception-Aware Navigation Utilizing MPC Layers - Available
Description: Safe and efficient navigation in challenging environments remains a fundamental problem in robotics. Reinforcement learning (RL) methods can learn complex navigation policies but often lack robustness when faced with perception errors or localization drift. On the other hand, model predictive control (MPC) offers strong guarantees in trajectory optimization but struggles with long-term adaptability in uncertain conditions. This project aims to combine the strengths of both approaches by embedding an MPC layer within an RL policy, while explicitly incorporating uncertainty-aware localization. By leveraging uncertainty estimates from perception modules, the navigation system can reason about when and how to rely on MPC corrections, enabling more reliable decision-making in environments with noise, dynamic obstacles, or degraded sensor data. The outcome is a perception-aware navigation framework that adapts to uncertainty while maintaining efficiency and safety.
Goal: The goal of this project is to design and evaluate an RL-based navigation policy augmented with an MPC layer and guided by uncertainty-aware localization. The system will be benchmarked in simulated and/or real-world environments with challenging perception conditions, focusing on metrics such as navigation success rate, robustness to sensor noise, and safety in obstacle-dense scenarios.
Contact Details: Interested candidates should send their CV and transcripts (bachelor’s and master’s) to Rudolf Reiter (rreiter@ifi.uzh.ch), Simone Nascivera (snascivera@ifi.uzh.ch)
Thesis Type: Master Thesis
Codesign of shape and control: A study in autonomous perching - Available
Description: This project establishes an automated co-design approach, specifically focusing on the development of an optimal control policy and the optimization of 3D shapes, targeting static design criteria, such as a desired lift-over-drag ratio for aerodynamic shapes. We are undertaking an ambitious project that aims to extend the concept of shape optimization to design a fully autonomous system. We propose to explore this project via an autonomous glider perching problem described in a paper by MIT [1]. We are looking for a master's student to help us design and build the glider to reproduce the experiments in [1]. In addition to designing and building the glider, the student needs to implement a controller that successfully executes the perching maneuver. This will require performing system identification, implementing motion planning algorithms, and designing tracking controllers. Project outcomes: - Build a physical RC glider for the perching task, made of foam - Develop a control law for the glider (e.g., optimal control/trajectory optimization and MPC) Prerequisites: Currently completing a master's in computer science or mechanical/electrical engineering at UZH or ETH in Switzerland. Prospective applicants need to have been exposed to the following topics through coursework or past projects: - Optimal control, model predictive control - Numerical optimization (convex, linear, quadratic programming) - Familiarity with deep learning and computational fluid dynamics is a plus - Hands-on robotics experience is also a plus The project is cosupervised by Prof. Pascal Fua. References: [1] Moore, Joseph & Cory, Rick & Tedrake, Russ. (2014). Robust post-stall perching with a simple fixed-wing glider using LQR-Trees. Bioinspiration & Biomimetics. 9. 025013. 10.1088/1748-3182/9/2/025013.
Goal: Can we optimize the design of an autonomous system for a complex dynamic maneuver?
Contact Details: Please send your CV, Bachelor's and Master's transcripts to Ming Xu, Mingda.xu@epfl.ch and Rudolf Reiter, rreiter@ifi.uzh.ch
Thesis Type: Master Thesis
Evolutionary Optimization Meets Differentiable Simulation - Available
Description: This master's project offers an exciting opportunity to work on real-world drone flight on the edge of the physical limits. The goal is to develop learning algorithms that enable quadrotors to fly autonomously by directly learning from real-world experience, combining global evolutionary optimization with differentiable simulation. We use visual features obtained from a VIO system as inputs. This approach pioneers new possibilities for the future of robotics and builds on the most recent research results from our laboratory. Differentiable simulation has shown outstanding performance to obtain vision-based policies in the real world, as shown in our recent work “Learning on the Fly: Rapid Policy Adaptation via Differentiable Simulation”. We advance this work by including evolutionary optimization to find globally optimal policies for challenging real-world tasks. This project is ideal for motivated master’s students interested in robotics, machine learning, computer vision, and real-world deployment. It offers a rare chance to contribute to a high-impact area, with strong potential for further academic or industrial opportunities. Applicant Requirements: - Proficiency in reinforcement learning, dynamical systems, computer vision, optimization, and robotics - Strong programming skills with Python, C++, and preferably JAX - Knowledge of simulation software and real-time data processing. - Understanding of drone dynamics, hardware, and control systems.
Goal: The goal is to investigate how the latest differentiable simulation strategies push the limits of learning real-world tasks such as agile flight.
Contact Details: Interested candidates should send their CV, transcripts (bachelor and master), and descriptions of relevant projects to Rudolf Reiter (rreiter AT ifi DOT uzh DOT ch), Jiaxu Xing (jixing AT ifi DOT uzh DOT ch), Yunfan Ren (yunfan AT ifi DOT uzh DOT ch)
Thesis Type: Semester Project / Master Thesis
Neural Quadrotor Dynamics - Available
Description: This project leverages the Neural Robot Dynamics (NeRD) framework to build a high-fidelity, high-speed neural simulator for a quadrotor UAV. Rather than relying solely on classical rigid-body dynamics and hand-crafted aero/actuation models, we will train a neural network to replace key low-level simulation components (e.g., unmodeled dynamics, actuator response, residual forces), enabling faster rollouts without sacrificing accuracy. The simulator will be fine-tuned on real flight logs to learn a vehicle-specific model and reduce the sim-to-real gap. Thanks to the model’s differentiability, the resulting engine also supports differentiable simulation (DiffSim) for gradient-based system identification, trajectory optimization, and policy learning. Ultimately, we aim to accelerate training of advanced flight control policies and improve zero-shot transfer by matching simulation to the target platform’s true dynamics. Applicants should be comfortable with control and learning for dynamical systems, have ML experience (e.g., JAX/PyTorch), and be proficient in C++ and Python.
Goal: Build a quadrotor neural simulator, fine-tune it with real flight data for a drone-specific accurate model, and validate differentiable-simulation (DiffSim) use cases and sim2real transfer improvements.
Contact Details: Interested candidates should send their CV, transcripts (bachelor and master), and descriptions of relevant projects to Yunfan Ren [yunfan (at) ifi (dot) uzh (dot) ch], Jiaxu Xing [jixing (at) ifi (dot) uzh (dot) ch], Ismail Geles [geles (at) ifi (dot) uzh (dot) ch], Prof. Davide Scaramuzza [sdavide (at) ifi (dot) uzh (dot) ch]
Thesis Type: Semester Project / Master Thesis
Drone Racing Meet Differentiable simulation - Available
Description: In this project, we investigate how DiffSim (differentiable simulation) can accelerate learning for drone racing by enabling end-to-end gradient-based training with Backpropagation Through Time (BPTT). Instead of relying solely on sample-inefficient trial-and-error, we use a differentiable simulator to propagate learning signals through the drone dynamics and control pipeline over time, allowing the controller to improve from trajectories much more efficiently. Our objective is to develop a training framework that learns high-speed, gate-to-gate racing behaviors in significantly less wall-clock time, while maintaining stable and agile flight. Applicants should be comfortable with control and learning for dynamical systems, have machine learning experience (e.g., Jax, PyTorch), and be proficient in C++ and Python.
Goal: Reduce training time for drone racing policies using DiffSim + BPTT, and demonstrate fast learning on a variety of racing tracks and flight conditions in simulation and in real-world experiments.
Contact Details: Interested candidates should send their CV, transcripts (bachelor and master), and descriptions of relevant projects to Yunfan Ren [yunfan (at) ifi (dot) uzh (dot) ch], Ismail Geles [geles (at) ifi (dot) uzh (dot) ch], Jiaxu Xing [jixing (at) ifi (dot) uzh (dot) ch], Prof. Davide Scaramuzza [sdavide (at) ifi (dot) uzh (dot) ch]
Thesis Type: Semester Project / Master Thesis
Vision-Based Drone Control with Structured Networks & Symmetry - Available
Description: Vision-based reinforcement learning controllers can achieve impressive drone flight performance, but training them is often slow and data-hungry because standard networks must re-learn the same behaviors across equivalent viewpoints and orientations. In this project, we will speed up vision-based drone control policy learning by using structured (symmetry-aware / equivariant) neural networks that encode physical and geometric symmetries directly into the policy. By enforcing these structure constraints, the controller can generalize better across rotations and scene variations, improving sample efficiency and sim-to-real transfer. Applicants should have a solid understanding of reinforcement learning, machine learning experience (Jax, PyTorch), and programming experience in Python and C++.
Goal: Develop and evaluate a symmetry-aware vision-to-control training pipeline, quantify improvements in learning speed and robustness versus standard baselines, and validate the approach across multiple flight/navigation tasks in simulation and real-world experiments.
Contact Details: Interested candidates should send their CV, transcripts (bachelor and master), and descriptions of relevant projects to Yunfan Ren [yunfan (at) ifi (dot) uzh (dot) ch], Rudolf Reiter [rreiter (at) ifi (dot) uzh (dot) ch], Prof. Davide Scaramuzza [sdavide (at) ifi (dot) uzh (dot) ch]
Thesis Type: Semester Project / Master Thesis
Learning Robust Agile Flight via Adaptive Curriculum - Available
Description: Reinforcement learning-based controllers have demonstrated remarkable success in enabling fast and agile flight. Currently, the training process of these reinforcement learning controllers relies on a static, pre-defined curriculum. In this project, our objective is to develop a dynamic and adaptable curriculum to enhance the robustness of the learning-based controllers. This curriculum will continually adapt in an online fashion based on the controller's performance during the training process. By using the adaptive curriculum, we expect the reinforcement learning controllers to enable more diverse, generalizable, and robust performance in unforeseen scenarios. Applicants should have a solid understanding of reinforcement learning, machine learning experience (PyTorch), and programming experience in C++ and Python.
Goal: Improve the robustness and generalizability of the training framework and validate the method in different navigation task settings. The approach will be demonstrated and validated both in simulated and real-world settings.
Contact Details: Jiaxu Xing (jixing@ifi.uzh.ch), Ismail Geles (geles@ifi.uzh.ch), Prof. Davide Scaramuzza (sdavide@ifi.uzh.ch)
Thesis Type: Semester Project / Master Thesis
Vision-based Navigation in Dynamic Environment via Reinforcement Learning - Available
Description: In this project, the goal is to develop a vision-based policy that enables autonomous navigation in complex, cluttered environments. The learned policy should enable the robot to effectively reach a designated target based on visual input while safely avoiding encountered obstacles. Some of the use cases for this approach will be to ensure a safe landing on a moving target in a cluttered environment or to track a moving target in the wild. Applicants should have a solid understanding of reinforcement learning, machine learning experience (PyTorch), and programming experience in C++ and Python.
Goal: Develop such a policy based on an existing reinforcement learning pipeline. Extend the training environment adapted for the task definition. The approach will be demonstrated and validated both in simulated and real-world settings.
Contact Details: Jiaxu Xing (jixing@ifi.uzh.ch), Yunfan Ren(yunfan@ifi.uzh.ch), Prof. Davide Scaramuzza (sdavide@ifi.uzh.ch)
Thesis Type: Master Thesis
Event camera-based drone landing on a dynamic platform - Available
Description: Commercial drones typically assume that their landing zone is a flat, stationary surface—often a square marker—and they use an RGB camera stream together with SLAM to localize themselves during the final descent, especially when GPS is unavailable. In many real-world scenarios, however, this assumption breaks down. A drone may need to land on a moving vehicle subject to strong vibrations, on an oscillating platform such as a boat, or even on another airborne vehicle like a plane. In these situations, conventional cameras struggle: motion blur and limited responsiveness make accurate localization difficult during fast maneuvers. Event cameras, on the other hand, offer high frame rates and a wide dynamic range, making them well-suited for these demanding conditions. Their ability to capture rapid changes without blurring allows drones to remain stable and responsive even when the landing surface is moving unpredictably. Requirements: - Strong programming skills in C++ and Python. - Background in Computer Vision, State Estimation and Deep Learning. - Experience with PyTorch and developing Deep Learning architectures
Goal: The goal of this project is to design and implement a complete pipeline that leverages an event camera to enable reliable drone landing on challenging, dynamically moving surfaces. The system aims to maintain accurate perception and control even when the landing platform is unstable, vibrating, or undergoing rapid motion.
Contact Details: Interested candidates should send their CV, transcripts (bachelor and master) to: Simone Nascivera [snascivera@ifi.uzh.ch], Roberto Pellerito [rpellerito@ifi.uzh.ch], Ismail Geles [geles@ifi.uzh.ch], Prof. Davide Scaramuzza [sdavide@ifi.uzh.ch]
Thesis Type: Semester Project / Master Thesis
Adaptive Frame Rate Management in Visual-Inertial Odometry Using Reinforcement Learning - Available
Description: Visual–inertial odometry (VIO) pipelines typically process every incoming camera frame to maintain accurate state estimation, leading to high computational load and energy consumption. This project investigates the use of reinforcement learning (RL) to adaptively select which frames should be processed based on inertial measurements and visual feature information. The objective is to learn a policy that predicts whether processing the next frame is necessary to sustain a desired estimation accuracy, or whether propagation using only inertial data is sufficient. The proposed pipeline integrates RL-based decision-making with conventional VIO modules and evaluates how different state representations—such as IMU dynamics, feature tracking statistics, and predicted uncertainty—affect frame-selection performance. Requirements: -Strong programming skills in C++ and Python. - Background in Computer Vision, State Estimation and Deep Learning. - Experience with PyTorch and developing Deep Learning architectures.
Goal: The objective of this project is to develop a pipeline that uses Reinforcement Learning to reduce the frame rate of a visual inertial odometry pipeline using the inertial measurement and map or features statistics as inputs.
Contact Details: Interested candidates should send their CV, transcripts (bachelor and master) to: Simone Nascivera [snascivera@ifi.uzh.ch], Roberto Pellerito [rpellerito@ifi.uzh.ch], Prof. Davide Scaramuzza [sdavide@ifi.uzh.ch]
Thesis Type: Semester Project
Safe Quadrotor Landing from Aggressive Racing Conditions via Reinforcement Learning - Available
Description: This project tackles the challenging problem of autonomous landing for a high-speed quadrotor immediately after completing an aggressive drone-racing maneuver, where reliable initial state estimation is not available. Relying solely on the onboard camera feed and the output of the existing gate-detection algorithm, the drone must safely land anywhere within the track boundaries while avoiding obstacles and preventing hard impacts with the floor. Although the track remains fixed between training and testing, its inherent symmetries and the absence of explicit state estimation make the task particularly difficult. Requirements: - Strong programming skills in C++ and Python. - Experience with ROS/ROS2 and Linux. - Experience with PyTorch and developing Deep - Learning architectures. - Experience with drones or reinforcement learning is a plus
Goal: The main goal is to develop a control policy for quadrotors that enables smooth and reliable landings within the track boundaries, using only the onboard video stream or the existing gate-detection signal.
Contact Details: Interested candidates should send their CV, transcripts (bachelor and master) to: Simone Nascivera [snascivera@ifi.uzh.ch], Ismail Geles [geles@ifi.uzh.ch], Victoria Catalan Pastor [pastor@ifi.uzh.ch], Prof. Davide Scaramuzza [sdavide@ifi.uzh.ch]
Thesis Type: Semester Project
Surgical HDR Imaging with an Event Camera (in collaboration with Balgrist) - Available
Description: In the operating room, strong light sources are used to illuminate the surgical field that enables surgeons to visually detect fine details. However, surgical environments present an extreme high dynamic range (HDR) challenge for technical systems, where standard frame-based cameras suffer from severe overexposure in the surgical site and underexposure in the periphery, compromising many downstream computer vision applications, such as surgical scene understanding, marker-less instrument tracking, or hand tracking. This project proposes a novel neuromorphic sensor fusion framework that combines the absolute intensity information of standard RGB sensors with the high temporal resolution and >120 dB dynamic range of asynchronous event cameras. By leveraging the complementary nature of these modalities, our approach recovers high-frequency texture and motion details in saturated regions where standard sensors blindly clip. We demonstrate that this bio-inspired fusion enables robust, artifact-free HDR image reconstruction and low-latency tracking in harsh lighting conditions, significantly outperforming traditional multi-exposure techniques. For evaluation, we will compare the proposed event-based method with classical image enhancement techniques based on optical images. This project is conducted at RPG and Balgrist under the supervision of Prof. Dr. Fürnstahl.
Goal: The primary goal is to develop a neuromorphic sensor fusion framework that overcomes extreme lighting conditions in surgery, enabling robust HDR image reconstruction and precise, low-latency tracking where standard cameras typically fail due to overexposure. **Requirements**: - Strong programming skills in Python or C++. - Background in Classical Computer Vision and Deep Learning. - Experience with PyTorch and developing Deep Learning architectures.
Contact Details: Interested candidates should send their CV, transcripts (bachelor and master) to: Roberto Pellerito [rpellerito@ifi.uzh.ch], Victoria Catalan Pastor [pastor@ifi.uzh.ch], Matthias Seibold [Matthias.Seibold@balgrist.ch], Lilian Calvet [Lilian.Calvet@balgrist.ch], Prof. Davide Scaramuzza [sdavide@ifi.uzh.ch]
Thesis Type: Semester Project / Master Thesis
Event-based Object Segmentation for Vision-Guided Autonomous Systems - Available
Description: Event cameras offer groundbreaking advantages: microsecond latency, high dynamic range, and sparse asynchronous output, which make them ideal for challenging perception tasks where traditional frame cameras falter (motion blur, low light, high contrast). While much of the current event-based research focuses on reconstruction, VO, or representation learning, precise object segmentation remains largely unexplored. Segmenting dynamic objects in complex environments is critical for downstream tasks such as agile navigation, collision avoidance, and robust control. By leveraging the temporal resolution and sparsity of event data, we aim to advance object segmentation methods that can outperform frame-based alternatives, especially under fast motion or harsh lighting. The primary goal of this project is to design, implement, and rigorously evaluate an end-to-end pipeline that performs object segmentation using event camera data. Success will be measured by segmentation accuracy, temporal consistency, latency, and robustness compared to conventional frame-based segmentation.
Goal: Dataset curation and preprocessing: Collect or simulate paired event streams and segmentation annotations (using existing datasets such as DSEC, EED, or custom setups). Align event data with ground-truth masks via synchronous frame capture or semi-synthetic generation. Method development: Develop or adapt deep learning architectures for segmentation on event data, e.g., spiking networks, graph neural nets, or sparse convolutional networks working on event voxel grids. Investigate hybrid methods that combine event streams with RGB frames to improve segmentation under challenging conditions. Performance evaluation: Quantitatively evaluate segmentation quality using metrics such as IoU, precision, recall, and temporal stability. Benchmark latency and computational efficiency, considering real-time deployment constraints. Robustness testing: Test across scenarios with dynamic lighting (HDR), rapid motion, motion blur, and occlusions. Compare performance to frame-based segmentation baselines. Optional real-time deployment: Integrate the segmentation pipeline onboard a drone or robot. Demonstrate real-time perception on dynamic tasks like object avoidance, tracking, or mapping. **Key Requirement** 1. Strong programming skills in Python, proficiency with PyTorch or similar deep learning frameworks. 2. Background in computer vision, deep learning, and ideally some experience with event-based vision. 3. Familiarity with segmentation tasks and performance evaluation. C++ knowledge for real-time/deployment aspects; ROS familiarity; hardware deployment experience are a plus.
Contact Details: Interested candidates should send their CV, transcripts (bachelor and master) to: Roberto Pellerito [rpellerito@ifi.uzh.ch], Rong Zou [zou@ifi.uzh.ch], Prof. Davide Scaramuzza [sdavide@ifi.uzh.ch]
Thesis Type: Semester Project / Master Thesis
Vision Language Action models for Drones - Available
Description: Designing a generative model for drone flight paths that obey high-level natural language descriptions (e.g. “fast 8 shape”) and pass through specific waypoints is a challenging multi-modal problem. The goal is to produce plausible 3D trajectories offline, which can then be used to train reinforcement learning (RL) policies by imitation or as goal demonstrations. Key challenges include: (1) combining textual commands with spatial constraints (control points) in a single model, (2) obtaining training data of trajectories paired with language descriptions (potentially from videos), and (3) ensuring generated paths are physically feasible and capture the qualitative style indicated by the language (e.g. *“fast”*implies higher speed, “8 shape” implies a figure-eight loop). In this project we will explore suitable model architectures, data sources, trajectory extraction methods, relevant research, and propose a system design to tackle this text-to-trajectory generation task.
Goal: The goal of this project is to design a generative model that can translate high-level natural language descriptions and waypoint constraints into physically feasible 3D drone trajectories, which can then be used to train or guide reinforcement learning policies for autonomous flight. **Key Requirement** 1. Strong programming skills in Python, proficiency with PyTorch or similar deep learning 2. frameworks. Strong background in computer vision, deep learning, and reinforcement learning, MPC and classical control. 3.Understanding of drone dynamics and control systems.
Contact Details: Interested candidates should send their CV, transcripts (bachelor and master) to: Roberto Pellerito [rpellerito@ifi.uzh.ch], Daniel Zhai [dzhai@ifi.uzh.ch], Prof. Davide Scaramuzza [sdavide@ifi.uzh.ch]
Thesis Type: Semester Project / Master Thesis
Vision-Based Tactile Sensor for Humanoid Hands (in collaboration with Soft Robotics Lab) - Available
Description: Humanoid robots are rapidly advancing, with dexterous hand manipulation emerging as a key research frontier. These systems currently rely primarily on vision-based perception for manipulation. However, such approaches face limitations in scenarios where the line of sight is blocked, or when precise force control is critical for stable manipulation. To enable fine-grained and robust manipulation, tactile sensing at multiple points on the fingertip and palm is fundamental. Several tactile sensing strategies exist, but vision-based tactile sensors stand out due to their compactness, low cost, and high spatial resolution. Their performance, however, is limited by the camera bandwidth and power consumption.
Goal: This project proposes the development of a novel event-based tactile sensor, replacing conventional cameras with event cameras. This approach leverages the asynchronous, high-bandwidth, and low-power properties of event-based vision to provide real-time, high-resolution tactile feedback. The ultimate goal is to integrate these sensors into a human-scale robotic hand and validate their effectiveness in dexterous manipulation tasks. The project will focus on building a method for estimating force from videos of a deformable material. **Key Requirement** We are looking for a highly skilled student with: 1. Background in mechatronics, robotics, or a related field. 2. Strong interest in tactile sensing and perception. 3. Strong experience with Deep Learning, in particular: CNNs, RNNs, GNNs and Transformers. 4. Strong experience with sensor characterization. 5. Basic knowledge of event-based vision and tactile sensors are a plus.
Contact Details: If you are interested in working on cutting-edge tactile sensing technologies and contributing to the future of humanoid robotics, please contact us with your CV, transcripts of Bachelor, Master and a small motivational introduction. Roberto Pellerito rpellerito@ifi.uzh.ch, Jaehoon Kim jaehoon.kim@srl.ethz.ch, Prof. Davide Scaramuzza [sdavide@ifi.uzh.ch]
Thesis Type: Semester Project / Master Thesis
Neural Vision for Celestial Landings (in collaboration with European Space Agency) - Available
Description: Event-based cameras offer significant benefits in difficult robotic scenarios characterized by high-dynamic range and rapid motion. These are precisely the challenges faced by spacecraft during landings on celestial bodies like Mars or the Moon, where sudden light changes, fast dynamics relative to the surface, and the need for quick reaction times can overwhelm vision-based navigation systems relying on standard cameras. In this work, we aim to design novel spacecraft navigation methods for the descent and landing phases, exploiting the power efficiency and sparsity of event cameras. Particular effort will be dedicated to developing a lightweight frontend, utilizing asynchronous convolutional and graph neural networks to effectively harness the sparsity of event data, ensuring efficient and reliable processing during these critical phases. The project is in collaboration with European Space Agency at the European Space Research and Technology Centre (ESTEC) in Noordwijk (NL).
Goal: Investigate the use of asynchronous neural networks (either regular or spiking) for building an efficient frontend system capable of processing event-based data in real-time. Experiments will be conducted both pre-recorded dataset as well as on data collected during the project. **Key Requirement** We look for students with strong programming (Pyhton/C++) and computer vision backgrounds. Knowledge in machine learning frameworks (pytorch, tensorflow) is required as well as familiarity with Visual Odometry, SLAM, feature tracking. Previous experience with IMU is a plus.
Contact Details: Interested candidates should send their CV, transcripts (bachelor and master) to: Roberto Pellerito (rpellerito@ifi.uzh.ch), Simone Nascivera (snascivera@ifi.uzh.ch) and Davide Scaramuzza (sdavide@ifi.uzh.ch).
Thesis Type: Master Thesis
SLAM at the Speed of Light with SPAD-based Vision - Available
Description: Single-Photon Avalanche Diodes (SPADs) are emerging as a game-changer for robotics, offering the ability to see in total darkness and capture high-speed motion without blur. Conventional cameras suffer from motion blur during high-speed maneuvers and blindness in low-light or high-dynamic-range (HDR) scenarios. While Event Cameras have revolutionized agile robotics by measuring brightness changes asynchronously, they are limited by their contrast sensitivity. Single-Photon Avalanche Diodes (SPADs) are effectively RGB/event cameras on steroids. They offer the higher microsecond temporal resolution and single-photon sensitivity. However, SPAD data significantly differs from standard camera or event data. Current SLAM pipelines are not optimized for this unique modality.
Goal: The goal of this project is to develop a robust pipeline for Simultaneous Localization and Mapping (SLAM) a high-resolution SPAD sensor. The student will investigate how the high temporal resolution of SPADs can be leveraged to robustly map environments where standard cameras and event cameras fail (e.g., low light, high dynamic range) **Requirements**: - Strong programming skills in Python or C++. - Background in Computer Vision, State Estimation and Deep Learning. - Experience with PyTorch and developing Deep Learning architectures. - Experience with ROS/ROS2 and Linux.
Contact Details: Interested candidates should send their CV, transcripts (bachelor and master) to: Roberto Pellerito [rpellerito@ifi.uzh.ch], Simone Nascivera [snascivera@ifi.uzh.ch], Prof. Davide Scaramuzza [sdavide@ifi.uzh.ch]
Thesis Type: Semester Project / Master Thesis
Spiking Architectures for Advanced Event-Based Temporal Reasoning - Available
Description: Biological neural systems excel at processing information with remarkable efficiency and robustness, largely due to their reliance on the precise timing and dynamic interplay of neural activity. In contrast, many conventional deep learning architectures simplify these temporal dynamics, often overlooking the rich information embedded in the precise timing of events. Event-based cameras offer a unique data stream that mirrors this biological principle, capturing asynchronous "spikes" of visual information in response to scene changes. This project aims to develop novel spiking neural network (SNN) architectures that harness these inherent characteristics of event data. We propose an approach that emphasizes neuron-level temporal processing. Furthermore, we will investigate how collective spiking synchronization can serve as a powerful latent representation for understanding dynamic scenes and sequential patterns. This paradigm seeks to strike a balance between biological plausibility and computational efficiency, leveraging the sparsity and high temporal resolution of event data to achieve robust and interpretable performance in complex, dynamic environments.
Goal: The primary goal is to design, implement, and rigorously evaluate an SNN architecture capable of advanced temporal reasoning on event-based data. This involves developing methods for individual spiking neurons to effectively process their historical event inputs and exploring how emergent synchronization patterns within the network can represent rich contextual information. The project will involve testing the developed models on challenging event-based vision tasks that require sequential understanding, such as gesture recognition, dynamic object tracking, or agile robot navigation. Performance will be assessed in terms of accuracy, computational efficiency, and robustness to noisy or complex event streams, with comparisons to existing event-based learning paradigms. Applicants should possess a strong background in spiking neural networks, deep learning frameworks (PyTorch), computer vision, and programming proficiency in Python. Experience with event cameras or neuromorphic computing is a significant advantage.
Contact Details: Interested candidates should send their CVs and transcripts (bachelor's and master's) to Nikola Zubic (zubic@ifi.uzh.ch), Roberto Pellerito (rpellerito@ifi.uzh.ch), and Davide Scaramuzza (sdavide@ifi.uzh.ch).
Thesis Type: Master Thesis
Rethinking RNNs for Neuromorphic Computing and Event-based Vision - Available
Description: While more recent sequence modeling architectures have gained prominence, traditional Recurrent Neural Networks (RNNs), such as LSTMs and GRUs, remain highly effective for tasks requiring strong state-tracking capabilities and continuous temporal reasoning, which are qualities crucial for processing dynamic time-series data. Event-based cameras, which produce sparse, asynchronous data streams in response to scene changes, generate precisely this kind of highly temporal information. However, efficiently processing these event streams with traditional RNNs, especially on resource-constrained platforms or future neuromorphic hardware, presents significant challenges due to their strictly sequential nature and inherent inefficiencies in current hardware implementations. This project aims to change the deployment of RNNs for event-based vision by developing hardware-aware optimization strategies. We will explore novel parallelization schemes that can process multiple, smaller hidden states concurrently, analogous to how multi-head mechanisms operate, thereby better utilizing modern parallel computing architectures. Furthermore, we will focus on fine-grained kernel optimization, targeting specific hardware characteristics such as internal cache sizes, memory access patterns, and compute handling, to unlock efficiency and throughput for RNNs processing event data. The ultimate goal is to enable RNNs to leverage the advantages of event-based sensors for real-time, low-latency applications.
Goal: The primary goal of this project is to design, implement, and rigorously evaluate highly optimized RNN architectures tailored for efficient processing of event-based vision data, with a strong focus on their potential for neuromorphic computing and modern GPU hardware. This involves developing custom kernels and optimization techniques that exploit the sparsity and asynchronous nature of event streams, alongside parallelization strategies that significantly accelerate RNN inference. The student will benchmark the developed solutions on representative event-based vision tasks (e.g., object detection, optical flow, motion estimation) to demonstrate substantial improvements in processing speed and computational efficiency compared to standard implementations. Applicants should possess strong programming skills in Python and C++, expertise in deep learning frameworks (e.g., PyTorch, JAX), and a solid understanding of RNN architectures. Experience with hardware-level optimization (CUDA, Triton) or neuromorphic computing concepts is highly advantageous.
Contact Details: Interested candidates should send their CVs and transcripts (bachelor's and master's) to Nikola Zubic (zubic@ifi.uzh.ch), Roberto Pellerito (rpellerito@ifi.uzh.ch), and Davide Scaramuzza (sdavide@ifi.uzh.ch).
Thesis Type: Master Thesis
Event Representation Learning for Control with Visual Distractors - Available
Description: Autonomous systems operating in complex, real-world environments often face significant challenges from visual distractors and high-speed dynamics. Traditional frame-based cameras can struggle with motion blur and high-dynamic range, leading to unreliable visual representations for control. Event-based cameras, with their microsecond latency and ability to capture only changes in a scene, offer a promising alternative for robust perception in such demanding scenarios. This project aims to investigate event-based representation learning for control tasks, specifically focusing on environments with static and dynamic visual distractors. Drawing inspiration from benchmarks like the DeepMind Control Vision Benchmark (DMC-VB), we will explore how event data's unique properties, sparsity, and high temporal resolution can be leveraged to learn more robust and efficient control policies. The goal is to develop representations that are inherently less susceptible to visual noise and rapid environmental changes, thereby improving the performance and reliability of autonomous agents.
Goal: The primary goal of this project is to design, implement, and evaluate novel event-based representation learning methods for control tasks, focusing on scenarios with visual distractors. This includes developing techniques to extract meaningful features from sparse event streams that are invariant to static or dynamic visual clutter. The student will work with simulated environments (adapted to generate event data or using existing event-based simulators) to benchmark the developed methods against traditional frame-based approaches. Experiments will cover various locomotion and navigation tasks, assessing the robustness, sample efficiency, and real-time performance of event-based control policies. Applicants should have a strong background in deep learning, reinforcement learning, computer vision, and programming skills in Python (PyTorch/JAX). Experience with event-based vision or control systems is highly beneficial.
Contact Details: Interested candidates should send their CVs and transcripts (bachelor's and master's) to Nikola Zubic (zubic@ifi.uzh.ch), Rong Zou (zou@ifi.uzh.ch), and Davide Scaramuzza (sdavide@ifi.uzh.ch).
Thesis Type: Semester Project / Master Thesis
Learning-Based Decoding for Event Cameras - Available
Description: There are some compact, relative event-stream formats to transmit events efficiently. While compact, this parsing on the CPU is complicated (vectorized addresses, base-state deltas, and protocol-violation handling), and it becomes a bottleneck for high-rate streams. This project proposes to learn a decoder. We target near-lossless decoding (minor discrepancies acceptable for downstream use), prioritizing high throughput and low latency over bit-perfect reconstruction.
Goal: Design, implement, and evaluate a learned decoder. Build paired datasets of byte stream to event tuples mappings, explore streaming sequence models over bytes/words (e.g., compact architectures) that map variable-length packets to event tokens with efficient causal inference. Distill from the reference decoder as a teacher, optimizing for event-level accuracy (position/time/polarity), as well as auxiliary objectives (e.g., protocol violation flags, timestamp shifts). Provide toggles for stricter vs. faster decoding. Implement an optimized baseline (bit-unpacking in CUDA/Triton) and the learned decoder in PyTorch/TensorRT with persistent-kernel style streaming. Evaluation will be conducted by measuring speed/latency, fidelity (mismatch rate to the teacher), and downstream impact (measuring end-to-end gains on standard event tasks once decoded). From candidates, we expect a strong background in deep learning, C++, and Python, and experience with PyTorch or TensorRT. Systems skills (CUDA/Triton) are a plus. Familiarity with event cameras or streaming sequence models is beneficial.
Contact Details: Send CV and transcripts (bachelor’s & master’s) to Nikola Zubic (zubic@ifi.uzh.ch), Leonard Bauersfeld (bauersfeld@ifi.uzh.ch), and Davide Scaramuzza (sdavide@ifi.uzh.ch).
Thesis Type: Semester Project / Master Thesis
High-Speed Object Pickup During Quadrotor Flight with Reinforcement Learning - Available
Description: Imagine a delivery quadrotor flying at high speed. To maximize efficiency and delivery turnover, the quadrotor does not stop to pick up objects. Instead, it picks up the payload during high-speed flight. This poses extreme challenges for robust control, since the forces involved during the pickup are a huge external disturbance that, if a conventional trajectory tracking controller is naively used, will cause severe instability after the pickup. In this project, we approach this issue by modelling the dynamics of the pickup and use reinforcement learning to handle the nonsmooth and nondifferentiable dynamics during object pickup, resulting in an overall smooth pickup trajectory with minimal interruptions to the original planned trajectory. Applicants are expected to be proficient in Python, C++, Pytorch, and Git, preferably with experience with RL and modeling.
Goal: This project aims to train an RL agent to handle the nonsmooth dynamics of object pickup during flight. The project will involve simulating the dynamics of picking up the object during flight, and adapting and integrating with our existing IsaacLab simulation and training environments. The project will finish with hardware experiments that validate the approach.
Contact Details: Interested candidates should send their CV, transcripts (bachelor and master), and description of relevant projects to Daniel Zhai [dzhai (at) ifi (dot) uzh (dot) ch], Ismail Geles [geles (at) ifi (dot) uzh (dot) ch], Yunfan Ren [yunfan (at) ifi (dot) uzh (dot) ch], and Prof Scaramuzza [sdavide (at) ifi (dot) uzh (dot) ch].
Thesis Type: Semester Project / Master Thesis
Vision-Based Drone Racing from Raw Pixels with Deep Reinforcement Learning - Available
Description: While reinforcement learning has achieved super-human performance in drone racing, demonstrating superior performance against traditional MPC methods, RL methods have been relying on perfect state observations of the quadrotor, filtered image inputs, or rendered RGB images. Drone racing using real onboard images only remains a challenging research problem. Due to the large sim-to-real gap between simulated images and real onboard RGB images, an RL agent flying from raw simulated pixels often performs poorly in the real world. This project aims to bridge this gap by using large-scale domain randomization, specifically the generative models for scene creation and the powerful rendering pipeline in IsaacLab. The students are expected to create accurate simulation environments and RGB image rendering, and design and train an RL agent for this task. Applicants are expected to be proficient in Python, C++, Pytorch, and Git, preferably with experience with RL and vision-based algorithms.
Goal: This project aims to train an RL agent that is able to race a quadrotor using onboard RGB image observations only. The project will involve investigating the SOTA RL algorithms and policy architectures, implementing the simulation in our IsaacLab environments. The project will finish with hardware experiments that validate the approach.
Contact Details: Interested candidates should send their CV, transcripts (bachelor and master), and description of relevant projects to Daniel Zhai [dzhai (at) ifi (dot) uzh (dot) ch], Ismail Geles [geles (at) ifi (dot) uzh (dot) ch], Yunfan Ren [yunfan (at) ifi (dot) uzh (dot) ch], and Prof Scaramuzza [sdavide (at) ifi (dot) uzh (dot) ch].
Thesis Type: Semester Project / Master Thesis
Observability-and-Perception-aware Planning and Control for Event-Based Object Reconstruction - Available
Description: Imagine a drone with an event camera onboard, and your task is to fly around a (stationary or even moving) object to gather vision data for 3D reconstructions of the object of interest. This task is challenging in two ways: 1) since you rely on VIO to estimate the state of the drone, the movement should produce good state estimates, which should include excitation of the rotation axes, maintaining feature parallax, and avoiding constant velocity; (2) the quality of the reconstruction depends on the quality and coverage of the sensor data, the drone trajectory should optimize for this. With these two concurrent objectives, the goal is to design a model-based or learning-based planner/controller that satisfies both objectives. Applicants should be proficient in Python, C++, preferably with knowledge of VIO and 3D reconstruction.
Goal: Investigate model-based or learning-based methods to design a perception-and-observability-aware controller for 3D object reconstruction using a quadrotor with an onboard event camera.
Contact Details: Interested candidates should send their CV, transcripts (bachelor and master), and description of relevant projects to Daniel Zhai [dzhai (at) ifi (dot) uzh (dot) ch], Simone Nascivera [snascivera (at) ifi (dot) uzh (dot) ch], Roberto Pellerito [pellerito (at) ifi (dot) uzh (dot) ch], and Prof Scaramuzza [sdavide (at) ifi (dot) uzh (dot) ch].
Thesis Type: Master Thesis
Vision-Based Reinforcement Learning in the Real World - Available
Description: This master's project offers an exciting opportunity to work on real-world vision-based drone flight without relying on simulators. The goal is to develop learning algorithms that enable quadrotors to fly autonomously using visual input, learned directly from real-world experience. By avoiding simulation, this approach opens up new possibilities for the future of robotics. A significant focus of the project is achieving high sample efficiency and designing a robust safety framework that enables effective exploration by leveraging the latest research results on optimization layers within RL policies. The project will begin with state-based learning as an intermediate step, progressing toward complete vision-based learning. It builds on recent research advances and a well-established drone navigation and control software stack. The lab provides access to multiple vision-capable quadrotors ready for immediate use. This project is ideal for outstanding master’s students interested in robotics, learning systems, and real-world deployment. It offers a rare chance to contribute to a high-impact area at the intersection of machine learning, control, and computer vision, with strong potential for further academic or industrial opportunities. Applicants should have proficiency in computer vision, reinforcement learning, and robotics, as well as strong programming skills in Python and C++. Initial experience with large neural network world models is expected, as well as familiarity with simulation software and real-time data processing. A solid understanding of drone dynamics and control systems is also essential.
Goal: The goal is to investigate how the latest optimization-based reinforcement learning advances push the limits of learning real-world tasks such as agile vision-based flight.
Contact Details: Interested candidates should send their CV, transcripts (bachelor and master), and descriptions of relevant projects to Rudolf Reiter (rreiter AT ifi DOT uzh DOT ch), Ismail Geles (geles AT ifi DOT uzh DOT ch), and Daniel Zhai dzhai AT ifi DOT uzh DOT ch
Thesis Type: Master Thesis
Learning Rapid UAV Exploration with Foundation Models - Available
Description: In this project, our objective is to efficiently explore unknown indoor environments using UAVs. Recent research has demonstrated significant success in integrating foundational models with robotic systems. Leveraging these foundational models, the drone will employ learned semantic relationships from large-world-scale data to actively explore and navigate through unknown environments. While most prior research has focused on ground-based robots, this project aims to investigate the potential of integrating foundational models with aerial robots to introduce more agility and flexibility. Applicants should have a solid understanding of mobile robot navigation, machine learning experience (PyTorch), and programming experience in C++ and Python.
Goal: Develop such a framework in simulation and conduct a comprehensive evaluation and analysis. If feasible, deploy such a model in a real-world environment.
Contact Details: Jiaxu Xing (jixing@ifi.uzh.ch), Daniel Zhai (dzhai@ifi.uzh.ch), Prof. Davide Scaramuzza (sdavide@ifi.uzh.ch)
Thesis Type: Semester Project / Master Thesis
Event-based Perception for Autonomous Driving - Available
Description: Autonomous vehicles must perceive and interpret their surroundings reliably under a wide range of conditions—including high-speed motion, rapid illumination changes, and adverse weather. Conventional frame-based cameras often face limitations such as motion blur and poor performance in low-light or high-dynamic-range scenes. Event-based sensors, inspired by biological vision, provide complementary information by capturing pixel-level brightness changes asynchronously, offering high temporal precision and wide dynamic range. This project investigates how event-based perception can contribute to robust and efficient environment understanding for autonomous driving. The focus is on exploring methods to leverage event data for improving detection, tracking, and scene understanding tasks, either standalone or in fusion with conventional sensors such as RGB cameras, LiDAR, or IMU.
Goal: The goal is to design and evaluate an event-based sensing framework that improves real-world perception performance for autonomous driving.
Contact Details: Interested candidates should send their CV, transcripts (bachelor’s and master’s), and desired start date to Rong Zou (zou@ifi.uzh.ch), Roberto Pellerito (rpellerito@ifi.uzh.ch), and Davide Scaramuzza (sdavide@ifi.uzh.ch).
Thesis Type: Semester Project / Master Thesis
Compact 3D Representations for Embedded Neural Vision - Available
Description: High-fidelity 3D scene representations such as 3D Gaussian Splatting have recently achieved remarkable realism in reconstruction and rendering tasks. However, their high memory requirements make them impractical for embedded or edge devices such as drones, mobile robots, and AR/VR systems. Event cameras offer an opportunity to overcome these limitations: by capturing sparse visual changes and providing a data-efficient representation of scene dynamics. This project investigates how neuromorphic signals can be leveraged to build compact 3D models that preserve essential structural and appearance information while drastically reducing storage usage.
Goal: The project aims to develop and analyze lightweight 3D scene representations tailored for resource-constrained platforms. The work will include implementing prototype pipelines, quantifying memory–fidelity trade-offs, and benchmarking performance against state-of-the-art methods.
Contact Details: Interested candidates should send their CV, transcripts (bachelor’s and master’s), and desired start date to Rong Zou (zou@ifi.uzh.ch), Nikola Zubic (zubic@ifi.uzh.ch), and Davide Scaramuzza (sdavide@ifi.uzh.ch).
Thesis Type: Semester Project / Master Thesis
Motion Segmentation with Neuromorphic Sensing - Available
Description: Understanding motion in dynamic environments is a fundamental challenge for autonomous agents, particularly when both the observer and surrounding objects move simultaneously. Traditional frame-based approaches often struggle with motion blur, latency, and ambiguity between camera motion and object motion. Neuromorphic sensors, which asynchronously capture intensity changes at microsecond precision, offer new opportunities for robust motion analysis even under fast motion or low-light conditions. This project explores how high-temporal-resolution visual signals can be used to distinguish between ego-motion and external motion in realistic, real-time scenarios. The focus is on developing a general and efficient framework for motion disambiguation that supports reliable mapping, tracking, and navigation in dynamic environments.
Goal: The objective is to design and evaluate an event-based motion-disambiguation framework that separates ego motion from independently moving objects. The resulting system will be assessed for robustness and accuracy on synthetic and real-world datasets.
Contact Details: Interested candidates should send their CV, transcripts (bachelor’s and master’s), and desired start date to Rong Zou (zou@ifi.uzh.ch), Roberto Pellerito (rpellerito@ifi.uzh.ch), and Davide Scaramuzza (sdavide@ifi.uzh.ch).
Thesis Type: Master Thesis
Vision-Based Agile Aerial Transportation - Available
Description: Transporting loads with drones is often constrained by traditional control systems that rely on predefined flight paths, GPS, or external motion capture systems. These methods limit a drone's adaptability and responsiveness, particularly in dynamic or cluttered environments. Vision-based control has the potential to revolutionize aerial transportation by enabling drones to perceive and respond to their surroundings in real-time. Imagine a drone that can swiftly navigate through complex environments and deliver payloads with precision using only onboard vision sensors. Applicants are expected to be proficient in Python, C++, and Git.
Goal: This project aims to develop a vision-based control system for drones capable of agile and efficient aerial transportation. The system will leverage real-time visual input to dynamically adapt to environmental conditions, navigate obstacles, and manage load variations with reinforcement or imitation learning.
Contact Details: Interested candidates should send their CV, transcripts (bachelor and master), and descriptions of relevant projects to Ismail Geles [geles (at) ifi (dot) uzh (dot) ch], Leonard Bauersfeld [bauersfeld (at) ifi (dot) uzh (dot) ch], Yunfan Ren [yunfan (at) ifi (dot) uzh (dot) ch], Prof. Davide Scaramuzza [sdavide (at) ifi (dot) uzh (dot) ch]
Thesis Type: Master Thesis