[SA/IDP/MA] Driving Scene Reconstruction and Novel Sensor Data Generation via Scenario Editing

Kurzbeschreibung

Beschreibung Motivation With the advancement of autonomous driving, many autonomous driving (AD) stacks are aiming at SAE level 4. However, even level 4 AD systems still encounter unpredictable corner cases (CCs) and require rigorous validation in diverse virtual environments. Testing these critical edge cases directly on real roads is often too dangerous, costly, and legally restricted. To address this, high-fidelity virtual simulation has emerged as a key solution. Rather than manually designing synthetic environments, reconstructing real-world dynamic scenes into highly photorealistic digital twins is pivotal. This project focuses on leveraging Gaussian Splatting (GS) to reconstruct complex 3D driving environments from real-world multi-sensor data, extract traffic scenarios, edit key dynamic elements (e.g., changing vehicle trajectories, adding obstacles), and simulate realistic sensor outputs. This pipeline will enable the scalable generation of diverse, safety-critical edge cases for testing and training AD systems. Voraussetzungen Work Packages • Literature review: Survey of GS for urban dynamic scenes, scenario editing, and sensor simulation in autonomous driving. • Implementation of the pipeline: • Implementing a 3D reconstruction pipeline using GS from multi-view dynamic driving squences. • Extracting dynamic traffic scenarios and individual participant models. • Developing a scenario-editing feature to modify dynamic object trajectories. • Generating synthetic sensor data (multiview images and pointcloud) based on the edited scenario. • Evaluation: Validating the visual and geometric quality of the reconstructed scenes and edited sensor outputs What you should bring along? • Very good programming skills in Python and PyTorch. • Knowledge of Computer Vision and Deep Learning (Must), 3D Reconstruction and 3D Gaussian Splatting (Desired). • High personal motivation and independent working style. • Very good language proficiency in English. Recommended literature: • DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenes • SplatAD: Real-Time Lidar and Camera Rendering with 3D Gaussian Splatting for Autonomous Driving • LangDriveCTRL: Natural Language Controllable Driving Scene Editing with Multi-modal Agents Possibility for publication in case of excellent work. If you are interested, please send me a grade sheet, your CV, short introduction (~5 sentences why this topic is interesting to you), and earliest possible date!