Photorealistic off-road digital twins: Simulating dynamic objects, changing lighting, and altered terrain with 3D Gaussian Splatting

Summary

Beschreibung Autonomous off-road driving (e.g., in agriculture, forestry or disaster response) must cope with unstructured terrain such as grass, gravel and mud, whose appearance varies strongly with season, weather and lighting. Real-world tests are costly and hard to reproduce, and classical simulators rarely capture this diversity, resulting in a large sim-to-real gap. 3D Gaussian Splatting (3DGS) is the state of the art for real-time, photorealistic scene reconstruction. Yet existing 3DGS simulators mostly target on-road scenes and only replay recorded drives instead of varying them, e.g., with other road users, different lighting or altered terrain. In this project, we build an off-road simulation from real vehicle sensor data in which reconstructed scenes can not only be replayed but also realistically edited and varied. The thesis can be carried out independently or as part of a team. Your Contribution You take ownership of one research area, for example: • Dynamic scenes: moving vehicles, people and swaying vegetation • Relighting and weather: time of day, wet or dusty conditions • Scene editing: inserting obstacles, altering terrain such as ruts or puddles • Generative priors: diffusion-aided 3DGS In your area, you review the state of the art, implement your method in the shared pipeline, and evaluate it with real vehicle data. Voraussetzungen What should you bring? • Very good programming knowledge in Python (PyTorch); C++ is a plus • Experience with AI-Tools to accelerate and improve development • Solid background in computer vision, 3D geometry or machine learning • High drive, self-motivation and an independent, structured way of working • Interest in off-road autonomy, 3D reconstruction and simulation ✉ richard.taupitz@tum.de

• Master Thesis at Airbus Helicopters: Investigating Design Principles for Damage Tolerant Structural Bonding in Aerospace Applications

  Thesis · Lehrstuhl für Produktentwicklung und Leichtbau (TUM-ED)

• Optimization-based Assembly and Production Line Planning: Extending the Open-Source Framework PyCAALP

  Thesis · Lehrstuhl für Umformtechnik und Gießereiwesen (TUM-ED)

• Recursive Training with Varying Model Capacity

  Thesis · AI for Scientific Modeling

• Why Do Models Learn Similar Representations?

  Thesis · AI for Scientific Modeling

• Experimental Projects on Heat Transfer at the R&D Fluid Dynamic Lab (Siemens Energy, Sweden)

  Thesis · Professur für Thermofluiddynamik (TUM-ED)

• HiWi für Verbundprojekt ForBat@Bau

  HiWi · Lehrstuhl für Fördertechnik Materialfluss Logistik (TUM-ED)