Spatial Intelligence Group

KITScenes Multimodal

We are proud to present KITScenes Multimodal, carried mainly by our group: a European dataset that pairs lidar beyond 400 m, 360° global-shutter cameras, and 4D imaging radar with the most complete public HD maps to date. Covering 62 km², our Lanelet2 maps provide full topology and reprojection-accurate 3D traffic elements, and are validated in closed-loop Autoware driving. Its four benchmarks — online HD map construction, long-range depth, novel view synthesis, and end-to-end driving — mark out the spatial reasoning current methods still lack, and invite the community to close that gap: kitscenes.com/multimodal.

Since this research repeatedly reached the limits of what public benchmarks can measure, we built our own: KITScenes Multimodal.

Group Leader: Dr.-Ing. Frank Bieder

Automated driving still depends on highly accurate maps that provide enriched layered knowledge, including lane geometries, traffic rules, and traffic lights. This information is not always perceivable, whether due to bad weather or occlusions. Formerly the Localization and Mapping group, the Spatial Intelligence Group investigates the challenges of map-based automated driving and addresses the future vision of mapless driving.

Deriving a suitable representation of the world from sensor data is Mapping. We estimate ego-motion with neural and continuous-time SLAM and build both dense 3D reconstructions and sparse, semantic, parametric maps that are compact in storage and low in maintenance — scaled from single drives to entire cities using aerial imagery, SD maps, and noisy fleet data from production vehicles. Lanelet2, our open-source HD map framework, is the common format underneath.

Where a map is outdated or missing, online Map Perception takes over. We predict map geometry, topology, and regulatory relations directly from sensor data, fusing priors from existing HD and SD maps and cutting label cost through self- and semi-supervised learning. In the other direction, maps serve as supervision: localizing precisely in a verified map produces training data without manual annotation, and even transfers knowledge between sensors that never share a field of view.

Projects:

Frequency Adaptive Neural SLAM

Frequency-aware neural map representations for faster and more robust LiDAR SLAM.

Lanelet2 HD Map Framework

Lanelet2: The open source real-world tested C++/Python automated driving HD map framework. It is designed to utilize high-definition map data to allow driving in the most complex traffic scenarios.

Scalable HD Mapping

Large-scale 3D Scene Reconstruction, Texturing and Semantic Mapping

We develop TSDF-based reconstruction methods that accurately recover large 3D scenes and enrich them with color and semantic texture information.

Semantic Parametric Mapping

Automated extraction of vectorized and parametric HD maps creates compact physical map layers and supports fully automated HD map generation.

Online Map Perception with Priors

We research how to perceive map information when offline HD maps are outdated or incomplete, and how navigation maps can extend onboard perception range.

Cross-Modal Domain Adaptation via Semantic Parametric Maps

Transferring perception knowledge across sensors via a semantic HD map — from front-view camera to full 360° LiDAR, without overlapping views or manual labeling.

Mapping with Fleet Data

Automated coreferencing, aggregation and map generation using fleet data obtained through production vehicles

Self-/Semi-Supervised Online HD Map Perception

Self- and semi-supervised learning boosts the performance of online HD map perception models when only limited labeled training data is available, by exploiting consistency cues that come for free with driving data.