KIT - MRT - Research - Railway - Perception and Scene Understanding in Railway Transport

Perception and Scene Understanding in Railway Transport

Environmental perception and scene understanding are essential components in the development of driver assistance systems and the automation of railway transport. Perception systems rely on data captured by sensors - including cameras, RADAR, and LiDAR - mounted on trains or strategically positioned along the track. Advanced algorithms from pattern recognition and machine learning can analyze this sensor data to detect dynamic objects such as pedestrians and trains, recognize and classify railway signals, and assess track conditions. By integrating and processing this information, scene understanding algorithms enable real-time decision-making, thus improving the safety, efficiency, and reliability of railway operations.

RAIL-BENCH: The first perception benchmark suite for the railway domain

The development of scene understanding algorithms requires high-quality, annotated datasets as a fundamental prerequisite. These datasets must capture the full diversity of real-world railway environments - varying lighting conditions, weather, track layouts, and traffic scenarios - in order to train and evaluate models effectively. Furthermore, predefined evaluation protocols and fixed training/test splits are necessary for a standardized comparison of algorithms. To meet these requirements, we have developed RAIL-BENCH, the first perception benchmark suite for the railway domain.

RAIL-BENCH comprises five challenges: rail track detection, object detection, vegetation segmentation, multi-object tracking, and monocular visual odometry. It provides curated datasets drawn from diverse real-world scenarios, along with standardized evaluation metrics and public leaderboards.

Stay tuned - our website is launching soon!

Final Project Presentation

We are pleased to announce our final project presentation, taking place on April 15th, 2026, from 10:00 to 11:30 (CET). If you are interested in attending, please don't hesitate to get in touch with M.Sc. Annika Bätz.

RAIL-BENCH Challenges:

	RAIL-BENCH Rail Task: Detection of all visible rails in the form of polylines.
	RAIL-BENCH Object Task: Detection of dynamic objects (trains, persons, road vehicles, bicycles) and infrastructure elements (signals, signal poles, catenary poles), which are marked with bounding boxes.
	RAIL-BENCH Vegetation Task: recognition and classification of vegetation into low and high growing type.
	RAIL-BENCH Tracking Task: detect and track pedestrians across frames within a short video sequence.
	RAIL-BENCH Odometry Task: estimation of the ego motion of the camera mounted on the train.