Rail Track Detection New: Second dataset release
Robust rail track detection serves as a fundamental and indispensable component in the pursuit of comprehensive scene understanding within automated railway systems, as it provides precise information about the region of interest for obstacle detection and can facilitate self-localization based on digital maps.
Dataset
The RAIL-BENCH Rail dataset comprises 2500 real-world RGB images, split into 1500 training, 500 validation, and 500 test images. All rails visible in the images are annotated as polylines with additional attributes, such as left/right rail and occlusion level. In addition, object-level annotations are also provided (see Object Detection).
New
Second release (November 2026): The annotations have been refined, and each rail now carries a
track_id attribute that links the two rails of the same track. We strongly recommend using this release.
All baseline results reported in the current version of our
paper
(arXiv v2) are based on it.
The entire dataset — with the exception of the test ground truth — is publicly available for download:
Annotation Policy
- All visible rails in the images are annotated with polylines at the position of the outer edges of the rail heads. Check and guard rails are excluded from the annotations.
- At turnouts, where several tracks merge/diverge, polylines terminate at the horizontal line connecting the two tips of the switch blades.
- Only visible rails are labeled. However, in cases of minor occlusions (e.g., poles, signs) or when the rail position can be reliably inferred (e.g., when straight rails are covered by another train, but are visible before and after the train), we interpolate through the occlusion to maintain continuity.
- Any rails that are present in the scene but not labeled (e.g., due to poor visibility, heavy occlusion, or ambiguity) or where it is unclear whether occluded rails should be annotated, ignore areas are added to mark these regions. Predictions in these regions are excluded during the evaluation to prevent penalizing reasonable model behavior.
- Each rail has additional attributes: whether it is a left or right rail, a
track_idthat groups the rails of the same track, and its occlusion level.
RAIL-BENCH Rail Challenge
In the RAIL-BENCH Rail challenge, predictions on the held-out test set are evaluated using two metrics. Both evaluate the positioning of rail predictions, but at different levels of detail:
ChamferAP
An average precision metric based on the Chamfer distance. It evaluates how well predicted polylines match ground truth rails on an instance level.
LineAP
A novel segment-based polyline average precision metric developed for this challenge. It performs a fine-grained geometric evaluation by dividing predicted and ground truth rails into small line segments.
Track-Width-Based Distance Threshold
Due to perspective, a fixed pixel threshold is too strict for distant rails and too lenient for nearby ones. Unlike common lane detection metrics, both ChamferAP and LineAP therefore use a novel track-width-based distance threshold that varies along the vertical image axis. Using the track width (the distance between the outer edges of the two rail heads) as a reference, the threshold approximates a constant real-world distance. A parameter α sets the threshold as a percentage of the track width. For example, LineAP@10 uses α = 10 %, which corresponds to roughly 16 cm.
| Metric | Thresholds α (% of track width) | Mean score |
|---|---|---|
| LineAP | 3, 5, 7, 10, 20 | mLineAP |
| ChamferAP | 5, 10, 20, 30, 50 | mChamferAP |
We report the individual scores at each threshold as well as their averages.
Ranking: Submissions are ranked by the mean of mChamferAP and mLineAP.
How to Participate
- Download the RAIL-BENCH Rail dataset and check the documentation of the Codabench Rail Track Detection challenge.
- Train your model with the train split of the dataset. Do not use the validation split for training, but only for hyperparameter tuning, early stopping, etc. You can use additional training data from other sources, but should state that when submitting your results.
- Use the RAIL-BENCH toolkit to check that your predictions are correctly formatted (recommended) and, optionally, to compute evaluation scores on the validation set locally.
- Submit your predictions on the Codabench Rail Track Detection challenge to get evaluated on the hidden test set.