Menu

RAIL-BENCH

is the world's first environmental perception benchmark for railway.

Accepted at WACV 2027 →
Codabench is currently experiencing disruptions. Submissions may be delayed or fail. Details

The Codabench platform, which hosts all RAIL-BENCH challenges, is currently affected by network infrastructure upgrades and security routing changes on the Codabench servers. As a result, you may experience slow page loads, and submissions may be delayed, stay in the queue for a long time, or fail.

This issue is outside our control and does not affect the RAIL-BENCH data or evaluation. If a submission is stuck, please wait and check again later instead of submitting it repeatedly, as repeated attempts may count towards your submission limit. We will remove this notice once the problem has been resolved. Thank you for your patience!

About RAIL-BENCH

RAIL-BENCH is the first perception benchmark suite for the railway domain. Automated train operation requires robust camera-based perception, yet the railway domain lacks public benchmark suites with standardized evaluation protocols that would enable reproducible comparison of approaches. RAIL-BENCH addresses this gap by providing curated training and test datasets drawn from diverse real-world scenarios, evaluation metrics, and public leaderboards — spanning five distinct perception tasks:

News

  • 10.11.2026 Our paper "Railway Artificial Intelligence Learning Benchmark (RAIL-BENCH): A Benchmark Suite for Perception in the Railway Domain" has been accepted at the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV 2027)! The preprint is already available on arXiv.
  • 10.11.2026 The RAIL-BENCH Rail challenge has been updated. The update includes (1) a second release of the RAIL-BENCH Rail dataset with refined annotations and new track IDs, (2) an improved evaluation protocol using a novel track-width-based distance threshold, and (3) a new submission phase on Codabench. See the Rail challenge page for details.
  • 02.06.2026 All RAIL-BENCH challenges are now open for submissions on the Codabench platform. Just search for "RAIL-BENCH" and you will find all five challenges or look for the direct links on our individual challenge pages. We can't wait to see your submissions and how you tackle the unique challenges of railway perception!
  • 24.04.2026 Our paper is on arXiv! Read about the motivation, design, and details of RAIL-BENCH in our preprint: View arXiv Preprint
  • 16.04.2026 Our project presentation slides are now available! Learn how and why we built RAIL-BENCH, and why benchmark suites are essential for advancing perception in the railway domain: View Presentation Slides

Citation

RAIL-BENCH has been accepted at WACV 2027. If you use RAIL-BENCH in your research, please cite it as follows:

@article{baetz2026railbench,
  title   = {Railway Artificial Intelligence Learning Benchmark ({RAIL-BENCH}): A Benchmark Suite for Perception in the Railway Domain},
  author  = {Baetz, Annika and Klasek, Pavel and Ham, Seo-Young and Lauer, Martin},
  journal = {arXiv preprint arXiv:2604.22507},
  year    = {2026}
}

License and Terms of Use

The RAIL-BENCH datasets are compiled from multiple sources whose owners granted permission for non-commercial research use. Some sources are licensed under CC BY-NC-SA 4.0; the others provided consent under individual agreements. By using RAIL-BENCH, you agree to:

  • Use the datasets for academic and non-commercial research or educational purposes only. Any commercial use is prohibited.
  • Cite RAIL-BENCH in any work that uses the datasets (see Citation).
  • Not redistribute the datasets; please refer others to the official download page instead.
  • Contact the original content owners directly if you want to use the data beyond the scope of this benchmark suite, e.g. for further processing or in a different context. All sources and their contact details are listed in the overview_datasets.csv file included in the download.

Contact

For questions or inquiries, please reach out to:

Annika Bätz Institute of Measurement and Control Systems (MRT), KIT annika.baetz@kit.edu