Institut für Mess- und Regelungstechnik (MRT)

Decision-Making and Motion Planning

Group Leader: Dr.-Ing. Maximilian Naumann

Motion planning is key to every automated physical system, and a well investigated subject in the field of robotics. In the field of automated vehicles, the task is to determine an appropriate behavior, resulting in a trajectory, i.e. the state of the vehicle as a function of time. This task is referred to as decision making or motion planning for automated vehicles. The decision is based on previously acquired knowledge about the environment, such as the drivable area and detected objects, but also traffic rules.

The key challenges in motion planning for automated driving arise from the collision risk with other traffic participants. Since human lives are at risk, safety takes the top priority. On the other hand, over-cautious behavior, sometimes referred to as “driving like a learner”, is not only inconvenient but can also cause dangerous situations, as this behavior might cause misunderstanding by humans or entice them to risky overtaking maneuvers, for example.


Inverse Reinforcement Learning for Highway Driving

Many planning problems are formulated as the minimization of a cost function. This cost functions then needs to account for different criteria such as comfort, safety and costs. As an alternative to manually defining a cost function, the cost function can be learned by Inverse Reinforcement Learning. To this end, a human expert provides demonstrations of optimal behavior. Using this data, the parameters of the cost function are optimized such that the demonstrated behavior is reproduced. We investigate how demonstrated behavior in highway driving can be reproduced by the learned cost function.

Contact: M.Sc. Johannes Fischer


Decision Making under Uncertainty


In real traffic it is necessary to make decisions under partial information. In particular, the intention of other traffic participants in important for planning. A principal framework for modelling such problems are partially observable Markov decision processes (POMDPs), which allow to optimally solve stochastic planning problems. POMDPs are usually solved approximately due to the high problem complexity, e.g. with Monte Carlo Tree Search. We investigate the application of POMDPs for modelling real traffic situations and efficient POMDP solvers.

Contact: M.Sc. Johannes Fischer


Interaction-aware Motion Planning

To reduce the computational complexity of motion planning, most state of the art approaches for automated vehicles separate the prediction of surrounding traffic participants from their motion planning step. This way, the interaction among traffic participants is neglected, leading to a pipeline approach.
While this simplifying assumption is valid for many single-lane maneuvers like ACC or lane-keeping, it is not suitable for lane changes in dense traffic.
Our work's focus is to explicitly consider these interactions during motion planning, anticipating other vehicles' reactions to our actions.

Shown is a single motion planning step of an interaction aware planning algorithm, planning the trajectory of the green vehicle for a merge scenario. The green vehicle approaches the left lane to trigger a reaction of the red car. The red vehicle can react differently to the merge attempt (opening the gap or not). The future evolution of the scene for both cases is shown at the bottom.

Contact: M.Sc. Christoph Burger


Cooperative Multi-Vehicle Motion Planning

Automated driving has the potential to greatly improve traffic efficiency and safety. This improvement becomes even more significant if not only a single automated vehicle is driving on the road by itself but multiple together.
The ability to communicate and react to each other much quicker and more precise than humans ever could enables the possibility of cooperative, coordinated maneuvers among automated vehicles. In our work, we investigate the potential of multi-vehicle motion planning.

Shown is a comparison between a traditional (non-cooperative) motion planning approach and one based on cooperative multi-vehicle planning.
By planning the trajectories jointly, a coordinated maneuver can be performed, and an overtaking can be performed much more efficiently.

Contact: M.Sc. Christoph Burger


Optimization based trajectory planning

In autonomous robots (and vehicles), trajectory planning is an important task. Once a computed global path/task/behavior is obtained, the trajectory planning will modify them to avoid unexpected obstacles and fulfill the requirements of our vehicle.

In our approach, optimization based methods are used to maximize comfort and safety inside the vehicle. This solutions are tested in simulated and real scenarios.

Contact: Dr. Eduardo José Molinos Vicente


Realistic simulation for motion planning and decision making


Testing on real vehicles is a expensive and, sometimes, risky task. For that reason, we perform a deep study on state of the art simulators, in a way that new planning approaches can be tested. We also focus on finding corner case scenarios that are rarely seen using on-road data.

Contact: Dr. Eduardo José Molinos Vicente


Realistic Single-Shot and Long-Term Collision Risk for Human-Style Safer Driving


Navigation in congested environments is a challenge for autonomous vehicles. In order to drive in a safer way, the potential collision risk with other obstacles should be considered in the planning algorithm. We focus on assessing realistic collision risk with other traffic participants, to finally enable generating a more human-style and safer driving behavior. In the demonstration videos, the ego vehicle always try to drive to the speed limit and follow the centerline of the road, but should also pay attention to the static and dynamic obstacles that result in certain collision risk to the ego vehicle. In order to reduce the collision risk, the ego vehicle reduces the speed or move laterally slightly to avoid passing the obstacles with low distance and high relative speed. After that, it should retrieve its origin lateral position and velocity. Future work will be done in learning how human drivers balance risk, utility and comfort from the real data. By applying machine learning techniques on recorded human trajectories, a realistic risk balanced driving behavior will be achieved.

Contact: M.Sc. Lingguang Wang


Decision-Making for Automated Vehicles Using a Hierarchical Behavior-Based Arbitration Scheme

Behavior planning and decision-making are some of the biggest challenges for highly automated systems. A fully automated vehicle (AV) is confronted with numerous tactical and strategical choices. Most state-of-the-art AV platforms are implementing tactical and strategical behavior generation using finite state machines. However, these usually result in poor explainability, maintainability and scalability. Research in robotics has raised many architectures to mitigate these problems, most interestingly behavior-based systems and hybrid derivatives.

Inspired by these approaches, we propose a hierarchical behavior-based architecture for tactical and strategical behavior generation in automated driving. It is a generalizing and scalable decision-making framework, utilizing modular behavior blocks to compose more complex behaviors in a bottom-up approach. The system is capable of combining a variety of scenario- and methodology-specific solutions, like POMDPs, RRT* or learning-based behavior, into one understandable and traceable architecture. We extend the hierarchical behavior-based arbitration concept to address scenarios where multiple behavior options are applicable, but have no clear priority against each other. Then, we formulate the behavior generation stack for automated driving in urban and highway environments, incorporating parking and emergency behaviors as well. Finally, we illustrate our design in an explanatory evaluation.

Contact: M.Sc. Piotr Orzechowski


Learning Path Tracking from Simulation

We focus on applying machine learning techniques for planning and control of automated vehicles. Off-line simulation can provide plenty of training data in order to learn challenging problems efficiently. However, one of the most important concerns is transferring learned algorithms from simulation to the real car. In this work, we investigate learning a path tracking agent on the simulation using several random training episodes to cover different situations and provide a generally applicable solution. The trained agent is then tested on a small racing car without further modification to evaluate its effectiveness for accurate and smooth control of the vehicle.

Contact: M.Sc. Danial Kamran


Scalable Reinforcement Learning for Automated Driving at Occluded Intersections

Contact: M.Sc. Danial Kamran


Probabilistic Motion Planning for Automated Vehicles

As a prerequisite for their on-road deployment, automated vehicles must show an appropriate and reliable driving behavior in mixed traffic, i.e. alongside human drivers. Besides the uncertainties resulting from imperfect perception, occlusions and limited sensor range, also the uncertainties in the behavior of other traffic participants have to be considered.

Related approaches for motion planning in mixed traffic often employ a deterministic problem formulation. The solution of such formulations is restricted to a single trajectory. Deviations from the prediction of other traffic participants are accounted for during replanning, while large uncertainties lead to conservative and over-cautious behavior. As a result of the shortcomings of these formulations in cooperative scenarios and scenarios with severe uncertainties, probabilistic approaches are pursued. Due to the need for real-time capability, however, a holistic uncertainty treatment often induces a strong limitation of the action space of automated vehicles. Moreover, safety and traffic rule compliance are often not considered.

Thus, we focus on motion planning approaches are targeted towards the different predominant uncertainties in different scenarios, while operating in a continuous action space and ensuring safety.

Contact: Dr.-Ing. Maximilian Naumann


Safety in Motion Planning and Decision Making

Since human lives must never be put at risk, the safety of automated vehicles must be ensured before their on-road deployment. As shown in many exemplary calculations, a validation solely via test drives is not expedient. In order to validate the safety of automated vehicles, formal methods are promising. Here, we focus on analyzing and extending existing concepts such as Responsibility Sensitive Safety (RSS) by Intel/Mobileye and set-based methods as proposed by Matthias Althoff et al. Further, we focus on motion planning and decision making that is safe, yet not over-cautious, since over-cautious behavior might entice human drivers to risky maneuvers (such as overtaking) and thus jeopardize the safety concept.

Contact: Dr.-Ing. Maximilian Naumann


Inverse Reinforcement Learning from Human Behavior

In order to better understand and predict human behavior, but also to generate human-like behavior, we created the Interaction Dataset, a dataset that contains human driving, cycling and walking behavior, along with high-definition lanelet2 maps. We now focus on analyzing this behavior using inverse reinforcement learning.

Contact: Dr.-Ing. Maximilian Naumann


Cooperative Motion Planning for Automated Vehicles in Mixed Traffic

While motion planning techniques for automated vehicles in a reactive and anticipatory manner are already widely presented, approaches to cooperative motion planning are still remaining. Thus, we focus on the enhancement of common motion planning algorithms, allowing for cooperation with human-driven vehicles.

The blue (automated) vehicle enters the narrowing first, as it is closer to the narrowing. However, if the black (human-driven) vehicle has the right of way, it drives first despite the blue vehicle being closer.

Contact: Dr.-Ing. Maximilian Naumann


Trajectory planning with verifiable safety

We perform research to investigate the metrics that enable reliable evaluation of the level of criticality and subsequently integrate them in our criticality-aware motion planner.

Contact: M.Sc. Ömer Sahin Tas