deep-learning machine-learning knowledge-graphs reasoning human-ai-interaction hybrid-AI automated-driving multi-objective-decision-making


Making autonomous public transport aware of what is required at any moment for safe travel.

Problem Context

An increasing number of tasks are being taken over from the human driver as automated driving technology is developed. Accidents have been reported in situations where the automated driving technology was not able to function according to specifications. As data-driven Artificial Intelligence (AI) systems are becoming more ubiquitous in automated vehicles, it is increasingly important to make AI systems situational aware (i.e. aware of their capabilities in their current situation). In the mobility domain, TNO Integrated Vehicle Safety (IVS) has already been working on Situational Aware AI for Automated Driving. For this, a Hybrid AI approach is used to determine the competence of an AI algorithm. Hybrid AI combines a data-driven approach with (expert-)knowledge that is captured by a Grakn knowledge graph. The knowledge graph allows the AI to reason about the results from the data-driven AI system. The research on Situational Aware AI for Automated Driving shows that the hybrid approach holds great promise to reduce the risk of automated driving functions. The approach was specifically tested, however, in the context of cut-in maneuvers in a simulated environment.


In the Operational Context Characterization for Trustworthy Automated Shuttles (OCCTAS) project, the previous research is continued by making autonomous public transport aware of what is required at any moment for safe travel. A risk estimation of the actions which the AI-algorithms proposes will be a part of the development. A shuttle bus project in the Netherlands (Fabulos) is chosen as application for the developed technology.


The original shuttle bus designed in the Fabulos project could drive safely and autonomously. However, it was not capable of handling certain situations well, like blocking objects on the road, dangerous crossings, and driving in the vicinity of a school. In the OCCTAS project, the shuttle bus' behavior in these situations was improved through the incorporation of increased situational awareness. Behavior of the shuttle bus was improved specifically in the situations of objects blocking the road and for driving in the vicinity of a school. Originally, the shuttle bus would stop for all blocking objects. TNO IVS used operational context characterization to determine what an object was and whether it was safe to continue driving, or whether the object neede to be passed. In the simulation, it was shown the shuttle bus adjusted its behavior to the more desired behavior of stopping for a blocking object only when it was appropriate. For driving in the vicinity of the school, TNO IVS implemented context characterisation by having the shuttle bus respond both to its vicinity to a school and whether there were children on the road. If there were no children, the shuttle bus would not slow down unnecessarily.

Although the project thus demonstrated how Hybrid AI could increase situation awareness in two shuttle bus use cases, more work is needed. A follow-up project SALVA (Situation Awareness to Leave Virtual Rails) will look further into how to improve automated driving.


  • Jan-Pieter Paardekooper, Scientist, TNO, e-mail: