When Robert Bosch GmbH set out to solve the problem of drivers going the wrong way on highways, the goal was to save lives. Other services like this existed in Germany, but precision and speed cannot be compromised. Could Bosch get precise enough location data—in real time—to do this? The company knew it had to try.
The result is the wrong-way driver warning (WDW) service and software development kit (SDK). Designed for use by app developers and original equipment manufacturers (OEMs), the architecture pivots on an innovative map-matching algorithm and the scalability of Microsoft Azure Kubernetes Service (AKS) in tandem with Azure HDInsight tools that integrate with the Apache Kafka streaming platform.
This article dives into the solution architecture.
“When we started our journey on Azure, we were a really small team—just one or two developers. Our partnership with Microsoft, the support from their advisory teams, the great AKS documentation and enterprise expertise—it all helped us very much to succeed.”
Bernhard Rode, software engineer, Bosch
The right way to solve the wrong-way problem
Bosch invests heavily in research and development (R&D), staffing more than 70,000 employees in R&D alone. These R&D staffers, along with 27,000 software and IT experts and 1,000 AI specialists, help maintain its position as Europe’s top provider of automotive solutions.
With headquarters in Stuttgart, Germany, Bosch has partnered closely with the world’s automakers for decades. The company continues to develop networked safety and assistance systems that support drivers today and pave the way for fully automated driving.
When the product team brainstormed the idea to solve the problem of wrong-way driving, they did not know whether it was technically possible. For such a service to work commercially, it had to locate vehicles in real time with pinpoint precision.
Smartphones or an onboard connectivity unit can anonymously record GPS coordinates and can send that location data to the cloud if the device is in a hotspot area, but GPS satellites broadcast their signals in space with only limited accuracy. What is received depends on many factors, including satellite geometry, signal blockage, atmospheric conditions, and the design and quality of the receiver. For example, GPS-enabled smartphones are typically accurate within a 4.9-meter (16-foot) radius under open sky.
“We were a little skeptical at first,” admits Bernhard Rode, a software engineer at Bosch who worked on the project. “To do it, we needed to map GPS data to very specific road elements—specific lanes on the highway—in order to decide whether a driver was going the wrong way or not.”
The Bosch team had to solve two major issues: first, to get the last piece of information out of the noisy sensor data; and second, to develop a highly scalable and ultra-flexible service to process the data in near real time. The question was how to build a real-time data ingestion and processing pipeline capable of returning notifications to drivers within seconds. Bosch Technical Lead Hai Dang Le developed a proof of concept using only a small technology stack, adding, “We really went for it!”
The problem was speed. The team assumed that devices emitting location information, such as smartphone apps and automotive head units, could eventually send thousands of data points to the solution per second, from all over Europe and eventually other countries. Bosch needed lightning fast compute capable of filtering events and pushing a notification back to an end device within 10 seconds—the time estimated to make the solution viable.
The team decided to offload the work of scaling and cluster maintenance to a managed service in a public cloud with a global reach. Thanks to the trusted partnership Bosch had with Microsoft, Azure Kubernetes Service was the obvious choice. A team of Microsoft cloud solution architects worked closely with Bosch engineers, who provided valuable feedback to Azure product teams. Microsoft continues to work with Bosch teams around the world. Working together, they devised a solution that produced the speed Bosch needed.
The key was orchestration. By orchestrating the deployment of containers using AKS, Bosch would get repeatable, manageable clusters of containers. Bosch already had a continuous integration (CI) and continuous deployment (CD) process to use in producing the container images and orchestration. The result: increased speed and reliability of deployments.
“We were looking for a cloud option where we could run our core business logic with zero changes on top of a new infrastructure,” explains Le.
AKS also offered the simplicity of a managed Kubernetes service in the cloud. It provided the elastic provisioning that Bosch wanted, without the need to manage its own infrastructure. In addition, the developers did not have to rethink all their design decisions. Instead, they could take the core business logic developed on-premises using the open-source tools they knew and run the solution virtually as is, within a faster infrastructure with a worldwide reach. The developers can deploy self-managed AKS clusters as needed, and they get the benefit of running their services within a secured network environment.
In addition, by running their solution on Azure and AKS, the average time to calculate whether a driver is going the wrong way could be improved to approximately 60 milliseconds.
The team was also interested in exploring other Azure services, such as solutions for managing APIs and security. “We didn’t want to have to handle security from the outside, like a web application firewall or something like that. With Azure, we get that,” Rode says.
“Using AKS was a strategic decision. We looked for a managed orchestrator so we could offload the work of patching, upgrading, and production-level services. That’s why we chose AKS—and it’s a very open, flexible platform.”
Hai Dang Le, technical lead, Bosch
How the solution works
The wrong-way driver warning solution runs as a service on Azure and provides an SDK. Service providers, such as smartphone app developers and OEM partners, can install the WDW SDK to make use of the service within their products. The SDK maintains a list of hotspots within which GPS data is collected anonymously. These hotspots include specific locations, such as segments of divided highways and on-ramps. Every time a driver enters a hotspot, the client generates a new ID, so the service remains anonymous.
Today the solution ingests approximately 6 million requests per day from devices emitting GPS data or from a partner’s back-end system. Anyone can download the SDK and try it out. The APIs grant a free request quota for test accounts. For production use, service providers request permission and then use the WDW SDK to register themselves for their own API authentication keys via the Azure API Management developer portal. Within their application, they configure the service’s endpoints by authenticating with their key for ingress and push notifications. The WDW service on Azure does the rest.
When a driver using a WDW-configured app or in-car system enters a hotspot, the WDW SDK begins to collect GPS signals and sensor events, such as acceleration and rotational data and heading information. These data points are packaged as observations and sent in the frequency of 1 Hertz (Hz)—one event per second—via HTTP to the WDW service on Azure, either directly or to the service provider’s back end, and then to Azure. The SDK supports both routes so that service providers stay in charge of the data that is sent to the WDW system.
If the WDW service determines that the driver is going the wrong way within a hotspot, it sends a notification to the originating device and to other drivers in the vicinity who are also running an app with the WDW SDK.
An architecture for wrong-way driving
The point of ingress for the WDW service is Azure API Management, which works in combination with Azure App Gateway, a managed global load-balancing service that can perform Layer-7 routing and SSL termination. The gateway decouples clients from services and acts as a reverse proxy, routing requests from clients to services.
Bosch uses API Management to send observations to an ingress engine (sendObservationAPI) and to send push notifications to client devices (sendPushAPI). “We use API Management like the entry door,” says Le. “It handles quota management, authentication of customers, and also terminates HTTPS for us.” To connect the microservices inside the cluster with API Management, the WDW service uses Traefik, an open-source reverse proxy that acts as ingress controller to ensure load balancing and dynamic discovery of the internal services by the Kubernetes API.
For years, Bosch developers had been using the Docker platform to containerize their code. For the first time, however, the WDW team hosted the Docker images in Azure Container Registry. “It was easy, safe, and secured to the outside,” notes Le. Azure Container Registry is compatible with the open-source Docker Registry v2, so the developers could use the same Docker CLI tools they already knew.
Within the first AKS cluster, the following apps process the streaming data:
- The ingest app receives the data from the gateway and stores it in MongoDB.
- The detector app does the heavy lifting by matching the location data on a map and then making the initial decision about whether the trip is suspicious.
- The alert validator app is a post-validation component used to verify the differing road characteristics of various countries.
- The data pump is solely tasked to persist data in an asynchronous way.
- The push registry app handles egress—the broadcast of the push notifications to the service providers. The WDW service supports Android, Google, and Apple push notifications and offers an option that service providers can use to configure custom alerts.
“We really enjoyed having Azure on our side because we could just spin up the service and route. We can deploy the service worldwide.”
Christian Jeschke, product owner, Bosch
The entire service is deployed using a CI/CD pipeline essentially lifted from on-premises and moved to Azure. Currently self-hosted in GitLab, the CI/CD pipeline is triggered when the code changes, at which point it automatically builds Docker images for every microservice. Each service is tested before being deployed into the AKS clusters.
AKS is deployed within a custom virtual network that keeps the applications isolated. “This allowed us to implement our security guidelines in a more elegant way,” explains Rode. “On the back end, our cluster is fully closed for external communication except through API Management. From a development perspective, it is very favorable for us to be able to deploy our apps in a very private virtual network environment.”
To operate such a complex service, the team uses Splunk software for searching, monitoring, and analyzing machine-generated big data.
Getting accuracy from unreliable GPS data
The team’s biggest technical challenge was to improve the reliability of the incoming GPS data. Bosch developed a custom sensor data-fusion and map-matching algorithm to verify a driver’s location and driving direction. Then the algorithm filters all suspicious trips and forwards them to the alert validator app. This multistep classification approach was used to reduce the computational complexity required for a cost-effective solution architecture.
The algorithm runs in the detector app, which receives millions of GPS coordinates daily. To handle the scale, it runs in its own AKS cluster. The detector app uses the Apache Kafka Streams framework to handle out-of-order data and to maintain the state of each trip locally. This approach enables the solution to operate reliably at scale with a latency of a few milliseconds.
The following diagram shows the map-matching algorithm in action. The red area represents a hotspot within which a driver’s GPS coordinates are recorded. Each blue dot represents one GPS location. Together, a series of these points depict a driven trajectory. The map-matching algorithm determines the most probable roads and the driving direction of the car. Based on this information, Bosch can reliably detect a wrong-way driver.
Additional Azure services
One goal of the project was to take advantage of Azure platform as a service (PaaS) tools whenever they would save time or costs. For example, Azure Cache for Redis provides fast, in-memory storage, while Azure Database for PostgreSQL delivers a highly available relational database that requires almost no administration. In addition, the team plans to migrate to Azure Data Explorer, a fast, fully managed data analytics service for real-time analysis on large volumes of streaming data.
The team also used the following services:
- Azure API Management provides the gateway to the back end. It pushes observations from client devices, currently serving about 6 million requests per day.
- Azure App Service was used to build and host multiple internal front ends used by the team for debugging and monitoring. For example, a real-time dashboard shows all the drivers currently passing a hotspot. App Service supports both Windows and Linux and works with the team’s automated deployment pipeline.
- Azure Content Delivery Network (CDN) uses the closest point of presence (POP) server to cache static objects locally, thus reducing load times, saving bandwidth, and speeding responsiveness of the WDW service.
- Azure Databricks is an Apache Spark–based analytics platform designed to support team collaboration. It enables Bosch data scientists, data engineers, and business analysts to make the most of the WDW service’s big data pipeline.
Next steps
The back-end cloud service team was among the first at Bosch to use AKS, and the developers appreciated working in a familiar Kubernetes experience. “We have local Kubernetes instances running on our development machines,” explains Le, “and we can spin up a cluster as needed. The basic Kubernetes experience is the same, and AKS was super easy.”
Today the WDW SDK is used in smartphone offerings by many service providers, including NDrive, Radioplayer, Antenne Bayern, V-Navi, BikerSOS, and TCS. Bosch is currently in talks with several OEMs interested in embedding the solution directly in cars.
The next step is to improve the way hotspots are downloaded. Currently, Azure App Services hosts all the known hotspots, which are downloaded to the end devices, regardless of the country a driver is traversing. The idea is to detect a driver’s locale and download only relevant hotspots, which will improve performance.
“Currently we already have a big community, with over 12 million downloads, and we detected some wrong-way drivers,” says Christian Jeschke, product owner. “For a small team of only five or six people, supporting a worldwide service wouldn’t be possible. With Azure, it was really quite easy.”
“What we like about AKS is the simplified Kubernetes experience. It's click and deploy, it’s click and scale. It’s infrastructure as code too, which is quite cool for us.”
Christian Jeschke, product owner, Bosch
Follow Microsoft