“A satisfied customer is the best business strategy of all” – Michael Leboeuf

In this two-part series, we will explore how we engineer the Delivery Partner-ETA predictive model to navigate successfully around roadblocks like sparsity of road mapping and dense networks of inroads in smaller towns and cities. 

Part 1: Identifying roadblocks to timely and accurate delivery of customer satisfaction

Brand equity and customer loyalty are two very valuable currencies for a company. Timely and accurate delivery of brand promise is critical to customer satisfaction. In a food delivery ecosystem, the accurate prediction and low-cost computation of Estimated Time of Arrival (ETA) is a game-changer. 

It is a brand promise that makes or breaks the customer experience. 

At Zomato, ETA is at the heart of menu browsing, restaurant selection, delivery partner (DP) assignment, order tracking, and delay communication experiences. The Time Microservice owns the ETA computation and subsequently distributes this information to other services. 

Our job is to set correct expectations with the customer and meet these expectations through a measure of punctuality and tolerance interval.

Zomato’s ETA goes beyond Delivery Partner Travel Time (DP-ETA). It factors in the computation of additional components like Kitchen Preparation Time (KPT), real-time dynamic buffers, and localised DP arrival times. 

DP-ETA is the transit time between the order pickup (restaurant location) and drop-zone-geo-fenced entry (customer location). While the DP-ETA is predicted in a feedforward method, real-time dynamic buffers adjust the estimated arrival time based on factors such as customer demand, traffic, road closures and repairs, DP supply, local weather, and real-time stress observed in the system. DP-ETA serves as a compensator in a variety of situations.  

On the contrary, highly precise values with low tolerance are more sensitive to dynamic settings, making them more error-prone in the event of disruptions such as traffic, diversions, and weather. Whereas the overstretched ETA and underpredictions have visible negative effects on customer metrics.

How do we engineer the DP-ETA predictive model?

  Roadblocks:

1. Forecasting a DP’s path is dependent on open-source maps and licensed map providers with routing engines. As we penetrate deeper into the city tier structure, the flaws in the representation of routing paths multiply. As some of these roads are less travelled and unexplored, road mapping can be severely inaccurate.

2. While mapping the Indian landscape, older city districts (with some of the world’s densest and most congested road networks) do not get computed and measured accurately. The network of Indian roads is like arteries, connecting different paths to the destination. Still, the lack of accuracy can affect the computation of DP-ETA. If one factors in multiple makeshift roads and inroads, then it becomes difficult and confusing to calculate distance, direction, and travel time precisely.

3. More often than otherwise, DPs select this path based on their own judgement as they react to unpredictable circumstances and experiences on the road. As a consequence, the forecasted route could differ significantly from the exact route taken by the DP. 

As a response, Zomato switched to a tree-based prediction model from the previously used map-graph-based model. 

Performance Indicator

Precision is measured using the 2-minute and 5-minute accuracy-compliance matrix, the R2 (R Square) score, error standard deviation and MAE (Mean Absolute Error). The performance indicators on the customer side include ORS (Order Requiring Support), Extreme Delays (>20 minutes), and user flows such as search to cart, cart to order, etc.

While in principle this made sense, we faced the following design challenges while building it as a model – 

• Response Time – Predictions should be generated in milliseconds, that too on a CPU-based deployment for optimising costs. These low-latency predictions are required for our apps’ smooth and lightning-fast scrolling
• Throughput – The requirement is in thousands of QPS (Queries per second) with resilience towards spikes
• Accuracy – Predictions should be substantially more accurate than the previous edition and should also pay more attention to catastrophic breaches and long-tail instances of delivery
• Scalability –
  • Vertical Scaling – To make this model one of the deepest tree models at Zomato, within the constraints of the latency requirement
  • Horizontal Scaling – To democratise the ability to add additional models into an ensemble. Specialised objectives, such as a rain model or a quantile model, might be included in these models
  • Parsimony of the features – To limit the number of features in the model in order to achieve high QPS and low inference latency. Features are input to the model and can represent particular states, properties, or traits

Stay tuned for part 2, where we will talk about eliminating these roadblocks.

This blog was written in collaboration with Shubh Chaurasia and Siddhartha Agnihotri.

If you found this to be an exciting problem to solve and would like to be a part of our engineering team, please reach out to me on LinkedIn.