Delivery Zone Planning
I wrote this case study because the insights gathered from the user testing of our MVP were quite interesting. As AI becomes more integrated across product types and industries, in this context—decision support systems in logistics, it is essential to thoughtfully study and design how users and AI work together. This human-AI collaboration will directly influence business outcomes.
Challenge
Creating a decision support system on Delivery Zone Planning.
My role
User Researcher
UX Designer
DELIVERABLES
Conducted a research to study the user’s decision-making process.
Design the MVP.
Conducted another user test to study user’s perception of the MVP.
result
Brought into light a problem in user’s perception: Overestimate the system’s capability.
Challenge
In our last-mile delivery, each truck is assigned a delivery zone made up of nearby areas (e.g., districts or wards). The process of grouping areas into these zones is called delivery zone planning.
Since order volumes in each area fluctuate, the zones must be regularly adjusted to optimize delivery. The goal is to maximize the number of orders delivered while preventing both overloading and underutilization.
Five Regional Managers are responsible for this task and periodically review and adjust the zones as needed.
We were asked to create an decision support system that could assist the Regional Managers with delivery zone planning.
The goal is, again, to
maximize order demand coveragewhile ensuringefficient truck utilization.
Example: Spoke A manages three delivery zones: Red, Blue, and Green, each serviced by a dedicated truck. The Red Zone consists of District 1, District 4, An Khanh Ward, and Thu Thiem Ward. The Blue Zone comprises only three wards due to the high order volume in this area.
Discovery research & MVP
Similar to the Demand Forecast project, I research and analyze how the five Regional Managers currently perform zone planning. These insights guide the data team in developing an algorithm to automate the task.
Our first MVP is a suggestion system, applied firstly for Ho Chi Minh City. Users can input their requirements (e.g., select historical data to be used for calculations), click a button, and then view the zone suggestions generated by the system.
The research
The MVP
User test
However, we knew that this very first version was not reliable enough to put into real usage, as we not yet found an optimal decision making process / algorithm, and also due to the lack of many critical data (eg., truck capacity fill rate).
While the data and development teams continued their work, I decided to test this MVP out to see how the users would respond to the product’s concept.
I conducted a group interview, where I met with all the Regional Managers, allowed them to use the MVP to generate a zone plan for Ho Chi Minh city, then asked them to rate the system’s suggestions, while also explaining their reasoning behind the ratings.
the goal was to study
How users assess the system’s suggestions: What criteria do they use to evaluate its effectiveness?
How users perceive the system’s role in delivery zone planning: Do they see it as a trustable assistant, and would they integrate it into their process?
The interesting insight
The findings from this research amazed me:
From the previous research findings, I was already aware of the process users followed to create zone plans, including the factors and criteria they considered. However, it wasn’t until now that they revealed other ad-hoc factors they took into account during planning (e.g., the time-consuming challenge of locating addresses in certain rural areas).
But what amazed me most was their (unexpected) perception of the system:
We asked them to rate each zone suggestion one by one as either “can apply (all or partly)” or ”cannot apply at all”. The final “can apply” rate was only ~50%. Despite this, at the end when we asked for their overall opinion on the system’s performance, they still believed that it had done a very good job and put together better plans than they could have.
Surprisingly, they didn’t question the system’s decision-making process or how it worked, all 5 of them, not once. Instead, they assumed there was some inexplicable “magic” behind the product that made it as intelligent as a human. As one remarked, “That’s how AI works, isn’t it?”
And lastly, they believed it would completely replace their role someday.
the insights
As the creators, we were fully aware that our “AI product” at that stage was far from “intelligent as a human”. The approach was rule-based AI, since the company could not afford the risk of a black-box mechanism for such a huge-impact decision. And at that time the rules were too simple, the system couldn’t even replicate half of the decision-making process of the users.
What we were not aware, however, was the perception of the users toward our product. Despite being high-level managers (they are Regional Managers!), when it comes to ‘advanced technology’ which they don’t fully understand, they approached it in a very unusual way. Their tendency to overestimate the system’s capabilities posed significant risks to the final outcome.
This realization brought to light a critical issue in our interaction design: Lack of transparency of how the systems works & its limitations.
Takeaways
Based on research findings, we’ve come to prioritize transparency, explainability, and trust in our AI product — elements that are just as critical as the underlying algorithms driving decision-making, but previously overlooked.
In the context of solving this “grouping areas into zones” problem, effective collaboration between the still-limited AI and human users (regional managers) is essential. For this partnership to work, users must clearly understand how the AI operates, its strengths, and how their expertise can complement the system’s limitations.
Transparency in AI products in business context is just as important as in societal context. Lacking of this quality could negatively affect the collaboration between the AI-human, as well as the final business outcome.