Users of Grab, the Philippines' most popular ride-hailing app, are at the mercy of an opaque algorithm that calculates service prices.
When they book a ride, the amount to be paid for the distance traveled and the travel time is usually coupled with a surge fee that, according to the company, "reflects the real-time market conditions" and allows it to balance supply and demand between drivers and users. Grab says that to calculate the final price, the algorithm considers factors such as traffic, location and the number of available drivers, but does not provide clear explanations as to when the surge fee is applied and exactly how it is calculated.
Our investigation, done by Philippine Center for Investigative Journalism (PCIJ) in partnership with the Pulitzer Center’s AI Accountability Network, revealed that this extra fee is always present, regardless of the time and location at which a ride is booked, and that paying for more expensive rides does not necessarily result in shorter waiting times. We came to this conclusion after collecting and analyzing thousands of data points.
The challenge of getting the data
One of the main problems in reporting on proprietary algorithms is the lack of transparency. Grab's pricing algorithm is a classic example of the so-called "black box": the system gets an input, in this case, the pick-up and drop-off locations selected by the user, and then returns an output, the price to be paid.
Between input and output there is a series of calculations involving a number of variables unavailable to the users.
Since we cannot see what happens inside the box, we worked systematically on the inputs and outputs to report this story. We chose a set of inputs (pairs of pick-up and drop-off locations) and held them constant over time, in order to analyze what happened to the outputs.
For one week, we collected data on Grab prices throughout the day for 10 routes in Metro Manila. We collected the data from two different sources: manually from the app itself, simulating the booking of a trip, and automatically through a public Grab API called Farefeed, that was available in the company's online fare check tool. An API, or Application Programming Interface allows you to interact more directly with the code and data of a program, circumventing the user interface, and usually allows you to systematically extract data from the program.
In total, we gathered 1,328 data points from the app and 6,720 from the API. Using two different sources allowed us to complement the limitations of each method and have different variables to analyze the data more thoroughly.
To explore the data we collected click here.
Two data collection methods
A team of 20 researchers attempted to book rides on 10 routes across Metro Manila every hour from 6 AM to midnight between February 14 and 20, 2024. At each booking attempt, they took screenshots of the app's booking page with information about the fare for a GrabCar four-seater, the estimated time of arrival at the destination, and the suggested route.
After each booking attempt on the app, the researchers also took a screenshot on Google Maps with the route automatically suggested by Google. The goal was to record the time and distance recommended by Google, since the app doesn’t display the distance.
If Google Maps’ default route didn’t match the app's, the researchers mimicked Grab’s route on Google Maps and took a second screenshot. This was necessary to get the estimated trip distance, a key piece of data for breaking down the fare.
The researchers conducted a dry-run of the process to detect shortcomings of the methodology and improve it before proceeding with the actual data collection.
During the same period as the manual collection, we created a Python script to fetch data from Grab's fare check API for the same 10 routes, but every 15 minutes, to get more granularity.
How we analyzed the data
After piecing together the data from both sources and checking for consistency, we did a breakdown of the fares using the fare matrix approved by the government.
Once all the fares were broken down, we found that a surge fee was always present. With the help of statisticians from two local universities, we conducted a statistical analysis to scientifically determine whether the waiting time for a ride decreases when the surge fee is higher.
The results didn’t show a significant correlation between the surge rate and waiting times. This contradicts the assumption that a higher rate attracts more cars to the street and lowers waiting times.
Lessons learned on how to report on algorithms
- Although we cannot get into the black box of an algorithm, we can obtain very relevant information if we work on its inputs and outputs. The key is to design a methodology for systematic data collection.
- When designing a manual data collection experiment, it’s important to test it before conducting the actual collection. The pre-meetings and the dry-run allowed us to establish a clear and consistent baseline to work with the data, identify weaknesses and improve it.
- You don’t need advanced coding skills to audit the outputs of an algorithm: a well-done manual data collection can be enough.
- When auditing an algorithm, think of other data sources that can complement the missing information. In our case, for example, Google Maps allowed us to establish the estimated distance of the trips, a data that neither the API nor the Grab app provided.
- If you have a lot of data and variables to analyze and the relation between them is complex, the help of a statistician will allow you to identify relevant patterns but also to avoid making hasty assumptions. It also strengthens your conclusions.
- When researching a technical topic, such as an algorithm that calculates fares, it’s key to include the human perspective as well. Testimonials from users and drivers allowed us to show how people are impacted by these systems and the true scale of the problem.
Photo by Marfil Graganza Aquino via Pexels.
This story was originally published by the Pulitzer Center. An edited version was published on IJNet with permission.