Uber is a worldwide marketplace of services, processing thousands of monetary transactions every second. As a marketplace, Uber takes on all of the risks associated with payment processing. Uber partners who use the marketplace to provide services are paid for their work even if Uber was unable to collect the payment. Fraud response is thus a very important operational component of Uber’s global marketplace.

Industry-wide, payment fraud losses are measured in terms of the fraction of gross amounts processed. Though only a small fraction of gross bookings, these losses impact profits significantly. Furthermore, if a fraudulent activity is not discovered and mitigated immediately, it could soon be further exploited, resulting in serious losses for the company. These dynamics make early fraud detection vital to the company’s financial health.

Modern fraud detection systems are a combination of classic 1980s AI (also known as an “expert system”) and modern machine learning. We would like to share the journey on how we build the best-in-class automatic fraud detection system and process, leveraging both machine algorithms and human knowledge.

Explainability Of Decisions

It’s important to understand the value of expert-driven rules in fraud detection. Explainability is of paramount importance when it comes to fraud detection. Uber plays the role of the societal operating system. Incorrect fraud-related decisions can be disruptive not just to individual users, but to whole communities. 

Fraud cannot be stopped by some black-box AI algorithms. The reliance on the engagement of human experts and human-readable rules gives the opportunity to humanize the process, examine past decisions, and have full traceability for every decision.

Signals of Payment Fraud

Following are the 2 primary payment fraud types that we are concerned with:

• DNS stands for “do not settle”, which occurs when the order has been fulfilled or a trip has been completed, but the payment could not be collected
• Chargeback happens when the payment has been collected, but the user disputes the transaction, and the bank refunds the money

In both cases, the losses detected change over time. If an analyst analyzes losses from transactions made on one day, these losses can change depending on what day they look at the data. This is because the data may not be fully mature. Payments that are initially uncollected can be collected after the date of the transaction. Alternatively, users may only choose to dispute losses days or weeks after the initial transaction date. Therefore, losses for a single day may increase or decrease over time. However, when enough time has passed from the initial transaction date, the fluctuations in losses will be minimal, and these losses are termed fully mature. 

Therefore, when analysts see early spikes in immature losses, they need to make a judgment call on whether or not these spikes are due to increased activity on the platform, or are signals indicative of fraudulent activity.

The Role of Risk Analysts in Manual Fraud Detection

Fraud analysts use thousands of fraud signals to manually set up the rules that stop various fraudulent operations. Some signals come from the marketplace transaction and some are produced by various models. This process is highly manual, labor-intensive, and time-consuming. However, the challenge with using supervised learning models to stop fraud is that they are great at catching past fraud patterns, but can’t stop unseen and evolved fraud patterns. Therefore, analysts play an integral role in analyzing, detecting, and mitigating fraud at Uber.

What is RADAR?

RADAR is an AI fraud detection and mitigation system with humans in the loop.

RADAR monitors fine-grained segments of Uber’s marketplace, detects the start of a fraud attack, and generates a rule to stop it. The fraud analysts are involved to review the rule and approve it as needed.

Uber has to deal with millions of unpaid orders in every region across the globe, and across many different payment methods used by its customer base. Manual investigation has the benefit of catching new fraud MOs that have never been observed before or are difficult to detect algorithmically, but it is time- and labor-intensive.

By introducing RADAR, we can ingest and analyze fraud patterns at a much faster rate, and only pull in analysts to investigate and mitigate the most complex instances of fraud. Therefore, we get the best of both worlds: the ability to respond to and mitigate new instances of fraud through continued use of manual review, while simplifying existing fraud detection and mitigation processes through a data-driven AI approach with RADAR.

RADAR fraud protection rules are generally short-lived and targeted reactions to the unexpected attacks. Each rule is monitored and carefully measured for relevance. Eventually the more generic models and rules cover the pattern detected by RADAR.

Environment and Engineering Processes

RADAR runs on top of Apache Spark^® and Peloton. Peloton helps us easily schedule the jobs and scale as needed to process the data. Apache Spark^® provides an easy-to-use Python^® and Scala API, which lets us separate data processing code from data access code. This makes it possible to do unit and integration testing of data processing and AI code. 

We have a rigorous process of automated testing for RADAR data pipelines, where synthetic data is used to validate the anomaly detection algorithms. Our assumptions about data generation processes and attack patterns are described in the code and probabilistic programming models. This makes the engineering team much more responsive to changes in the data generation processes and attack patterns.

RADAR Detect

RADAR detects fraud attacks by analyzing the activity time-series from the Uber payment platform and converts data insights into human action.

We look at the data in 2 different time dimensions. We will define some terminology to discuss time series data below:

• OT – “Order time” when the specific order has be