GRAPA Machine Learning

Case study

Combating welfare fraud with machine learning

Case 11/16 of applied Artificial Intelligence

A government organisation in the Netherlands had to contend with cases of fraud. The organisation was responsible for paying allowances to citizens, but hundreds of millions of euros were being paid to ineligible persons. An internal department had identified that processes were not running effectively, and in the aftermath of negative publicity it was clear that the method of tackling fraud needed to be improved.

The existing method consisted of human assessors who performed random checks based on what are known as business rules: patterns established by fraud experts that can point out when something is amiss. “We wanted to find out whether it was possible to improve processes with the aid of machine learning,” explains Hilko van Rooijen, Senior Manager of Financial Crime Analytics at Deloitte. “The benefit is twofold: fewer citizens are wrongly suspected of fraud, while the inspectors are able to spend their time more efficiently.”

Increasing the hit rate

Within four weeks Van Rooijen’s team had created a proof of concept. Following a comparative study of various artificial intelligence technologies, they decided on a form of logistic regression. Huge numbers of past cases were fed into the system, enabling it to learn in which new cases something was suspected to be wrong. The result was amazing: where cases of suspected fraud were assessed by human experts, fraud was found to have actually been committed in 10 per cent of cases; this hit rate rose to 50 per cent in the proof of concept.

The technology was improved and refined over the subsequent months, and one of the challenges in this respect was ensuring the representativeness of the data. “The system learns from the information you feed into it, so it is important that this information gives the fullest picture possible,” says Van Rooijen. “Once, for example, we used a sample containing many older people who had not committed fraud, and the system then concluded that older people never commit fraud. We had to correct that.”

A not insignificant part of the process was integrating the system into the corporate culture. “We did that by providing detailed explanations, but also by asking for input,” reveals Van Rooijen. “Staff within the organisation had a wealth of expertise about fraud, and that knowledge proved to be extremely valuable in developing the technology.” The system has since been connected to the existing IT infrastructure, and an internal team has been trained to manage the technology.

From detection to prevention

After six months, the technology’s hit rate had increased to 87 per cent. Another six months later, it fell again. “It showed us that simply less fraud was being committed,” remarks Van Rooijen. “Obviously, the detection techniques had become so effective that they served as a deterrent to fraudsters. As well as providing a means of detection, the technology had also begun to act as a form of prevention.”

The approach has successfully prevented fraud worth tens of millions of euros. “And those savings are still ongoing,” adds Van Rooijen. Deloitte has since assisted public authorities in various other European countries in the integration of this technology.

*) This case is part of the series of 16 Artificial Intelligence projects from Deloitte. Other cases in the series are in random order:

  1. TAX-I: A virtual legal research assistant
  2. AI Benchmark 
  3. SONAR: Find labelling errors in databases
  4. Transaction detector with regard to the Dutch work cost regulations
  5. GRAPA: assistance with risk strategies
  6. Chatbot as a handy search tool for the online technical library
  7. Argus: an eye for detail
  8. PostNL: optimising delivery times
  9. Virtual assistants: beyond the hype
  10. HR agent Edgy: the future of Human Resources
  11. Using machine learning to assess risks for insurance policies
  12. Predicting payment behaviour
  13. DocQMiner: contract analysis performed in no time at all
  14. Combating welfare fraud with machine learning
  15. Using machine learning and network analytics to search for a needle in a haystack
  16. Clustering unstructured information in BrainSpace

Sign up for the email alert to get all cases through email.

Vond u dit nuttig?