Fighting financial crime: How to safely profit from advanced analytics AML models?

Two trends that impact the AML framework

The last years have seen a substantial increase in regulatory scrutiny on anti-money laundering and counter terrorism financing (“AML”) implementations at financial institutions. Globally, billions of dollars in AML enforcement fines are levied annually. To maximise the efficacy and to minimise the risk of exposure to regulatory enforcement, financial institutions continue to invest in AML measures.

However, it is not always clear whether these investments will achieve the desired effect and result in accurate models that detect money laundering and terrorism financing. This is where validation of AML models comes into play. We identify two developments that impact AML frameworks and particularly the way they are being validated.

The first trend is the increasing complexity of the models underlying the AML implementations, particularly those that rely on machine learning or artificial intelligence (“AI”) based techniques. The second is the development of regulations that regard algorithmic decision-making. These two trends obviously interact: the reliance on more complex AI-based models to detect money laundering risks increases the need for mature governance to support model risk management.

These developments combine to affect the scope of AML validations, in which it is typically questioned whether a financial institution is in control over AML risks and regulatory obligations. In that context, we differentiate three ways to assess models in an AML validation:

1. in the context of the overall AML framework;
2. in the context of technically validating model development and implementation;
3. in terms of model risk management.

Each of these aspects implies the need for specific expertise to guarantee a well-founded appraisal of an institution’s AML practice in that regard.

Models in AML

Before we dig deeper into the different validation approaches, let’s first describe what models are and what role they play in AML.

Any implementation of AML measures uses models. Models assess client risk levels, identify potential money laundering behaviour, estimate expected transaction profiles, and so on. A model in the sense we use here is a method to estimate some quantity (e.g., the risk level of a set of transactions being part of a money laundering activity) from input data (e.g., attributes of the transactions and the parties involved in it). The model can be based on expert knowledge and/or statistically estimated relationships. A simple example could be a money laundering risk indicator that raises an alert if an account holder deposits cash amounts over a certain threshold.

The accuracy of these models determines the efficiency and efficacy of a financial institution’s AML activities. Models that often incorrectly label harmless behaviour as an indication of financial misconduct cause large numbers of false positives, overwhelming AML analysts with unproductive and unrewarding work. However, models that fail to recognise proper signals of financial misconduct undermine the gatekeeper role that regulators demand of financial institutions.

To date, the models used in AML implementations are mostly straightforward rules such as the example above: the rule compares the total amount of a set of cash deposits with a threshold to determine if an alert should be raised. Increasingly, however, financial institutions are turning towards more complex models, including machine learning models, to improve efficiency and efficacy.

Some examples of machine learning methods that we see being adopted in this context include techniques such as anomaly detection, clustering and classification. Anomaly detection, for instance, is often used to identify unusual client behaviour indicative of money laundering. Clustering can define peer groups and their expected transaction profiles. We mostly see classification models that deprioritise or even automatically close probable false positive alerts.

These and similar advanced analytics techniques improve both efficiency and effectiveness of AML activities. They increase the true positive rate of handled alerts in transaction monitoring, decrease the cost of customer due diligence and at the same time, enable the detection of more complex money laundering behaviour that goes undetected by traditional techniques. On the other hand, the increased complexity of the models implies increased model risks such as of faulty model development, (indirect) bias, inexplainable decisions, or incorrect interpretation of results.

To reliably exploit the potential offered by advanced analytics requires ample expertise on the technical aspects of model development in combination with AML expertise as well as mature governance to mitigate the model risks.

Models and AML validations

A validation of an AML framework must cover the models that implement steps in that framework. We can characterise AML validations in terms of the focus on the design and functioning of the models themselves, and on the governance and policies concerning the development and implementation of the models.

Conceptual AML Validation

The first level includes conceptual AML validations that consider models in terms of their role within the AML framework. They look at a model as a black box, disregarding how the model comes to an estimate, but closely regard what it is that the model estimates and how that fits in with the other components of the AML framework. Such validations are crucial to have confidence that the AML framework as a whole is sound and complies with rules and regulations such as the Wwft and DNB guidance. In short, conceptual AML validations answer the question whether the AML framework covers all relevant risks and whether it is compliant to regulatory AML obligations.

A conceptual AML validation considers whether the models are fit for purpose and fully address all the risks listed in the enterprise risk assessment for AML. This view also ensures that the risks covered by the models are consistent with sound industry practice and regulatory guidelines. Other issues considered are whether the data used by the models is sufficient and reliable, whether signals raised by the models are properly followed up, whether appropriate above- and below-the-line tests are in place, and so on.

Validations that consider models at this level of detail require ample experience of AML implementations as well as a solid understanding of industry practice and relevant guidance and regulations. The use of complex models implies the need for technical expertise as well, because complex models can be difficult to interpret. Thus, it may not be immediately apparent how or even if they relate to identified AML risks.

AML Model Validation

AML model validations take a more detailed look at how the models are developed and how they arrive at their estimates. This is similar to model validation as a longstanding practice in financial risk models. Such validations verify that models perform as expected and are in line with the requirements posed by the AML framework. In other words, this answers the question whether the models technically perform as they should.

Model validations have a strong technical character, assessing for instance whether the models have been developed according to accepted standards, whether design choices are supported by empirical or theoretical evidence, or whether statistical estimates of model performance are reliable and relevant. It also includes technical testing for a thorough appraisal of the reliability of the data sources and for verification of the implementation.

In addition to AML expertise these validations require a thorough understanding of modelling techniques, their strengths and their limitations. Experience with advanced analytics is crucial for an informed view of model development, data analysis and assumptions.

AML Model Risk Management

A rapidly developing set of regulations regards the risks of advanced models, prompted by various examples where models were implemented without proper considerations of the right to personal data protection, the risks related to discrimination and bias, ethical considerations around the scope and functioning of models, or the impact of automated decisions. Relevant guidance is provided by, for instance, the venerable US Federal Reserve’s Supervisory Guidance on Model Risk Management (SR 11-7), DNB’s SAFEST principles as well as the proposed EU Artificial Intelligence Act.

These regulations also cover AML models, and their risks must be managed to comply. Thus, AML validations need to extend their scope and need to go hand in hand with validations that focus on model risk management. In addition to comprehensively analysing the models in terms of their coverage of AML risks and soundness, the scope extends to analysing the controls and policies that ensure responsible development and use of models as well as reviewing whether model risk governance is appropriately arranged. Furthermore, the model risk management for AML models should align with organisation-wide model risk management frameworks.

Validating model risk management includes analysing relevant policies, the procedures that implement these policies and controls that test them. It also considers the standards for documenting model development and validation. Common controls to mitigate model risks include an inventory of the models in use, accurate and complete model documentation, as well as standards regarding transparency and comprehensibility of models and model-based decisions.

Validating model risk management requires knowledge of the developing regulations and comprehensive understanding of best industry practices. Experience with advanced analytics provides the context needed for a well-founded analysis of model documentation and transparency.

Where to start?

With the combined trends of increasing reliance on machine learning and AI, and the developing regulations regarding algorithmic decision making, the character of AML validations is changing.
We have seen how validations of AML frameworks can focus on the AML aspects, on the soundness of models that underpin them, and on the management of model risks. And while a validation can focus on one or more of these three aspects, the others cannot be entirely disregarded. Doing justice to these aspects requires ample experience with AML implementations combined with expertise about advanced analytics and model risk management.

The first step towards control over both AML and model risks often is an analysis of the maturity of an organisation’s model risk management and AML frameworks. Benchmarking the maturity level to comparable companies and industry standards will help to identify relevant activities to develop, support or validate model risk management as shown in the diagram below.

Conclusion

We believe that the solution to sustainably and effectively perform the ‘gatekeeper of the financial system’ role lies in leveraging advanced analytical models to assess client risks and monitor client behavior. Developing an AML framework that safely exploits advanced analytics implies a complex journey in which risks, perceptions and expectations must be managed. We are keen to help your organization navigate this journey towards the next-generation AML framework.

Read more in the NextGen AML blog series and discover our perspective on a more connected, impactful and sustainable fight against financial crime.