Ten principles for effective application of advanced analytics in transaction monitoring
Although it is widely recognized that advanced analytics can make Transaction Monitoring (TM) much more efficient and effective, financial institutions (FIs) are searching how to apply the advanced and complex methods in a way that is effective, transparent and acceptable to all stakeholders.
Is TM a model?
Central in answering the question if TM and especially advanced analytics methods are effective, is the way this can be validated. In various jurisdictions the debate whether TM should be perceived as a model and therefore should be subject to model validation, is already initiated or completed. A common point of departure is the question whether model standards applicable to the domain of financial risk (such as FED’s supervisory letter SR 11-7 and FIs internal standards) can be applied to anti-money laundering (AML) risk monitoring as well. While these standards obviously may provide structure to the question whether TM is effectively implemented, they have generally not been designed with application to non-financial risk models in mind. Therefore, applying existing model validation standards on TM will often lead to dead ends in the quantitative substantiation of TM outcomes as well as in the assessment whether regulatory expectations are sufficiently fulfilled.
The need for standards of development and governance specifically for TM (and perhaps for anti money laundering measures in general) is evident. With increasingly advanced and therefore more complex monitoring, the question how the validity of these methods should be assessed becomes ever more relevant. Specifically for the more advanced and data-driven methods, additional safeguards and conditions may apply so that all stakeholders can get an understanding of the effectiveness and reliability of TM. The existing validation standards may serve as inspiration and guidance, but will require tailoring for successful application within the field on AML. In our previous blog where we identified opportunities for advanced data analytics in TM. Now, we present ten technical principles that could be considered to enable effective application of advanced techniques within TM.
The 10 technical principles
- Know your data: ‘garbage in, garbage out’ is a well-known adage in data analytics, so an advanced TM model can only be valid and meaningful when based on valid data. This implies the need for identification and mitigation of issues with data quality, but it also means that the data must be tested for bias and artifacts that may adversely impact model performance and invalidate results.
- High quality labelling: specifically for supervised machine learning, monitoring models can only be valid when the historic data on which the model learns to predict AML risks consists of cases that are properly labelled as high or low risk. Moreover, it is important that this historic set comprises as many relevant AML risk scenarios as possible. When the model is not ‘fed’ with a wide variety of cases, it cannot learn to identify those cases in future client behavior.
- Make the model fit for purpose: advanced analytical techniques do not provide magic black boxes that instantly produce meaningful results. Appropriate techniques should be applied to perform precise monitoring tasks. It is key for success to identify and scope each monitoring task, and adopt the right analytical method for this task. As stated in our previous blog, the whole set of monitoring tasks should be addressed by an orchestrated combination of monitoring techniques, each performing an appropriately designated task.
- Risk based and data-driven: traditional TM designs are based on the expert based recognition and mitigation of AML risks. More advanced methods rely on (historic) data to determine relevant AML risk scenarios. It will be key for success to integrate these advanced methods with a risk based methodology. Only this will guarantee that TM is both risk based (what are the monitoring tasks to be performed?) as well as data-driven (what risk signals are truly relevant?).
- Respect the expert: although data evangelists may believe and preach that data-driven methods will outperform human expertise, this expertise is crucial to make the model fit for purpose and evaluate its effectiveness. A successful application of advanced TM utilizes human expertise to identify monitoring tasks, select and understand relevant data, and to evaluate model outcomes to facilitate meaningful feedback on model performance.
- Validation samples: in addition to analytical validations based on historical data, advanced techniques should always be tested on current transaction profiles before models are brought into production mode. This testing, which typically involves cases the techniques classified as high risk as well as low risk, may identify specific errors or flaws in model performance that are not evident in validation data sets.
- Transparency: all stakeholders will need to understand what is going on in TM. With increasingly complex techniques, the possibility for even expert users to understand what drives performance and identify possible pitfalls such as hidden biases decreases quickly. Although we do not think that all stakeholders should be able to understand all elements of a technique to its finest details, transparency of the why and how of advanced monitoring techniques (including key assumptions) is critical to gain sufficient stakeholder support.
- Explanation: even in the case of advanced analytics techniques, the operational outcomes of TM will still be analyzed by human ‘alert’ handlers to identify the actual AML risks. For efficient analysis, a meaningful explanation of the monitoring output (e.g., what triggered this particular risk signal and what are the relevant data points?) should be available for the any AML professional assessing the actual risk of the flagged transaction profile.
- Backtesting: in addition to testing TM before releasing it for production, the effectiveness of TM should be continuously evaluated by means of data analysis. This can help identify opportunities for enhancements as well as developments that impact reliability. Although this principle is not unique to advanced monitoring techniques (it equally applies to traditional TM business rules), backtesting is even more important for advanced techniques because those methods often evolve automatically and continuously based on new data as it becomes available.
- Model governance/validation: as discussed in the introduction, the need for broad governance and validation measures for TM is increasing in line with the growing complexity of TM processes. The purpose is clear: all stakeholders should be able to understand what has been done to manage TM effectiveness and that this has been done according to applicable standards. This should encompass all relevant aspects from TM, from a risk, data, IT and operational perspective. Existing validation standards may help to shape this process, but the process must be tailored to AML and evaluated by TM professionals who can grasp all relevant concepts in the TM process.
Application of the principles
Many FIs are already taking the step to constitute internal policies and standards for TM validation. We strongly encourage this development, as the application of advanced techniques can only create value when done right. However, from another perspective, further institutionalization of the field should not block advancements and investments in the domain. We also encourage regulators, supervisors and compliance professionals to understand that TM is shifting to its next paradigm. In this new paradigm, data and the advanced usage of data is, in combination with risk based methods, the key determinant for the design of AML monitoring methods. Therefore, new policies and standards must take the opportunities, requirements and implications of advanced analysis for TM into account. The principles presented above are not intended limitative nor prescriptive. However, we hope to further provoke the discussion around this topic and so bring the parties together and TM to the next level.
More information on Transaction Monitoring?
Do you want to know more on Transaction Monitoring? Please contact Hilko van Rooijen at +31 (0)88 288 7771 and Evert Haasdijk at +31 (0)88 288 0157.