Automation tooling: a future-proof approach to modelling


Automation tooling: a future-proof approach to modelling

The Road to Basel IV

The ever-increasing workload throughout the model lifecycle urges the banking sector to innovate continuously. Market developments and regulatory changes (e.g. Basel IV reforms) force banks to redevelop their existing models or develop new ones at an increasing pace. Automation can help to save time and reallocate resources more efficiently. In fact, automating data quality checks, model development, performance testing (for validation purposes) and regulatory tracking & reporting is the key to a future-proof approach to modelling.

This blog will focus on several aspects of the model lifecycle in which automation can be applied, resulting in a more efficient and streamlined workflow for model development, validation and monitoring. Automating repetitive and straightforward tasks will save valuable time and resources throughout the model lifecycle and reduces the amount of errors made. The most promising areas for the use of automation are explored below.

Data quality

Data quality is a key element throughout the model lifecycle, since most models are based on data. For model development or validation, an automated data quality framework can be designed to assess the data quality for multiple data sources according to a set of pre-defined checks. These checks may be generic, such as checking for missing values or duplicate records, but can also be more tailored (e.g. specific range for a variable). Also, for models that are already live, periodic automated checks can be used to assess data quality during the model life.sources throughout the model lifecycle and reduces the amount of errors made. The most promising areas for the use of automation are explored below.

Model development

During the model development stage, automation can support experts with feature selection and benchmarking. Firstly, the domain experts should identify the goal of the model and the corresponding target variable and potential features to be used in the model. Afterwards, an automated tool can provide initial insights into the predictive power of the set of features that are considered for the model, based on repetitive fitting of models with different feature subsets and analysing the results (e.g. bidirectional elimination). Finally, domain experts can combine the quantitative results with qualitative industry knowledge to select the optimal set of features to be included in the model. In the final stages of model development, benchmarking is essential. An automated challenger model can provide fast and reliable benchmarking to assess the quality of the model. In an automated challenger model, the features are analysed (with both single-factor as well as multi-factor analyses) to select a set of risk drivers. Based on these risk drivers, a robust challenger model is automatically trained and tested. Note that before the benchmarking exercise to analyse the predictive power of the models, an expert should do a final check on the generated challenger model for potential flaws.

Performance testing

After the model development stage, automated performance testing can provide a deep understanding of the calibration, stability, and discriminatory power of the model. In order to obtain the essential insights, there are market best practices available as well as (ECB) required performance tests that can be performed automatically. The following dimensions can be considered in automated performance testing: summary tables of the data (such as portfolio information tables or migration matrices), calibration power, stability of the model, and ranking order (differentiation). These tests are similar for many models and portfolios. Therefore, standardisation and automation offer a high return on investment and ensure that resources can be allocated efficiently instead of focusing on repetitive work involved in manual performance testing. The final step of automated performance testing entails automatically writing the results to the model or validation report, minimizing time spent on creating these reports.

Regulatory tracking & reporting

Implemented models remain subject to regulatory changes and reporting. Automation tooling can support the model owner with both tasks in order to ensure a regulatory compliant model and up-to-date reporting. In an automated regulatory tracking process, all relevant regulatory publications are instantly gathered and assessed in order to obtain a clear understanding of regulatory requirements. Automated tooling can also monitor new publications and provide insights into changes in regulatory papers and their impact (e.g. text scraping), which will render manual regulation tracking redundant. In addition, automation can streamline the process of repetitively publishing regulatory reports. As most regulatory reports concern the summing up of similar information, the automatic production of these reports can save time and resources. Instead of producing the regulatory report, the model owner can focus on reviewing it.

In short, the banking sector can save time and resources throughout the model lifecycle and reduce the amount of errors by means of the automation of repetitive tasks. Innovations offer a relief in the workload for the development, validation and monitoring of models. This offers opportunities for experts within banks to refocus their time on the projects that matter most. Please feel free to reach out for more information or for an exploratory meeting to discuss the innovations that await for your business.

Did you find this useful?