2023 EMEA Model Risk Management Survey

Audience

CROs, CFOs, Heads of model risk, Heads of model risk and validation, Chief Actuary, Chief Compliance Officer, Head of Data science, Heads of actuarial function, Heads of operational risk, Model risk managers, Model Owners, Model Developers, Model users.

Key findings at a glance

• Model Risk Management (MRM) frameworks now include a broader scope of models compared to 2021, including ESG models as a distinct category.
  
• MRM vendor solutions are most frequently applied in large banks and insurers, with MS Excel still being the most widely used choice of tooling for smaller firms. 
  
• Banks are migrating the model owner role from model developers to the model users, so the model developer takes a supplier role in firms. Insurers face resistance to from model owners to taking responsibility for all parts of the model role.
  
• More than half of the participants apply artificial intelligence (AI) and/or machine learning (ML) modelling techniques, with the majority of participants recognising the transparency challenges of using such techniques.
  

Introduction
During July and September 2023, 119 companies (85 banks and 34 insurers) across Europe, the Middle East and South Africa participated in the 2023 EMEA Model Risk Management (MRM) Survey. This latest MRM survey (following the previous one conducted in 2021) showed an increase in the participation of banks, and marked the inaugural participation of insurers. Our survey results create insights regarding the current state of MRM in banks and insurers across EMEA.

The survey covered four core themes:

1. model landscape and inventory;
2. technology and tooling;
3. governance; and
4. machine learning and AI.

Key findings of the survey
The survey results indicate a clear difference between the maturity of MRM in banks and insurers.

In the banking sector, the concept of model risk is well recognized, but the maturity of implementation varies by model types and use, with incomplete and partially effective Model Risk Management as a result.

In the insurance sector, the model risk topic is maturing and increasingly being recognized as a distinct and important risk type. More model types and uses are being identified within model inventories, as needing effective Model Risk Management.

Diving deeper into the four core themes of the survey, the results show some clear trends and patterns (compared to 2021).

Model landscape and inventory

• More than one third of banks and two thirds of insurers have developed ESG related models. 
  
• ESG models are included in the scope of MRM for more of the large banks and large / medium sized insurers, compared to the smaller banks and insurers respectively.
  
• Banks’ MRM frameworks include a broader scope of models compared to our survey results from 2021, with a greater number of compliance-related models (e.g., AML, KYC and transaction monitoring) being included within model inventories.
  

Technology and tooling

• Banks have been shifting in MRM technology and tooling from MS Excel to in-house developed solutions and vendor solutions. For insurers, MS Excel is more typically used as the basis for MRM tooling. 
  
• Vendor solutions are more often applied in large banks and insurers, but MS Excel is still the most widely used tooling. Over half of the medium and small insurers use tooling for MRM, and they use in-house solutions (including MS Excel based tools) more often. 
  

Governance

• For insurers, the main governance challenge is to ensure model owners take responsibility for all parts of the model. 
  
• For banks, the role of model owner is increasingly allocated to the model user, reducing the double role of the model developer + model owner.
  

Artificial intelligence

• More than half of the participants apply artificial intelligence (AI) and/or machine learning (ML) modelling techniques.
  
• Survey respondents indicated that the model definition was updated to include AI and/or ML models within the scope of the MRM framework. Around half of the respondents have no policies regarding generative AI and Large Language Models (e.g., ChatGPT, MS Copilot or Gemini). 
  
• AI/ML techniques are mainly being used for Anti Money Laundering (AML), transaction monitoring, fraud detection, enhancing customer experience, marketing and business decision-making, plus insurance pricing and underwriting models.
  
• The majority of participants recognise the transparency challenges relating to the use of AI/ML models.
  
• Currently, three out of five participants agree (at least to some extent) that AI/ML is critical to their organisation’s overall success in the next 5 years. 
  

Regulatory expectations on Model Risk Management have continued to expand since the publication of SR 11-7 Supervisory Guidance in 2011. The UK PRA supervisory statement on MRM principles for banks (published in 2023) highlights increased expectations for a wider scope of models. The EBA consultation on a supervisory handbook regarding the validation of IRB models (October 2022) clarified expectations for checks and controls. The Central Bank of the UAE publication of the Model Management Guidance in November 2022, illustrates the global expansion of MRM standards. Further MRM regulatory requirements are expected with the forthcoming EU AI Act, placing a strong emphasis on AI/ML-related topics, driving requirements for MRM into new industries and sectors.

The following sections elaborate on the implications for banks and insurers. If you would like to skip ahead to the section relevant to you, you can use these links for banks and insurers.

Implications for banks

The survey results revealed the following insights into the current state of MRM frameworks in banks. The model inventory is a central repository for model-related information and is the foundation of an effective MRM framework. It defines the scope of MRM in the bank and is the main source of information about model risk. While financial risk models are most frequently included in the inventory, 65% of participating banks have started to include other types of models, such as compliance, cyber, marketing, and HR models.

Successful MRM framework implementations are often supported by MRM tooling that integrates the model inventory, document repository, lifecycle management and workflow, and analytical and reporting capabilities into a single platform. The survey results indicate that banks have shifted from MS Excel to other in-house developed tools in recent years, as they recognise the need to move away from MS Excel-based solutions to further improve their MRM processes in their organisations.

Strong model governance across the entire model lifecycle is a key requirement for the MRM framework. The model owner role remains important, with 87% of responding banks having clearly defined and documented this role. Model owners are increasingly separated from model developers, and model users are acting more often as model owners. However, the key areas that need improvement relate to governance and controls, including monitoring and validation.

The head of the MRM function reports directly to the CRO in 46% of the banks, which is considered leading practice, while for the remaining 54% the head of MRM reports to a level below the CRO or conforms to another reporting structure; indicating that model risk is perceived as less critical than other risk types in enterprise risk management.

More than half of the banks surveyed are using some variation of AI/ML techniques, with 80% of large banks using such techniques compared to only 20% of small banks. Only 33% of banks have analysed the impact of the EU AI Act (expected to be adopted into EU law in H1 2024) on their businesses, and 31% of respondents found that their organization uses high-risk AI/ML models which will attract more stringent requirements (including those around governance, risk assessments and validation). Three out of five banks agree that AI/ML is critical to their organisations’ overall success in the next 5 years.

Implications for insurers

The survey results revealed the following insights into the current state of MRM frameworks in insurers. The model inventory is a central repository for model-related information and the foundation of an effective MRM framework, starting with a clear organisation-wide definition of a model. According to the survey, 74% of participating insurers have a documented model definition, compared to around 90% of banks. 60% of participating insurers have started to include other types of models (e.g., market risk models, ESG, cyber, marketing, and HR models) in the model inventory, which are not always subject to regulation. Further work is required to ensure consistent capture of the information that defines the scope and scale of MRM.

Successful MRM framework implementations for insurers are often supported by MRM tooling that integrates the model inventory, document repository, lifecycle management and workflow, and analytical and reporting capabilities into a single platform. However, according to the survey results, 40% of participating insurers stated that they do not use tooling for Model Risk Management.

Strong model governance across the entire model lifecycle is a key requirement for the MRM framework. The role of the model owner is important, with 74% participating insurers having clearly defined and documented this role and 64% appointing model owners from outside the model development teams.

More than half of the insurers surveyed are using some variation of AI/ML techniques, with 67% of large insurers using such techniques compared to only 27% of small insurers. Finding a balance between innovation and risk mitigating measures is crucial for organisations, as it ensures that AI models enhance accuracy while upholding fairness and stability within risk modelling. Our survey found that more than three out of five insurers agree that AI/ML is critical to their organisations’ overall success in the next 5 years. However, 86% of participating insurers have not established processes, methodologies, or tools to ensure the fairness of their AI/ML models.

Regulatory expectations for MRM are considered to be less mature within the insurance sector but are expected to increase rapidly. Specific requirements on model risk within the context of the operational risk framework and for internal capital models are already in place. Furthermore, we anticipate supervisors to increasingly focus on MRM, attaching great importance to the board’s oversight and challenge of the model; effective, independent model validation; and the organisational status of model risk management which enables these. Deloitte's MRM expertise can help insurers navigate these challenges and implement effective MRM frameworks. With a deep understanding of the implications for insurers, we can help build strong model governance across the entire model lifecycle, implement effective MRM tooling, and leverage AI/ML techniques to enhance accuracy while upholding fairness and stability.

Closing remarks and next steps

At Deloitte, our mission is to help our clients become more responsible businesses that can grow sustainably, and we believe that MRM is a crucial factor in achieving this goal. A mature MRM framework creates insights into the entire model landscape of the company, raising awareness and mitigating model risk across all steps of the model lifecycle. Deloitte's expertise in Model Risk Management can help firms stay ahead of regulatory expectations and implement effective MRM frameworks that meet the requirements of all stakeholders, ensuring management teams have appropriate safeguards to implement and use models to make better business decisions. 

The increasing risks from AI and machine learning are recognised by regulators and risk practitioners around the world, making MRM more critical than ever before. We encourage readers to reflect on our survey and continue the conversation about Model Risk Management, to help foster a deeper understanding of the challenges and opportunities faced by banks and insurers. 

As a next step, Deloitte will conduct a webinar to present the survey reports, followed by the release of the reports.

Finally, we would like to express our gratitude to all the survey participants for taking the time to provide responses and share valuable qualitative insights that have formed the foundation of this blog.