Model risk management

Get your model inventory working for you

Efficient model implementation takes careful planning

By Matt Devine

In order for models to be trusted predictive tools and consistent drivers of decision, your organization must have strong and integrated model governance. In addition to developing and maintaining a model inventory and model usage policies, there needs to be a clear, inclusive definition of exactly what the organization considers a model to be. With these measures in place, you can more confidently and consistently reap the strategic benefits of analytic modeling. Moreover, you will also be prepared for regulations that demand more robust governance and model risk management (MRM) from financial institutions.

Many organizations have already established a framework and an inventory for categorizing and tracking models, model use, and updates. These may include information such as the model owner, developer, and user; the type of model; risk and materiality considerations; frequency of monitoring and review; and the model’s current status.

Our previous posts in this series discussed issues and challenges in establishing these fundamental areas of the framework. However, once you are committed to practical and diligent implementation of modeling and model management, what are some of the key considerations around categorisation and usage? What’s involved in really getting your model inventory working for you? And what can it do?

Assess and understand your model inventory

From day one, having a good, well-evaluated model inventory accomplishes several things. First, it can help determine the magnitude of risk you’re facing. For example, how many old (and possibly out-of-date) models are currently in use, and by how much might your valuations be off as a result? If your economic capital estimates are inaccurate due to unreliable models, both regulatory risk and business risk come into play. Likewise, an inability to accurately predict fraudulent activity through modeling could cost you significantly in penalties and losses.

These are just some of the reasons why it’s important to fully understand your model inventory. This will also help you understand and manage ongoing change processes, such as “model drift,” or the degree to which your current models are being used in other ways than originally intended and whether they are still fit for purpose.

An effective model inventory and management system should account for a wide range of potential change management impacts and requirements. These include determining which stakeholders should be involved in the model inventory process and in what sequence, as well as assessing the organization’s overall level of engagement with the MRM concept as a whole. Any required additional training or improvements to internal communication and collaboration processes should be undertaken without delay, as should a full consideration of what role technology solutions could play going forward.

Other questions to ask about model risk management and usage include:

• Do all relevant stakeholders share the same understanding of the type and use of each model?
• Which areas of the organization are less familiar with model risk and management, or are not typically considered under the model risk umbrella?
• What key decisions, metrics, or results are materially influenced by models in use?
• For a given model, what are the degrees of complexity and judgment in use, and what impact will these have on the model risk profile?
• Does your organisation have a view on the potential impact of model risk on the business?

Managing and leveraging your model inventory to optimal effect means fully understanding its complexity, as well as the various change processes that will impact it. Organizations currently making extensive use of models would most likely benefit from reviewing and, if required, enhancing their model inventory and tracking framework. Of course, model inventory practices should be scaled to the size, nature and complexity of the organization but in most cases, asking the right questions and taking appropriate action should prove beneficial.

Deloitte has deep industry knowledge and global experience helping banks comply with model risk management requirements. Our comprehensive model risk management framework covers a range of governance and policy considerations. We can help you close gaps in current practices and identify opportunities to revise and improve key processes.