Model risk management

“To be or not to be”: Understanding what is and is not a “model”

By Azer Hann and Bevan Ferreira

Organizations are turning more and more to the use of models for the facilitation of strategic decision-making and to build competitive advantage. However, a lack of comprehensive and conceptual understanding of what models fundamentally are, how they can generate risk, and how that risk can best be managed remains.

What, indeed, is a “model”? Your interpretation could have an enormous impact on what is in-scope when it comes to model risk management (MRM), and consequently the level of effort and cost required to oversee it. The Office of the Superintendent of Financial Institutions (OSFI) expects all financial institutions (FIs) to gauge inherent model risk by identifying and cataloguing their existing models, as well as the scope and impact of any additions or updates. Your organization’s model definition must apply more accurate – and more flexible – criteria than they used to.

Too rigid a definition can increase risk
Any definition of model must include dimensions of both policy and process, delineating core attributes while also accounting for practical use and function. Strict adherence to a prescriptively narrow definition is not recommended.

Currently, many organizations apply rigid definitions with binary rigour. That leads to MRM being applied only when the specific definition is triggered, and not when it isn’t. This can sometimes prove overly simplistic and restrictive, and can be less sensitive to risk. For example, if a de facto model is incorrectly designated as outside the definition’s stringent parameters (i.e., as not a model), the appropriate mitigating activities may not be undertaken.

Understanding model complexity
Accepting that a model is not always covered by a rigid definition is important, but how do you then characterize additional complexities and categorize resulting models for MRM? First, organizations should be careful to consider model aspects that are both upstream (model inputs, including data, people, and processes) and downstream (model outputs and organizational interactions, including strategic decisions, capital allocation, etc.).

Next, it helps to realize that models and their included processes, functions, and data may occupy more than one category and can extend from the simple to the complex. For example, a Black ’76 option pricing computation can be run easily. However, it builds in a variety of fairly strict assumptions, and also requires careful selection of data and calibration of inputs. Along with Black ’76, models in FIs cover a wide range of types and uses, including:

• Regression models for econometric and stress test forecasting
  
• Monte Carlo simulations to generate exposures for counterparty risk
  
• Value-at-risk models that include very complex pricing and calibration functions, as well as less sophisticated but very data-intensive risk aggregation techniques
  
• Fraud detection and prediction models
  
• Data-mining procedures to assess customer behaviour
  

It’s also important to differentiate accurately at the other end of the spectrum, when something has model-like characteristics but does not in fact qualify as a model. To illustrate, consider the difference between a calculation and a model. Simply put, basic arithmetic (e.g., adding things together) is calculation; once an approximation is made or judgement is added, the result is a model. In other words, complex calculations aren’t necessarily models, and simple judgments can transcend mere calculation.

Models are essentially the simplification of real-world processes, which frequently rely on management judgment and/or approximations. Models typically apply statistical, economic, financial, or mathematical theories, techniques, and assumptions to process input data into quantitative estimates for inference, decision-making, and prediction.

Holistic thinking can drive effective model usage
Understanding what models are and the impacts of their use can lay the ground work to understanding model risk management. There’s been considerable debate over whether certain calculations should be categorized as a model. With our practical experience, Deloitte can help you understand and navigate these shifting, often complex, perspectives. In fact, in our next post, we’ll talk about some practical ways you can get your model inventory working for you.

Deloitte has deep industry knowledge and global experience helping banks comply with model risk management requirements. Our comprehensive model risk management framework covers a range of governance and policy considerations. We can help you close gaps in current practices and identify opportunities to revise and improve key processes.