Development of audit tools

As first step, it is important to consider the stages of the model lifecycle (see figure 2) and map the required controls on them.

Figure 2: Model lifecycle

The below questions can help to identify the necessary controls for each step:

Is a model required? What is the business purpose of the model?

What are the key assumptions behind the model (e.g. methodology)? How is it developed and implemented (e.g. python)?

What is the approach for the independent validation proportionate to the model materiality (e.g. review, light touch, full validation)?

Who can approve the model before the go-live? What is the approval process? Do we need a full validation involving, for example, rebuilding a separate model to check the outcome of the one under review?

What is the model purpose and limitations to be taken into consideration to use it?

What are the monitoring metrics required to evaluate the model performance? How often must they be produced? By whom must they be approved?

How is the periodic review (PV) performed (e.g. light touch, full validation)? How often must the PV be performed?

In order to establish a control framework for an MRM, there are some of the key elements to take into consideration:

GovernancePolicy and procedures are essential to establish key points such as model definition, materiality etc.Roles and responsibilitiesAllocation across the usual categories as developer, validators, user, and approver.Model inventoryThis is a main pillar of the model risk framework as it contains critical information such as model owner, weaknesses, validation due date etc.Quantification methodsThey are used to assess and monitor the model materiality according to the risk appetite and business model of the financial institution.Verify the feeding workflow between and among models to determine the parent-child relationship.Version controlDifferent teams use different releases of the same model.Check on data quality and integrity to verify the reliability and validity of the information and whether it's eligible to be used to feed a model for the intended purpose.SkillsThey determine the equilibrium between the key areas of MRM (e.g. development vs validation) because if there is as imbalance, the MRM cannot achieve its scope.ProcessThey are essential to norm the steps of key actives such as model development, validation etc.Generic and subject matter expert (SME) testingDrilling down in the validation stream, it is important to select the type of testing appropriate to the materiality of the individual model.MonitoringIt is a key element of the model development (often ignored at this phase of the model lifecycle) and vital to evaluate if the model is working as expected or if any corrective actions are required as described in detail in the next part of this article.

The outcome and associated guidelines provided by Targeted Review of Internal Model⁷ (TRIM) published by the European Banking Authority (EBA) represent a robust reference for the financial institutions who desire to enhance the above elements of the MRM mainly for the internal models.

Obviously, a robust model risk framework offers many benefits including the following⁸:

Improved decision-making process based on reliable and quality models
Cost-effective as the MRM allows to focus on the more material models and portfolios of the financial institution
Reduce risk of potential loss (financial and non- financial).