Prudential Regulation Authority Rulebook

7.1

Model justification and validation are two separate processes and represent important components of the Directive requirements. The PRA expects firms to demonstrate clearly this demarcation in their implementation.

7.2

The Statistical Quality Standards (SQS) for internal models in Solvency Capital Requirements – Internal Models 11 and Articles 228 to 237 of the Commission Delegated Regulation,^[9] set out requirements in particular on the methods and assumptions to be used in the internal model. Compliance with those requirements will have to be evidenced to the PRA (Solvency Capital Requirements – Internal Models 3.1).

7.3

For instance, firms will have to justify that the chosen actuarial and statistical methods are adequate^[10] and will have to justify the assumptions used taking into account the significance of those assumptions.^[11] Such justifications need to be satisfied as part of the SQS requirements, and it is not the aim of the validation to create a substitute for these requirements. Depending on the firm, such justification either could be completed by the first line of defence (as part of the production of outputs within the calculation kernel), or by the second line.

7.4

In contrast, the validation is a regular and independent (from the development and operation of the model) process which includes reviewing the model in terms of the appropriateness of its specifications, the correspondence of its results against experience and its overall performance over time.^[12]

7.5

Model justification, as part of the internal model development, is often covered by the validation using a bottom-up approach. Part of this includes the justification of why the modelling choices are reasonable and defensible. However, the bottom-up approach should cover the validation objectives fully and help boards and senior management to obtain an adequate understanding of the model.

7.6

The PRA expects validation to be a combination of detailed ‘bottom-up’ testing and ‘top-down’ ownership by boards.^[13] The PRA expects firms to be able to produce clear evidence showing how boards are overseeing and influencing the design of the validation process, how the findings from validation work are summarised and reported to them and how boards are then involved in tracking validation issues through to resolution. The PRA expects firms to demonstrate clearly that boards are using validation as a tool to enable them to gain a good understanding of a model and its strengths and weaknesses. The PRA has high-level expectations on board involvement in validation. See also Chapter 6 ‘Role of non-executive directors’.

7.7

Firms’ use of validation materials that are focussed primarily on the bottom-up justification of parameters and assumptions may not be aligned clearly enough to enable senior management and boards to challenge effectively the key assumptions and limitations of the model.

7.8

Although a bottom-up approach is an important aspect of the internal model validation, the PRA emphasises that boards should value the role that good validation can play in helping them to understand the key drivers and limitations of a model. The PRA expects that firms would be able to provide evidence that the board has:

• challenged the validation process and its results;
• understood and satisfied itself on the key assumptions and limitations of the model; 
• considered the possible quantification of these limitations; and
• taken appropriate mitigating actions.

7.9

The PRA expects boards to be tracking progress actively in addressing key issues identified by validation work.

7.10

The PRA does not expect boards or senior management to be able to discharge their duty in isolation. Boards should demand support from executive management to ensure that key features of models are explained in a way that directors can engage with properly, and they should draw on a wide range of sources, not limited to model developers, to ensure they are satisfied with the model.

7.11

To verify the robustness of the internal model, the T&S are designed to ensure that a model is well grounded in its technical content, with good sources of underlying data. The T&S also require that the model and its limitations are properly understood by its users and by senior management at firms, including the board.

7.12

In summary, a comprehensive validation process should put specific attention on those key assumptions and expert judgments that have a material impact on the model, and should also articulate how the sensitivity to the key assumptions and expert judgement are being assessed and taken into account in the decision process.

7.13

The PRA expects that validation of the internal model clearly evidences the review and challenge that has taken place in assisting the board to meet its objectives.