SS17/16 – Solvency II: internal models – assessment, model change and the role of non-executive directors

1.1

This supervisory statement is addressed to all UK Solvency II firms and to the Society of Lloyd’s, its members and managing agents. It sets out the Prudential Regulation Authority’s (PRA) expectations of firms regarding internal models.

1.2

This statement should be read in conjunction with the PRA’s rules in the Solvency II Firms Sector of the PRA Rulebook, the statement of policy (SoP) setting out the PRA’s approach to permissions and ongoing monitoring of internal models,¹ supervisory statement (SS) 1/24 – Expectations for meeting the PRA’s internal model requirements for insurers under Solvency II² and the PRA’s insurance approach document.³

1.3

This supervisory statement expands on the PRA’s general approach as set out in its insurance approach document. By clearly and consistently explaining its expectations of firms in relation to the particular areas addressed, the PRA seeks to advance its statutory objectives of ensuring the safety and soundness of the firms it regulates, and contributing to securing an appropriate degree of protection for policyholders.

1.4

In this SS, the PRA sets out its expectations for firms in the following areas:

• internal model applications;
• dealing with variability in premium provisions;
• the role of non-executive directors;
• model justification and validation and the role of boards;
• the PRA’s use of quantitative analysis as part of model permission;
• scope, identification and classification, governance and reporting of internal model changes; and
• reporting of analysis of change in SCR.

1.5

Firms should also refer to:

• the Bank of England and PRA SoP – Interpretation of EU Guidelines and Recommendations: Bank of England and PRA approach after the UK’s withdrawal from the EU;⁴
• SS1/19 – Non-binding materials: The PRA’s approach after the UK’s withdrawal from the EU;⁵ and
• SS2/19 – PRA approach to interpreting reporting and disclosure requirements and regulatory transactions forms after the UK’s withdrawal from the EU.⁶

1.6

Any reference to any provision of direct EU legislation is a reference to it as it forms part of retained EU law.

2.1

Firms are reminded that once a formal internal model application has been submitted to the PRA, there is limited opportunity for firms to make substantive changes. Firms should therefore make sure their applications are stable and approved by their internal governance processes prior to formal application. Where firms become aware that they may need to make changes during the application period, these should be discussed with their usual supervisory contact as soon as possible. Where changes are material, a new application is likely to be required. Alternatively, firms might reasonably be able to modify the existing application. If so, the PRA would need to reassess whether the modified application is complete and this may delay the PRA’s consideration of the application. Neither of these options should be approached lightly. If firms believe that significant model changes are likely in the formal application phase, they are encouraged to consider delaying their application to the PRA and to discuss options with their usual supervisory contact. In this respect, the preapplication process is a means to help firms verify they are on the right path before they submit an application.

2.2

The PRA will grant an internal model permission where the firm demonstrates to the PRA’s satisfaction that it satisfies the internal model requirements and calibration standards set out in the PRA Rulebook,⁷ or where safeguards mitigate non-compliance with the internal model requirements and ensure compliance with the calibration requirements. When reviewing an internal model application, the PRA may consider imposing two types of safeguards, a residual model limitation capital add-on (RML CAO) or a ‘requirement’ applicable in respect of firm’s business practices or model usage. For more information on the PRA’s approach to internal model permission applications, or model limitation adjustments (MLAs), firms should refer to SoP – Solvency II internal models: Permissions and ongoing monitoring.

2.3

Irrespective of the progress of the internal model application, firms should have an alternative approach that they can use if they fail to gain permission to use the model, for the purpose of calculating the SCR, after submitting an application, and ensure that they have a clear understanding of the actions they would take in those circumstances. For example, a merger or restructuring may make the existing standard formula inappropriate and therefore the applicant would need to have a contingency plan in the event that the PRA did not grant permission to use the internal model.

4.1

General insurance firms should consider variability in premium provisions on their year-end Solvency II balance sheet. In the same way that events can occur that cause claims provisions to vary, some of those same events will also cause the premium provision to vary. Examples include changes in court awards for liability claims or an unforeseen major natural catastrophe event, like the Tohoku earthquake.

4.2

Firms that do not consider this risk may fall short of the internal model requirements and calibration standards. The PRA considers this risk exists for all actively underwriting internal model firms who model on a one-year earned basis, although it may be that for some firms this risk will be small in the context of their total SCR.

6.1

This chapter sets out the PRA’s expectations regarding the role of the non-executive directors (NEDs) when considering a firm’s internal model.⁸ 

6.2

As part of the use test, firms must ensure that members of the board and others involved in running the firm have a sufficient understanding of the model. One of the methods the PRA may use to assess whether firms are meeting the use test is to speak to NEDs (either individually or collectively) to gauge their understanding. The PRA does⁹ not believe that it is necessary for board members to be technical experts in modelling techniques in order to meet the use test requirements. Rather, the PRA expects board members to understand and be able to explain areas such as the:

• key strengths, limitations, and judgements within the model
• assumptions and judgements that have the most material impact on the model output; and
• key sources of information and advice the board has relied upon in order to satisfy itself about the appropriateness of both the model design and the model output.

6.3

Board members should be supported by the executive management to understand and engage with the key features of models. Boards should draw on a wide range of sources to understand, challenge and make a decision on the validity of a model. For example, independent validation can play an important role in helping boards gain an overall understanding of a model and its strengths and limitations, as long as the validation work is focused appropriately on a critical appraisal of the most material aspects of the model, and if its conclusions are summarised and presented appropriately for a board-level audience.

6.4

Under the leadership of the chair, NEDs should consider and challenge, as appropriate, the executive management on all aspects of the firm’s strategy, which includes the viability and sustainability of the business model and the establishment, maintenance and use of the risk appetite and management framework. NEDs are expected to challenge how these elements are reflected in the internal model.

6.5

The executive management should be able to explain the firm’s internal model in simple and transparent terms to NEDs. This includes explaining the uncertainty around judgements, in what circumstances the results may deteriorate (eg the analysis or strategy could be wrong), and the implications on the internal model of different methods of measurement for issues such as liquidity. In order to explain complexity to the NEDs, the executive management should provide good management information to enable challenge and thus encourage accountability. If NEDs do not feel that they can meet these expectations, they should demand the time and support to enable them to do this.

7.1

Model justification and validation are two separate processes and represent important components of the Solvency Capital Requirement – Internal Models Part of the PRA Rulebook. The PRA expects firms to demonstrate clearly this demarcation in their implementation.

7.2

The Statistical Quality Standards (SQS) for internal models in Solvency Capital Requirement – Internal Models 11 and Solvency Capital Requirement – Internal Models 16.2 set out requirements on the principles that the methods and assumptions used in the internal model must adhere to. Compliance with those requirements must be evidenced to the PRA.

7.3

For instance, firms are required to ensure that the chosen actuarial and statistical methods used to calculate the probability distribution forecast are adequate, applicable, and relevant,¹⁰ and are required to ensure that the assumptions used make adequate allowance for uncertainty. Such requirements need to be satisfied as part of the SQS, and it is not the aim of the validation process to create a substitute for these requirements. Depending on the firm, such requirements could either be satisfied by the first line of defence (as part of the production of outputs within the calculation kernel), or by the second line.

7.4

In contrast, the validation is a regular and independent (from the development and operation of the model) process which includes reviewing the model in terms of the appropriateness of its specifications, the correspondence of its results against experience and its overall performance over time.¹¹

7.5

Model justification, as part of internal model development, is often addressed using a bottom-up approach. The bottom-up approach should cover the validation objectives fully and help boards and senior management to obtain sufficient understanding of the model, including a justification of why the modelling choices are reasonable and defensible.

7.6

The PRA expects validation to be a combination of detailed ‘bottom-up’ testing and ‘top-down’ ownership by boards.¹² The PRA expects firms to be able to produce clear evidence showing how boards are overseeing and influencing the design of the validation process, how the findings from validation work are summarised and reported to them and how boards are then involved in tracking validation issues through to resolution. The PRA expects firms to demonstrate clearly that boards are using validation as a tool to enable them to gain a sufficient understanding of a model and its strengths and weaknesses. The PRA has high-level expectations on board involvement in validation, as described in the following paragraphs. See also Chapter 6 ‘Role of non-executive directors’.

7.7

Firms’ use of validation materials that are focussed primarily on a bottom-up justification of their internal model may not enable senior management and boards to challenge effectively the key assumptions and limitations of the model.

7.8

Although a bottom-up approach is an important aspect of the internal model validation, the PRA emphasises that boards should value the role that good validation can play in helping them to understand the key drivers and limitations of a model. The PRA expects that firms would be able to provide evidence that the board has:

• challenged the validation process and its results;
• understood and satisfied itself on the key assumptions and limitations of the model; 
• considered the possible quantification of these limitations; and
• taken appropriate mitigating actions.

7.9

The PRA expects boards to be tracking progress actively in addressing key issues identified by validation work.

7.10

The PRA does not expect boards or senior management to be able to discharge their duty in isolation. Boards should demand support from executive management to ensure that key features of models are explained in a way that directors can engage with properly, and they should draw on a wide range of sources, not limited to model developers, to ensure they are satisfied with the model.

7.11

To verify the robustness of the internal model, the internal model requirements and calibration standards are designed to ensure that a model is well grounded in its technical content, with good sources of underlying data. The internal model requirements also require that the model and its limitations are properly understood by its users and by senior management at firms, including the board.

7.12

In summary, a comprehensive validation process should put specific attention on those key assumptions and expert judgments that have a material impact on the model, and should also articulate how the sensitivity to the key assumptions and expert judgement are being assessed and taken into account in the decision process.

7.13

The PRA expects that validation of the internal model clearly evidences the review and challenge that has taken place in assisting the board to meet its objectives.

8.1

The PRA’s quantitative framework for model assessment, in the context of the overall model review process, is structured around the internal model requirements and calibration standards. The PRA’s assessments are framed around these requirements so that the PRA is able to satisfy itself that the model meets the internal model requirements and calibration standards. In its assessments, the PRA applies a series of qualitative and quantitative tools to help identify areas of review focus. One of those tools is the quantitative framework for model reviews, which includes the use of specific quantitative indicators (‘QIs’) where risks are sufficiently homogeneous.

8.2

The PRA’s decision-making process will use a risk-based approach to determine areas of focus on a firm-by-firm basis. In this assessment process the PRA uses a series of indicators to determine the focus of review: these are both qualitative (eg a view on the embedding of the model from previous supervisory engagement) and quantitative.

8.3

The requirements which relate to calibration standards are set out in Solvency Capital Requirement – General Provisions 3.3 to 3.4. An assessment of these requirements is geared towards ensuring that the SCR produced by the model corresponds to the value at risk (VaR) of the firm’s basic own funds at the 99.5% confidence level over one year.¹³

8.4

The PRA uses its quantitative framework as:

1. (a) a diagnostic tool to help assess model rigour and capital adequacy and hence highlight areas of potential concern;
2. (b) a prioritisation tool, to help inform where review teams should direct their attention, eg by identifying risks or correlations which may be under-calibrated; and
3. (c) one contributor to decision making at internal model permission application stage as to whether a firm has met the internal model requirements and calibration standards, and therefore whether permission to use the internal model for the purpose of calculating the SCR should be granted, including whether the PRA should impose any safeguards.

8.5

Internal models are required to be calibrated to the standard specified in Solvency Capital Requirement – General Provisions 3.4. Where risks are homogeneous, a PRA quantitative assessment of the calibration of individual risks and their dependency structures can give an efficient diagnosis of whether there are areas of potential concern where the model has not been calibrated adequately to meet the internal model requirements and calibration standards. Where the risks are largely (but not totally) homogeneous, the PRA will tailor its quantitative assessments to reflect a firm’s specific risk profile.

8.6

Quantitative tools are also important in helping the PRA to prioritise areas for early review where firms may not have calibrated their risks or correlations adequately. However, they are not determinative of the PRA’s final view of a model or model component. It is also worth noting that the PRA looks at the calibration of any model as a whole, as well as in its constituent parts, with particular consideration being given to whether the model remains appropriate in a range of conditions and over time when the balance of risks may change.

8.7

Finally, the outputs of this quantitative analysis constitute one of the many indicators that are taken into account by the PRA in concluding whether the model meets the internal model requirements and calibration standards. Specifically, while the PRA’s quantitative analysis assists in verifying that the calibration standards have been met it does not negate the need for other aspects of the model to be reviewed, including SQS in Solvency Capital Requirement – Internal Models 11.

8.8

As set out above, the operation of the quantitative framework does not yield a mechanistic ‘pass/ fail’ decision. It is worth highlighting once more that the PRA’s decision on whether or not to grant permission to use an internal model does not hinge on meeting any particular quantitative criterion. It hinges on the internal model itself, in conjunction with any safeguards to mitigate residual non-compliance with the internal model requirements and ensure compliance with calibration standards. It must also be emphasised that the use of the tools underlying the PRA’s quantitative framework is always tailored having regard to a firm’s own risk profile.

9.1

Firms with permission to use an internal model are required to have an internal model change policy, which is covered by the firm’s internal model permission.¹⁴ Following a grant of permission, the internal model change policy is expected to play a central role in the wider governance of a firm’s internal model. For example, it should help ensure that the internal model continues to reflect the risks to which a firm is exposed and meet the requirements of the PRA Rulebook.

9.2

It is important that the internal model change policy is of a good standard. Firms should consider all the relevant PRA Rulebook requirements and the EIOPA Guidelines when developing and maintaining their internal model change policy. Firms are required to report a quarterly summary of minor and major model changes to the PRA using the QMC.01 template and supporting narrative documentation, in accordance with Article 6(3) of Chapter 2A of the Reporting Part in the PRA Rulebook.

9.3

This chapter outlines some further expectations for firms on setting out their internal model change policy.

9.4

When defining the scope of the policy, firms are reminded to consider Guideline 6 (scope of the policy for model changes) of the EIOPA Guidelines on the use of internal models. It is important for firms to consider whether the scope of the model change policy and model change definitions are sufficiently broad and appropriately flexible to be able to capture any changes that could have a material impact on the SCR or to enable the firm to meet the internal model requirements and calibration standards. For example, the policy recognises that a particular change to a technical provision model may be within scope if that change leads to an impact on the internal model SCR.

9.5

There may also be situations where firms consider it appropriate to exclude something from the scope of the model change policy. In these circumstances, it is good practice for firms to justify these exclusions clearly.

9.6

Firms should also be mindful of monitoring circumstances that might necessitate the need to change the scope of the policy.

9.7

It is important for firms to recognise that the need for model changes may arise from a wide range of potential sources. For example, model changes may be instigated through a firm’s model development plans, validation activities, the own risk and solvency assessment (ORSA) or evolving use of the model. In addition, changes in a firm’s own risk profile and factors external to the firm, such as the economic or commercial environment, may be potential triggers of model changes. A good model change policy would establish a robust process to identify, collate and manage all sources of potential model changes.

9.8

The EIOPA Guidelines on the use of internal models¹⁵ expects firms to develop and use a number of key quantitative and qualitative indicators for major changes.

9.9

In terms of quantitative indicators, the majority of firms define major changes based on a percentage change in the total SCR. An improved approach, adopted by some firms, specifies additional indicators at a more granular level, for example, indicators that relate to changes in the strength of the marginal risk distribution at certain percentiles or the amount of pre-diversified capital requirements for that risk.

9.10

It is important for the model change policy to include qualitative indicators for major changes. An example of a qualitative indicator is where a major change is triggered after a fundamental change in the methodology or a key expert judgement relating to a particular risk regardless of the impact that the change has on the SCR. Another potential qualitative major change indicator is if a proposed model change needs to be signed off at, or above, a certain level of seniority within the firm. Firms may also wish to consider what indicators might be appropriate to use to determine whether a major change might be triggered through ongoing model validation.

9.11

When developing major change indicators, the PRA encourages firms to consider the appropriateness of having different indicators or threshold levels for different risks or components of the model. For example, it may be desirable to include specific change thresholds for certain elements of the model that are of key interest because they are highly material, highly judgemental or have known limitations.

9.12

Finally, it is important that firms justify their choice of major change indicators including why any thresholds chosen are at an appropriate level for the ongoing supervision of the model. In this regard, it can be helpful if firms provide examples of model changes (eg past model changes) that meet their major change indicators in order to demonstrate the appropriateness of thresholds chosen.

9.13

Firms may struggle to articulate how they would define the circumstances in which a combination of minor model changes would constitute a major model change. Better model change policies have specified at least the following:

• how the impact of minor changes will be accumulated together;
• the time period over which these changes will be accumulated; and
• the indicators or thresholds used to determine when such an accumulation becomes a major change.

9.14

A reasonable starting point for each of these may be to:

• accumulate the absolute values of the impact of the minor changes together, unless it could be demonstrated why it would be reasonable to allow the impact of two minor changes to offset each other;
• accumulate changes from the date of the latest minor model change accumulation reset being, either at the end of an annual cycle (that firms may specify) or at the point of receiving a major change application (if permission is granted); and
• use indicators similar to those defined for single major changes, where considered appropriate.

9.15

A further consideration firms may wish to make is whether it is informative to group minor model changes together by risk or another common feature of the model.

9.16

The better model change policies clearly articulate the governance framework covering the internal process for identifying, approving and implementing the model changes. These included an articulation of how the model change policy fits in within the wider model governance, risk management and validation processes.

9.17

The PRA generally expects firms’ executive management to be responsible for the internal sign-off of major model changes and at least to be made aware of minor changes where appropriate.

9.18

It is important that firms also ensure that there is a robust governance process to agree whether changes should be classified as either major or minor, especially in cases where the classification is borderline or subject to judgement.

9.19

In accordance with Article 6(3) of Chapter 2A of the Reporting Part of the PRA Rulebook and the EIOPA Guidelines on the use of internal models, firms with permission to use a partial or full internal model are required to provide a quarterly summary of minor and major model changes to the PRA.

9.19A

[Deleted]

9.19B

[Deleted]

9.19C

[Deleted]

9.19D

[Deleted]

9.19E

[Deleted]

9.19F

[Deleted]

9.20

The PRA expects firms with permission to use an internal model to submit their quarterly minor and major model change information via the Bank of England Electronic Data Submission (BEEDs) portal.

9.21

Firms are required to submit the minor and major model change information quarterly, in  line with Reporting 2.5B in the PRA Rulebook. Where possible the PRA encourages earlier submission to allow early engagement.

9.22

The PRA has provided a template, named QMC.01, for firms to complete and submit,  containing key information regarding minor and major model changes, as set out in Article 6(3) of Chapter 2A of the Reporting Part of the PRA Rulebook. Firms are required to submit the relevant information to the PRA via the QMC.01 template.

9.23

Firms are also required to submit supporting narrative documentation to the PRA, in accordance with Article 6(3) of Chapter 2A of the Reporting Part of the PRA Rulebook, alongside the quantitative information in the QMC.01 template.

9.24

In submitting the model change information to the PRA, it may be helpful to group related changes together, for example by risk area or function of the model.

9.25

In the supporting narrative documentation, firms are required to provide a description of each model change submitted in the QMC.01 template. Firms are expected to provide sufficient detail in the description to ensure a clear explanation of each model change, which may vary depending on the complexity of the change. The explanation for each change may be both qualitative and quantitative and could include, but is not limited to, the following:

• the reference number, cross-referenced with the respective model change in the QMC.01 template;
• the reason for the model change; and
• the change made to the model.

9.26

In the supporting narrative documentation, firms are also encouraged to provide a brief summary of the model changes made since the last minor model change accumulation reset date. This is expected to include the cumulative figure of minor model changes made since the last accumulation reset date, and how this compares to the threshold that firms have defined in which a combination of minor model changes would constitute a major model change.

9.27

The PRA encourages firms to review the effectiveness of the model change policy on a regular basis to ensure that the internal model continues to reflect the firm’s risk profile and meets the internal model requirements and calibration standards. Firms are also reminded that changes to the model change policy itself, other than administrative changes, are subject to the PRA’s permission. Readers also are referred to SS12/16, ‘Changes to internal models used by UK insurance firms’.¹⁶

10.1

As part of the Internal Model Ongoing Review (IMOR) framework, firms with permission to use a partial or full internal model are required to carry out an analysis of change (AoC) exercise comparing the change in their SCR as at their most recent financial year end and their SCR as at their previous financial year end. The AoC requirements and expectations described in this SS also apply to groups using partial or full internal models in relation to their group SCR calculation. Firms should use the AoC exercises to identify causes of movements in their SCRs. Firms are required to submit the results to the PRA using the AoC.01 template. This requirement is in accordance with Solvency Capital Requirement – Internal Models 13A, and Articles 19 and 35 of Chapter 2A of the Reporting Part of the PRA Rulebook. Firms are required to submit supporting narrative documentation that provides reasons, and documentary evidence to support those reasons, explaining any change in SCR in accordance with Solvency Capital Requirement – Internal Models 13A.

10.2

The PRA may expect a firm seeking a new internal model permission, or variation to an existing permission, to carry out and submit to the PRA the results of an AoC exercise. For example, where there are material changes expected in a firm’s SCR between its most recent financial year-end and the application date to vary a permission, or complex changes are proposed to the internal model, the PRA may expect a firm to carry out, and submit the results of, an AoC exercise.

10.3

The PRA expects firms to submit the results of their AoC exercise and supporting narrative documentation via the Bank of England Electronic Data Submission (BEEDs) portal.

10.4

Commencing year-end 2025, firms are required to submit the results of their annual AoC exercise and supporting narrative documentation to the PRA annually, in line with Reporting 2.5B in the PRA Rulebook for groups and solos, as applicable. Where possible, the PRA encourages earlier submission to allow early engagement. When firms seek new internal model permissions or variations to existing permissions, firms may be expected to submit the results of an AoC exercise and supporting narrative documentation to the PRA and are expected to do so on the permission application date.

10.5

The PRA has provided an analysis of change template, named AoC.01, for firms to complete with key information regarding the movement in SCR and submit in accordance with Articles 19 and 35 of Chapter 2A of the Reporting Part of the PRA Rulebook.

10.6

Firms are also required to submit the supporting narrative documentation, in accordance with Solvency Capital Requirement – Internal Models 13A and Articles 19 and 35 of Chapter 2A of the Reporting Part of the PRA Rulebook, in a format they choose.

10.7

Firms are expected to submit a granular analysis of the material reasons contributing to the SCR movement over the period. Examples of such reasons include, but are not limited to, investment changes and risk profile changes.

10.8

In considering whether a reason is material, firms may use a number of criteria with a mixture of both qualitative and quantitative considerations. This definition is expected to align with practice used in firms’ internal AoC exercises in their SCR, where such exercises are completed. Examples of material reasons include whether the reason impacts the SCR by a specified percentage of the SCR, and whether the reason would lead to the firm considering implementing a management action.

10.9

Where the movement in SCR for firms using partial internal models includes material movements in the SCR of standard formula risk modules, firms are expected to provide this breakdown in the AoC exercise.

10.10

The AoC.01 is largely free-form, therefore firms are able to choose the order and categorisation with which the AoC exercise is completed and results submitted. The PRA expects that the order and categorisation of SCR movements in the AoC exercise should align with firms’ internal AoC in SCR, where such exercises are completed. In particular, the PRA expects the analysis submitted to be sufficiently granular to enable stakeholders, such as the board, to understand the key reasons, which may include changes in risk profile, of SCR movements over the relevant period.

10.11

Where firms consider it useful information, firms are encouraged to carry out, and provide the results of, an AoC in SCR movement split by a major ring-fenced fund, as appropriate.

10.12

Firms are required to provide reasons, and documentary evidence to support those reasons, explaining any change in SCR, in supporting narrative documentation. The supporting narrative documentation should include an explanation of each of the causes of SCR movements provided in the AoC.01 template, including the balancing item. Firms are expected to provide both quantitative and qualitative information in this supporting narrative documentation. In addition, in the supporting narrative documentation, firms are expected to detail:

• the governance process followed in completion of the AoC exercise. This may include, but is not limited to, any limitations in the results, the process the firm has taken to validate the results and the individual or committee by which the exercise was signed off;
• any material actions taken internally following completion of the AoC exercise;
• the reasons for any movement in diversification benefit over the period covered by the AoC exercise;
• the definition of material used in the AoC exercise, in line with 10.7 and 10.8; and
• a list of any MLAs and CAOs. This should be an up-to-date list of MLAs and CAOs which contributed to the calculation of the most recently reported SCR, for example, the most recent financial year-end.

10.13

Firms are encouraged to provide additional information, to that listed in 10.12, in the supporting narrative documentation, where helpful.