 By TIm Edwards, Director, Insurance and Investment Management, PwC
1- Independence: So many people get hung up about the difference between “first line” and “second line” validation. Solvency II does not use the terms “first line” or “second line”, so it is easier to distinguish between the activity of (a) “model design, build, document, test and deliver” and (b) “model validation”, which is a separate activity, broadly making sure that those actions have all been completed appropriately, and that the model is fit for purpose.
Independence therefore requires the validation to be entirely separate from the model build (etc), using people not involved in those processes. PRA feedback suggests that they have concerns over whether some firms’ validation is truly independent, and certainly they expect independence to be explicitly demonstrated.
2- Scope: Model validation has to cover Directive Articles 101, 112(5), 120-123 and 125, and all associated Delegated Acts (Level 2). Some 250 separate requirements in all, detailed in the PRA’s published “self-assessment template”, albeit with some update needed once the Delegated Acts are confirmed. So validation is not just about the calculation kernel itself, but also covers all the associated tests and standards. Many of these are not really technical – we are seeing an increasing role for Internal Audit in validation of qualitative elements.
3- Depth: Even if validation is independent enough, and encompasses all the requirements, there is still an issue of whether it is performed to adequate depth. This requires objective, constructive and extensive testing and challenge of assumptions, judgments and of outputs. It includes designing and running additional tests and stresses, to avoid the risk simply of replicating results, and to unearth potentially unknown weaknesses. One test is to ask yourself is: “what did the validation tell us about our model or risk profile that we didn’t know before?”
4- Quality: Depth does not necessarily equal quality. Considerable expertise and experience is required to be able to demonstrate that all appropriate challenges have been considered, and that the model is fit for purpose. In many cases, this needs experts in market, credit and operational risk in addition to traditional actuarial insurance-based disciplines. Then you can start addressing the problem of justifying the no doubt complex dependency structures that ultimately dictate the model output.
|
