On testing the hypothesis of population stability for credit risk scorecards

  • J du Pisanie Monocle Solutions
  • IJH Visagie School of Mathematical and Statistical Sciences, North-West University


Scorecards are models used in credit risk modelling. These models segments a population into various so-called "risk buckets" based on the risk characteristics of the individual clients. Once a scorecard has been developed, the credit provider typically prefers to keep this model in use for an extended period. As a result, it is important to test whether or not the model still fits the population. To this end, the hypothesis of population stability is tested; this hypothesis specifies that the current proportions of the population in the various risk buckets are the same as was the case at the point in time at which the scorecard was developed. In practice, this assumption is usually tested using a measure known as the population stability index (which corresponds to the asymmetric Kullback-Leibler discrepancy between discrete distributions) together with a well-known rule of thumb.This paper considers the statistical motivation for the use of the population stability index. Numerical examples are provided in order to demonstrate the effect of the rule of thumb as well as other critical values. Although previous numerical studies relating to this statistic are available, the sample sizes are not realistic for the South African credit market.The paper demonstrates that the population stability index has little statistical merit as either a goodness-of-fit statistic to test the hypothesis of population stability or as an intuitive discrepancy measure. As a result, a novel methodology for testing the mentioned hypothesis is proposed. This methodology includes a restatement of the hypothesis to specify a range of "acceptable" deviations from the specified model. An alternative test statistic is also employed as discrepancy measure; this measure has the advantage of having a simple heuristic interpretation in the context of credit risk modelling.
Research Articles