Developing a planning tool for South African prosecution resources : Challenges and approach

In every country the prosecution of criminal cases is governed by different laws, policies and processes. In South Africa, the National Prosecuting Authority (NPA) has the responsibility of planning and managing all prosecution functions. The NPA has certain unique characteristics that make it different from other similar organisations internationally. The development of a planning tool that the NPA could use to plan their future resource requirements over the short to medium term required extensive modelling, and its final form included features which, to the best knowledge of the development team, make it unique both locally and internationally. Model design was largely influenced by the challenges emanating from the special requirements and context of the problem. Resources were not forecasted directly, but were derived with the help of simulation models that traced docket flows through various resource-driven processes. Docket flows were derived as a proportion of reported crimes, and these were forecasted using a multivariate statistical model which could take into account explanatory variables as well as the correlations between the patterns observed within different crime categories. The simulation consisted of a number of smaller models which could be run independently, and not of one overarching model. This approach was found to make the best use of available data, and compensated for the fact that certain parameters, linking different courts and court types, were not available. In addition, it simplified scenario testing and sensitivity analysis. The various components of the planning tool, including inputs and outputs of the simulation models and the linkages between the forecasts and the simulation models, were implemented in a set of spreadsheets. By using spreadsheets as a common user interface, the planning tool could be used by prosecutors and managers who may not have extensive mathematical or modelling experience.


Introduction and problem statement
In South Africa, the National Prosecuting Authority (NPA) has the responsibility for managing the functions related to the prosecution of criminal cases and this responsibility is governed by specific laws, policies and processes.Prosecuting functions include assessing whether a case should and can be prosecuted, what charges to prosecute and in which court forum.The court forum may be the district court (DC), the regional court (RC), or the high court (HC).The NPA has to provide and oversee prosecutors and state advocates at the more than 700 court sites scattered across the country.
The Council for Scientific and Industrial Research (CSIR) was appointed, through a public tender process, by the NPA to develop a strategic planning tool for forecasting future prosecutor requirements.The model development team were required to understand the role of the NPA and its operation within the South African legal framework in order to develop a customised tool that would serve its needs.Significantly, the brief emphasised the forecasting of workload from which forecasts of resources would follow almost automatically.However, it was revealed during the initial preparation work that the tasks carried out by prosecutors are very complex and use many interlinked processes.Therefore, a simple conversion of the workload to resources did not seem possible and the emphasis of the model development was therefore shifted from workload estimation to resource allocation.In addition, all the required data on docket flows between different courts were not available in an appropriate format, and therefore a comprehensive model could not be developed to model activities within each entire planning region.
The features of the planning tool were designed to respond to the challenges emanating from the special requirements of the NPA, the context of the problem situation, and the available data.The final format of this planning tool included features which, to the best knowledge of the development team, make it unique both locally and internationally.Specifically, three unique aspects are highlighted: • the methods used for estimating future workload; • the "disaggregated" simulation modelling approach for processing workload; and • the way in which results were aggregated and scenarios investigated.
In order to understand why these aspects were considered unique, background is provided in the next section on other modelling approaches found internationally.Thereafter, each of the unique aspects is described in its own subsection.This is followed by a discussion regarding the results obtained from the model and how it may be used, and the paper concludes with a number of remarks and a short conclusion.

Background on existing approaches to the modelling of prosecutor-related functions
An extensive literature review revealed that, although considerable importance is currently placed on quantitative modelling related to the Criminal Justice System (CJS) environment internationally, no existing models were found that dealt specifically with the forecasting of prosecutor resources at a national level.Since required prosecutor resources are dependent on the workload received at courts, the literature study was focused on identifying appropriate techniques for forecasting workload, which in this context amounted to forecasting crime, as well as for estimating resources required to deal with a given workload.
The general trend for quantitative short to medium term crime forecasting found in the literature appears to be the use of time series based regression techniques, although a limited number of qualitative and expert-based judgemental approaches such as those by Walker (2002Walker ( , 2003) ) have also been reported.The time series based regression techniques involve the identification of potential drivers of crime to use as explanatory variables for modelling crime rates per crime category (Triggs 1997, Van Tulder and Van der Torre 1999, Van Tulder 2000).Forecasts of the explanatory variables were obtained via ARIMA or other univariate time series models or sourced from available agencies that produce such forecasts (Klepinger andWeis 1985, Triggs 1997).A number of articles point out problems with forecasting crime or court caseload by merely extrapolating the past trends (Moody and Thomas 1987;Krislov 1995).Although using a multivariate study that attempts to predict the future number of reported crimes from various influencing factors or "drivers" would mostly be preferred over extrapolation methods, the use of such methods is also not without difficulties (Arnett and Magnum 1995;Walker 2002).However, in cases where specific judgemental considerations have been identified and incorporated into the statistical forecasts, more accurate results have been reported (Goldman et al. 1976;Triggs 1997).
Typically the planning of staff resources within a service environment is complicated by the variability in demand for such services, as indicated by Abernathy et al. (1973).One approach to the optimisation of staff requirements within a context of fluctuating demand for services is to use stochastic programming as described by Joubert and Conradie (2005).
Several examples appear in the literature with regard to the application of simulation to resource planning, including the planning of police force resource needs (Freeman, 1992).Simulation models have also been developed and applied quite frequently for other aspects of the CJS in various countries and a comprehensive overview of such models is given in Stewart et al. (2004).Many of these models, however, have been for the purpose of forecasting prisoner populations, including those described by Van Tulder (2000), Lind et al. (2001) and Wang (2006).Two simulation models which were developed for strategic planning of the whole CJS, and which included the modelling of resources to carry out various activities in the court environment, were the recently built discrete-event simulation model in the USA that is summarised by Cangur et al. (2005), and the agent-based simulation model for England and Wales (Maresh 2005).However, not all CJS simulation models have been adopted successfully and the problems encountered as well as general pitfalls have been pointed out by several authors (Chaiken et al. 1975;Bohigian 1977;Jenkins 1995;Lind et al. 2001;Blumstein 2002).Two main warnings that have repeatedly been given in the literature are to avoid constructing simulation models that require large amounts of data that are problematic to collect, and to avoid incorporating too much complexity which may result in a model that is too difficult to maintain.The latter point is of particular concern within criminal justice agencies since the end-users tend to be people trained in law rather than as data analysts (Lind et al. 2001).
After consideration of the relative advantages and disadvantages of the various methods described in the literature, it was felt that a resource model based on simulation would be the most suitable in the NPA environment, with inputs provided from crime forecasts.These crime forecasts would be created by means of a regression-based approach using suitable drivers, rather than mere extrapolation techniques, but allowing for judgmental adjustments to the statistical forecasts.

Unique features of the model
The modelling approach that was selected can be represented by the graphical representation provided in Figure 1.Reported crimes were forecasted and these forecasts were used to estimate court workload.Resources were then derived from the estimated workload and the processes to be performed on the workload, using a simulation approach which incorporated the associated time and human resources required during such processing.In addition, the models allowed the user to change various variables and model parameters in order to do comprehensive scenario testing and sensitivity analysis.The approach of modelling the workload and resource needs separately, yet linking these together in order to determine the resource needs associated with a specific workload scenario required certain unique modelling features.These unique features are described in more detail in the following section.

Forecasting workload by estimating future docket flows
Forecasting the potential future workload of the NPA consisted of three parts.The first and second parts consisted of collecting appropriate data and deciding on a modelling approach for forecasting reported crimes, and these were carried out interactively.Once forecasts were obtained for crimes, then the third step was to translate the forecasted crime into forecasted "dockets to court."The term "dockets to court," simply called "dockets" in this document, refers to the reported crimes successfully investigated and referred to court by the South African Police Service (SAPS).
SAPS releases a summary of the number of crimes reported annually within a specific set of crime categories, and these data were used as the basis for the forecasting of reported crimes.The crime categories used by SAPS when releasing the crime data (Murder, Attempted murder, Assault with the intent to inflict grievous bodily harm, etc.), were also used in the forecasts.
A review of relevant South African crime studies revealed that most were of a qualitative nature, and the only quantitative studies located were the econometric studies of Brown (2001) and Blackmore (2003).The study by Brown (2001) seemed to be the first to attempt to quantify (correlate) the relationship between certain "drivers" of crime and crime rates.This study expanded on the traditional economic model of crime, namely the approach that persons would become involved in criminal activities if they could see financial benefits for doing so, by incorporating socio-economic measures as well as purely economic measures as "drivers" of crime.However, the study did not go as far as to develop a model linking crimes to the identified "drivers."Blackmore (2003) developed a model to predict crimes from "drivers" by using Seemingly Unrelated Regression (SUR).
Although some of the ideas from Blackmore's study were used for the NPA project, the study could not be duplicated since it was not possible to obtain updated data for all the variables included in that model (the study was executed on data up to 2001).A new model, containing a different set of variables as "drivers," had to be developed.
In order to develop the required forecasting model three possible approaches were considered, namely: • to forecast the number of crimes within each category separately per crime category and province using time series methods (different methods suited to each dataset); • to forecast the number of crimes separately per crime category using different "drivers," and individual multiple regression models for each category; and • to forecast the number of crimes based on the identified independent variables simultaneously within all crime categories using the Partial Least Squares regression (PLS) method.
PLS was first developed by Herman Wold, for applications in the field of chemometrics (see Wold, 1966 andWold, 1985) and is widely used within this field.Even though many statisticians propose using Ridge Regression rather than PLS (see Frank and Friedman 1993; Basak et al. 2003), and PLS is not such a well-known technique in the statistical field, this method has various advantages.The PLS method models the relationship between a group of dependent (y) variables and a group of independent (x) variables.While canonical correlation provides a method for producing a linear combination between a group of dependent variables as well as a linear combination between a group of independent variables in such a way that correlation between the two linear combinations is maximised, PLS goes one step further by providing the linear combinations as well as the "maximised" relationship between them in one model.The PLS method was used to compile a model for the different reported crime categories (the dependent or y variables) in terms of a set of economic and demographic variables (used as independent, i.e. explanatory or x variables).It was felt that such a modelling approach was suitable since the crime categories may not necessarily be assumed to be independent, and that including the underlying correlations between the patterns observed within different crime categories in the model may provide a better model than modelling the crime categories separately from each other.A second advantage is that PLS does not require any distributional assumptions on the data.Thirdly, the PLS method works well in cases where there are fewer data points than number of variables, when ordinary least squares regression cannot be used, and this was seen as an advantage due to the limited amount of available reported crime data.
The SAS statistical software (SAS, 2012) was used to develop the forecasting models.A historical data series of seven years could be obtained (reported crimes per province, per crime category and per year), and forecasts were produced for another seven years.
Due to a lack of data it was not possible to take a "hold-out" dataset to use for a quantitative measure for which of the three approaches produced the "best" forecasts.Instead, the first full set of forecasts for all provinces and crime categories for a year was compared to the actual number of reported crimes released by SAPS for that year, since these were released just after the forecasts had been completed.The forecasts obtained from the various approaches for that year were then compared using Mean Absolute Percentage Error (MAPE).From Table 1 it may be seen that the time series models approach delivered the lowest MAPE, and that the PLS gave a lower MAPE than the set of individual regression models.Even though the time series approach delivered a lower MAPE, within the context of the problem situation, models using explanatory variables to forecast crime were considered to be more suitable than time series models which only use extrapolation of past trends.Out of the two approaches which used explanatory variables, the PLS had the lowest MAPE value.Therefore, the forecasts obtained from the PLS approach were recommended for use in the NPA planning tool.
The final recommended set of forecasts for the number of reported crimes within each SAPS crime category for each province and each year within the forecast horizon of seven years was therefore obtained from the PLS approach.The forecasts used Gross Domestic Product (GDP) as well as age and gender population breakdowns as "drivers."Crime forecasts were broken down per province, and provincial indicators were incorporated into the forecasting model, since crime patterns were found to differ substantially between provinces.Forecasts for the explanatory variables were obtained and included in the PLS model to provide crime forecasts, and provision was made for judgemental updates to these forecasts, based on expert knowledge from within the CJS.The reported crime numbers were then adjusted to allow for investigation by SAPS, and final docket forecasts for each court were then obtained, as illustrated in Figure 2.

Simulating resource needs based on processing docket flows
The conceptual model of how to convert workload forecasts into resource forecasts was determined largely by the complexity of the court system and the available data.
Although various mathematical modelling approaches were considered, based on the review of literature, the knowledge gained of the court system and the need for the NPA to be able to understand and maintain the model, it was decided that simulation modelling would be the preferred approach.The initial approach was to use one overarching prosecution simulation model which would be run for each division (NPA planning region), linking the district, regional and high courts in the division and which would directly use the workload forecasts as inputs.Figure 3 illustrates this initial approach, showing the attempt to model the "real" flow of dockets.Dockets are first referred to district or regional courts, where some of the dockets will become cases that go to trial in district, regional or high court, and some of the cases may be appealed in the high courts.Such a model would allow one to model all flows in one high court planning region (division), and performance measures could then be assessed for each scenario of prosecutor resources in order to establish the "best" resource mix for the division.However, it became apparent during the data collection phase that the required information with regard to linkages between courts, i.e. the number of dockets or cases flowing between various court sites, was not available in an appropriate format.In addition, different court "types" could not be differentiated solely on the number of courtrooms and workload, since different combinations of district and regional court "types" were possible.Therefore, this initial conceptual design was reworked into a disaggregated model in which the focus shifted to the individual court sites in the division and the division was not modelled as a whole.This approach was considered to make the best use of the available data.
This disaggregated approach consisted of a number of smaller simulation models, representing court "types," which could be run independently and which differed with respect to certain functional procedures, types of prosecutors available, allocation of tasks to different types of prosecutors, size, as well as courtroom availability.Figure 4 indicates the main court type models used to simulate the lower courts using the ARENA discrete-event simulation software (Rockwell Automation, 2012).Each of the models was used to simulate the prosecution processes at that particular type of court over time by including the tasks carried out by prosecutors in the court environment, as well as the delays (per case) that are typically experienced in courts.Not all resources carry out the same tasks and therefore the inclusion of different categories and levels of prosecutors was necessary.The prosecutors' tasks related both to docket (or case) specific tasks that are undertaken from the time a docket is received until the completion of a case, as well as tasks that are not linked to specific cases, such as managerial tasks, meetings, public queries, and so on.All tasks and delays included in each court type model were assigned a stochastic distribution that would allow for variation in the model and these distributions were calibrated from data collected at specific court sites.Triangular distributions could be used to model most of the tasks, counts and delays, but lognormal distributions had to be used for very long-tailed distributions.Decision points in the models, which direct the route that a docket or case may take, were calibrated in a similar way but included other information collected at the NPA head office.In addition to the variation provided by the time distributions, the time taken on some tasks could also be adjusted according to the seniority level of the resource type selected for that task.
Since the simulation models were ultimately required for high-level resource planning purposes and not for operational purposes, it was assumed that all courts of a similar type would function in a similar way.Consequently, no allowances were made for differences in performance levels or for special circumstances at specific individual courts.The resources required at two courts of a similar type were differentiated based on the number of courtrooms and the size of the workload, as illustrated in Figure 5.The workload was defined in terms of the number of dockets and cases that are dealt with by a court during a year.Each of the court type simulation models requires the estimated annual workload and the number of courtrooms as input variables.During a simulation run, each model converts these annual workload inputs to daily docket inflow, according to typical monthly and weekday patterns.The number of resources (resource set) used in the court may be adjusted by the user, within certain guidelines per court type, in order to test combinations of resource numbers and resource types.While a simulation model runs, it also records specific performance measures, such as case cycle times, outstanding roll and hours worked by various prosecutor types over the simulated period.The user can then evaluate the performance of each particular resource set by comparing the performance measures to their respective targets.An associated cost per resource set is also provided so that the user can evaluate the balance between performance and cost, i.e. more senior prosecutors have a higher throughput but also cost more.Although this simulation process does not provide a mathematically optimal solution, it does allow the user to select the most practical "best case" scenario for a particular court structure and workload size.
In addition, the simulation models make use of Excel (Microsoft, 2010) interfaces, for both inputs and outputs, which allows the user to easily change input values, run the model and assess the summarised performance outputs.These interfaces remove, to a large extent, the need for the client (NPA staff) to interact directly with the ARENA simulation software.
Since each different court operates as an independent unit, a user could determine the required resource set for a group of courts within a particular planning region (such as an NPA Division or NPA cluster) by aggregating the estimated resource sets of all the courts in the region.A simplified example of how resource needs at individual courts would be aggregated for a particular planning region is provided in Figure 6, ignoring resource types for illustrative purposes.This aggregation is discussed in more detail in the next section.

Final product obtained from aggregating results
The overall objective was to provide a resource planning tool to determine prosecution resources nationally or for an appropriate planning region, given a certain workload forecast.However, the client (NPA staff) had to be able to determine resources for a single court or groups of courts without necessarily having to execute many simulation runs, if at all, or having to understand all the model details.

Results obtained from model
To summarise, a set of input values for workload forecasts and court structure may be compiled and corresponding resource requirement outputs can be "looked up," as shown in Figure 8.The simulation models may, however, still be used individually for scenario testing purposes by adjusting not only the workload and court structure inputs but also other time or case flow related inputs.Since the final product was intended for prosecutors and managers, the final planning tool was implemented in a set of spreadsheets as a common, and familiar, user interface.The final planning tool provides users with a rolling four-year forecast of prosecutor resource requirements for each NPA division and cluster across the country.It also allows resource requirements to be updated as workload forecasts change.In addition, users can interact with the simulation models in order to do scenario planning and sensitivity analysis at an individual court level.

Remarks and conclusions
In terms of verification, the overall model was checked in detail for internal consistency and logical operation, and the process flows within the simulation models were verified by experts within the NPA.Although the resource forecasts were based on an "ideal" wellfunctioning court and did not replicate the details of any particular existing court, final results were checked for similarities with the staffing profiles at the courts visited by the team.Despite the fact that a complete validation of all the forecasts and model elements was not possible, sufficient verification of processes and appropriate validation of outputs enabled the team to assess whether the final product provides a useful and rational basis for resource planning within the NPA.
Throughout any such model development project, client interaction is vital.Since it was not possible to run the complete set of models in a setup which could produce meaningful goodness of fit statistics for assessment purposes, it was imperative for the NPA to be confident in the logic and parameters which have been used.To this end, regular feedback was given to key NPA experts, and detailed written reports, which could be circulated within the organisation, were compiled in order to describe the different components of the final planning tool.
We are confident that this final planning tool, which provides a direct link between estimated future workload and required resources, will be a useful enhancement to the resource planning of the NPA.In addition, the flexibility provided by the workload forecasts and simulation models will enable the NPA to do a more comprehensive testing of future scenarios and of the sensitivity of resource requirements to changes in the criminal justice environment.

Forecast
number of reported crimes (per crime category, province and year) Estimate court workload resulting from crime forecasts (per court forum, NPA main planning region and year) process estimated Apply prosecution processes and resource workload (per type of court and year) Determine number of prosecution resources required (per court and per year)

Figure 1 :
Figure 1: A schematic representation of the selected overall modelling approach.

Figure 2 :
Figure 2: A schematic representation of the process of modelling NPA workload (dockets received at court).

Figure 3 :
Figure 3: Initial high level conceptual model which had to be revised.

Figure 4 :
Figure 4: The set of court type simulation models used in the final model for lower courts.

Figure 5 :
Figure 5: An illustration of the differences within one court type.

Figure 7 :
Figure 7: Different linkage paths used to determine resources.

Table 1 :
Armstrong et al. (2001)recommends the use of MAPE as the most appropriate measure for assessment of forecasts, and although one would prefer to use MAPE on a longer set of forecasts, this was practically not possible.The results from the MAPE comparison are summarised in Table1.Comparative values of MAPE for the three forecasting approaches.

Table 2 :
An illustrative example of summarised resource requirements per NPA division and cluster.