Comparing the Performance of Weapon Systems in terms of an Application Role

A method is presented for assigning a figure of merit to a weapon system, based on the weapon's suitability for a particular application role. The procedure involves correlating expected scenario parameter distributions with functions depicting the weapon system's dependence on these parameters. The procedure was motivated by the need for a choice between weapons to fulfil a role for which none of them were originally intended. The method is, however, suitable for any comparison where the application scenario plays a major role. Abbreviations AGL CAP CSH HC IR lat. ace. TAS above ground level combat air patrol combat support helicopter helicopter infra-red lateral acceleration true air speed


Introduction
In today's cost-conscious defence environment, it is often desirable to use existing weapons in new ways. Once a new mission requirement has been indentified, it can happen that a number of weapons exist that, while not the best solution, seem to have the potential for use in, or adaptation to, the new scenario.
The performance of most weapon systems is highly dependent on the scenario in which they are used. Certain weapon system attributes (e.g. lock-on ability for a missile system) are dependent on the scenario parameters prevailing at a certain time (e.g. sightline rate at the time of launch). However, performance limitations which might seem prohibitive at first glance may turn out to only affect a small percentage of the envisaged missions.
The operations researcher is thus faced with the problem of evaluating and comparing complex (and currently successful) weapons of known performance, in terms of a new set of criteria. The method presented in this paper can contribute to such an investigation, by providing a quantitative means for evaluating aspects of the weapon system's performance in a specific mission scenario.
The method comprises three processes: 1. a form of mission analysis by which the statistical distribution of possible scenario parameters is estimated; 2. the construction of a cross impact matrix, which shows the relationship between the parameters of the scenario in which the weapon system is used and the weapon system performance; 3. a calculation which correlates the expected scenario parameters with the weapon dependency on those parameters to assign a figure of merit to the weapon system in that scenario.
Alternatively, in the absence of a pre-defined mission, the matrix clarifies the interdependencies between the weapon and scenario to the extent that one can qualitatively describe scenario envelopes in which the weapon has a high probability of success.
The idea of using a matrix structure to represent a relationship between scenario parameters and weapon system attributes was developed and implemented by Dr C M Erasmus.
In reference [1], his matrix contains expected utilisation functions for certain weapon system attributes given the corresponding scenario parameters. This was used to aid in deciding between various weapon development options, and to avoid overspecification of the new weapon. The approach described in this paper is a variation on Dr Erasmus' technique, and the author would like to express appreciation for his advice and comments.

Mission Analysis
In order to use the cross impact matrix technique, it is necessary to define one or more missions that are regarded as being representative of the missions in whi~h the weapon would be used.
Each mission is then broken down into all the scenarios that could evolve, and those scenarios whose pertinent parameters at weapon launch are the same, are grouped together. The range of values each scenario's parameters could assume at weapon launch, as well as the probability distribution for t he values over that range, are then estimated.
These distributions for the scenario parameters are used as an input for the algorithm (derived from the cross-impact matrix) which evaluates the weapon as described in section 3. If a number of scenarios are evaluated, the mission analysis should produce a relative weighting for the probability of occurrence of each of the scenarios. A single overall figure of merit for the given mission can then be calculated as a weighted sum of the weapon's performance measure in each scenario.

Cross Impact Matrix
Each entry in the cross-impact matrix is a graph relating a weapon attribute to a scenario parameter. For example, at the time that a pilot wishes to fire a missile, certain prevailing scenario parameters such as range and sightline rate can be quantified. Certain of the weapon system's attributes such as lock-on ability are functions of these scenario parameters. If a mission is specified, probability distributions for the scenario parameters can be determined and then, using the matrix, a figure of merit for each weapon attribute can be found. The process is described in more detail below.
As an example, part of a cross impact matrix for a missile is given in table 1. The procedure for constructing the cross impact matrix is as follows: 1. A list is made of engagement scenario parameters that would have an effect on the performance of the weapon system (e.g. range to target).
In this form the matrix is already a useful tool, since the relationships between the scenario and weapon performance have been clarified. It can be used qualitatively to describe missions in which the weapon will be successful.
However, the matrix is most useful where one or more mission scenarios have been specified by a potential user, since the scenarios can be combined with the weapon parameters to determine how effective the weapon would be in those missions. The procedure continues as follows: 4. For a specific mission scenario defined by the user, the probability distribution for each scenario parameter is determined (e.g. a gaussian distribution centered around 3 km shows at which ranges the target is most likely to be).  Table 1: Part of a. cross impact matrix http://orion.journals.ac.za/ 5. A set of random values are chosen, one for each of the scenario parameters, according to the distributions specified for each parameter. This is equivalent to choosing one specific event: a set of values for range, sightline rate etc. For each chosen parameter value, its effect on each weapon system attribute is determined by the relationships in the matrix-for example weapon range is limited by target speed, relative altitude and target aspect angle. However range is itself a scenario parameter and a random number was drawn to determine a range value for this run, so one can see whether the weapon's range abilities under the limitations of this chosen event would be sufficient. Similarly each weapon system attribute can be evaluated against the chosen values, and if all the attributes are successful, a success is registered for the event. This process is repeated for a large number of sets of random numbers so that all probable combinations of the scenario parameters are represented. The percentage of the randomly chosen events in which the weapon was successful is an indication of the extent to which the mission scenario can be handled by the weapon system.
6. Such a correlation is performed for each of the mission scenarios provided by the user. If the user specifies a weighting that indicates the relative likelihood that each of the given scenarios will arise, the weapon success figures for the various scenarios can be combined by means of a weighted sum. This will produce a single figure of merit for the weapon for the whole mission.

Example
The methodology will perhaps be clarified if explained in terms of the operations research study that prompted it. The study was commissioned to address two main issues: 1. The evaluation of the usefulness of certain missiles in a helicopter air-to-air application, and 2. the definition of mission envelopes in which the missiles would be effective as helicopter air-to-air combat weapons. This involves determining bounds for the scenario parameters within which the weapon system will be useful.

Matrix
The following is a list of the mission scenario parameters that were considered to have the greatest effect on the performance of the missiles in question. Sightline angles (azimuth and elevation of the sightline with respect to the CSH's body x-axis) Sightline rates (azimuth and elevation) Target aspect (angles that the sightline makes with the target's body x-axis : indicates whether the CSH sees the target's side, tail etc.) Relative speed Relative height Since some of these parameters are mutually dependent, it was only necessary to specify the independent variables. For instance, once target range, speed, aspect and sightline have been determined, sightline rate can be calculated. Relative speed and height are also calculated from the other givens.
The weapon system attributes that were considered can be summarised under the following broad headings:

Range effectiveness Lock-on effectiveness Ground clearance Gathering effectiveness Manoeuvrability
The relationships between these parameters and the weapon system attributes are described in the detailed report on the study [2]. A few have been represented graphically in table 1.

Mission Analysis
Let us consider a hypothetical scenario for the helicopter air-to-air missile problem. A mission analysis would have to be drawn up in conjunction with the operational users of the system. An example is shown in figures 1 to 3 in appendix 6. The total mission scenario can be summarised by showing the breakdown of just those missions which lead to an air-to-air engagement. An example of this type of breakdown is shown in figure 4.
Note that the method produces figures indicating the relative frequency of occurrence of each scenario.
In this example, air-to-air engagements fall i~to one of eight major categories, so eight mission profiles would typically be defined. For the purposes of illustration, two of these eight profiles are described below. A rectangular probability distribution for a scenario parameter is characterised by a minimum and maximum value in the table, whereas a triangular distribution requires a modal value as well. Scene 1 : 6% of total air-to-air missions.
During a surprise daytime attack over enemy territory the CSH detects an enemy fixed wing aircraft in a combat air patrol (CAP) role over the prime target. The CSH has not been detected Itself, but smce the enemy aircraft IS mterfermg w1th Its pnme m1ssion It attacks the aircraft. Scene 5 : 24% of total air-to-air missions.
During an anti-tank battlefront mission the CSH detects and attacks an enemy helicopter that has not yet detected the CSH.

Simulation Results
Once the matrix for a weapon system has been drawn up, the algorithm described can be easily implemented in any high level computer language. The type of detailed results produced for two such scenarios is shown in appendix 6. These results would, for instance, indicate that although missile 1 is better able to cope with the expected sightline angles and rates, the target speeds and relative altitudes prove to be more limiting than for missile 2. The total weapon figure of merit indicates the percentage of encounters in the given scenario, in which the weapon is deemed to be successful.

Conclusion
A limitation of the current model is that it assumes a knife-edge success/failure according to the graphs in the matrix. In other words, if the weapon range at the given angles, speeds etc is less that the target range, success is assumed, otherwise failure. This is not unreasonable in the case of a missile system, but the method may need adjustment for other weapons.
Nonetheless, the cross impact matrix technique provides an effective means of combining detailed data about a weapon system, with expected mission scenarios, to obtain a figure of merit indicating the weapon's probability of success in the missions for which it is being evaluated. Where the performance is unsatisfactory, the weapon's limiting attributes can be identified. It should be emphasised that the figure of merit is useful for the comparison of different weapon options; its absolute value should be quoted with caution.
One of the advantages of this technique is that it lends visibility to the basic assumptions on which the trade-off is based, so that the effect of changes in assumptions due to new doctrines or opinions can easily be evaluated.
The method also lends itself to sensitivity studies of weapon performance versus scenario parameters which could assist in determining effective deployment tactics.