691. Confidence Problems in Performance Prediction

Paper

Abstract

Test programs for large systems rarely provide enough performance data points to allow prediction of performance guarantee confidence, utilizing statistical techniques developed for academic uses. The system engineer, however, often has enough data for an engineering/management type confidence guarantee. The systems performance data are developed from subsystem characteristics numbers by using a system simulation. Confidence curves for system numbers can be developed by a Mone Carlo of subsystem data distributions.
There are problems, however, in the practical applicaiton of this technique. These include an inability to construct an ideal sytem model, and problems in obtaining suitable input data for the model. Each of these problems is considered in detail. The conclusion is drawn that relatively small (say 20) data points per subsystem characteristic is adequate for sytem use; and tht theoretical distribution derived from test data should, in general, not be used for system simulation.