Statistical Deviation
Statistical analysis measures the difference between the actual output of a sensor and the value predicted by its mathematical calibration model. This residual fit error represents the inaccuracy that remains after all known compensations have been applied. It is a direct measure of how well the model describes the physical behavior of the device.
A lower value indicates a more accurate and predictable sensor.
Model Accuracy
Choosing a linear fit for a non-linear sensor will result in a high level of deviation. In the study of residual fit error, engineers look for patterns in the data that suggest the model is missing a physical effect. A random distribution of errors is preferred over a systematic trend.
Signal Noise
High frequency fluctuations in the electrical output can inflate the reported deviation. Because residual fit error includes the effects of noise and repeatability, it provides a realistic estimate of the best possible performance. This metric is used to sort sensors into different accuracy grades.
Performance Metric
Quality control departments use the maximum allowable deviation to accept or reject a batch of sensors. Reducing the residual fit error often requires more complex fitting algorithms or more calibration points. It is the final check on the effectiveness of the manufacturing process.
