Quantization Error
Mathematical rounding or chopping of digital filter multipliers reduces the word length of internal constants to fit fixed-point processor constraints. The occurrence of coefficient truncation changes the ideal transfer function into a realized version with shifted poles and zeros. It represents a fundamental trade-off between computational efficiency and numerical precision.
Stability Impact
Small changes in the values of recursive filter elements can lead to limit cycle oscillations or total system instability. Because coefficient truncation alters the feedback loop gain, it may push a marginally stable system into an unstable state. Fixed-point implementations are particularly sensitive to these shifts during low-level signal processing.
Proper scaling and overflow management are required to maintain reliable operation in these environments.
Frequency Response
The frequency response of a filter deviates from its designed specification when the word length is too short. The impact of coefficient truncation is seen when the stopband rejection decreases or the passband ripple exceeds the allowed tolerance.
Hardware Design
Designers use simulation tools to assess the numerical precision of the filter. They confirm how coefficient truncation affects the signal quality before the code is finalized. Proper scaling of inputs prevents the overflow conditions that often accompany the loss of precision in these environments.
