Accurate Floating-Point Argument Calculation for Sine-Fitting Algorithms

In this paper, accurate argument calculation for sine-fitting algorithms is investigated, assuming floating-point (FP) arithmetic. An easy-to-implement incremental calculation technique is suggested. In order to decrease error propagation, the algorithm is complemented with an advanced summation technique. Theoretical and numerical analyses on computational demand are performed to highlight that incremental argument calculation outperforms the method proposed in former research. Furthermore, an algorithm is implemented to mitigate the effect of imprecise representation of frequency on FP arithmetic. Monte Carlo analyses are carried out to demonstrate the accuracy of the suggested algorithms. Results show that phase information can be evaluated precisely even with single-precision FP arithmetic, applying incremental argument calculation. By this means, the cost of equipment that is needed to perform sine fitting can be reduced significantly. Finally, possible application areas are shown to demonstrate the applicability of the suggested solutions in the state-of-art measurement procedures.

I. Introduction

Sine-Fitting algorithms are widely used in the field of measurement technology. They can be applied to measure the complex value of an impedance [1], or to characterize the quality of the power system [2]. In particular, there are application fields where it is of great importance to have an accurate sine wave estimator. Among these fields, we can find the testing of analog-to-digital converters (ADCs) [3] and of digitizing waveform recorders [4].