Using Databases to Implement Algorithms: Estimation of Allan Variance Using B+ -tree Data Structure

This work develops and explains a method of using a database's organizational structure to implement data manipulations (grouping, addition, averaging) that enable the database to exhibit intentional algorithmic behavior. Specif-ically, the Allan VARiance (AVAR) estimation using B
⁺
-tree data structures is developed. AVAR is a crucial algorithm to determine the computational limits on data accuracy imposed by real-world noise sources. Unfortunately, AVAR is computationally challenging and is typically applied to large datasets (10
⁶
or larger), which can be difficult to manage without a database. In the previous work, the authors proposed Fast Allan VARiance (FAVAR) algorithms inspired by the FFT to improve computation speed by up to four orders of magnitude. These FAVAR algorithms apply to data sets that are manageable locally on a computer (MB to GB in memory) but can be difficult to deploy on “big data”, e.g., data sets that cannot fit in main memory. Notably, B
⁺
-trees index one-dimensional data similar to FAVAR, i.e., using data aggregations that scale in size as a function of tree order. This paper utilizes the similarity in B
⁺
-trees and FAVAR to use the database's operational process to automatically deploy an algorithm to estimate AVAR using a B
⁺
-trees for data corrupted by common noise types - white noise and random walk. Comparing AVAR estimates to algorithm-targeted B
⁺
-trees e calculations, the results match within 95% confidence bounds.