A Regular Expression-based DGL for Meaningful Synthetic Data Generation

Synthetic datasets are necessary for performance evaluation and function test in most database applications. In this paper, we propose a regular expression-based data generation language (DGL) for flexible test data generation. We extend the standard regular expressions to include references to external resources, sequential numbers, probability distributions, type/format inference, dictionary sampling. In order to implement the proposed scheme efficiently, result caching and database caching techniques are developed and evaluated by experiments.