Hybrid Framework for Generating Structured Data Synthetically while Maintaining Referential Integrity

There have been multiple attempts in academia and industry to generate synthetic data. The requirements for synthetic data have surged with the evolution of AI/ML in the industry. Organizations frequently face roadblocks due to the non-availability of data relating to compliance, etc. This leads to a significant loss of opportunities as insights are not available due to data limitations. Traditionally, manual mocking, masking of production data, Generative Adversarial Network (GAN) driven approaches, etc., have been explored, but they fail to address referential integrity. We present a Synthetic Data Generation framework to generate a relational data set comprising interlinked tables while maintaining referential integrity. The framework is an ensemble of many algorithms, including GAN, HMA (Hierarchical Modeling Algorithm), Explicit distribution extraction, etc. Further, we present an evaluation framework to measure the Veracity, Variety, and Volume of the generated data. The results have been validated on a variety of datasets across industries. We believe the framework can disrupt and help multiple organizations that are not able to leverage data due to any constraints.