Efficiently Building a Large Scale Dataset for Program Induction

One of the applications of machine learning with the most potential is speeding up expensive computational processes, i.e. learning to think. To do so, one first generates a large-scale dataset by a compute-intensive process and then trains a model to approximate the distribution. High performance computing (HPC) is a perfect fit for these processes, as one may efficiently deploy large amounts of computation to generate a dataset in a reasonable amount of time, to then learn a computationally-efficient solution. Here, we focus on generating a program synthesis dataset. Finding the program that fits a given input-output specification is very expensive, but generating the input-output pairs for a given program is a well-defined process. In this work, we show how we efficiently ran hundreds of thousands of C++ codes line-by-line and used intermediate variable states to generate a large-scale program synthesis dataset.