High-efficient Joinable Tables Discovery in Data Lakes: A Grey Relational Model-based Approach

Finding required datasets in a data lake that stores a vast amount of messy and disordered data is a challenge in data management. This paper addresses the problem of joinable tables discovery, which has many applications in scenarios such as augmenting training data, data cleaning, and extracting machine learning features. However, existing methods usually consider only one factor or assign equal weights to multiple factors in joinable tables discovery, resulting in low optimization precision and ignoring joinable tables discovery purpose. We propose GRMA, a joinable tables discovery method based on a grey relational model considering multiple factors. First, we select different factors according to the purpose of joinable tables discovery and calculate their scores for each table in the data lake. Then, we set weights for different factors using Analytic Hierarchy Process. Finally, we substitute the scores and weights into the grey relational analysis method to determine the final score and perform a top-k query. We also design a unique pruning method, screening pruning, for joinable tables discovery based on multi-factor problems and apply it to GRMA. Our experiments show that GRMA outperforms the best current joinable tables discovery methods by 28%, 13%, and 24% in precision, recall, and F1-score, respectively.