Word Embeddings for Model-Driven Engineering

Model-Driven Engineering practitioners have to deal with the construction of modelling evironments by devising meta-models, grammars, editors, etc. One of the goals of the application of Machine Learning to MDE is to use ML algorithms to assist the MDE expert in these tasks. These algorithms cannot directly receive raw models or meta-models as input, but they typically have to be transformed into a numeric representation, i.e., a vector. In this context, a common approach is to use pre-trained Word Embeddings, which define mapping functions that associate words to semantic vectors. However, current word embeddings are trained with general texts and lack the technical words which typically arise in the modelling domain. To tackle this issue, we have collected a corpus of modelling texts from well-known modelling venues, and we have trained two types of word embedding models. The resulting embeddings (named WordE4MDE) are specialised to address ML tasks in the MDE domain. We have performed an extensive evaluation using the Ecore models of the ModelSet dataset and two state-of-the-art word embeddings (GloVe and Word2Vec) as baselines. We show that WordE4MDE outperforms these two baselines in three meta-modelling tasks, namely meta-model classification, meta-model clustering, and meta-model concept recommendation. WordE4MDE embeddings are available at https://github.com/models-lab/worde4mde and can be loaded using standard Python libraries for their use in ML pipelines.