Effect of cooccurance weighting to English word embeddings

In this study, the effect of the weighting functions, which take place in the well known word embedding algorithms in the literature and which weight the co-occurrence statistics of the words, is examined. In the literature, it is assumed that the semantic relation between two words decreases inversely proportional as the distance between words increases. However, this assumption is not always acceptable. A new parametrically defined weighting function is proposed. In order to enable us to make systematic optimizations on the proposed parametric weighting function we first work on a small English corpus and optimize the parameters. These parameters are then tested on a larger English corpus. As a result of the experiments, it is shown that the new weighting method we proposed presents a higher performance for analogy test than the weighting method in the literature that uses inverse proportionality.