Oblique Logistic Function for the Rank-Frequency Distribution of Letters

The rank-frequency distribution of smaller subword units do not usually exhibit a linear behavior on the logarithmic scale in contrast to word n-grams. While the latter has been extensively studied in the literature since the seminal work by Zipf, relatively less attention has been paid to the distributional properties of letters both from an analytical and empirical point of view. In this paper, we propose a continuous function that can operate akin to a double pareto distribution with two regimes to fit rank-frequency data of letters. For evaluation, we compare it to the beta rank function, which has been consistently found to be the most accurate representation of such distributions by previous studies. Applied on a hundred different languages from a parallel corpus, it was shown that the proposed function yields a lower root mean square error on average outperforming the parameterized beta rank function for the majority of languages.