Unsupervised Style Transfer of Modern Hebrew using Generative Language Modeling and Zero-Shot Prompting

Style transfer is one of the most intriguing hallmarks of natural language processing. It involves the semantic preserving conversion of artistic “style”. Style transfer of the Hebrew language is an exceptionally challenging task due to the language’s intricate morphology, inflectional structure, and orthography, which have undergone significant transformations throughout its history. In this work, we present the first generative language model for unsupervised textual style transfer for modern Hebrew, which rewrites sentences in a target style in the absence of parallel style corpora. We create a pseudo-parallel corpus through back translation, fine-tunes a pre-trained Hebrew language model, and leverages zero-shot learning. Our results demonstrate the first significant results in Hebrew style transfer in terms of transfer accuracy, semantic similarity, and fluency.