Deep Learning for Natural Language Processing

This chapter describes the application of deep learning methods to natural language processing, with examples of text classification and text generation. It focuses on recurrent neural networks but also on transformer models, in particular, the BERT model, and compares the classification of texts by a recurrent network LSTM and by a transformer model DistilBERT. The chapter shows some examples of the implementation of deep neural networks, and, in particular, recurrent networks. If the source text is very long or if its vocabulary is simple, the ratio “total number of words / number of different words” will be high and it will be possible to generate the new text word‐by‐word. The words of a sentence can be represented in a way that allows convolutional neural networks to be naturally applied to them.