Performance of BERT word embedding technique for medical text classification

The integration of natural language processing (NLP) in the medical field has seen significant advances. Machine learning and deep learning techniques, combined with NLP, have revolutionized medical data management and improved diagnostic processes. An essential part of natural language processing (NLP) is feature engineering, which involves extracting relevant features from raw textual data and converting them into a format compatible with machine learning and deep learning models. BERT, a general linguistic model pre-trained on a corpus of texts is among the word embeddings techniques that has demonstrated remarkable performance in a variety of natural language processing tasks thanks to its task-specific fine-tuning. In this work, we apply BERT, for feature extraction to two datasets concerning the medical domain, one dataset on cancer types and the other on mental health, then trained classification models and evaluated the performance of each model.