DeepDiseaseInsight: A Deep Learning & NLP based Novel Framework for generating useful Insights from Disease News Articles

We live in an era of information explosion where media for information communication and information sharing have proliferated due to the onset of a digital age. One such important source and medium of consolidated, comprehensive and reliable information, which has persisted through centuries, is news articles. News articles have been considered as a primary source of information, ranging from local to global scope, and encompassing a wide array of domains, by most of the human population. However, as the technology for dissemination of information attracts innovations, it becomes equally important that the consumption and utilization of this information becomes smart as well. If processed properly and treated in an intelligent manner, the information contained in news articles can reveal useful insights and hidden patterns that might prove to be of critical importance in certain decision making pertaining to the topics and domains in question. The paper puts forward a novel framework created by combining Natural Language Processing methodologies like Sentiment Analysis and Information Retrieval, which are performed on top of a Deep Learning based classification with aptly defined classes for generating useful insights from a news article dataset. The paper contains the details regarding the implementation of this system and also a presentation of results on a particular data set of news articles - articles on HIV-AIDS, which have been categorized into meaningful classes and have been processed using aforementioned NLP tasks. Through this paper, a curated dataset of 3,000 news articles related to HIV-AIDS has also been introduced, which will be open sourced so as to inspire further research and experimentation in this specific domain.