Enhanced Data Analysis by Natural Language Query Processing

This project aims to empower non-technical users in conducting data analysis by enabling effortless data retrieval through natural language queries. These users often lack expertise in querying languages but require access to database-stored data. Traditional methods like SQL query templates and slot-filling techniques have limitations in coverage and understanding complex queries. Models relying on LSTM or BERT may require internet connectivity, limiting offline usability. To address these challenges, transformer-based models, specifically the CodeT5 transformer, were adopted. Unlike previous methods, CodeT5 excels in understanding and generating complex queries accurately, operating offline, and enhancing accessibility. By utilizing CodeT5 and implementing natural language processing techniques for query interpretation, significant improvements were made in the user experience in data analysis tasks. The system generates Pandas queries in Python, integrating seamlessly with Python-based data analysis workflows and supporting multiple data formats for enhanced flexibility. The system streamlines data analysis workflows and increases productivity. Users can upload data in various formats, including csv and txt, and export query results for further analysis or reporting. Its scalability allows adaptation to different databases and querying languages, such as MongoDB or MySQL, catering to diverse applications. The project demonstrated significant improvements in data analysis efficiency and acessibility. The system’s ability to generate and execute queries automatically reduces dependency on technical expertise, empowering users to extract insights from data effortlessly. With its scalability, cross-platform compatibility, and offline functionality, the system represents a significant advancement in democratizing data analysis.