Long Context Question Answering With BERT

Currently, BERT and other transformer-based models have a constraint on the sequence length of their inputs. To overcome this limitation, we propose breaking down the input text into smaller-sized chunks of a specified size, in this case being 512, using the Natural Language Toolkit's tokenizing features. We employ this model for question answering by fine-tuning it with the SQuAD dataset. To establish the feasibility of the solution, we have built a distributed, client-server system that enables clients to interface with the question-answering capabilities of the model via a RESTful API.