Fast and Accurate FSA System Using ELBERT: An Efficient and Lightweight BERT

The accuracy of many natural language processing (NLP) tasks has been significantly improved with the development of deep learning and Transformer-based pre-trained models like BERT. However, their large number of parameters and computations pose deployment challenges. For instance, while using BERT can improve predictions in financial sentiment analysis (FSA) tasks, it can also slow down processing, where throughput and accuracy are equally important for profitability. To address these issues, we first propose an efficient and lightweight BERT model called ELBERT, which incorporates a novel confidence-window-based (CWB) early exit mechanism. In addition, an innovative approach is proposed to accelerate text processing on the GPU platform, which addresses the challenge of effectively deploying the early exit mechanism with large input batch sizes. Using these advancements, we develop a fast and high-accuracy FSA system. Experimental results demonstrate that the proposed CWB early exit mechanism outperforms existing early exit methods on BERT in terms of accuracy, at the same computation cost. By leveraging this acceleration method, our FSA system can boost the processing throughput by nearly 40 times, enabling it to process over 1000 texts per second with sufficient accuracy. This processing rate is nearly twice as fast as FastBERT, providing modern trading systems with a more powerful text processing capability.