Computer Vision-Based Framework for Data Extraction From Heterogeneous Financial Tables: A Comprehensive Approach to Unlocking Financial Insights

Information extraction from financial document images is crucial in computer vision and NLP, as financial data often exists in image or PDF format, enabling organizations to analyze and make informed business decisions using OCR advancements. The table contents of financial document images are one of the prominent structures to confine important portions of data of the document and many Deep learning-based methods have been proposed to detect Table regions inside document images. The shortcomings of the current approach are that it is bounded within the detection of the table region and struggles in cases such as handling different layouts and preserving the relation among the different attributes of the table. Therefore, in this work, we proposed an end-to-end architecture to extract information from Financial table images while preserving the column row structures of the attributes within the table. We divided the task into four modules and generated synthesized data with different augmentation techniques to overcome data scarcity challenges and boost the performance of the pipeline modules. In terms of information extraction, the proposed method acquired 85% accuracy in the target invoice dataset.