Pre-trained Transformer-based Classification for Automated Patentability Examination

Patentability examination, which means checking whether claims of a patent application meet the requirements for being patentable, is highly reliant on experts’ arduous endeavors entailing domain knowledge. Therefore, automated patentability examination would be the immediate priority, though under-appreciated. In this work, being the first to cast deep-learning light on automated patentability examination, we formulate this task as a multi-label text classification problem, which is challenging due to learning cross-sectional characteristics of abstract requirements (labels) from text content replete with inventive terms. To address this problem, we fine-tune downstream multi-label classification models over pre-trained transformer variants (BERT-BaseLarge, RoBERTa-BaseLarge, and XLNet) in light of their state-of-the-art achievements on many tasks. On a large USPTO patent database, we assess the performance of our models and find the model outperforming others based on the metrics, namely micro-precision, micro-recall, and micro-F1.