FeBT: A Feature Balancing Transformer for Corporate ESG Forecasting

Environmental, social, and governance (ESG) serves as a crucial indicator for evaluating firms in terms of sustainable development. However, the existing ESG evaluation systems suffer from limitations, such as narrow coverage, subjective bias, and lack of timeliness. Therefore, there is a pressing need to leverage machine learning methods to predict the ESG performance of firms using their publicly available data. Traditional machine learning models encounter the feature imbalance problem due to the heterogeneity in ESG-related features. Common approaches typically involve unfolding all features, thereby granting high-dimensional folding features greater exposure and accessibility to downstream models, which results in the neglect of low-dimensional features. To fill the research gap regarding fully using the heterogeneous features of enterprises to enhance AI-based ESG prediction performance, we propose the Feature Balancing Transformer (FeBT), a model based on autoencoders and Transformer blocks. FeBT incorporates a novel feature balancing technique that compresses and enhances high-dimensional features from imbalanced data into low-dimensional representations, thereby ensuring a more balanced impact of high-dimensional and low-dimensional features on the model’s performance in the downstream ESG forecasting module. Extensive experiments verified the superior performance of FeBT compared with state-of-the-art methods in real-world ESG-related datasets and evidenced that our feature balancing module provides significant insights from high-dimensional folding features.