Generative Models Reimagined: Review into Resource-Efficient Training

In the era of rapid improvements to generative artificial intelligence, resource-effective training methods are important for optimising models like BERT, ChatGPT, and Gemini. These large-scale models require tremendous processing capacity, which might be a barrier to scalability and sustainability. This review paper dives into novel strategies that improve computing economy while maintaining model performance. Architectural optimisation, quantisation, models pruning, and effective ways for discovering high-quality datasets while maintaining accuracy are among the key topics explored. The research emphasises sustainable AI approaches by demonstrating methods for reducing resource use while maintaining the resilience for generative models across several datasets. Furthermore, this paper identifies additional areas for study, such as refining model structures to reduce energy usage, improving data selection approaches, and creating hybrid models that incorporate different resource-saving techniques. Finally, the paper seeks to contribute to current efforts to balance AI developments with the requirement for long-term, scalable, and efficient model training methods.