A Preprocessor for Creation of Large-Scale Computational Applications

The emergence of large-scale computing paradigms required the evolution of software development methods, moreover, the need to effectively transform traditionally sequential applications into parallelizable and scalable architectures. This transformation is key in harnessing the computing power of modern multicore and distributed systems. Python, with its vast ecosystem and simplicity, is becoming a powerful language that facilitates this transition. This article discusses the methodologies and development of a Python preprocessor tool aimed at converting sequential applications into forms suitable for large-scale computing environments. Through detailed analysis and implementation, we demonstrate the use of Spark functions and third-party libraries to automate this transition, making large-scale computing more accessible to a wider range of applications and developers. The tool includes a set of rules that identify different patterns of code input (for example, list-sequential looping) and transform consistent code. This article introduces a Python preprocessor designed to automate the transformation of sequential Python applications into ones that harness the power of large-scale computing environments.