Database Systems in the Big Data Era: Architectures, Performance, and Open Challenges

The advent of Big Data has fundamentally transformed the database management systems (DBMS) field, necessitating the development of innovative paradigms, architectures, and technologies to address unprecedented challenges. Despite their historical dominance, traditional systems falter under the high velocity, massive volume, and diverse variety of Big Data. These limitations have catalyzed the emergence of alternative solutions such as not only structured query language (NoSQL), NewSQL, and cloud-native databases, each offering unique approaches to scalability, flexibility, and performance optimization. This survey provides a comprehensive and systematic overview of the evolving database ecosystem in the Big Data era. It delves into the historical progression from traditional relational DBMS (RDBMS) to modern paradigms, emphasizing these transformations’ motivations, trade-offs, and innovations. The classification of databases based on data models, deployment strategies, scalability mechanisms, and consistency models is explored in depth, providing a structured framework for understanding their diverse capabilities. Furthermore, critical performance characteristics, including throughput, latency, fault tolerance, and cost efficiency, are analyzed to assess their effectiveness in real-world applications. By highlighting persistent challenges such as data heterogeneity, security, and interoperability, this survey outlines key research directions, fostering a holistic understanding of the domain and inspiring future advancements in database technologies.