5 Design and Implementation of Big Data Warehouses

Big Data is a concept of major relevance in today’s world, sometimes highlighted as a key asset for productivity growth, innovation, and customer relationship, whose popularity has increased considerably during the last years. Areas like smart cities, manufacturing, retail, finance, software development, environment, digital media, among others, can benefit from the collection, storage, processing, and analysis of Big Data, leveraging unprecedented data-driven workflows and considerably improved decision-making processes. The concept of a Big Data Warehouse (BDW) is emerging as either an augmentation or a replacement of the traditional Data Warehouse (DW), a concept that has a long history as one of the most valuable enterprise data assets. Nevertheless, research in Big Data Warehousing is still in its infancy, lacking an integrated and validated approach for designing and implementing both the logical layer (data models, data flows, and interoperability between components) and the physical layer (technological infrastructure) of these complex systems. This book addresses models and methods for designing and implementing Big Data Systems to support mixed and complex decision processes, giving special attention to BDWs as a way of efficiently storing and processing batch or streaming data for structured or semi-structured analytical problems.