Exploring Data Visualization Techniques for Large Datasets

In a world of Big Data, being able to easily visualize large data sets can make the difference between understanding & wisely judging that data - or not. Big Data is the word with which not many people are aware of, but these success stories make sure that organizations sit back & take notice to what this large data can show then when it collated together over time from different sources. The methods traditionally used for visualization rarely fit into the Big-Data world because they do not have scalability, given that volume (huge amounts), velocity (high speed), or variety (complex details) are now more prevalent in big-data processing. This paper then explores data visualization techniques that can be used to manage very large datasets, including streaming data visualization, dimensionality reduction (e.g. T-SNE), & hierarchical clustering among other Node XL analyses. In this paper, based on accumulated observations & implementations of these techniques over the years, an extensive review is provided for researchers or practitioners to select suitable visualization methodologies that not only suit the scale but also discover patterns in their data. The results highlight the need for enabling tools, methods, & practices to make sense of big data in an interactive visual context that enables better orchestration between complex systems using diverse hardware/software stacks. Additionally, the pros & cons of each method are explained, with real-world use cases & case studies demonstrating their practicality.