Rapid Prototyping of a Text Mining Application for Cryptocurrency Market Intelligence

Blockchain represents a technology for establishing a shared, immutable version of the truth between a network of participants that do not trust one another, and therefore has the potential to disrupt any financial or other industries that rely on third-parties to establish trust. In order to better understand the current ecosystem of Blockchain applications, a scalable proof-of-concept pipeline for analysis of multiple streams of semi-structured data posted on social media is demonstrated, based on open source components. Deep Web as well as conventional social media are considered. Preliminary analysis suggests that data found in the Deep Web is complimentary to that available on the conventional web. Future work is described that will scale the system to cloud-based, real-time, analysis of multiple data streams, with Information Extraction (IE) (ex. sentiment analysis) and Machine Learning capability.