A Hitchhiker's guide to developing software for hard and soft information fusion

An increasing number of domains require fusion of both "hard" (i.e. quantitative, sensor-based) data and "soft" data, which is derived from human sources and typically more subjective in nature. Although many of these domains have very similar (or even overlapping) requirements from a technology standpoint, the development processes tend to be independent with little exchange of software design patterns or lessons learned between development efforts from different organizations. Recent projects such as IBM's Watson system and Wolfram's Wolfram-Alpha engine have demonstrated the feasibility of working successfully with very large and unstructured datasets. However, most researchers lack the time and manpower of these well-funded efforts. This paper bridges that gap by describing the framework creation, software implementation, data generation, and simulation design necessary for fusing hard and soft information. Specific areas of implementation guidance include representation of heterogeneous data, complex event processing (CEP), multi-agent software systems, programming language selection, artificial intelligence (AI) techniques, user interface (visualization, sonification, and non-standard interfaces), and distributed processing techniques (including the use of message oriented middleware).