Data retrieval for client projects: Matching data onto an ontology map to produce a relevance assessment

Discerning relevant data is becoming more difficult, time consuming, and costly as the amount of data available dramatically increases. Currently, consulting firms strive to use data to support client's business decisions with evidence. To be effective at this, consultants must consider the applicability of both internal and external data libraries to their clients' requirements. Frequently, evaluating the applicability of data sets is a manual process, which can be costly to the firm and the client. This paper describes a technical approach to automate this process. Specifically, it details the structure of a software application, named UVa Open Miner, capable of assessing the applicability of data sources to client projects. This UVa Open Miner aims to maximize the scale and diversity of candidate data sets, increase the relevance of data found, and maintain manageable computational complexity. UVa Open Miner consists of two segments: mapping and matching. The mapping component text mines web pages to identify an ontology of keywords describing the business requirement. This enables users to handle diverse business requirements from various industry verticals. The matching component scores data sets based on a relevance factor obtained from the ontology map. To validate the application, subject matter experts provided business requirements for a problem in their domain, and validated the application's results. Professionals in environment science, political science and policy-making fields found the application to be useful. Therefore, the application, along with the framework used, can be refactored into a reusable solution for consulting firms to use for their clients.