TruthCore: Non-parametric estimation of truth from a collection of authoritative sources

Truth Finding is the problem of determining correct information from several conflicting sources and is required for data aggregation. Existing algorithms solve the problem by simultaneously estimating source qualities and fact confidences, working on either numeric or non-numeric data. However, in practice, datasets are a mixture of several different data types. In this work we present a unified framework for finding truth from a collection of conflicting, authoritative sources. We assume that a small subset of independent reliable sources are selected by a preprocessing step. We formulate truth finding as an outlier removal problem, by modeling the similarities between the values reported by these sources. Our algorithm works in two stages: it first models the similarity graph between the sources and then finds the truth by invoking an outlier removal algorithm. We report experiments on several datasets including results for fixing records in Open Library; an open, editable library catalog.