Relaxing Pairwise Conditional Dependency of NB Attributes using Hidden Variables

Conditional independence assumption of naive Bayes classifier as one of its major drawbacks, for real-world applications, has been recently tackled by various structural learning approaches. However, finding the optimal structure is itself reported to be an NP-hard problem. Compliance of the devised structure with general rules and concepts of the literature of Bayesian networks is also another challenge facing many researchers. Eventually, the computational complication of structural learning has impelled many scholars to adopt approaches to confine the search space of possible structures. In this paper, we incorporate a set of hidden variables in a predefined Bayesian network structure, for proposing Pairwise Hidden Naive Bayes Classifier (PHNB). In PHNB, a hidden variable is allocated for each pair of features with conditional mutual information higher than a specific threshold in a diverging connection to preserve their dependency whenever the class variable is instantiated. The Expectation-Maximization (EM) algorithm is modified using a Bayesian correction approach so as to consider an infinitesimal possibility for unforeseen events while converging to a local optimum. Experiments on 6 UCI benchmark datasets reveal substantial results based on classification accuracy(ACC) and area under the ROC curve(AUC).