Incorporating Proportional Sparse Penalty for Causal Structure Learning

Inferring causal relationships is key to data science. Learning causal structures in the form of directed acyclic graphs (DAGs) has been widely adopted for uncovering causal relationships, nonetheless, it is a challenging task owing to its exponential search space. A recent approach formulates the structure learning problem as a continuous constrained optimization task that aims to learn causal relation matrix. Following it are various variants, such as nonlinear variants which uncover nonlinear causal relationships. However, the nonlinear variant which considers an ordinary sparsity constraint as part of its optimization objective may not effectively eliminate spurious edges, i.e., wrongly predicted edges. In this paper, we investigate the defect that the ordinary sparsity constraint inherently contains, that is, when the model predicts spurious edges, the sparsity constraint may be unable to make the relation matrix sparse to help eliminate such spurious edges, and based on the theoretical and empirical analysis of the defect, we propose the proportional sparse penalty constraint PSP which replaces the ordinary sparsity constraint with a normalized first-order matrix norm. We then compare our proposed model GAE-PSP with three models to show considerable performance improvement. We further conduct three comparative experiments and show that the PSP can make the relation matrix sparse to eliminate spurious edges.