Novelty Encouraged Beam Clustering Search for Multi-Objective De Novo Diverse Drug Design

The generation of drug-like, high-quality molecules from scratch within the expansive chemical space is a significant challenge in drug discovery. In previous research, value-based reinforcement learning algorithms have been utilized to optimize multiple desired properties simultaneously. Randomness is injected into the decision-making process through ε-Greedy or stochastic sampling to enable the generation of a diverse ensemble of molecules, which usually encounters a trade-off between the optimality and diversity of these generated molecules. Moreover, novelty has not been explicitly addressed as an optimization objective, and the distinctiveness of generated molecules from the reference molecules is not guaranteed. In this paper, novelty-encouraged beam clustering (NeBC) search algorithm is proposed for de novo drug design. A clustering strategy is integrated with heuristic value-guided beam search to strike a balance between the optimality and diversity of the generated molecules. An intrinsic reward, which is measured by the disagreement of a group of experts trained on reference molecules, is proposed to encourage novelty explicitly. Experimental results demonstrate that NeBC search not only achieves a balanced trade-off between optimality and diversity but also effectively enhances the novelty of the generated molecules. The source code is publicly accessible on https://github.com/CMACH508/NeBC.