Neural Knapsack: A Neural Network Based Solver for the Knapsack Problem

This paper introduces a heuristic solver based on neural networks and deep learning for the knapsack problem. The solver is inspired by mechanisms and strategies used by both algorithmic solvers and humans. The neural model of the solver is based on introducing several biases in the architecture. We introduce a stored memory of vectors that holds up items representations and their relationship to the capacity of the knapsack and a module that allows the solver to access all the previous outputs it generated. The solver is trained and tested on synthetic datasets that represent a variety of instance types with different complexities. The solver neural model capabilities to generalize were tested on instances with up to 200 items, the model succeed to obtain near optimal solutions better than the greedy algorithm for instances in which there exists a correlation between items values and weights. The results also show that the capacity of the knapsack has a role in learning useful representations for each item in an instance and for the instance itself. Although the proposed solver may be not superior to other solvers, the results described here are insights for how the connection between combinatorial optimization, machine learning, and cognitive science could serve a great purpose in the operation research field. The goal of this work is not to design a state of the art solver, rather it full-fills some of the holes in the recent line of research that incorporates learning in combinatorial optimization problems.