Relaxed Combinatorial Optimization Networks with Self-Supervision: Theoretical and Empirical Notes on the Cardinality-Constrained Case

Self-supervised neural networks for combinatorial optimization (CO) handle non-differentiable constraints via relaxation. Despite their superiority in efficiency, one possible limitation is that these methods often put the constraints as soft penalty terms in the learning objective, and the degree of constraint-violation usually cannot be accurately or directly modulated. In this paper, we aim to develop a new paradigm to solve the CO problem by incorporating the constraints into the network architecture and computational operators, which is a more natural learning pipeline and decouples the constraint violation penalty from the raw objective optimization. Seeing such a paradigm may be rather general such that there only exist perturbation-based blackbox differentiable learning methods as generic solvers in literature, here we consider the commonly used cardinality constraints which in fact can incorporate many existing CO problem instances as its special cases. Specifically, the cardinality constraints are encoded by a differentiable optimal transport layer. We theoretically characterize the constraint-violations of two variants of our architecture (w.r.t. existing CO network whose constraint-violation is non-controlled), and we further show that their empirical performances are in line with our theoretical results. On self-supervised learning of pure CO problems on synthetic and real-world data, our networks surpass the state-of-the-art CO network, and are comparable to Gurobi and can sometimes even surpass. Our general paradigm also enables the application of end-to-end predictive portfolio optimization on real-world asset price data, improving the Sharpe ratio from 1.1 to 2.1 with a predict-then-optimize paradigm with LSTM+Gurobi.