In this study, we focus on finding exact binary optimal classification trees (OCTs) via mixed-integer (linear) programs (MIP). To alleviate the scalability issues of existing MIP-based OCT formulations, we propose a novel decomposable model that treats each leaf of the classification tree as a decision rule. The model exploits the order-independent nature of decision rules to reduce the problem size and the solution space, and lends itself to a Dantzig–Wolfe (extended) reformulation. We then implement a tailored branch-and-price solution with pricing subproblems designated for each leaf solved in parallel. We identify the conditions under which the extended formulation performs better in order to form a hybrid algorithm, OrderIndependentOCT, that decides when to invoke branch-and-price. On 18 benchmark datasets, OrderIndependentOCT proves optimality on 71.1% of instances within a one-hour time limit, compared to 60.0% for the well-known Benders decomposition approach (BendersOCT). In cases where neither method attains optimality, OrderIndependentOCT yields higher (or the same) training accuracy than BendersOCT in 50.68% (resp. 71.23% ) of the instances. Similarly, in 86.3% of all instances, OrderIndependentOCT matches or exceeds BendersOCT’s out-of-sample accuracy. These findings illustrate the efficiency of the proposed decomposition approach for the optimal classification tree training problem compared to the state-of-the-art.
The most cost-effective and energy-efficient way to handle many computationally expensive tasks in the modern computing environment is to offload those tasks to graphics processing units (GPUs). However, for the most expensive branch of mathematical optimization—deterministic global optimization—no existing mature solvers, to the best of our knowledge, use GPUs in any capacity. In this paper, we demonstrate that a core component of deterministic global optimization techniques—the generation of relaxations and relaxation subgradients—can be effectively performed on GPU hardware for arbitrarily complicated factorable expressions. This is accomplished using a source code generation approach for McCormick-based relaxation rules made available in the open-source software package SourceCodeMcCormick.jl. Across a benchmark test set of problems from MINLPLib, we achieve speedups in the calculation of relaxations and their subgradients in the range of one-to-four orders-of-magnitude over a typical CPU-based implementation of these rules at a level of parallelization that would be useful for a GPU-based deterministic global optimizer.
Feasibility Pump (FP) is a well-studied mixed integer linear programming primal heuristic that sequentially guides fractional solutions into integer feasibility by minimizing their distance to given target integer points, while preserving the linear constraints of the original problem. Due to its effectiveness and simplicity, FP has been embedded in most commercial solvers, and several variants of the basic method have been proposed, aiming at improving its convergence and the quality of the solutions obtained. This work introduces Kernel Pump (KP), a novel FP speed-up technique that relies on the intuition that not all binary variables are active (i.e., set to one) in a feasible solution, and can, thus, be sub-divided in a kernel search fashion. Precisely, KP identifies a subset of promising binary variables (the kernel) and distributes the rest into buckets ranked by their likelihood of composing a feasible/optimal solution. FP sub-problems are then solved iteratively, focusing on the current kernel and progressively adding variables from the buckets whenever necessary. Extensive computational experiments on benchmark instances from MIPLIB 2017, as well as two additional problems from the literature of vehicle routing and scheduling, reveal a considerable boost in success rate, also decreasing running times for the majority of instances tested. Notably, aside from outperforming FP, the new method was able to find feasible solutions in cases where CPLEX failed within an hour of search. To our knowledge, this is also the first study to introduce a decomposition approach within the FP framework.
Probing and dual fixing are two powerful presolve techniques in mixed integer programming (MIP) solvers. Probing tentatively sets some binary variables to 0 or 1, applies linear constraint based domain propagation techniques to derive better variable bounds, and extracts useful information such as stronger variable implications and better global variable bounds. Dual fixing attempts to fix variables to lower or upper bounds while ensuring that at least one optimal solution is retained, as long as the problem was feasible. In this paper, we investigate how to combine the two approaches in MIP solvers to achieve a better performance. In particular, we first embed dual fixing into the probing framework, deriving more useful variables' implications for enhancing the capability of probing. Then, we develop an improved dual fixing technique where more variable fixings can be applied, and use the probing framework to detect the reductions. Computational results on the MIPLIB 2017 benchmark instances demonstrate the potential of the two proposed techniques in combining probing and dual fixing on the open-source MIP solver HiGHS.
In this paper, we study a class of structured sparse optimization problems characterized by a convex, possibly non-smooth loss function and a capped- ℓ _1 penalty. This model can provide an exact continuous relaxation for the problems with a cardinality penalty. First, we propose an Accelerated Nested Proximal Gradient (ANPG) algorithm, which employs a nested proximal structure and selective extrapolation to enhance computational efficiency. Under some mild and easily verifiable conditions, we prove the subsequence convergence of the ANPG algorithm to the lifted stationary points of the considered problem, which correspond to the strong local minimizers of the corresponding cardinality penalty problems. Moreover, we establish an O(k^-1) convergence rate on the objective function values, while a refined extrapolation strategy ensures sequence convergence of the iterates, albeit with a potential reduction in the theoretical convergence rate on the objective values. Second, for the cases with a smooth loss function, we further propose an Accelerated Proximal Gradient (APG) algorithm that guarantees the sequence convergence on the iterates and achieves a faster convergence rate of o(k^-2) in terms of objective values. Finally, the effectiveness of the ANPG and APG algorithms, as well as the high quality of the solutions they produce, are verified through numerical experiments on the least absolute deviation regression problem and the sparse logistic regression problem, respectively.
