Depth-recurrence facilitates latent reasoning by sharing parameters across depths. However, prior work lacks combined FLOP-, parameter-, and memory-matched baselines, underutilizes depth-recurrence due to partially fixed layer stacks, and ignores the bottleneck of constant hidden-sizes that restricts many-step latent reasoning. To address this, we introduce a modular framework of depth-recurrent attention mixtures (Dreamer), combining sequence attention, depth attention, and sparse expert attention. It alleviates the hidden-size bottleneck through attention along depth, decouples scaling dimensions, and allows depth-recurrent models to scale efficiently and effectively. Across language reasoning benchmarks, our models require 2 to 8x fewer training tokens for the same accuracy as FLOP-, parameter-, and memory-matched SOTA, and outperform ca. 2x larger SOTA models with the same training tokens. We further present insights into knowledge usage across depths, e.g., showing 2 to 11x larger expert selection diversity than SOTA MoEs.
The Positive Influence Dominating Set Problem (PIDS) is a variant of the well-known Dominating Set Problem. It involves selecting a subset of vertices that positively dominate the remaining vertices in a given graph G=(V,E) . More formally, a vertex v_i ∈ V is said to be positively dominated if at least a portion of ρ deg_G(v_i) of its neighbors belongs to the selected set, where deg_G(v_i) is the degree of v_i , and 0<ρ <1 is the influence factor. The objective is to identify the smallest subset of positive influence dominants. This problem is NP-hard on general graphs and remains computationally challenging even for particular classes of graphs. In this paper, we propose an efficient algorithm for solving PIDS, based on the Local Branching approach combined with the CPLEX mathematical programming solver, especially to enhance intensification and provide a good partial solution. Also, to ensure solution diversification, we develop a destructive-reconstructive greedy heuristic to explore previously unvisited subspaces. We conduct extensive experiments to evaluate our method on real-world benchmark instances of varying sizes, including large-scale graphs, and compare it with five existing state-of-the-art solving approaches. The experimental results demonstrate that the proposed approach yields high-quality solutions within reasonable computational times, highlighting its performance for efficiently solving PIDS.
Recurrent-depth reasoners aim to solve harder problems by iterating their update longer at test time, but additional iterations can improve, preserve, or degrade an answer. We show that a measurable property of the trained operator, its finite-time dynamical regime (estimated as settling, marginal, or drifting), indicates which of these occurs. We give a sufficient condition for depth-safety: once an operator's per-step displacement is small relative to the decoder margin, the decoded answer cannot change under further iterations. Empirically, on algorithmic tasks trained from 800 unaugmented examples per difficulty tier, settling operators do not degrade with added depth, and on some tasks convert it into higher accuracy on harder unseen instances (Sudoku, 0.19 to 0.34 past the training horizon). A single terminal fixed-point objective moves the regime and the depth behavior together: removing it induces drift and removes the gains, and adding it to a generic recurrence yields depth-safe extrapolation on carry propagation. We give four operational criteria for useful test-time depth, use them to catalogue failure modes, and, as a consistency check, apply the same measurements to Huginn-3.5B, which falls in the non-settling family.
BACKGROUND:We previously showed that ancestral-specific anti-Spike binding IgG concentration and 50% inhibitory dilution neutralizing antibody titer (nAb-ID50) measured at 2 weeks postdose 2 (∼peak) were inverse correlates of risk (CoRs) of COVID-19 over 2 months post ∼peak in the PREVENT-19 trial of the NVX-CoV2373 vaccine; there were not sufficient data to assess CoRs of severe COVID-19. METHODS:Here, we assessed, in the same vaccinated cohort, Delta- and ancestral-specific Spike IgG and nAb-ID50 at ∼peak and over time as CoRs of severe COVID-19 and of Delta COVID-19 over 3.5-10 months post ∼peak (287 breakthrough Delta cases, including 8 severe; 446 noncases). RESULTS:Peak antibody levels were much higher for noncases versus severe cases (all inferred Delta), with nAb-ID50 Delta geometric mean 209.5 arbitrary units (AU)/mL (95% CI: 176.1, 249.1) versus 9.6 AU/mL (95% CI: 2.4, 38.6), respectively. Frequency of detectable nAb-ID50 titer was 98.3% (97.2, 99.0) for noncases versus 62.5% (22.3, 93.9) for severe cases. All markers were inverse CoRs of severe COVID-19, with a ∼peak hazard ratio (HR) of 0.13 (95% CI: .03, .57) per 10-fold nAb-ID50 Delta increase. Severe COVID-19 risk through 305 days postday 35 was 0.0338 (0.0043, 0.206) at the nAb-ID50 Delta 2.5th percentile (8.4 AU/mL), and 0.002 (0.0000, 0.0108) and 0.0002 (0.0000, 0.0035) at the 50th and 95th percentiles (210, 2522 AU/mL). CONCLUSIONS:Postvaccination NVX-CoV2373 antibody levels are stronger predictors of severe COVID-19 than any-severity Delta COVID-19. Low antibody responses indicate vulnerability to severe COVID-19.
Lensed quasars are key to many areas of study in astronomy, offering a unique probe into the intermediate and far universe. However, finding lensed quasars has proved difficult despite significant efforts from large collaborations. These challenges have limited catalogues of confirmed lensed quasars to the hundreds, despite theoretical predictions that they should be many times more numerous. We train machine learning classifiers to discover lensed quasar candidates. By using semi-supervised learning techniques we leverage the large number of potential candidates as unlabelled training data alongside the small number of known objects, greatly improving model performance. We present our two most successful models: (1) a variational autoencoder trained on millions of quasars to reduce the dimensionality of images for input to a dense neural network classifier that can make accurate predictions and (2) a convolutional neural network trained on a mix of labelled and unlabelled data via virtual adversarial training. These models are both capable of producing high-quality candidates, as evidenced by our discovery of GRALJ140833.73+042229.98. The success of our classifier, which uses only multi-band images, is particularly exciting as it can be combined with existing classifiers, which use other data than images, to improve the classifications of both models and discover more lensed quasars.