Hebbian learning limits Hopfield network storage capacity (pattern-to-neuron ratio around 0.14). We propose Kernel Logistic Regression (KLR) learning. Unlike linear methods, KLR uses kernels to implicitly map patterns to high-dimensional feature space, enhancing separability. By learning dual variables, KLR dramatically improves storage capacity, achieving perfect recall even when pattern numbers exceed neuron numbers (up to ratio 1.5 shown), and enhances noise robustness. KLR demonstrably outperforms Hebbian and linear logistic regression approaches.
Kernel-based learning methods can dramatically increase the storage capacity of Hopfield networks, yet the dynamical mechanisms behind this enhancement remain poorly understood. We address this gap by combining a geometric characterization of the attractor landscape with the spectral theory of kernel machines. Using a novel metric, Pinnacle Sharpness, we empirically uncover a rich phase diagram of attractor stability, identifying a Ridge of Optimization where the network achieves maximal robustness under high-load conditions. Phenomenologically, this ridge is characterized by a Force Antagonism, in which a strong driving force is counterbalanced by a collective feedback force. We theoretically interpret this behavior as a consequence of a specific reorganization of the weight spectrum, which we term Spectral Concentration. Unlike a simple rank-1 collapse, our analysis shows that the network on the ridge self-organizes into a critical regime: the leading eigenvalue is amplified to enhance global stability (Direct Force), while the trailing eigenvalues remain finite to sustain high memory capacity (Indirect Force). Together, these results suggest a spectral mechanism by which learning reconciles stability and capacity in high-dimensional associative memory models.
Purpose: Dynamic scene reconstruction in endoscopic data is crucial for minimally invasive surgery and surgical navigation. Recently, Gaussian splatting (GS)-based methods have performed outstandingly in the reconstruction of dynamic endoscopic scenes. However, more challenging scenes with deformable tissues, surgical tool occlusion, and changeable camera positions make it harder to initialize a GS model well. To address this, we propose Endo-PairGS, which leverages aligned point cloud pairs from two different frames to initialize a 4D GS model. Methods: Our framework contains two parts: static and dynamic 3D reconstruction. Firstly, we fine-tune a foundation model on endoscopic data to obtain aligned point clouds. To ensure the stability of fine-tuning video sequences with deformable tissues, we use an optical flow-based mask to reduce the influence of regions with large movement. Then, we used the generated point cloud pairs to initialize a 4D GS model for dynamic scene reconstruction. The paired point clouds can complement the region masked by the surgical tool and the unseen region caused by the camera position change, which presents a more complete 3D scene. Results: The experiments on EndoNeRF and StereoMIS datasets demonstrate our Endo-PairGS outperformed existing methods on both quantitative and qualitative results. Our method got 32.47 and 0.871 on PSNR and SSIM in StereoMIS (P3), improving 4% and 4.8% compared to the baseline approach, respectively. Conclusion: This study proposed Endo-PairGS with a more complete initialization method, significantly improving the performance of dynamic endoscopic scene reconstruction. The codes and data will be released at https://github.com/MoriLabNU/Endo-PairGS.
High-capacity associative memories based on Kernel Logistic Regression (KLR) are known for their exceptional performance but are hindered by high computational costs. This paper investigates the compressibility of KLR-trained Hopfield networks to understand the geometric principles of its robust encoding. We provide a comprehensive geometric theory based on spontaneous symmetry breaking and Walsh analysis, and validate it with compression experiments (quantization and pruning). Our experiments reveal a striking contrast: the network is extremely robust to low-precision quantization but highly sensitive to pruning. Our theory explains this via a “sparse function, dense representation” principle, where a sparse input mapping is implemented with a dense, bimodal parameterization. Our findings not only provide a practical path to hardware-efficient kernel memories but also offer new insights into the geometric principles of robust representation in neural systems.
