Modern Hopfield Networks (MHNs) have achieved widespread success across various domains but are confined to Euclidean/Hilbert spaces, failing to preserve the hierarchical structure of data due to geometric constraints—arbitrary tree structures cannot be embedded with low distortion, while hyperbolic spaces can naturally accommodate hierarchical structures through exponential volume growth. To address this issue, we propose Hyperbolic Associative Memory Networks (HAMNs), the first framework to embed modern associative memory into hyperbolic space: we map query and memory vectors from Euclidean space to a constant negative curvature manifold via exponential maps, define a regularized energy function based on the Minkowski inner product, and adopt curvature-aware Riemannian optimization combined with exponential map updates to achieve stable on-manifold retrieval. We put forward a hierarchy-sensitivity hypothesis—HAMNs outperform Euclidean MHNs on data with deep hierarchies but exhibit comparable performance on data with weak or shallow hierarchies, which is validated by depth-controlled experiments and cross-level metrics. As a plug-and-play, model-agnostic module, HAMNs are suitable for the storage and retrieval of representations in task architectures requiring hierarchical understanding, instantiated with the Poincaré ball in experiments, and also applicable to any hyperbolic model with constant negative curvature.

The task of estimating human dense correspondences from images is critical in human-centric analysis, yet existing methods face a trade-off between speed and accuracy. Direct regression approaches are fast but often lack geometric precision, while optimization-based techniques are more accurate but computationally expensive and prone to local minima. This work introduces PDCFMO, a cohesive framework that bridges this gap by harmoniously reconciling the paradigms of broad-scope regression and task-specific meta-optimization. The approach commences with an efficient method for estimating human dense correspondences using a 1D heatmap and a visibility confidence measure, supplemented by a novel technique that generates pseudo-groundtruth visibility using a soft z-buffering ordering scheme, addressing the lack of visibility labels. The key novelty lies in a task-specific neural network-based meta-optimizer that learns descent directions by fusing historical first- and second-order information, integrating task-specific prior knowledge into an iterative optimization process. This improves adaptability to specific settings and handling of complex gestures. Additionally, a memory-efficient Symmetric Rank-one (SR1) inverse Hessian approximation is integrated into the training process, enabling accurate approximations while minimizing memory usage. Evaluations on 3DPW, Human3.6M, and People Snapshot datasets demonstrate notable performance improvements, achieving an eightfold increase in convergence speed over conventional methods, underscoring the framework’s robustness and efficiency.

Cross-modal matching, a fundamental task in bridging vision and language, has recently garnered substantial research interest. Despite the development of numerous methods aimed at quantifying the semantic relatedness between image-text pairs, these methods often fall short of achieving both outstanding performance and high efficiency. In this paper, we propose the crOss-Modal sInkhorn maTching (OMIT) network as an effective solution to effectively improving performance while maintaining efficiency. Rooted in the theoretical foundations of Optimal Transport, OMIT harnesses the capabilities of Cross-modal Mover's Distance to precisely compute the similarity between fine-grained visual and textual fragments, utilizing Sinkhorn iterations for efficient approximation. To further alleviate the issue of redundant alignments, we seamlessly integrate partial matching into OMIT, leveraging local-to-global similarities to eliminate the interference of irrelevant fragments. We conduct extensive evaluations of OMIT on two benchmark image-text retrieval datasets, namely Flickr30K and MS-COCO. The superior performance achieved by OMIT on both datasets unequivocally demonstrates its effectiveness in cross-modal matching. Furthermore, through comprehensive visualization analysis, we elucidate OMIT's inherent tendency towards focal matching, thereby shedding light on its efficacy. Our code is publicly available at https://github.com/ppanzx/OMIT.

Cross-modal semantic alignment is essential for multimodal understanding, yet existing approaches rely on global embeddings that lose fine-grained structural relationships. This paper presents Graph Neural Operators (GNOs), a framework that learns function-to-function mappings between graph-structured representations across modalities. By treating visual and textual graphs as functions over their respective domains, GNOs integrate structure preservation, relational alignment, and semantic correspondence within a unified architecture. Experiments on Visual Genome and Flickr30K demonstrate competitive performance (F1=0.5662 on Flickr30K) using only 500–5000 training samples with interpretable entity-level correspondences. Key contributions include: (1) first application of neural operators to cross-modal alignment, (2) a weighted loss function achieving 180-fold improvement on sparse alignment labels, and (3) evidence that data quality and architectural simplicity outweigh large-scale training. Comprehensive robustness analysis, scaling studies, and comparisons with seven baselines validate the approach.

Long-distance truck transportation often results in stained, faded, or blurry frontal images, leading to fluctuations in image resolution and feature degradation. To enable high-precision truck brand recognition in challenging highway scenarios, this study evaluates convolutional neural networks (CNNs) enhanced by transfer learning (TL). First, four typical neural networks, such as InceptionV3 based on TL, Xception based on TL, Xception based on TL and DenseNet201 based on TL, are exploited for the recognition of truck brands. Second, a new network architecture is proposed, a fused deep neural network (FDNN), based on transfer learning for the recognition of truck brands (FDNN-TL-RTB), which integrates the convolution features of the last layer of InceptionV3-TL-RTB, Xception-TL-RTB, and DenseNet-201-TL-RTB networks based on the tandem fusion rules. Finally, the comparative experiments on CNNs are carried out using the Truck Brands Data Sets of Southeast University. The data set was obtained using continuous capture with cameras installed on the highway to obtain data under various conditions. The experimental results demonstrate that the proposed FDNN-TL-RTB network achieves a superior recognition accuracy of 98.16% on the test set. Although Shidai vehicles presented the most significant classification challenges among the 23 truck brand categories, the FDNN-TL-RTB method achieved a 95% recognition accuracy for this category. The high accuracy and robustness of the model highlight its significant potential for practical applications in intelligent transportation systems, such as automated toll collection and traffic flow monitoring.

Image-text matching, a core task in multimodal learning that aligns visual and textual semantics, faces two critical challenges: (1) existing graphbased methods often struggle to balance over-connection and semantic loss due to rigid thresholding strategies, and (2) single-level interaction mechanisms fail to capture hierarchical cross-modal dependencies effectively. As a response to the identified problems, our research proposes an advanced framework integrating Dynamic Semantic Graph Enhancement (DSGE) with Progressive Semantic Alignment (PSA). The DSGE module adaptively adjusts graph connectivity based on the statistical properties of similarity distributions, overcoming the limitations of manually defined thresholds that typically result in either over-connection or the omission of critical relationships. The PSA module establishes coarse-grained correspondences through bidirectional cross-modal attention and progressively refines alignment precision using a context-aware hierarchical strategy. Comprehensive evaluations on Flickr30K and MS-COCO, particularly in complex semantic scenarios, confirm our framework achieving significant performance gains over existing methods.

In this study, we propose utilizing the Hopfield network for cancer survival prognosis using Whole Slide Image (WSI) datasets. WSIs, known for their high-resolution histological details, are increasingly adopted in cancer diagnostics but pose challenges due to their massive data size and the need for both efficiency and precision in predictive models. By leveraging the associative memory capabilities of the Hopfield network, we model the complex relationships between histological features in WSIs and patient survival outcomes. Experimental results on specific dataset, e.g., TCGA, demonstrate that the Hopfield network outperforms traditional machine learning models, such as specific examples, e.g., random forests or support vector machines, and achieves comparable or superior accuracy to state-of-the-art deep learning approaches, with significantly lower computational overhead. These findings underscore the potential of the Hopfield network as an effective and interpretable tool for survival prognosis, opening promising directions for future research in medical image analysis and personalized medicine.

Effective representation of whole slide images (WSIs) is essential for survival prediction tasks. Previous studies have primarily focused on multimodal approaches, exploring complex fusion techniques to integrate information from different modalities. However, these methods are met with several challenges: (1) Increasingly complex modality fusion techniques result in prolonged model training and inference times. (2) Histology datasets are typically small, making these complex models vulnerable to overfitting. To address these issues and improve a model’s ability to capture effective representations without increasing complexity, we introduce multi-task learning into survival prediction. Specifically, we propose a multi-task survival prediction framework that incorporates tumor staging classification as an auxiliary task, trained simultaneously with the survival prediction task. To the best of our knowledge, this is the first study to integrate tumor staging information into survival prediction. Our method was comprehensively evaluated through unimodal and multimodal experiments across five TCGA datasets. Most experiments demonstrated improved performance, with the best C-Index showing a 16.3

In multi-domain time-series modeling, differences in underlying system dynamics across domains present significant challenges for effective knowledge transfer. Traditional domain adaptation methods often focus on aligning feature distributions, while overlooking the temporal evolution laws that govern the data. In this paper, we propose a shared-embedding Koopman framework for learning aligned dynamical models across multiple domains. Our method embeds observations from each domain into a common latent space, where domain-specific linear Koopman operators capture the temporal dynamics. Specifically, nonlinear systems are transformed into linear Koopman representations through learned neural network embeddings, enabling tractable analysis of complex dynamics. To achieve cross-domain consistency, we introduce a dynamics alignment mechanism that enforces similarity between the latent transition operators via operator alignment losses and spectral regularization. We employ Procrustes analysis for eigenvector alignment, which minimizes the Frobenius norm between aligned spectral bases, ensuring that domains share similar fundamental dynamical modes. Furthermore, we incorporate a self-supervised consistency loss to enhance the robustness of latent representations against sequence augmentations. The augmentation strategies include temporal jittering, scaling, and masking, which preserve the essential dynamical properties while introducing controlled variations. This joint modeling approach allows the framework to achieve both intra-domain accurate prediction and inter-domain transferable forecasting. We evaluate our method on three physiological time-series datasets: PAMAP2 Physical Activity Monitoring, UCI Human Activity Recognition (HAR), and EEG Motor Movement/Imagery datasets from multiple subjects, demonstrating superior transfer prediction performance compared to conventional non-aligned models and state-of-the-art domain adaptation baselines including transformer-based methods. Our approach achieves an average improvement of 49.4 % in Mean Absolute Error across all transfer tasks, with particularly strong performance (81.1 % improvement) on low-dimensional physiological signals. The results confirm that our approach effectively aligns latent dynamics across domains and provides a principled solution for time-series domain adaptation tasks. We discuss limitations regarding scalability to very high-dimensional systems and propose future directions for adaptive alignment strategies.

To propose a deep learning model and explore its performance in the auxiliary diagnosis of lung cancer associated with cystic airspaces (LCCA) in computed tomography (CT) images. This study is a retrospective analysis that incorporated a total of 342 CT series, comprising 272 series from patients diagnosed with LCCA and 70 series from patients with pulmonary bulla. A deep learning model named LungSSFNet, developed based on nnUnet, was utilized for image recognition and segmentation by experienced thoracic surgeons. The dataset was divided into a training set (245 series), a validation set (62 series), and a test set (35 series). The performance of LungSSFNet was compared with other models such as UNet, M2Snet, TANet, MADGNet, and nnUnet to evaluate its effectiveness in recognizing and segmenting LCCA and pulmonary bulla. LungSSFNet achieved an intersection over union of 81.05% and a Dice similarity coefficient of 75.15% for LCCA, and 93.03% and 92.04% for pulmonary bulla, respectively. These outcomes demonstrate that LungSSFNet outperformed many existing models in segmentation tasks. Additionally, it attained an accuracy of 96.77%, a precision of 100%, and a sensitivity of 96.15%. LungSSFNet, a new deep-learning model, substantially improved the diagnosis of early-stage LCCA and is potentially valuable for auxiliary clinical decision-making. Our LungSSFNet code is available at https://github.com/zx0412/LungSSFNet .

Unsupervised Domain Adaptation (UDA) aims to adapt a model trained on a labeled source domain to an unlabeled target domain. In recent years, vision-language models (VLMs) have emerged as powerful tools, achieving remarkable performance on various downstream tasks. However, when applied to UDA, these models often struggle to effectively learn domain-invariant features. In this paper, we propose an Optimal Transport-Based Prompt Alignment (OTPA) method for UDA to achieve fine-grained prompt alignment and learn domain-invariant features. OTPA leverages CLIP's zero-shot inference capabilities and the K-means algorithm to construct codebooks for both source and target domains, followed by a two-stage alignment process. In the first stage, we perform token-level Optimal Transport (OT) alignment between image features and textual prompts to establish a foundational performance. The second stage involves cross-attention between image features and domain-specific codebooks, followed by prompt-level alignment of the enhanced image features with textual features. This two-level OT alignment enables us to capture more fine-grained feature representations and learn domain-invariant features. Extensive experiments demonstrate that OTPA outperforms existing prompt learning methods in UDA tasks across various benchmarks.

AbstractLicense plate recognition is crucial in Intelligent Transportation Systems (ITS) for vehicle management, traffic monitoring, and security inspection. In highway scenarios, this task faces challenges such as diversity, blurriness, occlusion, and illumination variation of license plates. This article explores Recurrent Neural Networks based on Connectionist Temporal Classification (RNN‐CTC) for license plate recognition in challenging highway conditions. Four neural network models: ResNet50, ResNeXt, InceptionV3, and SENet, all combined with RNN‐CTC are comparatively evaluated. Furthermore, a novel architecture named ResNet50 Deep Fusion Network using Connectionist Temporal Classification (ResNet50‐DFN‐CTC) is proposed. Comparative and ablation experiments are conducted using the Highway License Plate Dataset of Southeast University (HLPD‐SU). Results demonstrate the superior performance of ResNet50‐DFN‐CTC in challenging highway conditions, achieving 93.158% accuracy with a processing time of 7.91 ms, outperforming other tested models. This research contributes to advancing license plate recognition technology for real‐world highway applications under adverse conditions.

Multi-label classification is a task with diverse applications, but current algorithms heavily rely on accurately labeled data, leading to time-consuming and labor-intensive data collection. However, multi-label classification with partial labels presents significant challenges. In this study, we propose Multi-modal Contextual Prompt Learning (MCPL), a novel approach that leverages large-scale visual-language models and exploits the strong image-text alignment in CLIP to address the scarcity of label annotations. We pre-train the visual language model’s encoder on a large number of image-text pairs.. We introduce multi-modal contextual prompt learning in both images and labeled text to better utilize the image-label correspondence within CLIP, resulting in enhanced multi-label classification performance, even when faced with partial labels. We also use the coupling function to couple the two modes and realize the interactive connection of the two modal prompts. Extensive experiments on the MS-COCO and VOC2007 datasets, demonstrating its superiority and achieving competitive performance.

Accurate medical image segmentation is crucial for early diagnosis in clinical medicine. However, neural networks for medical segmentation often overlook the combination of frequency and spatial domains, and the employed attention mechanisms treat each channel neuron equally, forming 1D or 2D weights. Such an approach fails to compute true 3D weights effectively. The SimAM paper mentions 3D attention, but its final expression formula suggests that the results are related only to the global mean and variance, without considering local information. We propose a multi-frequency attention model in multi-scale parameter-free attention (LungSSFNet) for lung segmentation in cystic adenocarcinoma datasets to address these challenges. The proposed model includes three key components: the parameter-free attention mechanism (S), the improved feature concatenation method (U), and the multi-scale, multi-frequency attention module (SSF). The U component improves upon traditional feature concatenation by providing a more effective method of capturing differences between deep semantic and shallow features. Thirdly, the SSF component is a multi-scale, multi-frequency attention module based on parameter-free 3D weights. This provides module that can capture the contours of small targets and tissue boundaries significantly. To ensure that the optimal model is not solely a result of parameter tuning, we leverage the automatic configuration module of nnU-Net to determine the parameters. These parameters will remain fixed during subsequent model evaluation. Through extensive experiments, we demonstrated that LungSSFNet consistently outperforms the state-of-the-art models by 1–2 https://github.com/zx0412/LungSSFNet .

Few-shot classification aims to recognize query samples from novel classes given scarce labeled data, which remains a challenging problem in machine learning. This paper proposes a Discriminative Feature Enhancement Network (DFENet) to distinguish the discriminative feature of a novel category with the following characteristics: (1) a Cross-Modal Guidance Module (CMG-Module) is proposed to enrich the query vectors via leveraging label information from the additional modality; (2) a Neural-Decoding based Attention Module (NDA-Module) further explores the relationship between the query and support samples, and the attention weights depend on contribution of the support to the reconstructed query. The main intuition of the NDAModule is to deepen our understanding of the self-attention mechanism from a neural decoding perspective, emphasizing that the best reconstruction can be used as a universal guideline; (3) a flexible triplet loss is designed to incorporate the semantic context among all classes and distinguish samples from different classes. The experimental results on two few-shot classification datasets show that our DFENet can learn a more discriminative representation for novel classes progressively. We also test the proposed method on image retrieval and facial expression recognition tasks, demonstrating consistent improvement.

Image-text matching aims to bridge vision and language so as to match the instance of one modality with the instance of another modality. Recent years have seen considerable progress in the research area by exploring local alignment between image regions and sentence words. However, there are still open questions regarding how to learn modality-invariant feature embeddings and effectively utilize hard negatives in the training set to infer more accurate matching scores. In this paper, we introduce a new approach called Image-Text Modality Contrastive Learning (abbreviated as ITContrast) for image-text matching. Our method addresses these challenges by leveraging a pre-trained vision-language model, OSCAR, which is firstly fine-tuned to obtain visual and textual features. We also introduce a hard negative synthesis module, which capitalizes on the difficulty of negative samples. This module profiles negative samples within a mini-match and generates representative embeddings that reflect their hardness in relation to the anchor sample. A novel cost function is designed to comprehensively integrate the information from positives, negatives and synthesized hard negatives. Extensive experiments on the MS COCO and Flickr30K datasets demonstrate that our approach is effective for image-text matching.

3-D human pose estimation or human tracking has always been the focus of research in the human–computer interaction community. As the calibration step of human pose estimation, subject-specific modeling is crucially important to the subsequent pose estimation process. It not only provides a priori knowledge but also clearly defines the tracking target. This article presents a fully automatic subject modeling framework to reconstruct human pose, shape, as well as the body texture in a challenging optimization scenario. By integrating powerful differentiable rendering into the subject-specific modeling pipeline, the proposed method transforms the texture reconstruction problem into analysis by synthesis minimization and solves it efficiently by a gradient-based method. Furthermore, a novel covariance matrix adaptation annealing algorithm is proposed to attack the high-dimensional multimodal optimization problem in an adaptive manner. The domain knowledge of hierarchical human anatomy is seamlessly injected to the annealing optimization process by using a soft covariance matrix mask. All together contributes to the novel algorithm robust to the temptation of local minima. Experiments on the Human3.6 M dataset and the People-Snapshot dataset demonstrate the competitive results to the state of the art both qualitatively and quantitatively.

Current state-of-the-art image-text matching methods implicitly align the visual-semantic fragments, like regions in images and words in sentences, and adopt cross-attention mechanism to discover fine-grained cross-modal semantic correspondence. However, the cross-attention mechanism may bring redundant or irrelevant region-word alignments, degenerating retrieval accuracy and limiting efficiency. Although many researchers have made progress in mining meaningful alignments and thus improving accuracy, the problem of poor efficiency remains unresolved. In this work, we propose to learn fine-grained image-text matching from the perspective of information coding. Specifically, we suggest a coding framework to explain the fragments aligning process, which provides a novel view to reexamine the cross-attention mechanism and analyze the problem of redundant alignments. Based on this framework, a Cross-modal Hard Aligning Network (CHAN) is designed, which comprehensively exploits the most relevant region-word pairs and eliminates all other alignments. Extensive experiments conducted on two public datasets, MS-COCO and Flickr30K, verify that the relevance of the most associated word-region pairs is discriminative enough as an indicator of the image-text similarity, with superior accuracy and efficiency over the state-of-the-art approaches on the bidirectional image and text retrieval tasks. Our code will be available at https://github.com/ppanzx/CHAN.

Classification of vehicle types using surveillance images is a challenging task in Intelligent Transportation Systems (ITS). In this paper, Convolutional Neural Networks for Vehicle types classification are comparatively studied. Firstly, GoogLeNet, ResNet50 and InceptionV4 are exploited as baselines for comparison. Secondly, we proposed a new network architecture based on GoogLeNet, ResNet50 and InceptionV4, named Fused Deep Convolutional Neural Networks (FDCNN), to take advantage of the ‘Inception’ module on parameter optimization and ‘Residual’ module on avoiding gradient vanishing, and applied the model to vehicle types classification. Thirdly, we created a vehicle dataset under the conditions of complicated and varied weather and lighting conditions, and conducted comparative experiments using the SEU vehicle dataset. Experimental results show much better performance of the proposed FDCNN with RMSprop optimizer on recognizing vehicle types. Specifically, the average classification accuracies of six vehicle types, such as truck, coach, sedan, minivan, pickup and SUV, are over 96.8%. Among the six classes of vehicle types, sedan is the most difficult to classify and the proposed FDCNN achieved over 93.81% accuracy in comparative experiments.

The results of a prior transcript profiling study identified a large difference in transcript accumulation of a pectin acetylesterase gene, designated as PAE8, in two genetically related lines of common bean (Phaseolus vulgaris). The results of reverse transcription-quantitative PCR experiments confirmed this difference and revealed that the gene is expressed specifically in the seed coat of developing seeds. Genomic sequence data identified a non-functional allele, due to a five-base pair insertion resulting in a frameshift and premature stop codon. The non-functional pae8 allele was associated with a lack of detectable protein as determined by Western blot. PAE8 accounted for approximately 65% of total pectin acetylesterase activity in the developing seed coat. Lack of functional PAE8 resulted in an approximately 2.5-fold increase in acetylation of soluble pectin in the mature seed coat. The presence of a non-functional pae8 allele was associated with an increased rate of water absorption by the seed and increased percentage of germination in aged seeds. The data suggest that the decreased acetylation of pectin leads to enhanced interaction with Ca2+, contributing to water impermeability.