In this work we introduce a novel application of Large Language Models (LLMs) towards the classification of malicious Windows Portable Executable (WinPE) files. Rather than fine-tuning an LLM based on the raw binary of the samples serving as the corpus, in this case we leverage our in depth knowledge of the underlying structure of the WinPE format to apply a two phased approach to translate each file into human readable format. For our studies we focus on the header portion of the files in order to reduce the required time for the fine-tuning process, performing Parameter Efficient Fine Tuning (PEFT) applying a QLoRA adapter to a light-weight baseline model. In addition, we deploy our fine-tuned model according to Software Defined Networking (SDN) principles. The model is containerized, with constraints applied in terms of the amount of CPU, memory, and networking resources that are available. In addition, the model is deployed as a virtualized LLM (vLLM) adhering to a server-client architecture. We benchmark the fine-tuned model against the SoReL-20M dataset and compare results in terms of both accuracy, including the macro-averaged F1 score, and latency, including the Time-to-First-Token (TTFT). By fine-tuning the model to understand the language of the Windows Portable Executable format, we demonstrate a significant improvement in macro-averaged F1 score, up to an absolute difference of 50 percentage points over generic LLMs, while maintaining comparable latency.

Malicious software presents significant risks to computer systems, networks, and sensitive data, making malware detection a critical cybersecurity challenge. Labeling malware data not only requires expert knowledge but is also time-intensive. Given the constantly evolving threat landscape, an ever increasing amount of malware remains unlabeled, and those samples that are labeled may be incomplete or inconsistent. In this work, we introduce a self-supervised method using contrastive learning to perform static analysis and classification of malicious Portable Executable (PE) files, reducing the dependency on labeled data. We also develop data augmentation techniques that generate multiple augmented views and design PE-specific augmentation operators to be used during self-supervised learning such as shuffle, encryption, and compression based on the IMAGE_SECTION_HEADER. Our method is built upon raw PE byte sequences extracted from a large-scale publicly available dataset, SoRel-20M, which contains 20 million PE samples. Utilizing a two-stage framework consisting of a self-supervised contrastive learning pre-training phase followed by a supervised fine-tuning phase with limited amounts of labeled data, our model learns label-free invariant representations of the PE structure and as a result outperforms a traditional supervised Convolutional Neural Network (CNN), achieving a macro-averaged F1 score of 78.6% with only 10% of the labeled data. In addition, our method only requires the first 1 KB of header data, whereas the supervised baseline requires 1 MB of the underlying PE header.

Graph Convolutional Networks (GCNs) have become a cornerstone in skeleton-based action recognition due to their ability to model non-Euclidean body structures. However, their capacity to construct expressive and structured representations is fundamentally constrained by the tight coupling of static graph topology represented by adjacency matrix. To address this, we introduce a novel architectural principle called Hierarchical Kernel Decoupling (HKD), which disentangles spatial aggregation semantics from fixed topological constraints, thereby enabling more structured and flexible representation learning. Our approach introduces two progressive modules. The first module initially decouples the adjacency matrix using a K-hop neighborhood partitioning strategy to build a multi-level convolutional neighborhood, allowing the model to learn from both local and distant joints in a structured and scalable manner. This partitioning strategy expands the receptive field while maintaining topological coherence, enabling diverse feature extraction. The second module further decouples the multiple joints within same partition by adaptively adjusting the contribution of each node within its partition using a context-aware partition enhancer. Together, these modules implement a hierarchical decoupling mechanism that transforms fixed, topology-coupled kernels into adaptive, structure-aware representations suited for skeleton-based action recognition. Integrated with a multi-branch temporal modeling design, the proposed network achieves state-of-the-art performance on three large-scale benchmarks: NTU RGB+D, NTU RGB+D 120, and NW-UCLA. Extensive experiments demonstrate that HKD-Graph Convolutional Network (HKD-GCN) not only improves accuracy but also perceives complex human actions with computational efficiency.

Acoustic sensors have been deployed to monitor the health of infrastructure such as wind turbine blades. Such sensors typically use wireless links to transmit sensor data; however, due to harsh, time-varying environmental conditions, these links are susceptible to interference and signal attenuation, leading to data loss. This study investigates how antenna configurations and sensor orientations can be used to address these problems. Three experimental phases were conducted in this study: (1) discrete rotation tests evaluating network performance in fixed rotational increments, (2) continuous rotation tests simulating real-world turbine operating conditions, and (3) validation using data collected from our sensing system deployed on an operational turbine. Performance metrics include received signal strength indicator (RSSI), round-trip time (RTT), and throughput. Results demonstrate that a dual-antenna sensor installed on the shear web of a turbine blade can provide reliable network performance. Using evaluations both in the laboratory and on an operational wind turbine, this work is the first to provide invaluable insights into attenuation effects from blade material and environmental interactions, aiming to optimize network performance for wireless sensors in real-world structural health monitoring.

This study presents a multimodal sensing and deep learning framework to enhance monitoring shrimp health and behavior. By integrating real-time water quality sensors, acoustic monitoring, and computer vision, we track key parameters closely tied to feed consumption in healthy and diseased conditions in shrimp. Over a 5-week growth period, we analyzed shrimp feeding behavior through acoustic signals, and performed a controlled disease challenge to identify unique patterns associated with healthy and diseased condition in shrimp. The resulting data can help offer actionable insights for farm operators to develop cost effective feeding strategies and reduce cost associated with aquafeed use for both commercial applications and small individual farm owners.

Image super-resolution is a classic computer vision problem with numerous practical applications on mobile and IoT devices. This creates a need for solutions that are not only performant but are additionally compatible with real mobile AI hardware such as neural processing units (NPUs). In this Mobile AI challenge, we address this problem and propose the participants to design efficient quantized deep learning super-resolution models that can demonstrate a real-time performance on mobile NPUs. For this, the participants were provided with the DIV2K dataset and trained quantized models to do an efficient $3 X$ image upscaling. The runtime of all models was evaluated on the Google Tensor NPU present in all recent Google Pixel smartphones. The proposed solutions are fully compatible with all major mobile AI accelerators and are capable of reconstructing Full HD images under 50 ms, delivering highfidelity results. A comprehensive description of the models developed in the challenge is provided in this paper.

In autonomous driving, the ability to detect pedestrians accurately is crucial for safety. Some detectors, however, often struggle with occlusions, where pedestrians partially hidden behind objects appear incomplete and are harder to be identified accurately. To alleviate this issue, we introduce Clear3D, which make previous partially unseen pedestrians visible by effectively removing occluders to enhance 3D pedestrian detection. The first essential step in Clear3D is to accurately identify occluded regions. This is achieved by calculating the intensity variation along the LiDAR ray direction, where regions with small changes are classified as occluded. After identifying the occluded areas, they are transformed as masks and fed into the inpainting module, reconstructing the originally unseen parts. Above process, encompassing both the perception and reconstruction of occluded regions, is termed occluder removal. Finally, by fusing the recovered occlusion-free image features with LiDAR features, Clear3D generates more complete representation, enhancing the performance of downstream 3D classification and localization tasks. Extensive evaluations show that Clear3D achieves state-of-the-art accuracy across various challenging settings on KITTI, particularly in heavy occlusion environments.

The current mainstream and state-of-the-art anomaly detection (AD) methods are substantially established on pretrained feature networks yielded by ImageNet pre- training. However, regardless of supervised or self-supervised pretraining, the pretraining process on ImageNet does not match the goal of anomaly detection (i.e., pretraining in natural images doesn’t aim to distinguish between normal and abnormal). Moreover, natural images and industrial image data in AD scenarios typically have the distribution shift. The two issues can cause ImageNet-pretrained features to be suboptimal for AD tasks. To further promote the development of the AD field, pretrained representations specially for AD tasks are eager and very valuable. To this end, we propose a novel AD representation learning framework specially designed for learning robust and discriminative pretrained representa- tions for industrial anomaly detection. Specifically, closely surrounding the goal of anomaly detection (i.e., focus on discrepancies between normals and anoma- lies), we propose angle- and norm-oriented contrastive losses to maximize the angle size and norm difference between normal and abnormal features simulta- neously. To avoid the distribution shift from natural images to AD images, our pretraining is performed on a large-scale AD dataset, RealIAD. To further alle- viate the potential shift between pretraining data and downstream AD datasets, we learn the pretrained AD representations based on the class-generalizable repre- sentation, residual features. For evaluation, based on five embedding-based AD methods, we simply replace their original features with our pretrained represen- tations. Extensive experiments on five AD datasets and five backbones consis- tently show the superiority of our pretrained features. The code is available at https://github.com/xcyao00/ADPretrain.

The task of instance segmentation in remote sensing images, aiming at performing per-pixel labeling of objects at the instance level, is of great importance for various civil applications. Despite previous successes, most existing instance segmentation methods designed for natural images encounter sharp performance degradations when they are directly applied to top-view remote sensing images. Through careful analysis, we observe that the challenges mainly come from the lack of discriminative object features due to severe scale variations, low contrasts, and clustered distributions. In order to address these problems, a novel context aggregation network (CATNet) is proposed to improve the feature extraction process. The proposed model exploits three lightweight plug-and-play modules, namely, dense feature pyramid network (DenseFPN), spatial context pyramid (SCP), and hierarchical region of interest extractor (HRoIE), to aggregate global visual context at feature, spatial, and instance domains, respectively. DenseFPN is a multi-scale feature propagation module that establishes more flexible information flows by adopting interlevel residual connections, cross-level dense connections, and feature reweighting strategy. Leveraging the attention mechanism, SCP further augments the features by aggregating global spatial context into local regions. For each instance, HRoIE adaptively generates RoI features for different downstream tasks. Extensive evaluations of the proposed scheme on iSAID, DIOR, NWPU VHR-10, and HRSID datasets demonstrate that the proposed approach outperforms state-of-the-arts under similar computational costs. Source code and pretrained models are available at https://github.com/yeliudev/CATNet.

One of the most challenges for anomaly detection (AD) is how to design one unified AD model, where the model is trained with normal instances from multiple classes with the objective to detect anomalies in these classes. For such a challenging task, popular normalizing flow (NF) based AD methods may fall into a ”homogeneous mapping” issue, where the NF-based AD models are biased to generate large log-likelihoods for both normal and abnormal samples, and thereby lead to a high missing rate of anomalies. In this paper, we propose a novel model with stronger unified AD performance, HGAD (Hierarchical Gaussian mixture). The proposed HGAD performs much better for multi-class anomaly detection by three key improvements. First, we propose to model NF-based AD networks with inter-class Gaussian mixture prior for more effectively capturing the complex multiclass distribution. Second, we propose a mutual information maximization loss to introduce the class repulsion property to the model for better structuring the latent feature space, where the class centers are repulsed from each other. In this way, different class centers are more distinguishable and more conducive to avoid the bias issue. Third, we introduce an intra-class mixed class centers learning strategy that can prompt the model to learn diverse normal patterns even within one class. Together with the inter-class Gaussian mixture modeling, we form a hierarchical Gaussian mixture normalizing flows modeling method to accomplish the multiclass AD task. We evaluate our method on four real-world AD benchmarks, where we can significantly improve the previous NF-based AD methods and also outperform the SOTA unified AD methods. Code will be available online.

Graph convolutional networks have achieved great success in skeleton-based action recognition area, in which topology learning is the key component for extracting representative features. In this paper, we propose a novel module called Motion-Aware Topology Graph Convolution (MAT-GC) that can boost the graph modeling ability with motion-aware features and time-wise feature aggregation for skeleton-based action recognition. In particular, feature-level temporal differences of joints from adjacent frames are sampled and combined to form enhanced motion representations for a given frame. Moreover, a two-pathway structure is adopted to model pairwise correlations at each time step taking both short-term and long-term temporal motion variations into account. Eventually, joint features are aggregated following the time-wise topologies. Combined with temporal modeling modules, the overall graph convolutional network MAT-GCN is constructed. Experimental results on three popular skeleton-based action recognition datasets verify the effectiveness of the proposed method.

Malware is one of the leading cybersecurity challenges, as it disrupts the normal use of devices, causes financial losses, and steals user information. Deep learning-based methods have been increasingly used in the malware analysis field recently. In this work, we propose a novel multi-input Transformer-based approach for detecting malicious Portable Executable (PE) files. The PE raw bytes were partitioned into different byte sequences as multiple inputs in our proposed multi-input framework. This framework is comprised of convolutional neural networks (CNNs) and Transformer networks and is capable of independent learning of each input, thereby enabling a more expressive representation of the data. As a result, it is possible to capture both local spatial and time-series features, resulting in greater data comprehension. Our proposed approach outperforms the two reference methods, a LightGBM and a CNN-based model, as indicated by four metrics: accuracy, recall, precision, and F1 score.

Accurate pedestrian classification and localization has garnered significant attention due to their extensive applications in various multimedia applications such as security monitoring, autonomous driving, and more. We have observed that the commonly employed Intersection over Union (IoU) metric in many pedestrian detectors is susceptible to an inconsistent GT-Proposal assignment issue. This issue arises when spatially adjacent proposals, which have highly similar features, are assigned to distinct ground-truth boxes, leading to confusion during the training process and an increased number of false positives during inference. To address this challenge, our work presents a novel algorithm named D irectional A ssignment S trategy ( DAS ). Firstly, in conjunction with depth distribution, our approach transforms the assignment metric from a two-dimensional (2D) view into a three-dimensional (3D) space, enabling the optimization of the regression head under the constraint of depth direction. Secondly, in contrast to the conventional IoU-based one-to-one assignment of one proposal to one ground-truth box, our method aims to establish a more reasoned matching between sets of proposals and ground-truth boxes. By doing so, the detector is less reliant on the setting of a specific threshold. Leveraging this strategy as a plug-in module within state-of-the-art pedestrian detectors, we demonstrate a notable improvement in performance.

Unified (multi-class and cross-class) anomaly detection (AD) is a growing area of interest in real-world applications. However, the popular reconstruction-based AD approach usually faces two significant challenges: the "identical shortcut" issue (copying the input as output) and the lack of class adaptability (the AD model cannot be directly applied to new classes). To address these challenges, we propose a novel unified AD method, named IRAD (Input-Reference Joint Driven). Our core insight is to effectively incorporate both input and references into the reconstruction process. Our IRAD consists of three components: 1) A Suspicious Anomaly Substituting module that replaces the potential abnormal regions of input with anomaly-free reference patches to prevent abnormal information leakage, effectively addressing the "identical shortcut". 2) An Input-Reference Fusing module that merges reference embeddings with input, which urges the subsequent Decoder to effectively utilize the normal reference patterns to reconstruct anomaly-free samples, making our model more class-adaptive. 3) A Rich Feature Preserving Decoder that efficiently preserves low-level details, mitigating low-level information degradation during reverse construction from high to low level. In multi-class AD, IRAD achieves better results on Mvtec-AD, BTAD, and VisA. In cross-class AD, IRAD also outperforms the baesline methods on Mvtec-AD and VisA.

Open-Set Object Detection (OSOD) aims to train detectors on closed-set datasets to detect known objects and identify unknown objects in open-set conditions. Traditional discriminative classifier-based OSOD methods struggle to accurately learn the decision boundary between known and unknown classes, often resulting in the misclassification of unknown samples. In this work, we aim to combine generative representation with discriminative classification to alleviate the issue of misclassification by transforming known-unknown recognition into a binary classification problem. The proposed two-stage OSOD approach proceeds as follows: during the generative representation stage, we employ Class-Conditioned Normalizing Flow (CCNF) to establish distribution mapping for each known category; In the discriminative classification stage, by utilizing a small number of unknown class samples, semi-push-pull supervised learning and entropy contrast learning are used to separate known and unknown classes. Extensive experiments demonstrate that our method significantly enhances OSOD performance, evidenced by a 25.8%-28.6% reduction in the Wilderness Index and a decrease of 4391-8870 units in Absolute Open-Set Errors on the test set VOC-COCO-T1.

LiDAR has become one of the primary 3D object detection sensors in autonomous driving. However, due to the inherent sparsity of point clouds, certain objects exhibit structure incompleteness in occluded and distant areas, which hampers the accurate perception of objects in 3D space. To tackle this challenge, we propose Semantic and Structure Completion Network (S $^{2}$ CNet) for 3D object detection. Concretely, we design the Semantic Completion (SeC) module to generate semantic features in Bird’s-Eye-View (BEV) space, utilizing a teacher-student paradigm. Notably, we adopt a coarse-to-fine guidance strategy to encourage student network to generate semantic features specifically within foreground regions. This ensures that the student network focuses on the generation of foreground object features. Besides, we introduce an attention-based module to adaptively fuse the generated features and raw features. SeC module faces particular limitation when dealing with objects containing only a few points, in such case, the network is prone to generating low quality proposals with inaccurate localization. Complementary to SeC module, we introduce the Structure Completion (StC) module, in which a group of structural proposals are obtained by traversing most structures in a structure-guided manner, and thus at least one proposal with ground truth similar structure can be guaranteed. Extensive experiments on the KITTI and nuScenes benchmarks demonstrate the effectiveness of our method, especially for the hard setting objects with fewer points.

Recently self-attention mechanisms have become increasingly popular for computer vision applications following the success of transformer in natural language processing. Yet, transformer remains under-appreciated compared to the dominant role of convolutional neural networks in the field of computer vision. In this study, we present various approaches for transformers and their application to human pose estimation. We propose a novel model (TransNet) using a convolutional neural network design with a parallel transformer encoder branch to capture the long-range spatial dependency simultaneously while fusing it with the local features extracted from the input images. Experiments results show that TransNet achieves the exceptional performance for human pose estimation on the COCO dataset. Our proposed model outperforms the competitors and achieves the Average Precision (AP) score of 78.3 on COCO val set. Specifically, there is a significant improvement in the average score between the proposed model and the advanced convolutional neural networks. We believe this research can contribute to a better understanding of transformers within computer vision models.

Humans recognize anomalies through two aspects: larger patch-wise representation discrepancies and weaker patch-to-normal-patch correlations. However, the previous AD methods didn't sufficiently combine the two complementary aspects to design AD models. To this end, we find that Transformer can ideally satisfy the two aspects as its great power in the unified modeling of patchwise representations and patch-to-patch correlations. In this paper, we propose a novel AD framework: FOcus-the- Discrepancy (FOD), which can simultaneously spot the patch-wise, intra- and inter-discrepancies of anomalies. The major characteristic of our method is that we renovate the self attention maps in transformers to Intra-Inter-Correlation (I2Correlation). The I2Correlation contains a two-branch structure to first explicitly establish intraand inter-image correlations, and then fuses the features of two-branch to spotlight the abnormal patterns. To learn the intra- and inter-correlations adaptively, we propose the RBF-kernel-based target-correlations as learning targets for self-supervised learning. Besides, we introduce an entropy constraint strategy to solve the mode collapse issue in optimization and further amplify the normal abnormal distinguishability. Extensive experiments on three unsupervised real-world AD benchmarks show the superior performance of our approach. Code will be available at https://github.com/xcyao00/FOD.

3D object detection from point clouds is one of the key components in autonomous driving. Current two-stage detectors generate a small number of proposals, and then refine them in the second RCNN procedure. However, due to the inherent sparsity of point clouds, the first stage may predict some low quality proposals with incomplete structure and inaccurate localization. These low quality proposals fail to obtain adequate and precise proposal features which are essential for the following refinement, inevitably degrading the overall detection performance. To alleviate this problem, we propose Structure guided Proposal Completion (SPC) for 3D object detection from point clouds. Specifically, two completion strategies are developed to obtain high quality proposals: one is Structure Completion, in which a group of structural proposals are obtained by traversing most structures, and thus at least one proposal with ground truth similar structure can be guaranteed. The other is RoI Feature Completion, which is used to fill the empty area of proposals with virtual points under structure-aware manner. With the proposed SPC, high quality proposals with clearer structure and more precise localization can be obtained, and further promote the RCNN to perceive adequate proposal features. Extensive experiments on KITTI benchmark demonstrate the effectiveness of our proposed method, especially for hard setting objects with fewer LiDAR points.

Three-dimensional digital image correlation (3D-DIC) has shown to be a powerful tool to extract full-field displacement and deformations of structures using a series of synchronized stereo images. Before performing 3D-DIC measurements, the stereovision system must be calibrated to determine the relative position of the two cameras. Traditionally, this has been done by taking pictures of a calibration target whose dimensions are well known. Because the calibration target must be as large as the inspected object to properly calibrate the entire measurement volume (i.e., large-area calibration), for large fields of view, the calibration procedure becomes a challenge. This research aims at reducing the complexity of large-area calibration by measuring the extrinsic parameters of the stereovision system using a suite of sensors. In particular, three inertial measurement units and a laser distance meter are used to measure the cameras’ relative position and orientation in space. In this chapter, the performance of the proposed sensor-based extrinsic calibration is compared with the traditional image-based calibration method. Laboratory tests show that the extrinsic parameters computed with the sensor-based method can be used for performing a 3D-DIC analysis that yields an error below 5