Applying conventional LLM Post-Training Quantization (PTQ) to Vision-Language Models (VLMs) often leads to a characteristic “Blind but Fluent” failure mode: textual fluency is largely preserved while visual reasoning degrades. Prior work suggests that calibration is biased toward language preservation because language-token gradients dominate those of vision tokens, but it remains unclear which layers are most vulnerable to modality-specific collapse and why global modality re-weighting alone is insufficient.We introduce two layer-wise diagnostics, the Vision Vulnerability Score (VVS) and the Language Vulnerability Score (LVS), which quantify layer-wise modality vulnerability by measuring how much the final-layer hidden-state response to vision or language intervention is attenuated when a target layer is quantized. Building on these diagnostics, we propose VVSQ (Vision Vulnerability Score-Guided Quantization). VVSQ applies a vision-preserving Selective Auxiliary Stabilization objective with gradient-weighted token reconstruction only to layers vulnerable to visual collapse and optionally increases precision on layers vulnerable to both visual and language collapse. Unlike prior methods that re-weight modalities globally, VVSQ controls quantization loss locally according to each layer’s modality vulnerability.Across Qwen2-VL, InternVL2, and LLaVA under W2A16, W3A16, W4A4, and W4A8, and on the newer Qwen2.5-VL-3B/7B backbones under W3A16 and W4A8, VVSQ consistently improves over Modality-Balanced Quantization, a directly comparable multimodal baseline. On Qwen2.5-VL-3B, calibration-free RTN collapses completely in our setting, whereas calibration-aware and vulnerability-guided methods retain usable multimodal accuracy. The gains are largest in more aggressive regimes, reaching up to Avg at W3A16 (Qwen2-VL-2B over MBQ), Avg at W4A8 (LLaVA-7B over MBQ), MMMU at W2A16 (LLaVA-7B over MBQ), and MMMU at W4A4 (LLaVA-7B over MBQ); these are peak single-model gains, with mean improvements of Avg (W3A16) and Avg (W4A8) across the nine-model suite. These results suggest that controlling quantization loss according to each layer’s modality vulnerability is more effective than applying a single modality-balancing objective uniformly across all layers.

In this paper, we introduce a method to tackle Domain Generalized Semantic Segmentation (DGSS) by utilizing domain-invariant semantic knowledge from text embeddings of vision-language models. We employ the text embeddings as object queries within a transformer-based segmentation framework (textual object queries). These queries are regarded as a domain-invariant basis for pixel grouping in DGSS. To leverage the power of textual object queries, we introduce a novel framework named the textual query-driven mask transformer (tqdm). Our tqdm aims to (1) generate textual object queries that maximally encode domain-invariant semantics and (2) enhance the semantic clarity of dense visual features. Additionally, we suggest three regularization losses to improve the efficacy of tqdm by aligning between visual and textual features. By utilizing our method, the model can comprehend inherent semantic information for classes of interest, enabling it to generalize to extreme domains (e.g., sketch style). Our tqdm achieves 68.9 mIoU on GTA5 → Cityscapes, outperforming the prior state-of-the-art method by 2.5 mIoU. The project page is available at https://byeonghyunpak.github.io/tqdm .

Data gathering trials remain an important part of the development and testing of imaging sensor systems. However, the role of trials has evolved to reflect emergent technology, engineering methodologies, operational requirements, and project constraints such as schedule and cost. The changing nature of data gathering trials is reviewed. Although trial programmes are still used for product acceptance, there has been an increased demand for trial data to support the design process. In this paper, the emphasis is on air-to-ground image-based military systems where the timely availability of relevant image data is critical to the development of advanced image processing software which, in turn, underpins the performance of the latest imaging systems. Other factors which affect the nature of trial programmes are also considered. These include the widespread availability of synthetic image generators and the use of low-cost drones as either targets or sensor platforms. Furthermore, the increasing use of AI data processing techniques demands a larger and more diverse image data set for training and evaluation purposes. Against this background of changing requirements, trial planning has become increasingly important. Although the great flexibility of low-cost commercial drones has resulted in them becoming a preferred solution for camera platforms, they present unique challenges, ranging from logistics through to image truthing of target locations. These issues are discussed, and recommendations made based on experience gained through multiple trial programmes.

Deep Neural Networks (DNNs)-based semantic segmentation models trained on a source domain often struggle to generalize to unseen target domains, i.e., a domain gap problem. Texture often contributes to the domain gap, making DNNs vulnerable to domain shift because they are prone to be texture-biased. Existing Domain Generalized Semantic Segmentation (DGSS) methods have alleviated the domain gap problem by guiding models to prioritize shape over texture. On the other hand, shape and texture are two prominent and complementary cues in semantic segmentation. This paper argues that leveraging texture is crucial for improving performance in DGSS. Specifically, we propose a novel framework, coined Texture Learning Domain Randomization (TLDR). TLDR includes two novel losses to effectively enhance texture learning in DGSS: (1) a texture regularization loss to prevent overfitting to source domain textures by using texture features from an ImageNet pre-trained model and (2) a texture generalization loss that utilizes random style images to learn diverse texture representations in a self-supervised manner. Extensive experimental results demonstrate the superiority of the proposed TLDR; e.g., TLDR achieves 46.5 mIoU on GTA-to-Cityscapes using ResNet-50, which improves the prior state-of-the-art method by 1.9 mIoU. The source code is available at https://github.com/ssssshwan/TLDR.

Since hyperspectral images (HSIs) contain visual information of multiple wavelengths, invisible signals to human eyes can also be detected. Therefore, it can be widely used for target object detection in bad weather and disaster environments. However, the channel dimension of the HSI is very large, and thus it is very inefficient to apply the existing object detector naively. In this letter, we present a lightweight convolutional neural network (CNN)-based channel sampler to estimate the importance score of each channel in the HSI. Based on the importance score of each channel, we can generate single-channel images that achieve the best object detection performance, as well as analyze the impact of the wavelength in the HSI on object detection performance. The proposed sampler is trained by a self-supervised adversarial learning method that recovers the original input HSI from the generated single-channel image. Therefore, our channel sampler can be seamlessly combined with any existing detectors. For experiments, we build a hyperspectral dataset for vehicle detection and then show the effectiveness of our method through various ablation studies.

Recently, infrared object detection (IOD) has been extensively studied due to the rapid growth of deep neural networks (DNNs). An adversarial attack using imperceptible perturbation can dramatically deteriorate the performance of DNNs. Most of the existing adversarial attacks are focused on visible image recognition (VIR), but there are few attacks for IOD. Moreover, the existing attacks are challenging to exploit for state-of-the-art detectors (e.g., EfficientDet) due to low compatibility. To solve the problem, we propose a novel upcycling adversarial attack for IOD by expanding the highly compatible adversarial attacks for the VIR task. We also propose a novel evaluation metric, attack efficiency (AE), to compare the effectiveness of different adversarial attacks. Since the AE value increases with the small perturbation size and the significant performance drop, we can concurrently compare the similarity and performance degradation between adversarial and clean images for various attacks. We validate our approaches through comprehensive experiments on two challenging datasets (e.g., FLIR and MSOD) for the infrared domain.

We indirectly predict a class by deriving user-defined (i.e., existing) attributes (UA) from an image in generalized zero-shot learning (GZSL). High-quality attributes are essential for GZSL, but the existing UAs are sometimes not discriminative. We observe that the hidden units at each layer in a convolutional neural network (CNN) contain highly discriminative semantic information across a range of objects, parts, scenes, textures, materials, and color. The semantic information in CNN features is similar to the attributes that can distinguish each class. Motivated by this observation, we employ CNN features like novel class representative semantic data, i.e., deep attribute (DA). Precisely, we propose three objective functions (e.g., compatible, discriminative, and intra-independent) to inject the fundamental properties into the generated DA. We substantially outperform the state-of-the-art approaches on four challenging GZSL datasets, including CUB, FLO, AWA1, and SUN. Furthermore, the existing UA and our proposed DA are complementary and can be combined to enhance performance further.

Fine-grained image recognition is a highly challenging problem due to subtle differences between images. There are many attempts to solve fine-grained image recognition problems using data augmentation, jointly optimizing deep metric learning. CutMix is one of the excellent data augmentation strategies which crops and merges to generate new images. However, it sometimes generates meaningless and obscured object images that degrade recognition performance. We propose a novel framework that solves the above problem and expands the CutMix leveraging localizing method. Also, we improve the recognition accuracy to joint optimizing with a pairwise margin loss using generated images from the improved CutMix. There are some images similar to the reference image among the generated images. They are generated by replacing similar parts from the reference image. Those generated images should not be located much farther than the margin value in embedding space because those generated images and a reference image have similar semantic meaning. However, the conventional margin loss can not consider those images which are located much farther than the margin. To solve this problem, we propose an additional margin loss to consider those generated images. The proposed framework consists of two stages: the part localization-aware CutMix and an adaptive pairwise margin loss. The proposed method achieves state-of-the-art performance on the CUB-200-2011, FGVC-Aircraft, Stanford Cars, and DeepFashion datasets. Furthermore, extensive experiments demonstrate that each stage improves the final performance.

Unsupervised domain adaptation often gives impressive solutions to handle domain shift of data. Most of current approaches assume that unlabeled target data to train is abundant. This assumption is not always true in practices. To tackle this issue, we propose a general solution to solve the domain gap minimization problem without any target data. Our method consists of two regularization steps. The first step is a pixel regularization by arbitrary style transfer. Recently, some methods bring style transfer algorithms to domain adaptation and domain generalization process. They use style transfer algorithms to remove texture bias in source domain data. We also use style transfer algorithms for removing texture bias, but our method depends on neither domain adaptation nor domain generalization paradigm. The second regularization step is a feature regularization by feature alignment. Adding a feature alignment loss term to the model loss, the model learns domain invariant representation more efficiently. We evaluate our regularization methods from several experiments both on small dataset and large dataset. From the experiments, we show that our model can learn domain invariant representation as much as unsupervised domain adaptation methods.

Recently, infrared object detection(IOD) has been extensively studied due to the rapid growth of deep neural networks(DNN). Adversarial attacks using imperceptible perturbation can dramatically deteriorate the performance of DNN. However, most adversarial attack works are focused on visible image recognition(VIR), and there are few methods for IOD. We propose deep learning-based adversarial attacks for IOD by expanding several state-of-the-art adversarial attacks for VIR. We effectively validate our claim through comprehensive experiments on two challenging IOD datasets, including FLIR and MSOD.

Zero-shot learning and self-supervised learning have been widely studied due to the advantage of performing representation learning in a data shortage situation efficiently. However, few studies consider zero-shot learning using semantic embeddings (e.g., CNN features or attributes) and self-supervision simultaneously. The reason is that most zero-shot learning works employ vector-level semantic embeddings. However, most self-supervision studies only consider image-level domains, so a novel self-supervision method for vector-level CNN features is needed. We propose a simple way to shuffle semantic embeddings. Furthermore, we propose a method to enrich feature representation and improve zero-shot learning performance effectively. We show that our model outperforms current state-of-the-art methods on the large-scale ImageNet 21K and the small-scale CUB and SUN datasets.

Zero-Shot Action Recognition (ZSAR) aims to recognize unseen action classes not included in the training dataset. Existing generative methods for ZSAR synthesize a feature of unseen action from a class embedding to overcome the absence of training data. Specifically, previous methods synthesize a feature which is averaged along a time axis, even though a video is extracted as a sequence of feature vectors. They suffer from the ambiguity of temporal information, which leads to confusion among actions sharing similar subactions. To tackle the problem, we first propose to synthesize not an averaged feature but a sequence consisting of feature vectors along the time axis. Hence, we design Sequence Feature Generative Adversarial Network (SFGAN) with Temporal Unrolling NEtwork (TUNE), which unrolls a class embedding into a set of condition vectors for generating sequences of features. Also, we employ a sequence discriminator as the second teacher. Through extensive experiments on the three benchmarks, HMDB51, UCF101, and Olympic, we validate the efficacy of sequence generation for ZSAR, and our method achieves the state-of-the-art generalized zero-shot learning performances.

In generalized zero-shot learning (GZSL), it is imperative to solve the bias problem due to extreme data imbalance between seen and unseen classes, i.e., unseen classes are misclassified as seen classes. We alleviate the bias problem by generating synthetic images of unseen classes. The most challenging part is that existing GAN methods are only focused on producing authentic seen images, so realistic unseen images cannot be generated. Specifically, we propose a novel zero-shot generative adversarial network (ZSGAN) which learns the relationship between images and attributes shared by seen and unseen classes. Unlike existing works that generate synthetic features of unseen classes, we can generate more generalizable realistic unseen images. For instance, generated unseen images can be used for zero-shot detection, segmentation, and image translation since images have spatial information. We also propose domain-free networks (DFN) that can effectively distinguish seen and unseen domains for input images. We evaluate our approaches on three challenging GZSL datasets, including CUB, FLO, and AWA2. We outperform the state-of-the-art methods and also empirically verify that our proposed method is a network-agnostic approach, i.e., the generated unseen images can improve performance regardless of the neural network type.

Predicting thermal comfort is one of the primary building research domains due to its technical and environmental significance. A metabolic rate, one of the significant variables for predicting an individual's thermal comfort, is primarily based on the human body's activity level. While other human and environmental factors, such as air temperature and relative humidity are easily measured and collected, with the help of sensory devices, a metabolic rate varies with time, and is not easy to measure to determine an accurate thermal comfort estimation in reality. Therefore, this study investigated the potential use of Deep Learning algorithm to accurately estimate the metabolic rate for a better thermal comfort estimation. A series of chamber tests were conducted with 23 test participants. The Kinect sensor was adopted to detect a user's physical motion, by capturing the motion images. With the help of a wearable sensor, a user's heart rate was also measured to estimate a metabolic rate. This study found that males showed higher MET than females, and the high BMI group generated higher MET than the low BMI group. The result also indicated that an estimated accurate range of 77%-89% was reasonably acceptable in the self-MET prediction modeling, while it was 65% in the third-party MET prediction. Therefore, the outcome of this research confirms that it is possible to use the Kinect sensor as a remote sensing device to estimate a user's metabolic rate, based on the use of a Deep Learning algorithm developed per individual.

Most of the conventional state-of-the-art methods for video analysis achieve outstanding performance by combining two or more different inputs, e.g. an RGB image, a motion image, or an audio signal, in a two-stream manner. Although these approaches generate pronounced performance, it underlines that each considered feature is tantamount in the classification of the video. This dilutes the nature of each class that every class depends on the different levels of information from different features. To incorporate the nature of each class, we present the class nature specific fusion that combines the features with a different level of weights for the optimal class result. In this work, we first represent each frame-level video feature as a spectral image to train convolutional neural networks (CNNs) on the RGB and audio features. We then revise the conventional two-stream fusion method to form a class nature specific one by combining features in different weight for different classes. We evaluate our method on the Comprehensive Video Understanding in the Wild dataset to understand how each class reacted on each feature in wild videos. Our experimental results not only show the advantage over conventional two-stream fusion, but also illustrate the correlation of two features: RGB and audio signal for each class.

Highlight detection from videos has been widely studied due to the fast growth of video contents. However, most existing approaches to highlight detection, either handcraft feature based or deep learning based, heavily rely on human-curated training data, which is very expensive to obtain and, thus, hinders the scalability to large datasets and unlabeled video categories. We observe that the largely available Web images can be applied as a weak supervision for highlight detection. For example, the top-ranked images in reference to the query “skiing” returned by a search engine may contain considerable positive samples of “skiing” highlights. Motivated by this observation, we propose a novel triplet deep ranking approach to video highlight detection using Web images as a weak supervision. The approach handles the relative preference of highlight scores between highlighting frames, nonhighlighting frames, and Web images by the triplet ranking constraints. Our approach can iteratively train two interdependent deep models (i.e., a triplet highlight model and a pairwise noise model) to deal with the noisy Web images in a single framework. We train the two models with relative preferences to generalize the capability regardless of the categories of training data. Therefore, our approach is fully category independent and exploits weakly supervised Web images. We evaluate our approach on two challenging datasets and achieve impressive results compared with the state-of-the-art pairwise ranking support vector machines, a robust recurrent autoencoder, and spatial deep convolution neural networks. We also empirically verify through cross-dataset evaluation that our category-independent model is fairly generalizable even if two different datasets do not share exactly the same categories.

A new non-intrusive ac voltage measurement technique based on stray electric filed energy harvesting is presented whereby the voltage of ac power lines can be measured without removing insulation and therefore without direct contact with conductors of the power lines. The energy of electric field around the power lines is picked up and stored in a capacitor by a wire wound around insulated 3-wire household power lines or by a conductor plate placed in the vicinity of overhead high-voltage power lines. Autonomous switches are used to monitor the stored electric field energy in a storage capacitor and to generate pulse train. Experimental results show that the repetition rate of the output pulse is a linear function of the line voltage. Since the voltage of insulated 3-wire household power line can be measured without removing insulation and the voltage of high-voltage power transmission line can be measured without any contact to the conductor of the power line, the proposed technique can be widely applied to smart grid system.

For the first time, under the actual three-phase 765 kV power transmission lines, electric field energy is harvested and Zigbee-based sensor node powered by the harvested energy is operated autonomously. A conductor tube is placed between power lines and the earth surface for electric field energy harvesting. The voltage difference between conductor tube and the earth surface is rectified and the harvested electric energy is stored in a storage capacitor. A self-sustaining power management circuit adopting low-power comparator with hysteresis is devised to monitor the stored energy and to operate the sensor node autonomously according to the stored energy. Experimental results under the actual three-phase 765 kV power lines show that the data transfer period is almost inversely proportional to the square of the height of the sensor module from the earth surface. The temperature data is transmitted in every 25 s when the conductor tube is located at 4 m from the earth surface. Since the presented non-contact energy harvesting apparatus is very simple, robust and low-cost, it is believed that the presented technology can be widely applied to power transmission, distribution and smart grid system.