Diffusion models demonstrate outstanding performance in image generation, but their iterative sampling processes incur substantial computational costs and error accumulation, hindering deployment on resource-constrained IoT devices. In this paper, we propose PQCAD-DM, a hybrid compression framework that tightly integrates Progressive Quantization (PQ) and Calibration-Assisted Distillation (CAD). PQ employs a two-stage strategy with momentum-guided adaptive bit-width transitions to suppress instability, while CAD leverages a dual-calibration dataset to reconstruct robust full-precision guidance for low-bit students. Aligned with an industrial server-to-device deployment paradigm, PQCAD-DM optimizes models on high-resource servers for efficient edge inference. Extensive experiments, including validation on aerial and urban surveillance IoT datasets, demonstrate that PQCAD-DM consistently outperforms fixed-bit quantization baselines and architecture-optimized hybrid models. On a mobile smartphone, our framework achieves a 2.8× higher throughput and 48% reduction in model size while maintaining high visual fidelity, making it uniquely suited for latency-sensitive IoT applications.
Style transfer aims to render a content image with the visual characteristics of a reference style while preserving its underlying semantic layout and structural geometry. While recent diffusion-based models demonstrate strong stylization capabilities by leveraging powerful generative priors and controllable internal representations, they typically assume a single global style. Extending them to multi-style scenarios often leads to boundary artifacts, unstable stylization, and structural inconsistency due to interference between multiple style representations. To overcome these limitations, we propose MAST (Mask-Guided Attention Mass Allocation for Training-Free Multi-Style Transfer), a novel training-free framework that explicitly controls content-style interactions within the diffusion attention mechanism. To achieve artifact-free and structure-preserving stylization, MAST integrates four connected modules. First, Layout-preserving Query Anchoring prevents global layout collapse by firmly anchoring the semantic structure using content queries. Second, Logit-level Attention Mass Allocation deterministically distributes attention probability mass across spatial regions, seamlessly fusing multiple styles without boundary artifacts. Third, Sharpness-aware Temperature Scaling restores the attention sharpness degraded by multi-style expansion. Finally, Discrepancy-aware Detail Injection adaptively compensates for localized high-frequency detail losses by measuring structural discrepancies. Extensive experiments demonstrate that MAST effectively mitigates boundary artifacts and maintains structural consistency, preserving texture fidelity and spatial coherence even as the number of applied styles increases.
Recent advances in artificial intelligence have expanded applications in the financial domain, particularly in fraud detection, a critical task for preventing losses for both customers and institutions. However, fraud detection is challenging due to severe class imbalance, which significantly degrades detection performance. Existing synthetic data generation methods for minority-class augmentation often fail to capture the heterogeneous structure and complex feature relationships of tabular data in the financial domain. To address these challenges, we propose MSTabVAE, a novel generative framework that extends TabNet, a deep learning architecture for tabular data, into a conditional variational autoencoder (CVAE) framework. MSTabVAE preserves TabNet’s step-wise feature-selection mechanism and introduces a multi-step latent mapping strategy to capture complex feature relationships in heterogeneous tabular data. Class-conditional information is further injected at every decision step to target minority-class synthesis. Experiments on four imbalanced financial datasets with two downstream classifiers show that augmenting training data with MSTabVAE-generated samples consistently improves classification performance over existing generative and traditional oversampling baselines, achieving up to a 42% relative improvement in Recall@1%FPR on the highly imbalanced BAF dataset. Additional experiments verify the fidelity of the generated data through inter-feature correlation and marginal distribution preservation, negligible privacy leakage as measured by the Distance to Closest Record metric, and up to 238× speedup in generation time over diffusion-based baselines. Finally, MSTabVAE provides structural interpretability for the generation process by revealing the features selected at each decision step.
Virtual try-on (VITON) aims to generate realistic images of a person wearing a target garment, requiring precise garment alignment in try-on regions and faithful preservation of identity and background in non-try-on regions. While latent diffusion models (LDMs) have advanced alignment and detail synthesis, preserving non-try-on regions remains challenging. A common post-hoc strategy directly replaces these regions with original content, but abrupt transitions often produce boundary artifacts. To overcome this, we reformulate VITON as a linear inverse problem and adopt trajectory-aligned solvers that progressively enforce measurement consistency, reducing abrupt changes in non-try-on regions. However, existing solvers still suffer from semantic drift during generation, leading to artifacts. We propose ART-VITON, a measurement-guided diffusion framework that ensures measurement adherence while maintaining artifact-free synthesis. Our method integrates residual prior-based initialization to mitigate training-inference mismatch and artifact-free measurement-guided sampling that combines data consistency, frequency-level correction, and periodic standard denoising. Experiments on VITON-HD, DressCode, and SHHQ-1.0 demonstrate that ART-VITON effectively preserves identity and background, eliminates boundary artifacts, and consistently improves visual fidelity and robustness over state-of-the-art baselines.
Facial expression recognition in the wild (FER-W) entails classifying facial emotions in natural environments. The major challenges in FER-W stem from the complexity and ambiguity of facial images, making it difficult to curate a large-scale labeled dataset for training. Additionally, the subtle differences in emotions often reside in the fine-grained details of local facial landmarks, demanding innovative solutions to capture these crucial features efficiently. To address these issues, we employ two distinct self-supervised methods. First, we adopt a contrastive learning method to capture generalized global representations, enabling the model to understand the semantic context of facial expressions without relying on labeled data. Simultaneously, we leverage masked image modeling to focus on embedding fine-grained, local facial landmark information at the patch-level. We introduce a novel module called FaceMAE, which aims to reconstruct the masked facial patches. The semantic masking scheme is designed to preserve highly activated feature activations, allowing the encoding of crucial details of unmasked facial landmarks and their relationships within the broader facial context at the patch-level. It finally guides the backbone network to calibrate the learned global features to be attentive to facial landmarks. Our proposed method, called Simple Facial Landmark Encoding (SimFLE), significantly outperforms supervised baseline and other self-supervised methods in terms of facial landmark localization and overall performance, as demonstrated through extensive experiments across several FER-W benchmarks.
The variational autoencoder (VAE) and generative adversarial networks (GAN) are two prominent approaches to achieving a probabilistic generative model by way of an autoencoder and a two-player minimax game. While VAEs often suffer from over-simplified posterior approximations, the adversarial autoencoder (AAE) has shown promise by adopting GAN to match the variational posterior to an arbitrary prior through adversarial training. Both VAEs and GANs face significant challenges such as training stability, mode collapse, and difficulty in extracting meaningful latent representations. In this paper, we propose the Multi-adversarial Autoencoder (MAAE), which extends the AAE framework by incorporating multiple discriminators and enabling soft-ensemble feedback. By adaptively regulating the collective feedback from multiple discriminators, MAAE captures a balance between fitting the data distribution and performing accurate inference and accelerates training stability while extracting meaningful and interpretable latent representations. Experimental evaluations on MNIST, CIFAR10, and CelebA datasets demonstrate significant improvements in latent representation, quality of generated samples, log-likelihood, and a pairwise comparison metric, with comparisons to recent methods.
Recent advancements in text-to-image models, such as Stable Diffusion, have showcased their ability to create visual images from natural language prompts. However, existing methods like DreamBooth struggle with capturing arbitrary art styles due to the abstract and multifaceted nature of stylistic attributes. We introduce Single-StyleForge, a novel approach for personalized text-to-image synthesis across diverse artistic styles. Using approximately 15 to 20 images of the target style, Single-StyleForge establishes a foundational binding of a unique token identifier with a broad range of attributes of the target style. Additionally, auxiliary images are incorporated for dual binding that guides the consistent representation of crucial elements such as people within the target style. Furthermore, we present Multi-StyleForge, which enhances image quality and text alignment by binding multiple tokens to partial style attributes. Experimental evaluations across six distinct artistic styles demonstrate significant improvements in image quality and perceptual fidelity, as measured by FID, KID, and CLIP scores.
Large-scale diffusion models have made significant advances in image generation, particularly through cross-attention mechanisms. While cross-attention has been well-studied in text-to-image tasks, their interpretability in image-to-image (I2I) diffusion models remains underexplored. This paper introduces Image-to-Image Attribution Maps $(\textbf{I}^2\textbf{AM})$, a method that enhances the interpretability of I2I models by visualizing bidirectional attribution maps, from the reference image to the generated image and vice versa. $\text{I}^2\text{AM}$ aggregates cross-attention scores across time steps, attention heads, and layers, offering insights into how critical features are transferred between images. We demonstrate the effectiveness of $\text{I}^2\text{AM}$ across object detection, inpainting, and super-resolution tasks. Our results demonstrate that $\text{I}^2\text{AM}$ successfully identifies key regions responsible for generating the output, even in complex scenes. Additionally, we introduce the Inpainting Mask Attention Consistency Score (IMACS) as a novel evaluation metric to assess the alignment between attribution maps and inpainting masks, which correlates strongly with existing performance metrics. Through extensive experiments, we show that $\text{I}^2\text{AM}$ enables model debugging and refinement, providing practical tools for improving I2I model's performance and interpretability.
Federated learning (FL) is a decentralized learning framework that allows a central server and multiple devices, referred to as clients, to collaboratively train a shared model without transmitting their private data to a central server. This approach helps to preserve data privacy and reduce the risk of information leakage. However, FL systems often face significant communication and computational overhead due to frequent exchanges of model parameters and the intensive local training required on resource-constrained clients. Existing solutions typically apply compression techniques such as quantization or pruning but only to a limited extent, constrained by the trade-off between model accuracy and compression efficiency. To address these challenges, we propose BiPruneFL, a communication- and computation-efficient FL framework that combines quantization and pruning while maintaining competitive accuracy. By leveraging recent advances in neural network pruning, BiPruneFL identifies subnetworks within binary neural networks without significantly compromising accuracy. Additionally, we employ communication compression strategies to enable efficient model updates and computationally lightweight local training. Through experiments, we demonstrate that BiPruneFL significantly outperforms other baselines, achieving up to 88.1x and 80.8x more efficient communication costs during upstream and downstream phases, respectively, and reducing computation costs by 3.9x to 34.9x depending on the degree of quantization. Despite these efficiency gains, BiPruneFL achieves accuracy comparable to, and in some cases surpassing, that of uncompressed federated learning models.
Electroencephalography (EEG)-based Automatic Emotion Recognition (AER) has gained increasing attention as a reliable tool for affective computing. While prior studies have explored various temporal, frequency, and spatial-domain representations of EEG signals, few have effectively integrated these domains within a unified framework. In this paper, we propose TFSNet, a multi-domain deep learning model that combines temporal, frequency, and spatial features for robust emotion recognition. TFSNet consists of a dual-encoder architecture: a temporal encoder based on state-space modeling (S4D), and a frequency-spatial encoder that leverages CNNs, attention mechanisms, and graph filtering using a physiologically-informed adjacency matrix. These domain-specific embeddings are fused and passed through a classifier for final prediction. Experimental results on the DREAMER dataset demonstrate that TFSNet achieves superior performance across Valence, Arousal, and Dominance emotions, outperforming state-of-the-art models. The results highlight the effectiveness of combining domain-aware representations and spatial connectivity priors for EEG-based emotion recognition and its potential for real-time applications.
Importance Consumer-level sleep analysis technologies have the potential to revolutionize the screening for obstructive sleep apnea (OSA). However, assessment of OSA prediction models based on in-home recording data is usually performed concurrently with level 1 in-laboratory polysomnography (PSG). Establishing the predictability of OSA using sound data recorded from smartphones based on level 2 PSG at home is important. Objective To validate the performance of a prediction model for OSA using breathing sound recorded from smartphones in conjunction with level 2 PSG at home. Design, Setting, and Participants This diagnostic study followed a prospective design, involving participants who underwent unattended level 2 home PSG. Breathing sounds were recorded during sleep using 2 smartphones, one with an iOS operating system and the other with an Android operating system, simultaneously with home PSG in participants’ own home environment. Participants were 19 years and older, slept alone, and had either been diagnosed with OSA or had no previous diagnosis. The study was performed between February 2022 and February 2023. Main Outcomes and Measures Sensitivity, specificity, positive predictive value, negative predictive value, and accuracy of the predictive model based on the recorded breathing sounds. Results Of the 101 participants included during the study duration, the mean (SD) age was 48.3 (14.9) years, and 51 (50.5%) were female. For the iOS smartphone, the sensitivity values at apnea-hypopnea index (AHI) levels of 5, 15, and 30 per hour were 92.6%, 90.9%, and 93.3%, respectively, with specificities of 84.3%, 94.4%, and 94.4%, respectively. Similarly, for the Android smartphone, the sensitivity values at AHI levels of 5, 15, and 30 per hour were 92.2%, 90.0%, and 92.9%, respectively, with specificities of 84.0%, 94.4%, and 94.3%, respectively. The accuracy for the iOS smartphone was 88.6%, 93.3%, and 94.3%, respectively, and for the Android smartphone was 88.1%, 93.1%, and 94.1% at AHI levels of 5, 15, and 30 per hour, respectively. Conclusions and Relevance This diagnostic study demonstrated the feasibility of predicting OSA with a reasonable level of accuracy using breathing sounds obtained by smartphones during sleep at home.
Previously, we reported that immunoproteasome (iP)-targeting linear peptide epoxyketones improve cognitive function in mouse models of Alzheimer's disease (AD) in a manner independent of amyloid β. However, these compounds' clinical prospect for AD is limited due to potential issues, such as poor brain penetration and metabolic instability. Here, we report the development of iP-selective macrocyclic peptide epoxyketones prepared by a ring-closing metathesis reaction between two terminal alkenes attached at the P2 and P3/P4 positions of linear counterparts. We show that a lead macrocyclic compound DB-60 (20) effectively inhibits the catalytic activity of iP in ABCB1-overexpressing cells (IC50: 105 nM) and has metabolic stability superior to its linear counterpart. DB-60 (20) also lowered the serum levels of IL-1α and ameliorated cognitive deficits in Tg2576 mice. The results collectively suggest that macrocyclic peptide epoxyketones have improved CNS drug properties than their linear counterparts and offer promising potential as an AD drug candidate.
Anomaly detection is critical in safety-sensitive fields, but faces challenges from scarce abnormal data and costly expert labeling. Time series anomaly detection is relatively challenging due to its reliance on sequential data, which imposes high computational and memory costs. In particular, it is often composed of real-time collected data that tends to be noisy, making preprocessing an essential step. In contrast, image anomaly detection has leveraged advancements in technologies for analyzing spatial patterns and visual features, achieving high accuracy and promoting research aimed at improving efficiency. We propose a novel framework that bridges image anomaly detection with time series data. Using Gramian Angular Field (GAF) transformations, we convert time series into images and apply state-of-the-art techniques, Reverse Distillation (RD) and EfficientAD (EAD), for efficient and accurate anomaly detection. Tailored preprocessing and transformations further enhance performance and interoperability. When evaluated on the multivariate time series anomaly detection dataset Secure Water Treatment (SWaT) and the univariate datasets University of California, Riverside (UCR) and Numenta Anomaly Benchmark (NAB), our approach demonstrated high recall overall and achieved approximately 99% F1 scores on some univariate datasets, proving its effectiveness as a novel solution for time series anomaly detection.
Recent advancements in text-to-image models, such as Stable Diffusion, have demonstrated their ability to synthesize visual images through natural language prompts. One approach of personalizing text-to-image models, exemplified by DreamBooth, fine-tunes the pre-trained model by binding unique text identifiers with a few images of a specific subject. Although existing fine-tuning methods have demonstrated competence in rendering images according to the styles of famous painters, it is still challenging to learn to produce images encapsulating distinct art styles due to abstract and broad visual perceptions of stylistic attributes such as lines, shapes, textures, and colors. In this paper, we introduce a new method, Single-StyleForge, for personalization. It fine-tunes pre-trained text-to-image diffusion models to generate diverse images in specified styles from text prompts. By using around 15-20 images of the target style, the approach establishes a foundational binding of a unique token identifier with a broad range of the target style. It also utilizes auxiliary images to strengthen this binding, resulting in offering specific guidance on representing elements such as persons in a target style-consistent manner. In addition, we present ways to improve the quality of style and text-image alignment through a method called Multi-StyleForge, which inherits the strategy used in StyleForge and learns tokens in multiple. Experimental evaluation conducted on six distinct artistic styles demonstrates substantial improvements in both the quality of generated images and the perceptual fidelity metrics, such as FID, KID, and CLIP scores.
Data augmentation is a widely used technique to enhance the performance of deep learning models. However, traditional augmentation methods, dependent solely on original data, often fall short in maintaining data diversity and generalization capabilities. In this paper, we propose a novel data augmentation approach leveraging pretrained text-to-image diffusion models to generate diverse and contextually rich images. Our approach integrates three advanced techniques: rich-text prompts, multi-object image generation, and inpainting. We demonstrate the effectiveness of these methods through extensive experiments on the Oxford-IIIT Pets and Caltech-101 datasets, where our diffusion-based augmentations significantly improved downstream classification accuracy and model generalization. No-tably, the inpainting technique excels in handling class imbalances by balancing the diversity and structural integrity of original data, while rich-text prompts and multi-object generation offer substantial gains by enhancing diversity and realism. Additionally, our methods show enhanced generalization to unseen data, proving their robustness and applicability to various deep learning tasks.
BACKGROUND:Multinight monitoring can be helpful for the diagnosis and management of obstructive sleep apnea (OSA). For this purpose, it is necessary to be able to detect OSA in real time in a noisy home environment. Sound-based OSA assessment holds great potential since it can be integrated with smartphones to provide full noncontact monitoring of OSA at home.OBJECTIVE:The purpose of this study is to develop a predictive model that can detect OSA in real time, even in a home environment where various noises exist.METHODS:This study included 1018 polysomnography (PSG) audio data sets, 297 smartphone audio data sets synced with PSG, and a home noise data set containing 22,500 noises to train the model to predict breathing events, such as apneas and hypopneas, based on breathing sounds that occur during sleep. The whole breathing sound of each night was divided into 30-second epochs and labeled as "apnea," "hypopnea," or "no-event," and the home noises were used to make the model robust to a noisy home environment. The performance of the prediction model was assessed using epoch-by-epoch prediction accuracy and OSA severity classification based on the apnea-hypopnea index (AHI).RESULTS:Epoch-by-epoch OSA event detection showed an accuracy of 86% and a macro F1-score of 0.75 for the 3-class OSA event detection task. The model had an accuracy of 92% for "no-event," 84% for "apnea," and 51% for "hypopnea." Most misclassifications were made for "hypopnea," with 15% and 34% of "hypopnea" being wrongly predicted as "apnea" and "no-event," respectively. The sensitivity and specificity of the OSA severity classification (AHI≥15) were 0.85 and 0.84, respectively.CONCLUSIONS:Our study presents a real-time epoch-by-epoch OSA detector that works in a variety of noisy home environments. Based on this, additional research is needed to verify the usefulness of various multinight monitoring and real-time diagnostic technologies in the home environment.
Self-supervised learning has been shown to be effective in various fields, proving its usefulness in contrastive learning. Recently, graph contrastive learning has shown state-of-the-art performance in the recommendation task. They created two views and learned node embeddings so that target nodes in the two views attract each other based on the target node, and non-target nodes in the two views repel each other. However, they overlooked the fact that false negatives can occur when negative pairs are repelled. It has been shown through various studies that false negatives in contrastive learning in various fields can have a negative impact on model training, but research on the impact of false negatives in link prediction tasks, such as recommendation tasks, where classes cannot be clearly defined, is still hardly explored. In this paper, we propose an approach to define false negatives in link prediction tasks and fully utilize them in learning. Learning by defining false negatives and removing them from negative pairs showed consistent improvements over existing graph contrastive learning on five benchmark datasets. In addition, we found through comprehensive experimental studies that learning by removing false negatives is of great advantage, especially for low-density datasets. On top of these advantages, our false negative detection and false negative elimination can be naturally integrated into any graph contrastive learning architecture.
With a growing interest in sleep monitoring at home, sound-based sleep staging with deep learning has emerged as a potential solution. However, collecting labeled data is restrictive in home environments due to the inconvenience of installing medical equipment at home. To handle this, we propose novel training approaches using accessible real-world sleep sound data. Our key contributions include a new semi-supervised learning technique called sequential consistency loss that considers the time-series nature of sleep sound and a semi-supervised contrastive learning method which handles out-of-distribution data in unlabeled home recordings. Our model was evaluated on various datasets including a labeled home sleep sound dataset and the public PSG-Audio dataset, demonstrating the robustness and generalizability of our model across real-world scenarios.
Recent advancements in text-to-image models, such as Stable Diffusion, have demonstrated their ability to synthesize visual images through natural language prompts. One approach of personalizing text-to-image models, exemplified by Dream-Booth, fine-tunes the pre-trained model by binding unique text identifiers with a few images of a specific subject. Although existing fine-tuning methods have demonstrated competence in rendering images asking to the styles of famous painters, it is still challenging to learn to produce images encapsulating distinct art styles due to abstract and broad visual perceptions of stylistic attributes such as lines, shapes, textures, and colors. In this paper, we present a new fine-tuning method, called StyleBoost, that equips pre-trained text-to-image models to produce diverse images in specified styles from text prompts. By leveraging around 15 to 20 images of StyleRef and Aux images each, our approach establishes a foundational binding of a unique token identifier with a broad realm of the target style, where the Aux images is carefully selected to strengthen the binding. This dual-binding strategy grasps the essential concept of art styles and accelerates learning of diverse and comprehensive attributes of the target style. Experimental evaluation conducted on three distinct styles - realism art, SureB art, and anime - demonstrates substantial improvements in both the quality of generated images and the perceptual fidelity metrics, such as FID and CLIP scores.
Abstract Introduction For diagnosis and management of Obstructive Sleep Apnea (OSA), long-term multi-night monitoring is crucial. Convenient detection of OSA at home is required for this purpose. Using sound recorded by smartphone can provide a convenient way to detect OSA. In this study, we present a sound-based OSA detection deep learning model that can detect OSA in real-time even in a home environment where various noises exist. The model is trained with home noise simulated sound to be robust for detecting home noises. Methods Two types of data were used for training and testing. The first type was sleep breathing sound data collected at the hospital while patients underwent a PSG. It included 1,154 and 297 nights recorded by a PSG microphone and a smartphone, respectively. We split them into 150 nights of smartphone data for testing and the rest for training. The second type was home noise data, which included 22,500 noises that might occur in a residential environment. The proposed acoustic apnea event detector inputs Mel spectrograms of sleep breathing sounds and outputs OSA event classes for each epoch (APNEA, HYPOPNEA, or NO-EVENT). The home noises were used to make the model robust to a noisy home environment. The performance of the prediction model was assessed by epoch-by-epoch prediction accuracy and OSA severity classification based on the apnea-hypopnea index (AHI). Results Our model achieved 86 % epoch-by-epoch agreement (0.75 in macro F1) for 3-class event detection task. The model had an accuracy of 92% for NO-EVENT, 84% for APNEA, and 51% for HYPOPNEA. Most misclassifications were made for HYPOPNEA, with 15% and 34% of HYPOPNEA being wrongly predicted as APNEA and NO-EVENT, respectively. The sensitivity and specificity of OSA severity classification (AHI ≥ 15) were 0.85 and 0.84, respectively. Conclusion Our study presents a real-time epoch-by-epoch OSA detector that works in a variety of noisy home environments. Based on this, additional research is needed to verify the usefulness of various multi-night monitoring and real-time diagnostic technologies in the home environment. Support (if any)
Jeong-Woo Cho合作论文数KTH Royal Institute of Technology
School of Information and Communication Technology4