Vision Language Models (VLMs) can be trained more efficiently if training sets can be reduced in size. Recent work has shown the benefits of masking text during VLM training using a variety of strategies (truncation, random masking, block masking and syntax masking) and has reported syntax masking as the top performer. In this paper, we analyze the impact of different text masking strategies on the word frequency in the training data, and show that this impact is connected to model success. This finding motivates Contrastive Language-Image Pre-training withWord Frequency Masking (CLIPF), our proposed masking approach, which directly leverages word frequency. Extensive experiments demonstrate the advantages of CLIPF over syntax masking and other existing approaches, particularly when the number of input tokens decreases. We show that not only CLIPF, but also other existing masking strategies, outperform syntax masking when enough epochs are used during training, a finding of practical importance for selecting a text masking method for VLM training.
Federated Learning (FL) enables collaborative model training without sharing raw data, but remains highly vulnerable to gradient manipulation and backdoor attacks, particularly under heterogeneous client distributions. Most existing defenses either target a narrow class of attacks, rely on client data, or fail to adapt in heterogeneous settings. We propose SD-CSFL (Synthetic Data-Driven Conformity Scoring for Federated Learning), a unified and privacy-preserving defense algorithm. SD-CSFL leverages a synthetic calibration dataset, independent of client data, to compute entropy-based nonconformity scores that capture irregularities in client updates. An adaptive percentile thresholding mechanism with stratified calibration dynamically distinguishes benign from malicious updates across training rounds. We establish a conformal prediction-based guarantee showing that percentile thresholds bound false positives under arbitrary score distributions. Experiments on CIFAR-10 and Birds-525 demonstrate up to 35% higher detection of gradient manipulation and an 80% reduction in backdoor success rates, outperforming recent defenses in heterogeneous environments.
Echocardiography, a widely used imaging modality, offers real-time assessments of cardiac morphology and function, with a particular emphasis on left ventricular dynamics. Despite its clinical importance, existing automated methods for echocardiographic analysis struggle to ensure temporal consistency in left ventricular key-point trajectories, largely due to their reliance on static frame annotations. To overcome these challenges, we propose a semi-supervised trajectory refinement framework that employs inter-frame correlations to enhance key-point trajectory estimation across echocardiography videos. A semi-supervised trajectory learning scheme is presented to improve the efficacy of key-point trajectory analysis using unannotated echocardiography videos. The experiments present considerable improvements in both spatial accuracy and temporal stability of the left ventricle key-point trajectories, outperforming state-of-the-art baselines and demonstrating the clinical applicability for robust echocardiography analysis.
In this paper, we propose joint optimization of a camera model and a deep neural network (DNN) for image classification and object detection tasks. The camera model consists of an image sensor model parameterized by the camera spectral sensitivity (CSS) and an image signal processing (ISP) model. We assume the camera model is composed of a three-sensor imager without a demosaicing process and an ISP with simple color correction and gamma correction. The DNNs then classify or detect objects. A key contribution of this paper is the joint optimization of not only the ISP model and DNN but also the image sensor model. For stable joint optimization, we have implemented a fully differentiable camera model. Therefore, we can jointly optimize the camera model and the DNN. Experimental comparisons with the flower and leaf datasets show that our approach outperforms existing approaches. Furthermore, we demonstrate that our approach is also effective for the object detection task. The source code is available at https://github.com/033labcodes/wacv26_jointopt_camera_model.git.
When spurious correlations between targets and data are present in training samples, deep neural networks may struggle to generalize, typically learning shortcuts corresponding to such undesired correlations (i.e., the bias), rather than fundamental target attributes. If bias attributes are assumed to be known, several strategies can be applied to mitigate a model’s dependency on bias, including upsampling or upweighting samples with no bias. However, this is hardly the case in real-world scenarios, and alternative unsupervised debiasing approaches have been proposed in recent years, assuming no bias information is available. In this work, we propose Lose Your Self (LoYS), a novel bias-unsupervised approach for model debiasing. The main design strategy in LoYS aims to force a model learning semantic features, discouraging it from learning bias-related descriptors, specifically focusing on the target classifier confidence. Exploiting an adversarial scheme based on entropy loss, against a shallow auxiliary classifier trained to match the predictions of a pre-trained biased model, and entropy regularizations, our experiments show how LoYS is competitive or outperforms state-of-the-art methods, on several common benchmarks.
