We release VS-Sim, a synthetic dataset of road scene images that can be used to study the robustness to viewpoint shift of Computer Vision models for several tasks. Our dataset includes images from a frontal camera in different positions, and annotations for tasks both in frontal view (depth, semantic segmentation) and Bird’s Eye View (semantic segmentation). The paper also includes an analysis of the robustness of a Bird’s Eye View segmentation model to viewpoint shifts. Experiments indicate that viewpoint shift significantly degrades the performance of a model trained on data from a single viewpoint, and suggest that using multiple viewpoints during training helps to mitigate the performance drop across different scenarios.
Vision transformers (ViTs) have become the go-to solution in many computer vision tasks due to their ability to capture long-range dependencies in data and, consequently, achieve remarkable performance in dense prediction tasks. However, their high computational and memory requirements make these architectures unsuitable for embedded and low-power devices. This work introduces a novel linear ViT architecture optimized for resource-constrained embedded hardware. Our model operates with less than 1MB of parameters and 400KB of RAM, relying only on operations compatible with most embedded runtimes. We propose a linear attention mechanism based solely on fully connected layers, enabling execution on platforms that do not support typical transformer operations. To ease deployment, we apply a Hardware-Aware Scaling (HAS) strategy that enables one-shot network scaling to meet specific hardware constraints, without requiring an expensive Network Architecture Search (NAS). Using just three hyperparameters, HAS supports independent control over model size, memory usage, and computational cost. Our network architecture shows better generalization performance and up to 80 90% reduction in latency compared to existing approaches, thanks to better optimization for the lightweight CPUs and small accelerators typical of tiny, resource-constrained devices.
Room segmentation in architectural floor plan documents is crucial for tasks such as property valuation, space optimization, and automated design analysis. Unlike natural images, floor plans consist of abstract and sparse visual elements that pose unique challenges for segmentation models. In this work, we investigate the applicability of the Segment Anything Model (SAM), a general-purpose foundation model, to room-level segmentation in floor plans. We evaluate SAM on two heterogeneous datasets: a private collection and the public R3D dataset. Our results demonstrate that SAM consistently outperforms other semantic segmentation state-of-the-art approaches, enabling accurate surface area computation with minimal user input. We further analyze the impact of prompt quantity and training source variability, confirming SAM’s robustness and practical utility for real-world applications in architectural analysis.
Super-Resolution (SISR) is a computer vision task that aims to generate high-resolution images from their low-resolution counterparts. Typically, Super-Resolution methods use scaling factors of x2, x3, or x4 to uniformly enhance the resolution of the entire image. However, some acquisition devices, such as 360 ^∘ cameras, produce images with non-uniform resolution across the frame. In this work, we propose to adapt state-of-the-art efficient methods for Single Image Super-Resolution to address the challenge of restoring images affected by spatially varying degradations. Specifically, we focus on the method that won the recent NTIRE 2024 Efficient Super-Resolution Challenge. For our experiments, synthetic images with different spatially varying types of degradation are generated, and the SISR method is specifically modified and trained to effectively handle such challenging scenarios. In addition to evaluating the developed method with traditional image quality metrics such as PSNR and SSIM, we also assess its practical impact on a downstream object detection task. The results on the WIDER FACE face-detection dataset, using the YOLOv8 object detection model, show that applying the proposed SISR approach to images with spatially varying degradations produces artifact-free outputs and enables object detectors to achieve superior performance compared to their application on degraded images.
The rapid development of Generative Models (GMs) for image synthesis poses challenges to identifying manipulated images accurately. Traditionally, detection methods are trained to identify image manipulations after they occur. However, these approaches often struggle to generalize to GMs not seen during training. Proactive methods have been introduced to address this limitation. In this paradigm, images are embedded with a protection template as a form of proactive defense. Previous methods exploited a finite set of templates to proactively counter image manipulation, however, this raises concerns about potential vulnerabilities since a finite number of templates can provide a predictable exploit for malicious attackers. This work presents a template-based detection system that is designed to generate personalized templates for each image. Our approach enhances protection while also improving the accuracy of manipulation detection. Our solution achieves high detection accuracy on unseen GMs while also outperforming existing solutions.
