Existing datasets for multimodal table understanding, such as MMTab, primarily provide short factual answers without explicit multi-step reasoning supervision. Models trained on these datasets often generate brief responses that offers insufficient accuracy and limited interpretability into how these models arrive at the final answer. We introduce CoReTab, a code-driven reasoning framework that produces scalable, interpretable, and automatically verifiable annotations by coupling multi-step reasoning with executable Python code. Using the CoReTab framework, we curate a dataset of 115K verified samples averaging 529 tokens per response and fine-tune open-source MLLMs through a three-stage pipeline. We evaluate the resulting model trained on CoReTab across 17 MMTab benchmarks spanning table question answering, fact verification, and table structure understanding. Our model achieves significant gains of +6.2

Text-to-Image editing using diffusion models faces challenges in balancing content preservation with edit application and handling real-image editing. To address these, we propose LAMS-Edit, leveraging intermediate states from the inversion process–an essential step in real-image editing–during edited image generation. Specifically, latent representations and attention maps from both processes are combined at each step using weighted interpolation, controlled by a scheduler. This technique, Latent and Attention Mixing with Schedulers (LAMS), integrates with Prompt-to-Prompt (P2P) to form LAMS-Edit–an extensible framework that supports precise editing with region masks and enables style transfer via LoRA. Extensive experiments demonstrate that LAMS-Edit effectively balances content preservation and edit application.

Multimodal Large Language Models (MLLMs) have shown impressive abilities in understanding and reasoning over conventional images. However, their perception of 360° images remains largely underexplored. Unlike conventional images, 360° images capture the entire surrounding environment, enabling holistic spatial reasoning but introducing challenges such as geometric distortion and complex spatial relations. To comprehensively assess MLLMs' capabilities to perceive 360° images, we introduce 360Bench, a Visual Question Answering (VQA) benchmark featuring 7K-resolution 360° images, seven representative (sub)tasks with annotations carefully curated by human annotators. Using 360Bench, we systematically evaluate seven MLLMs and six enhancement methods, revealing their shortcomings in 360° image perception. To address these challenges, we propose Free360, a training-free scene-graph-based framework for high-resolution 360° VQA. Free360 decomposes the reasoning process into modular steps, applies adaptive spherical image transformations to 360° images tailored to each step, and seamlessly integrates the resulting information into a unified graph representation for answer generation. Experiments show that Free360 consistently improves its base MLLM and provides a strong training-free solution for 360° VQA tasks. The source code and dataset will be publicly released upon acceptance.

BACKGROUND:Single-lead electrocardiograms (ECGs) contain subtle markers of left ventricular (LV) dysfunction. We hypothesized that a synthesized frontal-plane vector would improve single-lead detection of reduced LV ejection fraction (LVEF) compared with standard limb leads. METHODS AND RESULTS:This diagnostic study used paired 12-lead ECGs (10 s, 500 Hz) and echocardiograms obtained within 30 days from three Japanese hospital cohorts (one development and two external validation cohorts). For each beat, a derived lead, X, was synthesized from limb leads I and II as X(θ) = cosθ · I + sinθ · II, with θ scanned from 0° to 360°. A deep neural network generated beat-level predictions aggregated per recording to detect LVEF <50%. AUROC was the primary metric; accuracy, F1 score, sensitivity, and specificity were secondary metrics. Two performance peaks emerged near 30° and 210° (AUROC ≈0.857). Across repeated modeling from 180° to 270°, the 210° lead consistently ranked among the best single-lead models, matching or slightly exceeding aVR, although the 12-lead model remained superior. In external cohorts, the 210° lead achieved mean (±SD) AUROC 0.863±0.031, comparable to 12-lead and aVR (0.872±0.024 and 0.852±0.032; P>0.05). CONCLUSIONS:An aVR-adjacent frontal vector near 210° enables single-lead detection of LV dysfunction that approaches 12-lead performance and exceeds other individual leads. Requiring only standard limb electrodes, this strategy could facilitate screening; further validation and mechanistic studies are warranted.

Railway tracks are critical to social infrastructure, and their maintenance is essential for operational safety. Glued Insulated Rail Joints (GIRJs) are vital for railway signal control, where the insulator’s thickness helps prevent signal failures. This paper addresses detecting anomalies in GIRJ insulator thickness using images captured by devices on operational trains. Due to various factors, the insulator is not always clearly visible, and standard computer vision methods often struggle. In severe cases, judgments cannot be made from the image alone, requiring the system to return “unable to determine.” If these cases are rare, they can be manually inspected, still lowering overall inspection costs. We tackle this by framing the task as a one-dimensional regression problem, using convolutional neural networks (CNNs) to predict the boundary between the insulator and rail, while also estimating prediction uncertainty. Experiments with real-world data show that the model is accurate enough for practical use, even with challenging images. Additionally, we propose a robust method for detecting GIRJs in long-range railway images. This system is now operational in railway inspections.

3D Gaussian Splatting has emerged as a promising technique for high-quality 3D rendering, leading to increasing interest in integrating 3DGS into realism SLAM systems. However, existing methods face challenges such as Gaussian primitives redundancy, forgetting problem during continuous optimization, and difficulty in initializing primitives in monocular case due to lack of depth information. In order to achieve efficient and photorealistic mapping, we propose RP-SLAM, a 3D Gaussian splatting-based vision SLAM method for monocular and RGB-D cameras. RP-SLAM decouples camera poses estimation from Gaussian primitives optimization and consists of three key components. Firstly, we propose an efficient incremental mapping approach to achieve a compact and accurate representation of the scene through adaptive sampling and Gaussian primitives filtering. Secondly, a dynamic window optimization method is proposed to mitigate the forgetting problem and improve map consistency. Finally, for the monocular case, a monocular keyframe initialization method based on sparse point cloud is proposed to improve the initialization accuracy of Gaussian primitives, which provides a geometric basis for subsequent optimization. The results of numerous experiments demonstrate that RP-SLAM achieves state-of-the-art map rendering accuracy while ensuring real-time performance and model compactness.

In this study, we develop a super-resolution (SR) model for homogeneous isotropic turbulence (HIT) inspired by the recently proposed low-inference-cost ResShift diffusion model. The training data are obtained from direct numerical simulation of two three-dimensional HIT cases with varying grid resolutions and Reynolds numbers ( $ extit{Re}_\lambda = 94$ and 173) to increase the model's generalisability. The model is trained on two-dimensional snapshots rather than full three-dimensional fields, as training and inference on three-dimensional data would increase the computational cost significantly. Both the data from the whole domain and the data from a quarter of the domain are considered in the dataset to increase the diversity and quantity of training samples. This strategy also helps the model learn more localised flow structures and reduces dependence on global domain-specific patterns. The model is trained using single snapshots of velocity components for three upsampling factors of 4, 8 and 16. To assess the generalisability of the trained model, it is tested for flows under conditions different from those of the training data. Additionally, the high-resolution reconstruction of flow fields from low-resolution turbulent boundary layer data is performed to evaluate the model's performance in anisotropic turbulence. The results show that the diffusion model presented in this study performs well in predicting the velocity field even for high upsampling factors, and unlike bicubic interpolation, convolutional neural network (CNN)- and U-Net-based models, it does not generate a visually blurry flow field when applied to high upsampling factors. It also outperforms bicubic interpolation, CNN- and U-Net-based models, as well as the traditional conditional denoising diffusion probabilistic model designed for SR, in predicting flow statistics. The model effectively extracts flow features, generates flow structures of varying sizes and shows strong performance in predicting vorticity. It also reproduces the energy spectrum at high wavenumbers with reasonable accuracy, indicating the recovery of small-scale structures often lost in coarse data. This capability is valuable for subgrid-scale stress estimation and helps improve the physical fidelity of large eddy simulation frameworks.

This paper introduces a new benchmark test, Medical-Checklist, for assessing medical multimodal models. The recent advancements in multimodal models have demonstrated significant potential in the field of medical vision-language tasks. However, it is becoming increasingly clear that evaluating these models' performance, whether they are applied to natural or medical images, is challenging. The critical question is whether the models can accurately understand an input image while associating it with relevant input text. To address this, Medical-Checklist imposes a binary test on the models: they are given an image and two captions, where one is correct and the other incorrect, and the model must select the correct one. The incorrect caption contains a single medical concept (word or phrase) that is inaccurately substituted from the correct caption. Although the task is simple, this simplicity enables the unified assessment of diverse multimodal models designed and learned on different principles. It also enables us to verify whether models correctly understand a wide range of medical concepts across various medical sub-domains. Medical-Checklist is designed to reduce potential biases in data and to enable evaluation of the models' ability to handle out-of-distribution inputs, which were difficult in existing datasets. When evaluating four state-of-the-art medical multimodal models with Medical-Checklist, it was revealed that despite their excellent performance in specific tasks such as Med-VQA, they may not correctly understand images, suggesting a long journey ahead for clinical application. The dataset and code will be made public upon acceptance.

Open-set object detection (OSOD), a task involving the detection of unknown objects while accurately detecting known objects, has recently gained attention. However, we identify a fundamental issue with the problem formulation employed in current OSOD studies. Inherent to object detection is knowing "what to detect," which contradicts the idea of identifying "unknown" objects. This sets OSOD apart from open-set recognition (OSR). This contradiction complicates a proper evaluation of methods' performance, a fact that previous studies have overlooked. Next, we propose a novel formulation wherein detectors are required to detect both known and unknown classes within specified super-classes of object classes. This new formulation is free from the aforementioned issues and has practical applications. Finally, we design benchmark tests utilizing existing datasets and report the experimental evaluation of existing OSOD methods. The results show that existing methods fail to accurately detect unknown objects due to misclassification of known and unknown classes rather than incorrect bounding box prediction. As a byproduct, we introduce a taxonomy of OSOD, resolving confusion prevalent in the literature. We anticipate that our study will encourage the research community to reconsider OSOD and facilitate progress in the right direction.

Climate change exacerbates natural disasters, demanding rapid damage and risk assessment. However, expert‐reliant analyses delay responses despite drone‐aided data collection. This study develops and compares multimodal AI approaches using advanced large language models (LLMs) for expert‐level landslide image analysis. We tackle landslide‐specific challenges: capturing nuanced geotechnical reasoning beyond data digitization (specific to geological features and risk assessment), developing specialized transfer learning and data augmentation to mitigate data scarcity and geological diversity in landslide imagery, and establishing tailored evaluation metrics including geological accuracy, risk validity, and decision utility for landslide analysis. Evaluating a visual question answering‐large language model (VQA‐LLM) hybrid (sequential visual processing) and a multimodal large language model (MLLM, simultaneous vision/text processing) shows that MLLM excels in disaster identification, while the VQA‐LLM hybrid demonstrates superior performance in risk assessment, thereby informing optimal AI design choices. Our methodology, structuring 30+ years of expert commentary for AI training and employing a comprehensive evaluation framework including standard text metrics, LLM‐based semantic analysis, and expert domain assessment, highlights the potential of hybrid systems and addresses knowledge transfer in data‐sparse domains.

This paper studies the problem of inverting the DDIM image generation process to recover latent variables, particularly the initial noise map, from a generated image. Existing methods often struggle with accuracy in this task. We propose a novel hybrid approach that combines direct inversion via gradient descent for the first step, followed by a fixed-point method for subsequent steps. Empirical evaluations across three datasets demonstrate that our method significantly improves the prediction of initial latent variables while achieving superior reconstruction accuracy. Additionally, we introduce a new evaluation, called the self-interpolation test, which assesses the quality of images generated from interpolated points between the true and predicted latent maps, offering deeper insights into performance. Our results reveal that while existing methods perform reasonably well in reconstruction, they consistently fail to accurately predict the initial latent variables, resulting in poor performance on the self-interpolation test. In contrast, our method outperforms all others across all metrics, providing valuable insights into diffusion models and enhancing their applications in image generation and editing.

Smartphones with multi-camera systems, featuring cameras with varying field-of-views (FoVs), are increasingly common. This variation in FoVs results in content differences across videos, paving the way for an innovative approach to video super-resolution (VSR). This method enhances the VSR performance of lower resolution (LR) videos by leveraging higher resolution reference (Ref) videos. Previous works [14, 15], which operate on this principle, generally expand on traditional VSR models by combining LR and Ref inputs over time into a unified stream. However, we can expect that better results are obtained by independently aggregating these Ref image sequences temporally. Therefore, we introduce an improved method, RefVSR++, which performs the parallel aggregation of LR and Ref images in the temporal direction, aiming to optimize the use of the available data. RefVSR++ also incorporates improved mechanisms for aligning image features over time, crucial for effective VSR. Our experiments demonstrate that RefVSR++ outperforms previous works by over 1dB in PSNR, setting a new benchmark in the field.

Result diversification (RD) is a crucial technique in Text-to-Image Retrieval for enhancing the efficiency of a practical application. Conventional methods focus solely on increasing the diversity metric of image appearances. However, the diversity metric and its desired value vary depending on the application, which limits the applications of RD. This paper proposes a novel task called CDRCA (Contextual Diversity Refinement of Composite Attributes). CDR-CA aims to refine the diversities of multiple attributes, according to the application's context. To address this task, we propose Multi-Source DPPs, a simple yet strong baseline that extends the Determinantal Point Process (DPP) to multi-sources. We model MS-DPP as a single DPP model with a unified similarity matrix based on a manifold representation. We also introduce Tangent Normalization to reflect contexts. Extensive experiments demonstrate the effectiveness of the proposed method.

We present a novel event-based representation, named Motion-Encoded Time-Surface (METS), and how it can be used to address the challenge of pose tracking under high-speed scenarios with an event camera. The core concept is dynamically encoding the pixel-wise decay rate of the Time-Surface to account for localized spatio-temporal scene dynamics captured by events, rendering remarkable adaptability with respect to motion dynamics. The consistency between METS and the scene in highly dynamic conditions establishes a reliable foundation for robust pose estimation. Building upon this, we employ a semi-dense 3D-2D alignment pipeline to fully unlock the potential of the event camera for high-speed tracking applications. Given the intrinsic characteristics of METS, we further develop specialized lightweight operations aimed at minimizing the per-event computational cost. The proposed algorithm is successfully evaluated on public datasets and our high-speed motion datasets covering various scenes and motion complexities. It shows that our approach outperforms state-of-the-art pose tracking methods, especially in highly dynamic scenarios, and is capable of tracking accurately under incredibly fast motions that are inaccessible for other event- or frame-based counterparts. Due to its simplicity, our algorithm exhibits outstanding practicality, running at over 70 Hz on a standard CPU.

We propose action-agnostic point-level (AAPL) supervision for temporal action detection to achieve accurate action instance detection with a lightly annotated dataset. In the proposed scheme, a small portion of video frames is sampled in an unsupervised manner and presented to human annotators, who then label the frames with action categories. Unlike point-level supervision, which requires annotators to search for every action instance in an untrimmed video, frames to annotate are selected without human intervention in AAPL supervision. We also propose a detection model and learning method to effectively utilize the AAPL labels. Extensive experiments on the variety of datasets (THUMOS '14, FineAction, GTEA, BEOID, and ActivityNet 1.3) demonstrate that the proposed approach is competitive with or outperforms prior methods for video-level and point-level supervision in terms of the trade-off between the annotation cost and detection performance.

We present a novel self-supervised feature learning method using Vision Transformers (ViT) as the backbone, specifically designed for object detection and instance segmentation. Our approach addresses the challenge of extracting features that capture both class and positional information, which are crucial for these tasks. The method introduces two key components: (1) a positional encoding tied to the cropping process in contrastive learning, which utilizes a novel vector field representation for positional embeddings; and (2) masking and prediction, similar to conventional Masked Image Modeling (MIM), applied in parallel to both content and positional embeddings of image patches. These components enable the effective learning of intertwined content and positional features. We evaluate our method against state-of-the-art approaches, pre-training on ImageNet-1K and fine-tuning on downstream tasks. Our method outperforms the state-of-the-art SSL methods on the COCO object detection benchmark, achieving significant improvements with fewer pre-training epochs. These results suggest that better integration of positional information into self-supervised learning can improve performance on the dense prediction tasks.

The application of Multi-modal Large Language Models (MLLMs) in Autonomous Driving (AD) faces significant challenges due to their limited training on traffic-specific data and the absence of dedicated benchmarks for spatiotemporal understanding. This study addresses these issues by proposing TB-Bench, a comprehensive benchmark designed to evaluate MLLMs on understanding traffic behaviors across eight perception tasks from ego-centric views. We also introduce vision-language instruction tuning datasets, TB-100k and TB-250k, along with simple yet effective baselines for the tasks. Through extensive experiments, we show that existing MLLMs underperform in these tasks, with even a powerful model like GPT-4o achieving less than 35 contrast, when fine-tuned with TB-100k or TB-250k, our baseline models achieve average accuracy up to 85 Additionally, we demonstrate performance transfer by co-training TB-100k with another traffic dataset, leading to improved performance on the latter. Overall, this study represents a step forward by introducing a comprehensive benchmark, high-quality datasets, and baselines, thus supporting the gradual integration of MLLMs into the perception, prediction, and planning stages of AD.

Understanding how visual information is encoded in electrocorticography (ECoG) signals is essential for developing accurate and interpretable decoding models. In this study, we propose two novel approaches for multi-class visual classification based on ECoG data recorded from the inferior temporal cortex of macaque monkeys. The first model, MST-ECoGNet, combines traditional signal processing with neural networks by employing the Modified Stockwell Transform (MST) to map ECoG signals into a structured time-frequency-spatial domain. The second model, BiBand-3DECoGNet, replaces MST with a learnable convolutional module and utilizes a 3D spatial encoder to exploit the electrode array structure. Experimental results show that our models significantly outperform prior work, achieving up to a 12.87 percentage point improvement in classification accuracy while reducing model size by a factor of ten and increasing training speed by sixfold. Analysis of feature dimensions reveals that spatial and low-frequency components carry the most relevant information for visual decoding. These findings provide a foundation for further exploration of the neural mechanisms underlying visual object representation in the brain.

Visual localization is an important sub-task in SfM and visual SLAM that involves estimating a 6-DoF camera pose for an input query image relative to a given 3D model of the environment. The most accurate approach is a hierarchical one that splits the task into two stages: image retrieval and camera pose estimation. Each stage requires different image features, with global features compactly encoding holistic image information for the first stage and local features encoding the appearance around salient image points for the second stage. While existing methods use independent networks to extract these features, one for global and one for local, this strategy is suboptimal in terms of computational efficiency. In this paper, we propose a novel approach that achieves state-of-the-art inference accuracy with significantly improved efficiency. Our approach’s core component is SuperGF, a network that aggregates local features optimized for camera pose estimation to create a global feature that enables precise image retrieval. Through extensive experiments on the standard benchmark tests, we demonstrate that the method offers a better trade-off between accuracy and computational cost.

Introduction: Recent studies have demonstrated that artificial intelligence can effectively identify ventricular dysfunction using electrocardiograms (ECGs). However, the specific ECG waveforms indicative of left ventricular dysfunction remain poorly understood. This study aims to identify the ECG leads and segments that most accurately signal left ventricular dysfunction. Methods: We utilized ECG and echocardiography datasets comprising 17,422 cases from Japan and Germany. Convolutional neural networks with ten layers were developed to detect left ventricular ejection fractions below 50%. The models were trained and validated through 4-fold cross-validation using the Japanese dataset. The performance of the models was evaluated and compared among various ECG configurations (3-second strips, single-beat, and adjacent 2-beat overlay) and ECG segments (PQRST, QRST, P, QRS, PQRS). In addition to internal validation using Japanese data not employed during training, external validation was conducted with the German dataset. Results: Models based on 2-beat ECGs significantly outperformed the other models. Models with single-beat ECGs showed at least equivalent to superior performance compared to the 3-second models in both internal and external validations (AUC 0.908±0.001, 0.863±0.001 vs 0.826±0.016, respectively for 2-beat, single-beat, and 3-second models, P<0.0001, external validation). Employing single-beat models, our analysis found that limb leads, particularly leads I and aVR, were more indicative of left ventricular dysfunction. Furthermore, within the single-beat ECGs, the segments from QRS to T-wave were most revealing, while the addition of P segments significantly augmented the models’ performance ( Fig.1 ). Conclusions: This study confirms that using ECG segments from the P to the T-wave is more effective for assessing ventricular dysfunction than 3-second ECG strips, with performance further enhanced by multiple heartbeats. Notably, signals indicating ventricular dysfunction were predominantly distributed within the QRS to T-wave segments, while the P-segment also showed some significance. The concentration of dysfunction signals varied by lead, with leads I and aVR providing higher diagnostic utility. Further research is needed to identify specific ECG signals or waveforms for ECG-based diagnostics of ventricular dysfunction.