Layered image generation and editing is a fundamental capability that enables layer-wise reuse, editing, and composition of the generated visual content, analogous to word-level editing in natural language. Despite its importance, this remains an underexplored area at scale. To address this gap, we present the Masked Region Transformer, a 20B-parameter diffusion model tailored for multi-layer transparent image generation and editing, trained on over 10M multilingual design samples spanning diverse aspect ratios and textual prompts. To fully leverage this scale, we make three key technical contributions. First, we unify three complementary tasks---text-to-layers, image-to-layers, and layers-to-layers---within a shared masked region diffusion framework, where selective token masking enables flexible cross-modal generation and fine-grained layer-wise editing. Second, we design an efficient conditional diffusion decoder that incorporates Gated DeltaNet and gated attention mechanisms, enhancing visual fidelity while maintaining computational efficiency. Third, we introduce an overflow-aware canvas layer to handle boundary inconsistencies and support semi-transparent background synthesis, enabling complete editable layer generation beyond visible canvas boundaries. Additionally, we apply distribution matching distillation to achieve one-step, real-time multi-layer generation with minimal quality degradation. Extensive experiments demonstrate that our framework substantially outperforms prior state-of-the-art approaches across all three tasks, establishing a new benchmark for region-aware transparent image generation.

Photometric stereo recovers surface geometry by analyzing intensity variations under varying lighting conditions, offering high precision in fine-detail reconstruction. However, existing methods face challenges in unconstrained environments: per-pixel models lack global context, whereas all-pixel approaches often lose high-frequency detail. This study addresses the challenge of reconciling global contextual understanding with pixel-wise detail preservation in photometric stereo measurements. A novel dual-branch feature extraction neural architecture augmented by a multisuper-visory gradient propagation scheme is introduced. The proposed framework employs multiscale encoding and decoding mechanisms to integrate global and local representations, enabling end-to-end mapping from the input image sequences to surface normal estimations. Experimental results on benchmark datasets demonstrate that the proposed approach consistently outperforms nearly 20 state-of-the-art methods, highlighting robustness and practical applicability. The code and pretrained model weights are publicly available at https://github.com/zz-ux/ICASSP_2026.

Neural style transfer (NST) can create impressive artworks by transferring a reference style to a content image. Current image-to-image NST methods lack the fine-grained control often demanded for artistic editing. To mitigate this limitation, we propose a user-oriented interactive style transfer (IST) method, using which a harmonious image like drawing can be interactively created. Our IST method can serve as a brush, dipping style from anywhere, and then painting to any region of the target content image. To control the action scope, we formulate a fluid simulation algorithm, which takes styles as pigments around the position of brush interaction, and uses diffusion in style or content images according to similarity maps. By dipping and painting, even employing a single style image can produce thousands of eye-catching works. Our method expands the creative capabilities of NST.

Video summarization has garnered significant attention because of its valuable ability to enhance the efficiency of video browsing. Humans can effectively condense long videos into concise summaries by drawing on extensive prior knowledge and utilizing multi-source information to identify the most relevant content. However, existing video summarization approaches fail to incorporate multimodal cues and overlook implicit knowledge, resulting in lower-quality generated summaries. To address the limitations, we propose an innovative knowledge-guided multimodal network for the video summarization task, referred to as KGMNet. Specifically, KGMNet employs a visual-audio encoder to extract informative visual and audio representations from the input video. To complement these signals with high-level knowledge, a knowledge-guided encoder integrates affective information from an external knowledge source and expert attributes from image aesthetics, enabling the extraction of implicit knowledge that is not directly available from raw data. Based on these representations, a fine-to-coarse space projection module captures inter-relations among different modality spaces, strengthening cross-modal consistency. Moreover, a prediction head refines temporal structure by jointly estimating importance score, boundary descriptor, and centrality measure for each frame, supporting smoother transitions and more accurate localization of salient events. The experimental results demonstrate that the proposed network achieves superior performance compared to the state-of-the-art methods on two benchmark datasets. Notably, KGMNet attains an F-score of 60.4% on SumMe and 69.9% on TVSum, outperforming existing approaches. Furthermore, ablation studies validate the positive contribution of each module within the proposed KGMNet.

This research deals with the common problems of "not knowing the product before buying and assembling after buying" in the context of online sales of complex consumer products. A multimodal product understanding and assembly guidance optimization approach is proposed to address the information gap between product understanding, feature recognition and assembly execution. The research focuses on consumer products, such as bicycles, which are characterized by multiple components, significant parameters, and reliance on user self-assembly. This scenario is in close agreement with the applicant's actual experience in designing product display images, functional annotations, and installation instructions.Methodologically, the research uses CLIP for the semantic alignment of product images and text, BLIP-2 for the completion of text description and step consistency verification, and for the extraction of key structure visual characteristics. Additionally, a Transformer model encodes product parameter descriptions, component hierarchies, and assembly sequence information.On the basis of this framework, we construct a model of product understanding and a prediction model of assembly task. The User Behavior Analysis includes funnel analysis and Cognitive Load Stratification Assessment, and A/B Test is used to compare the performance of a conventional product page with an optimized solution at the time of purchasing decision-making and assembly execution. The experimental data include 52 product models, 6480 product page interaction records, 1,940 assembly task data points, and 14 months after sales service records.The results show that the optimized solution can increase the precision of product feature comprehension by 33.8%, the average time of assembly task is reduced by 26.9%, and the number of post-sales queries is reduced by 35.7%, and the relevant return request is reduced by 18.4%. Further results show that the problem of semantic mismatch between text and image has been reduced by over 40% with CLIP and BLIP-2. Research shows that the display system for complex consumer products should not be restricted to static visual presentation, but should serve as a digital information infrastructure that integrates product recognition, task orientation and after-sales support. This study is of practical importance to increase the transparency of electronic commerce for complex products, to reduce the risk of false tracking and misassembly, and to optimize the consumer experience.

Graphic design plays a vital role in visual communication across advertising, marketing, and multimedia entertainment. Prior work has explored automated graphic design generation using diffusion models, aiming to streamline creative workflows and democratize design capabilities. However, complex graphic design scenarios require accurately adhering to design intent specified by multiple heterogeneous user-provided elements (\eg images, layouts, and texts), which pose multi-condition control challenges for existing methods. Specifically, previous single-condition control models demonstrate effectiveness only within their specialized domains but fail to generalize to other conditions, while existing multi-condition methods often lack fine-grained control over each sub-condition and compromise overall compositional harmony. To address these limitations, we introduce CreatiDesign, a systematic solution for automated graphic design covering both model architecture and dataset construction. First, we design a unified multi-condition driven architecture that enables flexible and precise integration of heterogeneous design elements with minimal architectural modifications to the base diffusion model. Furthermore, to ensure that each condition precisely controls its designated image region and to avoid interference between conditions, we propose a multimodal attention mask mechanism. Additionally, we develop a fully automated pipeline for constructing graphic design datasets, and introduce a new dataset with 400K samples featuring multi-condition annotations, along with a comprehensive benchmark. Experimental results show that CreatiDesign outperforms existing models by a clear margin in faithfully adhering to user intent.

Background and Objective: Medical visual question answering requires reconciling clinical im-age content, structured biomedical knowledge, and natural language, yet prevailing architectures fuse these modalities once, in a single pass, and delegate all remaining cross-modal reasoning to a large language model. We propose an intelligent question-answering system that treats grounding and generation as distinct problems and performs explicit iterative grounding before generation begins. Methods: A Reasoning State Space Module (RSSM) maintains a reasoning state refined over 𝐾 cross-attention steps spanning image features, UMLS knowledge-graph embeddings, and intra-class visual prototypes, mirroring the cyclic hypothesis-revision of clinical reasoning. An Evidence Transport Module (ETM) then distils the final state into 𝐿 compact evidence tokens inserted immediately after the beginning-of-sequence token, placing the grounded evidence within the causal span of every generated token. The two modules are instantiated on the pre-aligned tri-modal embedding space of the BaMCo framework with a 3B-parameter LoRA-adapted backbone and trained end-to-end on a single GPU with a composite objective. Evaluation uses exact-match accuracy on the Slake and VQA-RAD benchmarks over three seeds, with paired bootstrap significance testing. Results: The system reaches 79.8% average accuracy on VQA-RAD and 87.0% on Slake. On VQA-RAD it exceeds a 14.7B-parameter generalist biomedical foundation model by 8.7 points, with gains concentrated in open-ended questions (+5.7 points over the base framework), while using roughly 4.9× fewer parameters. Both improvements over the base framework are statistically significant (𝑝 < 0.05 on Slake, 𝑝 < 0.01 on VQA-RAD). Ablations show the two modules are super-additive, each removing the constraint that limits the other, and characterise the effects of reasoning depth and evidence capacity. Conclusions: Explicit iterative grounding in representation space, rather than generative scale, is the decisive factor for compositional medical questions, yielding an accurate and parameter-efficient clinicalquestion-answering system.

Graphic design plays a crucial role in both commercial and personal contexts, yet creating high-quality, editable, and aesthetically pleasing graphic compositions remains a time-consuming and skill-intensive task, especially for beginners. Current AI tools automate parts of the workflow, but struggle to accurately incorporate user-supplied assets, maintain editability, and achieve professional visual appeal. Commercial systems, like Canva Magic Design, rely on vast template libraries, which are impractical for replicate. In this paper, we introduce CreatiPoster, a framework that generates editable, multi-layer compositions from optional natural-language instructions or assets. A protocol model, an RGBA large multimodal model, first produces a JSON specification detailing every layer (text or asset) with precise layout, hierarchy, content and style, plus a concise background prompt. A conditional background model then synthesizes a coherent background conditioned on this rendered foreground layers. We construct a benchmark with automated metrics for graphic-design generation and show that CreatiPoster surpasses leading open-source approaches and proprietary commercial systems. To catalyze further research, we release a copyright-free corpus of 100,000 multi-layer designs. CreatiPoster supports diverse applications such as canvas editing, text overlay, responsive resizing, multilingual adaptation, and animated posters, advancing the democratization of AI-assisted graphic design. Project homepage: https://github.com/graphic-design-ai/creatiposter

We propose Layer Decomposition of Graphic Designs (LDGD), a novel vision task that converts composite graphic design (e.g., posters) into structured representations comprising ordered RGB-A layers and metadata. By transforming visual content into structured data, LDGD facilitates precise image editing and offers significant advantages for digital content creation, management, and reuse. This task presents two core challenges: (1) predicting the attribute information (metadata) of each layer, and (2) recovering the occluded regions within overlapping layers to enable high-fidelity image reconstruction. To address this, we present the Decompose Layer Model (DeaM), a large unified multimodal model that integrates a conjoined visual encoder, a language model, and a condition-aware RGB-A decoder. DeaM adopts a two-stage processing pipeline: first generates layer-specific metadata containing information such as spatial coordinates and quantized encodings, and then reconstructs pixel-accurate layer images using a condition-aware RGB-A decoder. Beyond full decomposition, the model supports interactive decomposition via textual or point-based prompts. Extensive experiments demonstrate the effectiveness of the proposed method. The code is accessed at https://github.com/witnessai/DeaM.

With the continuous advancement of space technology, the number of defunct spacecraft, abandoned rocket bodies, and debris in space is increasing. These non-cooperative objects occupy a significant amount of orbital resources and pose a substantial threat to the safety of on-orbit spacecraft. This paper focuses on close-proximity operations in space and aims to address the limitation of camera resolution by proposing an optical flow-based multi-frame super-resolution reconstruction algorithm. This algorithm employs a multi-level wavelet convolutional network (MWCNN) for feature extraction and uses SpyNet to obtain multi-level optical flow between different frames. The multi-level optical flow pyramid alignment network is used to align features, and a recurrent network is utilized for frame-by-frame feature fusion. Finally, a reconstruction network generates high-resolution images. Extensive experiments have demonstrated that our proposed method effectively enhances the perception capabilities of space non-cooperative objects.

Video summarization task has gained widespread interest, benefiting from its valuable capabilities for efficient video browsing. Existing approaches generally focus on inter-frame temporal correlations, which may not be sufficient to identify crucial content because of the limited useful details that can be gleaned. To resolve these issues, we propose a novel transformer-based approach for video summarization, called Temporal-Channel Visual Transformer (TCVT). The proposed TCVT consists of three components, including a dual-stream embedding module, an inter-frame encoder, and an intra-segment encoder. The dual-stream embedding module creates the fusion embedding sequence by extracting visual features and short-range optical features, preserving appearance and motion details. The temporal-channel inter-frame correlations are learned by the inter-frame encoder with multiple temporal and channel attention modules. Meanwhile, the intra-segment representations are captured by the intra-segment encoder for the local temporal context modeling. Finally, we fuse the frame-level and segment-level representations for the frame-wise importance score prediction. Our network outperforms state-of-the-art methods on two benchmark datasets, with improvements from 55.3% to 56.9% on the SumMe dataset and from 69.3% to 70.4% on the TVSum dataset.

Visual relationships are crucial for visual perception and reasoning, and cover tasks like Scene Graph Generation, Human-Object Interaction, and object affordance. Despite significant efforts, this field still suffers from the following limitations: specialists for a specific task without considering similar ones, strict and complex task formulations with limited flexibility, and underexploited reasoning with language and knowledge. To solve these limitations, we seek to build a new framework, one model for all tasks, over Large Multimodal Models (LMMs). LMMs offer the potential of unifying tasks, flexible forms, and reasoning with language. However, they fail to handle visual relationship tasks well. We find the obstacles include the conflicts between different tasks and insufficient instance-level information. We solve these problems by reforming the data for LMMs, rather than architectures, considering their strong language-in language-out capability. We propose to disassemble tasks into simple and common sub-tasks, verbally estimate instance confidence, and augment instance diversity, all without additional modules. These strategies help us build a visual relationship generalist, RelationLMM, with a simple architecture. Exhaustive experiments demonstrate RelationLMM is strong, generalizable and flexible to different tasks, with one model and one suite of weight.

In the field of graphic design, automating the integration of design elements into a cohesive multi-layered artwork not only boosts productivity but also paves the way for the democratization of graphic design. One existing practice is Graphic Layout Generation (GLG), which aims to layout sequential design elements. It has been constrained by the necessity for a predefined correct sequence of layers, thus limiting creative potential and increasing user workload. In this paper, we present Hierarchical Layout Generation (HLG) as a more flexible and pragmatic setup, which creates graphic composition from any-ordered sets of design elements. To tackle the HLG task, we introduce Graphist, the first layout generation model based on large multimodal models. Graphist efficiently reframes the HLG as a sequence generation problem, utilizing RGB-A images as input, outputs a JSON draft protocol, indicating the coordinates, size, and order of each element. We develop multiple evaluation metrics for HLG. Graphist outperforms prior arts and establishes a strong baseline for this field.

The ability to learn from context with novel concepts, and deliver appropriate responses are essential in human conversations. Despite current Multimodal Large Language Models (MLLMs) and Large Language Models (LLMs) being trained on mega-scale datasets, recognizing unseen images or understanding novel concepts in a training-free manner remains a challenge. In-Context Learning (ICL) explores training-free few-shot learning, where models are encouraged to ``learn to learn" from limited tasks and generalize to unseen tasks. In this work, we propose link-context learning (LCL), which emphasizes "reasoning from cause and effect" to augment the learning capabilities of MLLMs. LCL goes beyond traditional ICL by explicitly strengthening the causal relationship between the support set and the query set. By providing demonstrations with causal links, LCL guides the model to discern not only the analogy but also the underlying causal associations between data points, which empowers MLLMs to recognize unseen images and understand novel concepts more effectively. To facilitate the evaluation of this novel approach, we introduce the ISEKAI dataset, comprising exclusively of unseen generated image-label pairs designed for link-context learning. Extensive experiments show that our LCL-MLLM exhibits strong link-context learning capabilities to novel concepts over vanilla MLLMs. Code and data will be released at https://github.com/isekai-portal/Link-Context-Learning.

Video summarization involves creating a succinct overview by merging the valuable parts of a video. Existing video summarization methods approach this task as a problem of selecting keyframes by frame- and shot-level techniques with unimodal or bimodal information. Besides underestimated inter-relations between various configurations of modality embedding spaces, current methods are also limited in their ability to maintain the integrity of the semantics within the same summary segment. To address these issues, we propose a novel multimodal temporal interest detection network (MTIDNet), to learn multimodal features in the fine- and coarse-grained embedding spaces using the mutual cross fusion layer. Furthermore, we design a temporal interest detection network to predict the importance scores and boundaries of each temporal segment that possesses local and global features across shots. Experimental results demonstrate the effectiveness of our MTIDNet on challenging datasets (SumMe and TVSum).

The objective of video summarization is to develop a concise and condensed summary that accurately captures the original video content. The methods currently used to summarize supervised videos and consider the task a sequence-to-sequence problem. However, modeling the order of long videos presents three challenges: (1) capturing both local and global relationships simultaneously is challenging; (2) the boundaries of video highlight segments are often incorrectly located, indicating that semantic integrity is incomplete; (3) efficient relation computing is difficult to do well. We design a novel coarse-to-fine network (C2F) for video summarization adapted to the multi-level semantic video structure, thus addressing these limitations. The multiscale representation scheme initially captures different scales of temporal relationships for the coarse classification results; Meanwhile, the action-wise proposal module is intended to provide the fine prediction of importance scores and regress the temporal locations of key-frames. In addition, a loss function is proposed to identify local differences among frames and analyze combinations of various loss functions. Extensive experimental results on two benchmark datasets have demonstrated that the proposed C2F achieves significant performance compared with state-of-the-art methods, and performs satisfactorily in efficient relation computing. For example, on the TVSum dataset, we improve the F-score from 69.4% to 72.8% by 3.4%. Furthermore, C2F includes 4.7 M parameters, accounting for only 10.7% of the parameters used in the SASUM model.

Referring Expression Segmentation (RES) is a widely explored multi-modal task, which endeavors to segment the pre-existing object within a single image with a given linguistic expression. However, in broader real-world scenarios, it is not always possible to determine if the described object exists in a specific image. Generally, a collection of images is available, some of which potentially contain the target objects. To this end, we propose a more realistic setting, named Group-wise Referring Expression Segmentation (GRES), which expands RES to a group of related images, allowing the described objects to exist in a subset of the input image group. To support this new setting, we introduce an elaborately compiled dataset named Grouped Referring Dataset (GRD), containing complete group-wise annotations of the target objects described by given expressions. Moreover, we also present a baseline method named Grouped Referring Segmenter (GRSer), which explicitly captures the language-vision and intra-group vision-vision interactions to achieve state-of-the-art results on the proposed GRES setting and related tasks, such as Co-Salient Object Detection and traditional RES. Our dataset and codes are publicly released in https://github.com/shikras/d-cube.

Interactive image segmentation (IIS) is an important technique for obtaining pixel-level annotations. In many cases, target objects share similar semantics. However, IIS methods neglect this connection and in particular the cues provided by representations of previously segmented objects, previous user interaction, and previous prediction masks, which can all provide suitable priors for the current annotation. In this paper, we formulate a sequential interactive image segmentation (SIIS) task for minimizing user interaction when segmenting sequences of related images, and we provide a practical approach to this task using two pertinent designs. The first is a novel interaction mode. When annotating a new sample, our method can automatically propose an initial click proposal based on previous annotation. This dramatically helps to reduce the interaction burden on the user. The second is an online optimization strategy, with the goal of providing semantic information when annotating specific targets, optimizing the model with dense supervision from previously labeled samples. Experiments demonstrate the effectiveness of regarding SIIS as a particular task, and our methods for addressing it.

Temporal action segmentation (TAS) of minute-long untrimmed videos involves locating and classifying human action segments using multiple action class labels. Previously, research on this task typically involved generating an initial estimate using designed temporal convolutional layers and gradually refining this estimate solely based on RGB features. This approach, however, exhibits several limitations, including the inability to capture inherent long-range dependencies and insufficient consideration of intricate spatial-temporal correlations in the changing relationships between human joints. To address these constraints, we introduce a novel spatial-temporal graph transformer network (STGT) for the skeleton-based TAS task. Our STGT employs a series of skeleton graph transformer blocks (SGT blocks) within an encoder-decoder architecture. Particularly, the spatial-temporal graph layer with an adaptive graph strategy enhances the graph structure, rendering it more flexible and robust. Additionally, the spatial-temporal transformer layer in the SGT block constructs parallel attention mechanisms to model the dynamic spatial and non-linear temporal correlations. Integrating these advancements into the TAS task represents a notable achievement. Experimental results on three challenging datasets (PKU-MMD, HuGaDB, and LARa) indicate the improved performance of the proposed framework compared with that of existing TAS models (MS-TCN, ASRF, BCN, ETSN, and ASFormer). Furthermore, our approach effectively addresses concerns regarding over-segmentation errors and ambiguous boundaries.