Objective Specular highlight detection and removal,with the primary objective of accurately identifying high-light regions exhibiting mirror-like reflection characteristics and precisely recovering the underlying pixel information,con-stitute a fundamental research topic in computer vision and image processing.This technology demonstrates considerable practical value across various visual applications,including object detection,medical image analysis,industrial defect identification,and 3D imaging.From an optical perspective,the specular highlight phenomenon originates from the direc-tional reflection behavior of incident light on smooth non-Lambertian surfaces,and its physical mechanism is theoretically describable through Fresnel equations.Traditional highlight removal methods typically rely on strict physical assumptions,which primarily encompass three classical approaches:1)chromaticity space analysis-based techniques,2)illumination estimation-based algorithms,and 3)polarization information-utilizing solutions.However,these methods exhibit limited adaptability to complex and variable highlight removal requirements in real-world scenarios because of their constraints regarding specific scenes or object types.By contrast,deep learning-based highlight detection and removal techniques adopt a data-driven approach to automatically learn the complex nonlinear mapping relationships between highlight reflec-tions and scene content,thereby effectively circumventing the dependency on prior assumptions inherent in traditional methods.The shifted window attention mechanism demonstrates unique advantages in highlight removal tasks and effec-tively addresses issues,such as texture information loss and color distortion,caused by specular highlights.This mecha-nism integrates local window partitioning with cross-window interaction,thus reducing computational complexity while maintaining global modeling capability.Building upon the exceptional modeling capability demonstrated by the shifted win-dow attention mechanism during the feature extraction stage,this study proposes a two-stage mask-guided highlight removal network architecture.The proposed framework employs the shifted window attention mechanism to extract multiscale cross-channel features and utilizes highlight masks as prior knowledge to guide subsequent pixel recovery in highlight regions,thereby achieving highlight localization and removal.Notably,existing deep learning methods face challenges regarding insufficient training data.Current publicly available highlight datasets predominantly employ synthetic generation or laboratory-controlled lighting acquisition methods,failing to meet the requirements of real-world applications.To address this limitation,this study additionally constructs an in-situ automotive highlight dataset captured under multiple illumina-tion conditions.This dataset has the following characteristics:1)coverage of diverse natural illumination conditions,2)inclusion of complex background interference,and 3)precise pixel-level annotations,thereby providing a data foundation for enhancing model generalization performance in real-world scenarios.Method In this study,we construct a real high-light dataset with cars as the main object,which is based on common cars in daily life with different highlight regions and intensities.Each pair of highlight images has its corresponding real highlight mask image.For highlight detection and removal,we propose a mask-guided two-stage highlight removal network(MG-TransUNet).This network explicitly divides the highlight detection and removal task into two phases.In the first phase,we train with a lightweight U-Net architecture to extract the highlight regions in the highlight images and generate the highlight mask map.In the second phase,a shifted window attention Transformer network that is based on the U-Net architecture is designed to recover the diffuse images of highlight regions by using the predicted highlight mask map as an a priori guide.Our experimental dataset is selected from three major public datasets(i.e.,paired specular-diffuse image dataset(PSD),specular highlight image quadruples(SHIQ)and synthetic specular highlight removal dataset(SSHR))with our proposed automotive dataset.Among the four datasets,only SHIQ and automobile datasets provide highlight masks.To make our method trainable on the PSD and SSHR datasets,we subtract and threshold the highlight images and the corresponding specular no-highlight images from the two datasets to obtain the highlight mask for each pair of images.The highlight detection network is trained with the highlight images and masks at the first stage,and the obtained highlight masks are used as a priori information to guide the highlight removal task at the second stage.Our mask-guided highlight removal network is implemented by the Pytorch toolbox.The input images and ground truth maps are resized to 256 × 256 for training,the momentum parameter is set to 0.99,and the initial learning rate is set to 2 × 10-4,which is reduced using a decay coefficient of 0.8 every 5 cycles until it reaches 1 × 10-6.The stochastic gradient descent learning process is accelerated using an NVIDIA GeForce GTX3090 GPU device,and 100 iterations requires about half a day.Result Experiments are conducted on the proposed automotive dataset and three publicly available datasets to compare eight state-of-the-art methods.The quantitative evaluation metrics include the structural similarity index measure and peak signal-to-noise ratio(PSNR).Experimental results show that the PSNR value ranks second on the PSD dataset.It is improved by 1.11 compared with that in the model with second-best performance on the SHIQ dataset and by 0.38 compared with that in the model with the second-best performance on the SSHR dataset.Comparative experiments are also performed on automotive datasets to validate the effectiveness of the proposed dataset.Conclusion We pro-pose a car highlight dataset for real scenarios and a mask-guided network incorporating a window-shift attention mechanism to perform the highlight removal task.The experimental results show that our highlight removal network can accurately rec-ognize highlight regions and remove highlight images.The code link:.
Driven by the rapid advancement of deep learning and large-scale computing,visual perception systems have achieved remarkable progress in a wide range of domains,including autonomous driving,intelligent transportation,secu-rity surveillance,medical diagnostics,industrial inspection,and human-robot interaction.Modern vision algorithms,empowered by massive datasets and increasingly sophisticated neural architectures,are now capable of performing object detection,semantic segmentation,scene understanding,and even causal reasoning with unprecedented accuracy.Despite this rapid growth,the development of visual intelligence still faces several critical bottlenecks.Chief among these chal-lenges are the highly imbalanced data distributions that characterize real-world environments,the scarcity of long-tail or rare-event samples essential for robustness,and the substantial human and financial cost associated with large-scale manual annotation.These factors significantly hinder the performance,safety,and generalizability of deep perception sys-tems,especially in complex,dynamic,or safety-critical scenarios.Parallel images technology,emerging as a novel image generation and modeling methodology grounded in parallel systems theory and the artificial systems,computational experi-ments,and parallel execution(ACP)framework,offers a promising pathway to address these limitations.The core idea behind parallel images is to construct controllable,high-fidelity artificial scene systems that reflect the structure,behavior,physics,and semantics of their real-world counterparts.Within these artificial systems,computational experiments can be conducted at scale,allowing for the controlled generation of diverse visual data that capture variations in illumination,geometry,environmental conditions,sensor characteristics,and task-specific factors.Through the interaction and iterative feedback between virtual and real environments,parallel images establish a closed-loop mechanism of"modeling-training-feedback-optimization",enabling perception models to evolve continuously,validate hypotheses,and improve perfor-mance under systematically generated variations.This closed-loop mechanism differentiates parallel images from traditional synthetic data generation in several important aspects.First,instead of passively producing static rendered images,paral-lel images emphasize dynamic parallelism,where virtual agents,environments,and tasks evolve in sync with real-world processes.Second,the approach integrates multimodal feedback,bridging visual,geometric,physical,and semantic modalities to ensure consistency and translatability across domains.Third,the framework supports scalable modeling of rare,dangerous,or expensive scenarios that are difficult or impossible to capture in real life,such as near-crash events in autonomous driving,rare diseases in medical imaging,or hazardous industrial operations.These capabilities make parallel images a powerful tool for enhancing the robustness,safety,and domain generalization of modern perception systems.This paper provides a comprehensive,systematic review of the theoretical foundations,methodological innovations,and devel-opmental trajectory of parallel images technology.This paper begins by revisiting its roots in parallel intelligence and the ACP paradigm and detailing how artificial systems serve as controlled experimental platforms that complement real-world data collection.Then,this paper examines recent technical advances encompassing three major research directions aligned with the ACP framework:1)multimodal data-driven virtual scene generation,which employs generative adversarial net-works,diffusion models,neural radiance fields,and 3D Gaussian splatting to overcome data scarcity and annotation bottle-necks,enabling the creation of controllable,editable,and semantically consistent synthetic environments;2)multiview feature fusion and virtual-real model transfer,aimed at addressing feature discrepancies and semantic misalignment across heterogeneous visual modalities through cross-modal alignment,multigranularity adaptive transfer,and domain-bridging strategies that enhance generalization and adaptability in hybrid virtual-real environments;3)parallel reasoning through heterogeneous data and knowledge fusion,which integrates structured information extraction,external knowledge guid-ance,scene graphs,temporal logic,and large language models to advance perceptual understanding toward semantic-level reasoning and decision-making,thereby supporting continuous optimization and closed-loop evolution in complex scenes.Beyond summarizing technological developments,this paper also situates parallel images within the broader context of emerging trends in generative artificial intelligence and foundation models.With the rise of diffusion models,neural radi-ance fields,and large-scale multimodal models,parallel images are poised to integrate more deeply with generative simula-tion pipelines.This paper discusses how these innovations can strengthen the fidelity,controllability,and adaptability of artificial visual data and potentially enable new capabilities such as task-conditioned scene synthesis,human-AI cosimula-tion,interactive data generation,and closed-loop autonomous scenario exploration,and provides key capabilities for build-ing general visual systems with continuous learning and feedback optimization.This paper provides crucial support for building general visual systems with continuous learning and feedback-driven optimization.Finally,this paper identifies several open challenges and future research directions that are essential for advancing the development of parallel images systems.These challenges include achieving high-quality expansion of virtual data,bridging the semantic gap between vir-tual and real domains,and enabling real-time,tightly coupled virtual-real interaction.Addressing these issues will require advances in intelligent generation models,self-supervised quality evaluation,unified data standards,causality-aware cross-domain alignment,and low-latency virtual-real collaboration supported by next-generation communication and sens-ing technologies.Solving these challenges will be critical for pushing forward the frontier of synthetic visual intelligence and unlocking the full potential of parallel images in real-world applications.
Intelligent digital humans have rapidly evolved with the advancement of computer graphics,computer vision,speech synthesis,and multimodal generative modeling.From early virtual avatars that focus on visual representation,digi-tal humans are currently developing toward dynamic motion modeling,emotion-aware interaction,and real-time deploy-ment.This study presents a systematic review of recent research progress in intelligent digital human content generation,organized around three core technical directions:video-to-digital human generation,3D human motion synthesis and edit-ing,and emotion-driven digital human generation.In addition,practical considerations for real-time on-device deployment are discussed.Video-to-digital human generation serves as the foundational stage for digital human construction.Its objec-tive is to reconstruct animatable 3D human avatars from monocular,multi-view,or in-the-wild video input.Early approaches primarily relied on implicit neural representations,such as neural radiance fields,frequently combined with parametric body models,such as the skinned multi-person linear model(SMPL).Although implicit methods provide con-tinuous and high-fidelity geometric representation,their rendering efficiency limits real-time applicability.Recent studies have shifted toward explicit or hybrid representations,particularly 3D Gaussian splatting,which significantly improves ren-dering speed while maintaining visual quality.Extensions that incorporate SMPL or SMPL-X priors further enhance geomet-ric stability and animatability.In multi-view settings,stronger geometric constraints improve reconstruction accuracy,while open-scene scenarios introduce additional challenges,such as occlusion handling,multi-person interaction,and background interference.Despite notable progress,maintaining temporal consistency and geometric robustness in complex environments remains an open problem.In addition to geometric reconstruction,3D human motion synthesis and editing enable digital humans to exhibit realistic dynamic behaviors.Compared with static modeling,motion generation requires accurate modeling of high-dimensional temporal distributions under kinematic and physical constraints.Early approaches based on statistical models or variational autoencoders improved representation capacity but frequently suffered from limited motion diversity.In recent years,diffusion models have become the dominant paradigm for motion generation due to their strong capability to model complex multimodal distributions.Representative frameworks demonstrate improved motion real-ism,diversity,and semantic alignment with textual or conditional input.Latent diffusion strategies further enhance effi-ciency by performing denoising processes in compact latent spaces.Beyond unconditional generation,condition-driven and fine-grained motion editing have attracted increasing attention.Text-guided editing frameworks allow local modification of specific joints or temporal segments while preserving overall motion style.Skeleton-aware and physics-guided diffusion models introduce structural constraints to improve anatomical plausibility and reduce artifacts,such as foot sliding.More-over,research has gradually expanded toward multi-person interaction modeling and long-sequence coherence,addressing challenges in action composition,interaction synchronization,and environment-aware motion planning.Nevertheless,bal-ancing physical consistency,computational efficiency,and controllability remains a critical challenge in practical applica-tions.Emotion-driven digital human generation further enhances interactivity and human-likeness.This direction includes facial expression synthesis,emotional speech synthesis,and multi-turn empathetic interaction modeling.In facial anima-tion,research has progressed from parameterized 3D morphable models to implicit neural rendering and,more recently,Gaussian-based explicit representations that achieve improved fidelity and real-time performance.In emotional speech syn-thesis,end-to-end neural architectures,non-autoregressive frameworks,and neural codec language models enable expres-sive,zero-shot,and fine-grained controllable speech generation.Meanwhile,emotion modeling in interactive dialogue sys-tems has evolved from passive emotion recognition toward empathetic response generation,incorporating graph-based con-textual modeling and large language model fine-tuning strategies.Although current systems can generate recognizable emo-tional expressions,challenges remain in maintaining emotional consistency over long interactions,decoupling emotion from underlying controllable factors,and ensuring cross-modal alignment in the speech,facial motion,and semantic contexts.To support real-world deployment,real-time on-device digital human systems have also gained attention.Lightweight mod-els,reduced-resolution rendering,intermediate parameter representations,and efficient inference frameworks are com-monly adopted under constrained computational resources.In practical applications,a collaborative architecture is fre-quently employed,where large language models handle semantic reasoning in the cloud while speech synthesis and avatar rendering are executed locally.This edge-cloud collaboration balances interaction latency and generation quality,facilitat-ing scalable deployment in mobile and desktop environments.In addition to reviewing representative models and technical routes,this work summarizes commonly used datasets and evaluation metrics across subfields,including text-to-motion benchmarks,emotional speech corpora,and multi-view reconstruction datasets.Performance is typically evaluated from multiple perspectives,such as perceptual realism,geometric accuracy,semantic alignment,motion stability,and subjec-tive human assessment.To facilitate reproducible research and provide a centralized resource for the community,we have curated all surveyed datasets,benchmark links,and a structured list of representative models into a public GitHub reposi-tory,which is available at: rapid prog-ress,unified benchmarks for long-term interactive digital humans are still lacking.Overall,intelligent digital human tech-nology is advancing along a progressive pathway from geometric reconstruction to motion generation,and finally,emotion-aware interaction.Future research is expected to focus on unified multimodal generative frameworks,improved long-term consistency modeling,physics-aware motion control,and efficient real-time deployment.By systematically organizing recent developments and open challenges,this review aims to provide a structured understanding of current progress and potential research directions in intelligent digital human content generation.
