We propose MM-Re3D, a multi-modal retrieval-augmented framework designed to accelerate and enhance text-to-3D generation. By leveraging retrieved image priors based on text prompts, our approach effectively reduces the number of sampling steps required during inference, leading to faster generation speeds. Experimental results on the Text2Shape dataset demonstrate that MM-Re3D achieves a 30% to 40% reduction in sampling steps compared to baselines. Additionally, our framework exhibits strong cross-category generalization, with consistent performance gains across multiple ShapeNet categories achieved without extra training for effective few/zeroshot 3D shape generation across diverse categories. The proposed coarse-to-fine sampling strategy integrates multi-modal knowledge to guide the diffusion process, providing a significant boost to both efficiency and output fidelity. Our code is made publicly available at: https://github.com/lingdoudoulingling/MM-Re3D.

High-accuracy point cloud (self-) reconstruction is crucial for point cloud editing, translation, and unsupervised representation learning. However, existing point cloud reconstruction methods often sacrifice many geometric details. Altough many techniques have proposed how to construct better point cloud decoders, only a few have designed point cloud encoders from a reconstruction perspective. We propose an autodecoder architecture to achieve detail-preserving point cloud reconstruction while bypassing the performance bottleneck of the encoder. Our architecture is theoretically applicable to any existing point cloud decoder. For training, both the weights of the decoder and the pre-initialised latent codes, corresponding to the input points, are updated simultaneously. Experimental results demonstrate that our autodecoder achieves an average reduction of 24.62% in Chamfer Distance compared to existing methods, significantly improving reconstruction quality on the ShapeNet dataset. Furthermore, we verify the effectiveness of our autodecoder in point cloud generation, upsampling, and unsupervised representation learning to demonstrate its performance on downstream tasks, which is comparable to the state-of-the-art methods. We will make our code publicly available after peer review.

Recently, significant progress has been made in condition-guided human motion generation. However, due to the inherent abstraction of conditional semantics like text, music and trajectory, these methods often fall short of generating precise motions that align with human intent. In contrast, free-hand sketches inherently and accurately depict human perspective intent, finding extensive applications across multiple domains. In this article, we introduce Sketch-guided human Motion Diffusion (SMD), to address a novel scenario: sketch-to-motion, aiming to generate plausible and natural human motions based on human motion sketches. Specifically, Our proposed SMD employs a Dual-branch Time-aware Transformer that utilizes both global semantic and local perspective level attention to condition 2D sketch information for 3D motion generation. At the global semantic level, we establish associations between the representation of the entire sketch and the sequential motion to ensure the generated motion aligns with the semantic content of the sketch. Meanwhile, at the local perspective level, a sketch-aware local attention is devised to correlate the sketch patches with the motion keyframes, aiming to precisely align the keyframes with the given sketches. Rooted in Diffusion model and Dual-branch Time-aware Transformer, our approach demonstrates proficiency in motion in-betweening and body part editing tasks, seamlessly generating natural motion sequences that harmonize with the provided context. Multiple experiments conducted on the curated sketch-to-motion datasets validate the efficacy of SMD, showcasing the state-of-the-art generation performances.

Fundus photography serves as a valuable modality for the comprehensive examination of the ocular fundus, enabling medical professionals to meticulously analyze these images and derive accurate diagnoses. However, images obtained using traditional fundus photography equipment are often blurred due to limitations in image resolution and optical performance. In contrast, imaging equipment based on confocal laser technology produces clearer images with enhanced detail and contrast. Nevertheless, this technology is susceptible to image degradation due to the angle of capture and environmental lighting conditions, which can impact the quality of the resulting images. If the coloring field is applied to confocal fundus images to show the anatomical features of the lesion more clearly in the image, the problem of information hiding caused by insufficient light and serious shadow occlusion of the original confocal fundus images is solved. Therefore, this paper proposes a fundus image coloring algorithm based on degradation model using CNN as foundation and GAN built by combining the theory of degradation model. Firstly, the semantic matching problem is modeled, and the feature extractor implemented by CNN is used to extract semantic features from the reference image and the confocal fundus image for semantic matching. Finally, the colorized confocal fundus image is reconstructed and inversely processed by GAN module to recover the high-resolution image, and the final color confocal fundus image is obtained. This paper's method breaks through in terms of color similarity, overall naturalness, and image clarity by experimental comparative reviews with other coloring methods.

Besides the challenges posed by the complex optical properties of water, underwater spectral imaging face significant issues, including system bulkiness, optical complexity, and high spectral resolution requirement. This study introduces a compact staring-type underwater spectral imaging system designed to overcome these limitations. Featuring a 6-channel rotary spectral camera, the system is optimized for optical simplicity, and spectral efficiency. It employs a k-Nearest Neighbor (KNN)-based spectral reconstruction method to substantially enhance spectral resolution. Auto-focusing and image registration techniques, utilizing the Tenengrad function and ScaleInvariant Feature Transform (SIFT) feature points, ensure sharp and aligned multi-channel imagery. Experimental validation demonstrates the system's capability to accurately reconstruct spectral reflectance data across the 400 nm to 700 nm band, significantly increasing spectral resolution from 6 to 31 channels. The spectral reflectance reconstruction results, achieving an average Root Mean Square Error (RMSE) of 0.0651 and Goodness of Fit Coefficient (GFC) of 0.9871, highlight the system's efficiency in precise spectral analysis. This advancement in spectral resolution and adaptability to diverse underwater applications marks a significant leap forward in underwater observation and monitoring through spectral imaging.

Recent GCN-based works have achieved remarkable results for skeleton-based human action recognition. Nevertheless, while existing approaches extensively investigate pairwise joint relationships, only a limited number of models explore the intricate, high-order relationships among multiple joints. In this paper, we propose a novel hypergraph convolution method that represents the relationships among multiple joints with hyperedges, and dynamically refines the height-order relationship between hyperedges in the spatial, temporal, and channel dimensions. Specifically, our method initiates with a temporal-channel refinement hypergraph convolutional network, dynamically learning temporal and channel topologies in a data-dependent manner, which facilitates the capture of non-physical structural information inherent in the human body. Furthermore, to model various inter-joint relationships across spatio-temporal dimensions, we propose a spatio-temporal hypergraph joint module, which aims to encapsulate the dynamic spatial–temporal characteristics of the human body. Through the integration of these modules, our proposed model achieves state-of-the-art performance on RGB+D 60 and NTU RGB+D 120 datasets.

Image-based virtual try-on aims to fit a target garment to a reference person. Most existing methods are limited to solving the Garment-To-Person (G2P) try-on task that transfers a garment from a clean product image to the reference person and do not consider the Person-To-Person (P2P) try-on task that transfers a garment from a clothed person image to the reference person, which limits the practical applicability. The P2P try-on task is more challenging due to spatial discrepancies caused by different poses, body shapes, and views between the reference person and the target person. To address this issue, we propose a novel Keypoints-Driven Flow Based Virtual Try-On Network (KF-VTON) for handling both the G2P and P2P try-on tasks. Our KF-VTON has two key innovations: (1) We propose a new keypoints-driven flow based deformation model to warp the garment. This model establishes spatial correspondences between the target garment and reference person by combining the robustness of Thin-plate Spline (TPS) based deformation and the flexibility of appearance flow based deformation. (2) We investigate a powerful Context-aware Spatially Adaptive Normalization (CSAN) generative module to synthesize the final try-on image. Particularly, CSAN integrates rich contextual information with semantic parsing guidance to properly infer unobserved garment appearances. Extensive experiments demonstrate that our KF-VTON is capable of producing photo-realistic and high-fidelity try-on results for the G2P as well as P2P try-on tasks and surpasses previous state-of-the-art methods both quantitatively and qualitatively. Our code is available at https://github.com/OIUIU/KF-VTON .

Aiming at the bit rate selection problem of MPEG-DASH adaptive streaming media in image processing, a hybrid method combining multiple artificial intelligence algorithms is proposed. Firstly, kernel principal component analysis, Grey Wolf optimization algorithm and least squares support vector machine are integrated to construct an efficient hybrid algorithm model. This model aims to optimize the image processing effect in streaming media transmission, especially in the dynamic network environment. The experimental results show that the accuracy of the hybrid algorithm reaches 0.945 in the training process, and the absolute error is only 0.0005, which is significantly better than other comparison algorithms. Further empirical analysis shows that the accuracy of the proposed rate selection technique in image processing is as high as 92.3%, which is far higher than the existing technique. This research not only improves the image quality of streaming media transmission, but also greatly improves the user experience. The research provides a new perspective for image processing technology in the field of digital media, and is of great significance for promoting the innovation and development of streaming media technology.

In recent years, the task of weakly supervised audio-visual violence detection has gained considerable attention. The goal of this task is to identify violent segments within multimodal data based on video-level labels. Despite advances in this field, traditional Euclidean neural networks, which have been used in prior research, encounter difficulties in capturing highly discriminative representations due to limitations of the feature space. To overcome this, we propose HyperVD, , a novel framework that learns snippet embeddings in hyperbolic space to improve model discrimination. We contribute two branches of fully hyperbolic graph convolutional networks that excavate feature similarities and temporal relationships among snippets in hyperbolic space. By learning snippet representations in this space, the framework effectively learns semantic discrepancies between violent snippets and normal ones. Extensive experiments on the XD-Violence benchmark demonstrate that our method achieves 85.67% AP, outperforming the state-of-the-art methods by a sizable margin.

Abstract Graph Convolutional Networks (GCNs) have been widely used in skeleton‐based action recognition. Though significant performance has been achieved, it is still challenging to effectively model the complex dynamics of skeleton sequences. A novel position‐aware spatio‐temporal GCN for skeleton‐based action recognition is proposed, where the positional encoding is investigated to enhance the capacity of typical baselines for comprehending the dynamic characteristics of action sequence. Specifically, the authors’ method systematically investigates the temporal position encoding and spatial position embedding, in favour of explicitly capturing the sequence ordering information and the identity information of nodes that are used in graphs. Additionally, to alleviate the redundancy and over‐smoothing problems of typical GCNs, the authors’ method further investigates a subgraph mask, which gears to mine the prominent subgraph patterns over the underlying graph, letting the model be robust against the impaction of some irrelevant joints. Extensive experiments on three large‐scale datasets demonstrate that our model can achieve competitive results comparing to the previous state‐of‐art methods.

This paper proposes a novel method for neural implicit 3D shape synthesis using a two-stage training strategy with an implicit autoencoder and a multi-scale denoising diffusion model. This presented approach is built on autoencoding implicit fields, performed multi-scale denoising diffusion process on the latent implicit grids encoded by our implicit autoencoder. To be specific, those latent implicit grids maintain the position awareness trait compared to traditional global implicit codes so that generating 3D shapes with more local details are further allowed when implementing our designed multi-scale denoising diffusion process. The approach’s effectiveness of shape reconstruction and synthesis, demonstrated on the ShapeNet dataset, make it a promising tool for multiple applications, including 3D modeling and analysis. Future integration with multimodal data sources, like text, audio, and images, could further expand its applicability to a wide range of real-world scenarios.

Few-shot colorization aims to learn a model to colorize images with little training data. Yet, existing models often fail to keep color consistency due to ignored patch correlations of the images. In this paper, we propose PCCNet, a novel Patch-wise Contrastive Colorization Network to learn color synthesis by measuring the similarities and variations of image patches in two different aspects: inter-image and intra-image. Specifically, for inter-image, we investigate a patch-wise contrastive learning mechanism with positive and negative samples constraint to distinguish color features between patches across images. For intra-image, we explore a new intra-image correlation loss function to measure the similarity distribution which reveals structural relations between patches within an image. Furthermore, we propose a novel color memory loss that improves the accuracy of the memory module to store and retrieve data. Experiments show that our method allows the correctly saturated color to spread naturally over objects and also achieves higher scores in quantitative comparisons with related methods.

Aiming at the problem that the existing models are difficult to accurately extract the global spatiotemporal features, a surface water quality prediction model GAT-DASeq2Seq based on the attention mechanism and fusion spatio-temporal features is proposed. Spatial features between water quality monitoring sites were extracted with GAT, and then the key features and key time points of historical water quality time series were extracted using the DASeq2Seq model that introduced a two-stage attention mechanism. The experiments were carried out with 15 months of historical data from 6 water quality monitoring stations in the Huangshui River and its surrounding areas in Xining City, Qinghai Province, and the comparison model of the ablation experiment was selected for the comparison experiment. The experimental results show that the proposed model can effectively Improve the accuracy of multi-step prediction of surface water quality.

In recent years, approaches based on graph convolutional networks (GCNs) have achieved state-of-the-art performance in point cloud learning. The typical pipeline of GCNs is modeled as a two-stage learning process: graph construction and feature learning. We argue that such process exhibits low efficiency because a high percentage of the total time is consumed during the graph construction process when a large amount of sparse data are required to be accessed rather than on actual feature learning. To alleviate this problem, we propose a graph-based parallel branch network (Graph-PBN) that introduces a parallel branch structure to point cloud learning in this study. In particular, Graph-PBN is composed of two branches: the PointNet branch and the GCN branch. PointNet exhibits advantages in memory access and computational cost, while GCN behaves better in local context modeling. The two branches are combined in our architecture to utilize the potential of PointNet and GCN fully, facilitating the achievement of efficient and accurate recognition results. To better aggregate the features of each node in GCN, we investigate a novel operator, called EAGConv, to augment their local context by fully utilizing geometric and semantic features in a local graph. We conduct experiments on several benchmark datasets, and experiment results validate the significant performance of our method compared with other state-of-the-art approaches. Our code will be made publicly available at https://github.com/zhangcheng828/Graph-PBN .

Segmentation of the lung parenchymal region in chest CT is an essential part of the automatic diagnosis of lung diseases. Therefore, the quality of the segmentation directly affects the results of the automatic diagnosis. This paper proposes a model for lung parenchymal segmentation in chest CT based on ResUnet. It introduces the residual learning unit to transfer low-level information and enhances the connection between layers using skip connections based on the U-Net architecture. Then, it achieves full feature extraction through down-convolution and up-sampling and uses image enhancement and data augmentation to preprocess the data set. Through experiment, the proposed segmentation model has better results than the IoU and Dice of other models and can better segment the lung parenchyma in chest CT.

At present, under extreme weather conditions, flood events caused by river overflow occur frequently, which poses a great threat to people's life and property safety. In order to prevent the disaster caused by flood and take preventive measures in advance, this paper constructs a hybrid short-term water level prediction model ATT-ConvLSTM which combined with attention mechanism and ConvLSTM. The model uses the water level, flow, temperature and rainfall data in the historical hydrological data set of Arnoia River to predict the short-term change of water level. The experimental results show that the prediction model proposed in this paper achieves good results in Mean Absolute Error (MAE), Root Mean Square Error (RMSE) and R-Squared (R2).

The rapid development of computer graphics and computer vision technology has made digital media technology widely used in various fields, e.g. media art. However, due to the limited sensing range of the sensor, the traditional interaction system can usually only do small-scale interactive recognition. In this paper, we propose a large scene interactive system which employs multiple Kinect devices. By fusing the multiple sensing data into our unified framework, our method greatly expands the interaction area. Furthermore, our method applies the vector field to our interactive art feedback system, which is visually impressive, vivid and efficient in computing complexity. The evaluation and user study have demonstrated the effectiveness and feasibility of our method.

Aiming at the complex dependence of water quality data in space and time, we propose a GCN-Seq2Seq model for surface water quality prediction. The model uses Graph Convolutional Network (GCN) to capture the spatial feature of water quality monitoring sites, uses the sequence to sequence (Seq2Seq) model constructed by GRU to extract the temporal feature of the water quality data sequence, and predicts multi-step water quality time series. Experiments were carried out with data from 6 water quality monitoring stations in the Huangshui River and surrounding areas in Xining City, Qinghai Province from November 2020 to June 2021, and compared with the baseline model. experimental results show that the proposed model can effectively improve the accuracy of multi-step prediction of surface water quality.

Artificial underwater illumination has a wide range of applications in coastal zone research, marine engineering, marine resource exploration and development, and marine ecological environment monitoring. In this article, the distribution of spectral irradiance has been discussed for an underwater light field generated by an underwater LED light source. Theoretical representations of the spectral irradiance have been derived according to physical principles and underwater spectral irradiance measurement have been accrued out along the optical axis of the light source and in the vertical plane perpendicular to the optical axis. Experimental results show that the spectral irradiance along the optical axis of the light source decays with respect to the distance, following the law of square attenuation and exponential attenuation. The irradiance in the vertical plane approximately matches the Gaussian distribution.

A method of air quality prediction based on deep learning is proposed in this paper, that is an air quality prediction model combining bidirectional gated recurrent unit and attention mechanism. Taking cities with air quality monitoring stations as reference, the change trend of air quality index in the future is predicted by analyzing and processing the historical values of PM2.5, PM10, SO2, NO2, O3, CO in the past five years. The experimental results show that the model has good results in the mean absolute percentage error, mean absolute error and root mean square error of air quality prediction, and the model can effectively improve the data accuracy and stability of urban air quality prediction.