Recent studies on 3D Gaussian splatting demonstrate promising applications in novel view synthesis using several posed images. These images captured in real-life scenarios may be blurry and lead to unsatisfactory synthesis quality. Utilizing 2D image deblurring models to sharpen the blurry image before 3D Gaussian splatting is a natural inspiration. However, directly applying 2D image deblurring models for novel view synthesis results in poor performance due to the inevitable inconsistencies. For the above problem, this paper introduces a new approach called view-consistent Gaussian splatting via alternating branch optimization strategy(VC-GS). VC-GS consists of two branches: preceding branch and subsequent branch. The preceding branch restores multi-view blurry inputs using a 2D deblurring model before 3D Gaussian splatting, while the subsequent branch directly applies 3D Gaussian splatting to the blurry inputs and restores the synthesized views with the same model. Through the alternating branch optimization of the preceding branch and subsequent branch, the 3D Gaussians in the preceding branch can achieve high-quality reconstruction and maintain view consistency under blurry inputs. To validate the effectiveness of the approach, we conducted qualitative and quantitative experiments on synthetic and real datasets, demonstrating that the proposed approach achieves better results. Our project page is available at https://github.com/DTG777/View-Consistent-Gaussian-Splatting-from-Blurry-Images-for-Novel-View-Synthesis .

Cone-beam computed tomography (CBCT) acquires three-dimensional internal images, particularly effective for high-mineral density structures like bones. However, especially in sparse view scenarios, its ability to visualize low-density soft tissues is limited, restricting its clinical applications. To address this problem, this study proposes a method called Decoupled Neural Attenuation Fields (DE-NAF). Specifically, DE-NAF utilizes an Adaptive Hybrid Encoder that includes both hash encoding and 3D feature grid encoding methods. This approach decouples the CBCT reconstruction into two components. Hash encoding is used for high-mineral density structures, such as bones, owing to its superior encoding quality and ability to retain features of high-mineral density structures during hash conflict resolution. The 3D feature grid is employed for low-density soft tissues, such as muscles, as it effectively preserves the feature information of low-density soft tissues. The Adaptive Hybrid Encoder extracts these features, which are then decoded by a multilayer perceptron (MLP) decoder to predict X-ray attenuation values for precise reconstruction. In addition, a loss of structural perception was introduced to enhance tissue contrast and detail, further aiding CBCT reconstruction. Extensive experiments demonstrated that DE-NAF effectively addresses the limitations of CBCT in imaging low-density soft tissues, maintaining complete structural integrity and exceeding other methods in reconstruction quality.

By harnessing the capabilities of large language models (LLMs), recent large multimodal models (LMMs) have shown remarkable versatility in open-world multimodal understanding. Nevertheless, they are usually parameter-heavy and computation-intensive, thus hindering their applicability in resource-constrained scenarios. To this end, several lightweight LMMs have been proposed successively to maximize the capabilities under constrained scale (e.g., 3B). Despite the encouraging results achieved by these methods, most of them only focus on one or two aspects of the design space, and the key design choices that influence model capability have not yet been thoroughly investigated. In this paper, we conduct a systematic study for lightweight LMMs from the aspects of model architecture, training strategy, and training data. Based on our findings, we obtain Imp-a family of highly capable LMMs at the 2B similar to 4B scales. Notably, our Imp-3B model steadily outperforms all the existing lightweight LMMs of similar size, and even surpasses the state-of-the-art LMMs at the 13B scale. With low-bit quantization and resolution reduction techniques, our Imp model can be deployed on a Qualcomm Snapdragon 8Gen3 mobile chip with a high inference speed of about 13 tokens/s.

Building click-through rate (CTR) and conversion rate (CVR) prediction models for cross-border e-commerce search requires modeling the correlations among multi-domains. Existing multi-domain methods would suffer severely from poor scalability and low efficiency when number of domains increases. To this end, we propose a Domain-Aware Multi-view mOdel (DAMO), which is domain-number-invariant, to effectively leverage cross-domain relations from a multi-view perspective. Specifically, instead of working in the original feature space defined by different domains, DAMO maps everything to a new low-rank multi-view space. To achieve this, DAMO firstly extracts multi-domain features in an explicit feature-interactive manner. These features are parsed to a multi-view extractor to obtain view-invariant and view-specific features. Then a multi-view predictor inputs these two sets of features and outputs view-based predictions. To enforce view-awareness in the predictor, we further propose a lightweight view-attention estimator to dynamically learn the optimal view-specific weights w.r.t. a view-guided loss. Extensive experiments on public and industrial datasets show that compared with state-of-the-art models, our DAMO achieves better performance with lower storage and computational costs. In addition, deploying DAMO to a large-scale cross-border e-commence platform leads to 1.21%, 1.76%, and 1.66% improvements over the existing CGC-based model in the online AB-testing experiment in terms of CTR, CVR, and Gross Merchandises Value, respectively.

Low-resolution (LR) multi-view capture limits the fidelity of 3D Gaussian Splatting (3DGS). 3DGS super-resolution (SR) is therefore important, yet challenging because it must recover missing high-frequency details while enforcing cross-view geometric consistency. We revisit SRGS, a simple baseline that couples plug-in 2D SR priors with geometry-aware cross-view regularization, and observe that most subsequent advances follow the same paradigm, either strengthening prior injection, refining cross-view constraints, or modulating the objective. However, this shared structure is rarely formalized as a unified objective with explicit modules, limiting principled attribution of improvements and reusable design guidance. In this paper, we formalize SRGS as a unified modular framework that factorizes 3DGS SR into two components, prior injection and cross-view regularization, within a joint objective. This abstraction subsumes a broad family of recent methods as instantiations of the same recipe, enabling analysis beyond single-method innovation. Across five public benchmarks, we consolidate nine representative follow-up methods and trace reported improvements to specific modules and settings. Ablations disentangle the roles of priors and consistency, and stress tests under sparse-view input and challenging capture conditions characterize robustness. Overall, our study consolidates 3DGS SR into a coherent foundation and offers practical guidance for robust, comparable 3DGS SR methods.

Neural Radiance Fields (NeRF) have achieved great success in the task of synthesizing novel views that preserve the same resolution as the training views. However, it is challenging for NeRF to synthesize high-quality high-resolution novel views with low-resolution training data. To solve this problem, we propose a zero-shot super-resolution training framework for NeRF. This framework aims to guide the NeRF model to synthesize high-resolution novel views via single-scene internal learning rather than requiring any external high-resolution training data. Our approach consists of two stages. First, we learn a scene-specific degradation mapping by performing internal learning on a pretrained low-resolution coarse NeRF. Second, we optimize a super-resolution fine NeRF by conducting inverse rendering with our mapping function so as to backpropagate the gradients from low-resolution 2D space into the super-resolution 3D sampling space. Then, we further introduce a temporal ensemble strategy in the inference phase to compensate for the scene estimation errors. Our method is featured on two points: (1) it does not consume high-resolution views or additional scene data to train super-resolution NeRF; (2) it can speed up the training process by adopting a coarse-to-fine strategy. By conducting extensive experiments on public datasets, we have qualitatively and quantitatively demonstrated the effectiveness of our method.

Neural radiance fields (NeRF) synthesize realistic novel views by estimating point attributes (density and color), followed by the volume rendering method. However, accurately predicting the arbitrary point attributes poses a challenge for the single NeRF-based model. Such limitation directly impacts the quality of novel view synthesis. To address this problem, a collaborative strategy with multiple NeRF-based models is proposed. This strategy is the first to introduce a multi-model cascaded architecture into NeRF for achieving high-quality novel view synthesis. Its purpose is to utilize a cascading architecture in space for the progressive enhancement of point attribute accuracy. The cascading architecture includes point adjustment and snapshots fusion. Specifically, point adjustment leverages a pretrained NeRF-based model to predict the initial density and color of each point in space. This step affords an initial rendering of target scene. Then, these initial density and color of points are directly transferred to the subsequent NeRF-based model. This process guides the subsequent NeRF-based model to focus on the refinement of initial point attributes and synthesize more realistic novel views. Finally, snapshots fusion fuses outputs (referred as snapshots) from multiple parallel subsequent NeRF-based models to synthesize the ultimate high-quality novel views. The proposed strategy is tested with a range of established NeRF-based methods, such as NeRF, Instant-NGP, and TensoRF. Experimental data for this research are sourced from the realistic 360 synthetic dataset and the LLFF dataset. Results indicate that the proposed collaborative strategy with established NeRF-based methods can improve the quality of novel view synthesis, surpassing the corresponding single model. Our project page is available at https://github.com/ZhenyangLiu/Collaborative-Neural-Radiance-Fields-for-Novel-View-Synthesis .

Adopting Graph Convolutional Networks (GCNs) for transductive node classification is a hot research direction in artificial intelligence. Vanilla GCNs are primarily under-confident and struggle to clarify the final classification results explicitly due to the lack of supervision. Existing works mainly alleviated this issue by improving annotation deficiency and introducing addition regularization terms. However, these methods need to re-train the model from the beginning, which is computationally expensive for large dataset and model. To deal with this problem, a novel confidence correction mechanism (CCM) for trained GCNs is proposed in this work. Such mechanism aims at calibrating the confidence output of each node in the inference stage by jointly inferring the feature and predicted pseudo label. Specifically, in the inference stage, it uses the predicted pseudo label to select target-related features over all network to obtain a more confident and better result. Such selectivity is formulated as an optimization problem to maximize the category score of each node. In addition, the greedy optimization strategy is utilized to solve this problem and we have mathematically proven that the proposed mechanism can reach the local optimum by mathematical induction. Note that such mechanism is flexible and can be introduced to most GCN-based model. Extensive experimental results on benchmark datasets show that the proposed method can promote the confidence of the final target category and improve the performance of GCNs in the inference stage.

Generating a realistic image which matches the given text description is a challenging task. The multi-stage framework obtains the high-resolution image by constructing a low-resolution image firstly, which is widely adopted for text-to-image synthesis task. However, subsequent stages of existing generator have to construct the whole image repeatedly, while the primitive features of the objects have been sketched out in the previously adjacent stage. In order to make the subsequent stages focus on enriching fine-grained details and improve the quality of the final generated image, an efficient multi-path structure is proposed for multi-stage framework in this paper. The proposed structure contains two parts: staged connection and multi-scale module. Staged connection is employed to transfer the feature maps of the generated image from previously adjacent stage to the end of current stage. Such path can avoid the requirement of long-term memory and guide the network focus on modifying and supplementing the details of generated image. In addition, the multi-scale module is explored to extract feature at different scales and generate image with more fine-grained details. The proposed multi-path structure can be introduced to multi-stage based algorithm such as StackGAN-v2 and AttnGAN. Extensive experiments are conducted on two widely used datasets, i.e. Oxford-102 and CUB dataset, for the text-to-image synthesis task. The results demonstrate the superior performance of the methods with multi-path structure over the base models.

•Novel ray allocation strategy enhances textures and edges in scenes.•Canny edge detector guides dynamic ray allocation.•Improves performance of NeRF-based algorithms quantitatively and qualitatively.

Convolutional Neural Networks (CNNs) are vulnerable to adversarial attacks in which visually imperceptible perturbations can deceive CNN-based models. While current research on adversarial attacks in single object tracking exists, it overlooks a critical aspect of manipulating predicted trajectories to follow user-defined paths regardless of the actual location of the targeted object. To address this, we propose the very first white-box attack algorithm that is capable of deceiving victim trackers by compelling them to generate trajectories that adhere to predetermined counterfeit paths. Specifically, we focus on Siamese-based trackers as our victim models. Given an arbitrary counterfeit path, we first decompose it into discrete target locations in each frame, with the assumption of constant velocity. These locations are converted to heatmap anchors, which represent the offset of their location from the target object's location in the previous frame. Later on, we design a novel loss function to minimize the gap between above-mentioned anchors and our predicted ones. Finally, the gradients computed by such loss are used to update the original video, resulting in our adversarial video. To validate our ideas, we design three sets of counterfeit paths as well as novel evaluation metrics to measure the path-following properties. Experiments with two victim models on three publicly available datasets, OTB100, VOT2018, and VOT2016, demonstrate that our algorithm not only outperforms SOTA methods significantly under conventional evaluation metrics, e.g. 90% and 68.4% precision and successful rate drop on OTB100, but also follows the counterfeit paths well, which is beyond any existing attack methods. The source code is available at https://github.com/loushengtao/Follow-me.

To solve the problem that complex radar emitter signals are difficult to identify under low signal-to-noise ratio, this paper proposes a novel radar signal recognition method based on an improved deep residual network. In this method, two IQ signals are used as the input of the method, which saves time for generating time-frequency images, and then the signal features are extracted through an improved deep residual network. A nonlinear transform layer is inserted into the network to automatically confirm the threshold value, and then the soft threshold method is used to denoise. The importance of features is weighted by attention unit, and then classified by softmax classifier. The experiments based on five kinds of radar signal datasets show higher accuracy at low signal-to-noise ratio compared with other methods. The experiments also verified its overall accuracy can still exceed 90% even at extremely low signal-to-noise ratio of -16dB.

Aiming at the problem that complex radar emitter signals are difficult to be recognized at low signal-to-noise ratio, a method based on improved coordinate attention network is proposed. Firstly, the radar signal is converted into a two-dimensional time-frequency image to reflect the signal feature information. Then the time-frequency image preprocessing and denoising by convolutional neural network. Finally, the coordinated attention network is used for feature extraction, and then the classification of radar emitter source signals are realized. Experiments results show that the proposed method can validly improve the accuracy of radar signal recognition under the condition of low SNR.

Abstract. An increasing amount of face images are being captured, shared, or applied in various applications. These images usually contain lots of sensitive information that may lead to privacy disclosure and misuse problems. Some pioneering works show that face image anonymization is one of the promising solutions. We present an innovative identity (ID) camouflage approach by synthesizing anonymous faces so that both artificial intelligence algorithms and humans are unable to recognize them and misuse them freely. Given a face image, our approach consists of two steps. First, we sample an anonymous ID feature point in the feature space. Then, we synthesize a camouflage face by training an anonymous deep generative adversarial network model. To reduce the risk of re-identification, we optimize our anonymous face generator based on the k-nearest neighbors to make a good balance between anonymity and utility of the original face image. The experimental results over the public dataset have verified the feasibility and state-of-the-art efficacy of our approach.

Recently, deep convolutional neural networks show good effect for single image deraining. These networks always adopt the conventional convolution method to extract features, which may neglect the characteristic of rain streak. A novelty vertical module is proposed to focus on the vertical characteristic of rain streak. Such module uses 1 × X convolution kernel to extract the vertical information of rain streaks and a X × X convolution kernel to keep relative location information. Use this module in the front of deraining network can better detach rain streaks from background. In addition, the contrastive learning is employed to improve the performance of the model. Extensive experimental results demonstrated the superiority of the deraining methods with the proposed methods in comparison with the base ones.

The attention mechanism is an important component of cross-modal research. It can improve the performance of convolutional neural networks by distinguishing the informative parts of the feature map from the useless ones. Various kinds of attention are proposed by recent studies. Different attentions use distinct division method to weight each part of the feature map. In this paper, we propose a full-dimension attention module, which is a lightweight, fully interactive 3-D attention mechanism. FDAM generates 3-D attention maps for both spatial and channel dimensions in parallel and then multiplies them to the feature map. It is difficult to obtain discriminative attention map cell under channel interaction at a low computational cost. Therefore, we adapt a generalized Elo rating mechanism to generate cell-level attention maps. We store historical information with a slight amount of non-training parameters to spread the computation over each training iteration. The proposed module can be seamlessly integrated into the end-to-end training of the CNN framework. Experiments demonstrate that it outperforms many existing attention mechanisms on different network structures and datasets for computer vision tasks, such as image classification and object detection.

Recently, deep convolutional neural networks have achieved great success for single-image deraining. However, affected by the intrinsic overlapping between rain streaks and background texture patterns, a majority of these methods tend to almost remove texture details in rain-free regions and lead to over-smoothing effects in the recovered background. To generate reasonable rain streak layers and improve the reconstruction quality of the background, we propose a distributed feedback network (DFN) in recurrent structure. A novel feedback block is designed to implement the feedback mechanism. In each feedback block, the hidden state with high-level information (output) will flow into the next iteration to correct the low-level representations (input). By stacking multiple feedback blocks, the proposed network where the hidden states are distributed can extract powerful high-level representations for rain streak layers. Curriculum learning is employed to connect the loss of each iteration and ensure that hidden states contain the notion of output. In addition, a self-ensemble strategy for rain removal task, which can retain the approximate vertical character of rain streaks, is explored to maximize the potential performance of the deraining model. Extensive experimental results demonstrated the superiority of the proposed method in comparison with other deraining methods.

In the area of pedestrian trajectory prediction, the hybrid structures of temporal feature extractor or spatial feature extractor have paved the way for the precise prediction model, and they are in larger and larger scale. Learning of specific feature encoding model not only influenced by the structure of the network, but also by the learning manners such as supervised learning and unsupervised learning. Previous works concentrated on more comprehensive encoders and more delicate designs of feature extractors. However, the mutual influence factors from the neighbour pedestrians associate with the distance to the centre pedestrian seldomly noticed. Most of the existed feature extractors in prediction models trained in the way of supervised learning other than unsupervised manners caused the problem that the extracted features are always handcrafted without the natural distinction of obscure situations. The graph contrastive accelerating encoder is proposed, which accelerates the pedestrian trajectory prediction training process of the state of the art method of spatio-temporal graph transformer networks. Employing the unsupervised contrastive learning process and the graph of neighbours representing distance affection of nearest and farthest pedestrian to the centre pedestrian, the graph contrastive accelerating encoder significantly shrinked the training time. Holding the final performance on to state of the art level, the proposed method let the lowest pedestrian trajectory prediction error show up in the obviously earlier training steps.

Applying graph convolutional networks to efficient nodes classification is still a hot spot in the research area of artificial intelligence. In order to fully exploit the neuron information in the graph model, we propose a feedback-based graph neural network, where the graph network takes the information of the known labels and the output of the model as the targets to optimize the network. Then, a top-down manner is adopted to select neurons associated with the target and remove the ones with negative contribution weights through a gating mechanism. The experimental results on benchmark datasets show that our proposed networks remarkably outperform the comparison methods.

通过生成对抗网络的对抗学习生成仿真图像,已成为人工智能领域的一个研究热点.为了进一步提高生成图像的质量,本文提出了多判别器协同合作的网络框架——采用多个判别器为唯一生成器提供联合损失量,并通过不同的学习率保持各个判别器的差异性.同时,为了满足判别器的Lipschitz连续条件,本文所有的判别器网络一律进行谱归一化操作.实验表明,本文提出的基于多判别器合作框架的生成对抗网络表现较优.