3D Gaussian splatting (3DGS) has recently demonstrated promising advancements in RGB-D online dense mapping. Nevertheless, existing methods excessively rely on per-pixel depth cues to perform map densification, which leads to significant redundancy and increased sensitivity to depth noise. Additionally, explicitly storing 3D Gaussian parameters of room-scale scene poses a significant storage challenge. In this paper, we introduce OG-Mapping, which leverages the robust scene structural representation capability of sparse octrees, combined with structured 3D Gaussian representations, to achieve efficient and robust online dense mapping. Moreover, OG-Mapping employs an anchor-based progressive map refinement strategy to recover the scene structures at multiple levels of detail. Instead of maintaining a small number of active keyframes with a fixed keyframe window as previous approaches do, a dynamic keyframe window is employed to allow OG-Mapping to better tackle false local minima and forgetting issues. Experimental results demonstrate that OG-Mapping delivers more robust and superior realism mapping results than existing Gaussian-based RGB-D online mapping methods with a compact model, and no additional post-processing is required.
Visual localization and object detection both play important roles in various tasks. In many indoor application scenarios where some detected objects have fixed positions, the two techniques work closely together. However, few researchers consider these two tasks simultaneously, because of a lack of datasets and the little attention paid to such environments. In this paper, we explore multi-task network design and joint refinement of detection and localization. To address the dataset problem, we construct a medium indoor scene of an aviation exhibition hall through a semi-automatic process. The dataset provides localization and detection information, and is publicly available at https://drive.google.com/drive/folders/1U28zkuN4_I0dbzkqyIAKlAl5k9oUK0jI?usp=sharing for benchmarking localization and object detection tasks. Targeting this dataset, we have designed a multi-task network, JLDNet, based on YOLO v3, that outputs a target point cloud and object bounding boxes. For dynamic environments, the detection branch also promotes the perception of dynamics. JLDNet includes image feature learning, point feature learning, feature fusion, detection construction, and point cloud regression. Moreover, object-level bundle adjustment is used to further improve localization and detection accuracy. To test JLDNet and compare it to other methods, we have conducted experiments on 7 static scenes, our constructed dataset, and the dynamic TUM RGB-D and Bonn datasets. Our results show state-of-the-art accuracy for both tasks, and the benefit of jointly working on both tasks is demonstrated.
The localization of the camera is essential in AR, MR, and robotics. Diverse pipelines employ a hand-crafted or learning based way to predict the camera pose as per the task. In the localization process, both weaknesses and strengths are maintained. However, few current frameworks consider these two features simultaneously. In this study, a novel relocalization pipeline for RGB or RGB-D input is proposed, including a coarse stage with learned features, further refinement with hand-crafted features, and a stable process to measure the confidence of both stages for improving localization robustness. Instead of directly regressing the camera pose, the coarse procedure uses registration to the known source and predicted weighted target point cloud to obtain the initial result. Therefore, we design a deep network called PGNet to construct the weighted target point cloud with the image and previous poses as inputs. Moreover, in consideration of dynamic surroundings, we add a segmentation branch distinguishing each point as either fixed or dynamic with the purpose of promoting dynamic perception. Correspondingly, the segmentation-extended Chamfer Distance is added to optimize PGNet. During the pose refinement, the feature space is established via hand-crafted feature extraction and matching on the training set. Based on the coarse pose, we obtain the accurate pose by applying Kabsch or Perspective-n-Point (PnP) algorithm to point-to-point correspondences built through searching the space and matching Oriented Fast and Rotated Brief (ORB) features. Furthermore, an additional process is presented by defining coarse and refinement metrics to gain a more stable performance. Finally, experiments on both static and dynamic scenes are conducted. On the one side, the results demonstrate the state-of-the-art performance over other existing methods on 7 Scenes, INDOOR-6, Cambridge Landmarks and TUM RGB-D. On the other side, the positive effects of the pose learning part, dynamic branch, confidence regression and hand-crafted feature based refinement are also provided.
Object pose estimation and camera localization are critical in various applications. However, achieving algorithm universality, which refers to category-level pose estimation and scene-independent camera localization, presents challenges for both techniques. Although the two tasks keep close relationships due to spatial geometry constraints, different tasks require distinct feature extractions. This paper pays attention to a unified RGB-D based framework that simultaneously performs category-level object pose estimation and scene-independent camera localization. The framework consists of a pose estimation branch called SLO-ObjNet, a localization branch called SLO-LocNet, a pose confidence calculation process and object-level optimization. At the start, we obtain the initial camera and object results from SLO-LocNet and SLO-ObjNet. In these two networks, we design there-level feature fusion modules as well as the loss function to achieve feature sharing between two tasks. Then the proposed approach involves a confidence calculation process to determine the accuracy of object poses obtained. Additionally, an object-level Bundle Adjustment (BA) optimization algorithm is further used to improve the precision of these techniques. The BA algorithm establishes relationships among feature points, objects, and cameras with the usage of camera-point, camera-object, and object-point metrics. To evaluate the performance of this approach, experiments are conducted on localization and pose estimation datasets including REAL275, CAMERA25, LineMOD, YCB-Video, 7 Scenes, ScanNet and TUM RGB-D. The results show that this approach outperforms existing methods in terms of both estimation and localization accuracy. Additionally, SLO-LocNet and SLO-ObjNet are trained on ScanNet data and tested on 7 Scenes and TUM RGB-D datasets to demonstrate its universality performance. Finally, we also highlight the positive effects of fusion modules, loss function, confidence process and BA for improving overall performance.
Novel View Synthesis (NVS) from unconstrained photo collections is challenging in computer graphics. Recently, 3D Gaussian Splatting (3DGS) has shown promise for photorealistic and real-time NVS of static scenes. Building on 3DGS, we propose an efficient point-based differentiable rendering framework for scene reconstruction from photo collections. Our key innovation is a residual-based spherical harmonic coefficients transfer module that adapts 3DGS to varying lighting conditions and photometric post-processing. This lightweight module can be pre-computed and ensures efficient gradient propagation from rendered images to 3D Gaussian attributes. Additionally, we observe that the appearance encoder and the transient mask predictor, the two most critical parts of NVS from unconstrained photo collections, can be mutually beneficial. We introduce a plug-and-play lightweight spatial attention module to simultaneously predict transient occluders and latent appearance representation for each image. After training and preprocessing, our method aligns with the standard 3DGS format and rendering pipeline, facilitating seamlessly integration into various 3DGS applications. Extensive experiments on diverse datasets show our approach outperforms existing approaches on the rendering quality of novel view and appearance synthesis with high converge and rendering speed.
In the past 30 years, Chinese enterprises have been a hot topic of discussion and concern among the general public in terms of economic and social status, ownership structure, business mechanism, and management level.Solving the problem of employment for the people is an important prerequisite for their peaceful living and work, as well as a prerequisite and foundation for building a harmonious society.The employment situation of private enterprises has always been of great concern to the outside world, and these two major jobs have always occupied an important position in the employment field of China that cannot be ignored.With the establishment of the market economy system, individual and private enterprises have become important components of the socialist economy, making significant contributions to economic development and social progress.The rapid development of China's economy, on the one hand, is the embodiment of the superiority of China's socialist market economic system, and on the other hand, it is the role of the tertiary industry and private enterprises in promoting the national economy.Since the 1990s, China's private enterprises have become a new economic growth point for local and even national countries, and are one of the important ways to arrange employment and achieve social stability.This paper studies the employment of private enterprises and individuals from the perspective of statistics, extracts relevant data from China statistical Yearbook, uses the relevant knowledge of statistics to process the data, obtains the conclusion and puts forward relevant constructive suggestions.
This paper introduces a novel continual learning framework for synthesising novel views of multiple scenes, learning multiple 3D scenes incrementally, and updating the network parameters only with the training data of the upcoming new scene. We build on Neural Radiance Fields (NeRF), which uses multi-layer perceptron to model the density and radiance field of a scene as the implicit function. While NeRF and its extensions have shown a powerful capability of rendering photo-realistic novel views in a single 3D scene, managing these growing 3D NeRF assets efficiently is a new scientific problem. Very few works focus on the efficient representation or continuous learning capability of multiple scenes, which is crucial for the practical applications of NeRF. To achieve these goals, our key idea is to represent multiple scenes as the linear combination of a cross-scene weight matrix and a set of scene-specific weight matrices generated from a global parameter generator. Furthermore, we propose an uncertain surface knowledge distillation strategy to transfer the radiance field knowledge of previous scenes to the new model. Representing multiple 3D scenes with such weight matrices significantly reduces memory requirements. At the same time, the uncertain surface distillation strategy greatly overcomes the catastrophic forgetting problem and maintains the photo-realistic rendering quality of previous scenes. Experiments show that the proposed approach achieves state-of-the-art rendering quality of continual learning NeRF on NeRF-Synthetic, LLFF, and TanksAndTemples datasets while preserving extra low storage cost.
Visual localization plays an essential role in a variety of different fields. The indirect learning based method obtains an excellent performance, but it requests a training process in the target scene before the localization. To achieve deep scene-independent localization, we start by proposing the representation called residual coordinate map between a pair of images. Based on the structure, we put forward a network called SILocNet with the proposed residual coordinate map as the output. The network consists of feature extraction, multi-level feature fusion and transformer based coordinate decoder. Moreover, considering the dynamic scene, we introduce an additional segmentation branch that distinguishes fixed and dynamic parts to promote network perception. With SILocNet in place, a cascaded localizer design is presented for reducing the accumulative error. Meanwhile, the simple mathematical analysis behind the cascaded localizers is also provided. To verify how well our algorithm could perform, we conduct experiments on static 7 Scenes, ScanNet and dynamic TUM RGB-D. In particular, we train the network on ScanNet and test it on 7 Scenes and TUM RGB-D to demonstrate the generality performance. All experiments demonstrate superior performance to other existing methods. Additionally, the effects of the cascaded localizer design, feature fusion, transformer based coordinate decoder and segmentation loss are also discussed.
Neural Radiance Field (NeRF) shows dramatic results in synthesising novel views. However, existing controllable and editable NeRF methods are still incapable of both fine-grained editing and cross-scene compositing, greatly limiting their creative editing as well as potential applications. When the radiance field is fine-grained edited and composited, a severe drawback is that varying the orientation of the corresponding explicit scaffold, such as point, mesh, volume, etc., may lead to the degradation of rendering quality. In this work, by taking the respective strengths of the implicit NeRF-based representation and the explicit point-based representation, we present a novel Rotation-Invariant Point-based NeRF (RIP-NeRF) for both fine-grained editing and cross-scene compositing of the radiance field. Specifically, we introduce a novel point-based radiance field representation to replace the Cartesian coordinate as the network input. This rotation-invariant representation is met by carefully designing a Neural Inverse Distance Weighting Interpolation (NIDWI) module to aggregate neural points, significantly improving the rendering quality for fine-grained editing. To achieve cross-scene compositing, we disentangle the rendering module and the neural point-based representation in NeRF. After simply manipulating the corresponding neural points, a cross-scene neural rendering module is applied to achieve controllable cross-scene compositing without retraining. The advantages of our RIP-NeRF on editing quality and capability are demonstrated by extensive editing and compositing experiments on room-scale real scenes and synthetic objects with complex geometry.
Recently, direct visual localization with convolutional neural networks has attracted researchers' attention with achieving an end-to-end process. However, on the one side, the lack of using 3D information leads to imprecise accuracy. Meanwhile, the single input image confuses the relocalization in the scenes that keep similar views at different positions. On the other side, the relocalization problem in variable or dynamic scenes is still challenging. Concentrating on these concerns, we propose two multitask relocalization networks called MMLNet and MMLNet+ for obtaining the 6-DoF camera pose in static, variable and dynamic scenes. Firstly, addressing the dataset lack of variable scenes, we construct a variable scene dataset with a semiautomatic process combining SFM and MVS algorithms with a few manual labels. Based on the process, three scenes covering an office, a bedroom and a sitting room are gathered and generated. Secondly, to enhance the perception between 2D images and 3D poses, we design a multitask network called MMLNet that regresses both camera pose and scene point cloud. Meanwhile, the Chamfer distance is joined into the original pose loss to optimize MMLNet. Moreover, MMLNet learns the pose trajectory feature by using LSTM layers to the additional pose array input, which meanwhile breaks through the limitation of single image input. Based on the MMLNet, aiming at dynamic and variable scenes, MMLNet+ outputs the auxiliary segmentation branch that distinguishes fixed, changeable or dynamic parts of the input image. Furthermore, we define the feature fusion block to implement the feature sharing among three tasks, further promoting the performance in dynamic and variable environments. Finally, experiments on static, dynamic and our constructed variable datasets demonstrate state-of-the-art relocalization performances of MMLNet and MMLNet+. Simultaneously, the positive effects of the pose learning part, reconstruction branch and segmentation task are also illustrated.
Extracellular vesicles (EVs) are increasingly recognized as important intermediaries of intercellular communication. They have significant roles in many physiological and pathological processes and show great promise as novel biomarkers of disease, therapeutic agents, and drug delivery tools. Existing studies have shown that natural killer cell-derived EVs (NEVs) can directly kill tumor cells and participate in the crosstalk of immune cells in the tumor microenvironment. NEVs own identical cytotoxic proteins, cytotoxic receptors, and cytokines as NK cells, which is the biological basis for their application in antitumor therapy. The nanoscale size and natural targeting property of NEVs enable precisely killing tumor cells. Moreover, endowing NEVs with a variety of fascinating capabilities via common engineering strategies has become a crucial direction for future research. Thus, here we provide a brief overview of the characteristics and physiological functions of the various types of NEVs, focusing on their production, isolation, functional characterization, and engineering strategies for their promising application as a cell-free modality for tumor immunotherapy.
This cross-sectional study assesses Medicaid spending associated with citrate-free vs original adalimumab.
In AR/MR applications, camera localization and object pose estimation both play crucial roles. The universality of learning techniques, often referred to as scene-independent localization and categorylevel pose estimation, presents challenges for both tasks. The two missions maintain close relationships due to the spatial geometry constraint, but differing task requirements result in distinct feature extraction. In this paper, we focus on simultaneous sceneindependent camera localization and category-level object pose estimation with a unified learning framework. The system consists of a localization branch called SLO-LocNet, a pose estimation branch called SLO-ObjNet, a feature fusion module for feature sharing between two tasks, and two decoders for creating coordinate maps. In SLO-LocNet, localization features are produced for anticipating the relative pose between two adjusted frames using inputs of color and depth images. Furthermore, we establish an image fusion module in order to promote feature sharing in depth and color branches. With SLO-ObjNet, we take the detected depth image and its corresponding point cloud as inputs, and produce object pose features for pose estimation. A geometry fusion module is created to combine depth and point cloud information simultaneously. Between the two tasks, the image fusion module is also exploited to accomplish feature sharing. In terms of the loss function, we present a mixed optimization function that is composed of the relative camera pose, geometry constraint, absolute and relative object pose terms. To verify how well our algorithm could perform, we conduct experiments on both localization and pose estimation datasets, covering 7 Scenes, ScanNet, REAL275 and YCB-Video. All experiments demonstrate superior performance to other existing methods. We specifically train the network on ScanNet and test it on 7 Scenes to demonstrate the universality performance. Additionally, the positive effects of fusion modules and loss function are also demonstrated.
Radio astronomy has emerged as a promising avenue for the detection and characterization of exoplanets, offering unique insights into the physical and atmospheric properties of these distant celestial bodies. This study commences with a historical overview of the early works by Zarka that laid the groundwork for the systematic exploration of radio exoplanet detection. It analyzes the theoretical principles and mechanisms that underpin the generation of radio emissions from exoplanetary systems. Detailed analysis delves into the specific instrumental capabilities of LOFAR, nenuFAR, and FAST, elucidating their respective strengths and limitations in detecting exoplanetary radio signals. In particular, the review points out the sensitivity and frequency coverage of each telescope and their potential to characterize a diverse range of exoplanets. Finally, this paper discusses the future prospects and potential synergies in advancing exoplanet radio detection. It concludes with an outlook on the role of radio astronomy in the exploration of exoplanetary systems and the exciting possibilities that lie ahead in understanding the diversity of exoplanets in the universe.
Augmented reality (AR) applications are increasingly being used in various fields (e.g., design, maintenance, assembly, repair, training, etc.), as AR techniques help improve efficiency and reduce costs. Moreover, collaborative AR systems extend applicability, allowing for collaborative environments for different roles. In this paper, we propose a multi-user collaborative AR system (aptly called the “multi-user collaborative system”, or MUCSys); it is composed of three ends—MUCStudio, MUCView, and MUCServer. MUCStudio aims to construct industrial content with CAD model transformation, simplification, database update, marker design, scene editing, and exportation, while MUCView contains sensor data analysis, real-time localization, scene loading, annotation editing, and virtual–real rendering. MUCServer—as the bridge between MUCStudio and MUCView—presents collaborative and database services. To achieve this, we implemented the algorithms of local map establishment, global map registration, optimization, and network synchronization. The system provides AR services for diverse industrial processes via three collaborative ways—remote support, collaborative annotation, and editing. According to the system, applications for cutting machines were presented to improve efficiency and reduce costs, covering cutting head designs, production line sales, and cutting machine inspections. Finally, a user study was performed to prove the usage experience of the system.
Visual localization plays an indispensable role in robotics. Both learning and hand-crafted feature based methods for relocalization process keep their effectiveness and weakness. However, current algorithms seldom consider these two kinds of features under one framework. In this paper, focusing on this task, we propose a novel relocalization framework for RGB or RGB-D data source, which is composed of coarse localization process by learning features and pose refinement by hand-crafted features. In particular, coarse stage contains deep point cloud generation and registration. In this stage, instead of regressing camera pose directly, the paper novelly designs a neural network called PGNet to construct sparse point cloud with RGB or RGB-D as inputs. Further more, by means of training set, hand-crafted feature space is established. Based on the obtained camera pose in coarse stage, accurate point-to-point correspondences are set up through searching the space. Then accurate camera pose is obtained by applying RANSAC to correspondences or solving PnP. Finally, experiments on both outdoor and indoor benchmark datasets demonstrate state-of-the-art performance over other existing methods.
This study aims to explore the role of long noncoding RNA (lncRNA)-HOX transcript antisense intergenic RNA (HOTAIR) in the occurrence and progression of glioma. Fresh glioma and normal brain tissues were classified into a glioma group (n = 67) and a normal group (n = 64) respectively. U87 cells were assigned into the blank, sh-NC, and sh-HOTAIR groups. Quantitative real-time polymerase chain reaction (qRT-PCR) was utilized to determine HOTAIR expression. Cell proliferation, cell cycle and cell apoptosis rates were detected by cell counting kit-8 (CCK-8) and flow cytometry (FCM). Scratch test and transwell assay were conducted for cell migration and invasion. Orthotopic glioma tumor model in nude mice was established by inoculating tumor cell suspension. Hematoxylin-Eosin (HE) staining was used to observe the growth and invasion of orthotopic glioma tumors. The expression of HOTAIR and cell viability was found to be lowest in the sh-HOTAIR group among the three groups. The sh-HOTAIR group exhibited a higher apoptotic rate and lower number of cell migration compared with the blank and sh-NC groups. Additionally, the speed of wound healing was slower, the migration distance decreased and the survival time of nude mice was extended in the sh-HOTAIR compared to the other groups. Moreover, the sh-HOTAIR group demonstrated reduced lesion sizes and inflammation, no convulsions or hemiplegia and lesser number of satellite metastases. Our findings support that down-regulation of HOTAIR could inhibit cell proliferation, promote cell apoptosis as well as suppress cell invasion and migration in the progression of glioma.