The height information of complex urban scenes is essential for urban understanding and 3-D reconstruction. Fine-grained stereo matching results provide detailed height relation of urban scenes, which is vital for localization, mapping, and 3-D object modeling, yet existing methods face challenges, such as limited scope of disparity prediction and low geometry consistency. To address these issues, a relation-geometry enhanced (RGE) network is proposed for stereo matching of optical satellite stereo image. To be specific, a pretrained monocular depth estimation model named DepthAnything V2 is applied to generate the relative relation of the images as well as to obtain features focused on the depth information. Given the produced depth map, a relation guided geometry fusion module and a relation constraint feature optimization module are introduced to generate the RGE information. Moreover, a relation-geometry gated recurrent unit based disparity update module is designed for precise stereo matching. The evaluation results of the proposed method on the US3D dataset and WHU-Stereo dataset demonstrated the effectiveness of the RGE-Net. The RGE-Net outperformed other stereo matching algorithms on the six cities by obtaining both the accurate disparity results and precise details of different regions.
Integrated 3D reconstruction from aerial-ground images is essential for generating high-precision urban 3D models, yet severe variations in viewpoint, scale, and rotation make robust feature matching highly challenging. To address these limitations, this study introduces a rotation-robust detector-free matching network coupled with multi-view track refinement for incremental Structure from Motion (ISfM). The proposed workflow features four key modules. First, rotation-aware feature extraction replaces traditional convolutions with an Omnidirectional State Space Block (OSS Block) that selectively scans across eight symmetrical directions to model long-range spatial dependencies and synthesize rotation-invariant feature maps. Second, multi-scale attention transformation utilizes quadtree attention to build a hierarchical token pyramid that isolates high-association token regions and discards irrelevant areas, capturing long-range context with linear computational complexity. Third, bi-directional feature matching executes a symmetric coarse-to-fine matching scheme where coarse alignment computes dual-direction Softmax confidence matrices under mutual nearest neighbor constraints, and fine alignment uses a multi-layer perceptron to regress sub-pixel coordinate offsets. Finally, multi-view track refinement employs an integrated indexing structure to evaluate localized spatial proximity and link disjoint sub-tracks to the highest-confidence anchor point, ensuring stable feature repeatability across the ISfM pipeline. By using real aerial-ground datasets, experimental results demonstrate that the proposed method improves AUC at 5° pose error by 93.9
Driven by the growing demand for high-precision 3-D spatial data in smart cities, accurate building instance segmentation from airborne light detection and ranging (LiDAR) has become critical. Traditional methods often struggle in complex urban environments, while existing deep learning approaches primarily rely on local geometry and neglect explicit building morphology. To address these limitations, we propose a contour-guided hybrid graph network (CG-HGN) for building instance segmentation. CG-HGN incorporates contour to explicitly encode building morphology and leverages a hybrid graph architecture: a superpoint graph captures semantic features, and an edge graph models intersuperpoint boundary connectivity. By formulating instance segmentation as a graph connectivity inference problem, CG-HGN improves boundary precision and alleviates over-segmentation. Experiments on four datasets with distinct characteristics demonstrate that CG-HGN consistently outperforms existing methods in terms of average precision (AP) and mean coverage (mCov), particularly in sparse, heavily occluded, or structurally complex areas. Cross-dataset evaluations further confirm the strong generalization ability. CG-HGN provides an effective solution for high-quality building instance segmentation in real-world airborne LiDAR applications.
With the rapid development of the low-altitude economy,the construction of low-altitude transportation infrastructure has progressed from conceptual exploration to scaled practice.The digital air-route network not only guides the construction of facility and air-internet networks,but also provides the service network with followable routes,making it a priority task in building low-altitude transportation infrastructure.However,the existing methods for constructing digital air-route networks insufficiently consider risk quantification,lack structured topology,and omit essential route attributes.In addition,they have not clarified the required types and geometric accuracies of geographic and constraint elements in low-altitude environments.Therefore,it is necessary to further improve relevant methodologies to better guide the construction and application of digital air-route networks for the low-altitude economy.To address these issues,this study begins with the interaction mechanism between unmanned aerial vehicles(UAVs)and their geographic constraint environments.It identifies the categories of geographic and constraint elements required for digital air-route network construction and specifies the geometric accuracy requirements for geographic elements.The feasibility and adequacy of spaceborne remote sensing techniques for acquiring these elements are analyzed.Based on these findings,a construction method for digital air-route networks is proposed,integrating geographic and constraint information while jointly optimizing topological structure and risk.Field experiments are conducted in Anyang to verify the feasibility of this method,including validation of the spaceborne geographic information base,meteorological constraints,the digital air-route network itself,and the communication and positioning quality along the routes.Results show that spaceborne remote sensing data achieve a DSM vertical accuracy better than 2m,a building white model accuracy of 3.83m,an overall obstacle recognition accuracy of 80.77%,and a land cover classification accuracy of 79.5%.These results collectively meet the meter level geometric and surface-attribute resolution requirements for digital air-route network construction.Compared with manually designed routes,the air-route network generated with this method reduces route length by 7.6%,cruise time by 12.6%,and the proportion of high-risk segments by 7.6%,while increasing the nonlinearity coefficient by 8.2%.Compared with pilot-planned ad-hoc routes,route length decreases by 4.2%,cruise time by 3.4%,and the nonlinearity coefficient improves by 18.5%.Overall,the proposed method effectively improves airspace utilization,reduces flight risk,and enhances flight efficiency,fulfilling the operational requirements for UAVs to fly,fly safely,and fly efficiently in large-scale low-altitude operations.
Vectorized high-definition (HD) map construction is formulated as the task of classifying and localizing typical map elements based on features in a bird's-eye view (BEV). This is essential for autonomous driving systems, providing interpretable environmental structured representations for decision and planning. Remarkable work has been achieved in recent years, but several major issues remain: (1) in the generation of the BEV features, single modality methods suffer from limited perception capability and range, while existing multi-modal fusion approaches underutilize cross-modal synergies and fail to resolve spatial disparities between modalities, resulting in misaligned BEV features with holes; (2) in the classification and localization of map elements, existing methods heavily rely on point-level modeling information while neglecting the information between elements and between point and element, leading to low accuracy with erroneous shapes and element entanglement. To address these limitations, we propose SuperMapNet, a multi-modal framework designed for long-range and high-accuracy vectorized HD map construction. This framework uses both camera images and LiDAR point clouds as input. It first tightly couples semantic information from camera images and geometric information from LiDAR point clouds by a cross-attention based synergy enhancement module and a flow-based disparity alignment module for long-range BEV feature generation. Subsequently, local information acquired by point queries and global information acquired by element queries are tightly coupled by three-level interactions for high-accuracy classification and localization, where Point2Point interaction captures local geometric consistency between points of the same element, Element2Element interaction learns global semantic relationships between elements, and Point2Element interaction complement element information for its constituent points. Experiments on the nuScenes and Argoverse2 datasets demonstrate high accuracy, surpassing previous state-of-the-art methods (SOTAs) by 14.9%/8.8% and 18.5%/3.1% mAP under the hard/easy settings, respectively, even over the double perception ranges (up to 120 m in the X-axis and 60 m in the Y-axis). The code is made publicly available at https://github.com/zhouruqin/SuperMapNet.
With the advancement of satellite imaging technology, acquiring high-resolution multiview satellite imagery has become increasingly accessible, enabling efficient and location-independent ground model reconstruction. However, traditional stereo matching methods often miss fine details, while neural radiance fields (NeRFs) provide high-quality reconstructions at the cost of prohibitively long training times. Moreover, challenges such as low visibility of building facades, illumination inconsistencies, and weakly textured regions further hinder accurate geometry recovery. To tackle these issues, we propose Sat-DN, a novel depth- and normal-guided surface reconstruction framework for satellite imagery. Given multiview satellite images and predicted depth maps, Sat-DN employs a progressively trained multiresolution hash grid architecture to efficiently represent scene geometry. Our pipeline integrates explicit depth supervision and surface normal consistency constraints to preserve structural boundaries and planar regularities. The progressive training strategy first fits low-frequency geometric structures and then refines high-frequency details under the guidance of depth and normals. Extensive experiments on the Data Fusion Contest 2019 dataset demonstrate that Sat-DN significantly outperforms traditional multiview stereo and NeRF-based baselines in both accuracy and efficiency.
Building information extraction holds significant application value in smart city development, urban planning, and management. With the accelerating process of urbanization, mid- and high-rise buildings are increasingly prevalent. In orthophotos, the roofs of tall buildings often do not fully overlap with their footprints. In satellite images from oblique angles, buildings may also be obstructed or affected by shadows. Therefore, building information extraction should evolve from a roof-only extraction task to a comprehensive task that includes both roofs and facades. Current methods predominantly employ convolutional neural networks (CNNs) and Transformer models, focusing on describing building boundary and global features. However, these methods have the following limitations: insufficient utilization of information between pixels and limited spatial information recovery capabilities in decoders. This makes it difficult to distinguish between roofs and facades, and the morphological structure of buildings is challenging to maintain. To address these issues, this paper proposes a new network architecture-NeSF-Net, designed to focus on the accurate extraction of roofs and facades. NeSFNet consists of two core modules: the neighborhood relationship awareness module (NRAM) and the scalefrequency modulation decoder (SFMD). NRAM enhances the connectivity between pixels by constructing subneighborhood relationship awareness in the latent space of deep features, effectively improving the integrity of the segmentation results. SFMD significantly reduces the loss of spatial information during the upsampling process by thoroughly extracting and integrating the scale and frequency features of buildings in the decoder. Experiments were conducted on the BANDON dataset, which contains images captured from oblique angles. The proposed method achieved a mIoU of 72.71 % and an F1 score of 83.04 %, outperforming state-of-the-art segmentation methods. The performance in facade extraction was particularly notable, with a mIoU score exceeding the second-best method by 4.92 %. Additionally, generalization experiments were conducted using GaoFen-7 satellite images, taking Shenzhen as a case study. The results demonstrate that the proposed method exhibits good generalization and robustness.
Extracting powerline point clouds from airborne LiDAR data and conducting 3D reconstruction has become a critical technical support for automatic transmission corridor inspection. To enhance data processing efficiency, this paper proposes an automatic method for span segmentation of powerline point clouds that accounts for adjacent powerline interference, aiming to provide “clean” data for the automatic reconstruction of powerline catenary curve models of each span. This method tackles a key challenge in airborne LiDAR data: interference from adjacent or cross-over powerlines when automatically extracting main-line pylon positions and powerline points. Leveraging the spatial relationship between pylons and powerlines in LiDAR point clouds, we developed a fast density clustering algorithm based on a novel point-counting grid (PCGrid), which greatly accelerates DBSCAN clustering while adaptively extracting main-line pylons and powerline point clouds. The method proceeds in three steps: first, using 2D density clustering to extract reliable pylon positions and 3D density clustering to filter out non-main-line point clouds; second, verifying pylon connection combinations via main-line point clouds and identifying the longest line in the connection matrix as the pylons of the main powerline; and third, assigning powerline points to their corresponding spans for segmented reconstruction. Experimental results demonstrate that the proposed PCGrid structure not only significantly improves clustering efficiency, but also enables a fully automated span segmentation process that effectively suppresses adjacent powerline interference, highlighting the novelty of integrating efficient PCGrid-based clustering with spatial-relationship-driven pylon verification into a unified framework for reliable 3D powerline reconstruction.
Dense matching plays an important role in 3D modeling from satellite images. Its purpose is to establish pixel-by-pixel correspondences between two stereo images. This study presents a learning-based dense matching approach that integrates selfsupervised learning with a multi-head attention mechanism to achieve feature fusion. Since stereo matching in satellite datasets is restricted by the disparity range, the pixel-by-pixel method can reduce the limitation. In the feature extraction module, we have performed attention-based in-depth learning on the smallest-scale feature using the self-supervised DINO. In addition, a CEP (Context-Enhanced Path) module is added outside the main matching path, and continuously enhanced position embedding is used to improve relative position encoding. The effectiveness of this method has been demonstrated through experiments on the US3D and WHU-Stereo datasets.
Semantic segmentation of LiDAR point clouds in powerline corridor environments is crucial for the intelligent inspection and maintenance of power infrastructure. However, existing deep learning methods often underperform in such scenarios due to severe class imbalance, sparse and long-range structures, and complex elevation variations. We propose EMPower-Net, an Elevation-Aware Multi-Resolution Network, which integrates an Elevation Distribution (ED) module to enhance vertical geometric awareness and a Multi-Resolution (MR) module to enhance segmentation accuracy for corridor structures with varying object scales. Experiments on real-world datasets from Yunnan and Guangdong show that EMPower-Net outperforms state-of-the-art baselines, especially in recognizing power lines and towers with high structural fidelity under occlusion and dense vegetation. Ablation studies confirm the complementary effects of the MR and ED modules, while transfer learning results reveal strong generalization with minimal performance degradation across different powerline regions. Additional tests on urban datasets indicate that the proposed elevation features are also effective for vertical structure recognition beyond powerline scenarios.
The height information of different semantic objects is crucial for urban understanding and three-dimensional (3-D) reconstruction. Fine-grained height maps provide essential details for localization, mapping, and 3-D modeling. However, single-view height estimation from remote sensing images faces challenges in handling complex pixel relations and noisy labels. This study introduces a relation-aware and edge-preserving height refinement network (RAEPHR-Net) for single-view height estimation. The proposed network incorporates a progressively relation mining and edge preservation module to generate smooth, difference, edge, direction maps, and a direction-weighted relation refinement module to refine pixel heights based on mined relations. The refined height map is used as a pseudolabel to mitigate label noise through self-supervision. Experiments on Vaihingen, Potsdam, and DFC2019 datasets demonstrate RAEPHR-Net's superior performance in accurate height estimation and semantic detail preservation compared to existing methods. The complexity and efficiency of the proposed method also outperformed with comparison methods.
Most transformer-based methods rely on pretraining weights on large datasets such as Imagenet or pretraining from specific change detection (CD) datasets and then fine-tuning on the target dataset. When the target dataset significantly diverges from the dataset used for pretraining, the model's ability to generalize to remote sensing imagery may be compromised due to the domain gap. In this letter, we propose HierFormer, which has the advantage of processing semantic features hierarchically, using simple operations for shallow features, spatial position transformation for middle-level features, and channel information interaction for high-level features. In addition, we propose a local-global-aware (LGA) attention block, which reduces the computational overhead of self-attention by sparse attention and increases the locality inductive bias (LIB) of the transformer by focusing attention on the local region and sparse part of the global region, which enables the model to be trained from scratch on small to medium-sized CD datasets. Finally, a new feature fusion decoder (FFD) is proposed to fuse the bitemporal features, which reweights the features of each channel through attention mechanism. Compared with other transformer-based or transformer-CNN-based hybrid networks, our method significantly improves F1, reaching 91.56% and 97.56% on the LEVIR-CD and CDD-CD change detection datasets. Our code is available at https://github.com/WesternTrail/HierFormer.
Disparity estimation is a key technology for recovering 3-D information from satellite imagery. Currently, very high-resolution images pose great challenges for context modeling in disparity estimation. While Transformer-based approaches outperform convolutional neural networks in long-distance modeling, their computational complexity necessitates the reduction of images to smaller patches during initial image processing, leading to substantial loss of contextual information and presenting challenges in ill-posed regions. To address these issues, we propose an end-to-end disparity estimation model, termed Mamba-based feature extraction and multifrequency information fusion for stereo matching of high-resolution satellite images. First, a Mamba-based feature extraction module similar to the U-Net architecture is employed to extract global information from stereo image pairs. Second, to address the challenges of multiscale objects matching and ill-posed regions in satellite imagery, a channel-spatial attention-enhanced multifrequency information fusion module is proposed, which effectively integrates multifrequency information contained in multiscale cost volumes. Finally, in the refinement module, we incorporate gradient data containing high-frequency information and original data with rich intensity information to extract convex weight combinations to obtain the full resolution disparity map, thereby updating a more accurate disparity map with enriched details and refined structures. In the experimental tests, systematic evaluation was conducted by using publicly available datasets, and the proposed algorithm was also compared with traditional and recent stereo matching techniques. Our method ranks first on the WHU-Stereo dataset, significantly outperforming the second-best method by 13.26% in EPE and 20.59% in D1.
Plane segmentation in three dimensions is a crucial step for many applications. A recent optimal-vector-field (OVF) technique demonstrated good generality across a variety of models. However, OVF is a rough approach that results in under-segmentation and missing points due to loss of details. Hence, this paper presents anew plane segmentation method that uses the detail-preserving OVF method to address these problems. There are three improvements to our proposed segmentation method. (1) To enlarge the vector difference between points on different planes, we split the model into a set of planar primitives leveraging the fine planar primitives extraction method, and then estimate the normal of each point in the primitive as the vector field. (2) We define a point-based Laplace operator to improve the vector field optimization process, thereby enhancing the accuracy of OVF for detail detection. (3) We innovatively take the magnitude of optimal-vector-field as the criterion for planar primitive-based growth to obtain the final segmentation result. The evaluation of four datasets shows that our method achieves higher average precision and recall than the OVF method by 16.43% and 20.79% respectively, and the global consistency error (GCE) decreases by 6.62%. The evaluation indicates that our method is capable of preserving finer details.
In recent years, change detection has been a hot research topic in remote sensing. Previous research has focused on binary change detection (BCD), limiting its practical applications. Therefore, semantic change detection (SCD), which can detect multiple change classes, is gradually becoming a more mainstream task. Most existing SCD methods use convolutional neural networks as the backbone to extract multiscale features and use relatively simple decoder structures, leading to unsatisfactory detection accuracy. We propose a multitask network for SCD, and in the encoder, given the great success of segment anything module (SAM) and vision transformer (VIT) in the field of general-purpose segmentation task, we introduce SAM-VIT into the backbone to enhance the encoder's ability to capture long-range contextual semantic relationships. We propose a transformer-based decoder structure for the semantic segmentation branch to extract local and global features effectively. We propose a convolutional attention-based change extractor for the BCD branch to enhance temporal information fusion. Also, we analyze in detail the semantic inconsistency that affects the performance of SCD. First, we introduce contrastive loss to establish the correlation between the output features of the BCD branch and the segmentation branch. Second, we design a bitemporal graph semantic interaction module to maintain semantic consistency between the output features of the two segmentation branches; the module assigns pixels with different land cover types to the corresponding graph nodes based on clustering techniques and then uses cross-attention to model the correlation between bitemporal semantic features in the graph space. Finally, a self-learning training scheme based on pseudolabel further mitigates the problem of semantic inconsistency. SCDVit achieves state-of-the-art performance on two popular high-resolution datasets. Meanwhile, adequate quantitative and qualitative analyses highlight the potential of SAM-VIT for change detection and the effectiveness of the module designed based on semantic consistency.
In photogrammetry and remote sensing, disparity estimation of satellite images has been a significant and challenging task, holding crucial importance for research and applications in this domain. Recent years have seen substantial progress in stereo matching methods, but challenges remain significant in ill-posed regions. Although deep learning-based stereo matching methods outperform traditional approaches in terms of performance and speed, their limited receptive field makes it difficult for networks to establish long-distance dependencies. This poses challenges in ill-posed areas such as textureless regions, repetitive patterns, and occluded areas. This paper proposes an end-to-end model for high-resolution satellite remote sensing images. First, in the feature extraction stage, we use two independent Transformer and CNN modules to extract global and local features of stereo image pairs. Subsequently, by designing effective fusion strategies, we merge these two types of features to obtain richer and more accurate feature representations. Next, we utilize multi-scale features to construct multi-level cost volumes, supervising each level of cost volume from coarse to fine. This allows lower-level cost volumes to provide prior knowledge to higher-level cost volumes, guiding them to acquire richer and more accurate information. Finally, we employ a ConvGRU-based recurrent module in the refinement module on geometrically encoded cost volumes containing geometric and contextual information to iteratively update disparity maps with finer details and structures. In experiments, we validate our approach using publicly available datasets and compare it with traditional methods. Experimental results demonstrate significant performance improvements in stereo matching tasks, proving the effectiveness of our proposed method.
The traditional method for extracting the heights of urban buildings involves utilizing dense matching algorithms on stereo images to generate a digital surface model (DSM). However, for urban buildings, the disparity discontinuity issue that troubles the dense matching algorithm makes the elevations of high-rise buildings and the surrounding areas inaccurate. The occlusion caused by trees in greenbelts makes it difficult to accurately extract the ground elevation around the building. To tackle these problems, a method for building height extraction from Gaofen-7 (GF-7) stereo images enhanced by contour matching is presented. Firstly, a contour matching algorithm was proposed to extract accurate building roof elevation from GF-7 images. Secondly, a ground filtering algorithm was employed on the DSM to generate a digital elevation model (DEM), and ground elevation can be extracted from this DEM. The difference between the rooftop elevation and the ground elevation represents the building height. The presented method was verified in Yingde, Guangzhou, Guangdong Province, and Xi’an, Shaanxi Province. The experimental results demonstrate that our proposed method outperforms existing methods in building height extraction concerning accuracy.
Affected by the vertical expansion of the city and non-nadir imaging, the roof of a building cannot completely overlap with its bottom footprint. Therefore, it is difficult to obtain the actual precise location of the building only by extracting the roof of the building. This paper proposes a new model framework that embeds the Axial Feature Enhancement Module in the network to represent the spatial dependence of the building roof and facade, and relies on the axial feature enhancement loss to constrain the network to learn the category distribution of the horizontal and vertical axes. As far as we know, this is the first time that the importance of building facade information is considered to carry out research on building roof and facade extraction. The experimental results also demonstrate the effectiveness of the method proposed in this article.
This paper primarily presents a parallel incremental Structure from Motion (ISfM) solution for large-scale images captured by unmanned aerial vehicles (UAVs). The core ideas are a local connection -constrained edge weighting strategy for match graph construction and an anchor -free parallel merging algorithm for the merged model generation. First, an effective algorithm is employed to retrieve spatially overlapped match pairs, utilizing the global descriptor for image representation and the graph indexing for nearest neighbor searching. Second, match pairs are used to create an undirected weighted match graph that is weighted by the local connection strength of the image. This match graph is then used to achieve parallel ISfM through graph clustering. Third, an anchor -free cluster merging algorithm, called AFP-Merging, is then designed by taking advantage of the independent connection between clusters, which increases the merging efficiency and stability. For robust estimation, AFP-Merging is implemented via a bidirectional mean square reprojection error. Finally, extensive evaluation and analysis have been carried out to verify its validation using large-scale UAV datasets captured from classical oblique photogrammetry and recent optimized views photogrammetry. Experiment results show that the proposed solution can generate more compact scene clusters and achieve a speedup ratio greater than 9.0 in cluster merging; compared with recent parallel ISfM, its orientation accuracy is higher in both relative bundle adjustment (BA) without GCPs (Ground Control Points) and absolute BA with GCPs. For the orientation of very large-scale UAV images, it has been successfully applied to a dataset containing ninety thousand images over an area of 50.0 km 2 . The proposed method provides a more efficient and reliable parallel SfM solution. The executable tool is made publicly available 1 .
In recent years, object detection on visible (RGB) and infrared (IR) has gained significant attention as a promising solution for robust detection in complex scenarios, especially in low-light conditions. With the help of IR images, object detectors have become more reliable and robust in practical condition by combining the RGB and IR information. Despite significant progress in this field, current methods ignore the distinct characteristics of the two modalities when extracting features. RGB images contain detailed texture and color information, which means they have many high-frequency signals. Meanwhile, IR images have smoother textures and edges but clear shapes, indicating a significant amount of low-frequency information. We must consider the differences between the two modalities when extracting corresponding features. To address this issue, we propose a novel network architecture: the frequency mining and complementary fusion network (FMCFNet), which accounts for the intermodal variability. Our network contains two critical modules: the frequency feature extraction (FFE) module and the complementary fusion (CF) module. The FFE module utilizes filters of varying kernel and pooling sizes to extract features with diverse frequency information and then adaptively selects the most responsive frequency component. The CF module uses the similarity scores generated by cross attention to model the interactions between two modalities. Comprehensive experimental results demonstrate that our method can effectively combine RGB-IR complementary information, achieving robust detection results.