Accurate height estimation from monocular remote sensing imagery is a fundamental task with broad applications in urban planning and management. However, existing vision-based methods rely solely on visual features, which limits their ability to fully exploit height-related semantic information. Vision-language models, e.g., CLIP, offer a promising alternative by leveraging multimodal information, yet directly adapting them to height regression remains challenging due to their weak perception of continuous numerical values. To overcome this limitation, we propose HeightCLIP, a novel vision-language framework tailored for height estimation from monocular remote sensing imagery. The task-adaptive textual prompts and image-side representations are optimized under height supervision to derive pixel-text similarity features for accurate height estimation. Specifically, we design an interval-based prompting strategy that integrates learnable context tokens and a fixed natural language template with parameterized interval values, allowing CLIP to better comprehend height-related semantics. In addition, we incorporate a reciprocal cross-attention fusion mechanism to facilitate bidirectional interaction between visual and textual features, thereby improving cross-modal feature integration. Finally, building upon this enriched feature space, we impose cross-modal consistency constraints to ensure pixel-level correspondence between height descriptions and visual features, providing more informative representations for subsequent height prediction. Extensive experiments on three benchmark datasets i.e., Vaihingen, Potsdam, and Data Fusion Contest 2019 (DFC2019), demonstrate that HeightCLIP achieves competitive or superior performance compared with existing state-of-the-art methods, and comprehensive ablation studies confirm the effectiveness of each proposed component. The source code will be made available at https://github.com/songtaowhu/HeightCLIP.

Geographic height information describes the vertical spatial structure of the city and serves as important foundational data for urban management. Obtaining height information from single-view remote sensing images is a relatively low-cost and convenient approach. However, there exist several bottlenecks in the current methods for inferring height information from monocular remote sensing images, such as difficulties in learning 3-D semantic information and accurately fitting the height morphology in various local scenes. In this study, we address these challenges by proposing a morphological-priors-guided network, termed MPG-Net, for accurate height estimation from single-view remote sensing images. First, considering the semantic morphological priors, we propose to explicitly enhance the 3-D visual cues (e.g., co-occurrence relationship between shadow buildings and shadow trees) and simultaneously design a semantic booster composed of a two-stream network with a multilevel cross-stream attention fusion mechanism to facilitate the 3-D feature learning for monocular height estimation. Second, taking into account the height distribution priors, we propose a scalable bins module that can create fully adaptive bins within a flexible height range for each input image, leading a more accurate delineation of height distribution pattern. The proposed MPG-Net is comprehensively evaluated on two datasets of different scenes (i.e., ISPRS Vaihingen and Potsdam datasets). Results indicate that the proposed MPG-Net significantly outperforms the existing methods, with the lowest root-mean-square error of 1.613 m and 1.947 m on Vaihingen and Potsdam, respectively. Furthermore, extensive ablation studies demonstrate the contribution of each designed component in the proposed method.

Vehicle detection in satellite videos is crucial for large-scale traffic monitoring and urban management, yet remains challenging due to the extremely small object size and limited appearance features. Existing methods generally focus on extracting spatiotemporal information at the local pixel level while neglecting the long-range dependencies between vehicle instances. This insufficient spatiotemporal feature aggregation limits the accuracy of detection. To address this issue, we propose a coarse-to-fine multilevel spatiotemporal attention network (MLSTA-Net) for detecting tiny vehicles in satellite videos. Specifically, a pixel-level spatiotemporal attention module is introduced, which leverages motion priors to guide the aggregation of spatiotemporal features at the pixel level, thereby enhancing the fine-grained representation of targets and generating coarse vehicle detection results. Subsequently, an instance-level spatiotemporal attention module is developed to refine initial detections by modeling the spatial relationships of instances within a single frame and the temporal consistency of instances across multiple frames. Experiments on VISO and SAT-MTB datasets demonstrate that the MLSTA-Net method achieves superior performance over state-of-the-art object detection approaches.

The mainstream instance segmentation in remote sensing images takes the way of “Detect then Segment.” Anchor-free detectors get rid of the predefined anchors and can be used for proposal generation. However, in the existing methods based on anchor-free detectors, incompact proposals are directly used for segmentation, which may result in incomplete mask segmentation. In addition, the commonly used mask segmentation module is based on the trimmed features, which is not sufficient for accurate mask segmentation due to the loss of spatial details. In this letter, an anchor-free network is proposed, for instance, segmentation in remote sensing images. In order to obtain more compact proposals, a box refinement module, which predicts the distance offsets of a pixel to four sides of a proposal, is designed for initial proposals generated by an anchor-free detector. Based on the refined proposals, a saliency supplement module is designed to obtain accurate instance-wise masks by embedding the saliency map into the coarse masks. Compared with other methods, the proposed method conducted on two remote sensing datasets achieves the optimal performance, with an average precision (AP) of 0.652 and 0.678.

建筑物作为三维模型的主体,其矢量化主要依赖人工勾画,虽有采用深度学习等方法进行建筑物提取的研究,但依然需要标注大量样本.针对上述问题,本文以天津市典型区域为试验区,提出一种融合高度和光谱信息的倾斜摄影数据建筑物自动提取方法.首先,通过高度初始分割、植被信息滤除、形态学后处理等,逐步优化建筑物提取结果,实现建筑物信息的自动提取,建筑物的总体识别精度达到94％.然后,通过对建筑物轮廓进行矢量化和规则化,在地理信息平台中实现了建筑物的对象化查询,拓展了实景三维模型的应用深度.

为了充分融合不同深度学习模型在建筑提取中的互补信息,该文提出一种基于深度学习概率决策融合的高分辨率影像建筑物提取方法,将不同深度学习模型的类别分割概率进行融合作为最终建筑提取的依据,以实现不同模型之间的优势互补,最后采用形态学后处理方法进一步优化建筑提取结果.采用3组不同分辨率,具有多种地物形态的建筑数据集验证该文方法的有效性.实验表明该文提出的概率决策融合方法取得了满意的精度(F指数分别为92.45％,90.56％,79.95％),优于单一模型的结果,并且显著提升了建筑提取结果的可靠性.

Deep-learning technologies, especially convolutional neural networks (CNNs), have achieved great success in building extraction from areal images. However, shape details are often lost during the down-sampling process, which results in discontinuous segmentation or inaccurate segmentation boundary. In order to compensate for the loss of shape information, two shape-related auxiliary tasks (i.e., boundary prediction and distance estimation) were jointly learned with building segmentation task in our proposed network. Meanwhile, two consistency constraint losses were designed based on the multi-task network to exploit the duality between the mask prediction and two shape-related information predictions. Specifically, an atrous spatial pyramid pooling (ASPP) module was appended to the top of the encoder of a U-shaped network to obtain multi-scale features. Based on the multi-scale features, one regression loss and two classification losses were used for predicting the distance-transform map, segmentation, and boundary. Two inter-task consistency-loss functions were constructed to ensure the consistency between distance maps and masks, and the consistency between masks and boundary maps. Experimental results on three public aerial image data sets showed that our method achieved superior performance over the recent state-of-the-art models.

针对当前基于卷积神经网络的建筑提取方法在数据输入层对建筑语义信息利用不足的问题,提出一种融合类别语义特征的卷积神经网络建筑物提取方法.首先,从影像上提取形态学建筑指数,该指数能够直接表征建筑覆盖信息,是一种高层次的语义特征;然后,将该建筑语义特征作为原始影像的补充通道一起输入到卷积神经网络模型中训练,从数据层进一步增强建筑与背景的可分性.采用国际上公开的具有多种地物形态的标准建筑数据集验证本文方法的有效性.实验表明,所提出的方法取得了满意的精度(准确率为85.6％,召回率为93.1％,F指数为88.4％),相对于原始的RGB影像输入,建筑物语义特征的加入整体上提升了建筑物提取的精度.

Urban villages (UVs), a special type of informal settlement in China. In this study, we proposed a method for UV extraction from high-resolution remote sensing imagery using landscape semantic metrics that can describe the complicated scene of UVs. In addition, an “uncertainty-feedback” strategy was proposed for large-scale practicable UV mapping. The experiment was performed in the urban areas of Guangzhou, with overall accuracy larger than 90%. The results reveal that the landscape semantic metrics have better ability to describe the essential characteristics of UVs compared to the traditional spectral and textural features. Besides, the “uncertainty-feedback” strategy can make full use of the classification reliability output by the machine learning, and produce more accurate UV mapping results with limited manual intervention. Thus, the proposed method can be effectively applied to large-scale UV extraction and mapping.

Vehicle detection in aerial images is an important and challenging task in the field of remote sensing. Recently, deep learning technologies have yielded superior performance for object detection in remote sensing images. However, the detection results of the existing methods are horizontal bounding boxes that ignore vehicle orientations, thereby having limited applicability in scenes with dense vehicles or clutter backgrounds. In this article, we propose a one-stage, anchor-free detection approach to detect arbitrarily oriented vehicles in high-resolution aerial images. The vehicle detection task is transformed into a multitask learning problem by directly predicting high-level vehicle features via a fully convolutional network. That is, a classification subtask is created to look for vehicle central points and three regression subtasks are created to predict vehicle orientations, scales, and offsets of vehicle central points. First, coarse and fine feature maps outputted from different stages of a residual network are concatenated together by a feature pyramid fusion strategy. Upon the concatenated features, four convolutional layers are attached in parallel to predict high-level vehicle features. During training, task uncertainty learned from the training data is used to weight loss function in the multitask learning setting. For inferencing, oriented bounding boxes are generated using the predicted vehicle features, and oriented nonmaximum suppression (NMS) postprocessing is used to reduce redundant results. Experiments on two public aerial image data sets have shown the effectiveness of the proposed approach.

以深圳电力线巡线时所获得的高分辨率航空影像为例,提出了基于集成卷积神经网络的架空铁塔护坡自动提取方法.该方法首先通过Faster R-CNN目标检测网络在多幅航空影像中框取护坡目标,然后通过SegNet语义分割网络进一步分割目标框中的护坡与背景,获取精确的护坡掩膜信息.与直接应用SegNet模型的"一步式"护坡提取方法相比,"先定位后分割"的方法在2个实验数据上的虚警率下降超过38.1％;与基于Mask R-CNN卷积神经网络提取方法相比,该方法在2个数据上的正确率均高于98％,优于Mask R-CNN所达到的95.96％.