The increasing accessibility of remotely sensed data and their potential to support large-scale decision-making have driven the development of deep learning models for many Earth Observation tasks. Traditionally, such models rely on large datasets. However, the common assumption that larger training datasets lead to better performance tends to overlook issues related to data redundancy, noise, and the computational cost of processing massive datasets. Effective solutions must therefore consider not only the quantity but also the quality of data. Towards this, in this paper, we introduce six basic core-set selection approaches – that rely on imagery only, labels only, or a combination of both – and investigate whether they can identify high-quality subsets of data capable of maintaining – or even surpassing – the performance achieved when using full datasets for remote sensing semantic segmentation. We benchmark such approaches against two traditional baselines on three widely used land-cover classification datasets (DFC2022, Vaihingen, and Potsdam) using two different architectures (SegFormer and U-Net), thus establishing a general baseline for future works. Our experiments show that all proposed methods consistently outperform the baselines across multiple subset sizes, with some approaches even selecting core sets that surpass training on all available data. Notably, on DFC2022, a selected subset comprising only 25% of the training data yields slightly higher SegFormer performance than training with the entire dataset. This result shows the importance and potential of data-centric learning for the remote sensing domain. The code is available at https://github.com/keillernogueira/data-centric-rs-classification/.

When a large disaster strikes, responders need a map of which buildings are damaged within hours. The models that do well on public benchmarks assume matched before-and-after imagery and a training set drawn from similar past events, and neither is usually available for a new disaster in its first day. We present HASTE (High-speed Assessment and Satellite Tracking for Emergencies), a no-code web platform that lets analysts who are not machine learning engineers produce per-building damage maps from post-disaster satellite imagery. HASTE implements two methods that share one interface. The first requires the user to label polygons over the post-disaster scene, trains a small semantic segmentation model on that single scene, runs it over the whole image, and joins the per-pixel output to existing building footprints. The second embeds every footprint with a pretrained vision model, requires the user to label a handful of buildings, and fits a logistic regression in the browser that scores the rest of the scene in seconds. We describe the platform, both methods, and the engineering that supports them. We also report preliminary experiments on xBD showing that foundation-model embeddings pooled over footprints separate damaged from intact buildings using post-disaster imagery alone, matching a fully supervised ResNet-50 baseline with a twentieth of its labels. HASTE and its predecessors have supported more than thirty real-world disaster responses since 2023, spanning earthquakes, hurricanes, cyclones, floods, wildfires, and tornadoes, delivering results to humanitarian partners within hours to days of imagery becoming available. We close with the directions we think are most promising, including vision-language assessment, active learning, and damage models for roads and other infrastructure. HASTE is open source at https://github.com/microsoft/haste.

Importance:Retinopathy of prematurity (ROP) is the leading cause of preventable childhood blindness worldwide. If detected and treated early, ROP-associated blindness is preventable; however, identifying patients who might respond to treatment requires screening over time, which is challenging in low-resource settings where access to pediatric ophthalmologists and pediatric ocular imaging cameras is limited. Objective:To develop and assess the performance of a machine learning algorithm that uses smartphone-collected videos to perform retinal screening for ROP in low-resource settings. Design, Setting, and Participants:This diagnostic study used smartphone-obtained videos of fundi in premature neonates with and without ROP in Mexico and Argentina between May 12, 2020, and October 31, 2023. Machine-learning (ML)-driven algorithms were developed to process a video, identify the best frames within the video, and use those frames to determine whether ROP was likely or not. Eligible neonates born with gestational age less than 36 weeks or birth weight less than 1500 g were included on the study. Exposures:An ML algorithm applied to a smartphone-obtained video. Main Outcomes and Measures:The ML algorithms' ability to identify high-quality retinal images and classify those images as indicating ROP or not at the frame and patient levels, measured by accuracy, specificity, and sensitivity, compared with classifications from 3 pediatric ophthalmologists. Results:A total of 524 videos were collected for 512 neonates with median gestational age of 32 weeks (range, 25-36 weeks) and median birth weight of 1610 g (range, 580-2800 g). The frame selection model identified high-quality retinal images from 397 of 456 videos (87.1%; 95% CI, 84.0%-90.1%) reserved for testing model performance. Across all test videos, 97.4% (95% CI, 96.7%-98.1%) of high-quality retinal images selected by the model contained fundus images. At the frame level, the ROP classifier model had a sensitivity of 76.7% (95% CI, 69.9%-83.5%); at the patient level, the classifier model had a sensitivity of 93.3% (95% CI, 86.4%-100%). At both levels, the model's sensitivity was higher than that for the panel of pediatric ophthalmologists (frame level: 71.4% [95% CI, 64.1%-78.7%]; patient level: 73.3% [95% CI, 61.0%-85.6%]). Specificity and accuracy were higher for ophthalmologist classification vs the ML model. Conclusions and Relevance:In this diagnostic study, a process that used smartphone-collected videos of premature neonates' fundi to determine whether high-quality retinal images were present had high sensitivity to classify such images as indicating or not indicating ROP but lower specificity and accuracy than ophthalmologist assessment. This process costs a fraction of the current process for retinal image collection and classification and could be used to expand access to ROP screening in low-resource settings, with potential to help prevent the most common cause of preventable childhood blindness.

Over 6.6 million people worldwide live in refugee camps, most of which lack comprehensive, up-to-date maps. This hinders effective resource distribution, infrastructure planning, and disaster response in these environments. Automated mapping with aerial imagery offers a promising solution, capturing the detail needed for effective camp management, but it requires datasets that reflect the distinct characteristics of refugee camps. Existing building foot-print datasets focus on urban or semi-urban areas leaving refugee camps-characterized by irregular layouts, diverse building sizes, and varied materials-underrepresented and poorly served by current models. This study introduces the KakumaAerial dataset, an open-source resource for humanitarian mapping. It pairs high-resolution aerial imagery from the Kakuma-Kalobeyei refugee camps in Kenya with annotations for buildings, solar panels, roof materials, and sanitation facilities. The dataset serves as a resource for benchmarking models on tasks crucial to humanitarian aid. Baseline machine learning models achieved strong performance on key tasks: building and solar panel segmentation (IoU of 0.848 and 0.813, respectively), roof material classification (accuracy of 85.6%), and toilet identification (accuracy of 97.8%). By applying these models to broader areas within the camps, the study provides actionable insights into camp infrastructure, including energy access and sanitation availability. This research demonstrates how geospatial technologies and machine learning can enable humanitarian organizations to improve operational efficiency while improving the living conditions and dignity of displaced populations.

Remote sensing imagery is dense with objects and contextual visual information. There is a recent trend to combine paired satellite images and text captions for pretraining performant encoders for downstream tasks. However, while contrastive image-text methods like CLIP enable vision-language alignment and zero-shot classification ability, vision-only downstream performance tends to degrade compared to image-only pretraining, such as MAE. In this paper, we propose FLAVARS, a pretraining method that combines the best of both contrastive learning and masked modeling, along with geospatial alignment via contrastive location encoding. We find that FLAVARS significantly outperforms a baseline of SkyCLIP for vision-only tasks such as KNN classification and semantic segmentation, +6% mIOU on SpaceNet1, while retaining the ability to perform zero-shot classification, unlike MAE pretrained methods.

We present a comprehensive global temporal dataset of commercial solar photovoltaic (PV) farms and onshore wind turbines, derived from high-resolution satellite imagery analyzed quarterly from the fourth quarter of 2017 to the second quarter of 2024. We create this dataset by training deep learning-based segmentation models to identify these renewable energy installations from satellite imagery, then deploy them on over 13 trillion pixels covering the world. For each detected feature, we estimate the construction date and the preceding land use type. This dataset offers crucial insights into progress toward sustainable development goals and serves as a valuable resource for policymakers, researchers, and stakeholders aiming to assess and promote effective strategies for renewable energy deployment. Our final spatial dataset includes 375,197 individual wind turbines and 86,410 solar PV installations. We aggregate our predictions to the country level – estimating total power capacity based on construction date, solar PV area, and number of windmills – and find an r^2 value of 0.96 and 0.93 for solar PV and onshore wind respectively compared to IRENA's most recent 2023 country-level capacity estimates.

As of 2023, a record 117 million people have been dis-placed worldwide, more than double the number from a decade ago [22]. Of these, 32 million are refugees under the UNHCR's mandate, with 8.7 million residing in refugee camps. A critical issue faced by these populations is the lack of access to electricity, with 80% of the 8.7 million refugees and displaced persons in camps globally relying on traditional biomass for cooking and lacking reliable power for essential tasks such as cooking and charging phones. Often, the burden of collecting firewood falls on women and children, who frequently travel up to 20 kilometers into dan-gerous areas, increasing their vulnerability. [7] Electricity access could significantly alleviate these challenges, but a major obstacle is the lack of accurate power grid infrastructure maps, particularly in resource-constrained environments like refugee camps, needed for energy access planning. Existing power grid maps are often outdated, incomplete, or dependent on costly, complex technologies, limiting their practicality. To address this issue, PGRID is a novel application-based approach, which utilizes high-resolution aerial imagery to detect electrical poles and segment electrical lines, creating precise power grid maps. PGRID was tested in the Turkana region of Kenya, specifically the Kakuma and Kalobeyei Camps, cov-ering 84 km2 and housing over 200,000 residents. Our findings show that PGRID delivers high-fidelity power grid maps especially in unplanned settlements, with F1-scores of 0.71 and 0.82 for pole detection and line segmentation, respectively. This study highlights a practical application for leveraging open data and limited labels to improve power grid mapping in unplanned settlements, where the growing number of displaced persons urgently need sustainable energy infrastructure solutions.

Training deep learning models on petabyte-scale Earth observation (EO) data requires separating compute resources from data storage. However, standard PyTorch data loaders cannot keep modern GPUs utilized when streaming GeoTIFF files directly from cloud storage. In this work, we benchmark GeoTIFF loading throughput from both cloud object storage and local SSD, systematically testing different loader configurations and data parameters. We focus on tile-aligned reads and worker thread pools, using Bayesian optimization to find optimal settings for each storage type. Our optimized configurations increase remote data loading throughput by 20x and local throughput by 4x compared to default settings. On three public EO benchmarks, models trained with optimized remote loading achieve the same accuracy as local training within identical time budgets. We improve validation IoU by 6-15 versus 0-30 https://github.com/microsoft/pytorch-cloud-geotiff-optimization

Fully understanding a complex high-resolution satellite or aerial imagery scene often requires spatial reasoning over a broad relevant context. The human object recognition system is able to understand object in a scene over a long-range relevant context. For example, if a human observes an aerial scene that shows sections of road broken up by tree canopy, then they will be unlikely to conclude that the road has actually been broken up into disjoint pieces by trees and instead think that the canopy of nearby trees is occluding the road. However, there is limited research being conducted to understand long-range context understanding of modern machine learning models. In this work we propose a road segmentation benchmark dataset, Chesapeake Roads Spatial Context (RSC), for evaluating the spatial long-range context understanding of geospatial machine learning models and show how commonly used semantic segmentation models can fail at this task. For example, we show that a U-Net trained to segment roads from background in aerial imagery achieves an 84% recall on unoccluded roads, but just 63.5% recall on roads covered by tree canopy despite being trained to model both the same way. We further analyze how the performance of models changes as the relevant context for a decision (unoccluded roads in our case) varies in distance. We release the code to reproduce our experiments and dataset of imagery and masks to encourage future research in this direction -- https://github.com/isaaccorley/ChesapeakeRSC.

Continued dependence on imported fossil fuels is rapidly becoming unsustainable in the face of the twin challenges of global climate change and energy security demands in Europe. Here we present scenarios in line with REPowerEU package to identify Renewables Acceleration Areas that support rapid renewable expansion, while ensuring minimal harm to places important for biodiversity and rural communities. We calculated the area needed to meet renewable energy objectives under Business-as-Usual (BAU) and Low-conflict (LCON) development scenarios within each country, providing a broad overview of the potential for renewable energy generation to reduce impacts when development is steered toward lower conflict lands. Our analysis shows that meeting renewable energy objectives would require a network of land-based wind turbines and solar arrays encompassing upwards of 164,789 km2 by 2030 and 445,654 km2 by 2050, the latter roughly equivalent to the land area of Sweden. Our results highlight that BAU development patterns disproportionately target high-conflict land cover types. By 2030, depending on the development pathway, solar and wind development are projected to impact approximately 4,386–20,996 km2 and 65,735–138,454 km2 of natural and agricultural lands, respectively. As renewable energy objectives increase from 2030 to 2050 impacts to natural and agricultural lands also increase, with upwards of 33,911 km2 from future solar development and 399,879 km2 from wind development. Despite this large footprint, low-conflict lands can generate substantial renewable energy: 6.6 million GWh of solar and 3.5 million GWh of wind, 8–31 times 2030 solar objectives and 3–5 times 2030 wind objectives. Given these patterns, we emphasize the need for careful planning in areas with greater impact potential, either due to a larger demand for land area or limited land availability. Top-emitting countries with large renewable energy objectives (Germany, Italy, Poland, France, Spain) and those with limited flexibility in meeting objectives on low-conflict land (Albania, Slovenia, Montenegro, Hungary, Croatia, Serbia, Bosnia Herzegovina, Finland, Greece, Portugal, and Norway) should be priorities for country-level customizations to guide low-conflict siting and avoid disproportionate impacts on high-value areas.

In recent years, there has been an explosion of proposed change detection deep learning architectures in the remote sensing literature. These approaches claim to offer state-of the-art performance on different standard benchmark datasets. However, has the field truly made significant progress? In this paper we perform experiments which conclude a simple U-Net segmentation baseline without training tricks or complicated architectural changes is still a top performer for the task of change detection.

Rapid and accurate building damage assessments from high-resolution satellite imagery following a natural disaster is essential to inform and optimize first responder efforts. However, performing such building damage assessments in an automated manner is non-trivial due to the challenges posed by variations in disaster-specific damage, diversity in satellite imagery, and the dearth of extensive, labeled datasets. To circumvent these issues, this paper introduces a human-in-the-loop workflow for rapidly training building damage assessment models after a natural disaster. This article details a case study using this workflow, executed in partnership with the American Red Cross during a tornado event in Rolling Fork, Mississippi in March, 2023. The output from our human-in-the-loop modeling process achieved a precision of 0.86 and recall of 0.80 for damaged buildings when compared to ground truth data collected post-disaster. This workflow was implemented end-to-end in under 2 hours per satellite imagery scene, highlighting its potential for real-time deployment.

This work presents an application of the epitome model to downscale low-resolution satellite GOES-16 LST products. The goal of the research work is to generate a high temporal high spatial resolution LST by fusing high temporal resolution GOES-16’s 2km LST product with high spatial resolution, 30m, auxiliary image products obtained from Landsat 8 imagery. The approach used seeks to extend an epitomic-based representation approach used for classification label super-resolution to downscaling a continuous variable like LST. Experiments were conducted using GOES-16 and Landsat 8 imagery from El Paso, Texas. The qualitative and quantitative results of the presented experiments paint an overall favorable picture for the proposed approach with some instances where the performance is lower than expected primarily due to the limitation of using sun synchronous Landsat 8 data for algorithm training. Furthermore, land cover and seasonal dependency were studied. The algorithm performed better for grassland & shrubland than for other land covers and performed better for winter and fall than for summer and spring.

In this paper we propose a mask-conditional synthetic image generation model for creating synthetic satellite imagery datasets. Given a dataset of real high-resolution images and accompanying land cover masks, we show that it is possible to train an upstream conditional synthetic imagery generator, use that generator to create synthetic imagery with the land cover masks, then train a downstream model on the synthetic imagery and land cover masks that achieves similar test performance to a model that was trained with the real imagery. Further, we find that incorporating a mixture of real and synthetic imagery acts as a data augmentation method, producing better models than using only real imagery (0.5834 vs. 0.5235 mIoU). Finally, we find that encouraging diversity of outputs in the upstream model is a necessary component for improved downstream task performance. We have released code for reproducing our work on GitHub, see https://github.com/ms-synthetic-satellite-image/synthetic-satellite-imagery .

Monitoring marine mammals is of broad interest to governments and individuals around the globe. Very high-resolution (VHR) satellites hold the promise of reaching remote and challenging locations to fill gaps in our knowledge of marine mammal distribution. The time has come to create an operational platform that leverages the increased resolution of satellite imagery, proof-of-concept research, advances in cloud computing, and machine learning to monitor the world’s oceans. The Geospatial Artificial Intelligence for Animals (GAIA) initiative was formed to address this challenge with collaborative innovation from government agencies, academia, and the private sector. In this paper, we share lessons learned, challenges faced, and our vision for how VHR satellite imagery can enhance our understanding of cetacean distribution in the future.

PURPOSE:Automatic and accurate segmentation of lesions in images of metastatic castration-resistant prostate cancer has the potential to enable personalized radiopharmaceutical therapy and advanced treatment response monitoring. The aim of this study is to develop a convolutional neural networks-based framework for fully-automated detection and segmentation of metastatic prostate cancer lesions in whole-body PET/CT images. METHODS:525 whole-body PET/CT images of patients with metastatic prostate cancer were available for the study, acquired with the [18F]DCFPyL radiotracer that targets prostate-specific membrane antigen (PSMA). U-Net (1)-based convolutional neural networks (CNNs) were trained to identify lesions on paired axial PET/CT slices. Baseline models were trained using batch-wise dice loss, as well as the proposed weighted batch-wise dice loss (wDice), and the lesion detection performance was quantified, with a particular emphasis on lesion size, intensity, and location. We used 418 images for model training, 30 for model validation, and 77 for model testing. In addition, we allowed our model to take n = 0,2, …, 12 neighboring axial slices to examine how incorporating greater amounts of 3D context influences model performance. We selected the optimal number of neighboring axial slices that maximized the detection rate on the 30 validation images, and trained five neural networks with different architectures. RESULTS:Model performance was evaluated using the detection rate, Dice similarity coefficient (DSC) and sensitivity. We found that the proposed wDice loss significantly improved the lesion detection rate, lesion-wise DSC and lesion-wise sensitivity compared to the baseline, with corresponding average increases of 0.07 (p-value = 0.01), 0.03 (p-value = 0.01) and 0.04 (p-value = 0.01), respectively. The inclusion of the first two neighboring axial slices in the input likewise increased the detection rate by 0.17, lesion-wise DSC by 0.05, and lesion-wise mean sensitivity by 0.16. However, there was a minimal effect from including more distant neighboring slices. We ultimately chose to use a number of neighboring slices equal to 2 and the wDice loss function to train our final model. To evaluate the model's performance, we trained three models using identical hyperparameters on three different data splits. The results showed that, on average, the model was able to detect 80% of all testing lesions, with a detection rate of 93% for lesions with maximum standardized uptake values (SUVmax) greater than 5.0. In addition, the average median lesion-wise DSC was 0.51 and 0.60 for all the lesions and lesions with SUVmax>5.0, respectively, on the testing set. Four additional neural networks with different architectures were trained, and they both yielded stronger performance of segmenting lesions whose SUVmax>5.0 compared to the rest of lesions. CONCLUSION:Our results demonstrate that prostate cancer metastases in PSMA PET/CT images can be detected and segmented using CNNs. The segmentation performance strongly depends on the intensity, size, and the location of lesions, and can be improved by using specialized loss functions. Specifically, the models performed best in detection of lesions with SUVmax>5.0. Another challenge was to accurately segment lesions close to the bladder. Future work will focus on improving the detection of lesions with lower SUV values by designing custom loss functions that take into account the lesion intensity, using additional data augmentation techniques, and reducing the number of false lesions by developing methods to better separate signal from noise.

This work presents an approach for combining household demographic and living standards survey questions with features derived from satellite imagery to predict the poverty rate of a region. Our approach utilizes visual features obtained from a single-step featurization method applied to freely available 10m/px Sentinel-2 surface reflectance satellite imagery. These visual features are combined with ten survey questions in a proxy means test (PMT) to estimate whether a household is below the poverty line. We show that the inclusion of visual features reduces the mean error in poverty rate estimates from 4.09% to 3.88% over a nationally representative out-of-sample test set. In addition to including satellite imagery features in proxy means tests, we propose an approach for selecting a subset of survey questions that are complementary to the visual features extracted from satellite imagery. Specifically, we design a survey variable selection approach guided by the full survey and image features and use the approach to determine the most relevant set of small survey questions to include in a PMT. We validate the choice of small survey questions in a downstream task of predicting the poverty rate using the small set of questions. This approach results in the best performance - errors in poverty rate decrease from 4.09% to 3.71%. We show that extracted visual features encode geographic and urbanization differences between regions.

The significance of renewable energy in achieving necessary reductions in emissions to limit global warming to 1.5 degrees Celsius is widely acknowledged. However, there is growing concern over the allocation of land for constructing the required new infrastructure. Nowhere is this conflict more apparent than in India, where renewable energy targets are ambitious and land use conflicts are already significant. India intends to increase renewable energy to 500 GW by 2030. This would require an additional 42 GW of renewable energy to be installed every year. Although renewable energy can provide the solution to both India’s growing need for cheap energy and climate change mitigation, the sustainable future of renewable energy deployment is far from simple due to its associated land use impacts and socio-ecological risk. While others have highlighted challenges to India’s renewable energy targets, here we focus on the land use change issues that will need to be addressed for India to meet its targets. We introduce a series of recommendations and highlight how these could contribute to mainstreaming land values and facilitate the implementation of India’s 2030 renewable energy targets. These recommendations include suggested planning approaches that would guide the development of standard siting guidelines, identification of preferential “go-to” areas for renewable energy, and the development of tools that allow access to data and information to site renewable right. Policy recommendations highlight utilizing converted lands and existing built infrastructure for renewable energy development, and adapting existing policies so they address land use impacts.

ABSTRACTBackground and ObjectiveRadiomics and deep learning have emerged as two distinct approaches to medical image analysis. However, their relative expressive power remains largely unknown. Theoretically, hand-crafted radiomic features represent a mere subset of features that neural networks can approximate, thus making deep learning a more powerful approach. On the other hand, automated learning of hand-crafted features may require a prohibitively large number of training samples. Here we directly test the ability of convolutional neural networks (CNNs) to learn and predict the intensity, shape, and texture properties of tumors as defined by standardized radiomic features.MethodsConventional 2D and 3D CNN architectures with an increasing number of convolutional layers were trained to predict the values of 16 standardized radiomic features from real and synthetic PET images of tumors, and tested. In addition, several ImageNet-pretrained advanced networks were tested. A total of 4000 images were used for training, 500 for validation, and 500 for testing.ResultsFeatures quantifying size and intensity were predicted with high accuracy, while shape irregularity and heterogeneity features had very high prediction errors and generalized poorly. For example, mean normalized prediction error of tumor diameter with a 5-layer CNN was 4.23 ± 0.25, while the error for tumor sphericity was 15.64 ± 0.93. We additionally found that learning shape features required an order of magnitude more samples compared to intensity and size features.ConclusionsOur findings imply that CNNs trained to perform various image-based clinical tasks may generally under-utilize the shape and texture information that is more easily captured by radiomics. We speculate that to improve the CNN performance, shape and texture features can be computed explicitly and added as auxiliary variables to the networks, or supplied as synthetic inputs.

In this article, we compare several approaches to segmenting glacial lakes in the Hindu Kush Himalayas in order to support glacial lake area mapping. More automatic mapping could support risk assessments of Glacial Lake Outburst Floods, a type of natural hazard that poses a risk to communities and infrastructure in valleys below glacial lakes. We propose and evaluate several approaches that incorporate labels from a 2015 survey using Landsat 7 ETM+ SLC-off imagery to guide segmentation on newer higher resolution satellite images such as Sentinel 2 and Bing Maps imagery, comparing them also to approaches that do not use this form of weak prior. We find that a guided version of U-Net and a properly initialized form of morphological snakes are most effective for these two datasets, respectively, each providing between an 8–10% intersection-over-union improvement over existing U-Net segmentation approaches. An error analysis highlights the strengths and limitations of each method. We design visualizations to support the discovery of lakes of potential concern, including an interactive exploratory interface. All the codes supporting our study are released in public repositories.