Vision–language models (VLMs) have shown significant promise in remote sensing applications, particularly for land-use and land-cover (LULC) mapping via zero-shot classification and retrieval. However, current approaches face several key challenges, such as the dependence on caption-based supervision, which is often not available or very limited in terms of the covered semantics, and the fact of being adapted from generic VLM architectures that are suitable for very high resolution images. Consequently, these models tend to prioritize spatial context over spectral and temporal information, limiting their effectiveness for medium-resolution remote sensing imagery.In this work, we present TimeSenCLIP, a lightweight VLM for remote sensing time series, using a cross-view temporal contrastive framework to align multispectral Sentinel-2 time series with geo-tagged ground-level imagery, without requiring textual annotations. Unlike prior VLMs, TimeSenCLIP emphasizes temporal and spectral signals over spatial context, investigating whether single-pixel time series contain sufficient information for solving a variety of tasks.Our approach is trained on the LUCAS and Sen4Map datasets and evaluated across four main mapping tasks: land cover, land use, habitat mapping and crop type classification. The CLIP text encoder can be used to probe the learned representations using semantically meaningful categories, enabling effective zero-shot generalization without task-specific text supervision. We further extend our evaluation to bioregions mapping and country-level image retrieval. Although coarse, these tasks are valuable for probing whether the model captures geographically meaningful representations, such as regional climate regimes, vegetation patterns, and land-use structures. TimeSenCLIP achieves consistently better performance than existing CLIP-based remote sensing models in both zero-shot classification and cross-modal retrieval. Notably, single-pixel multispectral time series variants remain highly competitive, particularly with extended temporal coverage, demonstrating that temporal–spectral dynamics can compensate to a substantial degree for the reduced spatial footprint.While larger spatial patches still offer advantages for tasks where spatial patterns are inherently informative, such as ecosystem type classification, the results suggest that single-pixel multispectral time series can provide effective remote sensing vision–language pipelines, supporting scalable and efficient modeling in scenarios where large spatial tiles or extensive textual annotations are impractical. Code is available at https://github.com/pallavijain-pj/TimeSenCLIP.
Food Security is a crucial global concern and a key pillar of the Sustainable Development Goals (SDG 2 - Zero Hunger). To monitor food insecurity situations, several early warning systems are active today, driven by agencies such as FAO and WFP. These systems use a limited set of data types, e.g., agroclimatic data and indicators extracted from household surveys. Household surveys are expensive and time-consuming, and by consequence not frequent, which ultimately limits the ability to provide timely responses in vulnerable regions. In this work, we present a deep learning (DL) framework able to classify regional food security levels into three main categories i.e., poor, borderline and acceptable. The idea is to obtain such predictions by using openly accessible heterogeneous data sources, including satellite imagery, climate variables, and socioeconomic indicators. The proposed architecture employs multi-branch neural network models—Convolutional Neural Networks (CNNs) for spatial rasters (population density, land use/land cover maps), Recurrent Neural Networks for time series (e.g., rainfall, temperature, commodity prices), and Random Forests for feature fusion that integrates representations extracted from the two DL branches with a set of conjunctural variables. We validate our approach on two African countries, Burkina Faso and Rwanda, achieving classification accuracies exceeding 85
Event Based Surveillance (EBS) monitors online sources such as broadcast, print, web news and generates early warning and response (EWAR) signals for use in disaster mitigation. These online sources provide a dynamic data source allowing for potential real-time EBS updates. However, in dealing with news articles, fragmented information exists in varied sources and redundant information are known to overburden EBS. In this study we propose a Large Language Model based approach that filters out redundancies while learning novel information from event centric online news corpora. We study this novelty task for events covering animal health, food security and climate change surveillance domains. Our approach focuses on features integrating spatio-temporal information (such as location and date of event) and thematic information (such as the name of disease, food insecurity triggers, climate change magnitude).We characterize novelty as presence of new and additional information (e.g., a newly mentioned disease name or additional location information) as distinguished from duplicate (e.g., an already seen disease name) and missing (expected but absent) information. To this regard, our approach proposes fine-grained classification of novelty in event surveillance and language modeling adoption with a multi-class classification objective to learn classifying of event information. Our LLM adoption strategy proposes question-based prompts whose extracted answers map to predefined feature types (e.g., location, date, name of disease) in order to enrich our classifier. In our empirical studies, we present comparative analysis with respect to language models and large language models for State-Of-The-Art performance in the event novelty classification task. Our findings demonstrates the ability of cross-domain novelty classification with our model EpidGPT (few-shot) achieving F1% scores of 82.3, 85.49 and 88.97 in animal health, food security and climate change domains while finetuned EpidGPT achieves F1% scores of 96.02, 86.0 and 88.45 on each respectively domains.
Accurate crop type mapping in Sub-Saharan Africa is a challenging task, due to the presence of smallholder farming systems characterized by fragmented landscapes and heterogeneous cropping practices. Persistent cloud cover, particularly significant during the cropping season, systematically limits the exploitation of optical satellite image time series, making things even harder. This study proposes MSF-Net (Multimodal Sentinel Fusion Network), a convolutional neural network-based late-fusion framework that combines Sentinel-1 synthetic aperture radar and Sentinel-2 multispectral time series for multi-class crop classification in the complex agricultural landscapes of central and northern Benin. The model was evaluated across six sites and three growing seasons (2022–2024) covering 12 land cover classes and compared with a Sentinel-2-only Temporal Convolutional Neural Network (TempCNN), a SAR-only baseline (S1-Branch), an ablated version of the proposed method, and two external state-of-the-art multimodal architectures, TSViT and TWINNS. MSF-Net achieved the highest or joint-highest overall accuracy in 10 of 14 site–year configurations, with overall accuracy ranging from 82.61% to 91.15% and kappa coefficients from 0.79 to 0.89, consistently outperforming both external baselines across all site–year configurations. The largest gains over TempCNN reached up to 30 percentage points for spectrally ambiguous classes such as Shrubby Savannah, Cotton, and Open Forest. In addition, MSF-Net produced more spatially coherent maps, with reduced salt-and-pepper noise, improved parcel-level homogeneity, and fewer modality-specific artefacts. These results demonstrate the value of SAR-optical fusion for operational crop monitoring in tropical West Africa.
The response of a network to connectivity disruption is central to understanding its robustness. While such questions have been thoroughly studied in terms of global cohesion, many questions remain open regarding the impact on local connectivity. Specifically, networks with different topologies display different distributions of small motifs, and it remains unclear how local connectivity patterns link to structural weaknesses within a network. This study presents a principled approach for assessing random connectivity disruptions among the nodes of a network to identify its structural vulnerabilities. Leveraging recent findings on random walks for exact local topology recollection, we introduce a method for quantifying changes within the (l)-hop neighborhood of the nodes. We derive vulnerability profiles for the nodes and introduce a new index that aggregates neighboring nodes’ vulnerabilities in a distance dependent weighting. Our conducted experiments show that structural weaknesses of a network are distributed across localized regions, where vulnerabilities are most likely driven by immediate neighborhoods of the nodes.
Multi-modal classification leverages complementary information across diverse data sources to enhance predictive performance. However, real-world scenarios subject to operational constraints, such as sensor failures or privacy restrictions, lead to inconsistent modality availability between training and inference times. To handle missing modalities, prior studies have mainly covered bimodal data setups and focused on designing robust fusion processes. Instead, we adopt a multi-modal co-learning framework that prioritizes inter-modal collaboration rather than multi-modal fusion. Specifically, we consider that any subset of modalities may be absent, without assuming predefined missing-modality patterns, an inference scenario we refer to as missing arbitrary modalities. To address this challenge, we introduce two alternative approaches that leverage information at both feature- and decision-level. Experiments on two multi-modal classification benchmarks demonstrate significant robustness gains in various missing modality conditions. The first method shows more robust behavior under minimal missing conditions, where a single modality is absent, whereas the second performs better under extreme missing conditions, where all-but-one modalities are missing. Our code is available at https://github.com/fmenat/Co4Miss.
The timely monitoring of land changes is of capital importance to support sustainable development, especially in regions that are particularly exposed to the effects of climate change. Deriving land cover maps from satellite image time series (SITS) is an established practice to enable the tracking of land dynamics, notably in the agricultural and environmental domain. While classical machine learning remains widely used to accomplish this task in operational settings, advances in deep learning offer new paradigms susceptible to improve the precision of these products.However, if deep learning based land cover mapping is proving its effectiveness in different contexts, including at global-scale, no clear evidence has emerged yet on the advantages that such technical shift may provide in the context of tropical agricultural landscapes. A certain number of factors, from the poor availability of exploitable satellite acquisition to the limitations of reference data bases, may hinder the potential of such techniques and make their use eventually inefficient.In order to give a full insight on a real-world application on challenging tropical agricultural landscapes, we here present a study aimed at comparing several state-of-the-art deep-learning approaches for land cover mapping using SITS on two areas located in Northern Benin, in the sub-humid West African region. Among the deep learning models explicitly exploiting the temporal dimension, TempCNN emerged as the most effective, outperforming Random Forest in overall accuracy, weighted F1-score, and computational efficiency, with notable robustness across sites and improved discrimination of minority classes. More complex architectures such as ConvTran and DuPLO provided no additional benefit, highlighting that model simplicity, when well-suited to the problem, can outperform elaborate designs for LULC classification from multi-temporal optical data.
In many scientific fields, complex systems are characterized by a multitude of heterogeneous interactions/relationships that are challenging to model. Multilayer graphs constitute valuable tools that can represent such complex systems, thus making possible their analysis for downstream decision-making processes. Nevertheless, modeling such complex information still remains challenging in real-world scenarios. On the one hand, holistically including all relationships may lead to noisy or computationally intensive graphs. On the other hand, limiting the amount of information to model through the selection of a portion of the available relationships can introduce boundary specification biases. However, the current research studies are demonstrating that it is more beneficial to retain as much information as possible and at a later stage perform graph simplification i.e., removing uninformative or redundant parts of the graph to facilitate the final analysis. While simplification strategies, based on deep learning methods, have been already extensively explored in the context of single-layer graphs, only a limited amount of efforts have been devoted to simplification strategies for multilayer graphs. In this work, we propose the MultilAyer gRaph simplificAtion (MARA) framework, a GNN-based approach designed to simplify multilayer graphs based on the downstream task. MARA generates node embeddings for a specific task by training jointly two main components: (i) an edge simplification module and (ii) a (multilayer) graph neural network. We tested MARA on different real-world multilayer graphs for node classification tasks. Experimental results show the effectiveness of the proposed approach: MARA reduces the dimension of the input graph while keeping and even improving the performance of node classification tasks in different domains and across graphs characterized by different structures. Moreover, deep learning-based simplification allows MARA to preserve and enhance important graph properties for the downstream task. To our knowledge, MARA represents the first simplification framework especially tailored for multilayer graphs analysis.
Due to its highly contagious nature, Avian Influenza (AI) is considered an animal health emergency affecting commercial sector and wild bird populations. Several genome sequencing databases have been created to help researchers understand how AI viruses evolve, spread, and cause disease. However, for a global epidemic monitoring approach, they need to be combined to public health surveillance systems, the well-one being EMPRES-i from the World Organisation for Animal Health (WOAH) and the Food and Agriculture Organization of the United Nations (FAO). This paper presents a new AI dataset, in which EMPRES-i is enriched thanks to the genome sequence data of Avian Influenza cases affecting bird species from 2012 to 2021, publicly provided by the Bacterial and Viral Bioinformatics Resource Center (BV-BRC). This dataset is obtained by automatically linking sequence information in BV-BRC to the AI events in EMPRES-i, which results in "putatively" linked events between these two sources. The collected data is structured by nature, but it is preprocessed and normalized for the purpose of high-quality data linkage. Moreover, several data linkage strategies and missing information handling are introduced. To show the usefulness of our dataset, we quantitatively evaluate the proposed strategies in randomly sampled events and present in the end a diffusion network inference task.
Pre-trained vision-language models (VLMs), such as CLIP, demonstrate impressive zero-shot classification capabilities with free-form prompts and even show some generalization in specialized domains. However, their performance on satellite imagery is limited due to the underrepresentation of such data in their training sets, which predominantly consist of ground-level images. Existing prompting techniques for satellite imagery are often restricted to generic phrases like a satellite image of ..., limiting their effectiveness for zero-shot land-use and land-cover (LULC) mapping. To address these challenges, we introduce SenCLIP, which transfers CLIPs representation to Sentinel-2 imagery by leveraging a large dataset of Sentinel-2 images paired with geotagged ground-level photos from across Europe. We evaluate SenCLIP alongside other SOTA remote sensing VLMs on zero-shot LULC mapping tasks using the EuroSAT and BigEarthNet datasets with both aerial and ground-level prompting styles. Our approach, which aligns ground-level representations with satellite imagery, demonstrates significant improvements in classification accuracy across both prompt styles, opening new possibilities for applying free-form textual descriptions in zero-shot LULC mapping.
The increasing availability of multi-modal data in Earth observation, such as optical and radar images, has enabled more comprehensive analyses for applications like land-cover classification. Despite the advantages, missing modality data can arise during inference due to factors like sensor malfunctions, cloud cover, or limited spatio-temporal coverage, presenting challenges to multi-modal model design. These issues can hinder model performance in the predictive tasks and its broad applicability. In this work, we propose a multi-modal co-learning framework designed to enhance single-modality predictions through multi-modal co-training, focused on the case where only partial modality data is available during inference. Our approach disentangles modality-invariant and modality-specific information by explicitly modeling shared and specific spaces using modality-dedicated encoders. To achieve this, we adopt a multi-task co-learning strategy with multiple loss functions promoting both discriminative and disentangled features. We evaluate our model using SPOT very high spatial resolution imagery and Sentinel-2 time series data for land-cover classification over a study site featured by contrasted landscape, located in the Indian Ocean, namely Reunion Island. The results demonstrate the effectiveness of our method with consistent improving over single-modality prediction. This work highlights the potential of multi-modal co-learning in the field of remote sensing, advancing land-cover classification, as it enables single-modality models to benefit from multi-modal data available at training time.
To address the current crises (climatic, social, economic), the self-sufficiency – a set of practices that combine energy sobriety, self-production of food and energy, and self-construction - arouses an increasing interest. The CNRS STAY project (Savoirs Techniques pour l’Auto-suffisance sur YouTube) explores this topic by analyzing techniques shared on YouTube. We present Agro-STAY, a platform designed for the collection, processing, querying and visualization of data from YouTube videos and their comments. We propose a full methodology dedicated to processing YouTube videos, and apply Natural Language Processing (NLP) techniques and language models, which enable a fine-grained analysis of alternative agricultural practices described online. In addition, we provide a well-adapted graphical user interface to help the experts analyzing the extracted knowledge.
Multi-sensor data has become a foundation of Earth Observation (EO) research, offering models with enhanced accuracy via optimal fusion strategies. However, the unavailability of sensor data at the regional or country scale during inference can significantly undermine model performance. The literature explores diverse approaches to increasing model robustness to missing sensor scenarios, i.e., to reducing the decline in accuracy caused by missing data at inference time. Nevertheless, most of them have suboptimal behavior when a single-sensor is available for prediction. To address this challenge, we propose a novel method for multi-sensor modeling, Decision-level Sensor Dropout with mutual distillation (DSensD+). This employs a decision-level fusion, ignoring predictions from missing sensors and incorporating the Sensor Dropout (SensD) technique. Unlike works that use the SensD at the input or feature level, we use it at the decision level. Moreover, we include a mutual distillation strategy to improve the robustness. From a practical viewpoint, the additional components in the DSensD+ method are incorporated only for the training phase. During inference, it operates as a standard decision-level fusion model that ignores missing sensors.We validate our method on three EO datasets, spanning binary, multi-class, and multi-label classification tasks for crop- and tree-mapping related applications. Notably, DSensD+ outperforms several state-of-the-art methods, achieving consistent improvements across moderate (single-sensor missing) and extreme (single-sensor available) conditions, as well as with full-sensor data. These results demonstrate the robustness of DSensD+ and highlight the effectiveness of our method for the missing sensor problem, advancing the field of multi-sensor modeling in EO.
The growing number of Earth observation satellites has led to increasingly diverse remote sensing data, with varying spatial, spectral, and temporal configurations. Most existing models rely on fixed input formats and modality-specific encoders, which require retraining when new configurations are introduced, limiting their ability to generalize across modalities. We introduce Atomizer, a flexible architecture that represents remote sensing images as sets of scalars, each corresponding to a spectral band value of a pixel. Each scalar is enriched with contextual metadata (acquisition time, spatial resolution, wavelength, and bandwidth), producing an atomic representation that allows a single encoder to process arbitrary modalities without interpolation or resampling. Atomizer uses structured tokenization with Fourier features and non-uniform radial basis functions to encode content and context, and maps tokens into a latent space via cross-attention. Under modality-disjoint evaluations, Atomizer outperforms standard models and demonstrates robust performance across varying resolutions and spatial sizes.
Multi-modal co-learning is emerging as an effective paradigm in machine learning, enabling models to collaboratively learn from different modalities to enhance single-modality predictions. Earth Observation (EO) represents a quintessential domain for multi-modal data analysis, wherein diverse remote sensors collect data to sense our planet. This unprecedented volume of data introduces novel challenges. Specifically, the access to the same sensor modalities at both training and inference stages becomes increasingly complex based on real-world constraints affecting remote sensing platforms. In this context, multi-modal co-learning presents a promising strategy to leverage the vast amount of sensor-derived data available at the training stage to improve single-modality models for inference-time deployment. Most current research efforts focus on designing customized solutions for either particular downstream tasks or specific modalities available at the inference stage. To address this, we propose a novel multi-modal co-learning framework capable of generalizing across various tasks without targeting a specific modality for inference. Our approach combines contrastive and modality discriminative learning together to guide single-modality models to structure the internal model manifold into modality-shared and modality-specific information. We evaluate our framework on four EO benchmarks spanning classification and regression tasks across different sensor modalities, where only one of the modalities available during training is accessible at inference time. Our results demonstrate consistent predictive improvements over state-of-the-art approaches from the recent machine learning and computer vision literature, as well as EO-specific methods. The obtained findings validate our framework in the single-modality inference scenarios across a diverse range of EO applications.
The widespread availability of satellite imagery data has enabled advancements in Land Use/Land Cover (LULC) and Urban Fabric (UF) mapping through deep learning. However, maintaining up-to-date urban land cover maps is challenged by the high cost and operational constraints of continuous field data collection. This study explores the feasibility of updating urban LULC maps using SITS-based semantic segmentation models trained on historical data, specifically examining a transfer scenario where a model trained on 2015 data is applied to 2020 imagery. We benchmark the performance of two convolution-based architectures (Unet and Unet3D), plus a recent spatio-temporal transformer-based approach (TSViT) and a proposed variant, named TSViT+SW, which incorporates a shifted window attention scheme. Experimental evaluations covering the urban area of Lyon, France, reveal that the proposed TSViT+SW model achieves the best results among transferred models, minimizing performance degradation compared to the ideal in-year training scenario. This work offers insights into the potential and limitations of using historical data to update urban land cover in the absence of fresh labeled data.
Language models now constitute essential tools for improving efficiency for many professional tasks such as writing, coding, or learning. For this reason, it is imperative to identify inherent biases. In the field of Natural Language Processing, five sources of bias are well-identified: data, annotation, representation, models, and research design. This study focuses on biases related to geographical knowledge. We explore the connection between geography and language models by highlighting their tendency to misrepresent spatial information, thus leading to distortions in the representation of geographical distances. This study introduces four indicators to assess these distortions, by comparing geographical and semantic distances. Experiments are conducted from these four indicators with eight widely used language models and their implementations are available on github ( https://github.com/tetis-nlp/geographical-biases-in-llms ). Results underscore the critical necessity of inspecting and rectifying spatial biases in language models to ensure accurate and equitable representations.
Pascal Poncelet合作论文数University Montpellier 2 - LIRMM2