Conference Agenda

The European Space Agency (ESA) has established the Third Party Missions (TPM) Programme to enhance the accessibility and utilization of satellite data from non-ESA missions. This initiative allows ESA to distribute and support data from a variety of missions operated by international and commercial partners, including NewSpace companies, providing scientists and researchers with additional resources for a broad range of Earth Observation applications.

In this presentation, an in-depth overview of the TPM Programme will be provided, highlighting its objectives, benefits, and the diverse range of missions included under its umbrella. We will explore how the programme facilitates the integration of complementary datasets, thereby enriching the scientific community's ability to monitor and analyse global environmental and climatic changes, among others.

Aligned with the objectives of URBIS24, this presentation will focus on how TPM data supports urban monitoring and innovative EO-integrated solutions to address urban challenges. By leveraging these datasets, urban policymakers, researchers, and service providers can gain valuable insights into urban planning, climate resilience, infrastructure management, and more, fostering new collaborations and advancements in urban Earth Observation.

SatVu, a start-up company based in London (UK) is developing a constellation of ten satellites (“HotSats”) flying a Medium Wave Infrared camera, with the mission to capture thermal information of any target of the Earth during both night and day. Our MWIR sensor offers a unique perspective of the world, imaging at an unprecedented resolution of 3.5 m, and is a significant improvement over any commercially available space-based thermal sensor.

One area where high resolution thermal data is a game-changing technology is the example is the Urban Heat Island Effect (UHI). The UN expects 68% of people to live in urban areas by 2050, up from 55% today. As global temperatures are also rising, many more people will be vulnerable to heat-related illness in the coming years. This is especially true where rapid urbanisation coincides with high vulnerability to climate change.

Every year, innovative solutions are adopted by city officials. The diversity of climate adaptation plans highlights that a strategy effective in one location may not be easily applicable to another. Each city is unique, influenced not only by its geographical location but also by factors such as demographics, land structure, and other distinctive characteristics. Regardless of that, more granular and frequent measurements of heat distribution in urban areas should become a tool in assessing the cost, effectiveness, and urgency of the local authorities’ response to extreme heat events.

In this presentation, we will showcase some of the images captured by HotSat-1 launched in June 2023. We will present the outcome of our partnership with Office of Planetary Observations who used one of our earliest data products as part of their nature-based solutions service, allowing better planning and management of cities across the world’s urban greening assets.

Challenge

The relevance of thermal remote sensing satellite data has become increasingly recognised for urban environmental monitoring in recent years, especially for tackling the growing Urban Heat Island effect. The problem with current thermal satellite data is that they are either only available in very high to high temporal resolution and low spatial resolution (>1,000m) respectively from geostationary or sun-synchronous satellites, or low temporal resolution and moderate spatial resolution of 30-100m. Such data are therefore only of limited suitability when regular monitoring of small-scale environmental factors is required.

Methodology

constellr develops the HiVE (High-precision Versatile Ecosphere monitoring mission) constellation of state-of-the-art high-resolution visible, near, and thermal infrared satellites, planned for launch by the end of 2024, to monitor land surface temperature (LST). With a 4- to 1-day revisit time (1 sat 2025 to 5 sats from 2028), 30 meters native spatial resolution, and 4-band multispectral TIR capabilities from 8-12µm enabling a temperature accuracy up 1.5 K, HiVE is uniquely equipped to provide accurate and timely data urban areas.

The HiVe´s secondary optical VNIR payload provides 10 spectral bands (similar to Sentinel-2) from 400-1000nm enabling the use of super resolution techniques for thermal sharpening.

Expected results

We will present constellr HiVE mission concept, status and provide insights into our upcoming activities to ensure high data quality from mid of 2025. On top we will present the added value of HiVE data for urban and infrastructure monitoring. We will demonstrate simulated HiVE data based on airborne campaigns as well as constellr LST30 which incorporates multiple available thermal public mission data and applies the constellr proprietary LST retrieval algorithm leading to higher sharpness at a 30m spatial resolution. We apply a time series of LST30 data on Freiburg, Germany during the 2023 heatwave to understand specific localized urban heat resilience challenges.

Lead author: Airbus Dimitri - BOULZE / Special campaigns & partnerships

Co-Author: Dassault Systèmes - Frederic BOS / Senior Client Executive

Airbus and Dassault Systèmes started discussions in 2020 about the possibility to combine Airbus expertise in 3D models production from satellite high resolution imagery; and Dassault Systèmes capabilities in physical simulation to generate “Virtual Twins”, trustworthy 3D replica of any area of interest worldwide, and use them as playground to simulate various physical phenomena. Airbus and Dassault Systèmes namely worked on Virtual Twins over various urban areas to demonstrate use cases around predictive maintenance, monitoring, prevention and crisis management.

The Rationale of Airbus and Dassault Systèmes to push for an ambition on virtual twins, namely over cities, encompasses the following assumptions:

• The number of EO sensors has soared in the last 10 years, with a resulting surge in data volume as well as data type available to EO users. With Pléiades Neo today and soon with CO3D and its native stereo acquisition, Airbus is therefore securing access to a high quality and reliable source of data for 3D content that can feed new usages.
• 3D datasets are the new frontier, and we can expect that they gradually replace 2D datasets in a large panel of uses, both in the commercial and institutional sector. Various GIS software providers (Google Earth, Google Maps, QGis, ArcGis, etc…) are defining new OGC standards allowing to visualise large 3D datasets in accessible, sometimes free tools. 3D is already included in Google Earth and it’s likely that the majority of these tools will follow the same trend, which will trigger a new leap in geospatial analysis. In a matter of years, Copernicus Services may need high resolution 3D coverage the same way they need 2D coverage today
• The advent of cloud computing: AI based solutions and analytics have transformed the way both defence and commercial users are consuming geospatial data, providing faster means to exploit data, automate processing and combine with other datasets to extract relevant insights. These technical bricks in which Airbus and Dassault Systèmes have started investing several years ago are a cornerstone of any strategy around virtual twins as the need for automation, big-data and in-depth analysis is mandatory.

In 2021, in the frame of the “France Relance” plan, Airbus and Dassault Systèmes agreed on a common goal and formed a partnership to generate trustworthy simulations and indicators that can help better anticipate environmental, social, economic or sanitary crises and establish adequate prevention and response plans. The project covered three main objectives:

• See how 3D models can be integrated in Dassault Systèmes’ simulation software environment. The exploration of use cases was spread across a wide spectrum of physical phenomena: fluid dynamics (hydro or aero), acoustic, thermal or electromagnetic propagation. The list of these physical domains is not exhaustive. A matrix of about forty use-cases/verticals/simulation services has been established and to date less than a dozen have been explored. The potential remains very important.
• Test and use different LODs (Level of Details) of 3D models and run simulations over urban areas, where the complexity of the 3D environment is the highest, and the need is the strongest. More than 10 urban areas have been produced already, using Pléiades and Pléiades Neo sensors, to calibrate the relevance and specific added value to different use cases. Several production chains were also used to test various levels of details and editing, and compare how simulations are affected.
• Analyse market size and select use cases with higher score between technical challenge and business potential. The idea being to address real use cases, on-boarding end-users with prototypes as early as possible to confirm added value of the solution, while building and packaging a service offer.

The project is still ongoing, and close to 10 additional 3D models will be produced by the end of the project, making them available for a wide variety of simulations.

End-users have been on-boarded all along the project (primarily cities and institutional actors) making sure to drive the approach and help assessing the right technical trade-offs.

The COSMO-SkyMed (CSK) system is an operational constellation of 5 SAR satellites (3 of first generation + 2 of second generation) with very high performing capabilities: very high resolution, large swaths, unique radiometric accuracy, high geolocation accuracy, better than 12 hours revisit, and several other characteristics.

Among the several COSMO-SkyMed capabilities, one of the most important is the possibility to perform interferometric acquisitions using Spotlight and Stripmap imaging modes. Especially important is the possibility to perform interferometry between Stripmap data acquired by the first and second generation, thus allowing to take advantage of a huge and very long-lasting archive with a projection to the future (2 new second generation satellites will follow in 2025-2026). Currently the system is able to acquire 5 interferometric acquisitions every cycle of 16 days.

During all the years, ASI has implemented (with the support of e-GEOS) a so-called background mission to populate the catalogue with useful acquisitions. For this we have focused on performing Stripmap interferometric acquisitions over several targets, including all the major world cities (having more than 200,000 inhabitants). In this way we have now a huge archive over all the most important urban areas that has monitored the situation for more than 10 years.

Using these acquisitions, it is possible to generate very precise change detection maps (using multitemporal combinations, with coherence analysis) and to analyze also millimetric vertical ground movements using the PS technique. Such activities allow to monitor closely the urban areas in time, regardless of the cloud cover, taking advantage of the very high resolution of the Stripmap mode (3 m) which is complementing the Sentinel-1 capabilities providing a more detailed analysis over dense urban areas.

Prométhée Earth Intelligence is a French start-up founded in 2020 aiming to democratize the use of satellite imagery to offer high value-added services in environmental or strategic intelligence. It relies on proprietary data from its own satellites, with ProtoMéthée currently in orbit, followed in 2025 by the Japetus demonstrator. By 2027, the Japetus constellation of 20 nanosatellites will be fully operational to provide high-revisit and high-reactivity data acquisition capabilities. The data will be available through the proprietary Earth Observation Platform (EOP), which enhances raw data by merging it with other complementary information sources to build solutions for operational services. This agile platform acts as a unique and aggregative access point which is linked to a cloud-based digital processing system. It is designed to facilitate EO access to a broader user community.

In particular, Prométhée Earth Intelligence offers an operational solution to protect cities from flooding, splitted in three application levels, by correlating physical and human geography :

• Situational Awareness Service to Understand, delivers a flood vulnerability map based on topographic (morphology of the relief) and contextual (land use and climate) information.
• Early Warning System to Prevent and Anticipate, highlights all the critical infrastructures and locates the potential landslide zones, evaluating the most endangered areas in case of flooding.
• Rapid Response service to Act, delivers a flood map emphasises impacted buildings and transport network status. A daily refresh will be computed on it, allowing accurate monitoring.

This new capability solution delivers a decision support capacity helpful for the authorities to handle flood events in urban areas. It is a scalable tool that complements and interfaces with existing information systems, based on the informational power of GEOINT and adaptable to current and future operational needs. This will be detailed through a scientific poster.

Urban surface fabric identification and mapping present a significant challenge in the field of Earth Observation. Precise knowledge of surface characteristics is essential for effective urban planning and climate research. However, the presence of artificial materials in urban areas and mixed pixels complicates the accurate analysis of spectral data from multispectral and hyperspectral sensors. In turn, the current hyperspectral libraries, which rely on spectroradiometers, do not offer enough spectral variability from artificial materials for successful Machine Learning model training and accurate material detection. To address these gaps, the urbisphere urban hyperspectral library is currently being developed, utilizing the Spectral Evolution RS-3500 spectroradiometer and HySpex VS-620 Camera.

Today, the urbisphere urban hyperspectral library contains more than 5000 in-situ hyperspectral measurements from various natural and artificial materials collected from several European cities (e.g., Heraklion, Paris, and Berlin) and is planned to be enriched further in the coming years. The library also contains the respective adjusted spectra for several satellite sensors (e.g., Sentinel-2, Landsat-8, Planet SuperDove, EnMap, PRISMA, etc.), enabling satellite image classification without the need for time-consuming on-site data collection.

In this study, the current performance of the urbisphere urban hyperspectral library was tested over the broad urban area of Heraklion city. Specifically, the X-SVM classifier was trained using only the adjusted satellite spectra from the library for the satellites of Planet SuperDove, Sentinel-2, Landsat-8, EnMap, and PRISMA, while the trained models were applied to the respective satellite images acquired between August 3rd and 5th, 2023. The results highlight the current performance of the library for satellite image classification and the unique limitations that originate from the low spatial resolution for the Hyperspectral satellites (EnMap, PRISMA) and, on the other hand, from the low spectral resolution from the multispectral sensors (Sentinel-2, Planet SuperDove).

Urban planning and city governance require innovative solutions to face new urgent requirements and priorities. Leveraging advancements and the integration of Earth Observation (EO) with Artificial Intelligence (AI) methodologies has become critical in urban management.

Using AI models to chain multiple processes in a pipeline, from Super Resolution (SR) to Change Detection (CD) and Building Footprints (BF) extraction, is crucial for urban delineation, providing stakeholders with accurate results for informed decision-making. The project IRIX4US aims to monitor urban dynamics accurately for mobility, sustainability, urban planning, and accessibility in urban areas.

Engaging a broad set of relevant users and stakeholders, from public organizations to private industry, provided an opportunity to develop an EO-integrated solution tailored to the needs of urban experts and decision-makers. The solution consisted in a comprehensive AI pipeline:

1. Fine-Tuned SR of Sentinel-2 (S2) images: An AI-driven solution has been conducted to enhance their spatial resolution up to 2.5 meters. The model was trained using data from very high-resolution satellite imagery at a European scale. This process outputs a realistic representation of buildings, ensuring sharp edges and clear differentiation from surrounding objects such as roads.
2. Segmentation and BF extraction: IRIX4US adapts tested segmentation models to the super-resolved S2 imagery in urban landscapes, extracting visible and near-infrared (VNIR) bands and BFs using high-quality labels provided by champion users.
3. CD with AI Models: AI models have been employed for identifying differences in BFs between different images over time. These AI models were trained using temporal change data provided by champion users, allowing for the effective monitoring of urban development patterns in a multi-temporal context.

The results within the IRIX4US provides a dynamic and time-saving advancement for urban planning, proven feasible and scalable applications such as building segmentation, building change detection, illegal settlement identification and construction damage assessment.

This study investigates the efficacy of image restoration techniques of satellite EO images with a focus on super-resolution of Sentinel-2 and PlanetScope products. The ultimate goal is to develop a robust image restoration model capable of producing enhanced multispectral aerial-like imagery. The study is investigating several semi-supervised generative algorithms including SR-GANs, EDSR-GANs, WDSR-GANs, Lambda-PNN and W-Net.

The neural network architectures examined exhibit variations in their learning approaches and the potential utilization of a HR panchromatic component. For instance, the SR-GANs, EDSR-GANs, WDSR-GANs and WNet architectures are trained within a semi-supervised framework, i.e. supervising the training process of the generative models by incorporating multispectral HR target images. The Lambda-PNN network is trained within a fully unsupervised framework, hence no HR target images are adopted during its training phase. On the other hand, Lambda-PNN and WNet include a HR resolution panchromatic channel aiding the image super-resolution task. For our assessment with Lambda-PNN and WNet architectures, we opted for a panchromatic channel from an aerial image captured at a spatial resolution of 75cm either within the same date and in year 2019.

From our preliminary experiments, we have observed that GAN architectures which do not require a high-resolution panchromatic band can reconstruct HR scenes only for synthetic LR image datasets at 3m resolution obtained by downscaling aerial images. Pan-sharpening architectures like Lambda-PNN do not require a multispectral ground truth for training. However, when fed with Sentinel-2 and PlanetScope images, such architecture can produce synthetic images whose spatial structure is preserved (low structural loss) but yields unrealistic results regarding spectral information.

Among the preliminarily investigated architectures, the only architecture yielding consistent and plausible predictions is W-NET, a GAN fed with panchromatic and LR images and trained using HR target images employing a supervised approach. Furthermore, we are dedicating our effort in quantifying the reliability of generated images with respect to the introduction of spatial and spectral artifacts.

A myriad of global/regional datasets have provided valuable insights into environmental dynamics at global scale, including urban environments, however they often fall short of capturing the fine-scale nuances of the state and dynamics at local levels. Most existing datasets lacks the resolution and specificity required to address the diverse monitoring needs of urban stakeholders, particularly in densely populated or rapidly changing urban landscapes.

In response to these limitations, there is a growing recognition of the need for human-in-the-loop approaches, wherein stakeholders actively engage in the process of data collection, analysis, and interpretation to augment existing datasets and tailor monitoring efforts to local contexts. In this presentation, we will present a new agile cloud-based solutions that empower users to independently create and update urban datasets using free and open Copernicus Sentinel and NASA Landsat data, thereby overcoming the challenges associated with existing global datasets and fostering a more dynamic and responsive approach to urban environmental monitoring.

Developed as part of the EU 100Ktrees activity, based on the elaborated needs of municipalities worldwide, you will hear how complex machine learning frameworks and automated satellite data acquisition has been turned into user friendly cloud-based web applications for scalable mapping of urban environments on demand. You will learn how multitemporal satellite data and automated data analysis can turn raw satellite data into scalable information about urban heat islands, green spaces, flood exposure and impervious surfaces. And you will discover how these tools, and others, are vital to address the existing data gap across urban landscapes worldwide and how they can be used to underpin comprehensive and dynamic monitoring regimes.

The characterization of structural changes in urban areas, and specifically the current trend towards verticalization, is a crucial step toward understanding urbanization phenomena at the global level. The use of spaceborne remote sensing data for this task has grown over the last decade, focusing on SAR and multispectral data, often jointly used. In particular, very high resolution (VHR) data, increasingly available in large volumes, calls for efficient and accurate 2D/3D change detection techniques, able to go beyond existing methods, focused either on detecting buildings footprint changes over time or on extracting changes in the number of floors/height of the same buildings. In this work, we present a complete framework applicable to VHR SAR temporal sequences able to characterize 2D & 3D changes at the same time. The methodology involves the extraction of altered buildings in the initial phase and subsequently performs a building segmentation, with specific emphasis on discerning changes in building height. The identification of changed urban core areas is performed using an unsupervised temporal clustering applied to the original sequence of SAR amplitude and coherence images. Subsequently, building segmentation in the initial and ending image of the sequence is performed thanks to a deep learning U-Net architecture, allowing a quantitative characterization of the 2D changes. Finally, the building height extraction is accomplished through the utilization of a novel ResNet deep learning architecture. The proposed framework has been tested using a time series of three years of COSMO-SkyMed data, from January 2019 to November 2021, over the city of Milan, Italy. Experimental results in the "Gae Aulenti" and "City Life" areas show that 2D and 3D changes are correctly detected and the use of the proposed machine and deep learning framework significantly increases the ability to achieve a better characterization of structural changes in urban areas.

While the significance of built-up volume and density for future sustainability and resilience of the built environment is widely recognized, the interplay between 3D built-up patterns, economic development, and local planning policies remains relatively unexplored. To address this gap, we integrate novel global data from the World Settlement Footprint 3D (WSF® 3D) - encompassing building area, height, and volume -, with socioeconomic statistics and information on planning policies across all countries and over 12,000 urban clusters worldwide. Our analysis begins with enhancing the original WSF3D dataset to provide a more accurate representation of high-rise buildings exceeding 50 meters. This enhancement involves integrating data from the Emporis database, a leading source of tall buildings information globally. Subsequently, we merge the enhanced WSF3D data (WSF3Dv2) with socioeconomic statistics sourced from the World Development Indicators (WDI) database and data on zoning regulations and land-use policies provided by the World Bank. For the city-level analyses, we additionally utilize the Urban Center database provided by the Joint Research Center (JRC) of the European Commission. Through our study, we offer a comprehensive and spatially detailed understanding of the global 3D building stock, unveiling intricate relationships between 3D settlement morphology, economic development, and local planning regulations. This empirical evidence strengthens the understanding of the global megatrend of urbanization and provides crucial insights to enhance the effectiveness of land use and spatial development policies aimed at promoting (urban) sustainability and resilience.

Accurate building height estimation is essential for sustainable urban planning, monitoring, and environmental impact analysis. However, conducting large-scale building height estimation at a fine spatial resolution is challenging, especially using open-access satellite data. Existing large-scale solutions provide height at coarse spatial resolution (500 m - 90 m), a better resolution can provide more comprehensive understanding of urban development.

We propose an advanced deep learning model, T-SwinUNet, specifically designed for large-scale building height estimation at a fine spatial resolution of 10 m. The model harnesses salient features from the spatial, spectral, and temporal dimensions of the Sentinel-1 SAR and Sentinel-2 MSI time-series data.

In T-SwinUNet, we integrated the semantic feature learning capabilities of the CNN encoder with the local/global feature comprehension capabilities of Swin transformers. With added temporal attention, the model learns the correlation between constant features (mostly geometry) and variable features (e.g. shadow) of building objects over time. This not only aids in differentiating building from non-building objects but also condition model to learn salient building height features. We equipped T-SwinUNet with uncertainty prediction, which helps in assessing model’s robustness and transferability to new areas. Knowing uncertainty in predictions helps stakeholders to make informed and careful decisions.

The model is evaluated on data from the Netherlands, Switzerland, Estonia, and Germany. The extensive evaluation and comparison with state-of-the-art DL models show that our proposed T-SwinUNet model surpasses SOTA by achieving an RMSE of 1.89 m at 10 m spatial resolution. We conducted a detailed ablation study to understand the impact of time-series data, each modality, multi-task learning and others. Further assessment at 100 m resolution shows that our predicted building heights (0.29 m RMSE, 0.75 $R^{2}$) also outperformed the global building height product GHSL-Built-H R2023A product (0.56 m RMSE and 0.37 $R^{2}$).

Accurate determination of building heights is crucial for urban 3D development analysis and disaster risk assessment. State-of-the-art (SOTA) techniques typically treat height retrieval from buildings as a regression problem. For instance, [1, 2] propose supervised Multimodal Deep Learning (DL) frameworks to estimate building heights using Sentinel-1 (Sen-1) and Sentinel-2 (Sen-2) data.

In this work, we employ an Attention-based U-Net model for building height estimation, relying solely on radar information provided by Very High-Resolution (VHR) COSMO-SkyMed (CSK) data. We utilize a CSK Level-1D Stripmap Himage acquisition mode with a spatial resolution of 2.5 meters in Ascending orbit direction, covering the entire metropolitan area of Milan city for the model training. Ground Truth (GT) labels are derived from the Normalized Digital Surface Model (nDSM) and refined by using a binary footprint mask from OpenStreetMap (OSM) data to exclude non-built-up (BU) areas.

Our proposed model exhibits robust performances in terms of Root Mean Square Error (RMSE) on external test sites in the cities of Pavia, Sant’Angelo Lodigiano, and Lodi, with final error values of 0.781, 0.731, and 1.487 meters, respectively, outperforming previous studies [1, 2]. To further validate the model, the average error outside the buildings was also evaluated. The results reveal a general underestimation of building height as their actual height increases, indicating an avenue for future research.

Notably, our method capitalizes on CSK radar data, offering a swift solution for mapping 3D BU area features owing to its VHR, weather-independent capabilities, and rapid emergency response.

REFERENCES

[1] Yadav, R., https://arxiv.org/abs/2307.01378

[2] Bowen, C. et al., https://www.sciencedirect.com/science/article/pii/S1569843223002236

CARS is a CNES open-source 3D reconstruction software part of the Constellation Optique 3D (CO3D) mission. CARS stands out from other MultiView-Stereo methods due to its highly parallelizable design, capable of addressing large volumes of data for processing on an HPC cluster or personal machine.

Designed as a modular set of applications, CARS allows user to plug in new public or confidential contributions, such as new applications or pipelines. Using this plug-in concept, we recently addressed the need for LOD2 semantized meshes on urban areas. The new CARS plug-in makes the most out of intermediate CARS products such as uncertainty and classification layers to create LOD2 meshes.

Surface temperatures are central to the surface energy balance, and particularly in cities link to human thermal comfort and energy consumption. Since, cities have a strong vertical component, estimating the surface temperature for the complete three dimensional (3D) urban surface is important. While satellite remote sensing holds a strong potential in observing city-wide and global urban surface temperatures, these need to be complemented with in-situ infrared observations and modeling to achieve the assessment of the complete 3D surface temperature.

In this study, a sub-building scale three-dimensional (3D) energy balance model is evaluated using a novel ground-based thermal camera observatory in Berlin. Four thermal cameras (OptrisPI160) were mounted on an 80 m tall building, overlooking a mix of low-rise residential, parks, trees from all cardinal directions. 3D urban surface temperature was modeled using an extension of the energy balance model TUF-3D model (Krayenhoff and Voogt, 2007) including a vegetation component (VTUF-3D, Nice et al., 2018).) Earth Observation products (land cover, building and vegetation heights) were used to parametrize the 3D urban form of the study domains, while forcing data were available from meteorological measurements on the same building. A total area of 0.2 km² was simulated with a 5 m spatial resolution for 3 – 6 August 2022. The camera brightness temperatures observations were corrected for emissivity to conclude to surface temperature. Detailed emissivity maps were derived using land cover and reference spectral library information.

The diurnal pattern of modelled surface temperature was found similar to the observed one for different surface types (MAE ranging between 3-10 C depending on the area and time of day, with error increasing during daytime). VTUF-3D simulated surface temperatures for more areas in Berlin will be used to assess the complete temperature from satellite thermal observations, such as Sentinel-3.