Conference Agenda

Urban surface fabric identification and mapping present a significant challenge in the field of Earth Observation. Precise knowledge of surface characteristics is essential for effective urban planning and climate research. However, the presence of artificial materials in urban areas and mixed pixels complicates the accurate analysis of spectral data from multispectral and hyperspectral sensors. In turn, the current hyperspectral libraries, which rely on spectroradiometers, do not offer enough spectral variability from artificial materials for successful Machine Learning model training and accurate material detection. To address these gaps, the urbisphere urban hyperspectral library is currently being developed, utilizing the Spectral Evolution RS-3500 spectroradiometer and HySpex VS-620 Camera.

Today, the urbisphere urban hyperspectral library contains more than 5000 in-situ hyperspectral measurements from various natural and artificial materials collected from several European cities (e.g., Heraklion, Paris, and Berlin) and is planned to be enriched further in the coming years. The library also contains the respective adjusted spectra for several satellite sensors (e.g., Sentinel-2, Landsat-8, Planet SuperDove, EnMap, PRISMA, etc.), enabling satellite image classification without the need for time-consuming on-site data collection.

In this study, the current performance of the urbisphere urban hyperspectral library was tested over the broad urban area of Heraklion city. Specifically, the X-SVM classifier was trained using only the adjusted satellite spectra from the library for the satellites of Planet SuperDove, Sentinel-2, Landsat-8, EnMap, and PRISMA, while the trained models were applied to the respective satellite images acquired between August 3rd and 5th, 2023. The results highlight the current performance of the library for satellite image classification and the unique limitations that originate from the low spatial resolution for the Hyperspectral satellites (EnMap, PRISMA) and, on the other hand, from the low spectral resolution from the multispectral sensors (Sentinel-2, Planet SuperDove).

Urban planning and city governance require innovative solutions to face new urgent requirements and priorities. Leveraging advancements and the integration of Earth Observation (EO) with Artificial Intelligence (AI) methodologies has become critical in urban management.

Using AI models to chain multiple processes in a pipeline, from Super Resolution (SR) to Change Detection (CD) and Building Footprints (BF) extraction, is crucial for urban delineation, providing stakeholders with accurate results for informed decision-making. The project IRIX4US aims to monitor urban dynamics accurately for mobility, sustainability, urban planning, and accessibility in urban areas.

Engaging a broad set of relevant users and stakeholders, from public organizations to private industry, provided an opportunity to develop an EO-integrated solution tailored to the needs of urban experts and decision-makers. The solution consisted in a comprehensive AI pipeline:

1. Fine-Tuned SR of Sentinel-2 (S2) images: An AI-driven solution has been conducted to enhance their spatial resolution up to 2.5 meters. The model was trained using data from very high-resolution satellite imagery at a European scale. This process outputs a realistic representation of buildings, ensuring sharp edges and clear differentiation from surrounding objects such as roads.
2. Segmentation and BF extraction: IRIX4US adapts tested segmentation models to the super-resolved S2 imagery in urban landscapes, extracting visible and near-infrared (VNIR) bands and BFs using high-quality labels provided by champion users.
3. CD with AI Models: AI models have been employed for identifying differences in BFs between different images over time. These AI models were trained using temporal change data provided by champion users, allowing for the effective monitoring of urban development patterns in a multi-temporal context.

The results within the IRIX4US provides a dynamic and time-saving advancement for urban planning, proven feasible and scalable applications such as building segmentation, building change detection, illegal settlement identification and construction damage assessment.

This study investigates the efficacy of image restoration techniques of satellite EO images with a focus on super-resolution of Sentinel-2 and PlanetScope products. The ultimate goal is to develop a robust image restoration model capable of producing enhanced multispectral aerial-like imagery. The study is investigating several semi-supervised generative algorithms including SR-GANs, EDSR-GANs, WDSR-GANs, Lambda-PNN and W-Net.

The neural network architectures examined exhibit variations in their learning approaches and the potential utilization of a HR panchromatic component. For instance, the SR-GANs, EDSR-GANs, WDSR-GANs and WNet architectures are trained within a semi-supervised framework, i.e. supervising the training process of the generative models by incorporating multispectral HR target images. The Lambda-PNN network is trained within a fully unsupervised framework, hence no HR target images are adopted during its training phase. On the other hand, Lambda-PNN and WNet include a HR resolution panchromatic channel aiding the image super-resolution task. For our assessment with Lambda-PNN and WNet architectures, we opted for a panchromatic channel from an aerial image captured at a spatial resolution of 75cm either within the same date and in year 2019.

From our preliminary experiments, we have observed that GAN architectures which do not require a high-resolution panchromatic band can reconstruct HR scenes only for synthetic LR image datasets at 3m resolution obtained by downscaling aerial images. Pan-sharpening architectures like Lambda-PNN do not require a multispectral ground truth for training. However, when fed with Sentinel-2 and PlanetScope images, such architecture can produce synthetic images whose spatial structure is preserved (low structural loss) but yields unrealistic results regarding spectral information.

Among the preliminarily investigated architectures, the only architecture yielding consistent and plausible predictions is W-NET, a GAN fed with panchromatic and LR images and trained using HR target images employing a supervised approach. Furthermore, we are dedicating our effort in quantifying the reliability of generated images with respect to the introduction of spatial and spectral artifacts.

A myriad of global/regional datasets have provided valuable insights into environmental dynamics at global scale, including urban environments, however they often fall short of capturing the fine-scale nuances of the state and dynamics at local levels. Most existing datasets lacks the resolution and specificity required to address the diverse monitoring needs of urban stakeholders, particularly in densely populated or rapidly changing urban landscapes.

In response to these limitations, there is a growing recognition of the need for human-in-the-loop approaches, wherein stakeholders actively engage in the process of data collection, analysis, and interpretation to augment existing datasets and tailor monitoring efforts to local contexts. In this presentation, we will present a new agile cloud-based solutions that empower users to independently create and update urban datasets using free and open Copernicus Sentinel and NASA Landsat data, thereby overcoming the challenges associated with existing global datasets and fostering a more dynamic and responsive approach to urban environmental monitoring.

Developed as part of the EU 100Ktrees activity, based on the elaborated needs of municipalities worldwide, you will hear how complex machine learning frameworks and automated satellite data acquisition has been turned into user friendly cloud-based web applications for scalable mapping of urban environments on demand. You will learn how multitemporal satellite data and automated data analysis can turn raw satellite data into scalable information about urban heat islands, green spaces, flood exposure and impervious surfaces. And you will discover how these tools, and others, are vital to address the existing data gap across urban landscapes worldwide and how they can be used to underpin comprehensive and dynamic monitoring regimes.