Conference Agenda

The characterization of structural changes in urban areas, and specifically the current trend towards verticalization, is a crucial step toward understanding urbanization phenomena at the global level. The use of spaceborne remote sensing data for this task has grown over the last decade, focusing on SAR and multispectral data, often jointly used. In particular, very high resolution (VHR) data, increasingly available in large volumes, calls for efficient and accurate 2D/3D change detection techniques, able to go beyond existing methods, focused either on detecting buildings footprint changes over time or on extracting changes in the number of floors/height of the same buildings. In this work, we present a complete framework applicable to VHR SAR temporal sequences able to characterize 2D & 3D changes at the same time. The methodology involves the extraction of altered buildings in the initial phase and subsequently performs a building segmentation, with specific emphasis on discerning changes in building height. The identification of changed urban core areas is performed using an unsupervised temporal clustering applied to the original sequence of SAR amplitude and coherence images. Subsequently, building segmentation in the initial and ending image of the sequence is performed thanks to a deep learning U-Net architecture, allowing a quantitative characterization of the 2D changes. Finally, the building height extraction is accomplished through the utilization of a novel ResNet deep learning architecture. The proposed framework has been tested using a time series of three years of COSMO-SkyMed data, from January 2019 to November 2021, over the city of Milan, Italy. Experimental results in the "Gae Aulenti" and "City Life" areas show that 2D and 3D changes are correctly detected and the use of the proposed machine and deep learning framework significantly increases the ability to achieve a better characterization of structural changes in urban areas.

While the significance of built-up volume and density for future sustainability and resilience of the built environment is widely recognized, the interplay between 3D built-up patterns, economic development, and local planning policies remains relatively unexplored. To address this gap, we integrate novel global data from the World Settlement Footprint 3D (WSF® 3D) - encompassing building area, height, and volume -, with socioeconomic statistics and information on planning policies across all countries and over 12,000 urban clusters worldwide. Our analysis begins with enhancing the original WSF3D dataset to provide a more accurate representation of high-rise buildings exceeding 50 meters. This enhancement involves integrating data from the Emporis database, a leading source of tall buildings information globally. Subsequently, we merge the enhanced WSF3D data (WSF3Dv2) with socioeconomic statistics sourced from the World Development Indicators (WDI) database and data on zoning regulations and land-use policies provided by the World Bank. For the city-level analyses, we additionally utilize the Urban Center database provided by the Joint Research Center (JRC) of the European Commission. Through our study, we offer a comprehensive and spatially detailed understanding of the global 3D building stock, unveiling intricate relationships between 3D settlement morphology, economic development, and local planning regulations. This empirical evidence strengthens the understanding of the global megatrend of urbanization and provides crucial insights to enhance the effectiveness of land use and spatial development policies aimed at promoting (urban) sustainability and resilience.

Accurate building height estimation is essential for sustainable urban planning, monitoring, and environmental impact analysis. However, conducting large-scale building height estimation at a fine spatial resolution is challenging, especially using open-access satellite data. Existing large-scale solutions provide height at coarse spatial resolution (500 m - 90 m), a better resolution can provide more comprehensive understanding of urban development.

We propose an advanced deep learning model, T-SwinUNet, specifically designed for large-scale building height estimation at a fine spatial resolution of 10 m. The model harnesses salient features from the spatial, spectral, and temporal dimensions of the Sentinel-1 SAR and Sentinel-2 MSI time-series data.

In T-SwinUNet, we integrated the semantic feature learning capabilities of the CNN encoder with the local/global feature comprehension capabilities of Swin transformers. With added temporal attention, the model learns the correlation between constant features (mostly geometry) and variable features (e.g. shadow) of building objects over time. This not only aids in differentiating building from non-building objects but also condition model to learn salient building height features. We equipped T-SwinUNet with uncertainty prediction, which helps in assessing model’s robustness and transferability to new areas. Knowing uncertainty in predictions helps stakeholders to make informed and careful decisions.

The model is evaluated on data from the Netherlands, Switzerland, Estonia, and Germany. The extensive evaluation and comparison with state-of-the-art DL models show that our proposed T-SwinUNet model surpasses SOTA by achieving an RMSE of 1.89 m at 10 m spatial resolution. We conducted a detailed ablation study to understand the impact of time-series data, each modality, multi-task learning and others. Further assessment at 100 m resolution shows that our predicted building heights (0.29 m RMSE, 0.75 $R^{2}$) also outperformed the global building height product GHSL-Built-H R2023A product (0.56 m RMSE and 0.37 $R^{2}$).

Accurate determination of building heights is crucial for urban 3D development analysis and disaster risk assessment. State-of-the-art (SOTA) techniques typically treat height retrieval from buildings as a regression problem. For instance, [1, 2] propose supervised Multimodal Deep Learning (DL) frameworks to estimate building heights using Sentinel-1 (Sen-1) and Sentinel-2 (Sen-2) data.

In this work, we employ an Attention-based U-Net model for building height estimation, relying solely on radar information provided by Very High-Resolution (VHR) COSMO-SkyMed (CSK) data. We utilize a CSK Level-1D Stripmap Himage acquisition mode with a spatial resolution of 2.5 meters in Ascending orbit direction, covering the entire metropolitan area of Milan city for the model training. Ground Truth (GT) labels are derived from the Normalized Digital Surface Model (nDSM) and refined by using a binary footprint mask from OpenStreetMap (OSM) data to exclude non-built-up (BU) areas.

Our proposed model exhibits robust performances in terms of Root Mean Square Error (RMSE) on external test sites in the cities of Pavia, Sant’Angelo Lodigiano, and Lodi, with final error values of 0.781, 0.731, and 1.487 meters, respectively, outperforming previous studies [1, 2]. To further validate the model, the average error outside the buildings was also evaluated. The results reveal a general underestimation of building height as their actual height increases, indicating an avenue for future research.

Notably, our method capitalizes on CSK radar data, offering a swift solution for mapping 3D BU area features owing to its VHR, weather-independent capabilities, and rapid emergency response.

REFERENCES

[1] Yadav, R., https://arxiv.org/abs/2307.01378

[2] Bowen, C. et al., https://www.sciencedirect.com/science/article/pii/S1569843223002236

CARS is a CNES open-source 3D reconstruction software part of the Constellation Optique 3D (CO3D) mission. CARS stands out from other MultiView-Stereo methods due to its highly parallelizable design, capable of addressing large volumes of data for processing on an HPC cluster or personal machine.

Designed as a modular set of applications, CARS allows user to plug in new public or confidential contributions, such as new applications or pipelines. Using this plug-in concept, we recently addressed the need for LOD2 semantized meshes on urban areas. The new CARS plug-in makes the most out of intermediate CARS products such as uncertainty and classification layers to create LOD2 meshes.

Surface temperatures are central to the surface energy balance, and particularly in cities link to human thermal comfort and energy consumption. Since, cities have a strong vertical component, estimating the surface temperature for the complete three dimensional (3D) urban surface is important. While satellite remote sensing holds a strong potential in observing city-wide and global urban surface temperatures, these need to be complemented with in-situ infrared observations and modeling to achieve the assessment of the complete 3D surface temperature.

In this study, a sub-building scale three-dimensional (3D) energy balance model is evaluated using a novel ground-based thermal camera observatory in Berlin. Four thermal cameras (OptrisPI160) were mounted on an 80 m tall building, overlooking a mix of low-rise residential, parks, trees from all cardinal directions. 3D urban surface temperature was modeled using an extension of the energy balance model TUF-3D model (Krayenhoff and Voogt, 2007) including a vegetation component (VTUF-3D, Nice et al., 2018).) Earth Observation products (land cover, building and vegetation heights) were used to parametrize the 3D urban form of the study domains, while forcing data were available from meteorological measurements on the same building. A total area of 0.2 km² was simulated with a 5 m spatial resolution for 3 – 6 August 2022. The camera brightness temperatures observations were corrected for emissivity to conclude to surface temperature. Detailed emissivity maps were derived using land cover and reference spectral library information.

The diurnal pattern of modelled surface temperature was found similar to the observed one for different surface types (MAE ranging between 3-10 C depending on the area and time of day, with error increasing during daytime). VTUF-3D simulated surface temperatures for more areas in Berlin will be used to assess the complete temperature from satellite thermal observations, such as Sentinel-3.