Patrick Aravena Pelizari defended his PhD Thesis titled “Multihazard-Expositionsmodellierung mit multimodalen Geobilddaten und Deep Learning” [in engl.: Multi-hazard exposure modelling with multimodal geospatial data and deep learning] at the Julius-Maximilians-University Würzburg successfully on the 12th of December 2024.
The Earth Observation Research Cluster congratulates him very much for his achievements in the research fields of remote sensing, AI and georisks. The defense was led by Prof. Katharina Kemmer. Patrick was supervised by Prof. Hannes Taubenböck (DLR/JMU), Prof. Stefan Dech (DLR/JMU) and Prof. Roland Baumhauer (JMU). His mentor at the German Remote Sensing Data Center (DFD) of the German Aerospace Center (DLR) was Prof. Christian Geiß.
We were very happy to welcome a large audience from the German Aerospace Center (DLR), the University of Würzburg and friends and family of the PhD candidate.
Here is the abstract of the thesis:
The ongoing processes of population growth, urbanization, and climate change have led to a drastic increase in the number of people and assets exposed to natural hazards worldwide. For effective risk reduction and disaster management, up-to-date exposure models with detailed, spatially localized information on the built environment and its vulnerability are essential. However, such data are often insufficiently available. At the same time, holistic vulnerability assessments across multiple natural hazards place high demands on exposure models in terms of thematic information and spatial resolution, as i) different building attributes may affect vulnerability to different hazards, and ii) natural hazards may differ in spatial scale and exhibit distinct spatial variabilities.
Today, geo-referenced imaging sensor data are an essential source for the automated extraction of spatial information. This has been largely driven by the ever-increasing availability of data collection initiatives – both remote and in-situ sensing – along with social media and advancements in data analysis methods, particularly in the field of artificial intelligence. The overarching goal of this dissertation is to demonstrate methods based on heterogeneous geospatial image data and current deep learning (DL) techniques that enable efficient, high spatial resolution, large-scale multicriteria characterization of the built environment for multi-risk analysis. This objective is addressed through three sub-studies, with the test area being the earthquake-prone metropolis of Santiago, Chile.
The first study investigates the potential of Convolutional Neural Networks (CNN) and StreetLevel Imagery (SLI) for the automated collection of vulnerability-related building characteristics. The approach involves a hierarchical workflow to acquire and structure heterogeneous SLI in an application-oriented manner. State-of-the-art CNN are used to derive: i) the Seismic Building Structure Type (SBST), ii) the Lateral Load Resisting System (LLRS) material, and iii) building height. These attributes reflect a building’s load-bearing structure and, consequently, its resistance to forces exerted by natural hazard events. The experimental results show classification accuracies above 𝜅 = 0.81 for all tasks, underlining the high potential of SLI and DL for in situ building data collection for natural hazard risk assessment at large spatial scales.
The second study aims at a synergistic and efficient multi-criteria characterization of buildings for multi-hazard risk assessment. Deep Multitask Learning (MTL) is used to address this challenge. The proposed deep MTL architecture enhances prediction accuracy in multiple image classification tasks by accounting for cross-task interdependencies. These interdependencies are inferred based on task-wise interim class label probability predictions from an intermediately supervised hard parameter sharing CNN: i) by directly stacking label probability sequences to the image feature vector (i.e., multitask stacking), and ii) by passing probability sequences to recurrent neural networks (Gated Reccurent Units) to explicitly learn cross-task interdependency representations (i.e., interdependency representation learning). The MTL architecture is applied to classify buildings according to five target variables: height, LLRS material, SBST, roof shape, and block position. The classification accuracies of the new MTL approaches outperform both single-task learning and classical hard-parameter sharing MTL. Already high initial estimated generalization capabilities can be significantly increased with accumulated task-specific residuals of more than +6% 𝜅, achieving mean cross-task accuracy values of up to 88.43 % overall accuracy and 84.49 % 𝜅. In terms of training time, the proposed MTL methods also prove very efficient.
The third study explores the potential of integrating heterogeneous multimodal geospatial image data – SLI, very high-resolution optical remote sensing data, and a normalized digital surface model – for large-scale characterization of buildings exposed to natural hazards. An
object-based multi-Input-/multi-Output DL methodology is presented, enabling the synergistic fusion of multi-sensor data for multi-criteria building classification. To address the issue of partially missing SLI data, the transformer-based SLI Spatial Context Encoder is proposed. This model leverages spatial correlations between physical-structural building attributes to make the semantic information of available SLI widely accessible. Task-wise Modality Attention (TMA) fusion is proposed to integrate the multimodal information. TMA optimizes feature representations for individual inference tasks separately, according to their specific requirements. Considering the two data scenarios – SLI available and SLI missing – a comprehensive experimental cross-comparison of generalization capabilities is conducted to evaluate the added value of the different modalities, their combinations, and TMA fusion. The results emphasize the high semantic value of SLI and the derived spatio-contextual representations for capturing physical-structural building characteristics. Additionally, they demonstrate that all modality combinations offer positive synergies. TMA fusion consistently outperforms benchmark methods. The most accurate models are subsequently applied to derive a spatially continuous exposure model.
The presented methodology enables the automated, large-scale collection of vulnerability-related building attributes with an unprecedented combination of spatial and thematic resolution. This level of detail is essential for a consistent assessment of multi-hazard vulnerability and, consequently, for effective risk and disaster management. The results of this dissertation offer in-depth insights into the potential of multimodal geoimage data and DL for the efficient provision of exposure information.
