New publication on deep multitask learning for street-level image classification

New publication on deep multitask learning for street-level image classification

September 27, 2023

New publication on deep multitask learning for street-level image classification

 

Researchers from the Earth Observation Center (EOC) of the German Aerospace Center (DLR) in Oberpfaffenhofen, the Department of Geography of the University of Bonn, and our Earth Observation Research Cluster of the University Würzburg teamed up to introduce a novel deep multitask learning architecture that specifically encodes cross-task interdependencies within the setting of multiple image classification problems. The paper titled “Deep multitask learning with label interdependency distillation for multicriteria street-level image classification” was just published in the ISPRS Journal of Photogrammetry and Remote Sensing by Patrick Aravena Pelizari, Christian Geiß, Sandro Groth and Hannes Taubenböck.

 

 

Here is the abstract of the paper:

Multitask learning (MTL) aims at beneficial joint solving of multiple prediction problems by sharing information across different tasks. However, without adequate consideration of interdependencies, MTL models are prone to miss valuable information. In this paper, we introduce a novel deep MTL architecture that specifically encodes cross-task interdependencies within the setting of multiple image classification problems. Based on task-wise interim class label probability predictions by an intermediately supervised hard parameter sharing convolutional neural network, interdependencies are inferred in two ways: i) by directly stacking label probability sequences to the image feature vector (i.e., multitask stacking), and ii) by passing probability sequences to gated recurrent unit-based recurrent neural networks to explicitly learn cross-task interdependency representations and stacking those to the image feature vector (i.e., interdependency representation learning). The proposed MTL architecture is applied as a tool for generic multi-criteria building characterization using street-level imagery related to risk assessments toward multiple natural hazards. Experimental results for classifying buildings according to five vulnerability-related target variables (i.e., five learning tasks), namely height, lateral load-resisting system material, seismic building structural type, roof shape, and block position are obtained for the Chilean capital Santiago de Chile. Our MTL methods with cross-task label interdependency modeling consistently outperform single task learning (STL) and classical hard parameter sharing MTL alike. Even when starting already from high classification accuracy levels, estimated generalization capabilities can be further improved by considerable margins of accumulated task-specific residuals beyond +6% κ. Thereby, the combination of multitask stacking and interdependency representation learning attains the highest accuracy estimates for the addressed task and data setting (up to cross-task accuracy mean values of 88.43% overall accuracy and 84.49% κ). From an efficiency perspective, the proposed MTL methods turn out to be substantially favorable compared to STL in terms of

training time consumption.

 

Here is the link to the paper: https://www.sciencedirect.com/science/article/pii/S0924271623002332

you may also like:

New staff member Luisa Pflumm

New staff member Luisa Pflumm

Luisa Pflumm joined the Earth Observation Research Cluster in May 2024 as part of the EcoGlob project and is working with the UAS team in the context of remote sensing for biodiversity and nature conservation. She received her Bachelor's degree in Geography from the...

New team member: Ása Dögg Adalsteinsdottir

New team member: Ása Dögg Adalsteinsdottir

Ása Dögg Adalsteinsdottir joined the Earth Observation Research Cluster in May 2024 as a member of the EO4CAM project team. After earning a bachelor's degree in geography from the University of Iceland, she moved to Germany to study in our EAGLE master's program. She...

NEW TEAM MEMBER: CHRISTIAN SCHÄFER

NEW TEAM MEMBER: CHRISTIAN SCHÄFER

Christian Schäfer joined the EO4CAM project in May 2024. He received his Master's degree in 2017 from Julius-Maximilians-Universität Würzburg (JMU), focusing on GIS-based synthesis of transboundary soil maps. During his work in the JMU BigData@Geo project, he enhanced...

GGW talk on geodata, mobility and social media

GGW talk on geodata, mobility and social media

On Monday the 13th of May our PhD students Ariane Droin and Johannes Mast were holding a talk at the Geographische Gesellschaft Würzburg organised by the Fachschaft Geographie about 'Geodaten, Mobilität und soziale Medien. Big data und die lokale Perspektive der...

NetCDA kick-off workshop

NetCDA kick-off workshop

Yesterday, on May 16th, the partners of the project "European Academic Network for Capacity Development in Climate Change Adaptations in Africa" (NetCDA) met to jointly and officially kick-off their project. The NetCDA team at the University of Würzburg invited all...