New publication on deep multitask learning for street-level image classification

New publication on deep multitask learning for street-level image classification

September 27, 2023

New publication on deep multitask learning for street-level image classification

 

Researchers from the Earth Observation Center (EOC) of the German Aerospace Center (DLR) in Oberpfaffenhofen, the Department of Geography of the University of Bonn, and our Earth Observation Research Cluster of the University Würzburg teamed up to introduce a novel deep multitask learning architecture that specifically encodes cross-task interdependencies within the setting of multiple image classification problems. The paper titled “Deep multitask learning with label interdependency distillation for multicriteria street-level image classification” was just published in the ISPRS Journal of Photogrammetry and Remote Sensing by Patrick Aravena Pelizari, Christian Geiß, Sandro Groth and Hannes Taubenböck.

 

 

Here is the abstract of the paper:

Multitask learning (MTL) aims at beneficial joint solving of multiple prediction problems by sharing information across different tasks. However, without adequate consideration of interdependencies, MTL models are prone to miss valuable information. In this paper, we introduce a novel deep MTL architecture that specifically encodes cross-task interdependencies within the setting of multiple image classification problems. Based on task-wise interim class label probability predictions by an intermediately supervised hard parameter sharing convolutional neural network, interdependencies are inferred in two ways: i) by directly stacking label probability sequences to the image feature vector (i.e., multitask stacking), and ii) by passing probability sequences to gated recurrent unit-based recurrent neural networks to explicitly learn cross-task interdependency representations and stacking those to the image feature vector (i.e., interdependency representation learning). The proposed MTL architecture is applied as a tool for generic multi-criteria building characterization using street-level imagery related to risk assessments toward multiple natural hazards. Experimental results for classifying buildings according to five vulnerability-related target variables (i.e., five learning tasks), namely height, lateral load-resisting system material, seismic building structural type, roof shape, and block position are obtained for the Chilean capital Santiago de Chile. Our MTL methods with cross-task label interdependency modeling consistently outperform single task learning (STL) and classical hard parameter sharing MTL alike. Even when starting already from high classification accuracy levels, estimated generalization capabilities can be further improved by considerable margins of accumulated task-specific residuals beyond +6% κ. Thereby, the combination of multitask stacking and interdependency representation learning attains the highest accuracy estimates for the addressed task and data setting (up to cross-task accuracy mean values of 88.43% overall accuracy and 84.49% κ). From an efficiency perspective, the proposed MTL methods turn out to be substantially favorable compared to STL in terms of

training time consumption.

 

Here is the link to the paper: https://www.sciencedirect.com/science/article/pii/S0924271623002332

you may also like:

‘Super Test Site Würzburg’ project meeting

‘Super Test Site Würzburg’ project meeting

After the successful "Super Test Site Würzburg" measurement campaign in June (please see here: https://remote-sensing.org/super-test-site-wurzburg-from-the-idea-to-realization/ ), the core team from the University of Würzburg, the Karlsruhe Institute of Technology,...

EORC Talk: Geolingual Studies: A New Research Direction

EORC Talk: Geolingual Studies: A New Research Direction

On July 19th, Lisa Lehnen and Richard Lemoine Rodríguez, two postdoctoral researchers of the Geolingual Studies project, gave an inspiring presentation at the EORC talk series.   In the talk titled "Geolingual Studies – a new research direction", they...

EO support for UrbanPArt field work

EO support for UrbanPArt field work

From May to September, Karla Wenner, a PhD student at the Juniorprofessorship for Applied Biodiversity Science, will be sampling urban green spaces and semi-natural grasslands in Würzburg as part of the UrbanPArt project. Our cargo bikes support the research project...

Cinematic drone shots

Cinematic drone shots

We spend quite some time in the field conducting field work, from lidar measurements to vegetation samples in order to correlate it with remote sensing data to answer various research questions concerning global change. Field work is always a 24/7 work load and...

Sommer event at DLR EOC

Sommer event at DLR EOC

Some of our staff joined the DLR EOC summer event and spend the day talking with various colleagues from DLR as well as experiencing the newest developments such as the virtual reality experiences by the department of Nils Sparwasser. Beside various topical...

Media reports on our work

Media reports on our work

We recently reported on our study published in The LANCET Planetary Health Journal on the impact of urban configuration types on urban heat islands, air pollution, CO2 emissions, and mortality – please see here:...