Multimodal Transfer Learning for Crisis Response: A Unified DeBERTa–ResNet Framework on CrisisMMD
DOI:
https://doi.org/10.57041/8fpqd649Keywords:
Multimodal deep learning, transfer learning, crisis informatics, DeBERTa, ResNet, social media, disaster response, CrisisMMDAbstract
Social media platforms generate large volumes of multimodal content during natural disasters that can support situational awareness and humanitarian decision-making. However, effectively analyzing both textual and visual information remains challenging due to weak cross-modal alignment, severe class imbalance, noisy user-generated data, and limited labeled samples. This paper presents a unified multimodal transfer-learning framework that combines a frozen DeBERTa-v3-base text encoder with a frozen ResNet-50 image encoder through a simple late-fusion strategy for crisis-related classification. The architecture projects the unimodal features into a common 256-dimensional space, concatenates them, and feeds the resulting representation into a lightweight multi-layer perceptron classifier. The framework is evaluated on all three official tasks of the CrisisMMD v2.0 dataset. Experimental results show an accuracy of 0.76 (macro-F1 = 0.74) on the informativeness task, 0.43 (macro-F1 = 0.39) on the humanitarian categories task, and 0.69 (macro-F1 = 0.64) on the damage severity task. The model performs relatively well on classes with distinctive multimodal signatures, while residual confusion remains on semantically overlapping or visually subtle categories. For the damage severity task, which is primarily image-based, the multimodal model achieves essentially the same performance as an image-only configuration, indicating limited contribution from text. The study also acknowledges that formal text-only and image-only ablation experiments under the same protocol were not conducted, so the precise benefit of fusion cannot be rigorously quantified. Overall, the work demonstrates that a computationally efficient frozen late-fusion architecture can serve as a practical and reproducible reference point for multimodal crisis informatics under the resource constraints typical of real-world disaster response.Downloads
Published
2026-11-13
Issue
Section
Proceedings of 12th APICEE
License
Copyright (c) 2026 https://grsh.org/journal1/index.php/ijeet/cr

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
How to Cite
Multimodal Transfer Learning for Crisis Response: A Unified DeBERTa–ResNet Framework on CrisisMMD. (2026). International Journal of Emerging Engineering and Technology, 5(1-1), 60-71. https://doi.org/10.57041/8fpqd649