Zero-Shot Social Media Crisis Classification: A Training-Free Multimodal Approach

Abstract

Rapid classification of social media content during humanitarian crises is essential for effective disaster relief; however, traditional approaches require extensive annotated training data, which are often unavailable during new disasters. This paper presents a training-free, multimodal classification framework that leverages zero-shot vision-language models to analyze disaster-related social media content without task-specific training. The framework employs a two-stage prompt-engineered pipeline using the locally deployable Mistral-Small-3.1-24B-Instruct model, performing binary informativeness detection followed by multiclass categorization into eight humanitarian categories through structured JSON output generation. Evaluation on the CrisisMMD dataset of 18,082 multimodal samples from seven natural disasters demonstrated binary F1 scores above 0.84 for both text and image informativeness detection and weighted F1 scores of 0.61 (text) and 0.72 (image) for humanitarian categorization. The framework generalizes consistently across all disaster types with minimal performance variance (standard deviation below 0.031) and operates entirely on local infrastructure without cloud dependencies, requiring only moderate GPU resources. By eliminating training data requirements, this approach enables immediate deployment during new disasters, demonstrating that zero-shot multimodal classification achieves practically relevant performance for real-time crisis response.