Toward High-Resolution UAV Imagery Open-Vocabulary Semantic Segmentation - Zimo Chen; Yuxiang Xie; Yingmei Wei (Drones)
Unmanned Aerial Vehicle (UAV) image semantic segmentation faces challenges in recognizing novel categories due to closed-set training paradigms and the high cost of annotation. While open-vocabulary semantic segmentation (OVSS) leverages vision-language models like CLIP to enable flexible class recognition, existing methods are limited to low-resolution images, hindering their applicability to high-resolution UAV data. Current adaptations—downsampling, cropping, or modifying CLIP—compromise either detail preservation, global context, or computational efficiency. To address these limitations, we propose HR-Seg, the first high-resolution OVSS framework for UAV imagery, which effectively integrates global context from downsampled images with local details from cropped sub-images through a novel cost-volume architecture. We introduce a detail-enhanced encoder with multi-scale embedding and a detail-aware decoder for progressive mask refinement, specifically designed to handle objects of varying sizes in aerial imagery. We evaluated existing OVSS methods alongside HR-Seg, training on the VDD dataset and testing across three benchmarks: VDD, UDD, and UAVid. HR-Seg achieved superior performance with mIoU scores of 89.38, 73.67, and 55.23, respectively, outperforming all compared state-of-the-art OVSS approaches. These results demonstrate HR-Seg’s exceptional capability in processing high-resolution UAV imagery.