PG-FM
A lightweight feature modulation method for RGB-T object detection that estimates the corruption type and severity of each modality to generate prompt vectors, applying input-dependent dynamic modulation to fused features and improving detection robustness under corrupted conditions without repairing the inputs.
Timeline (last 90 days)
The paper "Shifting Focus by Reliability: Geometric Fusion Modulation for Robust RGB-T Detection" proposes PG-FM (Prompt-Guided Feature Modulation), a lightweight adaptation scheme that does not restore the input but instead directly lets the detector adapt to changes in signal quality.
The core idea of PG-FM is not to restore the corrupted input but to estimate the corruption type and severity for each modality, generate prompt vectors, and use these prompts to modulate the fused features.
Under controlled RGB-T corruption simulation, PG-FM maintains performance on clean datasets while improving mAP50 under corruption conditions by up to 20.8 points on the FLIR dataset and up to 19.9 points on the KAIST dataset.
PG-FM can be transferred to unmodified, naturally captured external RGB-T datasets without retraining, demonstrating good generalization ability.
Directly letting the detector adapt to changes in signal quality is more efficient and more effective than the restore-then-detect route.
The lightweight nature of PG-FM makes it suitable for practical deployment, avoiding the latency problems caused by restoration methods.
All Facts (6)
The paper "Shifting Focus by Reliability: Geometric Fusion Modulation for Robust RGB-T Detection" proposes PG-FM (Prompt-Guided Feature Modulation), a lightweight adaptation scheme that does not restore the input but instead directly lets the detector adapt to changes in signal quality.
50%UnverifiedThe core idea of PG-FM is not to restore the corrupted input but to estimate the corruption type and severity for each modality, generate prompt vectors, and use these prompts to modulate the fused features.
50%UnverifiedUnder controlled RGB-T corruption simulation, PG-FM maintains performance on clean datasets while improving mAP50 under corruption conditions by up to 20.8 points on the FLIR dataset and up to 19.9 points on the KAIST dataset.
50%UnverifiedPG-FM can be transferred to unmodified, naturally captured external RGB-T datasets without retraining, demonstrating good generalization ability.
50%UnverifiedDirectly letting the detector adapt to changes in signal quality is more efficient and more effective than the restore-then-detect route.
50%UnverifiedThe lightweight nature of PG-FM makes it suitable for practical deployment, avoiding the latency problems caused by restoration methods.
50%