VCR-Bench
A modular open-source benchmark framework for evaluating video classification robustness. It integrates a complete evaluation pipeline covering video loading, classifier wrapping, adversarial attacks and defenses, and perceptual quality metrics. The framework supports 30 video classification models, 14 adversarial attack methods, and 10 defense wrappers, providing a unified and standardized protocol for video classification robustness research.
Timeline (last 90 days)
VCR-Bench is a modular open-source benchmark framework for evaluating the robustness of video classification.
VCR-Bench integrates 30 video classification models.
VCR-Bench integrates 14 adversarial attack methods.
VCR-Bench integrates 10 defense wrappers.
The research team evaluated video classifiers, attack methods, and defense strategies on a subset of the Kinetics-400 dataset.
VCR-Bench comprehensively measures robustness across multiple dimensions including clean accuracy, attack success rate, perceptual quality, runtime, and memory consumption.
The VCR-Bench project has been open-sourced on GitHub (msu-video-group/vcr-bench).
VCR-Bench comes with detailed installation documentation, reproducible run presets, component extension interfaces, and scripts for reproducing the paper's results.
VCR-Bench's component extension interface enables it to continuously evolve as new models, new attacks, and new defenses emerge.
VCR-Bench's current evaluation is based only on a subset of Kinetics-400, and the datasets and scenarios covered remain limited.
All Facts (10)
VCR-Bench's current evaluation is based only on a subset of Kinetics-400, and the datasets and scenarios covered remain limited.
50%UnverifiedVCR-Bench integrates 30 video classification models.
50%UnverifiedVCR-Bench integrates 14 adversarial attack methods.
50%UnverifiedVCR-Bench integrates 10 defense wrappers.
50%UnverifiedThe research team evaluated video classifiers, attack methods, and defense strategies on a subset of the Kinetics-400 dataset.
50%UnverifiedVCR-Bench comprehensively measures robustness across multiple dimensions including clean accuracy, attack success rate, perceptual quality, runtime, and memory consumption.
50%UnverifiedVCR-Bench is a modular open-source benchmark framework for evaluating the robustness of video classification.
50%UnverifiedVCR-Bench comes with detailed installation documentation, reproducible run presets, component extension interfaces, and scripts for reproducing the paper's results.
50%UnverifiedVCR-Bench's component extension interface enables it to continuously evolve as new models, new attacks, and new defenses emerge.
50%UnverifiedThe VCR-Bench project has been open-sourced on GitHub (msu-video-group/vcr-bench).
50%