[KongchangAI]
ConceptDocXEval框架

DocXEval

The first systematic evaluation framework for explainability (XAI) in multimodal document understanding. It comprises a benchmark dataset, evaluation metrics, and a meta-evaluation method for assessing the faithfulness and explainability of explanation methods.

Timeline (last 90 days)

Oct 28

DocXEval is the first systematic evaluation framework for explainability (XAI) in multimodal document understanding.

Unverified50%
Oct 28

DocXEval systematically evaluates 13 XAI methods, 3 types of document understanding tasks, 8 datasets, 4 models and 17 evaluation metrics.

Unverified50%
Oct 28

The evaluation results show that only a few methods maintain consistent performance across all evaluation criteria, among which Occlusion performs best overall on the two dimensions of faithfulness and interpretability.

Unverified50%
Oct 28

DocXEval's meta-evaluation found moderate to strong agreement among several faithfulness metrics and complexity metrics, while also identifying some metrics that produce unreliable results.

Unverified50%
Oct 28

DocXEval's code is publicly available on GitHub at https://github.com/saifullah3396/docxeval/.

Unverified50%
Oct 28

For tasks such as semantic entity recognition, DocXEval introduces efficient multi-target adapted versions of explanation methods and metrics.

Unverified50%

All Facts (6)

Source Articles