VERA: A Responsible AI Evaluation Dashboard for the EU AI Act

In regulated industries, deploying large language models (LLMs) often requires institutions to first obtain technical evidence of responsible AI practices before approval can be granted. However, the benchmarks that generate this evidence typically run via command line, while the people responsible for approval are mostly not engineers. This contradiction becomes especially acute under the compliance pressure of the EU AI Act. The open-source dashboard VERA was created precisely to bridge this gap.

From COMPL-AI to VERA: Making Compliance Evidence Readable

The COMPL-AI project provides a technical interpretation of the EU AI Act, organizing compliance evidence into six principle groups and eighteen technical requirements, twelve of which can be measured through existing benchmarks. These benchmark suites are the core tools for generating evidence, but their output consists of raw files that are difficult for non-technical approvers to interpret directly.

VERA (Verified Responsible AI) is an open-source, self-hosted dashboard that has been deployed internally at a large European bank to support responsible AI processes. Reviewers simply select a model and can run public benchmarks against it. VERA scores the twelve measurable requirements and records the energy consumed by each evaluation, providing run-level evidence of environmental impact. This design significantly narrows the gap between technical measurement and business approval.

COMPL-AI is an open-source compliance evaluation framework for the EU AI Act, developed through collaboration between academia and industry. Its core contribution lies in "translating" the abstract legal provisions of the Act into actionable technical requirements: it categorizes compliance evidence into six principle groups along dimensions such as fairness, transparency, and robustness, then further subdivides them into eighteen technical requirements, explicitly identifying which of these can be quantified through existing benchmarks (such as safety evaluations, bias detection, and adversarial robustness testing). COMPL-AI itself does not provide a user interface; instead, it invokes benchmark suites via command line and outputs raw result files, making it more suitable for engineering teams. It is often regarded as an "intermediate-layer specification" connecting legal text and engineering practice, and VERA is a visualization and workflow shell built on top of this specification.

How It Works: Model Selection and Automated Scoring

VERA's core workflow revolves around "select model—run benchmarks—generate scores." After reviewers select an open-weight model on the dashboard, the system automatically executes the preset benchmark suites and outputs structured scoring results according to the COMPL-AI requirements framework.

The research team tested four deployed open-weight models through this requirements pipeline, and the dashboard displayed corresponding scores for each measurable requirement. One detail worth noting: VERA not only focuses on model capabilities but also incorporates energy consumption into the evaluation dimensions—something uncommon among responsible AI tools. Quantifying environmental impact helps institutions consider deployment sustainability beyond compliance.

Open-weight models refer to models whose parameters are publicly downloadable and can be deployed and fine-tuned locally or in private environments, as opposed to closed-source models that offer only API access. In regulated industries (such as banking), open-weight models offer clear advantages: data stays within the internal network, inference processes are auditable, and versions can be pinned, making it easier to meet compliance and data protection requirements. But precisely because of this, institutions must bear the evaluation responsibility themselves—there is no vendor to provide compliance statements on their behalf, and tools like VERA are needed to complete benchmark testing and evidence retention internally. VERA's choice to prioritize open-weight models as evaluation targets aligns with the real-world needs of financial institutions and other organizations that deploy self-hosted solutions.

User Study: Dashboard Cuts Evaluation Time by Nearly Half

To validate the dashboard's practical effectiveness, the research team conducted a within-subjects study comparing participants reading results from raw output files versus reading results through the VERA dashboard.

The data shows that with the dashboard, the average time required to complete a responsible AI assessment dropped from 56 seconds to 30 seconds, an efficiency improvement of approximately 46%. In an internal questionnaire, the dashboard received a usefulness rating of 4.36 out of 5, indicating high acceptance among non-technical approvers. This result demonstrates that transforming complex benchmark outputs into an intuitive scoring interface can genuinely lower the barrier to compliance assessment.

Within-subjects studies are an experimental design in which the same group of participants experiences two or more experimental conditions sequentially (here, "reading raw output files" and "using the VERA dashboard") and their performance is compared. Compared to between-subjects designs, this approach has the advantage of eliminating individual differences (such as professional background and reading speed) as confounds and requires a smaller sample size. However, it carries risks of order effects and learning effects, which are typically mitigated through condition-order randomization or washout periods. The 56-second to 30-second comparison in this article comes from the same participants' task completion times under both conditions, making the efficiency improvement conclusion more convincing.

Open Source and Reproducible: Driving Cross-Organizational Adoption

The VERA dashboard and research materials have been publicly released to promote reproducibility and replicability across different organizational contexts. For institutions looking to implement EU AI Act compliance processes, this open-source solution provides a template worth emulating.

From a broader perspective, VERA's value lies not just in the tool itself but in the collaboration model it demonstrates: technical teams handle benchmarks and measurement, business teams complete approvals through the dashboard, and both fulfill their respective roles on the same platform. As AI regulatory requirements become more granular, intermediate-layer tools that connect technology and governance will become increasingly important.

Key Takeaways

• VERA transforms twelve of COMPL-AI's eighteen technical requirements—the measurable ones—into dashboard scores, lowering the interpretation barrier for non-technical personnel.
• In an actual deployment at a large European bank, the dashboard reduced responsible AI assessment time from 56 seconds to 30 seconds, with a usefulness rating of 4.36/5.
• The tool is open-source and self-hosted, supports cross-organizational reproduction, and additionally records energy consumption data to assess environmental impact.
• The solution demonstrates a viable path for collaboration between technical and governance teams, offering reference value for EU AI Act compliance implementation.

One-line Summary

Open-source dashboard VERA transforms LLM compliance benchmark outputs into visual scores, cutting evaluation time by 46%.

Paragraph Summary

VERA is an open-source, self-hosted dashboard designed to bridge the gap between technical measurement and business approval in LLM compliance assessment. Built on the COMPL-AI framework, it converts twelve of the eighteen measurable technical requirements of the EU AI Act into intuitive scores and additionally records the energy consumption of each evaluation. VERA has been deployed internally at a large European bank, where reviewers can select an open-weight model and automatically run benchmarks to generate structured results. A within-subjects study showed that evaluation time dropped from 56 seconds to 30 seconds with the dashboard, an efficiency improvement of approximately 46%, with a usefulness rating of 4.36/5. The tool is open-source and supports cross-organizational reproduction, providing a replicable template for responsible AI implementation in regulated industries.