Indexing data-testid and ARIA Attributes: A New Approach to Solving AI Test Hallucinations

AI coding agents generating integration and end-to-end tests is one of the highest-ROI uses in developer workflows today. But agents keep stumbling on one specific step: guessing DOM element selectors.

The typical failure scenario looks like this — the agent writes await page.click(".btn-primary-submit"), and the test fails because that class name doesn't exist. It's not that the logic is wrong; it's that the agent can't "see" the frontend's actual structure and can only guess. TokenCap's solution is to extract the frontend's test-locator contract ahead of time and feed it to the agent, rather than letting it guess blindly.

The Root Cause: Agents Guessing at Selectors

The most fragile part of AI-generated tests isn't the assertion logic — it's figuring out how to locate elements on the page. When developers write tests, they have the component structure in their heads: they know which button has a data-testid, which dropdown has role="combobox". The agent has none of this context.

What it can typically see is just scattered code snippets or vague page descriptions, so it falls back to the most primitive strategy: assembling a plausible-looking CSS selector. Class names, hierarchy paths, tag combinations — all based on the language model's priors. Even when such selectors occasionally hit the mark, they're extremely brittle — a single style refactor on the frontend can break the entire test suite, and the failure has nothing to do with business logic.

TokenCap's approach turns this from "guessing" into "looking up": it parses JSX AST nodes, extracts explicit test attributes, and indexes them in one place.

JSX AST: JSX is a syntax extension in React for describing UI structure. Browsers can't execute it directly — it must first be compiled into ordinary JavaScript function calls. An Abstract Syntax Tree (AST) is the tree structure formed after lexical and syntactic analysis of source code, where each node corresponds to a syntactic unit in the code, such as a JSX element, an attribute, or a piece of text. Unlike reading rendered HTML directly, the AST preserves source line numbers, attribute names, and nesting relationships, so it can precisely pinpoint "what the role attribute on that <select> on line 68 is." This is exactly the foundation for TokenCap's stable extraction of locator attributes — it processes structured source code, not page snapshots full of runtime noise.

ARIA: Short for Accessible Rich Internet Applications, a W3C-specified attribute standard used to describe element semantics and states to assistive technologies like screen readers. Attributes such as role="dialog" and aria-label="Close" give an otherwise plain <div> a programmatically understandable role and name. Test frameworks (like Playwright's getByRole) locate elements precisely through this semantic layer, so ARIA-based locators remain effective through style refactors — and they also indirectly improve the product's accessibility quality.

What TokenCap Indexes

TokenCap defines and extracts the following attribute categories in src/frontend/testIds.js:

1. Test identifiers: data-testid, data-test, data-cy
2. Accessibility roles: role="button", role="navigation", role="dialog"
3. Screen reader labels: aria-label, aria-labelledby, aria-describedby
4. Form associations: pairings of htmlFor with input id

These four categories cover the main approaches to element location in modern frontend testing. Form association in particular is worth highlighting — htmlFor and id pairings are scattered across JSX, and it's hard for an agent to infer from context "which input this label corresponds to," but AST parsing can capture this binding relationship precisely.

Putting ARIA attributes and test identifiers in the same index has an additional benefit: it naturally steers the agent toward accessibility best practices. Locators like getByRole("combobox", { name: "Billing Tier" }) are both more stable than CSS paths and incidentally validate the component's accessibility.

htmlFor and id pairing: In HTML and JSX, <label htmlFor="email">Email</label> and <input id="email" /> are associated through htmlFor (corresponding to HTML's for attribute) and id. Clicking the label focuses the input, and screen readers read the label text to the user. But this binding spans two elements that may be far apart, and a language model working from scattered context has a hard time reliably inferring "which input this label actually corresponds to." AST parsing can traverse nodes within the same scope and extract this pairing precisely, letting the agent use stable locators like getByLabel("Email") instead of guessing the input's class name.

The Snapshot Mechanism: Letting the Agent "See" the Locator Contract

The index itself won't automatically help the agent — what matters is how you put it in the agent's hands. TokenCap generates component snapshots via the tokencap make command, writing the test locator contract directly into the frontend component description:

### Component: UserBillingForm (src/components/UserBillingForm.jsx)
Testing Selectors:
  - Submit Button: `getByTestId("billing-submit-button")` (line 42)
  - Plan Selector: `getByRole("combobox", { name: "Billing Tier" })` (line 68)
  - Card Input: `getByTestId("card-element-input")` (line 95)

The information density of this snapshot is notable: every locator carries a line number, so the agent can follow it back to the source to confirm context; locators uniformly use Playwright's getByTestId / getByRole style rather than rigid CSS selectors.

With this context in hand, the agent no longer needs to speculate about DOM structure when generating tests. It knows the submit button is called billing-submit-button, and that the plan selector is a combobox named "Billing Tier." The resulting tests are naturally accurate in their locators and use modern accessibility queries.

Playwright locator style: Playwright recommends user-semantic locators. getByTestId matches test attributes like data-testid, while getByRole finds elements by ARIA role plus accessible name — for example, getByRole("combobox", { name: "Billing Tier" }) means find an element with the role of a combobox whose accessible name is "Billing Tier." Compared to CSS paths like div > form > div:nth-child(3) > select, semantic locators don't depend on DOM hierarchy or temporary class names and typically remain usable after frontend restructuring. Standardizing the snapshot into this style means the agent directly produces best-practice test code, rather than fragile selectors that need manual replacement afterward.

This Solves More Than Just "Hallucinations"

Framing this simply as "AI hallucination" undervalues it. The real shift is: making the frontend's implicit contract explicit.

data-testid and ARIA attributes are already "interfaces" that developers write for testing tools — it's just that in the past these interfaces only served human-written tests. Now they also serve AI agents. What the indexing layer does is expose this pre-existing interface to a new consumer.

This also changes the maintenance cost. As long as teams maintain data-testid naming conventions and fill in ARIA attributes completely, the agent-generated tests will remain stable over time. Conversely, if the frontend itself has chaotic locator attributes, the index can only faithfully reflect that chaos — the tool doesn't magically fix poor testability.

From an engineering standpoint, this approach is far more efficient than having the agent read the entire DOM tree or rendered HTML: AST parsing yields a precise, line-numbered attribute map, keeping token consumption manageable and accuracy high. For any team already using AI agents to write tests, bringing test locator attributes into the frontend index is a change with a very high return on investment.

One-line Summary

Explicitly index the frontend test locator contract and feed it to AI agents, sparing them from blindly guessing DOM selectors.

Paragraph Summary

When AI agents generate integration and end-to-end tests, the most fragile part isn't the assertion logic but element location — they can't see the frontend's actual structure and can only piece together plausible-looking CSS selectors, which then fail frequently due to nonexistent class names or style refactors. TokenCap's solution turns this from "guessing" into "looking up": parse the JSX AST, extract data-testid, ARIA roles, screen reader labels, and htmlFor/id form pairings, and index them centrally; then use tokencap make to generate component snapshots that write line-numbered, Playwright-style locators directly into the component description for the agent. Tests generated from this context locate accurately and naturally adopt accessibility queries. In essence, this approach makes the frontend's long-standing implicit testing contract explicit — but the tool can only reflect it faithfully, not fix inherently chaotic locator attributes.