Soei

In a prototype dataset containing 45 real coding-agent sessions, there were 102,233 records, 20,577 tool executions, and 2,336 interaction segments between humans and agents.

In the 45-session prototype study, 23.4% (95% CI 17.1–35.7) of code-editing segments received no independent check after the final edit.

Of the to-dos that agents marked as complete after editing, 57.8% lacked supporting evidence in the intermediate steps.

Judging session handoffs with a 10-minute time heuristic recovered only 6 of the 24 session pairs linked by content lineage.

42% of sessions ran longer than 24 hours, and 48.1% of transcript bytes exceeded the 2 MiB replay limit, a hard threshold that many agent runtimes set to control memory and latency.

In 16 sessions, 1,326 records contained strings suspected to be credentials.

Soei's hash-chained local ledger is only 20% the size of the raw data on the same corpus, stores no content or credentials, verifies at about 126k events per second, and can detect tampering and perform deterministic replay.

Soei comprises five components: a perception-confirmed event schema, a hash-chained local ledger, a graph structure aligned with W3C PROV, calibrated attribution and lineage inference, and a verification layer that strictly separates completion claims from completion evidence.

The study discusses its work alongside a concurrent 2026 proposal and, based on its own data, defines evidence-gated autonomy levels and harness metrics.

The open question the study raises is: how can evidence collection become the default behavior, rather than an after-the-fact remedy, without excessively increasing agent runtime overhead.