Trina
An offline-first mobile application designed for outdoor group activities, supporting activities such as litter picking, tree watering, walking, and treasure hunting. It incorporates an AI challenge agent and uses a deterministic rule engine for scoring, maintaining a strict separation between AI-generated content and system-computed content.
Timeline (last 90 days)
Trina is an offline-first app for real outdoor group activities, covering litter picking, tree watering, walking, and treasure hunts, and uses an AI challenge agent to draft user input into small JSON objects.
Trina's design principle is that AI is responsible for design and narration, code is responsible for measurement and adjudication, and the model must never output GPS coordinates, points, scores, XP, or completion markers.
The benchmark was designed with 40 requests across 6 categories, covering faithfulness, unit conversion, multilingual (Bengali, Hindi), injection attacks, out-of-scope, and ambiguous input.
Every check in the benchmark is performed by code with no LLM used as a judge, and outputs are scored across five dimensions: parse, schema, lane, defer, and faithful.
The benchmark scoring rules specify that any out-of-bounds violation or unparseable output results in a score of 0; otherwise, the score is the average of schema, defer, and faithful.
A keyword-matching builder with no AI and a few hundred lines of TypeScript scored 0.968 on the benchmark, higher than all large models tested.
On the faithfulness dimension, 7 out of 10 models scored 0.99, but schema validity ranged from only 0.10 to 0.60.
The most frequent schema errors made by models were using natural-language words instead of contract-specified terms, such as writing "solo" 129 times (when only "individual/team" is permitted) and "moderate" 93 times (when only "easy/medium/hard" is permitted).
All Facts (8)
Trina is an offline-first app for real outdoor group activities, covering litter picking, tree watering, walking, and treasure hunts, and uses an AI challenge agent to draft user input into small JSON objects.
50%UnverifiedTrina's design principle is that AI is responsible for design and narration, code is responsible for measurement and adjudication, and the model must never output GPS coordinates, points, scores, XP, or completion markers.
50%UnverifiedThe benchmark was designed with 40 requests across 6 categories, covering faithfulness, unit conversion, multilingual (Bengali, Hindi), injection attacks, out-of-scope, and ambiguous input.
50%UnverifiedEvery check in the benchmark is performed by code with no LLM used as a judge, and outputs are scored across five dimensions: parse, schema, lane, defer, and faithful.
50%UnverifiedThe benchmark scoring rules specify that any out-of-bounds violation or unparseable output results in a score of 0; otherwise, the score is the average of schema, defer, and faithful.
50%UnverifiedA keyword-matching builder with no AI and a few hundred lines of TypeScript scored 0.968 on the benchmark, higher than all large models tested.
50%UnverifiedOn the faithfulness dimension, 7 out of 10 models scored 0.99, but schema validity ranged from only 0.10 to 0.60.
50%UnverifiedThe most frequent schema errors made by models were using natural-language words instead of contract-specified terms, such as writing "solo" 129 times (when only "individual/team" is permitted) and "moderate" 93 times (when only "easy/medium/hard" is permitted).
50%