Unverified50% confidenceBenchmarkExact time
In this test, Lovable scored 100, went from prompt to launch in under 4 minutes, required 0 fix prompts, and consumed 2.10 credits on the free tier.
1
Sources
50%
Confidence
Short-term (~14 days)
Relevance
10/18/2026
First Seen
Valid until: 11/1/2026
Sources
Related Entities
Related Claims
UnverifiedEach question in the test is scored out of 100 points, consisting of 70 points for correctness plus 30 points for quality, with different questions carrying different weights.69% similarUnverifiedAfter moving the failure-logging instruction from a footnote into a numbered task step, the weak model's feedback-logging score rose from 0.00 to 1.80 (out of 2), and across twenty runs neither model triggered the instruction in the footnote version.68% similarUnverifiedAmong the five evaluators the authors tested, if the answer key had minor errors, the score was only 0.21 — lower than giving no key at all (0.46) and lower than giving a completely irrelevant key (0.50); while a human-verified model-generated answer scored 0.61, close to the 0.69 of a human-written one.68% similarUnverifiedThe pre-test execution baseline is fixed at: contribution value 1650, points 1000, and both orders and two types of transaction records at 0.67% similarUnverifiedIn the experiment, the task_score under the shaped reward configuration was only 0.125, compared to 0.417 for the naive reward, with an effect size of d=4.47.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1060624API
curl https://kongchang.com/api/v1/knowledge/claims/1060624MCP
get_claim(id=1060624)