How I Rebuilt a Scam-Ad Checker After My Own Evaluation Metric Failed

Title

How I Rebuilt a Scam-Ad Checker After My Own Evaluation Metric Failed

Meta Description

Developer retrospective on Awas, the anti-scam ad checker: after a self-imposed coverage gate passed only 8/20, the product was rebuilt around a pure-code citation gate that verifies every quoted evidence line appears verbatim on the page.

Article Content

A family member received a WhatsApp group job offer: 15 million Indonesian rupiah a month, a tourist visa is fine, flights included. The rules needed to stop such scams do exist—Cambodia is not an official placement country, a tourist visa is not a work permit, and charging fees before a contract is a warning sign on police checklists—but they are scattered across different ministries, police, and embassy pages, and no one in the group chat could read them all in one evening.

Developer edycutjong built Awas for the Nebius Global AI Hackathon, aiming to answer that message with official pages rather than a score. After launch, he wrote a retrospective focused not on success but on how failure changed the product.

• Try it online: https://awas.edycu.dev (the /judge page opens five real run records, no login required)
• Code (MIT license): https://github.com/edycutjong/awas
• Demo video (2 min 37 s): https://youtu.be/to4VOBm_eNM

How It Works

The user pastes the ad text, and Awas returns an evidence file plus one of three verdicts: STOP, VERIFY FIRST, or NO RED FLAGS FOUND (explicitly labeled "not a guarantee of safety"). No score.

The pipeline:

1. Nemotron 3 Ultra on Nebius Token Factory extracts structured fields from the ad (company, destination, position, visa, fees). Code checks each field for literal presence in the ad.
2. Fixed rules produce red flags, each citing an official page.
3. Tavily searches for published pages about the same company, number, or destination.
4. Nemotron 3 Super proposes evidence lines: a URL, a quote, a claim.
5. A citation gate written in pure code keeps only those lines whose quote is a verbatim substring of the page text fetched during this run.
6. The verdict comes from rules and verified lines, not from the model.
7. A short Indonesian message, based solely on verified lines, is generated for the family member.

Hitting the Wall: My Own Gate Failed

Before the first benchmark run, the author wrote a pass condition in his own plan (afterward found only in the repo history): for at least 12 of 20 known scam cases, Tavily must find a relevant official warning or news item. This promise—"here are published reports about this company"—was the core of the original pitch.

The result found only 8 of 20. (An agent read the results; no human has reviewed them yet.) A second problem followed immediately: reports almost never name the recruiter. Among the reported cases in the benchmark, the dev set had 1, the first batch 0, and the second batch 1 mention of the company name. So "company match rate" cannot be measured.

The author had two choices: stay silent and keep the original pitch, or redefine the scope. He chose to redefine and publish the numbers. Verdicts now come from rules citing official pages. Web search becomes background information for the family member, never an input to the rules. Both the README and the demo page state that the coverage gate failed, and by how much.

Coverage gate refers to the pass condition the author set before the benchmark run: for 20 known scam cases, web search must find a relevant official warning or news report for at least 12. Such "pre-registered" pass criteria share the same engineering and statistical lineage as hypothesis testing—write down the threshold before running the data, to avoid cherry-picking favorable results afterward. It measures whether the retrieval layer can provide external corroboration for the rule layer, not the accuracy of the classifier itself. A failed coverage gate (8/20) means the original selling point—"there are public reports about this company"—does not hold for most cases, partly because scam job ads rarely name the recruiter and news narratives often use "a certain company" generically. The author's response was to redefine the product scope and publish the numbers rather than quietly delete the metric; in prototype research this is more informative than raising a score, because it downgrades "retrieval augmentation" from a rule input to background information, avoiding the use of unverifiable web results as a basis for judgment.

The Part Worth Copying: The Citation Gate

The model proposes, the code decides. The core logic lives in src/lib/gate.ts, simplified to the matching step:

export function gateLine(p: Proposal, corpus: Corpus, ctx: GateContext): EvidenceLine {
  const doc = corpus.get(p.url);
  if (!doc) return dropped(p, "url_not_returned");
  if (words(p.quote).length < MIN_QUOTE_WORDS) return dropped(p, "too_short", doc.purpose);
  const nq = normalise(p.quote);
  // ...
  for (const chunk of doc.chunks) {
    const nc = normaliseWithMap(chunk.text);
    const at = nc.text.indexOf(nq);
    if (at < 0) continue;
    // ... return a "verified" line with the raw quote and its offset
  }
  const joined = normalise(doc.chunks.map((c) => c.text).join(" "));
  return dropped(p, joined.includes(nq) ? "spans_chunks" : "not_in_page", doc.purpose);
}

If a line's quote cannot be found verbatim on the page returned by this run, the line is dropped, and the drop reason is shown on screen. The displayed quotes can be re-checked against the page without Awas's code: in a held-out run, a script re-found 33 of 48; the rest needed a browser (14 pages rejected scripts or were PDF/JavaScript pages) or had changed since being fetched (1).

Citation gate is an architectural pattern that takes "fact verification" out of the generative model's hands and gives it to deterministic code. Its core assumption is that LLMs are good at proposing candidate claims but not at guaranteeing their truth, while string matching and offset location are operations code can perform with zero hallucination. Concretely: maintain a corpus of pages actually fetched in this run, normalise the model's quote (unify case, whitespace, punctuation), then search for the substring verbatim within text chunks of the corresponding page; if not found, drop the line and record the reason. The cost of this pattern is lower recall—legitimate quotes that are paraphrased, synonym-substituted, or stitched across paragraphs will be wrongly killed—so a minimum quote length threshold and cross-chunk detection are usually needed to reduce false drops. Its value is auditability: every piece of evidence shown on screen can be reproduced and verified on the original page by a third-party script independent of the original system, which is exactly why Awas could have an independent script re-find 33 of 48 quotes in the held-out run.

What the Held-Out Run Measured

The second benchmark was frozen before running (file hashed), then run once against the production app as a Nebius Serverless Job on code tagged code-freeze. Every number is in the repo's DEMO.md and bench/.

• Licensed-agency vacancies wrongly stopped: 0 of 10.
• 25 publicly reported scam job cases: 9 STOP, 15 VERIFY FIRST, 1 NO RED FLAGS FOUND. Eight of the nine STOPs came from cited rules alone. One case is just one case, not a rate.
• All 48 displayed evidence lines passed the citation gate. 6 of 54 proposed lines were dropped (11%).
• p50 13.7 s, p95 31.7 s from request to completed file (server-side); median cost 0.0078 USD in Token Factory tokens plus 1 to 3 Tavily credits.
• Across 20 reruns on 10 reported cases, 20 gave the same verdict; on 10 licensed vacancies, 19 were the same.

Things the Docs Didn't Tell Me

• The structured output guide's own json_schema example was rejected on Nemotron 3 Super and Lightning, returning HTTP 422; the name/schema/strict form in the API reference worked.
• Super is a reasoning model, so the author gave it generous max_tokens: an earlier project of his saw it spend all tokens thinking and return an empty answer (the empty answers recorded in this repo are Lightning's).
• Indonesian is not on any readable model card, so he measured it: field accuracy was 84.8% for Ultra, 70.7% for Super, and 59.6% for Lightning, against agent-drafted labels no human has reviewed. This caveat appears on every page that cites these numbers.
• Conditional writes to object storage (If-None-Match, returning 412 on duplicates) are enough to implement rate limiting without a database.

The full list, with every recorded call, is in FEEDBACK.md.

Structured output means having the model return fields per a given JSON Schema rather than free text, so downstream code can parse directly. Support varies across APIs: some require a top-level {name, schema, strict} wrapper, others accept a bare json_schema; non-compliant requests are often rejected with HTTP 422. The discrepancies documented here show that examples in model cards and guides may not match the live endpoint, so integration should follow the actual API reference. A related concept is the token budget of reasoning models: such models first generate a long internal chain of thought before answering, so if max_tokens is set too small, tokens may all be consumed in the thinking phase and an empty response returned. Give models like Nemotron 3 Super a generous output ceiling, and distinguish "empty answer" from "format error" in monitoring, or you will easily misattribute this to model capability.

Limitations

The coverage gate failed, so web evidence is background only. The benchmark's reported cases are journalists' descriptions of the recruitment, not the original ads, so real ads may contain more red-flag phrasing. The phone-number path has not been measured. NO RED FLAGS FOUND is never a guarantee, and no external users have used Awas. It is a research prototype, not legal advice, and not an official service.

If you've seen an overseas job ad in a group chat, paste the text (remove phone numbers and names) at https://awas.edycu.dev and tell the author where it went wrong.

One-line Summary

The developer built the anti-scam tool Awas with a "model proposes, code decides" citation gate, and published the scope redefinition and benchmark data after the coverage gate failed.

Paragraph Summary

Awas is a prototype against overseas job scams, built for the Nebius Global AI Hackathon: users paste ad text, Nemotron models extract fields, fixed rules produce red flags citing official pages, and a pure-code citation gate keeps only evidence lines that match verbatim on pages fetched in the run, yielding STOP, VERIFY FIRST, or NO RED FLAGS FOUND—no score. The author's retrospective emphasizes failure: the pre-set coverage gate passed only 8/20 and reports rarely name recruiters, so he published the numbers and downgraded web search to background. The held-out run showed 0 of 10 licensed vacancies wrongly stopped, 9 STOPs among 25 scam cases, all 48 displayed evidence lines passing the gate, median latency 13.7 s, and median cost about 0.0078 USD.