draft_stability.py
An offline Python script that computes a stability score for text passages across multiple samples. By means of Jaccard similarity and divergence detection of claim-type tokens, it assigns an author role to each passage: model draft, model outline, or human-written.
Timeline (last 90 days)
The provided Python script draft_stability.py contains no network code, so it can run in CI, on laptops, and in scheduled jobs.
In the fixture example samples.json, the three samples of the install paragraph are highly consistent in wording and receive a MODEL-DRAFT / HUMAN-APPROVE verdict; the three samples of the retry-behavior paragraph claim 3, 5, and 2 retries respectively, with backoff times ranging from 200ms to 100ms to 250ms, triggering a disputed-claim check and a HUMAN verdict.
Running the command python draft_stability.py --fixture samples.json --k 3 --out record.md generates the record output.
Fixture mode decouples "whether the tool is correct" from "whether the model is available", so the CI pipeline can always run fixture tests to ensure the scoring path has no regressions.
The tool does not measure model quality; it measures whether the input has been settled.
All Facts (5)
The provided Python script draft_stability.py contains no network code, so it can run in CI, on laptops, and in scheduled jobs.
50%UnverifiedRunning the command python draft_stability.py --fixture samples.json --k 3 --out record.md generates the record output.
50%UnverifiedFixture mode decouples "whether the tool is correct" from "whether the model is available", so the CI pipeline can always run fixture tests to ensure the scoring path has no regressions.
50%UnverifiedThe tool does not measure model quality; it measures whether the input has been settled.
50%UnverifiedIn the fixture example samples.json, the three samples of the install paragraph are highly consistent in wording and receive a MODEL-DRAFT / HUMAN-APPROVE verdict; the three samples of the retry-behavior paragraph claim 3, 5, and 2 retries respectively, with backoff times ranging from 200ms to 100ms to 250ms, triggering a disputed-claim check and a HUMAN verdict.
50%