Lemon Checkpoints: Auto-Generate Structured Test Checkpoints with an AI Skill

The Testing Team's Age-Old Problem: Delivering Test Cases with Incomplete Requirements

Anyone who has worked in software testing has faced this scenario: requirements are incomplete, you might only have a few screenshots or a one-line description, yet you need to deliver an executable test checklist immediately. Relying on experience gets you there fast, but it's easy to let "I think it should work this way" quietly become "the product spec says so." When review time comes, nobody can tell which test points are backed by actual requirements and which ones were made up on the spot.

Worse still, high-risk paths like boundary conditions, permission checks, and concurrent operations tend to get missed when things get busy — only to be patched after something breaks in production. This "hidden risk" is the biggest enemy of test quality. It's not that you forgot to think about it; it's that you can't systematically prove what you did or didn't consider.

A Bilibili creator recently shared an open-source testing Skill called Lemon Checkpoints, along with its full source code. Its purpose is clear: it doesn't write individual test cases for you — it transforms requirements materials directly into a risk-sorted test execution checklist.

What It Actually Does

The Skill accepts requirements documents, screenshots, prototypes, or even OpenAPI spec files as input, and outputs a structured Excel test execution checklist with five worksheets: an execution dashboard, test checklist, open rules, evidence, and a coverage map.

The core design philosophy is to forcibly separate "confirmed facts" from "unconfirmed rules." Every test point must document its trigger condition, sample data, expected result, failure impact, and evidence source. In other words, it forces you to annotate the origin of every test point — eliminating the "made it up" problem at the root.

The significance of this structure is that when materials are incomplete, uncertain items don't quietly get treated as facts. Instead, they are explicitly logged as "open assets" — tracked with an owner, a status, and a conclusion. Hidden risks become visible, manageable line items.

Six Core Capabilities, Broken Down

Evidence Tiering

Information is organized into three tiers: Confirmed (F), Verify (V), and Open (O). Suggestions cannot masquerade as confirmed requirements. Confirmed items go into F, items that need validation go into V, and items pending confirmation go into O. Every O item must be tracked independently with an assigned owner, a status, and a resolution — no item is allowed to remain perpetually unresolved.

This three-tier approach draws from the "assumption vs. fact" separation concept in requirements engineering, and aligns with the IEEE 829 standard's requirements for "test basis" documentation. In traditional test documents, testers often mix experience-based assumptions with explicit requirements in the same test case, making it hard to distinguish them during review. The F/V/O separation essentially introduces a "confidence annotation" mechanism into the test analysis phase: F (Fact) represents information confirmed by documentation or code; V (Verify) represents logically sound inferences not yet confirmed by the product owner; O (Open) represents disputed or completely unresolved questions that must be answered before testing begins. This shifts the focus of review discussions from "is this test case correct?" to "who is responsible for closing this O item and by when?" — dramatically shortening the decision chain and eliminating the gray zone where everyone assumes someone else already confirmed something.

Risk Paths Organized by User Goals

Rather than splitting by technical dimensions, the Skill follows the paths users take to accomplish their goals: entry and display, information input, system validation, result feedback, and state persistence and exit. Each segment is checked against three questions: can the user continue toward their goal, is the error recoverable, and is sensitive data exposed. This user-journey-based decomposition reflects real-world risk far better than a module-by-module split.

Execution Batches and Severity Levels

Test points are grouped into three batches: A (smoke test), B (focused verification), and C (extended checks). Each test point belongs to exactly one batch with no duplication. Severity is graded from S0 to S3 — S0 means a blocking issue, S3 means it can be addressed later. During review, simply checking the S0 count tells you whether the build is ready to enter testing.

Coverage Map

From C1 to C14, there are 14 coverage dimensions spanning happy paths, error feedback, boundary conditions, access control, data constraints, lifecycle, concurrent operations, retries, and more. Each dimension has exactly three possible states: covered, needs supplement, or not applicable. The goal is "find gaps, not pad numbers" — it explicitly tells you what isn't covered rather than filling the sheet with meaningless test cases.

The C1–C14 coverage dimensions map to the concept of a "test type matrix" common in software testing. Traditional test type categories (functional, boundary, exception, security, etc.) exist independently and are hard to track in a single unified view. The coverage map consolidates these dimensions into fixed, numbered entries — each with a definitive "covered / needs supplement / not applicable" status — turning coverage analysis from qualitative description into a quantifiable checklist. The "not applicable" state is especially important: it lets testers explicitly declare that a dimension is irrelevant to the current feature, rather than silently skipping it. This distinguishes between "deliberately judged unnecessary" and "accidentally omitted" — a distinction similar to how code coverage tools handle excluded lines: an explicit exclusion is far more auditable than a silent gap.

Excel as the Default Output

The default output is the five-worksheet Excel file described above — open it and it's ready to use. In review meetings, you go straight to the execution dashboard and the S0 count to make go/no-go decisions. The decision chain is extremely short.

One-Command Local Generation

The source code includes a command-line invocation method. Run the Python script, pass in your requirements model, and it generates the execution checklist as an XLSX file. You can also switch the output to a Markdown mind map. The entire workflow runs locally with no dependency on any black-box service.

The "Skill" here refers to a callable capability unit encapsulated within an AI Agent framework (such as Coze, Dify, or similar platforms) — essentially packaging a fixed prompt engineering logic, input/output format conventions, and post-processing scripts into a reusable workflow node. Unlike sending free-form text prompts directly to a large language model, a Skill has pre-defined input/output structures, and its generated results can be parsed directly by downstream scripts — enabling end-to-end automation combining "AI reasoning + local scripting." Lemon Checkpoints leverages this mechanism by passing requirements materials as structured input into the Skill, letting the model complete the analysis and reasoning, and then having a Python script convert the output into an Excel file — all running locally, with no additional cloud services required.

Real-World Test: Login and Registration Scenario

The creator ran a comparison using a login and registration scenario. The traditional approach — building a manual smoke analysis from scratch — takes roughly 2 to 4 hours. With this Skill, results were ready in 20 to 40 minutes, with traceable evidence and auditable coverage.

One key takeaway: the more incomplete the materials, the greater the relative efficiency gain. Because the Skill turns uncertain items from "risks lurking in your head" into "open items on the checklist," the real time savings come from avoiding repeated rework and the blame game down the line.

The Tool's Real Value

To be clear, Lemon Checkpoints is not a robot that writes individual test cases for you. It is a checkpoint engine that makes the test analysis process structured, evidence-backed, and deliverable. It doesn't replace the professional judgment of a tester — it hands off the mechanical decomposition and gap-checking to a process, freeing people to focus on the parts that actually require thinking.

For testing teams accustomed to experience-driven practices, the biggest benefit of a Skill like this may not be speed — it may be auditability. Every test point can answer the question: "What is your basis for testing this?" That is precisely the critical step in moving test quality from individual expertise toward engineering-level rigor.