CI Green Means Ready to Ship? The Version Correspondence Principle in Grok Bot Engineering Acceptance

CI all green doesn't mean it's deliverable. In the 15th episode of the Grok Bot series, Bilibili creator "马上花生" (Mashang Huasheng) used a set of simulated evidence exercises to break down this common misconception: a green CI status is only a necessary condition for delivery, not a sufficient one. What really determines whether something can enter the pending-review state is whether the evidence chain is complete, whether versions correspond, and whether the checks actually cover the original requirements.

Core Problem: Green Status ≠ Tests Passed

When many people see a CI pipeline all green, their first reaction is "it's ready to merge." But the creator explicitly points out that three questions must be asked: Were the tests actually executed? Which scopes were skipped? Is the current requirement covered?

GitHub's official documentation confirms this too: success, skipped, or neutral statuses can all satisfy platform-level check requirements, but this means a green status may hide a large amount of unverified content. A compelling example is when a smoke check passes but the core sorting regression fails—this situation cannot be described as "all passed."

Skipping tests itself isn't necessarily wrong, but the reason for skipping and the unverified items must be stated. The creator's core argument is: don't let color substitute for test content. Every piece of evidence must clearly state the commit, check name, command and environment, timestamp, exit status, and an openable log.

CI (Continuous Integration) refers to the engineering practice where developers frequently merge code changes into the main branch, and automated pipelines execute builds, tests, and checks. A pipeline typically consists of multiple Jobs, each corresponding to a check, ultimately converging into a platform-level status marker.

On GitHub, check statuses aren't just success and failure. Neutral means the check completed but the result is neither pass nor fail; skipped means the check wasn't executed because conditions weren't met. These two statuses may be treated as "passing" by default in branch protection rules, so the semantics of a green marker are broader than they appear literally.

Understanding this is the foundation for the discussion that follows: status color describes "whether the platform lets it through," not "whether the test verified the requirement." The gap between these two is precisely the source of risk that this episode repeatedly emphasizes.

Version Correspondence: The Most Easily Overlooked Evidence Trap

This episode features a classic "counterattack" exercise: the regression test for old version A passed, but new version B later modified the code yet still referenced A's passing screenshot. This version mismatch directly means the test cannot prove that the new commit satisfies the same requirements.

The correct approach is to return the commit hash of the actual result, the corresponding check name and logs, and then verify along the same regression path. A screenshot looking normal doesn't prove the test was executed on B. The version itself is part of the evidence.

GitHub has clear official guidance on this: you must check against the latest commit that needs to be checked; if there's a merge test commit status, check that, otherwise check the latest head commit. You can't mechanically use only the earliest run as the basis—you also need to verify the check name, trigger event, and repository rules.

Commit hash is the unique identifier Git generates for each commit, typically a 40-character hexadecimal string, with the first 7 characters sufficient for everyday reference. Since subsequent commits on the same branch change the branch pointer, any test evidence must be bound to a specific commit hash, not a branch name or relative description like "latest version."

Merge commit refers to the special commit automatically generated when a PR is merged, containing the history of both the target branch and the source branch. The platform prioritizes checking the check results on this commit when validating status; only if it doesn't exist does it fall back to the branch's latest head commit. This is why the same screenshot at different times and commits can lead to completely different conclusions—and the technical basis for the claim that "version is evidence."

Review Suggestions ≠ Blind Execution

A review comment points out that a certain check needs to remove duplicate values, but the original requirement clearly demands keeping them. Even if this check has successfully executed, this behavioral change still doesn't match the task—specific revisions and regression evidence are needed to support it.

The creator's view is clear: review suggestions also need to be judged for applicability—don't blindly change everything. Compare against requirements, impact, and scope to explain the reason for adoption. Disputes or changes beyond scope are decided by the person in charge—don't let the executor expand the task boundary on their own.

Failure Classification: Different Reds Require Different Handling

Not all red CI means "go fix the code again." The creator classifies failures into several types: results don't meet requirements, code behavior doesn't match the use case command, environment failure (dependencies inaccessible or tools reporting missing permissions).

For environment-type failures, after fixing the environment you must actually re-run the corresponding tests—restoring connectivity doesn't equal proving the code is correct. Missing permissions also can't be bypassed by repeated requests. Feedback to the executor must be specific failure versions, inputs, and expectations—not vague demands.

Deliverability Criteria

The operational framework given in this episode is: the current version B has core regression failures and duplicate values removed; only fix the relevant functions and tests; maintain the limited-number premise; return the new commit and corresponding logs; no longer reference A's passing screenshot. Limit modifications to at most two rounds—if the same problem persists or exceeds the allowed scope, escalate to manual handling.

The round limit is a management convention, not an official product restriction—the purpose is to avoid "writing a task into an infinite loop." Stopping operations won't undo existing changes; necessary corrections still need to be handled according to the real goals and authorization.

Once a reviewable result is achieved, it can be delivered to the pending-review state. When merging, separately verify the specific PR, latest commit, target branch, and authorization—you can't automatically merge and deploy just because of a link or green marker.

The creator concludes by emphasizing a reusable engineering acceptance checklist: version correspondence, verification of actual requirement fulfillment, traceable gaps, and pending-review stop points. Green status is meaningful, but it must be read together with this specific change and the original requirements. The next episode will switch to an office scenario, discussing the discrepancy in progress tables where "eight items completed, email says two items pending acceptance."

Escalation to manual handling is a common safety valve in automated workflows: when an automated agent cannot achieve its goal within a limited number of rounds, or when the scope of changes exceeds pre-authorization, the task is returned to a human owner for decision-making. The "two rounds" here is not a hard product-level restriction, but a management convention set by the team to control costs and risks.

There are three key points in designing an escalation mechanism: clear trigger conditions (repeated failures or scope violations), clear handoff content (current commit, logs, attempted fixes), and clear stopping boundaries (escalation doesn't equal rollback—existing changes still exist). Without any one of these, automated tasks easily degrade into unbounded loops or leave behind hard-to-trace half-finished states.

One-line Summary

CI all green is only a necessary condition for delivery; the completeness of the evidence chain and version correspondence are what really matter.

Paragraph Summary

Based on the simulated evidence exercises from episode 15 of Bilibili creator "马上花生"'s Grok Bot, this article breaks down the common misconception that "CI all green means ready to deliver." There are three core arguments: first, green status may include success, skipped, and neutral states—you must ask whether tests were actually executed, which scopes were skipped, and whether original requirements were covered; second, old version A's passing screenshot cannot prove that new version B satisfies the same requirements—commit hashes and corresponding logs are the valid evidence; third, review suggestions also need to be judged against original requirements for applicability—don't blindly change everything or let the executor expand scope on their own. The article also provides a complete operational framework including failure classification handling, escalation to manual after at most two rounds of fixes, and separate verification of PR and authorization during merge, with "version correspondence, verification of actual requirement fulfillment, traceable gaps, and pending-review stop points" as a reusable engineering acceptance checklist.