Grok Bot⑭｜Is the Sort Bug Really Fixed? How Do You Prove It?

"The sort is broken." "It's fixed." — These two statements together constitute no valid acceptance. What actually inspires confidence is an independently reproducible chain of evidence. In Lesson 14 of the Grok Bot series, Bilibili creator "马上花生" uses a numeric sorting failure as a teaching case to break down what counts as a "useful minimal reproduction" and how to accept a fix with evidence rather than verbal claims.

The value of this episode lies not in the sorting algorithm itself, but in the demonstration of a transferable acceptance methodology: from reproduction, shrinking, assertions, to reporting — every step fights against the illusion of "it looks like it passed."

Why Use a "Deliberately Written Bug" for Teaching

The course establishes a boundary right from the start: the numeric sorting failure deliberately written for this lesson is not a product issue of Grok Bot. The code was run and verified locally on the instructor's machine, and the commands and results delivered for launching the real Bot or Cloud Agent are explicitly recorded.

This disclaimer itself is important. It strictly separates "the behavior of the teaching code" from "the state of the real repository" — the old failure and candidate passing records from the instructor's local testing are in the appendix. They prove the behavior of the teaching code, not that the Bot has been fixed, and no real production issue has been closed as a result.

In other words, this lesson doesn't teach "how to fix sorting," but "how to prove that a fix actually happened."

First, Look at the Actual Behavior of Default Sorting

The core of the case is a seemingly simple function call: calling default sort directly on the array [12, 2, 1].

The result is not [1, 2, 12], but [1, 12, 2]. The reason is that the default rules first convert elements to strings, then compare them in string order rather than by numeric value.

There's also an easily overlooked side effect: this call mutates the input array in place and returns the same array reference.

This means the requirements are not just "correct order." At least three acceptance criteria hold simultaneously: sort in ascending numeric order, keep the original array unchanged, and return a new array reference. Looking only at the displayed order will miss the side-effect layer.

Why does default sorting convert numbers to strings?

JavaScript's Array.prototype.sort, when no comparator function is passed, converts each element to a string via the ToString abstract operation, then compares them character by character in UTF-16 code unit order. So [12, 2, 1] first becomes ["12", "2", "1"], with first characters being 1, 2, 1 respectively. "12" is sorted before "2" because its second character 2 compares less than "1", yielding [1, 12, 2]. This rule also causes problems for arrays like [10, 9], since "10" sorts before "9". It's default behavior defined at the specification level, not an engine bug, but it's almost always wrong in numeric scenarios. To sort numerically, you must explicitly pass a comparator like (a, b) => a - b, otherwise you get lexicographic order.

What do "mutation in place" and "returning the same reference" mean?

sort is one of the few methods that both modifies the original array and returns that same array. Many developers mistakenly assume it returns a new array like map and filter do. This side effect causes cascading problems in real code: the array the caller originally held is silently reordered, and downstream logic reads the changed order. Therefore, acceptance cannot just look at the function's return value — it must also check whether the input array is consistent before and after the call and whether the two references are equal. This is why the case emphasizes that all three acceptance criteria — "order, original input, new reference" — must hold simultaneously. Tests can separately assert deepEqual(input, originalCopy) and input !== result, covering the complete behavior with three independent checks.

Shrink the Problem, Don't Delete All Conditions

The instructor's advice is to shrink the problem into a single function call with one set of inputs, reduce interference, then verify that it still produces the same error — rather than deleting all conditions and declaring "it no longer fails."

When shrinking in a real project, preserve the version, inputs, and preconditions, and record what was removed and what was kept, so others can independently run it and see the same discrepancy. That's what makes a useful minimal reproduction.

Preconditions must also be clearly stated. The precondition of this lesson is arrays of finite numbers; negative numbers, zero, duplicate values, and empty arrays all fall within this scope. As for how to handle strings, non-numeric values, infinity, and null values — ask the business rules first, and don't sneak in type conversions within a small sorting fix.

The Candidate Fix Does Two Things, But Doesn't Claim to Do a Third

The candidate fix logic is: first copy the array, then use subtraction of two values as a comparator to sort numerically.

Copying protects the original array; the comparator solves the default string-order problem. But it doesn't itself perform finite-number type validation. To truly reject illegal input, rules and tests should be defined separately — this lesson does not count that follow-up requirement as already implemented.

Writing preconditions and implementation separately in the report avoids a common confusion: mistaking "passes within a limited scope" for "correct for all inputs."

12 Assertions — Don't Just Print the "Correct List"

The regression coverage cases include the original error, negative numbers, 0, duplicate values, and an empty array. Each case must verify three things: result order, unchanged original input, and returned new reference. That's 12 assertions total.

The candidate passes 12 assertions locally. The original case's input is still [12, 2, 1]; returning the correct result and preserving the old input are two things that hold simultaneously, and the acceptance record must reflect them separately.

The acceptance record should include: function, candidate code, environment and run command, and saved actual failure and passing output. After changing code, old logs are not the new version's output but evidence for the new version — a historical screenshot cannot complete the current acceptance.

What is an "assertion"? How does it differ from printing results?

An assertion is a statement that can automatically determine true or false and clearly error when false, such as assert.deepEqual(result, [1, 2, 12]) or assert.notEqual(result, input). Printing just writes values to the console, requires human eye comparison, and cannot automatically fail during regression. Printing the "correct list" is especially dangerous: seeing a column of seemingly correct output makes it easy to assume all conditions are covered, when in reality the three checks — order, unchanged original input, new reference — often only cover one of them. Writing each requirement as an independent assertion allows pinpointing the specific attribute when it fails and enables repeated execution in CI. In the example, 4 input cases times 3 acceptance requirements yields 12 assertions, and that number itself is a quantitative statement of coverage.

Why does regression testing emphasize keeping the "original error"?

If only new boundary cases are run after the fix, the original failure might still exist due to environment differences. Keeping the original [12, 2, 1] as one of the regression cases pins the "already-fixed bug" in the test suite, preventing future refactoring from reintroducing the same string-comparison problem. This also explains "old logs are not new evidence": the failure output before the fix only shows the old code failed; the passing output after the fix must come from the current code version, and the two cannot substitute for each other.

When It's Not Reproducible, Don't Close the Issue

If the same input doesn't fail in your environment, first record the code version, command, and symptoms, and compare where the user's conditions differ. Inability to reproduce doesn't mean the user is wrong, and you can't just close the issue.

You can supplement with the specific version, minimal steps, or sanitized input; for paths requiring real accounts, state the lack of permissions, hand back what can currently be confirmed, and avoid expanding scope or filling in causes with guesses.

The practical advice for this section: reproducible tasks require organizing the report first, then providing the candidate fix and regression, returning code, commands, actual output, and unverified items, limiting file scope, and stopping at a reviewable result without merging or deploying. Only fill in actual run records when there's a real environment; if you only have course materials, don't write instructor references.

When Writing the Report, Include What Wasn't Verified

The report structure is roughly: environment, code version, original input and exact invocation steps, then expected and actual. Logs, screenshots, or text output must correspond to this run. Separately list impact scope and conditions not yet verified.

The test scope attached to the report must also be clear: case passing does not cover network, permissions, real pages, or all illegal inputs. Conditions not verified remain unverified; don't write "everything is fine" just because it runs.

Official support cases recommend attaching reproduction steps and evidence of the bad state, but that doesn't mean any internal website is accessible without permissions. When accounts, environments, or data are needed, confirm first; if you don't have materials, list the gaps and don't fabricate screenshots. This case doesn't require pages, databases, or networks.

What do "evidence of the bad state" and "sanitized input" refer to respectively?

Evidence of the bad state is observable material captured at the scene of the problem, such as screenshots at the time of error, console logs, network responses, or database snapshots. It records the system's actual behavior under specific conditions, not a description retold from memory afterward. Attaching it lets the recipient independently judge the phenomenon without relying on the reporter's subjective interpretation. Sanitized input, in turn, is when submitting reproducible material, replacing real data with structurally equivalent content stripped of sensitive information — for example, replacing phone numbers, usernames, and order numbers with placeholders while keeping data types, lengths, and boundary characteristics consistent, so the problem can still be reproduced without leaking privacy.

What does "limit file scope, stop at a reviewable result" mean?

This means when delivering a candidate fix, only modify files and functions directly related to the issue, don't opportunistically refactor, don't expand the change surface, and stop the work at a "reviewable" state — attaching code, commands, and output, waiting for review, rather than merging or deploying yourself. The purpose is to let reviewers confirm the fix within a controlled scope, preventing one change from incidentally introducing unverified modifications, and avoiding the mistake of equating "passed local tests" with "ready for production."

Where Can This Methodology Transfer?

Abstracting this lesson's process yields a general acceptance template: clear inputs and steps, old version proven to fail, same assertions proving the candidate's change, then supplement with near-boundary and side-effect checks.

What's handed back is not a single sentence "it's fixed," but an independently checkable body of evidence. In the next episode, the instructor will bring this evidence to the PR and CI stage, checking the current commit and required inspection items — passing tests cannot substitute for new-commit acceptance, and a green status must also explain what exactly was run.

Background Supplement

Why use a - b instead of a > b as the comparator?

The comparator contract requires returning a number: negative means a should sort before b, positive means a should sort after b, and zero means their relative order stays unchanged. a - b naturally satisfies this contract and returns the correct positive, negative, and zero values within the numeric range. If written as a > b, it returns a boolean, which most engines coerce to 1 or 0, never returning a negative value, so the sorting result drifts with implementation details and cannot be stable. The comparator must also satisfy reflexivity and transitivity, otherwise different engines may produce different orders. For inputs containing NaN, a - b returns NaN, and sorting behavior becomes undefined — this lies outside the preconditions drawn in the case and should be handled by separate rules.

What are common ways to copy an array?

The most direct ways to copy and protect the original array are [...arr] or arr.slice(), both producing a shallow copy, which is sufficient for numeric arrays; Array.from(arr) has a similar effect. If the array contains nested objects, a shallow copy only guarantees the outer references differ while inner objects are still shared — in this case, reordering won't change inner data but replacing elements may affect each other. If complete isolation is needed, use a deep copy. Choosing to copy before sorting in the case is equivalent to writing "immutability" into the implementation as a contract, ensuring the same function doesn't cumulatively change the caller's state across consecutive calls.

One-line Summary

Using a numeric sorting failure case, it explains how to accept a fix with a reproducible chain of evidence rather than verbal claims.

Paragraph Summary

The article uses the numeric sorting failure from Bilibili creator "马上花生"'s Grok Bot Lesson 14 as an example to demonstrate a transferable acceptance methodology. The core point: "it's fixed" does not constitute acceptance — an independently reproducible chain of evidence is required. In the case, default sorting converts elements to strings for comparison, producing [1, 12, 2] from [12, 2, 1], and mutates in place while returning the same array, so acceptance must cover all three requirements: order, unchanged original input, and new reference. The candidate fix solves the first two via array copying and a numeric comparator but does not perform type validation; unimplemented requirements cannot count as accepted. The article also emphasizes that minimal reproduction means shrinking, not deleting all conditions, preserving version, inputs, and preconditions; when not reproducible, record differences and hand back confirmable parts; reports must state unverified items, and passing tests does not equal commit or deployment.