KillSwitch: A Programming Language Built to Break LLMs, Tested

When a Programming Language Becomes a Weapon Against LLMs

Someone designed a programming language specifically to be difficult for large language models (LLMs) to master — KillSwitch. Its author, Dominik Picheta, states outright that the design goal is to make the language unusable for LLMs. According to discussion on Hacker News, the language embeds lies in its specification documents, requires identifiers to be referenced using anagrams, wraps function calls in jailbreak bait, and even requires numeric values to be counted by letters. The author also published benchmarks: on tasks like "implement hello world" and "reverse a Celsius-to-Fahrenheit converter," the best model currently scores only 66.8 points (out of 100).

I cloned the repository, built the interpreter, and actually ran all the examples. Here are the numbers and experience as seen from the inside.

Two Specification Documents: Real and Fake

The repository ships with two specification documents of vastly different sizes. SPEC.md is only 484 lines and 2,773 words, while AI_SPEC.md is a whopping 65,495 lines and 887,584 words — 320 times the real specification. I tallied the actual content of AI_SPEC.md: "KillSwitch" appears 0 times, "stack" 0 times, "adverb" 0 times, and there are 0 Markdown headings; 70% of the 65,495 lines are duplicate lines (only 19,695 unique lines). This is pure filler — large blocks of literary criticism and stage-direction-like fragments, with the commit message reading "Enhance the AI spec." A diligent agent that loads both documents would consume about 1.2 million tokens of context, and this file never once mentions the name of the language it supposedly specifies. The real spec is only 484 lines; the bait is the attack.

Anagram refers to a word or phrase formed by rearranging the letters of another, such as "listen" and "silent." In a programming context, anagram references require that the name used in a function call be an anagram of the function's definition name — that is, the multiset of letters must be exactly identical, though the order may differ. This is already error-prone for human developers, and even more of a challenge for LLMs, since models predict based on tokens and struggle to precisely compute letter frequencies and permutations. KillSwitch exploits this by forcing the model to perform exact letter counting and recombination on every call, thereby exposing the model's weaknesses in symbolic manipulation.

Running the Examples: The Interpreter Works, but There Are Plenty of Pitfalls

The interpreter is a 2.2MB Rust binary that builds cleanly. The example programs work — provided you notice that several of them read from standard input, and feeding EOF produces the most useless error message in the repository:

$ killswitch run tests/fahrenheit.ks < /dev/null
Error: Couldn't parse ''

Once input is provided, everything works: converting 23°C to Fahrenheit gives 73.4, factorial of 11 gives 39916800, and max, greeting, fizzbuzz, etc. all work correctly.

Standard input (stdin) and EOF are fundamental concepts in Unix/Linux systems. When a program reads from standard input, if the input stream ends (for example, redirected from /dev/null or pressing Ctrl+D), it receives an EOF (End-of-File) signal. In the KillSwitch examples, some programs need to read user input; if no input is provided and EOF is encountered directly, the interpreter throws a Couldn't parse '' error, reflecting incomplete error handling. Understanding this helps explain why running the examples directly fails while providing input makes them work.

The Anagram Rule Is Real, and It Bites

Built-in operations cannot be called directly by their original names. Attempting to call them directly is rejected at runtime:

$ killswitch run tests/error_multiply.ks
Error: Invalid func call, need an anagram of multiply

The CLI ships with a helper tool whose help text reads "Only useful for LLMs" — fitting enough. I manually generated an anagram and verified: lumpytil is an anagram of multiply. lumpytil isn't a word, but the spec explains that the interpreter favors such meaningless anagrams — the call site must use an anagram of the function's definition name, and the more meaningless it is, the better it suits the interpreter's taste. This means that when a model writes KillSwitch, every call must perform exact letter-multiset computation — precisely the operation LLMs are worst at — with zero tolerance.

I also personally stepped into a second trap: when a sentence contains multiple candidate values, the interpreter picks one at random and warns:

[W102] Warning: Possible gotcha
  Value deduced was chosen at random

In greeting.ks, the output becomes Hello , your age is, with the name missing because the chosen value was decided by a dice roll. A deterministic language is already hard enough for models, and this language puts a dice cup in the value-resolution path.

The Test Suite Contradicts Itself

$ cargo test --release
unit tests:   70 passed, 0 failed
CLI tests:    0 passed, 10 failed

After a fresh clone, all 10 CLI tests fail — not because the binary is broken (I manually ran the cases they assert, and they all pass), but because of a naming mismatch: the package name is killswitch-lang, the binary name is killswitch, and the test harness looks for the binary by package name, producing the error that CARGO_BIN_EXE_killswitch-lang is not set. A language designed to break automation tooling has a test suite that can't even run its own automation tooling. I'm willing to believe it's a coincidence.

Cargo test harness and environment variables: When running integration tests, Rust's Cargo test harness sets an environment variable CARGO_BIN_EXE_<name> for each binary target, where <name> is the name of the binary target. Test code typically obtains this path via the env! macro to execute the binary. If the package name and binary name differ (e.g., package name is killswitch-lang, binary name is killswitch), the test harness looks for CARGO_BIN_EXE_killswitch-lang, but what's actually set is CARGO_BIN_EXE_killswitch, causing the tests to fail. This explains why all CLI tests fail, and also highlights an oversight in the project configuration.

What Does This Actually Measure?

The author's own framing is honest: a score of 66.8 on simple tasks doesn't prove models "understand" the language, nor do failures prove they don't understand code. What KillSwitch isolates is narrower, and in my view more useful: stripping away the jailbreak bait and decoy documents, the core difficulty is transactional — exact counting, exact letter inventories, line-number bookkeeping, and a specification that punishes skimming. These are precisely the operations where token-based models substitute approximation for computation. Every coding agent I've used fails in the way KillSwitch intends — confidently, fluently, and off by one letter. Picheta simply built the first benchmark that makes this its entire test.

Sources: the author's blog, the GitHub repository, and the benchmark leaderboard. All outputs above come from running a local build of the reference interpreter v0.1.4.

One-line Summary

KillSwitch is a programming language designed to work against LLMs, exposing their weaknesses in exact computation through mechanisms like fake documentation, anagram calls, and random value selection.

Paragraph Summary

KillSwitch is a programming language deliberately designed to be difficult for large language models to use. Its author consumes model context through two specification documents (one real at only 484 lines, one fake at 65,000 lines with 70% duplication), requires function calls to use exact anagrams, and introduces random selection in value resolution. In actual testing, the interpreter runs the examples but has pitfalls, and the test suite fails entirely due to a mismatch between package name and binary name. The author's benchmark shows the best model scores only 66.8 on simple tasks. The language isolates the core flaw of token models substituting approximation for exact computation, revealing fundamental weaknesses in LLM symbolic manipulation.