Why Open-Source LLMs Lose to Tiny Networks in Among Us Voting

Putting a language model into the meeting voting phase of Among Us seems like the scenario it is best suited for: crewmates need to find the impostor hidden among them through conversation. After someone gets killed, everyone speaks, reports positions, accuses others, and votes; the player with the most votes gets ejected from the ship. Developer Seth Wheeler ran this experiment in his own simulator, replacing the voting system originally driven by a small recurrent network with the open-weight model gpt-oss:20b. Across six configurations, the large model never helped the crew perform better.

Experimental Setup

Each surviving player (whether crewmate or impostor) is sent as an independent prompt to the same model, and the model's reply is that player's speech and vote. A crewmate's prompt contains only the information available to that role: where they went, when they saw whom, witnessed kills or vents, and a table of everyone's claimed positions. An impostor's prompt additionally contains the true movement path (marked as secret) and permission to lie. Behavior outside meetings—movement, tasks, kills—is still controlled by trained bots; only the meeting phase is replaced.

There are three meeting formats: players take turns speaking and can ask each other structured questions (for example, "Where were you 20 seconds ago?"); adding the voting network's "gut feeling" judgment to the crewmate prompt; and free-form turn-taking dialogue with the gut feeling. The baseline is the same set of bots, the same lobby, with the trained network handling all voting on both sides, for a total of 300 games.

Among Us is a multiplayer social deduction game released in 2018. Each round consists of 4 to 15 players, a minority of whom are randomly assigned as "Impostors," with the rest as "Crewmates." Crewmates win by completing tasks or voting out all impostors, while impostors create chaos by killing crewmates and sabotaging facilities. The game has two phases: a free-movement phase on the map, and a meeting phase triggered by discovering a body or pressing the emergency button. The conversation and voting in the meeting phase are the core decision-making parts of the game, and are also what this experiment replaces the model for.

gpt-oss:20b is an open-weight reasoning model released by OpenAI in 2025. It uses a MoE (mixture-of-experts) architecture, with roughly 21 billion total parameters and about 3.6 billion active parameters. It supports adjustable reasoning effort and was post-trained for dialogue and instruction-following tasks. The reason this experiment chose this model is that its weights are public, it can run locally, and it has strong general language and reasoning abilities—making it suitable for testing the hypothesis of whether a general large model can replace a specialized small model.

The small recurrent network in this experiment is a policy network trained inside the simulator to output voting decisions during meetings. Its inputs are structured game-state features (such as positions, witnessed events, and voting history), and its output is a suspicion score for each player. Unlike a language model, it does not process natural language and does not need general knowledge; all of its advantage comes from repeated calibration across thousands of matches in the same simulator, so its suspicion scores are highly aligned with the label of "who ultimately turned out to be the impostor." The experiment uses it as a baseline because it represents the upper bound of a specialized model trained extensively in a specific environment.

Eight Players, Two Impostors: The Large Model Performs Far Worse Than the Network Alone

When using the second-league bots, the trained network achieved a crewmate win rate of 52.0%, ejecting an average of 0.99 impostors per game. By contrast, gpt-oss, only in the turn-based Q&A format, won 2 out of 16 games (12.5%), ejected only 0.06 impostors per game, and incorrectly ejected 0.94 innocent crewmates. This gap far exceeds random fluctuation (one-sided Fisher's exact test, p = 0.0017), though the difference in simulator versions should be noted.

When the voting network's numerical output was inserted into the crewmate prompt as a "gut feeling," the situation changed. The gut-feeling line begins: "Your gut feeling (a trained model reads everything you have seen, plus the current accusations; it picks the real impostor about 70% of the time, so weigh it against the facts):" It then lists the percentage probability that each other player is the impostor. That 70% is a rough estimate the author wrote in the prompt, not an actual measurement. After adding the gut feeling and using the turn-based Q&A format, the crewmates won 8 out of 16 games, nearly indistinguishable from the network baseline (p = 0.54). If the model basically votes according to the gut-feeling indications, this is the expected result; the author did not measure how often the model actually followed the gut feeling.

Fifteen Players, Three Impostors: The Gap Narrows but It Still Trails

In a 15-player, three-impostor lobby, crewmate win rates were higher under both voting methods, but the ordering remained the same. The comparison across the three leagues all used the dialogue format with gut feeling:

Bot Source	Trained Network (300 games, current version)	gpt-oss Dialogue + Gut Feeling
Second league	89.7%	10 wins in 12 games (83%), earlier version
Fourth league	84.7%	11 wins in 20 games (55%), earlier version
Sixth league	93.3%	16 wins in 20 games (80%), current version

The sixth-league row is the only strictly comparable experiment: same bots, same simulator version, same settings, run on the same day. 16 wins in 20 games versus 280 wins in 300 games gives a one-sided Fisher's exact test p = 0.052, and the 95% confidence interval for 16 wins in 20 games is 58% to 92%. Looking at this row alone, the result is suggestive rather than definitive. The fourth-league gap is larger (p = 0.0025), but between that run and the baseline, the author changed how meetings handle persistent sabotage and what impostors know when doors are closed, so it should not be relied on too heavily.

Counting all six configurations: in one, the language model was clearly worse than the network (eight players, no gut feeling); among the five with gut feeling, it matched the network once and trailed in the other four, but each individual gap was either too small or too confounded to draw a conclusion. In none of the configurations did it perform better.

The Mismatch Between Cost and Performance

In the sixth league, a 15-player game using gpt-oss took an average of 524 seconds of wall-clock time for meetings. Almost all of that time was spent on turn-taking speech across about 4 meetings, with one model call per player per turn. The trained network's voting requires no model calls at all. If better voting were worth the cost, that would be fair, but the voting results were not better.

Why This Happens

This part is the author's interpretation rather than a measurement. Trained networks have an "unfair" advantage, and that is exactly the point: they are trained on the outcomes of thousands of meetings in this simulator, and their suspicion scores are calibrated to "who ultimately turned out to be the impostor." A language model brings general knowledge about what deception sounds like. But the impostor here does not sound abnormal at all—it simply names a room, and the only way to catch the lie is to check that claim against "who could have seen that room." The prompt contains what each player saw, but nothing suggests the model can reliably perform that check. And when the prompt contains the network's numbers, its performance merely matches the network.

An earlier benchmark by the author—ranking language models on identifying liars in Mafia—found that first-day accusations were pure luck. On the first day, information about the impostor is not yet knowable. But in this experiment, the evidence is right there in each crewmate's prompt, which makes losing to a small network even harder to excuse.

Fisher's exact test is a statistical significance test for 2×2 contingency tables, suitable for small sample sizes and low expected frequencies. In this experiment, it is used to compare whether the difference in crewmate win rates between the "trained network" and "gpt-oss" voting methods could be caused by random fluctuation. For example, in the eight-player no-gut-feeling configuration, p = 0.0017 means that if the two methods actually had the same win rate, the probability of observing such a large gap would be only about 0.17%, so the difference can be considered significant. In the 15-player sixth league, p = 0.052 is slightly above the commonly used 0.05 threshold, making it a borderline result that is "suggestive but not significant," which is why the author cautiously describes it as "between suggestive and definitive."

Conclusion

This experiment reveals a counterintuitive finding: in the meeting scenario of Among Us, the general reasoning ability of the open-weight language model gpt-oss:20b did not translate into a practical advantage. Against a small recurrent network that has seen thousands of matches in its own simulator, the language model was defeated across all six configurations. Even when the network's judgment was fed to the model in the form of a "gut feeling," it at best only matched it. This suggests that for tasks requiring precise alignment with the statistical characteristics of a specific environment, a general large model may still fall short of a specialized small model trained extensively in that domain—and this is exactly the kind of cognitive bias worth guarding against when deploying AI applications today.