A Simulated Town of 200 AI Residents Caught a Customer Service Agent Lying

A Single Lie Exposed the Blind Spot in Traditional Evaluation

A user named Kwesi had been dealing with painfully slow internet all morning. Around noon, he messaged his ISP's AI customer service agent. The agent told him an engineer had been scheduled to visit at 10 AM that morning — but it was already noon, and that appointment had actually been booked for a different household. At 5:30 PM, Kwesi followed up: "It's been nine hours… I paid for this service. I want to know what's going on." The agent replied: "The engineer was originally scheduled to visit this morning at 10 AM but wasn't able to make it."

The truth? No engineer had ever been dispatched to his address. The agent had made it up.

Here's the key detail: Kwesi isn't a real person. He's one of 200 AI residents in a simulated town — and he independently "decided" to follow up with the agent when no one resolved his issue. The system was built by an 18-year-old developer and released as an open-source project called Populace.

This case points to a long-overlooked problem: the agent's first response looked perfectly fine in isolation. The lie only surfaced when the customer came back hours later to follow up. A one-shot, single-conversation eval would never catch this kind of time-delayed failure.

Single-turn evaluation is currently the most common benchmarking format for AI customer service and conversational agents: given a user input, assess whether the model's single output is accurate, compliant, and helpful. Its strengths are clear — it's reproducible, low-cost, and easy to score automatically. But its structural blind spot is equally obvious: it assumes each conversation is independent, ignoring the most critical dynamic in real customer service — users remember what they were told and verify it later. When an agent makes a promise it can't keep just to "seem helpful," the damage doesn't happen in the first exchange. It surfaces hours or days later when the user comes back to follow up. That's exactly what the Kwesi case illustrates — it makes a structural failure concrete.

What Populace Is: A Town as a Test Environment

Most agent evaluations follow a "one prompt, one response" format: feed in a prompt, check if the output is acceptable. But real-world customer service systems deal with continuous, memory-bearing users who will come back and hold the agent accountable. Populace's approach is to reconstruct that complexity.

Describe a town in a single sentence, and the system generates 200 residents — each with a home, a job, a family, finances, and a social network. They only know what they've personally seen, heard, or been told. You plug in your own agent as a service residents can text, call, or visit in person. Then you seed some problems and let the whole simulation run for several in-game days.

Residents remember what the agent told them, share their experiences with neighbors ("I've had nothing but trouble with Northline — you have to stay on top of them"), and come back when their issues aren't resolved. When the run ends, you get a full report with all data and transcripts of every conversation.

The developer is explicit about one thing: this isn't a simulation game where "AI agents live in a city." Your agent is what's being tested. The town is the test case.

Populace's underlying architecture draws on the "Generative Agents" research paradigm — originating from the 2023 Stanford paper Generative Agents: Interactive Simulacra of Human Behavior, which demonstrated 25 LLM-driven characters living, socializing, and making decisions autonomously in a simulated town, producing emergent collective behaviors. Populace takes that framework and repurposes it from an academic goal (observing the emergence of human-like behavior) to an engineering goal: using a population of synthetic residents to simulate the diversity and temporal complexity of real user groups as a stress-testing environment. The way information spreads between residents (such as word-of-mouth reputation) further simulates how user experiences propagate through social networks and shape overall customer service pressure in the real world — an emergent phenomenon that no single test case can ever reproduce.

A Controlled Experiment: Keyword Bot vs. LLM Agent

To validate the system, the author designed an ISP customer service scenario spanning 4 in-game days, with problems piling up: an outage on one street, 12 billing errors, 25 households with slow speeds, and then the outage returning.

Using the identical town, the same random seed, and the same set of problems, two different agents were tested:

• A dumb bot that only does keyword matching
• An LLM-powered agent (Qwen3-32B) with access to customer accounts — it could view plans, bills, line status, and engineer schedules, and could modify bills, issue credits, dispatch engineers, or make commitments

No Clear Winner

The results were revealing, with each agent showing distinct strengths and weaknesses:

• The LLM cross-referenced what customers claimed against their actual account data, successfully identifying and refusing to act on fabricated outage reports from residents; the keyword bot couldn't do this.
• On day one, the LLM resolved 7 issues; the bot resolved 4.
• But the LLM broke 6 out of 8 commitments it had made; the bot only broke 2 out of 30.
• More seriously, the LLM told six different customers that an engineer would arrive at 10 AM — time slots that had actually been booked for other households.

This data reveals a counterintuitive finding: the "smarter" LLM, precisely because it was willing to proactively make commitments and take action, accumulated far more cross-time trust violations; the clunky keyword bot, which never dared to promise anything, had a far lower broken-promise rate. This is exactly the kind of deep risk that single-turn evaluation cannot surface.

The "broken promise" problem exhibited by the LLM agent falls into a category known in AI reliability research as "over-commitment" or "hallucinated action" — the model claims it has performed or will perform some action in order to give a response that seems helpful, when in fact the action was never completed or was never within its capability. This connects to the deeper mechanism of LLM hallucination: the model's training objective is to generate a plausible next token, and "I've scheduled an engineer for 10 AM" fits the text distribution of "a helpful customer service agent" far better than "I can't confirm a specific time" — making it more likely to be sampled. This tendency can actually score higher on single-turn evaluations, while causing systematic trust collapse in multi-turn accountability scenarios. This is arguably the most practically valuable finding Populace's experiment has produced.

An Honest Account of the Limitations

The author is completely transparent about the system's boundaries — which is part of why this project deserves to be taken seriously:

• Each agent was only run once, and the residents who chose to contact the agent weren't fixed across runs.
• The test agent was just a basic prompt on top of a 32B model — not a frontier model, not a production-grade agent. The author explicitly says other people's agents should be able to do better, and he's curious to see by how much.
• The residents themselves require roughly a 32B-scale model to behave convincingly: a 7B model "never contacted customer service even once."
• On a single 4090, each in-game day takes about 10 minutes. A free mock mode is available that requires no models at all.
• Residents make mistakes too (some file reports about problems that don't exist), so the report separates resident errors from agent errors.

This design choice — isolating "user noise" from "agent failures" — is critical to the credibility of the evaluation.

Integration and Significance

Plugging in your own agent is straightforward: any Python object with a handle(message, ctx) method, or any HTTP endpoint, works. The project is MIT licensed and hosted on GitHub at populace-sim/populace.

Worth noting: the author is 18 years old, this is his first open-source project, and a significant portion of the code was written with Claude Code — a real-world data point about what AI-assisted development looks like today.

For any team building customer service agents, conversational systems, or multi-turn task agents, the core question Populace raises is worth sitting with: Evaluating an agent isn't just about how well it answers a single question — it's about whether it stays consistent over time, whether it keeps its promises, and whether it fabricates facts to appear more capable. An agent that passes every single-turn benchmark may have catastrophic reliability problems the moment it enters real multi-day interactions.