Stop Asking "Would You Use This?" — One Question Helps You Filter Out Fake Demand

Most bad product decisions don't happen during code review — they stem from assumptions that were never questioned. While developing an on-device AI agent for Android, the StareBrain team learned this the hard way: one flawed research question cost them a great deal of time.

The Wrong Question: Why "Would You Use This?" Is Worthless

Early on, the StareBrain team asked potential users: "Would you use an AI agent that helps you control your phone?" Almost everyone said yes. But that answer provided almost no useful information.

"Would you use this?" is a hypothetical question. Saying yes costs the user nothing — no need to recall a specific situation, no commitment, no effort. It's the lowest-friction signal, and therefore the most meaningless one. The team collected plenty of yeses but had no idea whether a real problem existed.

As the original article notes: "Most bad product decisions don't happen during code review — they stem from assumptions that were never questioned." Before a single line of code was written, the wrong question had already planted the seeds of trouble.

Hypothetical bias is a classic concept in market research and behavioral economics. It refers to people's tendency to give more positive, idealized answers in hypothetical scenarios than their real behavior would suggest. The root cause is that hypothetical questions involve no real cost — respondents don't need to draw on specific memories or bear the psychological weight of commitment — so their answers reflect an "ideal self" rather than a "real self." In software product validation, this bias is especially dangerous, because a "yes" to "Would you use this?" predicts neither download rates nor retention. A common way to avoid hypothetical bias is behavioral tracing — asking whether the respondent has actually taken action in the past (paid, searched, tried a workaround), because past behavior is the most reliable predictor of future behavior.

The Key Question That Changed the Research: Focus on Specific Experiences

The turning point came when the team started asking: "Walk me through the last time you were doing something on your phone and felt it should have just handled itself. What was that like?"

This question forces respondents to recall real experiences rather than hypotheticals; it reveals actual friction points rather than imagined needs; it shows the workarounds users are currently relying on; and it helps you judge whether the problem occurs frequently enough to matter.

The specificity of the answers was unlike anything the first question ever produced. People described exact sequences: open an app, copy content, go back, paste, navigate to a particular screen, re-enter information the phone already knew. These frictions weren't abstract concepts — they were four-app chains of operations that users had repeated so often they no longer even found them annoying.

That kind of answer is the foundation for building products.

The Critical Incident Technique was introduced by psychologist John Flanagan in 1954, originally to analyze pilot errors, and later widely adopted in UX research and product discovery. Its core is to have respondents recall and describe in detail a specific event that actually happened, with a clear beginning and end, rather than offer general evaluations. In StareBrain's case, "the last time you felt something should have just handled itself" is a textbook critical-incident question: it anchors time in the past and the scenario in specific actions, thereby sidestepping hypothetical bias. When using the Critical Incident Technique, follow-up questions about details (which apps were opened, the order of steps, what you were doing at the time) are more valuable than questions about feelings, because details expose the real chain of operations and friction points, and also help you judge how often the problem occurs.

The Follow-Up That Filters Further: Behavioral Validation

After the story-based question, the team added another: "Have you ever paid for, or spent real time trying to make this easier?"

If someone has spent real time or money trying to solve the problem — searching for apps, trying shortcuts, buying tools — then it's a confirmed problem. They've already voted with their behavior.

If they just say "yeah, it's annoying" but have never tried to fix it, the pain point may not be sharp enough to drive adoption. Knowing that before you build is extremely useful.

The original article also includes a code example contrasting the tedium of manual user operations with the simplicity of agent automation:

// The status quo we're trying to eliminate:
// User intent: "Get a cab to the airport"
// Current reality:
fun userDoesManually() {
    openApp("Maps")
    searchFor("airport terminal")
    copyAddress()
    openApp("CabApp")
    pasteAddress()
    checkCalendarForFlightTime()
    switchBackToCabApp()
    confirmBooking()
}

// How StareBrain handles it:
fun agentHandles(intent: String) {
    val plan = planner.decompose(intent)
    plan.steps.forEach { step ->
        dispatcher.execute(step, requiresConfirmation = step.hasIrreversibleSideEffect)
    }
}

This contrast clearly shows the real friction the agent aims to solve: cross-app information copying and repeated input. And only through specific, story-based questions can you uncover friction that users have already grown so used to that they no longer notice it.

The logic behind the "have you paid for or worked on a solution" follow-up corresponds to the lean startup principle of behavioral validation: replace verbal intent with observable investment (time, money, attention). In demand filtering, this is often called the "pain point sharpness test" — for a genuine pain point, users have often already sought workarounds on their own (searching for apps, setting up shortcuts, buying third-party tools, writing scripts), and these behaviors constitute evidence of a "confirmed problem." Conversely, complaints that stop at "it's annoying" indicate the pain isn't sharp enough to drive users to change their existing habits. Notably, this question doesn't require respondents to have already purchased a perfect solution — only that they've attempted to solve it, because the attempt itself proves the problem is worth investing in.

The Change in Practice: From Hypothesis Testing to Behavioral Validation

The StareBrain team completely stopped validating features with "Would you use this?" Every user research conversation now begins with a story-based prompt and ends with a spend-or-effort question.

The story prompt tells you whether the problem truly exists; the spend question tells you whether the problem is worth solving. Between the two, you can filter out a huge amount of "building for imagined users" waste before you ever open Android Studio.

Code is the easy part. Asking the right questions is what makes writing code worthwhile.

This approach isn't only applicable to AI agent products — it's also a useful reference for early validation of any software or even hardware product. When resources are limited, replacing verbal commitments with behavioral data is an effective strategy for reducing product risk.

One-line Summary

The StareBrain team proved through firsthand experience that hypothetical questions can't validate demand — only story-based questions plus behavioral validation can filter out fake demand.

Paragraph Summary

While developing an on-device AI agent for Android, the StareBrain team discovered that early research questions like "Would you use this AI agent?" produced a flood of positive answers that were completely worthless, because saying yes costs users nothing. The turning point came from switching to story-based questions — "Walk me through the last time you felt something on your phone should have just handled itself" — which forced respondents to recall real experiences and revealed concrete friction like cross-app copy-paste that users had already grown accustomed to. The team then added a behavioral validation follow-up: "Have you paid for this, or spent real time trying to solve it?" If users had already voted with their behavior, the problem was confirmed; if they only complained verbally, the pain wasn't sharp enough. Based on this, the team completely abandoned hypothetical validation in favor of a research process built on story-based questions plus behavioral validation, filtering out a huge amount of "building for imagined users" waste before writing any code.