[KongchangAI]
· 4 min read· 2,141 words

The Boundaries of AI Productization: When to Use It and When Not To

The Boundaries of AI Productization: When to Use It and When Not To

AI is not a product staple; the test is whether it helps users accomplish their tasks.

Based on an interview with Ideo's Savannah Kunovsky, the article examines when AI integration is a breakthrough versus an obstacle. It criticizes companies that instinctively reach for LLMs, citing failures like the Humane AI Pin and a knobless cooktop, and argues physical controls should be kept for instant, by-feel operations. Good integration starts from users and tasks, and non-deterministic AI requires evaluations and clear human override.

When AI Becomes a Product Staple

Over the past few years, the tech industry has been eager to put an AI chip in every device, as if anything that can have AI must have it. Smart visual toothbrushes, household robots that read emotions... a flood of physical products has entered daily life, relying on artificial intelligence to learn, predict, and adjust user behavior. But as anyone who has suffered through a fridge firmware update knows, AI does not improve every product. Some product designers are starting to ask: where is the line between genuine innovation and "finding a solution for a problem"?

Savannah Kunovsky, head of Ideo's industrial product design team, shared her judgment on "when AI integration is an obstacle rather than a breakthrough" in an interview with IEEE Spectrum. The following analysis is based on the core points of that interview.

AI Integration Driven by Pressure

Kunovsky acknowledges that part of the current wave of AI integration is indeed driven by investor and shareholder pressure. Teams often come in feeling they must show they are "doing AI"—sometimes because the CEO promised an "AI story" to the board before an earnings call, sometimes because a competitor just launched an AI version of a product.

But Ideo's approach is to step back first and ask clearly who the product is for and what the user wants to accomplish. Once that is clear, what role AI should play—big, small, or invisible—usually becomes obvious. Kunovsky says there is a strong demand in the industry right now for this kind of "grounded" thinking; people want to cut through the hype and see what actually works in practice.

Distinguishing Real Needs from Gimmicks

The criterion for judging whether an AI feature is a gimmick is the same as for any product experience: does it help users do what they want to do? This requires understanding how people will actually use the product in real scenarios.

Many companies jump straight to large language models (LLMs) because that is the first thing that comes to mind. But the AI toolbox is far broader: computer vision, sensor fusion, on-device models, and more. Choosing the right capability matters as much as deciding whether to use AI at all.

Kunovsky cites the example of IDEO's predecessor company designing Apple's first mouse in the early 1980s: the team gave it only one button, because when people face a completely unfamiliar system, the interaction mechanism must be effortless. AI, as a new interaction paradigm, likewise requires users to change their behavior, so it must be simple.

On-device models refer to small AI models that run locally on devices such as phones, earbuds, and home appliances, rather than sending data to the cloud for processing. Their advantages are low latency, offline availability, and better privacy (data never leaves the device), at the cost of model size and capability being limited by local compute. Sensor fusion combines data from multiple sensors—cameras, microphones, accelerometers, radar—to judge context; for example, a smartwatch combines heart rate and motion data to tell whether you are running or sitting still. These two types of capability are often better suited to physical products than general-purpose LLMs: they respond quickly, consume little power, do not depend on the network, and fit specific tasks more closely. Choosing the right form of capability often determines the success of the experience more than whether AI is used at all.

Several Typical Ways AI Makes Products Worse

The clearest failure case is when AI replaces something that already worked well. Kunovsky respects Humane's AI Pin as a "provocation" but believes the technology is not yet mature—it asks users to give up a familiar interface for a slower, less reliable interaction.

Similar problems are everywhere: voice assistants mishearing simple commands and turning off the wrong light; customer service bots trapping you in a loop when all you want is a human. In cars, climate and volume controls have been moved to touchscreens and voice menus, so things once done by feel now require taking your eyes off the road. The European New Car Assessment Programme (Euro NCAP) has accordingly begun rewarding designs that retain physical controls.

People love certain products precisely because they are analog. Film cameras, good kitchen knives, and notebooks derive part of their value from craftsmanship and a sense of complete control. But it doesn't have to be either-or—in a smart home you can ask the voice assistant to turn off the light, use an app, or just press the switch on the wall. Multiple paths to the same goal are good, as long as complexity isn't added for its own sake.

Kunovsky shares a client case: the client wanted a cooktop without knobs—tell the AI what you're cooking and it manages the heat. During testing, a pot started smoking, the user reached for a knob that wasn't there, and had to shout "turn it down" twice before moving the pot off the burner—and the AI still hadn't responded. In the end they kept the knobs. Anything that needs to be done instantly, by feel, or under pressure is usually best kept physical: volume knobs, light switches, stove burners, emergency stop buttons. In those moments, mechanical controls are faster and more trustworthy than a system that has to interpret intent.

What Good AI Integration Looks Like

Companies often start from technical feasibility because the technology itself is so exciting. But understanding user needs is key, followed by technical feasibility and what makes sense for the business.

Sometimes AI is the core of the experience, such as a voice interface that lets users converse naturally; sometimes it works in the background, learning preferences so you don't have to repeat yourself. Either way, the measure is whether it helps users achieve their goals.

To prevent AI from making products unintuitive or cumbersome, the key is not to instinctively add a chat window to everything just because LLMs process text. Starting from people and tasks, the interface may look completely different. Physical products can communicate through light, sound, vibration, motion, or the feel of a knob or button. AI's intelligence can make these channels richer. The products Kunovsky is most excited about are those that extend our ability to perceive, understand, or act while keeping people at the center of the experience.

Accessibility offers the clearest examples: hearing aids use AI to pick out the voice you want to hear in a noisy restaurant; Be My Eyes lets blind or low-vision people point their phone at the world and hear a description of what's in front of them; real-time translation in earbuds lets two people who don't share a language converse freely.

Proactive AI systems need to anticipate needs without intruding, like a good waiter in a restaurant who notices an empty water glass and can also read whether you want to chat or be left alone. When people try something new, they often don't know what to ask for; a product that shows up with useful features can teach them what's possible. Personal agents such as Instinct, Dots, and Muse are emerging, handling matters in the background and coming back with new findings. Relevance and timing matter most: how loudly a product should speak should be proportional to the stakes, and it should learn from user reactions. If users repeatedly ignore it, that's a signal to adjust.

Personal agents are a class of AI systems that run continuously in the background and handle matters on the user's behalf. Unlike chatbots that require the user to ask questions, agents observe context, anticipate needs, and return with results at the right moment—for example, automatically organizing schedules, filtering email, or following up on to-dos. Products such as Instinct, Dots, and Muse are exploring this direction. The design challenge is to be "proactive without being intrusive": the agent must judge when to speak, how loudly, and learn to adjust from the user's ignoring or accepting it. This aligns with the "good waiter" metaphor in the article—relevance, timing, and restraint determine whether an agent is trusted more than the number of features.

The Testing Challenge Posed by Non-Determinism

Generative AI is non-deterministic; it may give different responses each time, which changes what product definition looks like. Traditional software writes specifications and tests whether the product precisely conforms; with generative AI you cannot specify every output, so you define the space in which the product should operate.

AI evaluations and constitutions (written principles that guide model training) have become the new form of product definition for LLMs. Evaluations describe what good performance looks like across thousands of situations; constitutions list the values and boundaries the system should hold. Together with model software, configuration, and runtime infrastructure, they shape the "container" the product lives in. Getting this right is now core design work, not a final engineering step.

When a product's responses are non-deterministic, consumer trust is under pressure. People trust traditional products because they do the same thing every time. Generative AI requires more design intent; teams must decide in advance where the product should be consistent and where it can explore. The product should clearly tell users which mode it is in and show what it is doing when uncertain.

Evaluations and constitutions are two key concepts in productizing generative AI. Evaluations refer to a set of systematic test cases and scoring criteria used to measure whether model outputs are "good enough" across thousands of input situations—since all outputs cannot be enumerated, teams instead define the characteristics and boundaries of good performance. Constitutions originate from Anthropic's "Constitutional AI" method, a written list of principles that specify the values the model should follow, the behaviors it should refuse, and tone boundaries, used to train and constrain the model. They work together: the constitution provides the value-level "what should be," and evaluations provide the measurable "whether it was achieved." For product teams, they are equivalent to requirements documents and acceptance tests in traditional software, except the object has shifted from deterministic features to a probabilistic behavior space.

Why Human Override Paths Are Essential

No AI system can be right forever. If people can easily step in, an error is just a minor incident; if they can't, it becomes a reason to stop trusting the product.

Override is not just about emergency buttons in home robots or cars. It can be as simple as undoing an action, editing a suggestion, or turning off a feature. In Ideo's AI writing tool Ethiqly, teachers approve every piece of feedback before students see it—this kind of control makes people more willing to hand more things over to the system step by step.

Manual paths should be prominent, fast, and available at the moment of need, not buried in a settings menu. When people know they can always take back control, they are more willing to let AI help.

A Practical Checklist for Product Teams

At an actionable level, you can start with a few questions: What task does this AI feature help users accomplish? How will users actually use it in real scenarios? Is there a more suitable capability than an LLM, such as computer vision or on-device models? If AI makes a mistake, can the user take over within seconds? Should physical controls be retained for instant, by-feel, or under-pressure operations? Does the product clearly show its current mode and uncertainty?

The boundary of AI productization is not whether the technology can do it, but whether AI should do it. Put users and tasks back at the center, and the boundary will emerge naturally.

One-line Summary

AI is not a product staple; the criterion is whether it helps users accomplish their tasks.

Paragraph Summary

Based on an interview with Savannah Kunovsky, head of industrial product design at Ideo, the article explores when AI integration is a breakthrough and when it is an obstacle. The author notes that the current AI wave is partly driven by investor and competitive pressure, and many companies instinctively jump to large language models while overlooking more suitable capabilities such as computer vision, sensor fusion, and on-device models. The most typical failures are when AI replaces interactions that already worked well: the Humane AI Pin, voice assistants that mishear commands, cars that move climate knobs into touchscreens, and a cooktop where a pot smoked during testing but there was no knob to find. The article argues that operations requiring instant, by-feel, or under-pressure action should retain physical controls, and Euro NCAP has accordingly begun rewarding physical buttons. Good integration starts from users and tasks; AI can be a front-end conversation or invisible in the background, but it must keep people at the center of the experience. The non-determinism of generative AI requires teams to define the product's operating space using evaluations and constitutions, and to retain prominent, fast human override paths.

#AI productization#AI integration#product design#physical controls#generative AI#Savannah Kunovsky#Ideo#Humane AI Pin#LLM#human override
Share:

Related articles