Frigade Assist API: Turning AI Customer Service from Document Dumps to Precise On-Screen Guidance

What Happens When AI Customer Service Can't "See" the Screen

Nearly every SaaS product now comes with an embedded AI assistant, but when a user asks "How do I do this?", the vast majority of AI support agents respond with a dense wall of text instructions. Users then have to hunt through the interface button by button on their own — a fragmented and inefficient experience.

The root of the problem: these embedded AI Agents simply can't "see" the screen. They can only generate text responses based on documentation knowledge, unable to perceive the user's current interface state, let alone tell them "click here." This is exactly the core pain point that Frigade Assist API aims to solve.

Recently launched on Product Hunt, Frigade Assist API earned 78 upvotes and ranked 16th on the daily leaderboard, categorized under "Customer Success," "Artificial Intelligence," and "SDK" — hitting a critical nerve in the AI customer service experience upgrade.

One Tool Call to Teach Your Agent to "Show the Way"

Core Mechanism: Guidance + Text Fallback + Screen Awareness

Frigade Assist API's design philosophy is remarkably restrained — it's not trying to replace your existing AI Agent, but rather exists as a single tool (tool call) that your Agent can invoke.

It's worth explaining the key concept of "tool calling" here. Tool Call (or Function Calling) is a core mechanism in modern LLM application architecture. Traditional LLMs can only generate text, but through tool calling, an LLM can identify scenarios during conversation that require external capabilities and request pre-defined external functions or APIs in a structured way. OpenAI's Function Calling and Anthropic's Tool Use are typical implementations of this mechanism. In Agent frameworks, developers pre-register a series of "tools" — each with a name, description, and parameter definitions — and the LLM autonomously decides whether to call a tool, which one to call, and what parameters to pass. Frigade leverages this mechanism to position itself as a new tool in the Agent's toolbox, rather than a replacement for the entire Agent system.

When a user asks an operation-related question, your existing Agent simply needs to make one tool call to Frigade, and Frigade handles three things:

• Draws step-by-step guidance directly on the page: Instead of text descriptions, it highlights and guides users through the actual interface, showing them exactly where to click, step by step;
• Returns plain text answers when visual guidance isn't possible: When guidance doesn't apply, Frigade hands a clear text response back to the Agent, ensuring the conversation doesn't "break";
• Tells the Agent what the user is currently looking at: This is especially critical — Frigade fills in the "screen awareness" piece of the puzzle, letting the Agent understand the user's current interface context.

This three-part design upgrades AI customer service from a "quiz machine" into a true operational guide that walks users through task completion.

Compatibility with Existing Tech Stacks

On the integration front, Frigade Assist API demonstrates strong openness. The team explicitly states it works with the Vercel AI SDK and supports any framework with tool calling capabilities.

Vercel AI SDK is an open-source SDK from Vercel, the company behind Next.js, designed to help frontend developers rapidly build AI-powered applications. It provides a unified interface for connecting to multiple model providers including OpenAI, Anthropic, and Google, with native support for streaming responses, tool calling, structured output, and more. Since the Vercel ecosystem holds a prominent position in modern web development — Next.js is one of the most widely deployed full-stack React frameworks — compatibility with Vercel AI SDK means Frigade can seamlessly integrate into a large number of existing SaaS product tech stacks, reaching a broad developer audience.

Developers don't need to restructure their existing Agent architecture — they just plug in Frigade as a callable tool, keeping migration costs extremely low.

Where Product Knowledge Comes From: Browser Agent Auto-Learning

This might be Frigade's most interesting technical highlight. Traditional product guidance tools rely on manually configuring every step — a massive workload that's difficult to maintain. Frigade claims it "knows your product" because — a browser Agent has already used your product first.

In other words, Frigade uses a browser automation Agent to actually operate the product interface, automatically learning feature paths and interaction logic to build a knowledge base for guidance.

Browser automation Agents are a rapidly evolving technology. Early representative frameworks include Playwright and Puppeteer, which control browsers programmatically to execute predefined actions. More recently, AI-driven browser Agents — such as Anthropic's Computer Use and OpenAI's Operator — go further, navigating, clicking, and typing in browsers like real users, even understanding page semantics. Core technologies include DOM parsing, visual recognition (understanding interfaces through screenshots combined with multimodal models), and action sequence planning. Frigade uses similar technology to automatically "try out" customer products. Essentially, it shifts browser Agents from "operating on behalf of users" to "learning on behalf of users" — it doesn't complete tasks for users but builds a product knowledge graph through automated exploration, then uses that knowledge to guide real users.

This mechanism solves the most frustrating maintenance challenge for product guidance tools:

"It re-learns every time you ship a release."

This statement highlights a long-standing pain point in product guidance. Product guidance tools aren't a new concept — early representative products include WalkMe (founded in 2011, acquired by SAP for $1.5 billion in 2021), Pendo, Appcues, Userflow, and others. These tools typically use a "no-code configuration" model: product managers select page elements in a visual editor, set trigger conditions and guidance steps, and generate tooltip popups or highlight guides. However, the biggest pain point of this approach is maintenance cost — every time the product interface is redesigned, broken CSS selectors and changed page structures can cause guidance to fail, requiring manual reconfiguration. By replacing manual configuration with browser Agent auto-learning, Frigade essentially uses AI capabilities to solve the most fundamental maintainability problem of this generation of product guidance tools, dramatically reducing operational overhead.

What Industry Trend Frigade Is Tapping Into

AI Agent Evolution: From "Conversation" to "Action"

The emergence of Frigade Assist API reflects a major trend: AI Agents are evolving from "being able to chat" to "being able to act." Over the past year, industry imagination around Agents has largely stayed at the text interaction level; the direction Frigade represents is embedding Agents deeply into product interfaces, turning them into real-time operational guides for users.

More specifically, AI Agent capability evolution can be divided into several tiers: the first tier is text conversation — Q&A based on knowledge bases (like traditional RAG approaches); the second tier is information aggregation, where Agents can call search, databases, and other tools to retrieve real-time information; the third tier is screen awareness, where Agents can understand the user's current operational context; the fourth tier is autonomous operation, where Agents can directly perform interface operations on behalf of users. Frigade currently covers primarily the third tier — it lets Agents "see" the user interface and provide visual guidance, but the actual operations are still performed by the user. Solutions like Anthropic's Computer Use, on the other hand, aim directly at the fourth tier. Notably, in SaaS customer service scenarios, the third tier may be more practically valuable than the fourth, because users typically want to maintain control over the process — they just need directional guidance rather than handing operational authority entirely to AI.

Behind this is a simple insight: what users often want isn't an "answer" — it's a "result." Rather than giving users an instruction manual, it's better to walk them through the operation directly.

A New Tool for Customer Success

Here's a telling detail: Frigade is explicitly categorized under "Customer Success." This signals that its value proposition isn't just technical innovation — it points to tangible business outcomes: lowering user learning curves, improving SaaS product activation and retention rates, and reducing support ticket volume.

The business significance of these metrics is worth unpacking. According to industry data, the average SaaS activation rate (users going from signup to completing a core action) typically falls between 20%-40%, meaning more than half of new users churn before experiencing the product's value. Meanwhile, the average cost per support ticket is approximately $5-15 (depending on complexity), with a large portion being low-complexity "how do I do this" questions. If AI customer service can resolve these issues directly through visual guidance, it not only significantly reduces ticket volume but, more importantly, shortens the user's "Time to Value" — one of the most critical factors affecting SaaS retention rates. For SaaS companies, these are all quantifiable core metrics.

Potential Questions and Observations

Despite the clear product vision, several questions deserve attention:

First is the accuracy of the browser Agent auto-learning mechanism. While auto-learning eliminates manual configuration, whether it can fully and accurately capture edge-case operation paths in complex products still needs real-world validation. Especially in scenarios involving permission controls, multi-role views, conditional branching, and other complex interactions, whether the browser Agent can correctly understand and record all possible operation paths remains a technical challenge worth watching.

Second is the applicability boundary of visual guidance. The team themselves acknowledge that scenarios exist where "visual guidance isn't possible," falling back to text answers. How frequently this fallback logic triggers directly determines the ceiling of the user experience.

Finally, there's multi-platform adaptability. Information disclosed so far mainly revolves around web browser interfaces, and support for desktop applications or mobile guidance remains unclear.

Conclusion

Frigade Assist API addresses a real and widespread pain point: embedded AI customer service that "can't see the screen and can only dump documentation." Through a lightweight "single tool call" integration approach, combined with the clever design of "browser Agents that auto-learn your product," it adds screen awareness and visual guidance capabilities to AI customer service.

As AI Agent capabilities rapidly expand outward, product guidance tools that focus on "having AI actually walk users through operations" may be closer to the real needs of SaaS products and their users than yet another general-purpose conversational model.