Instinct Founder Fireside Chat: Practical Lessons and Future Directions for Building AI Agents

At the Glow stage of Fireworks Forge, Instinct founder Noah Shinn and Fireworks CTO and co-founder Dzhulga Rakhimov held a fireside chat focused on lessons learned while building AI agents and where the field is headed next.

A Technical Conversation Focused on Putting Agents into Practice

This conversation took place at the Fireworks Forge event and was published by the official Fireworks account. From publicly available information, Shinn is the founder of Instinct, while Rakhimov, as Fireworks' CTO and co-founder, moderated the discussion. The core of their discussion was not broad industry trends, but the firsthand experience Shinn accumulated while personally building agent systems.

You may not have noticed that the value of this kind of fireside chat often lies in the "failure notes" rather than product promotion. Agent development is still at a stage where engineering practice is moving far faster than theoretical summaries, and firsthand founders willing to publicly share the real trade-offs made during the build process are more useful to peers as a reference. The specific content of the conversation was presented as a livestream, and the official account did not release a written transcript, so this article starts more from the themes of the conversation and outlines several key dimensions worth watching in agent building.

Building Agents: The Distance from Demo to Product

Engineering Complexity Is Underestimated

Agent systems are fundamentally different from traditional single-turn question-answering models. A usable agent needs to handle tool calls, multi-step reasoning, state management, and error recovery, and a problem in any one of these links can cause the entire task to fail. The "lessons learned" Shinn shared in the conversation most likely cannot avoid the reality of this kind of engineering complexity: tasks that run successfully in the demo stage often fail in real environments because of edge cases.

This is also a consensus in the current agent space: model capabilities themselves are advancing rapidly, but packaging those capabilities into stable and reliable products still requires substantial engineering investment. As an inference infrastructure provider, Fireworks is likely to perceive this kind of demand more directly.

State Management in the context of agents refers to the persistence and passing of intermediate results across multi-step tasks. Unlike traditional stateless inference, an agent needs to retain context, records of executed actions, and pending subgoals between multiple rounds of tool calls; otherwise, it cannot support resuming from checkpoints and error recovery. Common approaches include explicit task graphs, message history trimming, and external memory storage. Poor state design can lead to context bloat, repeated calls, and even task loops, making it a key engineering constraint on the stability of long-chain agents.

The Challenge of Evaluation and Iteration

An agent's behavior space is far larger than that of ordinary model calls, and how to evaluate whether an agent "does well" is itself an open question. The intermediate steps of multi-step tasks are difficult to score automatically, and end-to-end success rates are affected by environmental fluctuations. The experience mentioned in this conversation very likely involves how to establish an iterative evaluation loop, which is an infrastructure problem every agent team must face.

Tool Use is the core dividing line between agents and ordinary conversational models. It refers to the model's ability to actively call external functions, APIs, or retrieval systems during generation, usually by describing structured function signatures (such as JSON Schema) so the model outputs call parameters, which the host program then executes and returns results to. The difficulty of tool use is not a single call, but whether the model needs to judge when to call, whether parameters are correct, and whether to retry after failure. When multiple tools are chained, errors amplify along the call chain, so engineering practice often requires timeout control, idempotent design, and fallback strategies. This is also one of the first weaknesses exposed when agents move from demo to product.

End-to-end evaluation and intermediate-step evaluation are the two basic paths for agent evaluation. End-to-end evaluation only looks at whether the final task is completed; the metric is intuitive but the signal is sparse, making it difficult to locate the cause of failure. Intermediate-step evaluation scores each tool call or reasoning step; the signal is dense but depends on human annotation or strong model judges (LLM-as-judge), and both cost and noise are higher. In practice, a layered strategy is usually adopted: a small number of high-value end-to-end cases guard the bottom line, while step-level evaluation drives daily iteration. Environmental fluctuations (such as API instability and data changes) also introduce score drift, so evaluation sets need to be versioned and subjected to regular regression testing.

Speculation on the Direction of Next-Generation Agents

The second part of the conversation focused on "where they will go next." Although the official account did not disclose a specific roadmap, some reasonable inferences can be made from the combination of Shinn's and Rakhimov's backgrounds.

Shinn has cross-domain experience between academia and entrepreneurship, and Instinct, as the company he founded, is closely related to improving agent autonomy. Rakhimov's Fireworks focuses on model inference services, and the intersection of the two centers on "how to make agents run efficiently under real workloads." It can be expected that subsequent evolution will revolve around longer task chains, stronger tool-use capabilities, and lower inference costs.

It should be noted that the above is reasonable speculation based on public identities, not conclusions explicitly given in the conversation. Readers interested in specific technical directions can watch the full content through the official livestream link.

Inference cost and latency are hidden thresholds that determine whether agents can be deployed at scale. A multi-step agent may trigger dozens of model calls for a single task, and token consumption and latency grow approximately linearly, or even superlinearly, with the number of steps. Common ways to reduce costs include using small models for simple sub-steps, summarizing and compressing tool return results, caching high-frequency intermediate results, and adopting techniques such as batching and prefix caching on the inference side. The core value of inference infrastructure vendors such as Fireworks is precisely to reduce the cost and latency of a single call enough to support long-chain agents.

Why This Conversation Is Worth Watching

The agent space is full of information noise: many product launches, but few real experiences. A fireside chat with founder-level guests usually has a higher information density than marketing materials. Shinn shared practical insights from building agents, while Rakhimov asked questions from an infrastructure perspective; the collision of these two perspectives happens to cover the complete chain from agent R&D to deployment.

For teams currently building agent products, the value of this kind of conversation is not in providing standard answers, but in confirming which problems are common across the industry and which pitfalls others have already encountered. As an event platform, Fireworks Forge opens this kind of exchange in livestream form, also providing the community with a low-cost learning channel.

How to Access the Full Content

The official tweet included a livestream viewing link, and the conversation was presented in video form. Since no public written transcript is currently available, this article can only provide a framework-level interpretation based on the conversation themes and the guests' backgrounds; for specific views and cases, the original video should be taken as authoritative.

One-line Summary

Instinct founder Shinn and Fireworks CTO discuss agent-building experience and next steps.

Paragraph Summary

The article documents a Fireworks Forge fireside chat in which Instinct founder Noah Shinn and Fireworks CTO and co-founder Dzhulga Rakhimov exchanged views on lessons from building AI agents. Because no official transcript was released, the article provides a framework-level interpretation based on the conversation themes and the guests' backgrounds, noting that agents face two major challenges in moving from demo to product: first, engineering complexity from tool calls, multi-step reasoning, state management, and error recovery is underestimated; second, multi-step tasks are difficult to score automatically and end-to-end success rates are affected by environmental fluctuations, making an evaluation and iteration loop difficult to establish. Regarding next-generation directions, the article speculates that evolution will revolve around longer task chains, stronger tool-use capabilities, and lower inference costs, and emphasizes that the two guests represent application-layer and infrastructure-layer perspectives respectively, making this kind of conversation more information-dense than marketing materials.