Gemini API in Practice: Building AI Agents with Files

In this official hands-on session from Google DeepMind, two product managers, Mariano and Philip, systematically showcased Google's current AI development toolchain from two angles: app development with AI Studio and Agent building with the Interactions API. The core message is singular: going from idea to production is becoming faster than ever, and building Agents no longer requires writing complex Python loops—all you need to do is write files.

The Gemini Ecosystem: From Models to Multimodality

Mariano began by reviewing the iteration cadence of the Gemini family. From Gemini 1.5, to the widely acclaimed Gemini 2.5, and now to the flagship Gemini 3.5 Flash (the author's daily go-to model in the session), Google has maintained a high frequency of updates on frontier models. Meanwhile, the open-source side—Gemma, the generative media model Nano Banana 2, and the text-to-speech (TTS) models—are all advancing in parallel.

The reason Gemini 3 has been widely adopted by leading companies like Cursor, GitHub, Figma, and Vercel lies in its multimodal capabilities. It can accept a variety of inputs—system instructions, text, images, audio, video, speech, URLs, and more—and output text, images, audio, video, function calls, or code.

It's worth understanding this deeply: multimodal refers to a model's ability to understand and generate multiple types of data simultaneously. Traditional language models can only handle text, whereas multimodal models map data from different modalities into the same semantic dimension through a unified representation space (embedding space), enabling cross-modal reasoning and association. This relies on extending the Transformer architecture—splitting images into patches and converting audio into spectral features before feeding them into the model alongside text tokens. Gemini adopted a natively multimodal architecture from the outset, rather than stitching together separate vision and speech modules, which gives it more coherent performance on cross-modal tasks. For developers, this means a single API call can accomplish composite tasks like "describing an image" or "understanding meaning from audio," dramatically simplifying workflows that previously required chaining together multiple specialized models.

This "mix-and-match" capability lets developers build composite applications that were previously hard to realize. Mariano gave an example: he built an Agent himself that reads his inbox through Workspace integration and then generates a short podcast to listen to while making coffee in the morning.

AI Studio: A One-Stop Platform from Idea to App

AI Studio (ai.studio / ai.dev) was the focus of this session. It's not just a Playground for testing new features—it has evolved into a complete app-building environment.

A Zero-Config Full-Stack Development Experience

As a full-stack web developer, what Mariano values most is the full-stack Node.js application support. You can now build an entire application within AI Studio; the platform automatically configures both the Node.js server and frontend, provides access to the full npm ecosystem, and includes built-in secret management for server-side API keys. He admitted that authentication and key management have always been the most headache-inducing parts of frontend development, and now these pain points are directly resolved by built-in capabilities.

In addition, native Firebase integration brings out-of-the-box "Sign in with Google" authentication and automatic provisioning of Firestore (a NoSQL database), with Cloud SQL support coming in the future. Integration with Workspace is achieved through automated OAuth, with no manual configuration required in the Cloud Console—dramatically lowering the barrier to connecting with enterprise data from Gmail, Docs, Drive, and more.

OAuth (Open Authorization) is an open authorization standard that allows third-party applications to gain access to specific resources of a user on a given service without obtaining the user's password. When an application needs to read a user's Gmail or Drive data, OAuth grants minimal-privilege authorization by issuing time-limited, scope-restricted access tokens. Traditionally, configuring OAuth required manually creating credentials in the Google Cloud Console, setting up callback URLs, and declaring scope permissions—a tedious and error-prone process. An API Key, on the other hand, is a static credential that identifies the caller's identity; once leaked, it can be abused. Therefore, "Secret Management"—storing keys encrypted on the server side to avoid exposure in frontend code—is a basic security practice for production environments. AI Studio's built-in support for these capabilities directly addresses the authentication complexity that developers have long faced.

Generating Native Android Apps with Natural Language

A capability the author called a "game-changer" is generating native Kotlin and Jetpack Compose applications through natural language. According to Mariano, over 1 million apps have been created since its launch last month. The entire workflow requires no local environment setup—there's a live emulator running right in the browser, with support for one-click installation to your phone or one-click publishing to Play Console for testing.

To appreciate the significance of this capability, you need to understand the underlying stack. Kotlin is a programming language developed by JetBrains and established by Google in 2017 as the officially preferred language for Android. Compared to Java, it's more concise and offers stronger null safety features, significantly reducing the risk of crashes. Jetpack Compose is a modern declarative UI toolkit Google introduced in 2021, where developers simply describe "what the interface should look like," and the framework automatically handles redrawing when state changes—replacing the tedious traditional approach based on XML layouts and imperative operations. This "Kotlin + Compose" combination is the mainstream paradigm for native Android development today. AI Studio's ability to directly generate code that conforms to this modern standard through natural language means the output is not an outdated or toy-level prototype, but an engineering-grade artifact ready to move directly into production iteration—which is the technical backing behind the "game-changer" assessment.

Detail Improvements That Boost Development Efficiency

AI Studio has also introduced several experience optimizations:

• Focus Annotation: Click directly on a component on the page to issue precise modification instructions targeting that component, no longer having to laboriously describe in words "the heading of the second paragraph below the hero section."
• Design Previews: During the model's generation process, it first presents several different low-fidelity previews in various styles (such as high-density and lively, elegant dark mode, or immersive UI), letting developers choose a style ahead of time to avoid rework after generation is complete.
• Tab Autocomplete: Targeting the writing bottleneck of prompts—after typing a few words and pausing briefly, the model instantly completes and enhances your prompt in real time.

Seamless Export to Local

When you've built up to a certain stage in AI Studio and want to return to a familiar IDE (such as Anti-Gravity) to continue development, you can export with one click—losing no context or history, with a clean code structure, and it even asks whether to export along with the keys. Mariano demonstrated live a "Daily Stretch" app, going from a prompt to a runnable dark-themed app—including images generated by Nano Banana—with virtually zero manual intervention throughout.

Interactions API: A Unified Interface for Agents

The second half was led by Philip, from Germany, who introduced the officially GA (General Availability) Interactions API—Google's brand-new unified interface for calling Gemini models and Gemini Agents.

From "Turn-Based" to "Step-Based" Data Models

Philip pointed out that AI applications are shifting from simple Q&A to agentic use cases—we define tasks and goals, and let the model achieve them. To that end, the Interactions API made several key design choices:

• Unified models and Agents: The same codebase lets you choose to use a Gemini model, or use Agents like Anti-Gravity and Deep Research.
• Farewell to user/model turn-taking: The alternating "user input—model output" structure of the ChatGPT era can no longer accommodate new, complex interaction types like reasoning, function calls, and function results. The Interactions API shifts to a step-based data model, where each interaction becomes an independent data type—both easier for developers to build with and easier for the model to reason over.
• Server-side state management: Simply send new input and provide the ID from the previous turn, and the API automatically maintains context (you can also choose to manage it yourself).
• Asynchronous background execution: For long-running tasks, creating a request returns an ID, and you can poll for status or receive event updates via streaming.
• Mixing first-party and third-party tools: Previously, Google Search couldn't be used together with custom functions; now they can be freely combined. For example, you could build a security incident Agent that searches for the latest React CVE vulnerabilities and then queries an internal database.

The shift from turn-based to step-based here is worth expanding on. Early conversational AI (like ChatGPT) used a strictly alternating "user/model" turn-based data structure, where each conversation was modeled as a back-and-forth sequence of user messages and model replies. This structure was clear and concise for pure Q&A scenarios, but as model capabilities evolved, new interaction types emerged—chain-of-thought reasoning, function calling, tool execution result returns—which are neither "user input" nor "final reply." Forcing them into a turn-based framework causes semantic confusion. The step-based data model models each reasoning step, each tool call, and each result as an independent, typed data unit, forming a traceable execution trajectory. This not only makes it easier for developers to parse and debug Agent behavior, but also provides the model itself with more structured context, helping it maintain coherent reasoning across long-chain tasks.

Remote Environments: Managed Isolated Sandboxes

The environment parameter is what makes the Interactions API distinctive. A remote environment is an isolated sandbox running within Google Cloud, in which the Agent can execute commands and create files without performing any operations on your local machine.

A sandbox is a security mechanism that confines program execution to a controlled, isolated environment, preventing it from affecting the host system or accessing unauthorized resources. When an Agent needs to execute arbitrary code, install dependencies, or read/write files, running directly on the developer's local machine poses enormous security risks—malicious or erroneous code could damage the system or leak data. Moving the execution environment to an isolated cloud sandbox is essentially an application of containerization (such as lightweight virtualization technologies like gVisor and Firecracker) and cloud-native architecture. Persistence means the sandbox state is retained across multiple requests, avoiding the overhead of spinning up the environment and reinstalling dependencies every time.

These environments are persistent: you first have the Agent install dependencies like Pandas, and subsequent requests can directly use Pandas to analyze CSV files without repeating configuration. Resources can be mounted into the environment via GitHub repositories, GCS buckets, or inline files, with support for private repositories and private buckets.

Initial sandbox startup takes about 8 to 10 seconds. During Philip's live demo, the Agent reported the environment info: Ubuntu, Linux, an 8-core AMD virtual CPU, 15GB of memory, with Python, pip, and curl pre-installed. This 8-to-10-second cold start time reflects the engineering trade-off between security isolation and startup efficiency, while the Ubuntu, multi-core CPU, and pre-installed Python configuration indicates this is a fully functional Linux-like workstation, sufficient to support real computing tasks like data analysis and code building.

An Agent Is Just "A Composition of Files"

The most mind-bending point of this session was Philip's redefinition of the essence of an Agent:

Agents are no longer those weird and complex Python or JavaScript loops. We just write files, then provide those files, and we've built our own Agent.

Take the "AI Talk Radio" applet demonstrated live as an example—its agent folder structure is extremely simple: an agents.md file defines the role (turning any content into a radio show), a workflow describes the workflow (first do research, write the script, generate speech, generate background music, mix, generate a cover image), plus several skills. Each skill is just a documentation file plus a Python script—for instance, the TTS generation skill, whose script calls the Interactions API's Flash TTS model. In other words, the Anti-Gravity Agent runs on top of the Gemini API and, through skills, in turn calls other Gemini models to complete subtasks.

The accompanying Agents API lets you package "files + environment" into a reusable Agent, without having to re-provide the resource configuration for every request. A typical scenario is building a GitHub code review Agent: first have the Agent install the GitHub CLI and obtain the ID of that environment, then reuse this environment with the pre-installed CLI every time you create the Agent.

Google also provides an experimental Gemini API CLI—it's not a coding Agent (unlike Claude Code, Gemini CLI, or Anti-Gravity CLI), but an API CLI specifically for calling APIs and creating Agents.

Credential Security: The Network Proxy Mechanism

Addressing the concern of "could an Agent calling the Gemini API abuse credentials," Google built a layer of network proxy: on one hand it restricts the URLs or domains the Agent can access, and on the other it provides header transformation. The real credentials are stored in the proxy layer rather than inside the sandbox. The Agent only knows that it "can call the Gemini API," and when it makes an external request, the proxy injects the real credentials. This means credentials can be rotated or removed at any time, and the Agent can never touch the real keys.

This design embodies the "Zero Trust" security philosophy—that is, trusting no part of the chain by default, including the sandbox in which the Agent itself runs. The traditional approach was to inject API keys into the runtime environment, but once the sandbox is breached or the Agent behaves abnormally, the keys are at risk of leaking. The network proxy, acting as an intermediary layer between the Agent and external services, plays the role of a "credential custodian": when the Agent issues a request, it only carries a placeholder identifier, and the proxy layer injects the real credentials and completes header transformation before forwarding, while restricting the Agent to accessing only permitted endpoints through a URL/domain allowlist. This way, even if the Agent's code is fully exposed, an attacker cannot extract the real keys from it; and operations teams can rotate or revoke credentials at the proxy layer at any time without modifying the Agent itself. This architecture of "separating keys from the execution body" is a key security cornerstone for building trustworthy managed Agent services.

Key Takeaways

The entire session conveyed three clear messages:

1. AI Studio dramatically accelerates "idea to production" and is the starting point for all development;
2. The Interactions API is now officially GA, providing a unified, multimodal-aligned interface for agentic workflows;
3. The essence of an Agent is a composition of files—Markdown plus a few scripts is all it takes to define a powerful managed Agent.

For developers, the most direct course of action is: visit ai.studio to get an API key (with a generous free tier and configurable spending caps), rapidly build prototypes in AI Studio, then export to Anti-Gravity with one click for deeper development. Google is systematically lowering the barrier to building AI applications and Agents with the principles of "zero config," "one-click deployment," and "files as Agents."

Core Points