Azure Cosmos DB

Azure Cosmos DB's schema-agnostic flexible schema can simultaneously hold episodic memory (nested JSON event records) and semantic memory (flat-field preference tags).

Agent Memory Toolkit relies on Azure Cosmos DB's optimistic concurrency control (ETag mechanism) and tunable consistency levels to handle concurrent write conflicts.

Azure Cosmos DB product manager Madeline Egan points out that by default most AI applications start with no persistent memory, no preferences, no history, and no context.

The Agent Memory Toolkit provides a unified way for all agents to store, retrieve, and update memories in Azure Cosmos DB.

In Agent Memory Toolkit, memory entries are typically partitioned using the user ID as the partition key.

Developers should avoid mixing episodic memory and semantic memory in the same collection, otherwise relevance ranking during retrieval will be disturbed by noise.

Azure Cosmos DB's RU (Request Unit) is the unit of throughput and cost measurement, normalizing the CPU, IO, and memory consumed by read and write operations into a single number, with charges billed by provisioned or autoscaling RU/s.

The Azure Cosmos DB team proposed an AI Agent production operations framework comprising three phases—Observe, Evaluate, and Optimize—and these three phases form a continuous loop rather than a one-time process.

Each Agent request generates a complete trace recording key actions such as tool calls, routing decisions, and memory retrieval, and when the Agent interacts with Azure Cosmos DB, those interactions are also included in the same trace.

Cosmos DB-specific evaluations can verify whether cross-session memory recall is accurate—for example, after a user says they dislike red-eye flights, whether the Agent remembers and honors that preference in a brand-new session.

Optimization is divided into four layers: RU optimization (index strategy, partition model, and query patterns at the Cosmos DB layer), retrieval optimization (tuning vector search and hybrid search), memory optimization (retaining only key information through summarization and compression to reduce the number of round trips to the database), and LLM layer optimization (more compact prompts, smaller context windows, and fewer model calls to reduce token consumption).

In the Cosmos DB scenario, the MCP Toolkit acts as a bridge between the agent and the local emulator: the agent declares executable operations via MCP, and the tool side handles permission control and execution.

The goal of the first episode is to start from an empty folder and build a working Cosmos DB application using only a laptop, without relying on an Azure subscription at all.

In Cosmos DB, the partition key is the core concept that determines data distribution and horizontal scaling; each container must specify a partition key property, and Cosmos DB distributes data across different physical partitions based on the hash value of that property.

An ideal partition key should have three characteristics: high cardinality, even distribution, and frequent use as a filter condition in queries; using low-cardinality properties (such as status or type) or frequently updated properties as the partition key is a common mistake.

A request unit (RU) is the abstract unit in Cosmos DB for measuring throughput and cost; a point read (reading a single 1KB document by id and partition key) consumes 1 RU.

Under Cosmos DB's provisioned throughput mode, developers purchase capacity in request units per second (RU/s), so RU efficiency directly determines application cost.

The local emulator gives developers a real Cosmos DB environment on their laptop; Agent Kit teaches coding assistants best practices; and the MCP Toolkit lets agents connect to data securely.

Vector search is one of the query capabilities supported by Cosmos DB, used to find the semantically most similar results in vectorized data; Cosmos DB stores vectors in an array property of the document and provides the VectorDistance function to compute distance in queries.

Executing Cosmos DB vector queries usually requires a vector index (such as a DiskANN index) to ensure performance, and vector queries consume relatively many RUs.