LangChain Agent Signed HTTP Tools: Letting Websites Identify AI Agents vs. Crawlers

AIDR signs AI agent requests with RFC 9421 and Ed25519, letting sites distinguish legitimate agents from crawlers offline.
AIDR is an open-source project tackling the difficulty of telling AI agent requests apart from crawlers: both use cloud IPs, User-Agents are forgeable, and sites lack verifiable evidence. Its approach uses the RFC 9421 HTTP message signing standard with Ed25519 to sign each request and have agents carry a short-lived credential bound to organizational identity, so sites can verify request origin offline. In LangChain, integration only requires swapping in signed HTTP tools restricted to preset target origins to reduce credential leakage from prompt injection. Limitations: it only works on sites that enable verification, requires an organizational account, and its value depends on ecosystem adoption. A browser demo that sends no data is also provided.
AI agents visiting websites send requests that are nearly indistinguishable from those of crawlers: the same cloud IPs, and User-Agent strings that anyone can forge at will. Website administrators want to let valuable AI agents through while worrying about malicious crawlers slipping in. The open-source project AIDR attempts to solve this problem with cryptographic signatures—having each AI agent sign its HTTP requests and carry a verifiable organizational credential, so that websites can clearly distinguish "legitimate agents" from "crawlers".

The Problem: Why Agent Requests Are Hard to Distinguish from Crawlers
Developers point out that HTTP requests made by agents look exactly like those from crawlers: both come from cloud provider IP ranges, and the User-Agent field can be filled in by anyone. Even if a website wants to allow certain AI agents access, there is no verifiable basis for doing so. This leads to two outcomes: either the site blocks all automated traffic indiscriminately, inadvertently harming legitimate agents; or it lets crawlers in, bringing security and cost risks.
AIDR's approach is to add offline-verifiable signatures to agent requests, enabling websites to confirm on each request that "this is an agent from a registered organization" without relying on a third-party real-time service.
The Solution: RFC 9421 Signatures and Ed25519 Credentials
AIDR's implementation is based on the RFC 9421 HTTP message signing standard, using the Ed25519 algorithm to sign each request. The agent also carries a short-lived credential that binds its key to a verified organizational identity. As a result, every request a website receives carries verifiable identity information, rather than relying on User-Agent guesswork.
Integration in LangChain only requires swapping in the tools:
from aidr_agent import Agent
from aidr_agent.langchain import aidr_http_tools
agent = Agent.load_or_enroll("research-01", audiences=["https://shop.example"])
tools = aidr_http_tools(agent) # http_get, http_post, signed
These tools only allow access to pre-listed target origins. This is critical: if the LLM were allowed to choose URLs freely, a prompt injection attack could cause the agent to present its identity credentials to sites it should not access. Restricting target domains via an allowlist mitigates this risk.
RFC 9421 is the HTTP message signing standard published by the IETF in 2024, defining a general mechanism: the sender selects several components from the request (such as @method, @target-uri, content-digest, etc.), concatenates them in a fixed format into a signature base string, signs that base string, and places the signature and the list of components used into the Signature and Signature-Input headers. The receiver reconstructs the base string using the same rules and verifies the signature, so the signature covers the key semantics of the request, and any tampering will cause verification to fail. Compared with earlier HTTP signature drafts or proprietary schemes, RFC 9421 does not bind to a specific algorithm, its signature parameters are negotiable, and it supports key rotation and expiration—making it the foundation for standardizing "request-level verifiable identity" today.
Ed25519 is a digital signature algorithm based on twisted Edwards curves (defined in RFC 8032), with 32-byte private and public keys and 64-byte signatures—much shorter than RSA at equivalent security strength, making it suitable for HTTP headers. Its signing and verification are both fast, and implementations are less prone to side-channel errors, which is why it is widely used for SSH keys, Sigstore, and various modern credential systems. For scenarios like AIDR that require per-request signing, Ed25519's low overhead and short signature length are key advantages; its public key can be published via organizational credentials, and websites can cache it for offline verification without querying the origin on every request.
Cloudflare Web Bot Auth is an automated traffic identity verification mechanism promoted by Cloudflare, allowing legitimate bots to carry identity credentials based on HTTP message signatures in their requests. After verifying the signature at the edge, Cloudflare can mark the traffic as an authenticated bot rather than treating it as a crawler. It relies on a public key directory: bot operators publish their public key sets at agreed locations, and verifiers fetch and cache the public keys accordingly. AIDR's integration with this mechanism means its signatures can be verified not only by self-hosted middleware but potentially also recognized at the CDN edge layer, expanding the scheme's applicability. However, registration and maintenance of the key directory remain the operator's responsibility, which is one concrete manifestation of the fact that this is "not zero-configuration".
Limitations: Only Effective on Sites That Have Enabled Verification
AIDR is not a master key. It cannot help agents bypass websites that do not verify signatures. The signature only takes effect when the target site actively enables verification (for example, by deploying AIDR middleware or registering a key directory in Cloudflare Web Bot Auth). In addition, registering an organizational account is a required step, so it is not a zero-configuration solution.
This limitation means AIDR's value depends on ecosystem adoption: only when enough websites are willing to verify signatures can agent identity authentication form a positive feedback loop.
Demo and Reflections: Offline-Verifiable, Request-Level Identity
The developer provides an in-browser demo where you can observe the signature handshake process, and no data is sent to any server. In a detailed article, the author also shares three bugs encountered and fixed during development, emphasizing that "the answer must be offline-verifiable and per-request".
For website operators who want to manage AI agent access permissions, AIDR offers an approach based on open standards; for agent developers, it raises a question worth considering: how do you identify your agent's identity to websites today?
(This article is based on the Reddit post and the project's public materials; the SDK is licensed under Apache-2.0.)
One-line Summary
AIDR uses RFC 9421 and Ed25519 to sign AI agent requests, enabling websites to distinguish legitimate agents from crawlers offline.
Paragraph Summary
AIDR is an open-source project that aims to solve the problem of distinguishing AI agent requests from crawlers: both come from cloud IPs, User-Agents can be forged at will, and websites lack a verifiable basis. Its solution is based on the RFC 9421 HTTP message signing standard, using Ed25519 to sign each request and having agents carry a short-lived credential bound to organizational identity, so websites can verify the request's origin offline. Integration in LangChain only requires swapping in signed HTTP tools, and the tools only allow access to preset target origins to reduce the risk of credential leakage from prompt injection. The limitation is that the scheme only works on sites that actively enable verification, requires registering an organizational account, and its value depends on ecosystem adoption. The developer also provides an in-browser demo that sends no data.
Related articles

OpenAI to Shut Down Agent Builder and Two Other Services on November 30, Developers Must Migrate Urgently
OpenAI will shut down Agent Builder, the evals dashboard, and reusable prompts on November 30; evals is already read-only. Here's the scope, OpenAI's migration paths (Agent SDK, PromptFu, your own code), and what developers should export and rebuild this week.

RankSpot: A Pay-Per-Use SEO & AI Search Execution Layer That Lets Agents Write, Publish, and Build Backlinks for You
RankSpot is an SEO & AI search execution layer for AI Agents. Connect via MCP to Claude or ChatGPT to write, publish & build backlinks. Pay-per-use, $3/article.

Appxiom: Giving AI Coding Assistants Production Context
Appxiom gives AI coding agents production context—crashes, jank, memory leaks, failed network calls—across Android, iOS, Flutter, KMP, and Web, ranking bugs by Goal Friction Impact.