LLM API
A service model in which the capabilities of a cloud-based large language model are invoked through an API interface. Developers can integrate AI functionality without local deployment and pay according to request usage.
Core Facts
Timeline (last 90 days)
LLM APIs are priced separately for input and output tokens, and the per-token price does not change with the temperature and seed parameters.
LLM APIs rate-limit on two independent dimensions, requests per minute and tokens per minute; even with monthly budget to spare, an endpoint can still return 429 during traffic peaks.
Traditional cloud FinOps tools focus on infrastructure utilization (CPU/memory) and do not parse token metrics from API payloads, causing API-based AI consumption to become a blind spot.
The correct way to track AI agent costs is: inject a correlation ID at the API gateway, propagate these IDs through the orchestration framework, and intercept all LLM API calls, emitting standard OpenTelemetry spans with token metadata.
In LLM repair loops, the recommended typical circuit breaker parameters are: 5 consecutive failures trigger the break, a 30-60 second cooldown, and rate-limited probing during the half-open phase.
Pressing all LLM traffic for critical business onto a single model API makes annual cumulative downtime reaching the hour level almost the norm rather than the exception.
By adjusting only the API-layer call structure, without rewriting application logic and with user volume and request volume basically unchanged, a developer reduced monthly LLM API spending from $212 to $61, a reduction of about 70%.
Mainstream LLM APIs usually rate-limit on both requests per minute (RPM) and tokens per minute (TPM).
The request-response cycle handled by a traditional API gateway usually completes within 50 to 200 milliseconds, whereas an LLM API often needs to keep the connection open for 30 to 120 seconds.
Mainstream LLM APIs all send discrete JSON payloads in the data field of SSE, ending the stream with the sentinel string data: [DONE].
17 more timeline events
All Facts (20)
LLM APIs are billed by token, with approximately 1–2 tokens per English word and approximately 1–2 tokens per Chinese character.
65%VerifiedSince 2023, API prices for mainstream LLMs have fallen by more than 90%
65%Unverified调用第三方LLM API的AI产品数据流向通常经历「用户设备→产品服务器→LLM服务商」三个节点，每个节点都存在独立的安全风险面
80%UnverifiedLLM APIs are priced separately for input and output tokens, and the per-token price does not change with the temperature and seed parameters.
50%UnverifiedLLM APIs rate-limit on two independent dimensions, requests per minute and tokens per minute; even with monthly budget to spare, an endpoint can still return 429 during traffic peaks.
50%UnverifiedTraditional cloud FinOps tools focus on infrastructure utilization (CPU/memory) and do not parse token metrics from API payloads, causing API-based AI consumption to become a blind spot.
50%UnverifiedThe correct way to track AI agent costs is: inject a correlation ID at the API gateway, propagate these IDs through the orchestration framework, and intercept all LLM API calls, emitting standard OpenTelemetry spans with token metadata.
50%UnverifiedIn LLM repair loops, the recommended typical circuit breaker parameters are: 5 consecutive failures trigger the break, a 30-60 second cooldown, and rate-limited probing during the half-open phase.
50%UnverifiedPressing all LLM traffic for critical business onto a single model API makes annual cumulative downtime reaching the hour level almost the norm rather than the exception.
50%UnverifiedMainstream LLM APIs usually rate-limit on both requests per minute (RPM) and tokens per minute (TPM).
50%UnverifiedThe request-response cycle handled by a traditional API gateway usually completes within 50 to 200 milliseconds, whereas an LLM API often needs to keep the connection open for 30 to 120 seconds.
50%UnverifiedMainstream LLM APIs all send discrete JSON payloads in the data field of SSE, ending the stream with the sentinel string data: [DONE].
50%UnverifiedLarge model API calls are stateless: each HTTP request is independent, the server retains no cross-request context, and it does not automatically concatenate the previous round's user message and assistant reply.
50%UnverifiedWhen using large model APIs, raw user input is usually spliced directly into the messages array, resulting in the simultaneous presence of three types of risk: PII leakage, prompt injection, and MCP tool poisoning.
50%UnverifiedThe LLM API splits the tokens of a request into four categories priced separately: fresh input (input), cache write, cache read, and output.
50%UnverifiedSelf-hosting n8n requires a server, costing about $5-20 per month; calling LLM APIs incurs token fees.
50%UnverifiedSaaS products based on LLM APIs have two typical forms of runaway cost: the runaway script user (scripted calls or client-side retry loops causing a single account to burn a large amount of money overnight) and the silent loss-making account (a low-paying customer sending an extremely long context, with token cost far exceeding what they pay).
50%UnverifiedMost platforms provide a certain amount of free tokens or a trial period, but the terms of the free quota are often modified in sync when prices are adjusted.
50%UnverifiedOver the past two years, LLM API prices have generally trended downward.
50%UnverifiedThe current pricing systems of mainstream LLM APIs are usually composed of several dimensions: per-token billing (distinguishing input/output), per-call billing, tiered pricing, and cache-hit discounts.
50%