Agent Reliability
An open-source Python SDK focused on measuring AI agent reliability, adapting SRE (Site Reliability Engineering) reliability measurement methods to agent evaluation scenarios, with support for SLO assertions, error budget tracking, and CI integration.
Timeline (last 90 days)
Agent Reliability is an open-source Python SDK focused on measuring the reliability of AI Agents.
The core idea of Agent Reliability is to migrate SRE (Site Reliability Engineering) reliability measurement methods into AI Agent evaluation.
Agent Reliability introduces three explicit outcome states: PASS, FAIL, and UNKNOWN.
In Agent Reliability, the UNKNOWN state does not artificially lower the reliability score.
Agent Reliability strictly distinguishes between the evaluator's own execution failures and Agent failures, and tracks measurement health and Agent reliability separately.
Agent Reliability introduces SLO and error-budget semantics, and provides SLO assertions that can be used in tests/CI.
Agent Reliability adopts a local-first approach, requiring no account registration, API keys, or hosted services.
Agent Reliability's base package has zero mandatory runtime dependencies and includes built-in deterministic evaluators.
Agent Reliability provides optional OpenTelemetry interoperability.
Agent Reliability can be installed via pip install agent-reliability and has been published to PyPI.
1 more timeline events
All Facts (10)
Agent Reliability is an open-source Python SDK focused on measuring the reliability of AI Agents.
50%UnverifiedThe core idea of Agent Reliability is to migrate SRE (Site Reliability Engineering) reliability measurement methods into AI Agent evaluation.
50%UnverifiedAgent Reliability introduces three explicit outcome states: PASS, FAIL, and UNKNOWN.
50%UnverifiedIn Agent Reliability, the UNKNOWN state does not artificially lower the reliability score.
50%UnverifiedAgent Reliability strictly distinguishes between the evaluator's own execution failures and Agent failures, and tracks measurement health and Agent reliability separately.
50%UnverifiedAgent Reliability introduces SLO and error-budget semantics, and provides SLO assertions that can be used in tests/CI.
50%UnverifiedAgent Reliability adopts a local-first approach, requiring no account registration, API keys, or hosted services.
50%UnverifiedAgent Reliability's base package has zero mandatory runtime dependencies and includes built-in deterministic evaluators.
50%UnverifiedAgent Reliability provides optional OpenTelemetry interoperability.
50%UnverifiedAgent Reliability can be installed via pip install agent-reliability and has been published to PyPI.
50%