#agent evaluation
1319 related articles
·4 min
AI Agent Reward Hacking: Why High Evaluation Scores Don't Equal Real Capability
Deep analysis of reward hacking in AI Agent evaluation: how models exploit evaluation loopholes for high scores, Poolside's four-pronged defense strategy, and why the evaluation path matters as much as the score.
Read more →
·3 min
MCP Session Analytics: Building Production-Grade Observability and Evaluation for AI Agents
How to build product analytics and evaluation capabilities for AI Agents at the MCP protocol layer, covering session-level tracing, tool call observability, and quality Evals.
Read more →
·2 min
Prefactor: Production-Grade Monitoring and Quality Assurance Tool for Real-Time AI Agent Evaluation
Prefactor is a production-grade monitoring tool for real-time AI Agent evaluation, using live scoring, quality drift detection, and performance visualization to solve the core problem of Agents passing offline tests but failing in production.
Read more →
·2 min
Prefactor: Production-Grade Monitoring and Quality Assurance Tool for Real-Time AI Agent Evaluation
Prefactor is a production-grade monitoring tool for real-time AI Agent evaluation, using real-time scoring, quality drift detection, and performance visualization to solve the core pain point of Agents passing offline tests but failing in production.
Read more →
·3 min
How to Choose an AI Agent Platform for Your Team: 5 Evaluation Dimensions and a Decision Framework
Choose the right AI Agent platform by evaluating model flexibility, observability, tool integration, security compliance, and total cost. A complete decision framework to help technical leaders avoid vendor lock-in.
Read more →
·3 min
How to Choose an AI Agent Platform for Your Team: 5 Evaluation Dimensions & Decision Framework
Choose an AI Agent platform by evaluating model flexibility, observability, tool integration, security compliance, and total cost. A complete decision framework to avoid vendor lock-in.
Read more →
·2 min
The New Bar for AI/LLM Roles: A Practical Guide to Agent Observability and Evaluation
Enterprise AI/LLM roles now demand engineering skills: streaming recovery, high concurrency, multi-tenancy, LLM gateways, Langfuse observability, and evaluation platforms. Master these 8 core competencies.
Read more →
·4 min
Stop Shipping Skills Blindly! A Google DeepMind Engineer's Hands-On Guide to Agent Evaluation
Google DeepMind engineer Philip reveals: almost everyone uses coding agent Skills, yet almost no one writes evals for them. A deep dive into Skill evaluation methods, 8 actionable tips, and a real Gemini API case study.
Read more →
·3 min
Agent Performance Evaluation: A Three-Layer Framework and Engineering Implementation Guide
A deep dive into the three-layer AI Agent evaluation framework — outcome, process, and system layers — covering trajectory evaluation, tool call accuracy, automated testing, and key engineering challenges.
Read more →
·3 min
How to Evaluate AI Agents in Production: Four Online Evaluation Methods Explained
Offline tests passing doesn't mean production reliability. This guide breaks down Online Evals for AI Agents: rule checks, LLM-as-a-Judge, user feedback, and human review.
Read more →
·3 min
Can MiniMax Serve as a Hermes Agent's Brain? A Complete V0.18 Evaluation
A hands-on test of MiniMax as an AI Agent brain using the Hermes V0.18 framework: fast inference, low cost, stable long tasks—but clear flaws in complex reasoning and tool calling. Learn its three strengths, three weaknesses, and best-fit workflows.
Read more →
·7 min
Agentic AI Development in Practice: Evaluation and Error Analysis Are the Real Competitive Edge
Cut through the Agentic AI hype to see the real value of agentic applications. Based on Andrew Ng's course, learn why Evals and error analysis—not framework choice—separate top developers.
Read more →
·2 min
Andrew Ng on AI Agent Development: Evaluation and Error Analysis Are the Core Competitive Advantage
Andrew Ng argues that the core gap in AI Agent development isn't model selection — it's systematic evals and error analysis. A breakdown of his methodology.
Read more →
Tutorials·3 min
Advanced LangGraph in Practice: Complete Guide to Agent Optimization, Evaluation, and Cloud Deployment
Deep dive into three advanced LangGraph topics: multi-agent architecture optimization, evaluation frameworks for non-deterministic AI systems, and cloud deployment with LangGraph Platform.
Read more →
Product Reviews·2 min
Kimi K2.6 In-Depth Review: Comprehensive Evaluation of Coding, Multi-Agent, and Frontend Development Capabilities
In-depth review of Kimi K2.6 open-source model across frontend development, multi-agent collaboration, and long-horizon tasks, covering four professional modes, 3D/SVG generation, and pricing analysis.
Read more →
Product Reviews·5 min
Nano AI Agent Hands-On Review: Real Experience and In-Depth Evaluation of Free MCP Tools
In-depth hands-on review of Nano AI agent with Guizhou travel guide and Pop Mart business analysis tasks. Free MCP tools, zero setup, 10-min reports with detailed pros/cons analysis.
Read more →
Product Reviews·4 min
Kimi OK Computer Hands-On Review: In-Depth Evaluation of AI Agent Performance Across Four Scenarios
In-depth hands-on review of Kimi's AI Agent 'OK Computer' across website building, data analysis, audio picture books, and PPT creation. Can an agent with 20+ built-in tools truly do your work?
Read more →
Tutorials·4 min
Avoiding Pitfalls in Agent Skill Maintenance: A Practical Guide to Defending Against Perplexity's "Action at a Distance" and Building Evaluation Systems
Deep dive into Perplexity's "Action at a Distance" risk in Agent Skill maintenance, covering precise fixes for three failure types, the Gotcha flywheel, and a four-layer evaluation system.
Read more →
Deep Dives·4 min
The Five-Dimensional Agent Evaluation Framework: A Complete Guide to the Must-Ask AI Product Manager Interview Question
A complete guide to the five-dimensional AI Agent evaluation framework covering task set design, process and result evaluation, and controlled experiments for AI product managers.
Read more →
Tutorials·4 min
Complete Guide to Agent Observability: Tracing, Evaluation, and Red Teaming in Practice
Deep dive into Microsoft Foundry's Agent observability solution covering multi-Agent tracing, AI quality evaluation, Red Teaming security testing, and automated prompt optimization to bridge the gap between expected and actual Agent behavior.
Read more →