#LLM-as-judge
83 related articles
·3 min
From Production Incidents to Eval Cases: A Practical Methodology for LLM Regression Testing
Aggregate metrics mask LLM long-tail failures. Learn how teams convert real production incidents into regression test cases, building evolving eval systems that prevent repeated mistakes during model upgrades.
Read more →
·3 min
A Complete Guide to AI Agent Observability and Evaluation Tech Stacks
Deep dive into three technical approaches for AI Agent observability and evaluation: LangSmith native integration, open-source self-hosted solutions like LangFuse, and unified platforms like Lyzr.
Read more →
·2 min
Code Preference Dataset: Manually Reviewed Training Material for DPO/RLHF Fine-Tuning
A manually reviewed code preference dataset designed for DPO/RLHF fine-tuning, covering Python and JavaScript with multi-dimensional quality assessments including correctness bugs, security issues, and performance tradeoffs.
Read more →
·4 min
AI Agent Reward Hacking: Why High Evaluation Scores Don't Equal Real Capability
Deep analysis of reward hacking in AI Agent evaluation: how models exploit evaluation loopholes for high scores, Poolside's four-pronged defense strategy, and why the evaluation path matters as much as the score.
Read more →
·3 min
Stickblade Arena: A New Benchmark That Pits LLMs Against Each Other in a Physics-Based Combat Arena
Stickblade Arena is a physics-engine-based LLM benchmark where models battle in a 2D arena, testing spatial reasoning and dynamic decision-making while avoiding training data leakage. Its six-axis Elo system reveals fine-grained capability differences.
Read more →
·3 min
LangChain Managed DeepAgents: Hosted Agent Infrastructure So You Can Focus on Core Logic
LangChain launches Managed DeepAgents public beta, hosting evals, memory, OAuth, Slack integration, and sandbox infrastructure so developers can focus on Agent core logic.
Read more →
·2 min
Deep Dive into the LangChain Ecosystem: A Guide to LangGraph, deepagents, and LangSmith Collaboration
A deep analysis of three core LangChain ecosystem components: LangGraph stateful agent orchestration, deepagents deep agent paradigm, and LangSmith observability platform for production AI apps.
Read more →
·3 min
Kopai: Turn Your Expertise into AI Agents and Earn Passive Income from Knowledge
Kopai is a no-code AI agent platform where experts upload knowledge to publish sellable AI agents, with per-message billing and 70% revenue share for creators.
Read more →
·3 min
System Prompts Should Be Version-Controlled Like Code
System prompts drive LLM apps but often lack version control and regression testing. Learn how to manage them with versioning, structured separation, testing, and code review.
Read more →
·4 min
11-Node LangGraph in Practice: Complete Architecture Breakdown of a Compliance-Grade RAG Agent
Deep dive into an 11-node Agentic RAG agent built with LangGraph, featuring 6-way intelligent routing, hallucination guards, PII masking, circuit breakers, and zero-cost deployment.
Read more →
·3 min
From Demo to Production: The Real Challenges of Deploying LLM Systems
Deep analysis of deploying LLM systems from prototype to production: a real-world AI incident investigation assistant case study revealing key engineering challenges beyond the model.
Read more →
·3 min
From Demo to Production: The Real Challenges of Deploying LLM Systems
Deep analysis of deploying LLM systems from prototype to production: a real-world AI incident investigation assistant case revealing critical engineering challenges beyond the model.
Read more →
·4 min
Harness Engineering: The Core Infrastructure for Taking AI Agents from Demo to Production
Deep dive into Harness Engineering: why AI Agents need memory management, durable execution, guardrails & approvals to go from demo to production.
Read more →
·4 min
Harness Engineering: The Core Infrastructure for Taking AI Agents from Demo to Production
Deep dive into Harness Engineering: why AI Agents need memory management, durable execution, guardrails & approvals to reach production. Based on Scott Moss's workshop.
Read more →
·3 min
AI Not Performing Well? Start by Getting These Simple but Often Overlooked Things Right
When your AI system underperforms, the problem is often not the model or algorithm — it's basic work like data cleaning, prompt writing, and evaluation that hasn't been done right.
Read more →
·3 min
AI Not Working Well? Start by Getting These Simple but Often Overlooked Things Right
When AI systems underperform, the problem often isn't the model or algorithm — it's that basics like data cleaning, prompt writing, and evaluation aren't done right. Learn the simple fixes that matter most.
Read more →
·8 min
A Complete Guide to Agent Testing: Five Core Dimensions and Automation in Practice
In-depth analysis of the five core dimensions of AI Agent testing: command safety, tool-calling accuracy, task planning, output consistency, and error self-repair. Master automated testing and the transition path for test engineers.
Read more →
·8 min
The Complete Guide to Agent Testing: Five Core Dimensions and Automation in Practice
An in-depth analysis of the five core dimensions of AI Agent testing: command safety, tool-calling accuracy, task planning, output consistency, and error self-repair. Master automated testing methods and the transition path for test engineers.
Read more →
·2 min
Dependency Drift in Production AI Agents: Detection, Response, and Building Resilient Systems
AI Agents in production face systemic dependency drift risks — model updates silently change outputs and silent failures are hard to detect. Learn detection strategies, hidden costs, and engineering practices.
Read more →
·4 min
7 Core Components for Designing Long-Running AI Agents
A deep dive into the 7 core components for building long-running AI Agents: Goal, Evaluator, Verifier, Loop, Orchestration, Observability, and Memory.
Read more →