traces
The complete records left during an agent's execution, including reasoning steps, tool-call sequences, intermediate outputs and final results. They can be placed under version control to enable diffing, rollback and forked reuse.
Timeline (last 90 days)
The two necessary prerequisites for a recursive loop are that the Agent runs in code-mode and that traces are version-controlled
The essential difference between versioned traces and ordinary logs lies in structure and reproducibility: logs are for after-the-fact troubleshooting, while versioned traces are for proactive reuse and recursive continuation
traces refer to the complete records left during an Agent's execution, including reasoning steps, tool call sequences, intermediate outputs, and final results; putting them under version control means differences can be diffed, intermediate states rolled back, and different strategies tried at fork points
Evals and traces are complementary: evals answer whether overall quality meets the bar, traces answer at which step a given failure occurred, and feeding typical failure cases from traces back into the evals dataset can form a continuous improvement loop.
When model call volume is high, exporting all traces puts pressure on the backend, so you can configure a sampling rate at the TracerProvider layer or do targeted sampling of high-value requests.
Trace capture (traces) is borrowed from observability monitoring practices in distributed systems and is used to reconstruct the complete execution path of each AI request.
LangSmith traces are structured logs of LLM application runs, recording the inputs, outputs, intermediate steps, latency, token consumption, and other information in each call chain.
Fine-tuning with real trace data more closely reflects the actual usage scenarios of the application compared to manually constructed datasets, and can reduce the performance gap of the fine-tuned model in production.
All Facts (13)
Agents produce execution traces including each step's reasoning process, tool calls, intermediate results, and final outputs
90%UnverifiedTraces are a core concept in distributed system observability, originating from standards like OpenTelemetry
90%UnverifiedEach operation node in a trace is called a span, and spans form a tree structure through parent-child relationships
90%UnverifiedIn the Agent context, a trace records the complete call chain from user input to final output, including LLM inference calls, function/tool calls, retrieval operations, and intermediate state changes
85%UnverifiedThe ability to transform execution traces or raw data into Harbor evaluation tasks is a key skill in Agent evaluation
75%UnverifiedThe essential difference between versioned traces and ordinary logs lies in structure and reproducibility: logs are for after-the-fact troubleshooting, while versioned traces are for proactive reuse and recursive continuation
50%UnverifiedThe two necessary prerequisites for a recursive loop are that the Agent runs in code-mode and that traces are version-controlled
50%Unverifiedtraces refer to the complete records left during an Agent's execution, including reasoning steps, tool call sequences, intermediate outputs, and final results; putting them under version control means differences can be diffed, intermediate states rolled back, and different strategies tried at fork points
50%UnverifiedEvals and traces are complementary: evals answer whether overall quality meets the bar, traces answer at which step a given failure occurred, and feeding typical failure cases from traces back into the evals dataset can form a continuous improvement loop.
50%UnverifiedWhen model call volume is high, exporting all traces puts pressure on the backend, so you can configure a sampling rate at the TracerProvider layer or do targeted sampling of high-value requests.
50%UnverifiedTrace capture (traces) is borrowed from observability monitoring practices in distributed systems and is used to reconstruct the complete execution path of each AI request.
50%UnverifiedLangSmith traces are structured logs of LLM application runs, recording the inputs, outputs, intermediate steps, latency, token consumption, and other information in each call chain.
50%UnverifiedFine-tuning with real trace data more closely reflects the actual usage scenarios of the application compared to manually constructed datasets, and can reduce the performance gap of the fine-tuned model in production.
50%