Function Calling

Function Calling (Tool Use) is a standardized mechanism for large models to invoke external tools, where developers pre-declare tool names, descriptions, and parameter schemas, and the model autonomously determines when to make a call.

The classic paradigm of Agent workflows is the separation of the planning layer and the execution layer, where the planning layer is responsible for understanding goals, decomposing tasks, and generating execution plans, while the execution layer handles actual operations via dedicated tools or smaller models.

The core mechanism of Function Calling is that developers describe external function signatures in JSON Schema format, and the model outputs structured JSON invocation instructions that conform to the schema specification.

Constrained decoding and function calling can restrict the selectable tokens during the decoding stage, but they still belong to the generation paradigm and output text rather than a directly usable probability distribution.

An agent's action capability relies on the reasoning and function calling mechanisms of large language models.

The tool-call requests output by the model and its natural-language replies are two independent generation streams that do not share a strong consistency constraint, so "saying one thing and doing another" is architecturally possible.

Function calling makes the model emit a structured call that explicitly names the tool and parameters, and the system can validate and reliably execute that structure.

Small models may have incomplete or poor support for function calling, and enum value deviations can still occur.

Interface drift refers to changes after deployment in tool description text, parameter schemas, or the candidate tool list that cause model function calls to fail, even when the underlying semantics are unchanged.

The second step is letting the model call tools, starting with three common tool chains: retrieval, function calling, and code execution or interface calling.

Function calling is the mechanism by which a model proactively issues a structured request during a conversation, which an external system executes and returns the result of; Google Search grounding is essentially a platform-built-in form of function calling.

The function calling approach depends on the provider's implementation, and behavior may differ across platforms.

Common implementations of structured output include providing a function calling interface, constrained decoding, or post-processing parsing plus validation.

Contract drift in tool calls (such as a tool changing from returning plain text to returning JSON, or a schema adding a required field) usually does not directly produce an error pointing to the root cause, but instead manifests as the model repeatedly failing to parse, outputting garbled text, or retrying.

The essence of agentic workflows is a sequential reasoning loop: make a plan, choose tools, execute calls, validate outputs, and retry on failure, which naturally increases latency.

OpenAI's official guidance draws three lines: the Decisions API for classification, routing, scoring, and gating that require a determination; Structured Outputs (Responses API) for scenarios that require extracting fields or written explanations in a structured form; and function calling for scenarios where the model triggers tools with parameters.

A tool-calling/function-calling architecture that separates reasoning from computation can circumvent persuasion-based failures and is a general strategy for addressing hallucination and consistency issues

OpenAI's function calling outputs invocation parameters in structured JSON format, greatly reducing parsing error rates.

Function calling allows a model to recognize user intent and output structured function call instructions (including function names and parameters) rather than pure natural language text.