GPT-OSS-120b Returning Empty Responses? A Troubleshooting Guide for Tool Calling and Reasoning Content Retrieval

Issue: Model Returns Empty Content and Tool Calls Fail

A developer hosted the gpt-oss-120b model on Azure and called it through AzureChatOpenAI. The model was bound to two tools (with simple parameters, at most two keys), but when called, the model sometimes returned responses with neither content nor tool_calls—i.e., completely empty responses. The same prompts worked fine on other models (such as stronger models like gpt-54). In addition, the developer wanted to retrieve reasoning_content from the AIMessage object but found that the attribute was not present.

This problem is not uncommon in open-source model deployments, especially when the model is large and has strong reasoning capabilities, as its behavior may differ from commercial closed-source models. Below we analyze the possible causes from several angles and provide troubleshooting directions.

Why Do Empty Responses Occur?

1. The model's output format is incompatible with the bound tools

As an open-source large model, gpt-oss-120b's implementation of function calling may depend on a specific prompt template or output format. If the AzureChatOpenAI wrapper is not fully adapted to the model's tool calling protocol, the model may generate output that does not match the parsing expectations, causing the client to parse it as empty. For example, the model may return the intent to call a tool, but the format is not OpenAI-standard JSON, so it is discarded by the parser.

The OpenAI-standard function calling protocol requires the model to return a structured JSON object, typically contained in the tool_calls field, with each call having id, type, function.name, and function.arguments. Commercial closed-source models have undergone extensive alignment training and can stably output this format; open-source models, however, are often exposed to diverse templates during training. If their specified chat template (e.g., special tokens like <|python_tag|> or <|tool_call|> markers) is not strictly applied during deployment, the generated text may deviate from the standard structure. LangChain's parser only recognizes standard fields; once the format does not match, it silently discards them, resulting in both content and tool_calls being empty. Therefore, checking the recommended prompt template and tool calling examples in the model card is the first step to determine compatibility.

2. Tool definitions are too simple, confusing the model

The developer mentioned that tool parameters have at most two keys, which seems simple, but it may be precisely this simplicity that makes it difficult for the model to decide when to call the tool. Some open-source models are highly sensitive to tool descriptions; if the tool description is not clear enough, the model may choose not to call the tool and not to generate content, but instead return empty. It is recommended to check whether the tool definition's description is clear, whether it explicitly states the tool's purpose and when to call it.

3. Interference from reasoning content (reasoning_content)

gpt-oss-120b may be a model that supports reasoning content output (similar to the o1 series), which first generates a reasoning process internally and then generates the final reply. If the reasoning process consumes a large number of tokens and max_tokens is set too small, the final reply may be truncated to empty. In addition, the reasoning content may be encapsulated in a specific field of the response, and AzureChatOpenAI's AIMessage object does not expose that field by default.

reasoning_content is not a standard field in the official OpenAI Chat Completions specification, but rather chain-of-thought text additionally returned in responses by some reasoning models (such as DeepSeek^Ad-R1 and some open-source reasoning models). Field naming is not uniform across vendors; common ones include reasoning_content, reasoning, thinking, etc. OpenAI's o-series models count reasoning_tokens in usage and hide the specific reasoning text. LangChain's AIMessage only maps standard fields; unknown fields are usually placed in additional_kwargs or simply discarded, so directly accessing message.reasoning_content will throw an attribute error. To get that content, you need to extract it from the raw response JSON or additional_kwargs, rather than relying on attribute access.

4. Model deployment configuration issues

When hosting open-source models on Azure, the default values of inference parameters (such as temperature, top_p, max_tokens) may not match the model's expectations. For example, too high a temperature may cause unstable output, while too low a max_tokens may directly truncate the response. It is recommended to check the parameter configuration during deployment and compare it with the recommended values in the model documentation.

When hosting open-source models on Azure AI Foundry or Azure OpenAI-compatible endpoints, what actually takes effect are the inference parameters in the deployment configuration, not the model's built-in defaults. max_tokens is especially critical in reasoning model scenarios: it limits the sum of reasoning tokens and final reply tokens. If the reasoning process consumes a lot, the final content may be truncated to empty. In addition, some hosting services use vLLM or TGI as the inference backend for open-source models, whose defaults for temperature, top_p, repetition_penalty, etc. may differ from the model documentation's recommended values, and improper stop sequence configuration can also terminate generation prematurely. During troubleshooting, these parameters should be set explicitly, and the actual consumption of prompt_tokens and completion_tokens should be compared in endpoint logs or response usage.

How to Retrieve reasoning_content?

The developer mentioned being unable to find the reasoning_content attribute in the AIMessage object. This is usually because LangChain's AzureChatOpenAI wrapper has not yet adapted to the model's reasoning content field. You can try the following methods:

• Check the raw JSON of the response object: print the dict or JSON representation of the response in the call chain to see whether there is a reasoning_content or similar field (such as reasoning, thinking, etc.).
• Use the model's raw API: bypass the LangChain wrapper and directly call Azure's inference endpoint to view the complete returned structure.
• Custom callback or parser: if the raw response contains reasoning content, you can write custom parsing logic to extract it and attach it to the AIMessage's additional_kwargs.

Troubleshooting Steps and Recommendations

1. Simplify reproduction: Test with the simplest prompt and a single tool to confirm whether the empty response occurs consistently.
2. Check the raw response: Print the raw JSON returned by the model in the code to confirm whether there is really no content or whether information was lost during parsing.
3. Adjust max_tokens: Appropriately increase max_tokens to rule out truncation as the cause of empty responses.
4. Verify tool descriptions: Ensure the tool descriptions are clear, unambiguous, and explicitly state the meaning of parameters.
5. Consult the model documentation: Confirm whether gpt-oss-120b supports tool calling and its recommended invocation method.
6. Compare with other models: As the developer said, other models work fine, indicating the problem may lie in the model adaptation layer; you can report it to the Azure or LangChain community.

Conclusion

gpt-oss-120b returning empty responses and failing tool calls is likely due to a mismatch between the model's output format and client parsing, or the reasoning content field not being properly exposed. By checking the raw response, adjusting parameters, optimizing tool definitions, and trying to call the API directly, developers can gradually pinpoint the issue. For reasoning_content, custom parsing or waiting for framework updates may currently be necessary. Deploying open-source models often requires more debugging work, but once adapted, their flexibility and cost advantages are still worth the investment.

One-line Summary

gpt-oss-120b empty responses and tool call failures stem from format parsing mismatches; check the raw response.

Paragraph Summary

The article analyzes two types of issues that occur when hosting gpt-oss-120b on Azure and calling it through AzureChatOpenAI: the model sometimes returns empty responses with neither content nor tool_calls, and reasoning_content cannot be retrieved from AIMessage. The author provides causes from four angles: the open-source model's tool calling format is incompatible with LangChain's parser, tool definitions are too simple and confuse the model, reasoning content consumes tokens or the field is not exposed, and inference parameters on the deployment side are improperly configured. For reasoning_content, it is recommended to check the raw JSON, bypass the wrapper and call the API directly, or write a custom parser. Finally, six troubleshooting recommendations are given, including simplifying reproduction, printing the raw response, adjusting max_tokens, optimizing tool descriptions, consulting the model documentation, and reporting to the community.