InferenceX
An AI inference benchmarking suite developed by Semi-Analysis. Its core metric is the latency–throughput trade-off curve, which simulates real-world business scenarios to measure the actual inference efficiency of systems operating under quality-of-service constraints.
Timeline (last 90 days)
In the InferenceX public benchmark, Jalapeño achieved 1.5 to 1.9 times the tasks per watt at peak throughput compared to NVIDIA's reference system, with end-to-end latency reduced by 1.7 to 3.6 times.
OpenAI uses an open-source benchmarking tool called InferenceX to test Jalapeño's performance.
OpenAI used Semi-Analysis's public benchmark InferenceX to test Jalapeño on three open models: GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2 1T.
The core metric of InferenceX is the latency-throughput trade-off curve, which measures the number of requests a system can handle and its energy consumption under the constraint of a set maximum acceptable response latency for users.
The Jalapeño benchmark published by OpenAI is labeled only as 8K 1K (8000 input tokens, 1000 output tokens), sourced from the open-source test suite InferenceX, with no publicly disclosed test data for agentic scenarios such as AgentX.
All Facts (5)
In the InferenceX public benchmark, Jalapeño achieved 1.5 to 1.9 times the tasks per watt at peak throughput compared to NVIDIA's reference system, with end-to-end latency reduced by 1.7 to 3.6 times.
50%UnverifiedOpenAI uses an open-source benchmarking tool called InferenceX to test Jalapeño's performance.
50%UnverifiedOpenAI used Semi-Analysis's public benchmark InferenceX to test Jalapeño on three open models: GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2 1T.
50%UnverifiedThe core metric of InferenceX is the latency-throughput trade-off curve, which measures the number of requests a system can handle and its energy consumption under the constraint of a set maximum acceptable response latency for users.
50%UnverifiedThe Jalapeño benchmark published by OpenAI is labeled only as 8K 1K (8000 input tokens, 1000 output tokens), sourced from the open-source test suite InferenceX, with no publicly disclosed test data for agentic scenarios such as AgentX.
50%