[KongchangAI]
Product

Together Compiler

A compiler system that uses AI agents to continuously maintain inference tuning state in production environments, automatically adjusting inference configurations through a closed loop of perception, decision-making, execution, and verification.

Core Facts

Timeline (last 90 days)

Nov 5

The "up to 2.6 times" figure is a result under a specific model and a specific baseline, and actual gains will vary with model architecture, traffic patterns and hardware environment.

Unverified50%
Nov 5

For inference scenarios that require long-term online serving, continuous compilation capability may be more important than single-run peak performance.

Unverified50%
Nov 5

Together Compiler limits the optimization target to inference forward computation, focusing on the execution efficiency of the computation graph during serving rather than the training or fine-tuning process.

Unverified50%
Nov 5

Together Compiler can continuously re-optimize the inference forward computation as traffic patterns change.

Unverified50%
Nov 5

If the InferenceX configuration has already been highly optimized for the target workload, the room for improvement from Together Compiler will shrink; conversely, in scenarios that are not fully tuned, the gains from continuous compilation may be more significant.

Unverified50%
Nov 5

Comparing Together Compiler with the best InferenceX configuration is in essence competing against a strong baseline that has been finely hand-tuned, rather than comparing against default parameters.

Unverified50%
Nov 5

The core idea of Together Compiler is to turn the compilation process into a closed loop: the agent continuously observes inference performance, generates and validates optimization plans targeting the bottleneck of the current forward computation, and then retains the effective changes.

Unverified50%
Nov 5

On a single model, the TPS that Together Compiler reached in under 24 hours was up to 2.6 times that of the best InferenceX configuration.

Unverified50%
Nov 4

The 3.7x throughput improvement comes from the official description and is a peak figure under ideal conditions; actual gains vary with model type, hardware configuration and the magnitude of traffic fluctuations.

Unverified50%
Nov 4

Together Compiler claims it can deliver up to a 3.7x throughput improvement when traffic changes.

Unverified50%

12 more timeline events

All Facts (20)

Partially Verified

Together Compiler uses AI agents to automate inference tuning into the production environment, keeping production inference continuously in a tuned state.

75%
Unverified

Together Compiler claims it can complete one round of optimization within 24 hours.

70%
Unverified

Together Compiler can continuously re-optimize the inference forward computation as traffic patterns change.

50%
Unverified

On a single model, the TPS that Together Compiler reached in under 24 hours was up to 2.6 times that of the best InferenceX configuration.

50%
Unverified

The core idea of Together Compiler is to turn the compilation process into a closed loop: the agent continuously observes inference performance, generates and validates optimization plans targeting the bottleneck of the current forward computation, and then retains the effective changes.

50%
Unverified

Comparing Together Compiler with the best InferenceX configuration is in essence competing against a strong baseline that has been finely hand-tuned, rather than comparing against default parameters.

50%
Unverified

The "up to 2.6 times" figure is a result under a specific model and a specific baseline, and actual gains will vary with model architecture, traffic patterns and hardware environment.

50%
Unverified

If the InferenceX configuration has already been highly optimized for the target workload, the room for improvement from Together Compiler will shrink; conversely, in scenarios that are not fully tuned, the gains from continuous compilation may be more significant.

50%
Unverified

For inference scenarios that require long-term online serving, continuous compilation capability may be more important than single-run peak performance.

50%
Unverified

Together Compiler limits the optimization target to inference forward computation, focusing on the execution efficiency of the computation graph during serving rather than the training or fine-tuning process.

50%
Unverified

Together Compiler is mainly aimed at inference optimization for open-source models, and its applicability is limited for teams using closed-source APIs.

50%
Unverified

Together Compiler claims it can deliver up to a 3.7x throughput improvement when traffic changes.

50%
Unverified

Together Compiler's agents are a set of automated programs that can perceive compilation feedback, call tools and iterate decisions, rather than a single large model.

50%
Unverified

The persistence engine saves the state of compilation artifacts, including intermediate representations, configurations that have been tried, measured performance data for each configuration and hardware environment fingerprints, enabling incremental compilation rather than starting over from scratch.

50%
Unverified

The 3.7x throughput improvement comes from the official description and is a peak figure under ideal conditions; actual gains vary with model type, hardware configuration and the magnitude of traffic fluctuations.

50%
Unverified

When evaluating the 3.7x throughput improvement, one needs to check whether the baseline was tuned to the same degree, the batch size and sequence length range tested, whether compilation and re-tuning time are included, and whether it is average throughput or a P99 latency metric.

50%
Unverified

Within the 1.3 to 3 times throughput improvement range, the lower bound represents steady gains under normal conditions, while the upper bound appears in scenarios where load characteristics shift and statically compiled schemes are clearly mismatched.

50%
Unverified

Together Compiler claims it can deliver a 1.3 to 3 times throughput improvement.

50%
Unverified

The combination of Polymorphic Inference and Together Compiler points toward a 'tuning as a service' capability, turning inference optimization that depends on senior engineers into a built-in continuous self-adaptation mechanism of the system.

50%
Unverified

Together launched Together Compiler, a tool that turns compilation into a continuously running agent system and uses a persistent engine to continuously re-optimize open models as traffic changes.

50%

Source Articles