Flame
An open-source distributed training engine that supports splitting the training loop into independent, scalable Python services, offering features such as hot executors, elastic rollout and incremental weight synchronization for efficient training of large language model agents.
Timeline (last 90 days)
There is a trade-off between the complexity introduced by incremental synchronization and relaxing synchronization constraints in exchange for higher GPU utilization.
The open-source distributed engine Flame has added a Search-R1-style training example for training agents that can search.
Flame abstracts the training loop into multiple independently scalable Python services, where the vLLM executor generates trajectories, PyTorch trainer replicas use group relative advantages to update the policy, and the object cache handles the transfer of batches and weight updates.
Hot executors keep the model, retrieval index, and optimizer state resident in memory across invocations, avoiding repeated initialization overhead.
Elastic rollout allows generation capacity to grow elastically with pending tasks, temporarily adding executors when the queue backs up.
Flame trainers use gang scheduling, requiring a group of trainers to obtain their minimum resource quota simultaneously, avoiding partial-scheduling deadlock and ensuring collective communication synchronization.
Incremental weight synchronization pulls only the bf16 values that have changed after the first full export.
In the measured 100-step training run, the steady-state weight publication per step was only about 165MB, while the full model size is 6.17GB.
With a configuration of 14 rollout GPUs plus 2 trainer GPUs, after 100 training steps, exact match on the held-out set rose from 14.5% to 47%.
After the first two steps, the average time per step was 24.3 seconds.
4 more timeline events
All Facts (14)
There is a trade-off between the complexity introduced by incremental synchronization and relaxing synchronization constraints in exchange for higher GPU utilization.
50%UnverifiedThe synchronous training loop still leaves substantial GPU idle time.
50%UnverifiedFlame's approach is to compress synchronization cost under the synchronous loop by replacing full broadcast with incremental transfer.
50%UnverifiedFlame abstracts the training loop into multiple independently scalable Python services, where the vLLM executor generates trajectories, PyTorch trainer replicas use group relative advantages to update the policy, and the object cache handles the transfer of batches and weight updates.
50%UnverifiedHot executors keep the model, retrieval index, and optimizer state resident in memory across invocations, avoiding repeated initialization overhead.
50%UnverifiedElastic rollout allows generation capacity to grow elastically with pending tasks, temporarily adding executors when the queue backs up.
50%UnverifiedFlame trainers use gang scheduling, requiring a group of trainers to obtain their minimum resource quota simultaneously, avoiding partial-scheduling deadlock and ensuring collective communication synchronization.
50%UnverifiedIncremental weight synchronization pulls only the bf16 values that have changed after the first full export.
50%UnverifiedIn the measured 100-step training run, the steady-state weight publication per step was only about 165MB, while the full model size is 6.17GB.
50%UnverifiedWith a configuration of 14 rollout GPUs plus 2 trainer GPUs, after 100 training steps, exact match on the held-out set rose from 14.5% to 47%.
50%UnverifiedAfter the first two steps, the average time per step was 24.3 seconds.
50%UnverifiedThe experiment used only a single random seed, the evaluation set contained only 200 questions, and the corpus was built from the context paragraphs of the selected questions, so the scores cannot be directly compared with published Search-R1 benchmark results.
50%UnverifiedFlame (2012) was an oversized espionage platform of up to 20MB, capable of audio recording, screenshot capture, and Bluetooth device scanning.
50%UnverifiedThe open-source distributed engine Flame has added a Search-R1-style training example for training agents that can search.
50%