[KongchangAI]
Concept集合通信 / Collective Communication

Collective Communication Primitive

A fundamental operation for cooperative data transfer among devices in distributed training. Common examples include broadcast, all-reduce, all-gather, reduce-scatter, and all-to-all, which typically assume that the data layouts of participating devices are symmetric or homogeneous.

Timeline (last 90 days)

Nov 6

Conventional collective communication primitives (such as all-gather and all-to-all) assume that the data layouts of participating devices are symmetric or homogeneous, whereas in weight refit scenarios the source and target may differ in sharding dimensions, device counts and mapping relationships.

Unverified50%

All Facts (1)

Source Articles