Dragonfly
An open-source file distribution system based on P2P technology that supports large-scale distribution and acceleration of OCI container images, AI models, and general files. The project has graduated from CNCF incubation and is widely adopted in large-scale cluster environments.
Timeline (last 90 days)
Dragonfly is a P2P-based file distribution system that can accelerate the distribution of various content including OCI images and AI models.
The Dragonfly project graduated from CNCF last year, marking the project's achievement of production maturity.
Dragonfly's community contributors come from multiple companies including Ant Group, Alibaba Cloud, and ByteDance.
The Dragonfly architecture includes components such as Manager, Scheduler, Seed Peer, and Peer; the Manager is analogous to the K8s API Server, and the Scheduler uses piece as its scheduling granularity.
Dragonfly's piece is the smallest unit of scheduling, with a default size typically around 4MB.
The dfget command in Dragonfly version 2.5 supports downloading models directly from Hugging Face and ModelScope.
The Blocklist feature in Dragonfly 2.5 can intercept tasks that consume bandwidth; GRPC calls return permission denied, and HTTP calls return 403 Forbidden.
Dragonfly 2.5 introduces Adaptive Rate Limit, which can automatically adjust rate limit values based on resource configurations such as memory and CPU.
Dragonfly 2.5 adds the dfcontrol tool, similar to kubectl, which can list node tasks, delete local cache, and pre-warm files or images to the cluster.
Dragonfly bundles AI model weights and configurations into OCI images for distribution, packaging each safetensors file as a separate layer, with OCI images being generatable using the ModelKit tool.
1 more timeline events
All Facts (11)
Dragonfly is a P2P-based file distribution system that can accelerate the distribution of various content including OCI images and AI models.
50%UnverifiedThe Dragonfly project graduated from CNCF last year, marking the project's achievement of production maturity.
50%UnverifiedDragonfly's community contributors come from multiple companies including Ant Group, Alibaba Cloud, and ByteDance.
50%UnverifiedThe Dragonfly architecture includes components such as Manager, Scheduler, Seed Peer, and Peer; the Manager is analogous to the K8s API Server, and the Scheduler uses piece as its scheduling granularity.
50%UnverifiedDragonfly's piece is the smallest unit of scheduling, with a default size typically around 4MB.
50%UnverifiedDragonfly bundles AI model weights and configurations into OCI images for distribution, packaging each safetensors file as a separate layer, with OCI images being generatable using the ModelKit tool.
50%UnverifiedDragonfly provides a Model CSI Driver solution for older clusters that cannot be upgraded to the latest K8s version
50%UnverifiedDragonfly 2.5 introduces Adaptive Rate Limit, which can automatically adjust rate limit values based on resource configurations such as memory and CPU.
50%UnverifiedDragonfly 2.5 adds the dfcontrol tool, similar to kubectl, which can list node tasks, delete local cache, and pre-warm files or images to the cluster.
50%UnverifiedThe dfget command in Dragonfly version 2.5 supports downloading models directly from Hugging Face and ModelScope.
50%UnverifiedThe Blocklist feature in Dragonfly 2.5 can intercept tasks that consume bandwidth; GRPC calls return permission denied, and HTTP calls return 403 Forbidden.
50%