Cube Studio: Tencent's Open-Source All-in-One AI Platform Covering the Full Pipeline from Large Model Training to Inference

Project Overview

In the wave of AI engineering, how to efficiently manage the entire pipeline from data labeling, model training to inference deployment has always been a core challenge for enterprises. Cube Studio, open-sourced by Tencent Music, was born to address this pain point — it is a cloud-native all-in-one machine learning/deep learning/large model AI platform that covers the complete MLOps algorithm pipeline.

MLOps (Machine Learning Operations) is a methodology that brings DevOps principles into the machine learning domain, aiming to solve the "last mile" problem of getting models from the lab to production. In traditional ML development, experiments completed by data scientists in Jupyter Notebooks are often difficult to deploy directly as production services — model version management is chaotic, training environments are non-reproducible, and deployment processes are manual and error-prone. MLOps standardizes the ML lifecycle through automated CI/CD pipelines, model version control, feature stores, model monitoring, and other mechanisms. Cloud-native architecture, with containerization (Docker), container orchestration (Kubernetes), microservices, and declarative APIs at its core, provides an elastic, portable, and observable infrastructure layer that is naturally suited for hosting MLOps workloads.

The project has currently earned nearly 5,000 Stars and 877 Forks on GitHub, developed in Python, with continuously growing community activity. From a positioning perspective, Cube Studio is not just a training platform — it's a comprehensive AI infrastructure encompassing compute resource management, development environments, task orchestration, distributed training, inference services, edge computing, and data labeling.

Notebook Online Development & Pipeline Task Orchestration

Cube Studio provides a web-based Notebook online development environment, allowing developers to write and debug code directly without configuring local environments. More importantly, the platform supports drag-and-drop Pipeline orchestration, where users can visually chain together data processing, feature engineering, model training, evaluation, and other steps into complete workflows.

The ML Pipeline concept originates from the DAG (Directed Acyclic Graph) scheduling paradigm in data engineering. Each Pipeline consists of multiple Steps with data dependencies between them, and the scheduler executes tasks according to DAG topological order. In the Kubernetes ecosystem, Argo Workflows and Tekton are two mainstream Pipeline engines that encapsulate each Step as a Pod and pass intermediate results through Artifacts. Drag-and-drop visual orchestration essentially transforms DAG definition from YAML/code into graphical interface operations, reducing the cognitive cost of understanding and using Pipelines. This pattern has been proven effective in data engineering tools like Apache Airflow, and Cube Studio introduces it into ML scenarios with deep integration with GPU resource scheduling.

This low-code orchestration approach significantly lowers the barrier to using MLOps, allowing algorithm engineers to focus on models themselves rather than infrastructure setup. Compared to the traditional approach of manually writing scheduling scripts, visual Pipeline orchestration offers qualitative improvements in collaboration efficiency and maintainability.

Multi-Node Multi-GPU Distributed Training

Distributed training capability is one of Cube Studio's core competitive advantages. The platform supports an extremely rich ecosystem of distributed training frameworks, including:

• PyTorch / TensorFlow / MXNet: Full coverage of mainstream deep learning frameworks
• DeepSpeed / ColossalAI: Large model training acceleration frameworks
• Horovod / PaddlePaddle: Cross-framework distributed training
• Ray / Volcano: Elastic scheduling and batch computing

The core challenge of multi-node multi-GPU distributed training lies in the communication overhead of gradient synchronization. Mainstream parallelism strategies include Data Parallelism, Model Parallelism (Tensor Parallelism), and Pipeline Parallelism. In data parallelism, each GPU holds a complete model replica, trains on different data shards, and synchronizes gradients through AllReduce operations; model parallelism partitions model parameters across different devices. DeepSpeed's ZeRO optimizer significantly reduces per-GPU memory usage by sharding optimizer states, gradients, and parameters across devices, making it possible to train models with tens of billions of parameters on limited hardware.

The platform also supports RDMA high-speed network interconnection, which is crucial for communication efficiency in multi-node multi-GPU training scenarios. RDMA (Remote Direct Memory Access) technology allows network cards to directly access remote host memory, bypassing the CPU and OS kernel, reducing network latency from microsecond to sub-microsecond levels, with bandwidth reaching 100-400Gbps (e.g., InfiniBand HDR/NDR) — critical for frequent gradient communication during large model training. Combined with hyperparameter search functionality, users can automatically explore optimal model configurations and significantly improve training efficiency.

Full Pipeline for Large Model Training & Fine-Tuning

In the era of large models, Cube Studio stays at the forefront of technology, providing complete large model training pipeline support:

• SFT (Supervised Fine-Tuning): Supports instruction fine-tuning for mainstream large models like DeepSeek^Ad and LLaMA
• Reward Model Training: A critical component in the RLHF pipeline
• Reinforcement Learning Training: Complete alignment training Pipeline

RLHF (Reinforcement Learning from Human Feedback) is the mainstream technical approach for large model alignment, systematically proposed by OpenAI in the InstructGPT paper. The complete RLHF process consists of three stages: the first stage is SFT (Supervised Fine-Tuning), which uses high-quality human-written instruction-response pairs to fine-tune the pretrained model, enabling it to learn basic instruction-following capabilities; the second stage trains a Reward Model by collecting human preference ranking data from multiple model responses to the same prompt, training a model that can predict human preference scores; the third stage uses reinforcement learning algorithms like PPO (Proximal Policy Optimization), with the reward model's output as the reward signal, to optimize the language model's generation strategy. While recent methods like DPO (Direct Preference Optimization) attempt to skip the reward model and optimize directly from preference data, RLHF remains the mainstream choice in industry.

This means enterprises can complete the entire process from pretrained models to customized large models on Cube Studio, without cobbling together multiple toolchains. Compared to using fine-tuning tools like LLaMA-Factory alone, Cube Studio's advantage lies in connecting fine-tuning with data management, resource scheduling, and model deployment into a closed loop.

Inference Service Deployment & VGPU Virtualization

On the model deployment side, Cube Studio supports mainstream large model inference engines including vLLM, Ollama, and MindIE, with multi-node inference support to handle large-scale concurrent request scenarios.

The core bottleneck of large model inference is KV Cache memory management. During autoregressive generation, each token generation requires accessing the Key-Value cache of all previous tokens, and KV Cache memory usage grows linearly with sequence length. The PagedAttention mechanism proposed by vLLM borrows from the paging concept in operating system virtual memory, dividing KV Cache into fixed-size Blocks and managing non-contiguous memory through Block Tables, solving the memory fragmentation and waste problems caused by pre-allocating contiguous memory in traditional implementations. This enables vLLM to achieve 2-24x throughput improvement over HuggingFace Transformers. Ollama focuses on ease of use for local deployment, providing a Docker-like model management experience. MindIE is Huawei's Ascend ecosystem inference engine, deeply optimized for the architectural characteristics of Ascend NPUs.

Particularly noteworthy is the platform's VGPU virtualization capability. Through GPU virtualization technology, multiple inference tasks can share the same physical GPU, greatly improving GPU utilization. GPU virtualization addresses the problem of low GPU resource utilization — in real production environments, many inference tasks and development/debugging scenarios don't require the full computing power and memory of an entire GPU, but traditional scheduling can only allocate in whole-card units, leaving significant compute resources idle. VGPU technology implements fine-grained GPU compute and memory partitioning by intercepting at the driver or runtime layer. NVIDIA's MPS (Multi-Process Service) and MIG (Multi-Instance GPU) are native solutions from the hardware vendor, while open-source community solutions (such as HAMi/vGPU-scheduler) achieve logical GPU partitioning and quota management at the scheduling level through Kubernetes Device Plugin mechanisms. This allows a single A100 80GB GPU to simultaneously serve multiple small inference tasks, improving GPU utilization from a typical 30% to over 70%.

For enterprises with high compute costs, this feature offers significant economic value. The platform also positions itself as a compute rental platform with comprehensive resource management and billing capabilities.

Domestic Ecosystem & Ascend NPU Adaptation

In the context of Xinchuang (information technology application innovation) and domestic substitution in China, a major highlight of Cube Studio is its comprehensive support for domestic CPUs/GPUs/NPUs, particularly deep adaptation of the Huawei Ascend ecosystem. This enables the platform to run stably in domestic hardware environments, meeting compliance requirements for government and enterprise customers.

Huawei Ascend is currently the most complete domestic AI chip ecosystem solution. Its hardware product line includes the Ascend 910 series for training and the Ascend 310 series for inference, with a supporting software stack including the underlying driver CANN (Compute Architecture for Neural Networks), the deep learning framework MindSpore, and the model development tool MindStudio. Ascend's Da Vinci architecture uses 3D Cube computing units specifically optimized for matrix operations. At the adaptation layer, PyTorch can run on Ascend hardware through the torch_npu plugin, but operator coverage and performance tuning require ongoing investment. Beyond Huawei, domestic chips from Cambricon (MLU), Hygon (DCU), Enflame (GCU), and others are also developing rapidly, though their ecosystem maturity still lags behind Ascend. Xinchuang policy requires critical industries to gradually achieve self-reliance in core technologies, creating enormous market demand for domestic AI chips and compatible platforms.

This feature is quite rare among similar open-source projects. Neither Kubeflow nor MLflow currently offers meaningful support for domestic hardware, and Cube Studio's investment in this area reflects the project team's deep understanding of domestic market needs.

Data Labeling & Edge Computing Support

The platform also integrates an automated labeling platform, incorporating the data labeling process into unified management. Combined with AI-assisted labeling capabilities, it can significantly improve labeling efficiency and quality.

Additionally, edge computing support enables trained models to be deployed to edge devices, covering scenarios like IoT, industrial quality inspection, and smart security. The end-to-end integration from data labeling to edge deployment reduces the cost of switching between different tools for enterprises.

Cloud-Native Architecture Advantages

Cube Studio is designed with cloud-native architecture, naturally possessing the following advantages:

• Elastic Scaling: Kubernetes-based resource scheduling with on-demand compute allocation
• Multi-Tenancy Isolation: Support for resource isolation and permission management across multiple teams and projects
• Observability: Comprehensive logging, monitoring, and alerting systems
• Extensibility: Modular design supporting custom component integration

The platform also provides an AI Model Marketplace and Private Knowledge Base functionality. The former facilitates internal model sharing and reuse across teams, while the latter supports RAG-based enterprise knowledge Q&A scenarios. RAG (Retrieval-Augmented Generation) is the mainstream technical approach for addressing large model "hallucination" and knowledge timeliness issues. Its core idea is to retrieve document fragments related to the user's question from an external knowledge base before generating an answer, injecting the retrieval results as context into the large model's prompt to guide the model to generate answers based on factual information. A typical RAG system includes document parsing, text chunking, embedding (vectorization), vector database storage (e.g., Milvus, Chroma), similarity retrieval, and answer generation. Enterprise private knowledge bases leverage RAG technology to enable large models to accurately answer professional questions based on internal documents without exposing training data — highly valuable in scenarios like customer service, legal affairs, and technical support.

Comparison with Kubeflow, MLflow, and Other Competitors

Compared to international open-source MLOps platforms like Kubeflow and MLflow, Cube Studio's differentiated advantages are primarily reflected in:

Comparison Dimension	Cube Studio	Kubeflow	MLflow
Large Model Support	Built-in full RLHF pipeline, multiple inference engines	Requires additional integration	Not applicable
Domestic Hardware Adaptation	Deep Ascend NPU support	Essentially unsupported	Not applicable
Feature Completeness	Full pipeline from labeling to inference	Primarily training	Primarily experiment management
Chinese Ecosystem	Documentation and community primarily in Chinese	Primarily in English	Primarily in English

The platform is suitable for AI teams in medium-to-large enterprises, particularly organizations that need to manage multiple model training tasks, pursue efficient GPU resource utilization, and have domestic compliance requirements.

Summary

As an AI infrastructure platform open-sourced by Tencent Music, Cube Studio demonstrates the engineering capabilities of Chinese enterprises in the MLOps domain. It not only covers the complete traditional machine learning pipeline but also leads in large model training, inference, and domestic hardware adaptation.

For teams that are building or upgrading their AI platforms, Cube Studio is an open-source option worth thorough evaluation. Especially in the current climate where demand for large model deployment is growing and domestic substitution is accelerating, an open-source platform that can simultaneously meet both needs carries considerable practical value.

Key Takeaways

• Cube Studio is a cloud-native all-in-one AI platform open-sourced by Tencent Music, covering the full MLOps pipeline from data labeling, model training to inference deployment
• Supports over ten distributed training frameworks including PyTorch, DeepSpeed, and ColossalAI, with RDMA high-speed networking and VGPU virtualization capabilities
• Built-in full pipeline for large model SFT fine-tuning, reward model, and reinforcement learning training, with integrated mainstream inference engines like vLLM/Ollama
• Comprehensive support for domestic CPUs/GPUs/NPUs and the Huawei Ascend ecosystem, meeting Xinchuang compliance requirements
• The project has earned nearly 5,000 GitHub Stars and offers enterprise-grade features including visual Pipeline orchestration, AI model marketplace, and private knowledge base