Cube Studio Deep Dive: Tencent's Open-Source One-Stop MLOps Platform

Overview

In the wave of AI engineering implementation, efficiently managing the entire workflow from data labeling, model training to inference deployment has been a core challenge for enterprises. Cube Studio, open-sourced by Tencent Music, was built to address this pain point — it is a cloud-native one-stop machine learning/deep learning/large model AI platform that covers the entire MLOps pipeline, with nearly 5,000 Stars on GitHub.

This article provides a comprehensive analysis of Cube Studio from the perspectives of architecture design, core features, technology ecosystem, and applicable scenarios.

Cube Studio's Core Positioning: Full MLOps Lifecycle Coverage

Cube Studio's core value lies in integrating all stages of AI development into a unified platform, eliminating the fragmentation of toolchains.

MLOps (Machine Learning Operations) is a methodology that applies DevOps principles to the machine learning domain. In traditional AI development, data scientists use Jupyter Notebooks for experiments, engineers use different tools for deployment, and operations teams have their own monitoring systems. This toolchain fragmentation often extends the cycle from experimentation to production to several months. According to Gartner, over 85% of AI projects never make it to production, largely due to the lack of a unified engineering platform. The core goal of an MLOps platform is to bridge these gaps and enable continuous integration, continuous delivery, and continuous monitoring (CI/CD/CM) for models.

Cube Studio covers the following core layers:

• Data Layer: Automated labeling platform to reduce data preparation costs
• Development Layer: Online Notebook development environment, ready to use out of the box
• Training Layer: Distributed training, hyperparameter search, Pipeline orchestration
• Deployment Layer: Inference services, VGPU virtualization, edge computing
• Application Layer: AI model marketplace, private knowledge bases

This end-to-end design philosophy eliminates the need for teams to switch between multiple tools, significantly improving AI project delivery efficiency.

Deep Dive into Cube Studio's Key Features

Drag-and-Drop Pipeline Orchestration

Cube Studio provides visual task flow Pipeline orchestration capabilities, allowing users to build complex algorithm pipelines through drag-and-drop operations.

Pipeline orchestration is one of the core capabilities of an MLOps platform. Its essence is abstracting machine learning workflows into Directed Acyclic Graphs (DAGs). In the cloud-native ecosystem, Argo Workflows and Kubeflow Pipelines are two mainstream open-source pipeline engines that implement task scheduling based on Kubernetes CRD (Custom Resource Definition) mechanisms. The value of visual drag-and-drop orchestration is that it transforms the complex process definitions — which originally required writing YAML or Python DSL — into graphical operations, enabling algorithm engineers without infrastructure backgrounds to independently build complete pipelines from data preprocessing, feature engineering, and model training to model evaluation.

This low-code design significantly lowers the barrier to using MLOps, allowing algorithm engineers to focus on the models themselves rather than infrastructure setup.

Multi-Node Multi-GPU Distributed Training

In the era of large models, distributed training capability is a core competitive advantage for AI platforms. Cube Studio offers extremely comprehensive support in this area, compatible with mainstream distributed training frameworks:

• PyTorch / TensorFlow / MXNet: Classic deep learning frameworks
• DeepSpeed / ColossalAI / Horovod: Large model training acceleration frameworks
• PaddlePaddle: Baidu's PaddlePaddle ecosystem
• Ray / Volcano: Distributed computing and batch scheduling

The core challenge of distributed training is how to efficiently distribute computation tasks across multiple GPUs while maintaining gradient synchronization. Current mainstream parallelism strategies include Data Parallelism, Model Parallelism, Pipeline Parallelism, and Tensor Parallelism. DeepSpeed, developed by Microsoft, uses ZeRO (Zero Redundancy Optimizer) technology to partition optimizer states, gradients, and parameters, making it possible to train extremely large models with limited GPU memory. ColossalAI, developed by Singapore's HPC-AI Tech team, offers more flexible multi-dimensional parallelism strategy combinations. Horovod, originally developed by Uber, uses the Ring-AllReduce algorithm for efficient gradient aggregation. These frameworks each have their own strengths, and Cube Studio's comprehensive support for all of them means users can choose the optimal solution based on model scale and cluster configuration.

It also supports RDMA high-speed network interconnection, which is critical for communication efficiency in multi-node multi-GPU scenarios and directly impacts training scalability and speed. RDMA (Remote Direct Memory Access) is a technology that allows computers in a network to directly access each other's memory without operating system kernel involvement, achieving ultra-low latency and ultra-high bandwidth data transfer. In multi-node multi-GPU training scenarios, GPUs need to frequently exchange gradient data, and traditional TCP/IP network stacks introduce significant CPU overhead and latency. RDMA technology (including two main implementations: InfiniBand and RoCE) can reduce inter-node communication latency from microseconds to sub-microseconds, with bandwidth reaching 200Gbps or even 400Gbps. For training models with hundreds of billions of parameters, RDMA networking is virtually mandatory — otherwise, communication bottlenecks can cause GPU utilization to drop dramatically, potentially reducing training efficiency by over 50%.

Large Model Training and Fine-Tuning Capabilities

Cube Studio closely follows large model development trends and provides complete large model training pipeline support:

• SFT Fine-Tuning: Supports supervised fine-tuning of mainstream large models like DeepSeek
• Reward Model Training: A critical component in the RLHF process
• Reinforcement Learning Training: Complete alignment training support

RLHF (Reinforcement Learning from Human Feedback) is the mainstream technical approach for current large model alignment, systematically proposed by OpenAI in the InstructGPT paper. The complete RLHF process consists of three stages: the first stage is SFT (Supervised Fine-Tuning), using high-quality human-annotated instruction-response pairs for supervised fine-tuning; the second stage is training a Reward Model, where human annotators rank multiple model responses to train a scoring model that can predict human preferences; the third stage uses reinforcement learning algorithms such as PPO (Proximal Policy Optimization) to optimize the language model's policy using the reward model's scores as signals. These three stages are interconnected, and Cube Studio's support for the full process means enterprises can completely reproduce ChatGPT-level training paradigms.

This means enterprises can complete the entire training workflow from pre-training to alignment on Cube Studio based on open-source large models.

Large Model Inference Service Deployment

On the inference side, Cube Studio integrates the most popular inference engines:

• vLLM: High-throughput LLM inference engine
• Ollama: Lightweight local large model runtime solution
• MindIE (Ascend): Inference engine for Huawei's Ascend ecosystem

vLLM is a high-performance LLM inference engine developed by the UC Berkeley team, with PagedAttention as its core innovation. In traditional LLM inference, KV Cache memory management is extremely inefficient — since sequence lengths are uncertain, systems typically pre-allocate continuous memory space for the maximum length, resulting in 60%-80% of GPU memory being wasted. PagedAttention borrows the paging concept from operating system virtual memory, dividing KV Cache into fixed-size blocks that are allocated and released on demand, bringing memory utilization close to the theoretical optimum. This allows vLLM to achieve 14-24x the throughput of HuggingFace Transformers on the same hardware. Additionally, vLLM supports Continuous Batching, which dynamically inserts new requests into batches being processed, further improving GPU utilization.

It supports multi-node inference deployment, and combined with VGPU virtualization technology, enables fine-grained management and sharing of GPU resources, significantly improving hardware utilization. VGPU (Virtual GPU) virtualization technology allows a single physical GPU to be split into multiple virtual GPU instances, each with independent memory and compute quotas. In AI platform scenarios, VGPU addresses the core problem of GPU resource fragmentation — many inference tasks or development/debugging tasks don't require the full compute power of an entire GPU, but Kubernetes' native GPU scheduling only supports whole-card allocation. Through VGPU technology (such as NVIDIA MIG, third-party solutions like HAMi, etc.), a single A100 can be split into multiple instances assigned to different users, increasing GPU utilization from an average of 30% to over 70%. Cube Studio's integration of VGPU capabilities means platform administrators can allocate GPU resources at finer granularity (such as 0.5 cards or 0.25 cards), which has direct economic value for reducing enterprise AI infrastructure costs.

Compute Resource Leasing and Management Platform

Notably, Cube Studio also includes built-in compute resource leasing platform capabilities. This means it can serve not only as an internal enterprise AI platform but also as the underlying platform for compute service providers to offer GPU/NPU compute leasing services externally, with potential for commercial operations.

Domestic Ecosystem Adaptation

Cube Studio demonstrates outstanding performance in domestic adaptation, with explicit support for:

• Domestic CPUs/GPUs: Adapted to the domestic chip ecosystem
• Huawei Ascend NPU: Deep integration with the Ascend computing ecosystem
• MindIE Inference Engine: Ascend-native inference solution

Xinchuang (Information Technology Application Innovation) is China's national strategy to promote autonomous control of critical information infrastructure, covering the full stack from chips, operating systems, databases, and middleware to application software. In the AI chip domain, Huawei's Ascend series is currently the most ecosystem-complete domestic AI accelerator solution, with its Atlas 900 cluster's compute power comparable to NVIDIA A100 clusters. The Ascend ecosystem includes the CANN (Compute Architecture for Neural Networks) low-level operator library, MindSpore training framework, and MindIE inference engine. Under the current international landscape, many AI projects in finance, telecommunications, and government sectors have already made domestic adaptation a mandatory requirement in their procurement specifications. Cube Studio's deep integration with the Ascend ecosystem enables it to directly participate in these project bids — an advantage that many purely NVIDIA-based open-source platforms lack.

In the current context of Xinchuang and autonomous control, this feature is quite important for government and enterprise customers. For many enterprises, domestic adaptation capability has already become a hard requirement when selecting AI platforms.

Cloud-Native Architecture Advantages

Cube Studio is designed on a cloud-native architecture, which brings several significant advantages:

1. Elastic Scaling: Container-based deployment on Kubernetes, dynamically adjusting resources based on load
2. Resource Isolation: Resource isolation and quota management in multi-tenant environments
3. Portability: Support for public cloud, private cloud, and hybrid cloud deployment
4. Ecosystem Compatibility: Native integration with the cloud-native ecosystem (such as Kubeflow, Argo, etc.)

The core philosophy of Cloud Native architecture is that applications are designed for cloud environments from the start, fully leveraging technologies such as containerization, microservices, declarative APIs, and immutable infrastructure. Kubernetes, as the de facto standard orchestration platform for cloud native, provides powerful scheduling capabilities for AI workloads. In AI scenarios, Kubernetes' value is demonstrated through: managing heterogeneous hardware resources like GPUs/NPUs via the Device Plugin mechanism; managing the lifecycle of distributed training tasks through the Operator pattern (such as Training Operator); enabling automatic elastic scaling of inference services through HPA/VPA; and implementing multi-tenant resource isolation through Namespaces and ResourceQuotas. Volcano is a CNCF batch scheduler designed specifically for high-performance computing and AI scenarios, supporting Gang Scheduling to ensure all Pods for distributed training start simultaneously, avoiding resource deadlocks.

The project is developed in Python, which lowers the barrier for AI teams to do secondary development and customization.

Applicable Scenario Analysis

Based on Cube Studio's feature matrix, it is particularly suitable for the following scenarios:

• Enterprise AI Platform for Mid-to-Large Companies: Enterprises that need to centrally manage multiple AI projects and teams
• Compute Service Providers: Service providers that need to build compute leasing and management platforms
• Private Large Model Deployment: Scenarios requiring large model training and inference in private environments
• Xinchuang Environments: Government and enterprise customers that need to adapt to domestic chips and operating systems

Summary

As an AI platform project open-sourced by Tencent Music, Cube Studio has achieved a high standard in both feature completeness and ecosystem compatibility. It not only covers the entire traditional MLOps workflow but also keeps pace with the demands of the large model era, providing a complete solution from fine-tuning training to inference deployment. Its capabilities in domestic adaptation and compute management give it a unique competitive advantage in the current market environment.

For teams looking for open-source AI platform solutions, Cube Studio is a choice worth evaluating in depth. The nearly 5,000 GitHub Stars also validate the community's recognition of this project.

Key Takeaways

• Cube Studio is a cloud-native one-stop AI platform open-sourced by Tencent Music, covering the entire MLOps workflow including data labeling, model development, distributed training, and inference deployment
• Supports SFT fine-tuning, reward model, and reinforcement learning training for large models like DeepSeek, with integrated mainstream inference engines including vLLM/Ollama
• Compatible with over ten distributed training frameworks including PyTorch, DeepSpeed, and ColossalAI, with RDMA high-speed networking support
• Deeply adapted to domestic CPUs/GPUs and Huawei Ascend NPU ecosystem, meeting Xinchuang environment requirements
• Built-in compute leasing platform and VGPU virtualization capabilities, suitable for enterprise AI platform building or compute service provider operations