Cube Studio Deep Dive: A Practical Guide to Tencent's Open-Source One-Stop MLOps Platform

Overview

In the wave of AI engineering adoption, efficiently managing the entire pipeline from data labeling and model training to inference deployment remains a core challenge for enterprises. Cube Studio, open-sourced by Tencent Music, was built to address this pain point—it's a cloud-native, one-stop machine learning/deep learning/large model AI platform covering the entire MLOps algorithm pipeline. It has earned nearly 5,000 stars on GitHub.

This article provides a comprehensive analysis of Cube Studio from the perspectives of architecture design, core capabilities, and ecosystem compatibility, helping you determine whether it's the right fit for your team and business scenario.

Platform Positioning: Full MLOps Lifecycle Coverage

Cube Studio's core positioning is as a one-stop AI platform, meaning it's not just a training framework or inference engine, but covers the complete lifecycle of an AI project from zero to one.

To understand the value of this positioning, it helps to understand the MLOps background. MLOps (Machine Learning Operations) is an engineering practice system that brings DevOps principles to the machine learning domain, aiming to solve the "last mile" problem of getting models from the lab to production. In traditional AI development, data scientists use Jupyter Notebooks for experiments, but after models go live, they face issues like chaotic version management, non-reproducible training, and broken data pipelines. MLOps connects model development, training, deployment, and monitoring into a closed loop through standardized pipelines, automated CI/CD mechanisms, and unified metadata management. Companies like Google and Microsoft began heavily promoting the MLOps concept around 2020, and Gartner listed it as a key technology trend.

Specifically, Cube Studio covers the following key stages:

• Data Labeling: Built-in labeling platform with automated labeling support, significantly reducing manual labeling costs
• Online Development: Provides Notebook-based online development environments where developers can write and debug code without local configuration
• Task Orchestration: Supports drag-and-drop Pipeline orchestration, lowering the barrier to workflow construction
• Model Training: Supports multi-node, multi-GPU distributed training and hyperparameter search
• Inference Deployment: Offers inference services and VGPU virtualization capabilities
• Edge Computing: Extends AI capabilities to edge devices

This end-to-end design allows teams to complete all AI-related work on a single unified MLOps platform without switching between multiple tools—enterprises no longer need to separately maintain labeling tools (like Label Studio), experiment management tools (like MLflow), scheduling systems (like Airflow), and deployment platforms (like Seldon), while also avoiding data silo problems.

Core Capabilities: Deep Dive

Large Model Training and Fine-tuning

In the large model era, Cube Studio keeps pace with technology trends by providing complete training support for mainstream large models like DeepSeek, including:

• SFT (Supervised Fine-Tuning): Supervised fine-tuning on pre-trained models to quickly adapt to specific business scenarios
• Reward Model Training: Training capability for reward models in the RLHF pipeline
• RLHF Training (Reinforcement Learning from Human Feedback): Full support for the RLHF training workflow

These three stages constitute the standard paradigm for current large model alignment training, first systematically proposed by OpenAI in the InstructGPT paper and later widely applied in the training of products like ChatGPT and Claude. Specifically, the SFT stage uses high-quality, human-written instruction-response pairs to perform supervised fine-tuning on the base model, teaching it basic instruction-following abilities; the reward model stage has human annotators rank multiple model-generated responses by preference, training a scoring model that can automatically evaluate response quality; the RLHF stage uses the reward model's scores as reward signals to further optimize the language model's generation strategy through reinforcement learning algorithms like PPO (Proximal Policy Optimization). These three stages are interconnected, each requiring different training configurations and compute resources—building the complete pipeline from scratch typically requires weeks of engineering effort.

Cube Studio integrates them into a unified platform, eliminating the extensive engineering work of building training pipelines independently. This is highly practical for enterprises looking to privately deploy and fine-tune large models.

Large Model Inference Services

On the inference side, Cube Studio integrates the most mainstream large model inference frameworks:

• vLLM: High-performance LLM inference engine supporting advanced techniques like PagedAttention, with excellent throughput
• Ollama: Lightweight local large model runtime framework, suitable for quick validation and small-scale deployment
• MindIE: Huawei Ascend ecosystem inference engine, targeting domestic hardware scenarios

Among these, vLLM is an inference engine open-sourced by the UC Berkeley team in 2023, with PagedAttention as its core innovation borrowing from the paged memory management concept in operating systems. In traditional LLM inference, KV Cache memory management is the biggest bottleneck—each request's KV Cache requires pre-allocated contiguous GPU memory space, and since generated text length is unpredictable, systems often reserve memory at maximum length, resulting in 60%-80% memory waste. PagedAttention splits the KV Cache into fixed-size "pages" (Blocks), allocating them dynamically on demand. KV Cache pages from different requests can be stored non-contiguously in physical GPU memory, mapped through page tables. This design improves memory utilization to nearly 100%, boosting inference throughput by 2-4x on the same hardware. Additionally, vLLM supports Continuous Batching, allowing new requests to dynamically join without waiting for the current batch to complete, further reducing queuing latency.

Notably, the platform supports multi-node inference, which is crucial for deploying ultra-large models (e.g., models with tens of billions of parameters)—models that cannot fit in a single machine's GPU memory can be served through multi-node collaboration, breaking through hardware limitations.

Distributed Training Framework Ecosystem

Cube Studio demonstrates exceptional ecosystem compatibility in distributed training, supporting frameworks that cover virtually all mainstream options in the industry:

Framework	Use Case
PyTorch	General deep learning training
TensorFlow	Production-grade deep learning
MXNet	Flexible and efficient deep learning
DeepSpeed	Efficient large model training
PaddlePaddle	Baidu ecosystem
ColossalAI	Large model parallel training
Horovod	Distributed training communication
Ray	Distributed computing and scheduling
Volcano	Kubernetes batch scheduling

Among these, DeepSpeed and ColossalAI are the two most prominent distributed training frameworks in the large model era. DeepSpeed, developed by Microsoft, uses ZeRO (Zero Redundancy Optimizer) technology to partition optimizer states, gradients, and model parameters across different GPUs, reducing per-GPU memory requirements by several to dozens of times, making it possible to train models with tens of billions of parameters on consumer-grade GPUs. ColossalAI, developed by Singapore's HPC-AI Tech team, offers richer combinations of parallel strategies, including flexible mixing of data parallelism, tensor parallelism, pipeline parallelism, and sequence parallelism.

This broad framework support means teams don't need to switch their technology stack due to platform limitations. Existing training code can be migrated to Cube Studio at relatively low cost.

Domestic Hardware and Xinchuang Ecosystem Support

In the current context of domestic technology substitution, one of Cube Studio's important highlights is its comprehensive adaptation to domestic hardware ecosystems:

• Domestic CPU/GPU: Compatible with domestic processors and graphics processors
• NPU Ascend Ecosystem: Deep adaptation to Huawei Ascend AI processors, including MindIE inference engine integration
• RDMA Support: Supports Remote Direct Memory Access, which is crucial for communication efficiency in multi-node distributed training

Regarding RDMA, this is a network communication technology that allows computers to directly read and write each other's memory without involving the operating system kernel or CPU. In large model distributed training scenarios, multiple machines need to frequently synchronize gradient data. Each data transfer through the traditional TCP/IP network protocol stack requires multiple copies and context switches between user space and kernel space, with latency typically in the microsecond to millisecond range. RDMA offloads the network protocol stack to dedicated network cards (such as NVIDIA ConnectX series InfiniBand NICs), achieving sub-microsecond latency and near-line-rate bandwidth utilization. Current mainstream RDMA implementations include InfiniBand (IB), RoCE v2 (RDMA over Converged Ethernet), and iWARP. In training models with hundreds of billions of parameters, inter-node communication overhead can account for 30%-50% of total training time. RDMA can reduce communication overhead by an order of magnitude, making it a standard infrastructure requirement for large-scale distributed training.

For enterprises and institutions that need to meet Xinchuang compliance requirements, Cube Studio provides a viable path for smooth transition from mainstream international hardware to domestic hardware, without having to build domestic AI infrastructure from scratch.

Compute Management and Resource Scheduling

Cube Studio is not only an AI development platform but also has compute rental platform capabilities. Combined with VGPU virtualization technology, the platform enables fine-grained GPU resource allocation and sharing, improving utilization of expensive GPU resources.

VGPU (Virtual GPU) virtualization technology splits a physical GPU's compute resources and memory into multiple virtual GPU instances, allowing multiple users or tasks to share the same GPU. In AI platform scenarios, VGPU's value is particularly prominent: a single A100 80GB GPU costs over 100,000 RMB to procure, but many development and debugging tasks actually only need a few GB of memory and minimal compute power. If each developer is exclusively allocated an entire card, resource waste is extreme. NVIDIA officially provides MIG (Multi-Instance GPU) and vGPU as two virtualization solutions, but the former only supports high-end cards like A100/H100 with limited partitioning granularity, while the latter requires additional commercial licenses. The open-source community has therefore produced various alternatives, such as Fourth Paradigm's VGPUScheduler (now the HAMi project), which intercepts at the CUDA layer to implement more flexible GPU compute and memory partitioning. With VGPU capabilities integrated into Cube Studio, administrators can proportionally allocate a single GPU to different tasks—for example, splitting an A100 with 80GB memory into multiple virtual instances, significantly improving overall cluster GPU utilization.

With cloud-native architecture, the platform inherently possesses elastic scaling capabilities, dynamically adjusting resource allocation based on training and inference workloads. For teams with limited GPU resources, this fine-grained compute management capability directly impacts infrastructure cost control.

Additional Capabilities: Knowledge Base and Model Marketplace

Beyond core training and inference capabilities, Cube Studio provides two noteworthy additional features:

• Private Knowledge Base: Supports building enterprise private knowledge bases, combined with large models to enable RAG (Retrieval-Augmented Generation) applications, allowing large models to perform Q&A based on internal enterprise data
• AI Model Marketplace: Provides model marketplace functionality for internal team model sharing and reuse, avoiding redundant work

RAG (Retrieval-Augmented Generation) is one of the most mainstream technical architectures for enterprise large model applications, first proposed by Meta AI in 2020. Its core idea is to retrieve document fragments relevant to the user's question from an external knowledge base before the large model generates an answer, inject these fragments as context into the prompt, and then have the large model generate an answer based on the retrieved information. This architecture solves two core pain points of large models: first, the knowledge timeliness problem—pre-training data has a cutoff date, while RAG can retrieve the latest information in real-time; second, the hallucination problem—model-generated content can be traced back to specific source documents. A typical RAG system includes document parsing, text chunking, embedding (vectorization), vector database storage (such as Milvus, Chroma), similarity retrieval, and answer generation stages. Enterprise private knowledge bases are the most typical RAG application scenario—importing internal documents, product manuals, technical specifications, etc. into the knowledge base allows employees to obtain precise information through natural language Q&A while ensuring sensitive data doesn't leak externally.

While these two features aren't the platform's core selling points, they often deliver significant efficiency improvements in actual enterprise deployments.

Summary and Outlook

As a one-stop AI platform open-sourced by Tencent Music, Cube Studio's greatest value lies in integrating the entire AI engineering workflow into a unified cloud-native platform. From data labeling to model training, from hyperparameter search to inference deployment, from traditional deep learning to large model fine-tuning, the platform's coverage is remarkably comprehensive.

Especially in the large model era, Cube Studio's SFT/RLHF training support for models like DeepSeek, along with its integration of inference frameworks like vLLM and Ollama, makes it a powerful tool for enterprises building private large model capabilities. Combined with deep adaptation to the domestic Ascend ecosystem, the platform also has strong competitiveness in Xinchuang scenarios.

For teams looking for open-source AI platform solutions, Cube Studio is worth in-depth evaluation—especially for enterprises that need to simultaneously handle traditional ML tasks and large model applications while having domestic hardware requirements. You can start from the GitHub repository, first run through core workflows in a test environment, and then gradually evaluate whether it's suitable for production deployment.

Key Takeaways

• Cube Studio is a cloud-native one-stop AI platform open-sourced by Tencent Music, covering the full MLOps pipeline including data labeling, model training, and inference deployment
• Deep support for large model training paradigms, including SFT fine-tuning, reward model, and RLHF training for models like DeepSeek
• Integrates mainstream inference frameworks including vLLM, Ollama, and MindIE, with multi-node multi-GPU distributed inference support
• Compatible with nearly ten distributed training frameworks including PyTorch, DeepSpeed, and ColossalAI, offering exceptional ecosystem compatibility
• Comprehensive support for domestic CPU/GPU/NPU Ascend ecosystem and RDMA, meeting Xinchuang compliance requirements