Cube Studio Deep Dive: Tencent's Open-Source One-Stop Cloud-Native AI Platform

Overview

Cube Studio is a cloud-native, one-stop machine learning/deep learning/large model AI platform open-sourced by Tencent Music. It has earned nearly 5,000 Stars on GitHub. The project covers the entire MLOps workflow from data labeling and model training to inference deployment, while also supporting domestic hardware ecosystems. It stands as one of the most feature-complete AI platforms in China's open-source community.

MLOps (Machine Learning Operations) is an engineering practice that brings DevOps principles into the machine learning domain, aiming to solve the "last mile" problem of getting models from the lab into production. In traditional AI development workflows, data processing, model training, evaluation, and deployment often use different tools and platforms, resulting in extensive manual handoffs and environment inconsistencies between stages. MLOps connects these stages into reproducible, traceable end-to-end workflows through standardized pipelines, automated testing, and continuous integration/continuous deployment (CI/CD). Analyst firms like Gartner have repeatedly noted that the lack of MLOps capabilities is one of the biggest bottlenecks preventing enterprise AI projects from moving beyond PoC to scaled production.

For teams seeking enterprise-grade AI infrastructure solutions, Cube Studio offers a compelling option worth exploring in depth.

Notebook Online Development & Pipeline Orchestration

Cube Studio includes a built-in Notebook online development environment, allowing algorithm engineers to write and debug code directly without configuring local environments. The Notebook environment (typically based on JupyterHub or similar technologies) is a standard interactive development tool in data science. It lets users write code, run experiments, and view results instantly in the browser, with code, documentation, and visualizations mixed in a single document. In cloud-native scenarios, the Notebook environment runs inside containers on a Kubernetes cluster—users don't need to install CUDA drivers, deep learning frameworks, or other complex dependencies locally. The platform automatically allocates GPU resources and mounts pre-configured image environments, greatly alleviating the "environment configuration hell" problem.

More importantly, the platform offers drag-and-drop Pipeline orchestration capabilities, allowing users to visually connect data processing, feature engineering, model training, evaluation, and other steps into complete workflows. This orchestration essentially abstracts the MLOps process into a Directed Acyclic Graph (DAG), where each node represents an independent computational task (such as data cleaning, feature extraction, or model training), and edges between nodes define data dependencies and execution order. This approach draws from the design philosophy of mature workflow engines like Apache Airflow and Kubeflow Pipelines. Compared to purely code-defined pipelines, visual orchestration enables algorithm engineers without infrastructure backgrounds to understand and manage complex multi-step training processes, while the platform automatically handles task scheduling, failure retries, resource allocation, and other underlying logic.

This low-code orchestration approach dramatically lowers the barrier to building MLOps pipelines, enabling even algorithm practitioners without infrastructure experience to quickly construct end-to-end training workflows.

Distributed Training & Hyperparameter Search

At the model training layer, Cube Studio offers remarkably broad support. The platform natively supports multi-node, multi-GPU distributed training, compatible with mainstream frameworks including PyTorch, TensorFlow, and MXNet, as well as DeepSpeed, ColossalAI, and Horovod, which are widely used in large model training. It also supports PaddlePaddle, Ray, Volcano, and other distributed scheduling frameworks.

Distributed training is the core technology for large-scale model training. The basic idea is to distribute computational tasks across multiple GPUs on multiple machines for parallel execution. PyTorch's DistributedDataParallel (DDP) and TensorFlow's MirroredStrategy are the most fundamental data parallelism approaches. DeepSpeed, developed by Microsoft, is renowned for its ZeRO (Zero Redundancy Optimizer) technology, which shards optimizer states, gradients, and parameters across different GPUs, dramatically reducing memory consumption and making it possible to train models with tens or even hundreds of billions of parameters on limited hardware. ColossalAI, developed by the HPC-AI Tech team, provides multi-dimensional parallelism strategies including tensor parallelism, pipeline parallelism, and sequence parallelism. Horovod, originally open-sourced by Uber, uses a Ring-AllReduce communication pattern and is known for its clean API and framework-agnostic design. Ray is a general-purpose distributed computing framework whose sub-project Ray Train focuses on distributed training scenarios. Volcano is a CNCF batch scheduling system specifically designed for high-performance computing and AI training workloads on Kubernetes.

The integration of Hyperparameter Search functionality is another major highlight. Hyperparameters are parameters that must be set manually before model training begins, such as learning rate, batch size, number of network layers, and regularization coefficients—they directly affect model convergence speed and final performance. Traditional manual tuning relies on engineers' experience and intuition, which is inefficient and unlikely to find the global optimum. Automated hyperparameter search techniques include Grid Search, Random Search, Bayesian Optimization, and early-stopping-based methods like Hyperband. In the Kubernetes ecosystem, Katib is the most commonly used hyperparameter search component—it automatically creates multiple training jobs with different hyperparameter configurations to run in parallel, automatically selecting the optimal configuration based on target metrics (such as validation accuracy). Users can perform automated hyperparameter tuning directly within the platform, eliminating the inefficient cycle of manual tuning and significantly improving model iteration efficiency.

Large Model Training & Fine-Tuning Capabilities

In the era of large models, Cube Studio keeps pace with technological trends by providing complete support for SFT fine-tuning, Reward Model training, and RLHF (Reinforcement Learning from Human Feedback) training for large models like DeepSeek.

SFT (Supervised Fine-Tuning) is the first step in large model customization—continuing to train a pre-trained model on domain-specific labeled data to give it task-specific capabilities. Reward Model training is a critical intermediate step in the RLHF pipeline: first, human preference ranking data for different model outputs is collected, then a reward model is trained to learn human preference criteria. RLHF uses the trained reward model as a reward signal to further optimize the language model through reinforcement learning algorithms like PPO (Proximal Policy Optimization), making outputs better aligned with human expectations. This technical chain from SFT to Reward Model to RLHF is the core training paradigm behind conversational large models like ChatGPT and DeepSeek. Recently, DeepSeek has also introduced innovative methods like GRPO (Group Relative Policy Optimization), further advancing reinforcement learning training techniques.

Enterprises can complete the entire workflow from supervised fine-tuning to alignment training on their own data using open-source large models. For teams looking to train and customize large models in private environments, this capability chain offers significant practical value, effectively ensuring data security and model controllability.

Inference Services & VGPU Virtualization

After model training is complete, Cube Studio provides comprehensive inference service deployment capabilities. The platform integrates mainstream large model inference engines including vLLM, Ollama, and MindIE, supports multi-node inference deployment, and can handle high-concurrency online inference scenarios.

vLLM is a high-performance large model inference engine developed by a team at UC Berkeley. Its core innovation is PagedAttention technology, which borrows the paged memory management concept from operating systems to manage KV Cache, improving inference throughput by 2-4x compared to traditional implementations. Ollama is positioned as a lightweight local large model runtime tool, known for its minimalist user experience and suitable for rapid deployment and testing scenarios. MindIE (Mind Inference Engine) is the inference engine for Huawei's Ascend ecosystem, specifically optimized for Ascend NPU hardware, making it irreplaceable in domestic deployment scenarios. Multi-node inference (also called tensor parallel inference or pipeline parallel inference) is the key technology for handling ultra-large models (with hundreds of billions of parameters) that exceed single-GPU memory capacity, distributing model parameters across multiple GPUs to collaboratively complete inference computation.

Particularly noteworthy is the VGPU virtualization feature. In the real-world environment of scarce GPU resources, virtualization technology can partition a single physical GPU into multiple virtual GPUs, significantly improving resource utilization and reducing compute costs. The core idea of VGPU virtualization is to add a virtualization layer between physical GPUs and applications, partitioning a single physical GPU's compute power and memory on-demand for multiple containers or virtual machines. In the Kubernetes ecosystem, common VGPU solutions include NVIDIA's official MIG (Multi-Instance GPU, supported only on high-end cards like A100/H100), NVIDIA's commercial vGPU solution, and domestic open-source projects like HAMi (formerly vGPU-scheduler). These solutions achieve resource partitioning through different technical paths including API interception, time-slice rotation, or hardware-level isolation. In production environments, low GPU utilization is a widespread problem—many development and inference tasks use only 10%-30% of GPU compute capacity. VGPU virtualization can aggregate and utilize this fragmented compute power; according to industry practice data, overall GPU utilization can increase from 30% to over 70%.

Combined with the platform's compute leasing functionality, Cube Studio can effectively serve as an enterprise's internal GPU compute management and scheduling center.

Labeling Platform & Private Knowledge Base

Cube Studio also includes a built-in automated labeling platform, bringing the data labeling step—which typically requires external tools—into the unified platform. Additionally, the platform provides a private knowledge base feature that, combined with large model inference capabilities, enables rapid deployment of enterprise-grade RAG (Retrieval-Augmented Generation) applications.

RAG (Retrieval-Augmented Generation) is the mainstream architecture pattern for enterprise large model applications today. Its core approach is to retrieve document fragments relevant to the user's question from an external knowledge base before the large model generates a response, inject these fragments as context into the prompt, and then have the large model generate answers based on this reference information. Compared to relying solely on knowledge stored in model parameters, RAG effectively addresses large model knowledge timeliness issues and hallucination problems, while avoiding the expensive model fine-tuning required for each specific domain. A complete RAG system typically includes document parsing, text chunking, vectorization (embedding), vector database storage, similarity retrieval, and large model generation. Cube Studio integrates the private knowledge base with inference engines on the same platform, simplifying the RAG application development process.

Additionally, the AI Model Marketplace design enables team model assets to be accumulated and reused, effectively avoiding the problem of reinventing the wheel.

Domestic Ecosystem Support

In the broader context of information technology innovation and domestic substitution, Cube Studio's domestic hardware support is a significant differentiating advantage. The platform explicitly supports domestic CPUs, GPUs, and Huawei Ascend NPU ecosystems, along with RDMA high-speed networking, ensuring efficient distributed training communication even in domestic hardware environments.

Huawei Ascend NPU is currently one of the most mature domestic AI chip solutions in terms of ecosystem. Its flagship product, the Ascend 910B, has been adopted by multiple enterprises for large model training scenarios. The Ascend ecosystem includes underlying hardware, CANN (Compute Architecture for Neural Networks) operator library, MindSpore framework, and MindIE inference engine—a complete technology stack. RDMA (Remote Direct Memory Access) is a critical networking technology in high-performance computing and distributed training. It allows network cards to directly access remote machine memory, bypassing the operating system kernel and CPU, reducing network communication latency from microsecond to sub-microsecond levels. In large model distributed training, nodes need to frequently synchronize gradient data. RDMA networking (typically based on InfiniBand or RoCE v2 protocols) can reduce communication overhead by an order of magnitude, serving as an infrastructure prerequisite for efficient multi-node training.

For industry users in finance, government, telecommunications, and other sectors with mandatory domestic hardware requirements, Cube Studio provides a battle-tested technology option.

Cloud-Native Architecture Analysis

Cube Studio is built on a cloud-native architecture, leveraging Kubernetes for container orchestration and resource management at its foundation. Kubernetes (abbreviated K8s), originally open-sourced by Google, has become the de facto standard for container orchestration. In AI platform scenarios, Kubernetes' value extends far beyond container management: it provides a unified resource abstraction layer that enables heterogeneous compute resources such as CPUs, GPUs, and NPUs to be uniformly scheduled through the Device Plugin mechanism; its Namespace and ResourceQuota mechanisms naturally support multi-tenant resource isolation and quota management; and the Operator pattern allows platform developers to encapsulate complex distributed training tasks (such as PyTorchJob, TFJob) as Kubernetes-native resources, with custom controllers automatically managing task lifecycles.

This architectural choice brings several significant advantages:

• Elastic Scaling: Training and inference resources can be dynamically adjusted based on load. Kubernetes' Horizontal Pod Autoscaler (HPA) and Cluster Autoscaler mechanisms provide native support for inference service auto-scaling, automatically adjusting the number of inference instances based on request volume to optimize resource costs while maintaining service quality.
• Multi-Tenant Isolation: Supports multiple teams within an enterprise sharing the same platform, achieving resource isolation and fair scheduling between teams through Namespaces and resource quotas.
• Standardized Deployment: Container-based delivery reduces environment consistency issues, ensuring high consistency between training and production environments.
• Edge Computing Support: Cloud-native architecture naturally adapts to edge scenario model deployment. Leveraging edge computing frameworks like KubeEdge, trained models can be seamlessly pushed to edge nodes for execution.

Community Activity & Project Maturity

As of now, Cube Studio has 4,983 Stars and 877 Forks on GitHub, with Python as its primary development language. From a community activity perspective, the project maintains continuous updates with an ever-expanding feature set.

As a project open-sourced after being refined through Tencent Music's internal practices, Cube Studio's feature design has a clear production-environment orientation—it's not an experimental demo but an engineering platform validated through large-scale business operations.

Conclusion

In the AI infrastructure space, Cube Studio's positioning is crystal clear: to be a one-stop cloud-native AI platform that covers the full MLOps lifecycle, adapts to the demands of the large model era, and supports domestic hardware ecosystems. From Notebook development to distributed training, from large model fine-tuning to inference deployment, from data labeling to knowledge base construction, it aims to solve all core AI engineering problems within a single platform.

Comprehensive functionality also implies higher deployment and operational complexity—enterprises should evaluate their team's technical capabilities and actual needs during selection. However, for mid-to-large teams with infrastructure capabilities that want to build a unified AI platform, Cube Studio is undoubtedly a highly competitive open-source choice.