Cube Studio: Tencent's Open-Source One-Stop AI Platform Covering Everything from Large Model Training to Inference

Project Overview

Cube Studio is a cloud-native, one-stop machine learning/deep learning/large model AI platform open-sourced by Tencent Music. The project has earned nearly 5,000 stars on GitHub with 877 forks. Built with Python, it stands as one of the most comprehensive open-source MLOps platforms in China.

Cloud Native refers to applications designed from the ground up for cloud environments, fully leveraging containerization, microservices, declarative APIs, and automated orchestration. Cube Studio is built on Kubernetes (K8s), which means it inherently possesses enterprise-grade operational capabilities such as elastic scaling, service self-healing, and rolling updates. MLOps (Machine Learning Operations) is an engineering practice that has emerged in recent years, borrowing from DevOps principles to standardize and automate the development, deployment, and operations of machine learning models. In the MLOps maturity model, a well-rounded platform needs to cover data management, experiment tracking, model registry, automated pipelines, monitoring and alerting, among other dimensions—and Cube Studio provides full-stack capabilities within this framework.

The platform connects the entire AI lifecycle from data labeling, model training, and hyperparameter search to inference deployment. Especially in the era of large models, Cube Studio offers comprehensive support for fine-tuning and multi-node inference of models like DeepSeek, making it a popular choice for enterprises building private AI infrastructure.

Detailed Core Feature Modules

Notebook Online Development Environment

Cube Studio includes a built-in web-based Notebook online development environment, allowing developers to conduct algorithm development and debugging without configuring local environments. This cloud-based development model stems from the maturity of the Jupyter Notebook ecosystem—Jupyter was originally spun off from the IPython project by Fernando Pérez in 2014 and has since become the de facto standard tool for data science and AI development. Its core philosophy is "Literate Programming," merging code, visual output, and documentation into a single interactive document. In enterprise scenarios, hosting Notebooks in the cloud (rather than running them on developers' local machines) offers several clear advantages:

• Significantly lowers the barrier to entry for AI development—developers no longer need to spend hours configuring CUDA drivers, Python dependencies, and GPU environments
• Team members share computing resources, eliminating the "it works on my machine" environment inconsistency problem while preventing expensive GPU resources from sitting idle on personal workstations
• Supports multiple kernels (such as Python, R, Julia) to accommodate different development needs, with the ability to quickly switch runtime environments via pre-built Docker images

Drag-and-Drop Pipeline Task Orchestration

The platform supports visual Pipeline orchestration, allowing users to build complex algorithm workflows through drag-and-drop. This low-code orchestration approach shares design inspiration with well-known workflow engines in the industry—such as Apache Airflow (focused on general data pipelines), Kubeflow Pipelines (Google's open-source ML workflow engine), and Argo Workflows (cloud-native workflow engine). The core value of Pipeline orchestration lies in abstracting each step in AI development (data preprocessing, feature engineering, model training, evaluation) into reusable "Operators" that define execution order and dependencies through Directed Acyclic Graphs (DAGs). This allows algorithm engineers to focus on the model itself rather than tedious engineering work. More importantly, standardized Pipelines make experiments reproducible and auditable, with the entire process executable automatically—particularly crucial in industries like finance and healthcare where model compliance requirements are strict.

Distributed Training Capabilities

Cube Studio offers comprehensive support for distributed training:

• Framework Support: PyTorch, TensorFlow, MXNet, DeepSpeed, PaddlePaddle, ColossalAI, Horovod, Ray, Volcano, and other mainstream distributed training frameworks
• Training Modes: Multi-node, multi-GPU distributed training with RDMA high-speed network interconnection
• Hyperparameter Search: Built-in automatic hyperparameter search to improve model tuning efficiency

To understand the significance of this framework list, it's important to know the two core paradigms of distributed training: Data Parallelism and Model Parallelism. Data parallelism splits training data across multiple GPUs, with each card holding a complete model replica and independently computing gradients, then synchronizing gradients through collective communication operations like AllReduce—Horovod is Uber's classic open-source data parallelism framework, known for its ease of use. Model parallelism splits the model itself across multiple cards, suitable for ultra-large models that cannot fit in a single card's memory—DeepSpeed (open-sourced by Microsoft) and ColossalAI (open-sourced by HPC-AI Tech) are representatives in this field, using techniques like ZeRO (Zero Redundancy Optimizer) to shard optimizer states, gradients, and parameters, dramatically reducing memory consumption. Ray positions itself as a general-purpose distributed computing framework, supporting not only training but also reinforcement learning and hyperparameter search. Volcano is a batch computing scheduling engine contributed by Huawei to CNCF, specifically addressing scheduling of high-performance computing tasks on Kubernetes. PaddlePaddle is Baidu's open-source deep learning framework, widely used in Chinese NLP and domestic scenarios.

RDMA (Remote Direct Memory Access) mentioned above is a critical network technology in distributed training. Traditional TCP/IP networks require multiple memory copies and kernel-mode switches when transferring gradient data between GPUs, with latencies reaching tens of microseconds. RDMA allows network cards to directly read and write remote machine memory, bypassing the CPU and operating system kernel, reducing latency to the 1-2 microsecond level. In large-scale distributed training, communication overhead is often the performance bottleneck, and RDMA (typically implemented through InfiniBand or RoCE v2 networks) can improve multi-node training communication efficiency by several times, making it standard equipment for thousand-GPU training clusters.

Hyperparameter Optimization (HPO) is another important capability. The choice of hyperparameters like learning rate, batch size, and number of network layers has a significant impact on model performance, but traditional manual tuning is extremely inefficient. Automated search methods include Grid Search, Random Search, Bayesian Optimization, and early-stopping-based Hyperband. Cube Studio's built-in HPO functionality means users can define a search space and let the platform automatically find optimal hyperparameter combinations, dramatically improving tuning efficiency.

This broad framework compatibility means teams can flexibly choose training frameworks based on actual needs without being locked into the platform.

Full Pipeline for Large Model Training and Inference

Facing the large model wave, Cube Studio provides a complete large model support solution:

• Training Side: Supports the full workflow of SFT supervised fine-tuning, reward model training, and reinforcement learning (RLHF) for large models like DeepSeek
• Inference Side: Integrates mainstream inference engines including vLLM, Ollama, and MindIE, with multi-node inference deployment support
• Application Side: Built-in private knowledge base functionality for quickly building enterprise-grade RAG applications

The large model training workflow involved here deserves deeper understanding. SFT (Supervised Fine-Tuning) is the first step in large model alignment: building upon a pre-trained base model, using high-quality human-annotated instruction-response pairs for supervised training, teaching the model to answer questions in the way humans expect. But SFT alone isn't enough—the model may generate content that seems fluent but is actually harmful or inaccurate. This necessitates the RLHF (Reinforcement Learning from Human Feedback) stage: first training a Reward Model that learns human preference rankings for different responses; then using reinforcement learning algorithms like PPO (Proximal Policy Optimization) with the reward model's scores as signals to further optimize the language model. This "SFT → Reward Model → RLHF" three-stage process was first systematically proposed by OpenAI in the InstructGPT paper and has become the standard paradigm for large model alignment. DeepSeek, as a leading open-source large model series in China, uses MoE (Mixture of Experts) architecture in its V2/V3 versions, dramatically reducing inference costs while maintaining strong performance, making it a popular choice for enterprise private deployment.

On the inference side, vLLM is a high-performance large model inference engine open-sourced by UC Berkeley. Its core innovation is PagedAttention technology—borrowing the paged memory management concept from operating system virtual memory, it dynamically allocates KV Cache (Key-Value Cache, the memory area storing historical token attention information during Transformer inference) by pages, solving the memory fragmentation problem in traditional inference engines that leads to wasted GPU memory. This achieves throughput improvements of several times to tens of times compared to HuggingFace Transformers. Ollama is a lightweight large model runtime tool designed for individuals and small teams, known for its minimalist command-line experience—a single command can download and run various open-source models. MindIE (Mind Inference Engine) is the inference engine for Huawei's Ascend ecosystem, deeply optimized for Ascend NPUs, serving as a core component for domestic inference deployment. Multi-node inference deployment addresses the challenge of insufficient single-machine memory for ultra-large models (hundreds of billions or even trillions of parameters), splitting the model across multiple machines for collaborative inference through Tensor Parallelism and Pipeline Parallelism.

On the application side, RAG (Retrieval-Augmented Generation) is the mainstream architecture pattern for enterprise-grade large model applications. Pure large models have limitations including knowledge cutoff dates, tendency to produce hallucinations, and inability to access enterprise private data. RAG's core approach is: when a user asks a question, first retrieve relevant document fragments from the enterprise private knowledge base (typically using semantic search based on vector databases), then input the retrieved context along with the user's question into the large model, letting the model generate answers based on real documents. This approach leverages both the language understanding and generation capabilities of large models while ensuring answer accuracy and timeliness through external knowledge sources. Cube Studio's built-in private knowledge base functionality means enterprises can import internal documents, product manuals, technical specifications, and more into the system to quickly build proprietary intelligent Q&A services without sending sensitive data to external APIs.

This complete training-inference-application pipeline allows enterprises to achieve private large model deployment without cobbling together multiple tools.

VGPU Computing Power Management and Virtualization

The platform includes built-in computing power lease management with VGPU virtualization technology, enabling fine-grained partitioning and scheduling of physical GPU resources.

GPU virtualization is a key technology for solving the low utilization rates of enterprise AI computing resources. According to industry research data, the average utilization of enterprise GPU clusters is typically below 30%—developers often request entire GPU cards for debugging but actually use only a small amount of memory and computing power, causing serious waste. VGPU (Virtual GPU) technology splits a single physical GPU into multiple virtual GPU instances at the software level, with each instance having independent memory quotas and computing power shares, allowing multiple tasks or users to share the same physical card. Current mainstream GPU virtualization solutions include: NVIDIA's official MIG (Multi-Instance GPU, supporting only high-end cards like A100/H100 with hardware-level isolation), MPS (Multi-Process Service, software-level sharing), and third-party open-source solutions like HAMi (formerly k8s-vgpu-scheduler). Cube Studio's VGPU capability combined with computing power lease management enables GPU quota allocation and billing by project or team. For scenarios where multiple teams share a GPU cluster, this feature can significantly improve GPU utilization and reduce overall computing costs. Given the current GPU scarcity and high per-card prices (a single NVIDIA H100 costs over $30,000 on the market), the economic value of this capability is particularly significant.

Domestic Ecosystem Support: Huawei Ascend Adaptation

Cube Studio explicitly supports domestic CPUs/GPUs/NPUs, with particularly deep adaptation for the Huawei Ascend ecosystem.

Huawei Ascend is Huawei's self-developed AI computing architecture, with core chips including the Ascend 910 series for training and the Ascend 310 series for inference. The Ascend 910B/910C uses Huawei's self-developed Da Vinci architecture, whose core computing unit is the AI Core, containing a Cube Unit (matrix computation) and Vector Unit, specifically hardware-optimized for matrix multiplication and activation functions in deep learning. At the software stack level, the Ascend ecosystem uses CANN (Compute Architecture for Neural Networks) as its heterogeneous computing architecture, similar to NVIDIA's CUDA ecosystem, providing operator libraries, graph compilers, and runtime environments. The upper layer builds a complete development toolchain through the MindSpore deep learning framework and the aforementioned MindIE inference engine.

In the context where domestic substitution of AI infrastructure has become a rigid demand, this positioning gives Cube Studio a unique competitive advantage in Xinchuang scenarios. Xinchuang (Information Technology Application Innovation) is China's national strategy for promoting autonomous and controllable critical information infrastructure, covering the full stack from chips, operating systems, databases, and middleware to application software. Since 2020, key industries including finance, telecommunications, energy, and government affairs have accelerated Xinchuang substitution, and the demand for AI infrastructure adaptation to domestic chips has become increasingly urgent. However, migrating from the NVIDIA CUDA ecosystem to the Ascend CANN ecosystem is not simply a matter of "swapping cards"—it involves extensive adaptation work including operator compatibility, precision alignment, and performance tuning. Cube Studio has completed these adaptations at the platform level, meaning upper-layer users can relatively transparently run AI tasks on domestic hardware, significantly lowering the technical barrier for enterprise domestic migration.

Other Noteworthy Features

• Automated Labeling Platform: Built-in labeling tools to reduce data annotation costs. Data labeling is one of the most time-consuming and labor-intensive aspects of AI development, with industry saying that "data labeling accounts for 80% of AI project workload." Automated labeling typically combines pre-trained models for pre-labeling, followed by human review and correction, improving labeling efficiency by 3-5x.
• Edge Computing Deployment: Supports model deployment to edge devices. Edge inference runs models on devices close to data sources (such as industrial gateways, smart cameras, vehicle terminals), avoiding the latency and bandwidth costs of uploading data to the cloud, suitable for scenarios with high real-time requirements or limited network conditions.
• AI Model Marketplace: Provides model sharing and reuse mechanisms to promote team collaboration. Similar to Docker Hub for container images, a Model Registry is a key component in the MLOps system, responsible for model version management, metadata recording, lineage tracking, and access control.
• Cloud-Native Architecture: Built on Kubernetes with elastic scaling and high availability capabilities. Kubernetes (K8s) is Google's open-source container orchestration system that has become the de facto standard for cloud-native infrastructure, with over 80% of global containerized workloads running on K8s.

Use Cases and Selection Recommendations

Cube Studio is suitable for the following typical scenarios:

1. AI Platform Construction for Medium and Large Enterprises: A one-stop platform reduces the complexity of multi-tool integration and unified AI asset management. The core concept of an AI platform is "platform capabilities sink down, business innovation rises up"—consolidating common capabilities like data processing, model training, and inference services into platform services that algorithm teams across business lines can quickly invoke, avoiding reinventing the wheel.
2. Centralized Computing Resource Management: VGPU virtualization and computing power leasing features are ideal for multi-team shared GPU clusters
3. Private Large Model Deployment: The complete training and inference pipeline supports enterprises in building their own large model services. Core drivers for private deployment include data security compliance (sensitive data stays within boundaries), customization needs (industry-specific models), and cost control (self-hosting is more economical than API calls for high-frequency scenarios).
4. Xinchuang Domestic Substitution: Support for domestic chips like Ascend meets compliance requirements

If your team is small or only needs a single function (e.g., inference deployment only), lighter-weight tools may be more appropriate—for example, consider using vLLM or Triton Inference Server directly for inference services only, or MLflow or Weights & Biases for experiment management only. But for medium to large teams that need unified management of the entire AI workflow, Cube Studio's comprehensive capabilities are worth serious evaluation.

Summary

As an open-source project with nearly 5,000 stars, Cube Studio reflects Tencent Music's deep expertise in the MLOps domain. Its broad feature coverage, rapid adoption of new technologies (DeepSeek fine-tuning, vLLM inference, etc.), and support for the domestic ecosystem make it an important candidate for Chinese enterprises building AI platforms. Internationally, platforms with similar positioning include Google's Vertex AI, AWS SageMaker, and open-source options like Kubeflow and MLflow, but Cube Studio has differentiated advantages in domestic hardware adaptation and full-pipeline large model support. For teams looking for a one-stop AI platform, Cube Studio offers a feature-complete and continuously evolving open-source choice.