GPUStack in Action: Complete Guide to Day 0 Deployment of DeepSeek-V4-Flash

Introduction: Why Day 0 Deployment Matters

Whether a new model can go live in a production environment on the day it's released is a direct measure of an inference platform's maturity. This article is based on a hands-on demo from the GPUStack technical channel on Bilibili, fully reproducing the end-to-end deployment of DeepSeek-V4-Flash (DSpark version) on an 8× H20-141G setup using GPUStack.

"Day 0 deployment" means going live on the same day a model is released — a concept borrowed from software engineering as the ultimate test of deployment agility. In the era of large language models, where model architectures evolve rapidly, inference frameworks must continuously adapt to new attention mechanisms, quantization schemes, and parallelism strategies. Traditional manual deployment is a serious challenge: it often means compiling inference frameworks from source, configuring dependencies, and iterating through parameters — a process that can take anywhere from a few hours to several days.

With GPUStack's built-in SGLang inference backend and image-based management, the entire process is compressed into a few clicks in the Web UI. That's the core value this test demonstrates.

As an open-source GPU cluster management and inference platform, GPUStack containerizes inference backends and makes configuration visual, lowering the operational complexity from "expert-level" to "operator-level" — enabling any team with basic Web skills to complete a production-grade deployment on launch day.

Step 1: Add the DSpark Image Version for SGLang

To support the DSpark variant of DeepSeek-V4-Flash, you first need to attach a dedicated image version to the SGLang backend. The entire process requires no command line — everything is done in the Web UI:

1. Go to "Inference Backends" and select Edit for SGLang
2. Click "Add Version" and create a new version named DSpark
3. Fill in the corresponding image address and configuration details

About SGLang and DSpark: SGLang (Structured Generation Language) is a high-performance LLM inference framework developed jointly by academia and industry. Its core advantage is the RadixAttention mechanism — through prefix sharing and reuse of the KV Cache, it significantly reduces VRAM usage and computational redundancy in multi-turn conversations and batch inference. Compared to frameworks like vLLM, SGLang often delivers superior throughput in high-concurrency, long-sequence scenarios. DSpark is a variant of SGLang deeply optimized for the DeepSeek model architecture, specifically adapted to DeepSeek's MLA (Multi-head Latent Attention) compression mechanism, achieving substantial inference efficiency gains without sacrificing model accuracy.

The "image as version" design is elegant: it abstracts different variants of an inference framework as pluggable version modules, preserving SGLang's core stability while allowing flexible attachment of model-specific optimized images. Teams that need to keep up with new models don't have to rebuild their entire environment — just add a new version entry.

Step 2: Create a New Deployment and Select the Model

After configuring the backend, return to the deployment page and click "Deploy Model" in the top-right corner. Select ModelScope as the model source — for users in China, pulling models from ModelScope offers better network reliability and download speeds.

ModelScope is China's largest open-source model community, jointly built by Alibaba Cloud and DAMO Academy, hosting major Chinese LLMs including DeepSeek, Qwen, and ChatGLM. Compared to Hugging Face, ModelScope offers significantly faster download speeds in Chinese network environments, while providing model formats and SDK interfaces compatible with Hugging Face — allowing domestic teams to switch model sources without modifying any code. GPUStack natively supports ModelScope as a model source, effectively preventing download interruptions caused by network instability in production deployments.

In the search box, type and select DeepSeek-AI/DeepSeek-V4-Flash-DSpark, then choose the SGLang DSpark version configured earlier as the backend.

The decoupled "model + backend" selection approach allows the same model to be tested against different inference engines for side-by-side comparison, and the same backend to serve multiple models simultaneously — significantly improving resource utilization.

Step 3: Configure Backend Parameters and Environment Variables

Once the model is selected, you enter the critical parameter tuning phase. In the "Advanced" options, fill in the backend parameters one by one. These parameters directly determine the tensor parallelism strategy, VRAM allocation, and throughput performance on the 8× H20 setup.

Tensor Parallelism and MoE Architecture: Tensor Parallelism is one of the core multi-GPU parallelism strategies for large model deployment. It splits individual weight matrices along specific dimensions across multiple GPUs, with each card handling a portion of the matrix multiplication and then aggregating results via AllReduce communication. DeepSeek-V4-Flash uses a MoE (Mixture of Experts) architecture, where the Expert layer parallelism involves not just tensor parallelism but also Expert Parallelism (EP) — distributing different Experts across devices and using a dynamic routing mechanism to determine token assignment. On the 8× H20 configuration, properly setting the tensor parallelism degree (typically 8) maximizes VRAM utilization while keeping inter-GPU communication overhead within NVLink bandwidth limits, achieving the optimal balance between throughput and latency.

About the H20-141G Hardware: The NVIDIA H20 is a data center GPU designed specifically for the Chinese market, based on the Hopper architecture, with 141GB of HBM3 VRAM per card — one of the highest VRAM capacities among NVIDIA's export-compliant product lines. Compared to the H100/H800, the H20 has some limitations in FP16/BF16 compute, but its enormous memory bandwidth (4.0 TB/s) and VRAM capacity make it particularly well-suited for inference on large MoE models. An 8× H20 configuration provides a total of 1,128GB of VRAM, enabling full loading of ultra-large models like DeepSeek-V4-Flash without any VRAM quantization.

In addition to backend parameters, you also need to add an environment variable to ensure dependencies are installed correctly. While this may seem like a minor detail, it is a necessary condition for the DSpark version to start successfully. GPUStack centralizes all these configuration items in a unified visual interface, effectively preventing common errors in handwritten startup scripts — such as path mistakes and missing variables.

Step 4: Launch Deployment and Monitor Logs

After submitting the deployment, the model begins to start up. Through the log window, you can observe the following key startup signals:

• CUDA Graph Capture — CUDA graph capture in progress, a core step in SGLang's inference latency optimization
• Application Startup Complete — Application has finished starting
• Uvicorn listening on inference port — Service is ready and accepting requests

The Performance Significance of CUDA Graph: CUDA Graph is a performance optimization technology introduced by NVIDIA in CUDA 10. In traditional CUDA inference, each computation step requires the CPU to issue instructions to the GPU one by one, generating significant kernel launch overhead. CUDA Graph uses a "record and replay" mechanism to pre-capture a fixed computation graph (such as a single Transformer forward pass) as a unified execution unit. Subsequent inference only needs to trigger one replay to complete the entire computation, dramatically reducing CPU-GPU synchronization latency. For LLM inference workloads with relatively fixed computation graphs, CUDA Graph typically delivers 10%-30% latency improvements, with especially pronounced effects in low-latency, high-concurrency scenarios. This is why observing CUDA Graph Capture completion in the logs is an important indicator that the service is truly ready.

When the instance status changes to Running, deployment is complete. Thanks to image-based management, the entire process skips the lengthy wait of on-site compilation.

Benchmark Results: Single-Concurrency Throughput Stable at 200 tokens/sec

After deployment, head to the "Playground" for online validation. After a few rounds of conversation, the real-time throughput data displayed in the bottom-right corner of the interface shows:

Output: 185.94 tokens/sec, single-concurrency TPS stable around 200

On the 8× H20-141G configuration, the DeepSeek-V4-Flash DSpark variant demonstrates impressive single-concurrency generation speed. TPS (Tokens Per Second) is the core metric for measuring LLM inference throughput. Single-concurrency TPS reflects the model's generation speed under a single request and directly determines the user's perceived response experience — generally, 100 tokens/sec or above is considered sufficient for a smooth "typewriter effect," while approximately 200 tokens/sec means that even for long-text generation, the user experience remains fluid: generating a 2,000-character document takes only about 10 seconds, far exceeding human reading speed.

It's worth noting that high single-concurrency TPS does not equate to excellent overall system throughput. Under multi-concurrency load, VRAM bandwidth bottlenecks and batch scheduling strategies become the dominant factors. For formal stress testing, click on the instance details to obtain the inference service's IP and port, then connect directly to a benchmark tool (such as vLLM's benchmark_serving.py) for multi-concurrency validation.

Summary and Reflections

The biggest takeaway from this Day 0 test is: the value of an inference platform lies not only in its performance metrics, but in its ability to lower the barrier for getting new models online.

GPUStack consolidates inference backend management, image version control, model source selection, and parameter configuration into a single visual interface, transforming a deployment process that once required professional DevOps expertise into a series of trackable UI operations. For teams that want to quickly evaluate cutting-edge models without spending significant effort on environment setup, this is a genuine productivity gain.

Of course, the data in this article comes from a single-scenario demo. The 200 TPS single-concurrency result was achieved under specific hardware and parameter configurations — multi-concurrency performance and long-term stability in actual production environments still require further validation. But as a demonstration of "deploy on release day" capability, the combination of GPUStack and SGLang DSpark has fully proven its competitiveness in rapidly adopting frontier large models.

Key Takeaways