AI Infra Salaries Skyrocket: The Truth Behind the Divide of 8 Million Yuan Annual Packages

"In the past, people in the AI industry called those doing Infra and engineering 'the civil engineering department of computer science' — doing the dirtiest, hardest work for the lowest pay. Who could have guessed they'd be this valuable now?" Liu Li, an executive at a large model unicorn, remarked to Leiphone.

Behind this metaphor is a salary revaluation now underway. Previously, AI Infra engineering roles at big tech companies rarely reached an annual package of 3 million yuan; now, good Infra talent can command 7 to 8 million. There's still a gap compared to algorithm roles (5 to 14 million), but it's already a remarkable turnaround.

Why Does the "Civil Engineering Department" Command High Salaries?

What exactly are AI Infra talents? Simply put, they squeeze the highest possible utilization out of expensive GPU compute cards, genuinely saving money for their companies.

Lu Bingning, a senior AI business expert for South China at listed HR services firm Career International, witnessed this surge firsthand: "In 2024, I met an Infra candidate in Shenzhen whose annual salary was still 300,000 yuan; in 2025 it rose to 500,000; this year he switched jobs and jumped straight to 1 million."

The trigger for this round of salary hikes was the release of DeepSeek^Ad R1. According to Lu Bingning, after R1's release, the entire industry was pondering one thing: DeepSeek had so few people yet built such a great product and made so much money — how did they do it? Part of the answer later landed on Infra cost reduction.

The Hidden Funnel in Compute Bills

For large model companies, the most expensive cost is almost all piled onto compute cards. An industry-circulated budget allocation goes like this: for every 100 yuan an AI model company invests in AI training, 40 yuan goes to compute, 30 yuan to talent, 20 yuan to advertising, and the remaining 10 yuan to data.

But the money gets spent, and the cards often sit idle. Zhang Tong, a compute scheduling expert at a large model lab, revealed that actual GPU utilization in the industry has long been low — averaging only 25% to 27%, meaning more than half of the purchased compute never actually runs.

And this is precisely where Infra talents come in. "With the same batch of cards, a good Infra team can push utilization to 40%. That gap of over ten percentage points is the moat."

Lu Bingning estimates that doing Infra cost reduction well can save at least tens of millions in compute expenses. "Infra's role now is to save money for vendors. Saving money is also making money."

GPU utilization refers to the proportion of time within a given window that a GPU is actually performing effective compute tasks. It has become one of the most closely watched metrics at large model companies because GPU procurement, power delivery, cooling, racks, and network bandwidth are all provisioned under full-load assumptions, yet real training tasks involve a lot of waiting: data loading can't keep up, all-reduce communication blocks, insufficient VRAM forces smaller batches, and poor task queue scheduling. An industry average of 25%–27% means large numbers of expensive H100/A100s sit half-idle most of the time. Raising utilization from 25% to 40% is equivalent to conjuring roughly 60% more effective compute out of thin air with the same total number of cards — which is the direct value source that Infra teams create for their companies.

VLLM and SGLang are the two mainstream large model inference frameworks today. VLLM was open-sourced by a team at UC Berkeley, with its core innovation being PagedAttention, which manages the KV Cache in pages to significantly reduce VRAM fragmentation and boost concurrent throughput; SGLang was launched by the LMSYS team, featuring RadixAttention and a front-end DSL, offering better scheduling efficiency in scenarios like structured output and multi-turn dialogue. They allow small and mid-sized teams to run services without rewriting an inference engine from scratch, making "deploying VLLM to production" a relatively low-barrier Infra role; what truly creates the salary gap is the deep optimizers who can build on top of these frameworks with operator fusion, quantization, speculative sampling, and distributed parallel scheduling.

Algorithm Dividends Peak, Systems Engineering Takes Over

At a deeper level, the large model tech race has moved from competing on algorithmic innovation to waging hard battles in底层 systems engineering.

Zhou Yue, a senior AI Infra engineer at a large model unicorn, frankly stated: "The efficiency leap you get from squeezing out a bit of optimization in Infra is far more worthwhile than grinding away at algorithms."

Large model training is now almost an open-book exam. Thanks to the proliferation of open-source models, foundational architectures like Transformer have matured and converged, and the industry's competitive focus has shifted from basic pre-training to engineering implementation and inference optimization. "Now in large model training, truly irreplaceable creative work is rare — most of the job is data cleaning, data ratio experiments, and engineering tuning. These tasks aren't high-tech, but they deliver real money."

Only 5% Reap This Round of Premium

On the surface, "salary doubling" has become a buzzword in the AI Infra recruitment market. But in the view of Guo Quanwei, co-founder and CTO of StartLux, this is more like mistaking a few outliers for the universal norm.

Job salaries on Maimai show that AI Infra talents in mainland China earn roughly 73,000 yuan per month, already slightly higher than those doing large model algorithms. But averages mask internal divergence. What truly pulls the salary curve up is that wave of people who can boost compute card utilization; the many doing "ordinary Infra" are still far from doubling. For example, Infra roles that deploy inference frameworks like VLLM and SGLang to production mostly stall at 40,000 to 50,000 yuan per month — nowhere near "doubling."

Who's Doubling, Who's Standing Still

The answers from several interviewees converge on two directions: inference optimization and model quantization. What these people do is squeeze every GPU's compute to its absolute limit. "If you help the company save 30% of training time, or add 30% more token throughput, costs drop directly by a notch," Guo Quanwei added.

Lu Bingning divides Infra into three parts: inference optimization and acceleration, training frameworks, and operator development. "All three have strong salary growth, but they differ in how much attention they get." He offered a plain analogy: operator developers are like polishing kitchen utensils; training framework people are building the kitchen system; inference people are figuring out how to serve dishes fastest — the direct monetization link. The industry generally considers operators and inference more important.

But the problem is, this group isn't as large as one might imagine. Guo Quanwei estimates that those truly reaping the high premium account for only about 5%–10% of the entire AI Infra population. Most are veterans who have深耕ed 5 to 10 years in Infra, having early careers in distributed systems, high-performance computing, search and recommendation, and databases at large internet companies — their starting salaries were already high, and their experience is hard to replace.

Model quantization is the process of compressing model weights and activations from higher bit-widths like FP16/BF16 down to FP8, INT8, or even INT4, thereby reducing VRAM usage, improving memory bandwidth utilization, and accelerating computation. Common quantization paths include post-training quantization (PTQ), quantization-aware training (QAT), and hardware-specific formats like W8A8 and W4A16. It is one of the high-premium Infra directions because quantization directly determines how large a model a single card can hold and how high a throughput it can achieve, all while minimizing accuracy loss. Doing quantization reliably requires understanding both model numerical characteristics and the operator instructions and runtime scheduling of CUDA or domestic chips — making it one of the roles with the deepest intersection of algorithms and systems engineering.

Who's Paying Big: Five Types of Buyers, Five Sets of Calculations

The divergence in job hotness and coldness reflects different employers' private calculations and anxieties. Across the industry, buyers willing to pay the Infra premium fall into roughly five categories.

ByteDance has enormous business scale and needs people who can keep training and inference scheduling stable. Whether it's Seedance or Doubao, if inference engineers can accelerate by 10%, the annual compute savings are considerable.

Tencent takes a different path: its foundation models were weaker in earlier years, and over the past two years it has clearly been catching up on foundation models and full-stack Infra, setting up a standalone AI Infra department and poaching mature teams from outside.

Cloud vendors like Alibaba and Baidu operate at yet another rhythm. During the previous cloud computing and big data boom, they accumulated large numbers of Infra engineers doing distributed systems and foundational work. When the AI wave arrived, many could transfer directly to fill gaps. Plus, both have in-house chips and focus more on heterogeneity, making hiring a routine, steady effort with little frenzy of salary explosions.

The ones with truly surging demand for Infra talent are ByteDance, Tencent, plus unicorns like DeepSeek, Kimi, and MiniMax that want to squeeze maximum value from every card. Giants prefer regular army formations — an Infra department typically starts at fifty or sixty people; unicorns go for a "lean special forces" model, with Infra teams of around twenty, favoring versatile hands who understand both low-level operators and can operate inference engines.

Finally, there's a group of domestic chip makers "betting on the nation's fate." "Many domestic chip makers have reached the IPO stage, and options look set to cash out. Some Infra talents jump ship hoping to 'bet big and achieve financial freedom,'" Liu Li told Leiphone. While cash raises from chip makers generally hover around 45%, that potentially cashable stock is itself a highly attractive bargaining chip.

Interestingly, familiarity with NVIDIA's CUDA ecosystem remains "hard currency," with job-switch raises for such talent guaranteed at 80% or more; domestic cards rely more on hardware-software co-design and complex ecosystem adaptation, with raises typically around 50%. However, constrained by external export restrictions, NV-side talent is already scarce, and policy is pushing companies to procure domestic chips — meaning those on the domestic card track actually have more imagination in their prospects.

The North American market is equally hungry for AI Infra talent. OpenAI's recent job postings are almost entirely for inference engines, GPU low-level operators, and performance acceleration; Anthropic also has over a dozen corresponding positions open.

The CUDA ecosystem is the hardware-software system NVIDIA built around its GPUs, including the programming model, acceleration libraries like cuDNN/cuBLAS, the Nsight performance analysis tool, and a vast accumulation of operator and tuning knowledge. CUDA-savvy talent is "hard currency" because they can directly do kernel optimization, memory transfer optimization, and communication overlap at the GPU low level, truly squeezing out the hardware's theoretical peak. By contrast, domestic chips (such as Ascend, Cambricon, Hygon, etc.) still have rapidly evolving software stacks, with operator library coverage and tuning tool maturity lagging CUDA. Engineers often need to handle more tedious work across hardware characteristics, compiler adaptation, and framework integration simultaneously, so job-switch raises are slightly lower — but they are also less replaceable.

How Long Can This Round of Salary Hikes Last?

Lu Bingning's judgment is that this round of salary hikes can last at least two to three years. "We're still in a state of supply falling short of demand. Vendors are basically all hiring, and compensation is set very high."

The reason salaries will keep rising is that Infra has spawned many new business scenarios. For general conversational foundation models, many domestic vendors have largely completed their layouts; but the implementation of complex vertical scenarios has only just begun. Whether it's AI Coding, which devours compute, or the high-definition video generation track with extreme throughput and latency requirements, the full-chain commercialization loop is still incomplete in many cases.

The turnaround story of AI Infra is essentially a microcosm of the industry shifting from "telling stories" to "doing the math." When large model commercialization is continually stung by expensive compute bills, those who can squeeze profits out of compute cards will naturally command ever-rising salary premiums. But this round of dividends belongs only to those who truly go deep in systems engineering.

One-line Summary

DeepSeek R1 ignited a salary revaluation for AI Infra, but only 5%–10% of deep systems engineers actually reap the doubling premium.

Paragraph Summary

The article describes how AI Infra engineering roles went from being jokingly called "the civil engineering department of computer science" to seeing skyrocketing salaries. The trigger was DeepSeek R1's release, after which the industry realized Infra cost reduction was the key: average GPU utilization is only 25%–27%, while excellent teams can push it to 40%, with that dozen-plus percentage points directly corresponding to tens of millions in cost savings. In the recruitment market, good Infra talent's annual pay rose from under 3 million to 7–8 million, with individual cases going from 300,000 to 1 million in two years. But the high premium belongs only to the few who can squeeze compute cards to the limit through inference optimization and model quantization — just 5%–10% of the population. Many doing ordinary framework deployment still earn 40,000–50,000 yuan per month. Buyers include five types — ByteDance, Tencent, cloud vendors, AI unicorns, and domestic chip companies — each with their own strategy; CUDA talent gets at least 80% raises, domestic card roles about 50% but with promising prospects driven by policy. Interviewees judge this salary surge can last another two to three years.