NVIDIA L4
A data center GPU introduced by NVIDIA, based on the Ada Lovelace architecture and positioned for mid-range inference and lightweight training tasks, with support for AI acceleration.
Timeline (last 90 days)
The break-even point for per-million-document cost between L4 and cpu-repack is a 3.4% busy ratio: when the L4 is busy more than 3.4% of its uptime, its per-document cost is lower than repack's.
An L4 instance costs about $1.0465 per hour, roughly ten times that of a 1 vCPU instance.
On Google Cloud Run, the minimum instance specification required to attach a GPU to an L4 service is 4 vCPU plus 16 GiB of memory.
On NVIDIA L4, the bf16 model (6.32 GiB) has a theoretical memory-bandwidth ceiling of about 47 tokens/s, while the int4 model (2.85 GiB) has a theoretical ceiling of about 105 tokens/s, with measured peak reaching 131.9 tokens/s.
NVIDIA L4 (g6/SageMaker) natively supports bf16, fp8, and int8, and int4 is widened to 16 bits; NVIDIA T4G (g5g) supports bf16 (emulated) and int8, does not support fp8, and widens int4 to 16 bits; NeuronCore-v2 (inf2/trn1) supports bf16 and int8, fp8 has hardware support but the compiler has yet to catch up, and int4 is widened to bf16 and is 27 times slower per call.
Inferentia2 and Trainium have 2.0 to 2.7 times the memory bandwidth of NVIDIA L4, but their decode speed is only 0.28 times that of L4.
Nodus 在 2026 年 10 月 11 日的试点中，使用 Qwen2.5-0.5B 模型在单张 NVIDIA L4 上运行相同配方，监督微调费用比 Modal 低 67%，持续预训练费用低 42.5%；计入一次失败尝试后微调节省降至 24.7%。
The training results of the 0.8B model are reportedly verified on an NVIDIA L4 GPU.
All Facts (8)
The break-even point for per-million-document cost between L4 and cpu-repack is a 3.4% busy ratio: when the L4 is busy more than 3.4% of its uptime, its per-document cost is lower than repack's.
50%UnverifiedAn L4 instance costs about $1.0465 per hour, roughly ten times that of a 1 vCPU instance.
50%UnverifiedOn Google Cloud Run, the minimum instance specification required to attach a GPU to an L4 service is 4 vCPU plus 16 GiB of memory.
50%UnverifiedOn NVIDIA L4, the bf16 model (6.32 GiB) has a theoretical memory-bandwidth ceiling of about 47 tokens/s, while the int4 model (2.85 GiB) has a theoretical ceiling of about 105 tokens/s, with measured peak reaching 131.9 tokens/s.
50%UnverifiedNVIDIA L4 (g6/SageMaker) natively supports bf16, fp8, and int8, and int4 is widened to 16 bits; NVIDIA T4G (g5g) supports bf16 (emulated) and int8, does not support fp8, and widens int4 to 16 bits; NeuronCore-v2 (inf2/trn1) supports bf16 and int8, fp8 has hardware support but the compiler has yet to catch up, and int4 is widened to bf16 and is 27 times slower per call.
50%UnverifiedInferentia2 and Trainium have 2.0 to 2.7 times the memory bandwidth of NVIDIA L4, but their decode speed is only 0.28 times that of L4.
50%UnverifiedNodus 在 2026 年 10 月 11 日的试点中，使用 Qwen2.5-0.5B 模型在单张 NVIDIA L4 上运行相同配方，监督微调费用比 Modal 低 67%，持续预训练费用低 42.5%；计入一次失败尝试后微调节省降至 24.7%。
50%UnverifiedThe training results of the 0.8B model are reportedly verified on an NVIDIA L4 GPU.
50%