A100
Core Facts
Timeline (last 90 days)
FP8 is natively supported on Hopper and Blackwell architecture GPUs (H100, H200, B200, B300), but the A100 has no FP8 hardware support.
Running a 4-bit quantized 70B model on 1×A100 80GB costs about $1.09 per hour and can support about 98,000 in-flight tokens.
AWS, GCP, and Azure typically have low default quotas for GPU instances (such as 0-8 A100s per account), and if you do not apply for a quota increase in advance before filing season, you may find that machines cannot be provisioned even if you have money.
Using the command colab new --session gemma-test --gpu A100 to request an A100 instance returns Session READY after about 5 seconds
In the A100 instance name gpu-a100-s-kkb-usc1f1, a100-s means A100 standard slice, and usc1f1 is the data center code (US Central zone 1, facility 1)
A Colab A100 instance took about 30 minutes to go from 200 compute units down to 197.28, at a billing rate of 5.30 units/hour
Running gemma4:26b (GGUF Q4_K_M) via Ollama on a Colab A100 instance achieves a generation speed of 179.42 tok/s, taking 2.16 seconds for 369 tokens
Running a ComfyUI custom node graph locally with models such as Wan 2.1 or Wan 3.0 requires more than 24GB of VRAM (NVIDIA RTX 3090/4090 or a cloud A100 instance).
NVIDIA 有意在 RTX 4090 上取消了 NVLink，业界普遍认为这与数据中心 A100/H100 的产品分层策略有关。
Most mainstream inference service providers currently rely on NVIDIA A100 or H100 clusters.
40 more timeline events
All Facts (20)
训练一次GPT-4级别的模型需要数万块A100/H100 GPU协同工作数月
90%Verified美国自2022年起对中国实施先进半导体出口管制，限制高端AI训练芯片如NVIDIA A100/H100及其后续产品的出口
85%VerifiedEach SM on the NVIDIA A100 has 65,536 32-bit registers.
80%VerifiedThe RTX 4090's GDDR6X memory interface bandwidth reaches up to 1008 GB/s
80%VerifiedNVIDIA's MIG technology enables hardware-level resource isolation on high-end GPUs such as the A100/H100.
80%VerifiedIndustry estimates put the training cost of GPT-4 at over $100 million, involving tens of thousands of A100 GPUs, months of continuous computation, and petabyte-scale training data.
80%VerifiedThe United States has restricted exports of high-end GPUs to China, including NVIDIA's A100 and H100
75%VerifiedThe model weights alone of a 175-billion-parameter model (in BF16 precision) require 350 GB of storage, and the Adam optimizer must maintain two sets of states—first-order moments and second-order moments—totaling more than 1 TB.
75%VerifiedEven after quantization, 70B-level models often require more than 40 GB of VRAM or a multi-GPU configuration.
75%VerifiedIn October 2022, the Biden administration introduced its first export controls targeting advanced computing chips, restricting the export of high-end GPUs such as the NVIDIA A100 and H100 to China.
75%VerifiedThe KV Cache of a 70B-parameter model with 128K context support can require hundreds of gigabytes of VRAM at full load, exceeding the 80GB VRAM capacity of a single H100.
70%VerifiedNVLink 4.0 (used in the H100) can achieve a unidirectional bandwidth of 900 GB/s.
70%VerifiedMIG leverages the GPC physical isolation capability of the Ampere architecture to partition GPU compute units, L2 cache, and memory bandwidth into independent partitions at the hardware level, providing true hardware-level isolation.
65%VerifiedThe U.S. Department of Commerce has included high-end GPUs such as the A100 and H100 on its export control list since 2022.
65%UnverifiedNVIDIA MIG仅支持A100/H100等高端卡且切分粒度有限，vGPU方案需要额外购买商业许可证
85%UnverifiedNVIDIA MIG仅A100/H100支持
85%Unverified单卡RTX 4090即可运行GPT-OSS 20B，单卡A100即可运行GPT-OSS 120B
80%Partially VerifiedNVIDIA's H100/A100 GPUs hold approximately 80% of the AI market share.
75%Unverified部署DeepSeek-R1 671B MoE模型需要至少数张高端GPU（如H100或A100），显存需求超过320GB
75%Partially VerifiedNVIDIA A100专业训练卡拥有40GB或80GB HBM2e显存，单卡售价超过1万美元
65%