ProductNeuronCore v2
NeuronCore-v2
AWS's second-generation neural processing core, used in Inferentia2 and Trainium chips, supporting bf16 and int8 precision.
Timeline (last 90 days)
Oct 12
NVIDIA L4 (g6/SageMaker) natively supports bf16, fp8, and int8, and int4 is widened to 16 bits; NVIDIA T4G (g5g) supports bf16 (emulated) and int8, does not support fp8, and widens int4 to 16 bits; NeuronCore-v2 (inf2/trn1) supports bf16 and int8, fp8 has hardware support but the compiler has yet to catch up, and int4 is widened to bf16 and is 27 times slower per call.
Unverified50%