Llama-3.1-8B
An 8-billion-parameter instruction-tuned large language model in Meta's Llama 3.1 series, widely used for inference and evaluation.
Core Facts
Timeline (last 90 days)
The framework was applied to six 7-9B instruction-tuned models (three general-purpose: Mistral-7B, Phi-3.5-mini, Llama-3.1-8B; three medically adapted: BioMistral-7B, OpenBioLLM-Llama3-8B, Med42-Llama3-8B), tested on 200 real patient questions from the K-QA benchmark.
Full Llama-3.1-8B decodes in 2.8 milliseconds per step at batch 1 using FuTransformer, compared with 8.1 milliseconds for the compiled transformers implementation.
The Hugging Face API can only list a model's direct children, 1,000 per page; finding all 21,639 descendants of Llama-3.1-8B this way requires 21,653 calls, which at 0.61 seconds per call takes about 3.7 hours, whereas FalkorDB returns the same result in just 70 milliseconds.
For cases where the base model is undeclared, FinePrint introduces a constrained LLM agent (the deepseek-ai/DeepSeek-R1-Distill-Llama-8B case), which reads the model card, quotes verbatim statements such as "DeepSeek-R1-Distill-Llama-8B is derived from Llama3.1-8B-Base," finds meta-llama/Llama-3.1-8B in the graph, and writes a typed edge with source and citation for the rule engine to use.
CARE is currently validated only on four LIBERO suites and two agents (Qwen3.5-9B and Llama-3.1-8B).
Taking Llama-3.1-8B as an example, each token has a hidden state vector of dimension 4096 at each layer, encoding the semantic, syntactic, and even pragmatic information of that token in the current context.
Under the same conditions of Llama-3.1-8B, batch size 32, and context 2048, vLLM achieves about 450 tok/s, llama.cpp about 280 tok/s, and Ollama about 120 tok/s.
By offloading frozen layers to system memory via layer offloading (offload_layers), the 4-bit training of Llama-3.1-8B can reduce the VRAM upper limit to 7 GiB.
This rotation-decay geometric phenomenon has been reproduced on three models: Llama-3.1-8B, Qwen3-8B, and Gemma-4-E4B.
All Facts (10)
以LLaMA-3 70B（80层、64头、每头128维）为例，单个token的KV Cache约占2.6MB（FP16精度），32K上下文的KV Cache总量约达83GB
80%UnverifiedThe framework was applied to six 7-9B instruction-tuned models (three general-purpose: Mistral-7B, Phi-3.5-mini, Llama-3.1-8B; three medically adapted: BioMistral-7B, OpenBioLLM-Llama3-8B, Med42-Llama3-8B), tested on 200 real patient questions from the K-QA benchmark.
50%UnverifiedFull Llama-3.1-8B decodes in 2.8 milliseconds per step at batch 1 using FuTransformer, compared with 8.1 milliseconds for the compiled transformers implementation.
50%UnverifiedThe Hugging Face API can only list a model's direct children, 1,000 per page; finding all 21,639 descendants of Llama-3.1-8B this way requires 21,653 calls, which at 0.61 seconds per call takes about 3.7 hours, whereas FalkorDB returns the same result in just 70 milliseconds.
50%UnverifiedFor cases where the base model is undeclared, FinePrint introduces a constrained LLM agent (the deepseek-ai/DeepSeek-R1-Distill-Llama-8B case), which reads the model card, quotes verbatim statements such as "DeepSeek-R1-Distill-Llama-8B is derived from Llama3.1-8B-Base," finds meta-llama/Llama-3.1-8B in the graph, and writes a typed edge with source and citation for the rule engine to use.
50%UnverifiedCARE is currently validated only on four LIBERO suites and two agents (Qwen3.5-9B and Llama-3.1-8B).
50%UnverifiedTaking Llama-3.1-8B as an example, each token has a hidden state vector of dimension 4096 at each layer, encoding the semantic, syntactic, and even pragmatic information of that token in the current context.
50%UnverifiedUnder the same conditions of Llama-3.1-8B, batch size 32, and context 2048, vLLM achieves about 450 tok/s, llama.cpp about 280 tok/s, and Ollama about 120 tok/s.
50%UnverifiedBy offloading frozen layers to system memory via layer offloading (offload_layers), the 4-bit training of Llama-3.1-8B can reduce the VRAM upper limit to 7 GiB.
50%UnverifiedThis rotation-decay geometric phenomenon has been reproduced on three models: Llama-3.1-8B, Qwen3-8B, and Gemma-4-E4B.
50%