Unverified50% confidenceFactExact time
The Dory model specifications are: a 131,072-token vocabulary, hidden dimension 2560, 16 query heads, 4 KV heads, head dimension 256, about 6T pretraining tokens, English-only support, and training covering 43 programming languages.
1
Sources
50%
Confidence
Long-term
Relevance
10/18/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedIn experiments with 85 million non-embedding parameters and matched training token budgets, unconstrained fitting produced higher held-out average bits-per-byte across all 12 languages under a shared unconstrained decoder.66% similarUnverifiedIn large language models, one English word typically corresponds to 1–2 tokens, one Chinese character typically corresponds to 1.5–2.5 tokens, and GPT-4's single-conversation context window is approximately 128,000 tokens.65% similarUnverifiedOn Odia and Swahili, the back-translation method, using only 1000 parallel sentence pairs, outperforms continued pre-training by 7.7 to 16.4 XCOMET-XXL points in in-domain evaluation.64% similarUnverifiedThe final trained model has approximately 400 million (0.4B) parameters, targets Bengali as the primary language, and uses a corpus of approximately 2 billion tokens.63% similarUnverifiedKeera uses a custom tokenizer with a vocabulary size of 16,384, trained on Python code, at an average of 3.9 characters/token.62% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1063738API
curl https://kongchang.com/api/v1/knowledge/claims/1063738MCP
get_claim(id=1063738)