TinyStories
A dataset of short children's stories written in simple English, designed for training and evaluating small language models. It provides a controlled benchmark environment widely used in language model research to study emergent capabilities and performance of compact architectures.
Timeline (last 90 days)
The publicly released VKORE-46M-NPU is a 46M-parameter, character-level model trained on the TinyStories dataset, and its purpose is to verify the feasibility of the training path rather than model quality.
The benchmark uses llama2.c locked to a specific commit, paired with TinyStories FP32 checkpoints (15M, 42M, 110M parameters); these are small models running on llama2.c rather than the full Llama 2.
The study validates on three datasets—TinyStories, WikiText-103, and CNN/DailyMail—scaling each corpus from 512 chunks to 2,048 chunks, with each chunk consisting of 64 tokens.
All Facts (3)
The publicly released VKORE-46M-NPU is a 46M-parameter, character-level model trained on the TinyStories dataset, and its purpose is to verify the feasibility of the training path rather than model quality.
50%UnverifiedThe benchmark uses llama2.c locked to a specific commit, paired with TinyStories FP32 checkpoints (15M, 42M, 110M parameters); these are small models running on llama2.c rather than the full Llama 2.
50%UnverifiedThe study validates on three datasets—TinyStories, WikiText-103, and CNN/DailyMail—scaling each corpus from 512 chunks to 2,048 chunks, with each chunk consisting of 64 tokens.
50%