EleutherAI
A nonprofit organization focused on open AI research, releasing language models, training datasets, and training logs in a fully open manner to advance the democratization of AI research.
Core Facts
Timeline (last 90 days)
In the whatmodel.sh data, 14,436 results come from independent evaluations (ACL Anthology, arXiv, Epoch AI, LMArena, EvalPlus, EleutherAI harness, etc.) and 7,677 from lab self-reports, with independent evaluations accounting for about 77%.
Pythia is a series of standard GPT-NeoX-style Transformer models released by EleutherAI, with a fully public and transparent training process.
Open-source communities represented by EleutherAI and Hugging Face emphasize that Security through Obscurity has historically been a fragile strategy.
Organizations such as EleutherAI and Allen AI pursue a fully open approach, releasing even training logs and datasets.
Publicly recognized evaluation frameworks in the community include LMSYS Chatbot Arena, EleutherAI's LM Evaluation Harness, and the HuggingFace Open LLM Leaderboard.
Eval Harness was first widely used in model benchmarking, such as EleutherAI's lm-evaluation-harness.
EleutherAI developed the lm-evaluation-harness standardized evaluation framework, attempting to reduce measurement noise by unifying testing procedures
The Pile is an approximately 825 GB open-source text dataset created by EleutherAI, consisting of 22 sub-datasets covering academic papers, code, web pages, books, Wikipedia, and more.
EleutherAI created The Pile—an 800 GB high-quality training dataset.
EleutherAI has successively released open-source models including GPT-Neo, GPT-J-6B, and GPT-NeoX-20B.
3 more timeline events
All Facts (16)
EleutherAI's lm-evaluation-harness is the most well-known reference implementation of an Eval Harness in the large model field, providing standardized benchmark testing capabilities for language models.
75%UnverifiedMeta的LLaMA官方实现采用非交错格式（half-half格式），GPT-NeoX/EleutherAI的实现采用交错格式
90%UnverifiedEleutherAI的Pythia系列和BLOOM是达到真正开源标准（包括训练数据、训练代码、模型架构和权重全面公开）的少数AI项目代表
80%UnverifiedEleutherAI's Language Model Evaluation Harness focuses on capability benchmark testing
70%UnverifiedThe Pile is an approximately 825 GB open-source text dataset created by EleutherAI, consisting of 22 sub-datasets covering academic papers, code, web pages, books, Wikipedia, and more.
60%UnverifiedIn the whatmodel.sh data, 14,436 results come from independent evaluations (ACL Anthology, arXiv, Epoch AI, LMArena, EvalPlus, EleutherAI harness, etc.) and 7,677 from lab self-reports, with independent evaluations accounting for about 77%.
50%UnverifiedPythia is a series of standard GPT-NeoX-style Transformer models released by EleutherAI, with a fully public and transparent training process.
50%UnverifiedOpen-source communities represented by EleutherAI and Hugging Face emphasize that Security through Obscurity has historically been a fragile strategy.
50%UnverifiedOrganizations such as EleutherAI and Allen AI pursue a fully open approach, releasing even training logs and datasets.
50%UnverifiedPublicly recognized evaluation frameworks in the community include LMSYS Chatbot Arena, EleutherAI's LM Evaluation Harness, and the HuggingFace Open LLM Leaderboard.
50%UnverifiedEval Harness was first widely used in model benchmarking, such as EleutherAI's lm-evaluation-harness.
50%UnverifiedEleutherAI developed the lm-evaluation-harness standardized evaluation framework, attempting to reduce measurement noise by unifying testing procedures
50%UnverifiedEleutherAI is a decentralized AI research organization spontaneously formed in 2020 by a group of independent researchers on Discord.
50%UnverifiedEleutherAI created The Pile—an 800 GB high-quality training dataset.
50%UnverifiedEleutherAI等开源研究社区聚集了大量独立研究者进行协作
50%UnverifiedEleutherAI has successively released open-source models including GPT-Neo, GPT-J-6B, and GPT-NeoX-20B.
50%