GPT-3
A large-scale language model released by OpenAI in 2020 with 175 billion parameters. It formally established the zero-shot, one-shot, and few-shot inference paradigms and laid the conceptual foundation for prompt engineering as a mode of human–AI interaction.
Core Facts
The context window in the GPT-3 era was only 2,048 tokens, whereas Google Gemini already supports millions of tokens, Claude supports 200K tokens, and GPT-4 Turbo supports 128K tokens.
GPT-3拥有1750亿参数
OpenAI Codex was originally released in 2021 as a code generation model fine-tuned on GPT-3 and was the core engine behind GitHub Copilot
Timeline (last 90 days)
Model costs have fallen rapidly since GPT-3, until Google began raising prices for the Gemini 2.0 generation in 2025, with OpenAI following suit afterward.
Three key differences between ChatGPT and GPT-3 are: conversational interaction eliminating the ML and software barrier, the viral spread of screenshots, and the free version lowering the barrier to entry.
The inference paradigm adopted by TabFM is In-Context Learning (ICL), a concept first systematically demonstrated in GPT-3.
Taking GPT-3's attention layers as an example, the original weight matrix is approximately 12,288×12,288 (about 150 million parameters), while a LoRA decomposition with rank r=8 requires training only about 200,000 parameters, a compression ratio exceeding 700×.
According to research estimates, training GPT-3 consumed approximately 700,000 liters of fresh water, equivalent to the water usage for manufacturing about 370 automobiles.
After in-context learning capabilities were observed in large models such as GPT-3, ICL rapidly became a research hotspot.
The conclusions of the Chinchilla scaling law challenged the prevailing practice of the GPT-3 era, in which the industry invested most compute into scaling model size while training data was relatively insufficient.
During the legislative period of the EU AI Act, large language models underwent a generational leap from GPT-3 to GPT-4 and then to multimodal systems.
GPT-3 has officially been discontinued (retired).
Switching GPT-3 models often entails a full regression test and prompt rewriting effort, rather than simply swapping an API endpoint string.
37 more timeline events
All Facts (20)
The context window in the GPT-3 era was only 2,048 tokens, whereas Google Gemini already supports millions of tokens, Claude supports 200K tokens, and GPT-4 Turbo supports 128K tokens.
90%VerifiedGPT-3拥有1750亿参数
90%VerifiedOpenAI Codex was originally released in 2021 as a code generation model fine-tuned on GPT-3 and was the core engine behind GitHub Copilot
90%VerifiedGitHub Copilot is built on OpenAI's Codex model, which is a code-specialized fine-tuned version of GPT-3 trained on tens of billions of lines of public code.
90%VerifiedThe in-context learning capability of Few-Shot prompting was first systematically described in the GPT-3 paper.
90%Verified早期GPT-3的上下文窗口仅有4K Token，而现代模型已扩展至128K甚至更长
90%Verified训练一次GPT-4级别的模型需要数万块A100/H100 GPU协同工作数月
90%VerifiedGPT-4 was launched in 2023 with an estimated parameter scale exceeding one trillion.
90%VerifiedThe context window of the early GPT-3 was only 4,096 tokens, approximately 3,000 English words.
90%VerifiedThe computing power required to train a GPT-4-level model is approximately 100 times greater than that required for GPT-3.
85%VerifiedGPT-3 has 175 billion parameters and demonstrated emergent capabilities (zero-shot or few-shot reasoning) in 2020.
80%VerifiedThe FLOPs required to train a GPT-4-scale model reach the order of 10²⁴ to 10²⁵.
80%VerifiedOpenAI has not disclosed complete technical details since GPT-3, and GPT-4 strictly keeps key information such as model architecture, parameter scale, and training data confidential
80%VerifiedThe early GPT-3 had a context window of only 2,048 tokens.
80%VerifiedGPT-3's training data contains hundreds of billions of tokens of internet text.
80%VerifiedIn 2020, OpenAI released the GPT-3 API.
80%VerifiedThe context windows of current mainstream large language models (such as GPT-4, Claude, etc.) have been expanded to 128K or even longer tokens.
80%VerifiedAgentic AI represents the third stage of large language model application evolution: from single-turn Q&A to multi-turn conversational assistants, and then to the agent paradigm capable of autonomous planning, calling external tools, and continuously executing tasks.
80%VerifiedCodex is a large language model based on the GPT architecture, fine-tuned specifically for code understanding and generation tasks.
80%VerifiedThe original Codex was released in 2021, fine-tuned based on GPT-3, and was once the underlying engine of GitHub Copilot.
80%