GPT-3.5

Over the past two years the cost of running GDP-3.5-level capability models has fallen by about 280 times; according to Stanford HAI's AI Index data, a task that cost 1 dollar in late 2022 now costs less than one cent.

不同模型（如 GPT-4、GPT-3.5、text-embedding-ada-002）使用的是不同的 tokenizer，同一段文本在不同模型下的 token 数量可能不同。

Experiments based on early GPT-3.5 revealed the existence of "critical pivot tokens"—positions at which the model's answer uncertainty suddenly collapses and locks onto a particular result.

The capability boundary of the local 27B model is roughly equivalent to the level between early GPT-3.5 and GPT-4.

GPT-3.5 exhibits notable volatility in robustness tests, performing comparably to other models in some scenarios while lagging significantly behind in others.

After Meta released the Llama series in 2023, developers for the first time could run a model approaching GPT-3.5 levels entirely on a local machine or even a consumer-grade GPU.

AutoGPT's success relies heavily on the powerful capabilities of GPT-4; when using GPT-3.5 or open-source models, issues such as infinite loops and goal deviation frequently occur.

A parameter scale of 30B is large enough to exhibit capability levels close to GPT-3.5, yet small enough to run on high-end consumer graphics cards (such as the NVIDIA RTX 4090) or mid-scale servers.

The 7B version of the Qwen model can run on consumer-grade graphics cards but has limited capability; the 70B version approaches GPT-3.5 but requires professional hardware; Qwen-Max reaches the hundred-billion parameter scale.

The parameter count of frontier models has jumped from GPT-3's 175 billion to the trillion-parameter level, and they are typically based on the Transformer architecture.