Chain-of-Thought Reasoning

Emergent abilities refer to capabilities that appear suddenly when a model's scale exceeds a certain threshold and are absent in smaller-scale models, such as chain-of-thought reasoning.

The task planning process is typically realized through chain-of-thought reasoning and task decomposition, while more advanced implementations adopt a tree-of-thought framework to simultaneously explore multiple planning paths.

Evaluating the per-task cost of large models requires considering both the token unit price and the number of tokens consumed to complete the task; the output token consumption of chain-of-thought reasoning models may increase by several times or even dozens of times.

The training of deep learning models is not a fully interpretable process, and researchers often discover capabilities that were not explicitly designed during post-training evaluations, such as in-context learning, chain-of-thought reasoning, tool-calling tendencies, and so on.

The main industry strategies for addressing hallucinations include RAG, Chain-of-Thought reasoning, and Reinforcement Learning from Human Feedback (RLHF), but no fundamental solution exists yet.