Large Language Model

Large language models have an inherent limitation of a knowledge cutoff date, meaning the model only contains information up to the cutoff date of its training data.

Decision models are a specialized architecture distinct from generative language models, with the core goal of outputting structured judgment results for given inputs rather than generating natural language text.

Every inference operation of a large language model service consumes substantial GPU computing power, making capacity planning far more difficult than for ordinary applications.

Because large language models are based on probabilistic sampling, even if instructions are written in the context, the model may still forget or skip certain instructions in complex reasoning chains, which is known as instruction-following instability.

Large language models generate content based on predicting the next most likely token word by word, rather than retrieving answers from a factual database.

大语言模型本质上是一个基于概率的文本预测系统，其核心机制是「下一个词预测」（Next Token Prediction），并不真正理解语义

大型语言模型能够自动化分析恶意代码、生成漏洞利用脚本、模拟社会工程学攻击话术，将原本需要高技能黑客数小时完成的工作压缩至分钟级别

Critics question whether AI's breakthrough represents genuine mathematical reasoning or merely advanced retrieval and recombination of solutions already present in training data.

The training corpora of large language models may include unpublished academic preprints and research notes, raising concerns about data contamination.

Each model, when answering, is not challenged by other models' viewpoints, so it tends to give comprehensive 'safe' answers rather than taking a clear stance.

Large language models fine-tuned with RLHF tend to systematically exhibit an overconfidence bias.

General-purpose AI question-generation tools habitually draw distractors from frequency dictionaries, producing incorrect options that are often more plausible than the correct answer, thereby distorting the assessment objective.

Large language models default to optimizing for plausibility rather than discriminability when generating options, because the training objective is to predict plausible words.

训练大型语言模型需要成千上万块加速芯片协同工作，持续运转数周甚至数月

Large language models are being deployed in scenarios that directly affect individuals' interests, such as customer service, recruitment screening, and credit approval.

As the scale of large language model training clusters expands, HBM demand is growing at an approximate rate of doubling per year.

A large body of research shows that deep learning models are naturally prone to overconfidence, tending to assign confidence levels close to 100% even when answers are wrong, and this phenomenon also exists in large language models.

The emergent capabilities of large language models are regarded by some researchers as signals of general intelligence, while others believe these capabilities may still represent efficient compression and retrieval of implicit patterns in training data.

Large language models can align natural-language driver feedback with structured telemetry data, and machine learning algorithms can identify patterns in high-dimensional parameter spaces that are difficult for humans to detect

Each model invocation consumes both input tokens (prompts and context) and output tokens (the model's generated response), with the two typically priced separately.