Embedding Model

RAG（检索增强生成）技术架构将文档切片、向量化、存入向量数据库，查询时先检索相关片段再交给模型生成回答

嵌入模型将文本转换为高维向量时，通常生成768维或1536维的浮点数数组

Vector databases (such as Pinecone, Weaviate, Milvus, etc.) convert text into high-dimensional vectors via embedding models and use cosine similarity or Euclidean distance for approximate nearest neighbor (ANN) search.

向量数据库将文本内容通过嵌入模型（Embedding Model）转化为高维数值向量，检索时通过余弦相似度等算法找出语义最接近的内容

Vector embedding is the process of transforming discrete text, code, or experiences into numerical vectors in a continuous high-dimensional space, so that semantically similar content is closer in vector space.

In code scenarios, slicing by function boundaries preserves more complete semantic units than slicing by fixed character counts.

Embedding models typically compress text into dense vectors of 768 or 1536 dimensions.

If a RAG pipeline returns results that are technically correct but contextually useless, the problem is almost certainly in the chunking or metadata, not the embedding model.

When recall@5 on adversarial questions is low, the correct action is to go back and check the chunking strategy; only after recall@5 meets the bar is it meaningful to invest in optimizing the embedding model or reranker.

The process of vector retrieval is to convert code snippets into high-dimensional vectors through an embedding model, and then return the snippets most relevant to the current task based on semantic similarity.

Small specialized models (such as embedding models) may be more suitable than general-purpose large language models for specific tasks, especially in privacy-sensitive scenarios.

The embedding model's maximum length means that inputs exceeding 512 tokens are silently truncated, so the example sets the chunk budget to 480 tokens.

In a real RAG pipeline, merely swapping the embedding model can make recall jump by 22 percentage points.

A reranker cannot create relevant chunks that were not retrieved; it can only rearrange existing results, and therefore cannot solve problems caused by poor embedding model quality.

Embedding models age with language, and when corpus vocabulary changes (product launches, reorganizations, new regulations), the evaluation scoreboard needs to be rerun.

The embedding model is the encoder component in a RAG pipeline, responsible for mapping query text and document chunks into the same high-dimensional vector space and retrieving relevant document chunks via vector similarity (such as cosine similarity or inner product).

Domain fine-tuning of the embedding model should be considered only when you have thousands of labeled query-chunk pairs and the general-purpose embedding model still fails on terminology despite good chunking.

Embedding models still generate vectors for grammatically broken text that are close to those for normal contract sentences, because they essentially statistically model character co-occurrence patterns rather than checking grammar.

The quality of an embedding model depends heavily on the domain coverage of its training data; models trained on general corpora may perform poorly on specialized texts such as patents, so some patent search platforms use patent corpora for domain fine-tuning.

An embedding model turns any text into a fixed-length list of numbers (a vector).