Document Chunking
The first step of a RAG pipeline, splitting long documents into segments of a few hundred tokens. Chunking quality directly affects subsequent retrieval and citation, and the clarity of page structure determines chunking quality.
Timeline (last 90 days)
In FAQ and technical documentation scenarios, structure-based chunking can improve retrieval accuracy by 15-30% compared to fixed-length chunking.
In chunk granularity selection, chunks of 128–512 tokens are generally suitable for precise retrieval, chunks of 512–2,048 tokens better preserve context, and retaining 10–20% overlapping text between adjacent chunks can mitigate information loss.
All Facts (2)
In FAQ and technical documentation scenarios, structure-based chunking can improve retrieval accuracy by 15-30% compared to fixed-length chunking.
50%UnverifiedIn chunk granularity selection, chunks of 128–512 tokens are generally suitable for precise retrieval, chunks of 512–2,048 tokens better preserve context, and retaining 10–20% overlapping text between adjacent chunks can mitigate information loss.
50%