CodebaseRAG
An open-source RAG system project built on hexagonal architecture, incorporating golden-set evaluation and CI gates to prevent regressions in retrieval quality.
Timeline (last 90 days)
The CodebaseRAG project was created by Jordi Marsal and is an open-source RAG project.
CodebaseRAG's golden set contains at least 40 public question-answer pairs, each question accompanied by the location of its answer in the codebase.
CodebaseRAG's CI gate measures average recall@5, MRR and nDCG@5 on the golden set, and the build fails when average recall@5 falls below the committed baseline of 0.409.
CodebaseRAG adopts a hexagonal architecture, where query use cases depend on ports, and pgvector, Qdrant, Ollama and Anthropic sit behind the ports as adapters.
The gate uses only recall@5 as a hard threshold, with MRR and nDCG@5 as observational metrics, to avoid ranking fluctuations causing unstable builds.
Moving the baseline (raising or lowering the average recall@5 threshold) requires an explicit ADR decision, rather than being changed just to make CI green.
All Facts (6)
The CodebaseRAG project was created by Jordi Marsal and is an open-source RAG project.
50%UnverifiedCodebaseRAG's golden set contains at least 40 public question-answer pairs, each question accompanied by the location of its answer in the codebase.
50%UnverifiedCodebaseRAG's CI gate measures average recall@5, MRR and nDCG@5 on the golden set, and the build fails when average recall@5 falls below the committed baseline of 0.409.
50%UnverifiedCodebaseRAG adopts a hexagonal architecture, where query use cases depend on ports, and pgvector, Qdrant, Ollama and Anthropic sit behind the ports as adapters.
50%UnverifiedThe gate uses only recall@5 as a hard threshold, with MRR and nDCG@5 as observational metrics, to avoid ranking fluctuations causing unstable builds.
50%UnverifiedMoving the baseline (raising or lowering the average recall@5 threshold) requires an explicit ADR decision, rather than being changed just to make CI green.
50%