[KongchangAI]
BenchmarkBharatSynthQA数据集

BharatSynthQA

A passage-grounded question-answering dataset on Indian temples and religious heritage. It contains boolean, factoid and explanatory question-answer pairs, and is used to evaluate LLMs' domain knowledge of cultural heritage and the effectiveness of RAG.

Timeline (last 90 days)

Nov 10

The research paper "BharatSynthQA: A Context-Grounded Question Answering Dataset on Indian Temples and Religious Heritage" points out that existing models have weak knowledge of the Indian temple domain.

Unverified50%
Nov 10

The BharatSynthQA dataset construction pipeline includes: obtaining seed topics from authoritative heritage sources, crawling Wikipedia articles, filtering by topic relevance and Indian locations, and then using a large language model with few-shot prompting to generate question-answer pairs.

Unverified50%
Nov 10

The BharatSynthQA dataset ultimately retained 98,677 of 131,494 generated question-answer pairs.

Unverified50%
Nov 10

The BharatSynthQA dataset separately released 300 manually annotated question-answer pairs as a reference set.

Unverified50%
Nov 10

Manual evaluation of a subsample of the BharatSynthQA dataset showed that 89.83% of the questions were correctly constructed.

Unverified50%
Nov 10

The BharatSynthQA dataset has been made public on Hugging Face at https://huggingface.co/datasets/theshivam7/bharatsynth-qa.

Unverified50%
Nov 10

The BharatSynthQA dataset is a passage-based question answering dataset focused on Indian temples and covering other related religious sites.

Unverified50%

All Facts (7)

Source Articles