BharatSynthQA
A passage-grounded question-answering dataset on Indian temples and religious heritage. It contains boolean, factoid and explanatory question-answer pairs, and is used to evaluate LLMs' domain knowledge of cultural heritage and the effectiveness of RAG.
Timeline (last 90 days)
The research paper "BharatSynthQA: A Context-Grounded Question Answering Dataset on Indian Temples and Religious Heritage" points out that existing models have weak knowledge of the Indian temple domain.
The BharatSynthQA dataset construction pipeline includes: obtaining seed topics from authoritative heritage sources, crawling Wikipedia articles, filtering by topic relevance and Indian locations, and then using a large language model with few-shot prompting to generate question-answer pairs.
The BharatSynthQA dataset ultimately retained 98,677 of 131,494 generated question-answer pairs.
The BharatSynthQA dataset separately released 300 manually annotated question-answer pairs as a reference set.
Manual evaluation of a subsample of the BharatSynthQA dataset showed that 89.83% of the questions were correctly constructed.
The BharatSynthQA dataset has been made public on Hugging Face at https://huggingface.co/datasets/theshivam7/bharatsynth-qa.
The BharatSynthQA dataset is a passage-based question answering dataset focused on Indian temples and covering other related religious sites.
All Facts (7)
The BharatSynthQA dataset separately released 300 manually annotated question-answer pairs as a reference set.
50%UnverifiedThe BharatSynthQA dataset construction pipeline includes: obtaining seed topics from authoritative heritage sources, crawling Wikipedia articles, filtering by topic relevance and Indian locations, and then using a large language model with few-shot prompting to generate question-answer pairs.
50%UnverifiedThe BharatSynthQA dataset ultimately retained 98,677 of 131,494 generated question-answer pairs.
50%UnverifiedThe research paper "BharatSynthQA: A Context-Grounded Question Answering Dataset on Indian Temples and Religious Heritage" points out that existing models have weak knowledge of the Indian temple domain.
50%UnverifiedManual evaluation of a subsample of the BharatSynthQA dataset showed that 89.83% of the questions were correctly constructed.
50%UnverifiedThe BharatSynthQA dataset is a passage-based question answering dataset focused on Indian temples and covering other related religious sites.
50%UnverifiedThe BharatSynthQA dataset has been made public on Hugging Face at https://huggingface.co/datasets/theshivam7/bharatsynth-qa.
50%