Lakh, Crore and Carpet Area
A mathematical benchmark for large language models designed for Indian home-buying scenarios. It contains 60 deterministic questions covering lakh and crore conversions, area unit conversions, EMI calculations, and Indian numeric formatting conventions in real estate and personal finance contexts.
Timeline (last 90 days)
A benchmark called Lakh, Crore and Carpet Area consists of 60 deterministic questions covering money and real-estate calculations for Indian homebuyers, divided into four groups of 15 questions each: conversion, area, EMI and tax, and formatting.
Each question in the benchmark has a single correct answer derived by Python computation; the scorer requires models to end with ANSWER:<value>, EMI allows a rounding tolerance of ±₹2, and the formatting group requires exact string matching.
All participating models scored 15 out of 15 on area conversion questions (loading percentage, super built-up, gaj conversion, etc.).
The author fixed the benchmark by switching to serial calls, adding retries, and printing "answered: X of 60" before reporting the score, in order to ensure the honesty of benchmark scores.
All Facts (4)
A benchmark called Lakh, Crore and Carpet Area consists of 60 deterministic questions covering money and real-estate calculations for Indian homebuyers, divided into four groups of 15 questions each: conversion, area, EMI and tax, and formatting.
50%UnverifiedEach question in the benchmark has a single correct answer derived by Python computation; the scorer requires models to end with ANSWER:<value>, EMI allows a rounding tolerance of ±₹2, and the formatting group requires exact string matching.
50%UnverifiedAll participating models scored 15 out of 15 on area conversion questions (loading percentage, super built-up, gaj conversion, etc.).
50%UnverifiedThe author fixed the benchmark by switching to serial calls, adding retries, and printing "answered: X of 60" before reporting the score, in order to ensure the honesty of benchmark scores.
50%