GhostInk
A benchmark designed to evaluate whether vision-language models (VLMs) can honestly acknowledge their inability to recognize content in damaged or faded documents. It contains 239 entries spanning categories such as bills, document fields, words, and code, and employs a character-level scoring mechanism that does not require a judge model.
Timeline (last 90 days)
GhostInk benchmark measures not recognition accuracy, but whether a model admits what it cannot read, rather than filling the gap with a plausible-sounding answer.
The GhostInk benchmark contains a total of 239 entries, most of which are everyday Indian documents.
GhostInk's scoring rules label characters as C (clearly visible), U (partially visible), or D (damaged), where FABRICATED (writing out a character that cannot be seen) incurs a -1 penalty even if the guess happens to be correct.
All Facts (3)
GhostInk benchmark measures not recognition accuracy, but whether a model admits what it cannot read, rather than filling the gap with a plausible-sounding answer.
50%UnverifiedThe GhostInk benchmark contains a total of 239 entries, most of which are everyday Indian documents.
50%UnverifiedGhostInk's scoring rules label characters as C (clearly visible), U (partially visible), or D (damaged), where FABRICATED (writing out a character that cannot be seen) incurs a -1 penalty even if the guess happens to be correct.
50%