SAGE
A Schema-guided Aspectized Grant Evaluation system that structures scoring criteria into actionable checklist items and binds judgments to evidence from application materials. It generates auditable drafts for human expert review, enabling a human-AI collaborative professional review workflow.
Timeline (last 90 days)
The full name of SAGE is Schema-Guided Aspect-Based Grant Evaluation, and its core idea is to translate the scoring criteria of grant review into a series of structured checklist items and link each judgment to specific evidence in the application materials.
SAGE was evaluated on 35 grant applications from nonprofit organizations.
In the Phase 1 post-hoc comparison, SAGE's evaluation results were compared against 105 human reviews from the original competition, yielding an ordinal consistency of kappa = 0.29, which falls in the "fair" range.
In the Phase 2 human-AI collaborative assisted re-review mode, the foundation produced 202 evaluations through criterion-by-criterion re-assessment, and SAGE's consistency improved to kappa = 0.58, reaching a moderate-to-high level.
As a comparison baseline, the simple "one-prompt-per-criterion" approach achieved only kappa = 0.33 on the common subset, whereas SAGE achieved 0.58.
SAGE outperforms the simple baseline on rank correlation and has a lower error rate.
SAGE introduces a claim-level audit mechanism that labels each judgment in the structured draft as "confirmed," "disputed," or "unaddressed."
SAGE does not attempt to replace human reviewers, but rather generates a detailed, evidence-linked, auditable draft for expert revision.
All Facts (8)
The full name of SAGE is Schema-Guided Aspect-Based Grant Evaluation, and its core idea is to translate the scoring criteria of grant review into a series of structured checklist items and link each judgment to specific evidence in the application materials.
50%UnverifiedSAGE was evaluated on 35 grant applications from nonprofit organizations.
50%UnverifiedIn the Phase 1 post-hoc comparison, SAGE's evaluation results were compared against 105 human reviews from the original competition, yielding an ordinal consistency of kappa = 0.29, which falls in the "fair" range.
50%UnverifiedIn the Phase 2 human-AI collaborative assisted re-review mode, the foundation produced 202 evaluations through criterion-by-criterion re-assessment, and SAGE's consistency improved to kappa = 0.58, reaching a moderate-to-high level.
50%UnverifiedAs a comparison baseline, the simple "one-prompt-per-criterion" approach achieved only kappa = 0.33 on the common subset, whereas SAGE achieved 0.58.
50%UnverifiedSAGE outperforms the simple baseline on rank correlation and has a lower error rate.
50%UnverifiedSAGE introduces a claim-level audit mechanism that labels each judgment in the structured draft as "confirmed," "disputed," or "unaddressed."
50%UnverifiedSAGE does not attempt to replace human reviewers, but rather generates a detailed, evidence-linked, auditable draft for expert revision.
50%