Header-Driven Interpretable Semantic Table Parsing: A New Approach to Metadata Quality Assessment

Starting from Column Headers: The Hidden Bottleneck in Knowledge Graph Quality

Knowledge graph (KG) quality is often attributed to downstream graph validation steps, but an earlier and more subtle problem tends to go unnoticed — long before data is integrated, the quality of tabular metadata itself determines success or failure. When cell data is missing, too noisy, or simply unavailable, column headers become the primary signal for semantic interpretation.

This arXiv paper proposes a header-centric, interpretable framework specifically designed for metadata-only scenarios in Semantic Table Interpretation (STI). It addresses two tasks simultaneously: Column Type Annotation (CTA) and Data Quality Assessment (DQA). The key insight is that it shifts quality control from the tail end of the knowledge graph pipeline to the very front of data preparation.

Semantic Table Interpretation (STI) is the process of mapping columns, cells, and rows in semi-structured tables to knowledge base entities and properties. Its main subtasks include Column Type Annotation (CTA, mapping columns to ontology types), Column Property Annotation (CPA, identifying semantic relationships between columns), and Cell Entity Annotation (CEA, linking cell values to specific entities). Traditional STI methods rely heavily on cell content for string matching or entity linking. In metadata-only scenarios, however, cell data is often unavailable due to privacy protection, data licensing restrictions, or collection-stage limitations — leaving column headers as the sole semantic signal. This constraint renders all content-based methods ineffective, forcing systems to extract type information from the natural language vocabulary of headers while keeping decision processes auditable.

How the Framework Works: From Headers to Traceable Types

39 Interpretable Types and Token-Level Provenance

The core mechanism maps column headers to 39 interpretable FinalFormat types. This mapping relies on carefully curated lexical resources rather than black-box models. Crucially, it preserves token-level traceability through SourceKeywords — meaning every type assignment can be traced back to the specific word in the header that triggered it.

This interpretability is especially valuable in knowledge graph preparation contexts. Compared to having a large language model produce an annotation that is difficult to audit, this rule- and lexicon-based approach makes every decision transparent and verifiable, facilitating human review and accountability.

Quality Issue Detection and the HeadersIQ Metric

Each assigned type activates a set of validation rules grounded in a taxonomy of Data Quality Issues (DQIs). The framework detects a range of common problems:

• Missing data
• Duplicates
• Domain violations
• Wrong data types
• Temporal mismatches

These detection results are aggregated into HeadersIQ — a lightweight, unweighted, data-source-level quality indicator. Rather than providing fine-grained per-column scores, it offers a rapid health profile of an entire data source, making it well-suited for large-scale metadata monitoring.

Systematic classification of data quality dimensions typically follows standards such as ISO 25012, covering dimensions like completeness, accuracy, consistency, and timeliness. In tabular data contexts, a "domain violation" refers to column values falling outside the range permitted by the column's semantic type — for example, an "age" column containing negative numbers or integers with more than three digits. A "temporal mismatch" refers to timestamps or date values that are logically inconsistent with the current context or with other time-related columns in the same table. HeadersIQ consolidates detection results across these issue types into a single metric. Its unweighted design deliberately avoids subjective prioritization among different quality issues, making it more suitable as a standardized tool for comparing data source health across datasets rather than for fine-tuning individual datasets.

Large-Scale Evaluation: Over 120,000 Column Headers

The validation scale is substantial. The research team evaluated the framework across multiple heterogeneous benchmarks, including UCI, Prague, Kaggle, VizNet/Sato, SOTAB, T2Dv2, and the SemTab 2024 Metadata-to-KG track, totaling approximately 120,000 column headers.

Results demonstrate broad practical coverage on noisy, real-world metadata. A parallel KG mapping pathway supports alignment to both DBpedia and Schema.org, enabling seamless integration into knowledge graph ecosystems.

Honest Performance Reporting: Official Scores and Diagnostic Audits

One commendable aspect of this paper is its candor about evaluation results. On the SemTab 2024 Metadata-to-KG track, the official GT-strict (ground truth strict) evaluation scores were only moderate.

Rather than leaving it there, the researchers conducted a blinded diagnostic audit. The results revealed that many apparent "errors" stemmed from issues with the benchmark itself — granularity mismatches, aliasing, and ontology selection effects — rather than genuinely unreasonable predictions from the framework.

The researchers explicitly present this audit as diagnostic evidence of disagreement patterns, not as a revision of benchmark performance. This careful distinction is rigorous and restrained, avoiding self-serving rationalization of scores, and instead highlights inherent limitations in benchmark evaluation — a valuable reminder for the entire STI community.

SemTab (Semantic Web Challenge on Tabular Data to Knowledge Graph Matching) is the most representative evaluation track in semantic table interpretation, held annually alongside the International Semantic Web Conference (ISWC). Its evaluation uses two scoring modes: GT-strict (strict ground truth), which requires predicted types to exactly match benchmark annotations, and GT-relaxed (relaxed ground truth), which tolerates some ancestor-descendant deviation within ontology hierarchies. The "aliasing effect" refers to the same concept having multiple equivalent URIs or labels across different ontologies, causing semantically correct predictions to be marked as errors. "Granularity mismatch" refers to benchmark annotations selecting a coarser or finer level in an ontology type hierarchy than the system's prediction, where both are semantically valid but textually non-matching. These are long-standing methodological challenges in STI benchmark evaluation.

Practical Impact: A Reusable Workflow

Taken together, this work delivers more than an annotation tool — it provides a complete, reusable workflow serving three purposes:

1. Metadata-driven semantic annotation — column type identification without cell data;
2. Data source-level quality monitoring — lightweight health assessment via HeadersIQ;
3. KG-oriented benchmark diagnostics — exposing systematic biases in existing benchmarks around granularity, aliasing, and ontology selection.

For teams working on knowledge graph construction, data governance, and enterprise metadata management, this interpretable, traceable, and scalable approach offers a pragmatic path to enforcing quality standards before data ever enters a graph. It serves as a reminder: before pursuing complex downstream modeling, getting the fundamentals of metadata quality right often delivers the greatest returns.