VisKG-LM
A knowledge-graph-augmented question-answering model that renders subgraphs as images and caches them offline, avoiding repeated graph-structure encoding at inference time. This decouples graph encoding from language reasoning, enabling the model to achieve performance comparable to 7B-parameter vision-language models using approximately 400M online parameters.
Timeline (last 90 days)
VisKG-LM decouples knowledge graph encoding from language reasoning by offline-compiling retrieved subgraphs once and caching them as read-only visual memory for reuse, thereby eliminating repeated graph propagation computation during the online phase.
VisKG-LM first serializes the retrieved candidate subgraphs into Relation-Labeled Paths, then renders these paths into two-dimensional images that preserve the branching structure.
VisKG-LM achieves improvements of 1.2, 0.8, and 4.3 points over the GreaseLM baseline on the three benchmarks CommonsenseQA, OpenBookQA, and MedQA-USMLE, respectively.
VisKG-LM, using only approximately 400M active parameters, matches or surpasses the 7B-parameter vision-language model GraphVis.
VisKG-LM outperforms a text-only control model that receives the same relation-labeled path text by 4.2, 6.5, and 5.1 points on the three benchmarks, respectively.
The improvement of VisKG-LM on MedQA-USMLE is particularly significant, partly because medical knowledge graphs have high relation density and long reasoning chains, making them well-suited for visual 2D layouts that preserve multi-path crossing structures.
VisKG-LM is an arXiv preprint, and its generalization capability, performance on larger-scale knowledge graphs, and image rendering cost still await further validation.
All Facts (7)
VisKG-LM decouples knowledge graph encoding from language reasoning by offline-compiling retrieved subgraphs once and caching them as read-only visual memory for reuse, thereby eliminating repeated graph propagation computation during the online phase.
50%UnverifiedVisKG-LM first serializes the retrieved candidate subgraphs into Relation-Labeled Paths, then renders these paths into two-dimensional images that preserve the branching structure.
50%UnverifiedVisKG-LM achieves improvements of 1.2, 0.8, and 4.3 points over the GreaseLM baseline on the three benchmarks CommonsenseQA, OpenBookQA, and MedQA-USMLE, respectively.
50%UnverifiedVisKG-LM, using only approximately 400M active parameters, matches or surpasses the 7B-parameter vision-language model GraphVis.
50%UnverifiedVisKG-LM outperforms a text-only control model that receives the same relation-labeled path text by 4.2, 6.5, and 5.1 points on the three benchmarks, respectively.
50%UnverifiedThe improvement of VisKG-LM on MedQA-USMLE is particularly significant, partly because medical knowledge graphs have high relation density and long reasoning chains, making them well-suited for visual 2D layouts that preserve multi-path crossing structures.
50%UnverifiedVisKG-LM is an arXiv preprint, and its generalization capability, performance on larger-scale knowledge graphs, and image rendering cost still await further validation.
50%