IMA
An AI knowledge base and agent-building tool launched by Tencent, designed to support enterprise private knowledge management and intelligent question-answering scenarios.
Timeline (last 90 days)
The results show that the inductive, data-driven coding scheme achieved higher inter-model agreement (IMA) while maintaining validity.
Inter-model agreement (IMA) draws on ideas from ensemble learning and multi-annotator agreement research, with the core assumption that different LLMs carry different systematic biases due to differences in training data, architecture and fine-tuning strategy.
IMA is typically measured by the proportion of agreement or a statistic (such as Krippendorff's alpha) between pairs of models' annotation results.
The study designed two evaluation paths: external proxy indicators and inter-model agreement (IMA).
Experiments on 100 Israeli parliamentary records show that the data-driven scheme outperforms the theory-driven scheme on IMA without sacrificing validity.
High inter-model agreement does not automatically equal high validity; multiple models may share the same bias.
Inter-model agreement is only a proxy signal of reliability and cannot replace human validity verification; the validity of the data-driven scheme still needs to be tested on more corpora and in more contexts.
All Facts (7)
The results show that the inductive, data-driven coding scheme achieved higher inter-model agreement (IMA) while maintaining validity.
50%UnverifiedInter-model agreement (IMA) draws on ideas from ensemble learning and multi-annotator agreement research, with the core assumption that different LLMs carry different systematic biases due to differences in training data, architecture and fine-tuning strategy.
50%UnverifiedIMA is typically measured by the proportion of agreement or a statistic (such as Krippendorff's alpha) between pairs of models' annotation results.
50%UnverifiedThe study designed two evaluation paths: external proxy indicators and inter-model agreement (IMA).
50%UnverifiedExperiments on 100 Israeli parliamentary records show that the data-driven scheme outperforms the theory-driven scheme on IMA without sacrificing validity.
50%UnverifiedHigh inter-model agreement does not automatically equal high validity; multiple models may share the same bias.
50%UnverifiedInter-model agreement is only a proxy signal of reliability and cannot replace human validity verification; the validity of the data-driven scheme still needs to be tested on more corpora and in more contexts.
50%