[KongchangAI]
ConceptFleiss' kappa

Fleiss' kappa

A statistical measure of agreement among multiple annotators, applicable to settings involving more than two annotators.

Timeline (last 90 days)

Nov 11

Three annotators, following a written guideline, reached a Fleiss' kappa of 0.788 agreement on 816 data items.

Unverified50%
Nov 10

Fleiss' κ values typically range from 0 to 1, where 0 indicates agreement equivalent to random guessing and 1 indicates complete agreement; academia generally regards 0.61–0.80 as strong agreement and above 0.81 as very strong agreement.

Unverified50%
Nov 6

For annotation consistency, Cohen's kappa was 0.781 (range 0.753-0.807) and Fleiss' kappa was 0.77.

Unverified50%
Oct 26

PGT-FG uses Fleiss' kappa to measure inter-annotator agreement on the shared subset, with a kappa value of 0.6521 for the overall toxicity label.

Unverified50%
Oct 26

Fleiss' kappa is a statistical metric used to measure the degree of agreement among multiple annotators, applicable to scenarios where three or more annotators assign categories to the same set of samples.

Unverified50%

All Facts (5)

Source Articles