[KongchangAI]
Concept

SPLINTER

A pre-tokenization reduction method for consonantal scripts (abjads) that reduces vocabulary sparsity by mapping words to their consonantal skeletons.

Timeline (last 90 days)

Nov 17

SPLINTER is a pre-tokenization reduction method for abjads (consonantal writing systems), originally used for languages such as Arabic, whose core idea is to map vocabulary to consonantal skeletons to reduce vocabulary sparsity.

Unverified50%
Nov 17

FIDEL extends the SPLINTER reduction method from abjad scripts to Ethiopian abugida scripts.

Unverified50%

All Facts (2)

Source Articles