Person
Rico Sennrich
A computational linguist and researcher at the University of Edinburgh who in 2016 proposed applying the BPE algorithm to subword tokenization for neural machine translation, making him one of the founders of subword tokenization methods.
Timeline (last 90 days)
Oct 13
Rico Sennrich et al. quantified in their 2016 paper "Neural Machine Translation of Rare Words with Subword Units": on a 100-million-word German corpus, keeping whole words requires a vocabulary of 1.75 million entries, and the test set still leaves 1,079 unknown words; splitting into characters requires about 3,000 symbols, but the corpus balloons to 550 million positions; the subword method BPE uses about 63,000 symbols, 112 million positions, and zero unknown words.
Unverified50%