MinHash
A locality-sensitive hashing algorithm based on random hashing, used to quickly estimate the Jaccard similarity between sets. It is widely applied in scenarios such as recommendation systems, deduplication, and clustering.
Timeline (last 90 days)
CMinHash is a MinHash algorithm designed through circulant permutation, which can reduce both variance and storage space.
The theoretical basis of MinHash is: if a random permutation is applied to set elements, then the probability that the respective minimum hash values of two sets are equal is exactly equal to their Jaccard similarity.
The Minner estimator is an improved scheme in the MinHash family that further reduces variance by combining multiple minimum hash values, and typically has lower variance than standard MinHash under the same hash budget.
The introduction of circulant permutation makes MinHash more attractive in storage-constrained scenarios (such as embedded devices and large-scale distributed systems).
在文档摄入中，每个文档生成MD5哈希作为document_id，内容哈希用于精确去重，生产环境建议加入MinHash做近似去重，失败文档进入死信队列支持断点续传。
After data deduplication (using locality-sensitive hashing algorithms such as MinHash and SimHash), model convergence speed can improve by 10–20%.
All Facts (6)
CMinHash is a MinHash algorithm designed through circulant permutation, which can reduce both variance and storage space.
50%UnverifiedThe theoretical basis of MinHash is: if a random permutation is applied to set elements, then the probability that the respective minimum hash values of two sets are equal is exactly equal to their Jaccard similarity.
50%UnverifiedThe Minner estimator is an improved scheme in the MinHash family that further reduces variance by combining multiple minimum hash values, and typically has lower variance than standard MinHash under the same hash budget.
50%UnverifiedThe introduction of circulant permutation makes MinHash more attractive in storage-constrained scenarios (such as embedded devices and large-scale distributed systems).
50%Unverified在文档摄入中，每个文档生成MD5哈希作为document_id，内容哈希用于精确去重，生产环境建议加入MinHash做近似去重，失败文档进入死信队列支持断点续传。
50%UnverifiedAfter data deduplication (using locality-sensitive hashing algorithms such as MinHash and SimHash), model convergence speed can improve by 10–20%.
50%