OLMo-Twin
A controlled experiment project released by AI2 that reruns OLMo 2 training with matched document pairs inserted, in order to study the causal effect of membership inference.
Timeline (last 90 days)
The research team reran the official 50-billion-token mid-training phase of OLMo 2 1B, replacing selected training data chunks with documents from 7,814 matched document pairs, where for each pair a coin flip decided which document was inserted into training.
The research team commits to releasing checkpoints, per-token log probabilities and evaluation documents, providing a controlled benchmark for subsequent MI research.
All Facts (2)
The research team commits to releasing checkpoints, per-token log probabilities and evaluation documents, providing a controlled benchmark for subsequent MI research.
50%UnverifiedThe research team reran the official 50-billion-token mid-training phase of OLMo 2 1B, replacing selected training data chunks with documents from 7,814 matched document pairs, where for each pair a coin flip decided which document was inserted into training.
50%