Meter
An AI safety evaluation organization named by Dario Amodei in his article "We Must Pace the Frontier." Meter is proposed as an embedded third-party evaluator for frontier AI companies, intended to conduct independent safety assessments of advanced AI systems.
Core Facts
Timeline (last 90 days)
In agent development, one should not hard-code model unit prices in the source code, but use environment variables with a default value of 0, because published prices change and an outdated number is worse than no number.
On free models, the dollars property should honestly be 0, but the token count is still real, because what actually runs out first is tokens, the window or rate limits, not money.
OpenAI brought in independent assessment firm Meter to investigate the incident, with investigators relying on AI agents to sort through hundreds of thousands of messages and logs.
Meter analyzed AI pull requests that passed tests and found that approximately half would not be merged by project maintainers.
The only evaluation organization named by Dario in his article is Meter.
Investor David Sachs criticized Meter for not being an independent organization, arguing that it is deeply intertwined with Anthropic's investors and employees.
Meter was incubated out of ARK, which was founded by Paul Cristiano, and has received funding from ARK, the Survival and Flourishing Fund, Open Philanthropy, and FTX.
Paul Cristiano is a trustee of Anthropic's Long-Term Benefit Trust (LTBT), his wife Ajaya works at Meter, and previously oversaw AI safety at Open Philanthropy.
FTX founder Sam Bankman-Fried is an early investor in Anthropic, and FTX provided over one million dollars in funding to Meter.
All Facts (10)
Token billing is calculated based on the total volume of input and output; the longer a conversation, the more each call pays for an increasingly bloated history.
65%UnverifiedOn free models, the dollars property should honestly be 0, but the token count is still real, because what actually runs out first is tokens, the window or rate limits, not money.
50%UnverifiedIn agent development, one should not hard-code model unit prices in the source code, but use environment variables with a default value of 0, because published prices change and an outdated number is worse than no number.
50%UnverifiedMeter analyzed AI pull requests that passed tests and found that approximately half would not be merged by project maintainers.
50%UnverifiedFTX founder Sam Bankman-Fried is an early investor in Anthropic, and FTX provided over one million dollars in funding to Meter.
50%UnverifiedInvestor David Sachs criticized Meter for not being an independent organization, arguing that it is deeply intertwined with Anthropic's investors and employees.
50%UnverifiedPaul Cristiano is a trustee of Anthropic's Long-Term Benefit Trust (LTBT), his wife Ajaya works at Meter, and previously oversaw AI safety at Open Philanthropy.
50%UnverifiedThe only evaluation organization named by Dario in his article is Meter.
50%UnverifiedMeter was incubated out of ARK, which was founded by Paul Cristiano, and has received funding from ARK, the Survival and Flourishing Fund, Open Philanthropy, and FTX.
50%UnverifiedOpenAI brought in independent assessment firm Meter to investigate the incident, with investigators relying on AI agents to sort through hundreds of thousands of messages and logs.
50%