[KongchangAI]
BenchmarkPyrite基准

Pyrite

A pairwise conformation ranking benchmark for evaluating the accuracy of protein-ligand conformation ranking, comprising 1,500 conformation pairs.

Timeline (last 90 days)

Nov 11

The Pyrite benchmark contains 1,500 protein-ligand conformation pairs, each requiring a judgment of which of two candidate poses is closer to the true binding mode.

Unverified50%
Nov 11

On the Pyrite benchmark, expert chemists equipped with 3D visualization tools and physics-based scoring ranked conformation pairs close to random guessing.

Unverified50%
Nov 11

PyriteReasoner is an LLM agent that directly compares conformation pairs through text-based reasoning.

Unverified50%
Nov 11

PyriteFeaturizer is an LLM agent that generates executable code to build features for logistic regression, optionally incorporating features derived from LLM sub-calls.

Unverified50%
Nov 11

In the Pyrite benchmark evaluation, PyriteReasoner and PyriteFeaturizer achieved ranking accuracies of 67% and 74%, respectively.

Unverified50%
Nov 11

Both LLM agents, PyriteReasoner and PyriteFeaturizer, exceeded the observed expert performance in conformation ranking accuracy.

Unverified50%
Nov 11

In the Pyrite study, GRPO was used to improve open-weight models' performance on conformation ranking, enabling them to match or even surpass closed-source frontier models.

Unverified50%
Nov 11

One of the conclusions from the Pyrite study is that LLM-guided feature discovery generally outperforms direct LLM reasoning.

Unverified50%

All Facts (8)

Source Articles