We Tested 20 of the Most Popular Agent Skills: The #1 Most-Installed One Actually Hurts Performance

Does a high install count mean a skill is actually good? Someone ran a controlled experiment on the 20 most-installed Agent skills: the same batch of tasks, each run once with the skill loaded and once without, then compared answer quality. The conclusion is a bit counterintuitive — the #1 most-installed skill, find-skills, made answers worse on both models.

This test comes from Driftproof (an open-source evaluation tool; the author is also the tester, and the conflict of interest is disclosed in the article), with data based on a skills.sh snapshot from October 17. Claude Sonnet 5.5 and Claude Opus 5.5 served as the answering models, Claude Haiku 5.5 as the judge, producing 40 result sets in total.

Overall Scoreboard: More Than Half the Skills Genuinely Help

Start with the big picture. Of the 20 skills, 11 showed "clearly helpful" results on both models.

Result	Sonnet 5.5	Opus 5.5
Clearly helpful	12	12
Clearly worse	4	2
Insufficient samples to judge	4	6

This distribution shows that most high-install skills aren't just "installed for peace of mind" — on pure text tasks they really do change output quality. But it also shows that the install leaderboard contains items that drag performance down, and the higher the ranking, the more glaring the problem.

The Ones That Hurt: The Top-Ranked Skill Fails

The most closely watched failure case is find-skills (vercel-labs/skills). It's #1 by install count, yet it was judged "clearly worse" on both models. Both failures occurred on the same search-request type task: with the skill loaded the score was about 0.75, while without it the score was 0.92 to 0.95.

The test boundary needs to be made clear here. find-skills is essentially a router that points the Agent toward other skills. The tester scored the answer it wrote itself, not the effect of the skills it handed off to. In other words, this is more like "the router's description text interferes with direct answering" rather than the routing function itself failing.

Other skills judged to be dragging things down:

• reddit-automation (flowkit-labs/skills): worse on both models, affecting a ranking task and, on Sonnet, a reply task.
• grill-with-docs and hyperframes: clearly worse on Sonnet only.

The same skill performing differently across models shows there's a compatibility issue between skills and models — drawing conclusions from a single model can easily lead to misjudgment.

The Biggest Gains: Doubling Scores Is No Exaggeration

The positive cases are just as dramatic:

• hyperframes-registry (heygen-com/hyperframes): from 0.43 to 0.98 on Sonnet, and from 0.62 to 0.97 on Opus.
• prototype (mattpocock/skills): from 0.36 to 0.93 on Opus.
• triage (mattpocock/skills): a perfect 1.00 on both models, versus about 0.7 without the skill.

triage scoring 1.00 on both models is the most stable positive signal in this dataset. hyperframes-registry had the largest increase, nearly pulling "failing" up to "near-perfect."

10 Result Sets the Tester Chose Not to Publish

Of the 40 result sets, 10 were flagged as "insufficient samples to judge." grill-me, tdd, and Anthropic's own frontend-design all fell into this range on both models.

The tester specifically stresses that this does not mean "no effect." It means that across repeated runs of the same task (up to 10 per experimental group), the answers fluctuated too much to draw a credible conclusion. Rather than report a number that would flip on a rerun, they chose to label it "can't be measured." This kind of handling is uncommon in evaluation content and deserves credit.

Evaluation Method: What Counts as "Clearly Helpful"

The judging criteria are the key to this experiment's credibility:

1. Each skill is paired with three pure text tasks, written based on the skill's own description and README, and fixed before any run begins.
2. Each task is answered both with and without the skill, with each group repeated 3 to 10 times and each answer scored 5 times.
3. "Clearly helpful" is defined as: on at least one task, the two groups' score intervals (mean plus or minus one standard deviation) do not overlap and differ by at least 0.05, while no task moves clearly in the opposite direction. Any clear decline on any task overrides improvements on other tasks.

The third rule is quite strict — it means "a higher overall average score" is not enough to be judged helpful.

The non-overlap of intervals defined as "mean plus or minus one standard deviation" is an intuitive approximation of the statistical notion of two datasets being "significantly different." Strictly speaking, such comparisons usually employ a two-sample t-test or Mann-Whitney U test and report p-values and effect sizes. This article uses the interval method, which has the advantage of intuitive thresholds and accessibility to non-statistical readers, but is also more conservative: when sample sizes are small and variance is large, intervals easily overlap, causing real differences to be judged "can't be measured"; conversely, if variance is tiny, even a 0.05 gap may be judged "clear." So the three-way classification of "clearly helpful," "clearly worse," and "insufficient samples" is essentially a joint judgment of effect direction and stability, not a strict hypothesis-test conclusion.

Another notable design choice is "scoring each answer 5 times." This is common practice when using an LLM as a judge: single scores have high variance, and averaging or majority-voting multiple scores reduces randomness. But repeated scoring only reduces judge-side noise; it cannot eliminate the judge model's systematic preferences (for example, preferring longer, more clearly structured answers). The use of a single judge, Claude Haiku 5.5, is precisely the source of this limitation.

Limitations: Conclusions Can't Be Extrapolated Too Far

The tester lists three limitations: each skill has only three pure text tasks, written by the testing party rather than the skill authors; tool calling, media generation, and deployment services were not tested; and there is only a single judge model. Also, "popular" here refers to reported install counts, which is not the same as quality.

These limitations define the scope of the conclusions: they answer "on three fixed text tasks, does loading this skill change answer quality," not "is this skill useful in real workflows." For skills involving tool calling, this method basically doesn't cover them.

If you think a task is unfair or a result is wrong, every skill page has a rerun request entry point, and all scoring records, task sets, and retained answers are public. See the full table at driftproofhq.com/skills/; the detailed report is #015.

Install counts are a market signal, not a quality signal. The value of this test isn't in ranking 20 skills against each other, but in a reminder: before loading a skill into your Agent, it's worth asking — does it actually make the output better, or does it just make the configuration look more professional?

One-line Summary

We tested 20 high-install Agent skills: most help, but the #1 skill, find-skills, actually lowers answer quality.

Paragraph Summary

Driftproof ran a controlled experiment on the 20 most-installed Agent skills on skills.sh: each skill got three fixed pure text tasks, answered with and without the skill loaded, judged by Claude Haiku 5.5, producing 40 result sets. The conclusion: 11 skills clearly improve quality on both Sonnet 5.5 and Opus 5.5, with triage scoring perfect on both models and hyperframes-registry showing the largest gain; but the #1 most-installed skill, find-skills, was judged clearly worse on both models, and skills like reddit-automation dragged performance down too. 10 result sets were labeled "insufficient samples" due to excessive fluctuation, and the tester refused to report unstable numbers. The author stresses that the evaluation covers only pure text tasks, and that install counts are a market signal rather than a quality signal.