17,040 AI Signals in 43 Days: Four Truths Trend Leaderboards Won't Tell You
Four counterintuitive findings about trend data after 43 days and 17,040 AI signals.
The author ran an automated pipeline for 43 days, collecting 17,040 AI signals from 8 public sources (~398/day). Key findings: rotating leaderboards leave only 23 of 1,493 Hugging Face models with ≥20 days of history; top entities barely move over 48 days; 38% of data comes from one aggregator while the four digest-producing sources are just 29%; all derived metrics must be recomputed and audited. The proposed fix is dual-track collection: rotating leaderboards for discovery plus a fixed set for time series. The dataset, API, and MCP tool are open-sourced.
Running an automated pipeline that continuously collects AI signals from public sources for 43 days yielded 17,040 data points, averaging about 398 per day. This dataset was originally meant to support a daily digest, but it turned out that the data itself is more interesting than the digest — especially the things it cannot tell you.
Here are the four findings most worth knowing before building any AI trend application.
1. Of 1,493 Tracked Models, Only 23 Have Enough History to Compute a Trend
This is the core problem. The collector scrapes trending leaderboards — GitHub trending, Hugging Face trending, and so on. But the leaderboards themselves rotate: the model ranked #4 today may vanish tomorrow, and its history vanishes with it.
The actual results:
| Category | Tracked | Ever Ranked | Has ≥20 Days of History |
|---|---|---|---|
| Hugging Face models | 1493 | 228 | 23 |
| GitHub repos | 63 | 63 | 14 |
In other words, when a tool shows you the "momentum" of an AI model, you should ask what its baseline is. If an entity has only been observed twice, then the so-called "trend" is just two points and a straight line.
The author's choice was to output unranked with a reason, rather than filling the gap with a plausible-looking number. Every row in the JSON either has a derivable value or carries an explicit "not ranked, for the following reason." This decision sacrifices a lot of "attention-grabbing" numbers, but it remains correct — a fabricated ranking is indistinguishable from a real one to downstream consumers.
If you're building this yourself, the solution is: track a fixed set (take the top N by a stable metric, recorded daily) and run it in parallel with the rotating trending leaderboards. The rotating leaderboards handle discovery; the fixed set handles time series. Without the latter, you get neither.
A trending list is a ranking dynamically generated by platforms like GitHub and Hugging Face based on recent activity, usually computed on a rolling basis from short-window metrics such as download counts, star growth, and forks. Its essence is a "snapshot of the day," and it doesn't guarantee that an entry will still be on the list the next day; once it drops off, the collector can never get its metric readings again. This has a fatal consequence: for a leaderboard-driven pipeline, the historical series is a product of survivorship bias — you only keep the entities that stayed popular, and those happen to be the ones that change the slowest. Therefore, if you want leaderboard data to support time-series analysis, you must split "being discovered" and "being tracked" into two separate things: leaderboards handle discovery, and a separately maintained fixed entity list handles day-by-day recording. This is also the fundamental reason the article repeatedly emphasizes collecting from "rotating leaderboards + a fixed set" in parallel.
2. The Head Is Boringly Stable; the Really Interesting Signal Is "Churn"
The expectation was that leaders would change frequently, but 48 days of daily snapshots showed:
- Largest Hugging Face download change: −11.7% (
moonshotai/Kimi-K3) - Largest increase in the same column: +7.6% (
deepseek-ai/DeepSeek-R1) - Everything else in the top 8 ranged from −5% to −10%
- GitHub stars, top repos: only +0.08% to +0.54% over 48 days
The leaderboards are almost frozen, and most large-model downloads are slowly decaying. Growth is the exception, not the norm.
At the same time, single-day changes (2026-10-12 → 2026-10-13) looked like this: 2 models entered, 41 no longer appeared, 201 changed rank. The real movement is at the edges, not the top.
If you're going to build alerts, target entries and exits, not rank movement at the top. "This model appeared for the first time / this model disappeared" carries far more information than "moved up 3 spots."
Rank churn refers to the total amount of entries, exits, and rank changes of leaderboard items between two adjacent snapshots. It is a metric for measuring the "metabolic rate" of a leaderboard, independent of absolute ranking. Empirically, the top of a leaderboard tends to be occupied by a few strong entities for a long time, with position fluctuations compressed to within a few percentage points; the middle and lower sections, however, are replaced at high frequency. This means alerts based on "rank moved up N spots" will generate a lot of noise at the top or simply stay silent, while alerts based on "first entry / no longer appears" can catch new things earlier. When computing churn, be careful to keep the definition consistent: are you counting rank changes across all entities, or only the intersection of those present in both snapshots? The two mean completely different things, and conflating them will cause the metric to distort as leaderboard length fluctuates.
3. Most of the Volume Comes from a Single Source, and It Isn't the Most Interesting One
Source distribution over 43 days:
| Source | Items | Share |
|---|---|---|
| Trending aggregation | 6455 | 38% |
| arXiv | 2551 | 15% |
| Hugging Face | 2107 | 12% |
| Arena leaderboards | 2041 | 12% |
| Silicon Valley news | 1198 | 7% |
| Hardware/chips | 1089 | 6% |
| GitHub | 860 | 5% |
| YouTube | 739 | 4% |
Aggregation sources alone account for more than a third. They're cheap to collect, and much of their content duplicates the primary sources.
The four sources that actually produce the digest (GitHub, Hugging Face, YouTube, Silicon Valley news) account for only 29% of the data volume. If you rank sources by item count, you'll optimize the loudest one rather than the most useful one. The author ultimately tracked each source's yield rate and dedup rate, not raw data volume — volume is the kind of metric that makes a pipeline look busy while actually producing nothing.
Dedup rate refers to the proportion of items collected from a given source that duplicate other sources. Aggregation-type sources (such as trending aggregators and newsletters) naturally rely on secondary reposting; they have large item counts but low original information density, and their dedup rates tend to be high. First-hand sources (GitHub, Hugging Face, arXiv, official blogs) have few items, but they are the first place information lands. Therefore, when evaluating source value, a more reasonable metric is "deduplicated unique items ÷ collection cost," not raw item count. If you allocate compute by item count, aggregation sources will crowd out the quota for first-hand sources, and the pipeline will look data-rich while actually being repeatedly filled with the same batch of information. This is also the origin of the "yield × dedup rate" two-dimensional evaluation method mentioned in the article.
4. Derive Numbers from Raw Snapshots, or Don't Publish Them
Every published derived number (rank, rank change, 7-day momentum, position within an arXiv category) is recomputed from raw snapshots by an automated audit, and the audit also reconciles the rendered page against the JSON API. If the two don't match, that's a failure, not a rounding error.
There are two reasons:
- Derived numbers drift silently. A rank computed and stored at write time is a claim from the past; only after recomputation is it a claim you can defend.
- It catches your own bugs. The audit has already caught errors such as a "content quality" check passing on a page whose body was actually empty, and a given day's version not matching the same day's API response.
The author's principle is: better to publish a boring number that can be re-derived than an exciting number that can't be verified.
A derived metric is a value that is not directly observed but calculated from raw data, such as "7-day momentum," "rank change," or "position within a category." Its relationship to the "raw snapshot" is similar to that of a report to a ledger. The hidden danger of derived metrics is that once the computation logic at the write stage has a bug or the definition changes, the stored value silently goes stale, and downstream consumers can't tell what's real. The audit approach is to save immutable raw snapshots, recompute all derived metrics from the snapshots using the current code before each publication, and then reconcile the rendered result against the JSON API — any mismatch is judged a failure rather than a tolerance. The cost of this "replayable" principle is recomputation overhead; the benefit is that every number can be traced back to a specific set of raw inputs.
How This Dataset Is Made Public
The pipeline runs unattended (collects 4 times a day, publishes one issue daily, self-audits daily). The data is publicly available for free:
- Daily dataset (CC BY 4.0):
github.com/jiangzhangcc-glitch/ai-signals-daily, with the fullsignals.jsonl(all 17,040 rows) - Today's signals JSON:
data.jiangzhang.ca/signals4/api/today.json - Changes relative to the last snapshot:
.../api/changes.json - Model/repo rankings with citations:
.../api/models.json,.../api/repos.json - As an MCP tool for Claude/Cursor:
signals-api-mcp
A pitfall worth noting: if you use a bare Python-urllib User-Agent (or none at all), you'll receive a 403 from the CDN edge — the request never reaches the server, and the server logs show nothing. It looks like a dead API. requests, httpx, aiohttp, Go, node-fetch, axios, and curl are all fine. Remember to set a User-Agent in your examples.
If you need scale beyond the free tier (8 sources instead of 4, 4 updates a day, full history behind a stable key), that falls under the paid feed.
If you're building something similar, the author genuinely wants to know: how do you handle the leaderboard rotation problem? It's the most underrated part.
One-line Summary
Four counterintuitive findings about trend data after collecting 17,040 AI signals over 43 days.
Paragraph Summary
The author ran an automated pipeline for 43 days, collecting 17,040 AI signals from 8 public sources, averaging about 398 per day, expecting it to support trend judgments but finding that what the data "cannot say" is more valuable than what it "can say." Core conclusions include: rotating leaderboards mean only 23 of 1,493 tracked Hugging Face models have ≥20 days of history, making trends statistically unreliable; top entities are nearly static over 48 days (largest change −11.7%), and the real information comes from daily entries and exits; 38% of data comes from a single aggregation source, while the four sources that produce the digest account for only 29%; all derived metrics must be recomputed from raw snapshots and reconciled by audit. Based on this, the author proposes a dual-track collection scheme of "rotating leaderboards for discovery + a fixed set for time series," and has open-sourced the dataset, API, and MCP tool.
Related articles
Beyond the Mainstream World: Security and Privacy Struggles in Overlooked Regions
Human-centered security research has long focused on the West, causing security solutions to fail in the mainstream world. Researcher Collins Minando draws on Kenyan mobile loan apps, internet café authentication dilemmas, and local expert studies across nine countries to reveal how socioeconomic constraints reshape privacy practices, and proposes mobile money privacy redesign and PETs deployment pathways.
Claude Reportedly Able to Hide Secret Messages in Benign Images
A Reddit post claims Claude Fable 5.1 can hide secret messages in benign images. This article reviews the background, image steganography principles, and potential security risks, and suggests a testing path from rumor to verification.
Federal Government Awards $85 Million; CFS to Develop Fusion-Specific Semiconductors
The federal government awards $85 million to Commonwealth Fusion Systems to develop conductors and power semiconductors for fusion plants. This article analyzes the significance of this funding for the fusion supply chain and commercialization path.