HN Community Poll: How Many 'AI Can't Do This' Challenges Have Actually Been Solved?

An Intriguing Community Experiment

A fascinating Show HN project has appeared on Hacker News: the creator compiled a list of past community claims about things "AI can't do" and invited users to vote on which challenges today's AI has genuinely overcome — and which remain unsolved.

The project itself isn't technically sophisticated, but it touches on a compelling question: how accurate have tech communities been at predicting the limits of AI capability? In the fast-moving wave of generative AI, many tasks once considered permanently beyond AI's reach are being conquered at a pace that exceeds even optimistic expectations.

Why These "Proven Wrong" Lists Matter

Tech communities have never been short of confident claims about AI limitations. Some insisted AI could never write usable code; others believed it would never truly understand context; still others maintained that creative work was an exclusively human domain. These views were often reasonable given the technology of the time — but technological progress has a way of making prophets look foolish.

Collecting these historical claims and putting them to a vote is essentially a form of collective cognitive retrospective calibration. It pushes practitioners to ask: how much of our judgment about technological limits is grounded in solid first-principles analysis, and how much is simply intuition shaped by the constraints of today's tools?

This kind of retrospective has real practical value. When setting technical roadmaps, evaluating investment directions, or planning product capabilities, judgments about "what can and can't be done" directly affect decision quality. If the community has consistently underestimated AI's rate of progress, related strategic decisions may be systematically too conservative.

The Limitations Behind the Voting Mechanism

That said, this type of poll has obvious methodological problems. "A challenge has been met" rarely comes with a shared definition — does it mean AI has surpassed humans on a benchmark test, or that it reliably completes the task in real-world conditions? The gap between those two things is enormous.

Take code generation: an AI producing elegant functions in a demo environment is not the same as independently delivering a maintainable system in a complex production setting. If voters apply different standards when judging whether something "counts" as solved, the results lose much of their reference value.

Community polls are also naturally susceptible to group bias. Hacker News users skew tech-optimistic overall and may tend to overestimate how mature AI capabilities actually are. Without a rigorous verification process, poll results reflect community perception more than objective reality.

The standards for measuring AI capability have long been contested in both academia and industry. Academic benchmarks like MMLU (general reasoning) and HumanEval (code generation) quantify model performance, but carry the risk of "data contamination" — models may have seen test questions during training, inflating scores. Industry practitioners care more about real-world reliability: how a model performs against noisy inputs, edge cases, and long-tail requirements in production. The chasm between these two frameworks is precisely why claims like "AI can now do X" so often spark debate. With HN voters coming from vastly different backgrounds, their standards for judging any given capability can vary wildly — making it hard for aggregated results to carry precise technical meaning.

What This Reveals About Shifting AI Perceptions

Setting aside methodological rigor, the project's greatest value is the window it opens onto how perceptions of AI have shifted. Over the past few years, breakthroughs in generative AI across language, images, and code have genuinely rewritten many people's mental models of what technology can do.

Tasks once considered "impossible within a decade" are now baseline features in everyday tools. That acceleration is itself a reminder: when it comes to AI, any claim of "it will never be able to do X" deserves serious humility. The boundary of what's technically possible isn't fixed — it's being continuously redefined.

For practitioners tracking AI development, rather than fixating on whether a given capability "can or can't" be done, the more useful posture is to dynamically follow technical progress and periodically recalibrate your mental model. This small voting project happens to offer a lightweight way to do exactly that.

The core breakthroughs in generative AI have unfolded from 2022 to the present. Large language models (LLMs) like GPT-4, Claude, and Gemini have achieved fluency in language understanding and generation that was previously unimaginable. Image generation models like Stable Diffusion and Midjourney transformed "AI art" from a research prototype into a mainstream tool. Copilot-style products brought AI code assistance into standard development workflows. The underlying driver of this acceleration is the scaling law — the simultaneous expansion of model parameters, training data, and compute has produced emergent capabilities: abilities that appear suddenly once model scale crosses certain thresholds, with no prior indication they were coming. This nonlinear leap in capability is precisely why so many "impossible in a decade" predictions collapsed within two or three years — and why any static "capability checklist" continues to fail.

Conclusion

This is a simple but thought-provoking community experiment. Through the mechanism of a vote, it puts the tech community's collective predictions about AI on the table for scrutiny. While the objectivity of the poll results is limited, the reflection it invites — about how we evaluate technological limits and how we avoid being trapped by present-day constraints — is far more valuable than the vote tallies themselves.

(Note: The project currently has modest traction on Hacker News, and related discussion is still in its early stages.)