Not Everything Needs an LLM: A Sober Look at the Boundaries of AI Applications

A rational look at LLM boundaries: use rules for deterministic tasks, and bring in models only for fuzzy scenarios.
Inspired by a Hacker News post titled 'Not Everything Needs an LLM,' the article discusses how, amid the inertia of integrating large models into nearly every product, technical decision-making logic often gets diluted by business motives. The author proposes three dimensions for deciding whether an LLM is needed — task determinism, cost structure sustainability, and acceptability of error cost: using an LLM for rule-coverable tasks replaces determinism with uncertainty; high-frequency, low-value calls amplify latency and cost; and hallucination is an architectural fact requiring fallbacks on critical paths. The article emphasizes that 'not using an LLM' is not opposition to AI but a call for technical decision discipline, advocates a hybrid architecture of rule engines and LLMs, and offers practical recommendations such as requirement baselines, cost quantification, and degradation design.

A post with a strikingly blunt title appeared on Hacker News — Not Everything Needs an LLM. The post itself doesn't launch into a lengthy argument, but it's precisely this restraint that makes the proposition feel sharper: at a moment when nearly every product is racing to integrate large models, has anyone seriously asked which scenarios don't need one at all?
A Question Obscured by Speed
For some time now, integrating an LLM has become the default move in product iteration. Whether it's a writing tool, a customer service system, or an internal admin panel, anything that touches "intelligence" gets asked to "add a large model." There are legitimate business motives behind this inertia — telling a story, raising funding, differentiating — but the logic of technical decision-making often gets diluted in the process.
This title is worth discussing because it points to a more fundamental question: LLMs are an extremely capable but also extremely expensive tool — where exactly are their boundaries of applicability? If a team can't answer this question, it easily falls into a cycle of "using a hammer to find nails."
Three Dimensions for Deciding Whether You Need an LLM
Task Determinism
If a problem has clear rules that can cover it, then introducing an LLM often means replacing determinism with uncertainty. Form validation, state machine transitions, fixed-format text parsing — these tasks have clear input-output relationships, and traditional code is not only more reliable but far cheaper to debug. The value of LLMs lies precisely in handling the fuzzy areas where rules are hard to enumerate exhaustively, not in taking over processes that are already fully governed by rules.
Rule engines are a class of systems that extract business decisions from code and express them as configurable rules — typical examples include Drools or decision tables in risk control. Their core advantage is determinism: the same input always yields the same output, and rules are auditable, versionable, and directly maintainable by business stakeholders. Compared with LLMs, rule engines cannot handle fuzzy semantics not covered by rules, but for clearly bounded branch decisions, compliance checks, and process flows, their reliability and interpretability far exceed those of probabilistic models. A common division of labor in hybrid architectures is: the rule engine handles rigid constraints and safety fallbacks, while the LLM handles open-ended semantic understanding and generation, with the two connected through clear interfaces to avoid making the model bear deterministic responsibilities that rules should guarantee.
Cost Structure Sustainability
LLM calls have latency, cost, and output variability. For high-frequency, low-value-density call scenarios, these costs multiply rapidly. By contrast, deterministic logic written once and running long-term has a natural advantage in marginal cost. When deciding whether to integrate an LLM, putting per-call cost and call frequency together into a single calculation often filters out a considerable portion of unnecessary requirements.
Acceptability of Error Cost
LLMs hallucinate — this is an architectural fact, not a flaw that can be completely eliminated through prompting. In scenarios where error costs are low and human fallback exists (such as draft generation or inspiration assistance), this uncertainty is acceptable; but on pathways involving money, compliance, or irreversible operations, putting a probabilistic component on the critical path is itself a design failure in risk management.
Hallucination refers to the phenomenon where an LLM generates content that appears plausible but is factually incorrect or fabricated out of thin air. The root cause is that the model fundamentally predicts the next token based on probability distributions rather than retrieving or verifying facts, so even when the output sounds confident, it may be entirely false. Hallucination cannot be completely eradicated through prompt engineering; it can only be reduced in probability through retrieval augmentation (RAG), tool calling, structured output constraints, and similar means. The standard engineering practice is to treat it as a systemic risk: add fact-checking layers, human review, or deterministic fallbacks on critical paths, rather than assuming the model "probably won't be wrong this time." Understanding this is essential to reasonably defining the boundaries within which an LLM can be trusted.
"Not Using an LLM" Doesn't Mean "Opposing AI"
This title is easily misread as a rejection of AI, but a more accurate interpretation is a call for discipline in technical decision-making. Many excellent product practices only hold up after clearly delineating the boundaries of LLMs: using rule engines for deterministic branches and LLMs for open-ended semantics, each playing its own role.
From an engineering perspective, this hybrid architecture actually better demonstrates respect for LLM capabilities — using them where they are truly needed, rather than letting them erode user trust in scenarios they're not good at.
Practical Implications for Developers
For teams currently making technical decisions, this discussion can translate into several concrete actions: clarify "what is the traditional implementation approach for this task" during requirements review, as a baseline for comparison; perform quantitative assessments of cost and latency for newly introduced LLM calls; and design degradation and fallback strategies for LLM outputs on critical paths.
The value of Not Everything Needs an LLM isn't in providing answers, but in reminding us: amid a technology frenzy, having the courage to say "we don't need it here" is itself a form of professional judgment.
Degradation and fallback strategies refer to the system falling back to acceptable alternatives along preset paths when an LLM call fails, times out, or produces output that doesn't meet constraints. Common approaches include: setting timeouts and retry limits, switching to rule templates or cached results after exceeding limits; performing structured validation on output (such as JSON Schema, regex, business invariants), and rejecting or escalating to humans if validation fails; on pathways involving money and irreversible operations, positioning the LLM as an "advisor" rather than an "executor," with deterministic code making the final decision. The essence of degradation design is to confine the failures of probabilistic components to a local scope, preventing a single anomaly from amplifying along the call chain into a systemic incident.
Related articles

Gemini 4 Neon Review: Benchmark Scores Rival Carbon, API Pricing and Quota Breakdown
Google Gemini 4 Neon is now live, benchmark scores closely tracking Carbon, API pricing 3/15 at twice Hydrogen's rate. This article compiles Bilibili creator testing: Neon can replace 90% of Carbon scenarios, AntiGravity quota consumption is only one-third of Carbon, plus subscription channel restrictions and 2API ban risks.

Gemini 4 Helium Coming Soon? Signals of Google Loosening Subscription Restrictions
Google team member hints releases will continue; community speculates Gemini 4 Helium (Flash Lite) may be next. Subscription ban controversy gets a response as product lead says "Soon," suggesting restrictions may be lifted. This article covers Gemini developments, OpenAI Coding Agent and risk management, Doubao real-time audio-video API, and compute business dynamics.

AI-Assisted React Development in Practice: Optimizing a Video App and Handling Errors
A 14-year veteran developer demonstrates AI-assisted React video app optimization: compressing video from 80MB to 5MB (~94%) via ImageKit, plus reliable error handling and retry mechanisms. Includes planning, code understanding, and verification methods for working with AI.