When Shouldn't You Use an LLM? A Five-Gate Engineering Decision Framework

"Can this requirement be built with an LLM?"—this has become almost a reflex for every technical team at project kickoff. But the more important question to ask first is: does this requirement really need an LLM?

Not every software problem needs an LLM. If a task can be solved reliably with deterministic rules, SQL queries, standard APIs, search algorithms, or traditional machine learning, then bringing in an LLM only adds cost, latency, security exposure, and maintenance complexity—without delivering better results.

This analysis framework, drawn from engineering practice, offers a core dividing line: when a system requires precise computation, predictable outputs, strict execution guarantees, extremely low latency, or cannot tolerate unverified results, you should not use an LLM.

Define the Problem Before Choosing the Technology

The most common architectural mistake is choosing the technology before defining the task. Take invoice processing: it actually consists of document extraction, field validation, tax calculation, purchase order matching, discrepancy detection, and approval—and these steps don't all need the same technology.

Document extraction, when layouts vary widely, may suit OCR, document AI, or an LLM; tax calculation should use deterministic business logic; PO matching can use queries and predefined rules; and anomalous discrepancies may call for statistical anomaly detection or manual review.

So the first question should be: does this task need language understanding or generation, or does it just need to execute known business logic? If the answer is the latter, start with conventional software.

Seven Scenarios Where You Shouldn't Use an LLM

Outputs must be mathematically exact

Financial reconciliation, tax calculation, inventory balances, and contract pricing should all rely on validated algorithms and appropriate numerical representations. An LLM can interpret the request, but the computation should remain deterministic. In the example given in the original, the discount calculation uses Decimal to handle floating-point issues and performs boundary validation on price and discount rate—handing this kind of logic to a model is unnecessary risk. The right approach is to use validated business logic, decimal arithmetic, and specialized computation libraries.

IEEE 754 and decimal arithmetic: The default floating-point numbers (float/double) in most programming languages follow the IEEE 754 standard and use binary representation, so decimal fractions like 0.1 and 0.2 cannot be stored exactly, and addition or multiplication produces tiny errors. In financial and inventory scenarios, these errors accumulate across large numbers of transactions and cause reconciliation to fail. The solution is to use decimal floating point (such as Python's decimal.Decimal, Java's BigDecimal, or a database's NUMERIC/DECIMAL type), which stores decimal numbers in fixed-point form and computes exactly according to the expected rounding rules, at the cost of lower performance than binary floating point. If you let an LLM generate the expressions or results, the model can't guarantee bit-for-bit correctness, nor can it guarantee it will follow the same computational path every time.

Boundary validation: This means explicitly checking the value range, type, and precision of inputs and outputs before and after computation—for example, price cannot be negative, discount rate must fall between 0 and 1, and the total must not exceed a cap. It's the key means of turning "the model might make things up" into "the system will definitely reject it," intercepting errors before they enter the books rather than relying on after-the-fact manual discovery.

Deterministic computation and reproducibility: Financial auditing requires that the same input produces the same output at any time, in any environment, with a traceable basis for the calculation. Deterministic code naturally satisfies this requirement, whereas LLM sampling is random—even with temperature set to 0, outputs may differ due to version, backend, or parallelization strategy changes, making it hard to use as audit evidence.

Processes have clearly defined rules

A refund policy with explicit time limits, exclusion categories, and approval thresholds is better expressed with a rules engine or conventional application logic. In the simplified example in the original, requests over 30 days or otherwise ineligible are rejected outright, amounts over 500 go to manual review, and the rest are approved—note that the original explicitly warns this is not a complete refund policy and that a production system still needs identity verification, transaction validation, fraud controls, and audit records.

Latency requirements are extremely strict

LLM inference introduces multiple layers of latency from networking, input processing, generation, and tool calls. Industrial control loops, low-level network packet processing, high-frequency trading, and safety-critical device control are better served by optimized algorithms, embedded logic, or purpose-built low-latency models with verified timing.

The answer already exists in structured data

If the revenue figure is already in the authoritative database, query the database directly instead of having the model "generate" it. The example SQL in the original tallies revenue for a given month from paid invoices—while noting that this query assumes "revenue" equals the amount of paid invoices, and that financial reporting may use a different definition. An LLM can translate a user question into a constrained query, but authorization and aggregation should always be controlled by the application layer.

Sensitive data can't meet the processing architecture requirements

If the implementation can't satisfy data processing, retention, access control, and deployment requirements, don't bring in an LLM. Private deployment and governed managed services can support sensitive workloads, but you must be able to prove the controls are effective. OWASP lists prompt injection and sensitive information disclosure as major risks for LLM applications.

Prompt injection: OWASP ranks this first among the top ten risks for LLM applications. It refers to attackers hiding malicious instructions in user input, external documents, web pages, or tool responses to trick the model into ignoring its original system prompt, leaking secrets, or performing unauthorized actions. Unlike SQL injection, prompt injection has no simple escaping solution, because the boundary between natural-language instructions and data is inherently fuzzy. Defense relies on multiple layers of control: input filtering, separation of instructions and data, least-privilege tool calls, output validation, and human review.

Sensitive information disclosure: A model may inadvertently repeat training data, system prompts, other users' sessions, or private fields returned by tools in its answers. The risk comes not only from the model itself but also from logs, caches, vector databases, and third-party managed services. So-called "processing architecture requirements" typically include data residency, whether data is used for retraining, retention periods, access auditing, encryption, and tenant isolation. If these requirements cannot be demonstrably met, the model should not be given access to that data.

Private deployment and governed hosting: Private deployment means both model weights and data stay within an environment the enterprise controls; governed hosting means constraining data usage through contractual and technical controls on a third-party cloud. Both must be paired with access control, key management, network isolation, and monitoring to pass compliance review—a promise of "we won't make it public" is not enough on its own.

Purpose-built models are more efficient

Fraud detection, demand forecasting, defect classification, anomaly detection, and predictive maintenance are often better served by statistical methods or task-specific machine learning. An LLM can explain the results, but it doesn't need to generate the underlying prediction. The key is to compare on representative data.

You can't evaluate whether the model is correct

Without representative test cases, defined unacceptable outcomes, and evaluation or review mechanisms, you shouldn't approve autonomous production use. Until measurable acceptance criteria exist, a controlled prototype or manual review process is preferable.

A Three-Way Comparison of Technology Choices

The original provides a requirements-to-solution comparison table with a clear core logic: exact arithmetic uses deterministic code, business rule execution uses a rules engine, structured data retrieval uses SQL or APIs, exact keyword matching uses a search engine, and demand forecasting uses statistical or ML models; while flexible document summarization, semantic document discovery, and natural-language interfaces suit LLMs, and high-risk automated actions must be paired with deterministic controls and human oversight.

These are starting points, not universal rules. The original emphasizes benchmarking against your own acceptance criteria.

Work Out the True Cost of an LLM

Total cost of ownership = development + infrastructure + inference + evaluation + monitoring + maintenance + exception handling.

The original gives a hypothetical comparison at a million requests per month (explicitly labeled as illustrative, not vendor pricing or a public benchmark): a deterministic service costs $0.001 per call with a 99.9% success rate, about $0.001 per successful task; an LLM service costs $0.02 per call with a 97% success rate, about $0.0206 per successful task.

The two may be solving different problems—the deterministic service may not be able to handle language-intensive tasks. What you need to compare is deployment time, cost per completed task, failure rate, review requirements, maintenance, monitoring, and security overhead.

The Five-Gate Decision Framework

This is the most actionable part of the piece; the original calls it "a proposed engineering method, not an externally validated standard":

Gate 1: Is language intelligence really needed? If the task doesn't require flexible language interpretation or generation, start with deterministic software.

Gate 2: Can a simpler approach meet the requirement? Benchmark against rules, search, APIs, or purpose-built ML—don't compare the LLM only against "doing nothing."

Gate 3: Can the system tolerate probabilistic behavior? Define how errors, incomplete results, or inconsistent outputs will be detected and handled.

Gate 4: Does the LLM meet the operational constraints? Test latency, throughput, cost, availability, privacy, and security under realistic load.

Gate 5: Is the extra capability worth the extra complexity? Approve the LLM only when the benefits are substantial and the risks are controllable.

The decision flow runs: is language intelligence needed → if not, use deterministic software; if yes → compare against simpler alternatives; if alternatives meet the bar → prefer the alternatives; otherwise → test error containment, operational constraints, and measurable incremental value, then approve a controlled LLM implementation.

The logic behind the gates: These five gates are a layer-by-layer filter from "is it necessary" to "is it worth it," not a parallel checklist. Gates 1 and 2 sit at the requirements and alternatives level, answering "is there a simpler, more reliable path"; Gates 3 and 4 sit at the system capability level, answering "are probabilistic behavior and operational constraints acceptable"; Gate 5 sits at the economic and governance level, answering "does the added complexity buy substantial incremental benefit." If an earlier gate fails, later discussion is meaningless, so review should proceed in order to keep teams from arguing about model selection right out of the gate.

How this connects to "define the problem before choosing the technology": Gate 1 corresponds to the core judgment at the start of the article—does the task need language understanding, or does it just need to execute known logic. Making Gate 1 a hard prerequisite prevents "reflexive LLM use" from derailing architecture reviews.

Why measurable incremental value is emphasized: Gate 5 requires substantial benefits and controllable risks because the failure modes LLMs introduce (hallucination, uncertain retries, output format drift) cannot be fully eliminated—only contained with evaluation, monitoring, and human fallback. If those costs exceed the cost of handling the task with a deterministic solution, then even a better model experience doesn't justify approval. The original explicitly states that this framework is a proposed engineering method, not an industry standard, and that teams should adjust the gate order and thresholds to their own acceptance criteria.

Three Architectures for Customer Support Ticket Routing

A SaaS platform needs to route requests such as duplicate charges, changing an account email, and API errors. The original compares three architectures:

Option A: Rule-based routing. Keyword matching sends tickets to billing, account, technical, or manual review. Transparent and testable, but limited coverage when customers phrase things unexpectedly or describe multiple issues.

Option B: LLM-based routing. It understands varied phrasing, but adds inference cost, model evaluation, output validation, and additional failure modes.

Option C: Hybrid routing. Clear requests go through rules, common variations go through a lightweight classifier, ambiguous language goes to an LLM, and low-confidence or high-impact cases go to humans. Routing policy is still controlled by the application.

Comparison metrics include classification precision and recall, misrouting rate, human review rate, P95 latency, cost per correctly routed ticket, and maintenance effort. A hybrid system is only better when it delivers measurable benefits.

Put the LLM at the Boundary, Not Everywhere

The core architectural principle is: separate language interpretation from authoritative business execution. The LLM can translate intent into a structured request; the deterministic service validates the schema, enforces permissions, applies business rules, and executes critical operations.

The flow chain in the original is clear at a glance: user request → LLM interprets intent → structured request → schema validation → authorization + business rules → deterministic API execution → verified result → user response.

Structured output can constrain the response format, but syntactic correctness doesn't mean the interpretation is correct or that execution is authorized. This is especially important for AI agents connected to enterprise systems.

Structured output: Through JSON Schema, function calling, or grammar constraints, the model is required to return results in fixed fields for easy programmatic parsing. But syntactic correctness only means the format is valid—it doesn't mean the semantics are correct. The model may fill in the wrong field, supply a nonexistent enum value, or fabricate parameters. So the application side must re-validate types, ranges, required fields, and business rules, and send failing results back for retry or manual handling.

The principle of separation of duties: Position the LLM as an "intent translator," not an "executor." The user request is first converted by the LLM into a structured request (e.g., {action: "refund", orderId: "...", amount: 120}), after which a deterministic service performs schema validation, identity verification, permission checks, business rule application, and transaction execution. That way, even if the model is induced to produce an unauthorized intent, it gets blocked at the authorization layer.

Why this matters especially for AI agents: Agents can often call tools, write to databases, or trigger external APIs, giving them a far larger action space than pure text chat. If authorization and business rules are also left to the model's judgment, a single prompt injection could turn into real financial loss or data leakage. So agent tools should follow the principle of least privilege, critical operations should require confirmation or human approval, and a complete audit trail of calls should be retained.

Validation and idempotency: Before executing, the deterministic service should verify whether the request is valid and whether it's a duplicate submission, and return a verifiable result. The LLM does not participate in deciding "is this authorized" or "has this been charged"—it only converts natural-language intent into a structured request that a program can check. This is where the article's architectural principle lands.

Ten Questions to Ask Before Approving an LLM

Problem definition: what needs an LLM, and what alternatives were considered? Baseline: how does the strongest non-LLM approach perform? Quality: what dataset and metrics are used to evaluate correctness? Reliability: what happens when the model fails or produces invalid output? Security: can the model access or change anything beyond the user's authorization? Performance: what are the P50, P95, and P99 latencies under load? Economics: what is the cost per successful business task? Ownership: who maintains the prompts, datasets, configuration, and monitoring? Change management: what is the retesting mechanism for model and data updates? Exit strategy: can the model be replaced without refactoring the application?

One Practical Rule for Architecture Review

The original closes with a line you can take straight into a review meeting: if removing the LLM doesn't materially reduce the system's ability to meet the requirement, then it probably shouldn't be on the critical path.

It's not that LLMs can't be used for business logic—it's that they're better suited to interpreting requests; authoritative rules, computations, permissions, and transactions should generally remain deterministic. The goal is always the same: build the simplest system that reliably delivers business outcomes.

One-line Summary

Not every requirement needs an LLM. This article lays out seven scenarios where you shouldn't use one and a five-gate decision framework.

Paragraph Summary

Responding to technical teams' tendency to reach for LLMs reflexively, the article proposes an engineering framework for judging whether an LLM is truly needed. The core dividing line: when a system requires precise computation, predictable outputs, strict execution guarantees, extremely low latency, or cannot tolerate unverified results, it shouldn't use an LLM. It lists seven unsuitable scenarios: mathematically exact outputs, clearly defined business rules, strict latency, answers already in structured data, inadequate data processing architecture for sensitive data, purpose-built models being more efficient, and an inability to evaluate correctness. It then presents five gates: is language intelligence really needed, can a simpler approach meet the requirement, can probabilistic behavior be tolerated, do operational constraints hold, and is the incremental benefit worth the extra complexity. The architectural principle is to separate language interpretation from authoritative business execution—the LLM translates intent, while deterministic services handle validation, authorization, execution, and verification. Cost should be measured per successful task, not per call. The final criterion: if removing the LLM doesn't affect meeting the requirement, it shouldn't be on the critical path.