Why Pet Health AI Should Separate Vision Models from Rule Tables

The Bug That Rated an Obese Cat as "Ideal Weight"

When building a health recognition app for pet owners, one seemingly reasonable approach is to let the vision model deliver conclusions directly — "This cat needs to see a vet immediately" or "This cat is in ideal shape." But this architecture carries hidden risks. One developer shared a painful lesson: a vision model rated an obviously obese cat as having an "ideal body condition," and once that error was surfaced directly to users, it could delay a genuinely necessary health intervention.

This case reveals a core design principle — the vision model is responsible for describing, not deciding. The model sees the color and consistency of a pet's stool, the condition of the ear canal, the color of the gums. Its job is to translate these visual features into structured descriptions. What those features mean — whether a vet visit is warranted — should be left to a rule table that the model can never directly overwrite.

Why Description and Judgment Must Stay Separate

Coupling the perception layer with the decision layer is one of the most common early mistakes in AI application development. Vision models excel at recognizing patterns in images, but they have no stable, auditable concept of "clinical severity." When you ask the same model to both describe and conclude, any hallucination or misclassification flows directly into the recommendation the user sees — and it becomes extremely difficult to trace.

Separating the two makes the architecture clear:

Perception Layer: Output Only Observable Facts

The vision model's output is constrained to objective descriptions of visible features — stool appears dark, ear canal shows discharge, gums look pale. This layer makes no value judgments and assigns no labels like "urgent" or "normal." The benefit is that the model's output can be independently validated and is far easier to debug when something goes wrong.

This design is commonly referred to in ML engineering as Output Constraining or Structured Prompting. In practice, developers use carefully crafted system prompts or JSON Schemas to force the model to return only predefined fields — such as stool_color, ear_discharge_level, gum_color as enum values — with no free-text value judgments allowed. This thinking mirrors Tool Calling / Function Calling in large language model applications: locking model output into a machine-parseable format so that deterministic code handles all downstream logic. A nice side effect is that structured output is naturally suited to logging and retrospective analysis — when a dispute arises, you can precisely replay what the model "saw" in any given request.

Decision Layer: A Human-Readable Rule Table

What actually determines "should this pet see a vet" is a rule table (a row in a table) driven by domain knowledge. Each descriptive feature maps to a corresponding recommended action. This table is written and reviewed by humans, and the model cannot touch it. When a rule needs updating — for instance, if a certain stool characteristic turns out to warrant a higher alert level — you change one row in the table without retraining the model.

This separation also brings a hidden advantage: auditability. When an error like "obese cat rated as ideal" occurs, you can immediately determine whether the problem originated in the perception layer (the model misjudged the body condition) or the decision layer (the rule mapping was wrong), allowing you to contain the damage quickly.

In engineering terms, this "rule table" might be a configuration table in a database, a YAML file, or a more sophisticated rules engine such as Drools or Open Policy Agent. The essence is externalizing domain expert judgment logic so it exists independently of any machine learning model. This pattern has decades of history in traditional medical information systems — the core of a Clinical Decision Support System (CDSS) is a knowledge base written and reviewed by physicians and medical experts, not a black-box model. Grafting AI's perceptual capabilities onto this kind of battle-tested deterministic architecture lets you gain the flexibility of visual recognition while preserving the auditability and compliance of a rule-based system — a mature engineering path for high-stakes vertical applications.

What This Case Teaches Us

For any team building high-stakes AI applications, this thinking offers a valuable reference. Medical, health, and safety scenarios share a common characteristic: the cost of errors is high, and model output cannot be blindly trusted.

Strictly separating "what the model can do" from "what the system should recommend" is fundamentally about preserving a deterministic, human-controllable layer of logic alongside the model's probabilistic output. A vision model may rate an obese cat as ideal — but as long as the decision rules exist independently, a vetted body condition standard can intercept that misclassification before it leads a user to make the wrong call.

The Iteration Flexibility Rule Tables Provide

It's worth emphasizing the maintenance advantages of this architecture. Retraining a model is expensive, slow, and yields unpredictable results. Rule table updates, by contrast, are immediate, versionable, and rollback-friendly. When the business discovers that the threshold for a certain symptom needs to be tightened or relaxed, the engineering team can push a one-line config change to production without touching the expensive model pipeline.

For a domain like pet health — where veterinary expertise needs to be continuously incorporated — this division of labor, where the model looks and the rules judge, is precisely what allows specialized knowledge to flow into the system continuously and at low cost.

Summary

This developer experience from Reddit reminds us: before plugging AI into a critical decision chain, ask yourself — "Should this conclusion come from the model, or from human-controlled rules?" Decoupling description from judgment reduces the risk of hallucinations reaching users, and makes the system more transparent and maintainable. That misclassified obese cat turns out to be an excellent architecture lesson.