DecisionShield: Using AI to Assess Decision Readiness Instead of Giving Direct Advice

When AI can produce a seemingly well-reasoned recommendation in seconds, it's easy to forget one thing: its confidence doesn't mean we have enough evidence. The open-source project DecisionShield, from Hacktoberfest Hack Day Coimbatore, tries to make AI answer a more fundamental question before recommending options — are you ready to make this decision?

From "What to Choose" to "Can You Even Choose"

We use AI to support decisions every day: which tech stack should we use? Should we switch databases? Which cloud provider should we pick? Build or buy? Modern AI assistants have an answer for almost all of these, and the answers are often well-structured and confidently stated.

But one overlooked fact is this: AI giving a confident recommendation does not mean we have enough evidence to make the right decision.

Take the question of whether a startup should migrate from PostgreSQL to MongoDB. A typical AI assistant might explain that MongoDB's document model is more flexible and supports horizontal scaling — which is true, but before actually making the call, a whole series of questions still needs answers: What do the query patterns look like? How strong are the transaction and consistency requirements? Does the application rely on complex relational queries? How much migration downtime can be tolerated? What is the three-year total infrastructure cost? Does the team have experience with the new database? What risks could arise during migration?

Without this information, switching databases based on a generic recommendation could introduce unnecessary complexity, unexpected costs, or operational problems. The issue isn't that AI can't give useful answers — it's that AI still gives answers even when critical information is missing.

How DecisionShield Works

DecisionShield is an AI-powered "decision readiness and risk audit" platform. Instead of immediately choosing an option for the user, it first analyzes the decision context and identifies what information supports or undermines the decision.

The analysis is organized into several categories: facts (information supported by context), assumptions (statements not yet sufficiently validated), unknowns (important information currently missing), claims (assertions that need further verification), risks (issues that could affect the outcome), missing evidence (information needed for a more confident assessment of the decision), and stress-test scenarios (how the decision holds up if conditions change).

These structured analyses are then fed into a deterministic decision engine that calculates a "decision readiness" score from 0 to 100. For example, the assessment might be: Readiness 64/100 — Not Yet Ready, along with three key pieces of missing evidence that should be investigated before making the final call.

The key design principle of the entire workflow is: the language model handles semantic analysis, and the backend handles validation and scoring. Gemma processes the decision context and generates structured findings; the backend validates the data through Pydantic and passes it to the decision engine, with strictly separated responsibilities.

Gemma 4 E4B is a version of Google's open-source Gemma model series, and the "E4B" suffix typically refers to a 4-billion-parameter scale (Effective 4 Billion). It is a lightweight model that can run locally on consumer GPUs and even some high-end laptops. Unlike large models that require calling cloud APIs, local inference means data never leaves the device, making it more controllable in hackathons or privacy-sensitive decision scenarios. The 4-bit quantization mentioned in the article is a technique that compresses model weights from 16-bit floating point to 4-bit integers, reducing VRAM usage to about one quarter of the original at the cost of a slight loss in inference precision. This combination (small-parameter model + quantization + local inference) is a typical approach for individual developers to deploy LLMs with low barriers to entry, and it also explains why DecisionShield needs a fallback mechanism when inference fails.

Why Not Just Build an AI Chatbot

This is the project's most central design tradeoff. The usual AI assistant path is: user asks a question → LLM → text recommendation. DecisionShield's path is: decision context → evidence extraction → information classification → risk analysis → missing evidence detection → deterministic scoring → stress testing → decision audit report.

The difference lies in the focus of the analysis. A chatbot mainly responds to questions, while DecisionShield examines the quality and completeness of the information behind a decision. It does not intend to replace human judgment, but rather to help decision-makers see more clearly: what evidence exists, what risks exist, and what questions remain unresolved.

Core Features Breakdown

Decision Workspace lets users define the decision to be assessed, including title, description, alternatives, background, requirements, constraints, and existing evidence. Providing context and constraints allows the system to analyze the real situation described by the user rather than looking at the problem in isolation.

AI Evidence Analysis uses Gemma 4 as the semantic reasoning layer to break information into facts, assumptions, unknowns, and claims. In the database migration example, "PostgreSQL is the current database" and "the data volume is about 2TB" are facts; "MongoDB will automatically solve scaling problems" and "migration will reduce long-term costs" are assumptions that need investigation rather than immediate acceptance.

Decision Readiness Scoring can synthesize dimensions such as evidence quality, requirement coverage, evidence completeness, risk coverage, assumption load, unknown information, and decision stability. One key architectural principle is: Gemma does not directly generate the final numerical score, but outputs structured JSON, which is validated by Pydantic and then passed to the Python decision engine for calculation.

Missing Evidence Detection answers the question, "What other information could materially change this decision?" For example, in a database migration scenario, the system might identify key items to investigate: current query load and performance, transaction and consistency requirements, and expected peak traffic; as well as secondary items: migration downtime tolerance, three-year infrastructure and maintenance costs, and the team's experience with the new database. It also suggests validation actions, turning abstract concerns into concrete next steps.

Risk Analysis identifies potential risks related to the decision, including severity, likelihood, impact, confidence, and supporting evidence. For example: complex relational workloads, migration downtime, unexpected infrastructure costs, and team learning curve. These should be treated as potential risks to investigate, not as a conclusion that a particular database is unsuitable.

Stress Testing lets users explore whether the decision still holds when assumptions change: What if traffic grows 10x? What if the budget is cut by 50%? What if new data residency compliance requirements are added? What if the team loses its existing database expertise? What if the data volume doubles? For example, a decision initially scoring 72 might drop to 43 under a new compliance requirement scenario, showing that the decision is sensitive to changing requirements — a high initial score does not mean it is suitable under all conditions.

Stress Testing is also called sensitivity analysis in decision science, and its core idea is that whether a decision is robust depends on whether it still holds when key assumptions change. The example mentioned in the article — a decision initially scoring 72 dropping to 43 after a new compliance requirement is added — illustrates the potential misleading effect of a single-point score. This method borrows from risk modeling in finance (such as Monte Carlo simulation) and fault injection testing in engineering, but DecisionShield uses a lighter-weight approach: the user or system defines several hypothetical scenarios (traffic doubling, budget cuts, team changes, etc.), reruns the scoring process one by one, and observes the magnitude of score fluctuations. The larger the fluctuation, the more sensitive the decision is to unknown factors, and the more necessary it is to fill in the relevant evidence before making the final call.

Technical Architecture and Choices

The project uses a frontend-backend separated architecture. The frontend is React + Vite + Tailwind CSS + Recharts + Framer Motion; the backend is Python + FastAPI + Pydantic + Uvicorn; the AI layer uses Gemma 4 E4B, 4-bit quantization, and local inference; architecturally it adopts REST API, structured JSON, and a deterministic scoring engine, with in-memory storage during the MVP phase and a designed fallback flow between the AI provider and the demo provider.

Each component has clear responsibilities: Gemma 4 E4B handles semantic reasoning and classification, Pydantic handles structured output validation, the Python decision engine handles deterministic scoring, FastAPI handles the backend API and orchestration, React handles the interface and visualization, and the final decision remains in the user's hands.

This division of labor matters because language model analysis and numerical computation serve different purposes. The model interprets context, the backend validates data and executes scoring, and humans are responsible for the final judgment.

Pydantic is a mainstream library in the Python ecosystem for data validation and parsing. It constrains field types, value ranges, and required items by defining data models (classes). In DecisionShield, its role is to act as the "gatekeeper" between AI output and backend logic: the JSON generated by Gemma must conform to a predefined structure (for example, each risk must have fields such as severity, likelihood, and impact), otherwise validation fails. This effectively prevents the language model from "freestyling" and producing unparseable output, and also allows backend logic to be tested independently of the model. The deterministic scoring engine complements Pydantic — the former ensures the data is "correctly formatted," while the latter ensures the computation process is "reproducible," and together they form the "verifiable and auditable" analysis pipeline emphasized in the article.

Reliability Design and Demo Fallback

The project explicitly refuses to let the model directly return unsupported conclusions like "I think readiness is 87." The intended flow is: Gemma outputs facts/assumptions/risks/unknowns → structured JSON → Pydantic validation → Python decision engine → readiness score. This makes the analysis structure verifiable, the scoring process auditable, and the backend testable independently of the model.

However, the article also honestly points out that deterministic computation does not automatically guarantee correct results; assessment quality still depends on the input evidence, classification quality, and scoring rules.

Local AI inference in development and hackathon scenarios is prone to issues such as model loading failures, insufficient GPU VRAM, and CUDA configuration problems. Therefore, DecisionShield designed a fallback/demo provider approach. Both types of providers enter the same downstream processing pipeline through a unified structured output format, and the application distinguishes whether the analysis comes from real AI or demo mode.

Limitations and Roadmap

The author is quite candid about the limitations: AI classification may be imperfect; missing evidence suggestions may not cover all important factors; results depend on the quality and completeness of the provided context; stress-test scenarios are hypothetical analyses rather than predictions of the future; readiness scores are analytical indicators rather than objective truth; and the deterministic engine can also produce misleading results when inputs or rules are inappropriate. For high-impact decisions, users should still verify key claims, consult experts, and use independent evidence.

The future roadmap includes: document/PDF/spreadsheet evidence intake, evidence provenance (tracking which document and which page each fact comes from), advanced decision graphs (showing relationships among evidence, assumptions, risks, requirements, and decisions), collaborative decision rooms, decision history, external knowledge validation, more sophisticated stress testing, and specialized audit templates for domains such as software architecture, cloud infrastructure, procurement, hiring, and business strategy.

The Evidence Provenance mentioned in the article is an increasingly important concept in the field of AI trustworthiness, referring to the ability to trace each analytical conclusion back to its original source of information (which document, which page, which paragraph). In the context of DecisionShield, if the system points out that "the team lacks experience with the new database" is a key piece of missing evidence, the user should be able to click and see which passage of input text that judgment is based on. Implementing evidence provenance usually requires combining retrieval-augmented generation (RAG) and citation annotation techniques. The Decision Graph in the roadmap goes even further, presenting the dependency chain among facts, assumptions, risks, requirements, and the final decision as a network of nodes and edges, helping users intuitively understand "if a certain assumption is overturned, which conclusions will be affected as a result." Together, these two features point to the same goal: turning AI's analysis process from a "black-box output" into a decision-support tool that is "traceable, challengeable, and correctable."

A Shift in Thinking Worth Noticing

Perhaps the most valuable part of DecisionShield is not the technical implementation itself, but the question it raises: What if AI's most useful capability isn't telling you what to choose, but telling you what's still missing before you choose?

In the past, we asked AI, "What should I choose?" Now we can ask one more question: "Do I have enough evidence to make this decision?" If the answer is no, then we should first figure out: what information is missing, which assumptions need validation, and which risks deserve attention.

The project was developed by the Wakie Wakie team during Hacktoberfest Hack Day Coimbatore, organized by INIT Club and IDEA Club, and participated in the Gemma 4 additional challenge. The code is open-sourced on GitHub.

One-line Summary

DecisionShield is an open-source project that makes AI first assess decision readiness and point out evidence gaps rather than directly choosing an answer for you.

Paragraph Summary

DecisionShield is an open-source project from Hacktoberfest Hack Day Coimbatore. Its core claim is that a confident AI recommendation does not mean the user has enough evidence to make a decision. It breaks decision context into seven categories — facts, assumptions, unknowns, claims, risks, missing evidence, and stress-test scenarios. Gemma 4 E4B handles semantic analysis, Pydantic validates structured output, and a deterministic Python engine calculates a 0-100 decision readiness score. A key architectural principle is that the language model does not directly generate the final score and is only responsible for classification and explanation. The platform also provides missing evidence detection, risk analysis, and hypothetical-scenario stress testing to help users see unresolved questions. The author acknowledges that scores are analytical indicators rather than objective truth, the final decision remains in human hands, and future features such as evidence provenance and decision graphs are planned.