Good Predictions Don't Equal Good Decisions: Lessons from the FPL Decision Engine

From a Football Problem to a Decision System

While building tools for FPL (Fantasy Premier League), a developer kept coming back to a deceptively simple yet fundamental question: if you can predict the best player, but signing them isn't actually the optimal move, what's the point of the prediction?

The question is far more complex than it appears. In FPL, you have to simultaneously juggle budget constraints, transfer limits, your existing squad, upcoming fixtures, and the lingering fallout from a bad decision you made three gameweeks ago. Simply ranking players isn't enough — the real challenge is translating predictions into decisions that are both actionable and compliant with the rules.

That frustration gave rise to the developer's open-source project: FPL Decision Engine.

The Fundamental Difference Between Prediction and Decision

The core idea behind this project is to combine three things: football data, AI reasoning, and a rule-based validation layer. Its goal isn't to rank players — it's to compare actual, concrete decisions.

Prediction answers "which player will perform best." Decision answers "given my current constraints, what should I actually do?" The former is a statistical problem; the latter is one of optimization under constraints. A high-scoring player you can't afford, or one whose transfer would break the rules, renders even the most accurate prediction completely useless.

This distinction shows up everywhere in AI applications. Whether in recommendation systems, investment advice, or operational decision-making, a model's "optimal answer" is typically built on idealized assumptions — while the real world is full of hard constraints like budgets, compliance requirements, and legacy baggage. Predictions are useful, but figuring out what to do with them is the harder part.

The Boundary Between AI Suggestions and What the System Allows

The developer highlights one particularly interesting design choice in the project: explicitly separating AI recommendations from system-permitted actions.

An AI model might recommend a brilliant player, but if that transfer exceeds the budget or violates the "maximum three players from the same club" rule, the suggestion should be rejected — regardless of how confident the model is.

Confidence cannot make an invalid decision valid.

This is a line every AI product designer should keep in mind. Large language models have a natural tendency to deliver answers with conviction, but high confidence doesn't mean an answer is legal or feasible. Adding a hard rule-based validation layer to an AI system is essentially putting a deterministic guardrail around probabilistic intelligence. This hybrid architecture — AI reasoning backed by rule-based enforcement — is proving to be a key pattern for deploying practical AI systems today.

This architecture has a well-known name in engineering: the combination of Constraint Satisfaction with probabilistic models. Pure machine learning models output probability distributions or ranked scores — they have no inherent awareness of business rules. Rule engines, by contrast, are deterministic and can directly evaluate whether an action is permissible. The typical pattern for combining both is: let the AI generate a set of candidate solutions, then pass them through a rule layer that filters or adjusts each one, surfacing only those that satisfy all hard constraints. This approach is widely used in financial risk management (model scoring + compliance rules), medical decision support (AI recommendations + drug contraindication checks), and autonomous driving (perception predictions + traffic law constraints). The core value is clear: let the highly flexible AI handle "discovering good possibilities," and let the zero-tolerance rule layer handle "ensuring those possibilities are genuinely feasible." With these two responsibilities clearly separated, the system retains both intelligence and trustworthiness.

When Human Judgment Diverges from AI

The developer is also probing a deeper question: what happens when your own judgment conflicts with the AI's recommendation? Does that disagreement actually improve decisions, or does it just make you more confident in what you already thought?

This is an honest and important thing to ask. AI-assisted decision-making is often pitched as "augmenting human judgment," but in practice it can easily become a mirror — people tend to accept suggestions that confirm their existing views while discounting genuinely valuable dissenting ones. Whether AI is correcting for bias or amplifying it is something that demands rigorous verification.

The developer openly admits this part of the research is still ongoing, and that he has not yet proven the engine actually improves real FPL performance — there's a lot of testing left to do. This refusal to overpromise or rush to conclusions is refreshingly rare in an environment saturated with "AI changes everything" rhetoric.

The phenomenon the developer touches on has a precise term in behavioral economics: Confirmation Bias. When processing new information, people tend to notice, remember, and act on content that aligns with their existing beliefs, while unconsciously discounting contradictory evidence. AI decision-support tools should theoretically counter this bias by providing objective data — but experimental research (such as Dietvorst et al.'s work on "algorithm aversion") shows that when AI recommendations contradict a user's own judgment, the common reaction is to question the model rather than reconsider their own thinking. There's also a subtler, opposite phenomenon: Automation Bias — in certain contexts, users over-trust AI and abandon the critical scrutiny they should apply. These two biases point in opposite directions yet can coexist across different users or different situations. Genuinely validating whether an AI tool improves decision quality requires controlled experiments, not subjective impressions — which is precisely why the developer acknowledges the work is still in progress.

The Broader Value of an Open-Source Project

The public repository includes well-organized code, architectural documentation, and a complete working example for interested developers to explore.

Perhaps more interesting is what the developer himself took away from the experience: a project that started as a simple football tool ultimately sparked a deep interest in decision systems as a discipline. This trajectory — starting from a concrete use case and gradually abstracting toward a general methodology — is a pattern shared by many of the best technical explorations.

For AI application developers, FPL Decision Engine offers a small but complete reference point: how to make AI's predictive capabilities genuinely serve executable decisions under real-world constraints, rather than just producing impressive-looking leaderboards.

Project repository: FPL Decision Engine on GitHub

Closing Thoughts

Between prediction and decision lies a gap filled with constraints, rules, and human judgment. The biggest takeaway from this project may not be whether it can help you win at FPL — it's the reminder that in the age of AI, getting the model to "say the right thing" is only the first step. Getting the system to do the right thing is the real engineering challenge.