Generalizing Causal Decision-Making: Markov Equivalence, Transportability and Counterfactuals

The paper extends structural causal bandits to three realistic settings: partially known structure, cross-environment transportability, and counterfactual actions.
The paper extends the structural causal bandit framework to three more realistic settings: partially known causal structure, information from other environments, and implementable counterfactual actions. For Markov equivalence classes, the authors develop graphical criteria and search algorithms to identify minimal intervention sets; they derive target reward bounds via partial transportability and use dominance relations for action pruning with regret guarantees; and they formalize counterfactual structural causal bandits constrained by experimental capabilities, providing graphical characterizations and algorithms for minimality and possible optimality. The core idea is that causal knowledge and experimental capability jointly determine the action space worth exploring, shrinking exploration without losing the optimal action.
The intersection of causal inference and reinforcement learning is evolving rapidly. A new paper, Generalizing Causal Decision-Making: Markov Equivalence, Transportability and Counterfactuals, extends the structural causal bandit framework to three more realistic settings: partially known causal structure, information from other environments, and implementable counterfactual actions. This research provides new theoretical tools for how agents can explore efficiently when causal knowledge is incomplete.

From fully known to partially known: minimal intervention sets under Markov equivalence
The classic structural causal bandit assumes the causal graph is fully known, allowing the agent to select intervention actions accordingly. In reality, however, we often only observe partial information about the data, yielding a Markov equivalence class—multiple causal graphs that are indistinguishable from observational data. The paper first develops graphical criteria for this setting to identify minimal intervention sets under Markov equivalence.
Specifically, the authors characterize "possibly optimal" minimal intervention sets in maximal ancestral graphs and design a search algorithm that can find possibly optimal sets in at least one graph represented by a partial ancestral graph. This means the agent does not need to determine the unique true causal graph to screen out intervention actions worth exploring.
A Markov equivalence class is a set of causal graphs that are indistinguishable from observational data, sharing the same conditional independence relations. In causal discovery, observational data alone typically identifies only an equivalence class, not the unique true graph. A partial ancestral graph (PAG) is a graphical tool for representing Markov equivalence classes, with edges marked by circles, arrowheads, and other symbols to encode possible ancestral relations across all equivalent graphs. A maximal ancestral graph (MAG) is the graph in the equivalence class with the most ancestral edges, often used to define possibly optimal intervention sets. Understanding these graphical representations is essential to grasping the paper's notion of "possibly optimal."
Cross-environment transportability: partial transportability and reward bounds
The second extension addresses information reuse. When the target environment's reward distribution is unknown, can data from a source environment constrain the target reward? The paper derives bounds on the target reward through "partial transportability," with particular attention to the influence of non-manipulable variables.
These bounds further yield dominance relations that can be used for action pruning and restricted exploration. Under valid prior bounds, the authors provide regret guarantees. According to the paper, this transportability mechanism can significantly reduce the action space that needs to be explored while preserving the optimal action.
Transportability is a formal framework in causal inference, proposed by Pearl and Bareinboim, for studying how to transfer causal knowledge from a source environment to a target environment. It encodes differences between environments via selection diagrams and uses do-calculus to derive whether causal effects in the target environment are identifiable. Partial transportability refers to cases where only some mechanisms are transportable or only partial data is available; in such cases the target reward cannot be exactly identified, but bounds can be derived. Dominance relations mean that if an action's expected payoff is no better than another action's under all possible reward distributions, it can be pruned, thereby shrinking the exploration space.
Counterfactual actions: learning under experimental capability constraints
The third extension introduces counterfactuals into the structural causal bandit. The actions available to the agent are not arbitrary interventions but are restricted by specific experimental capabilities—that is, the counterfactual actions that can actually be implemented in reality. The paper formalizes this class of counterfactual structural causal bandits and provides graphical characterizations of minimality and possible optimality.
Based on these characterizations, the authors propose algorithms that remove equivalent or dominated choices while guaranteeing that the optimal action is not lost. The experimental section uses specified structural causal models to examine how these reductions and transportability bounds affect the learning process.
In causal inference, a counterfactual refers to reasoning about the outcome for the same unit under an intervention different from the one that actually occurred, typically represented using do-calculus and counterfactual variables in structural causal models. In structural causal bandits, counterfactual actions are not arbitrary interventions but are restricted by the experimental capabilities the agent can actually implement—for example, it may only be able to intervene on certain variables or only implement specific types of interventions. Formalizing such constraints requires extending the definition of the action space to incorporate feasibility conditions and redefining the optimal action and regret. Graphical characterizations use the structural information of the causal graph to determine which counterfactual actions may be optimal, thereby avoiding unnecessary experiments.
A unified view: causal knowledge and experimental capability jointly determine the exploration space
The three extensions share a core idea: causal knowledge and experimental capability together determine which actions are worth exploring. Markov equivalence addresses structural uncertainty, transportability addresses information reuse across environments, and counterfactuals address action feasibility constraints. The paper's graphical criteria and algorithms provide actionable action-screening methods for each case, shrinking the exploration space while preserving the optimal action.
For researchers building practical causal reinforcement learning systems, this work suggests a direction: rather than pursuing a complete causal graph, use partial structural information, cross-environment bounds, and action feasibility to constrain exploration—perhaps a more realistic path to deploying causal decision-making agents in the real world.
One-line Summary
The paper extends structural causal bandits to three realistic settings: partially known structure, cross-environment transportability, and counterfactual actions.
Related articles

Gemini 4 Hydrogen Hands-On: Google's Fastest and Cheapest AI Model
Hands-on with Gemini 4 Hydrogen: the fastest, cheapest model in the Gemini 4 family. We break down the Carbon/Argon/Hydrogen split, API pricing traps, free-tier limits, and the cost trade-offs of four thinking levels—with a real bug-debugging demo.

Gemini 4 Hydrogen vs Sonnet Comparison Test: Uses Only 1% of Weekly Quota, Performance on Par
Hands-on test: Gemini 4 Hydrogen vs Sonnet on the same CRUD task—Hydrogen used only 1% weekly quota with comparable performance and better UI details, while Anthropic's model used 7%. Developer says daily tasks don't require expensive models.

Gemini 4 Neon Released: Beats Opus 5.5 on Multiple Benchmarks, Google Returns to the Top Tier
Google DeepMind releases Gemini 4 Neon: 61 on the Artificial Analysis intelligence index (3rd) and 1st on the coding index, beating Opus 5.5 on multiple benchmarks. $3/$15 per million tokens; available in the Gemini API and AI Studio.