ContextAdapt: Evaluating Context-Adaptive Value Alignment in Large Language Models

The Hidden Challenge of Value Alignment: Having the Right Principles Isn't Enough

Honesty, autonomy, confidentiality — these values are often treated as universal pillars of AI alignment. But a long-overlooked question remains: what it means to uphold these values changes depending on context. "Confidentiality" means something different for a doctor deciding whether to report a communicable disease versus a lawyer protecting client privilege.

A newly published arXiv paper, ContextAdapt, takes aim at exactly this blind spot. The authors pose a fundamental question: can large language models (LLMs) appropriately adapt how they apply values across different professional contexts, while remaining consistent when contextual changes don't actually alter the relevant professional norms? This seemingly subtle question exposes a core flaw in current value alignment evaluation — we tend to test whether models "know" abstract principles, but rarely whether they can apply those principles correctly in specific contexts.

The ContextAdapt Framework: A Value × Domain Cross-Evaluation

ContextAdapt is a purpose-built evaluation framework covering three categories of values — honesty, autonomy, and confidentiality — across four high-stakes professional domains: healthcare, law, finance, and national security.

The framework's construction is noteworthy. Rather than designing scenarios from scratch, the researchers built a two-dimensional "value × domain" matrix grounded in primary professional and regulatory documents. From this matrix, they developed two types of scenarios: those testing default professional rules and those testing recognised exceptions. The elegance of this design lies in its ability to distinguish whether a model is mechanically applying rules or genuinely understanding the professional logic behind them — and their limits.

The evaluation covers 12 mainstream LLMs. Crucially, the study examines not just what action a model recommends, but also what reasoning it provides — because a correct decision paired with incorrect justification reveals that the model's grasp of professional norms is purely superficial.

---

The distinction between "default rules" and "recognised exceptions" is critical in professional ethics. Take confidentiality: the default rule requires lawyers not to disclose client information, but "recognised exceptions" include explicit client consent and preventing imminent serious crimes. Healthcare operates similarly — mandatory reporting of communicable diseases and obligations to report child abuse are legally defined exceptions to general confidentiality principles. These exceptions aren't vague moral intuitions; they're clearly defined by statutes, professional codes, or case law. By incorporating these exceptions into the test matrix, ContextAdapt evaluates not rote memorization of principles, but a model's grasp of "rule boundaries" — precisely the area most prone to error in real-world professional decision-making, and the hardest to assess.

---

Key Findings: Actions Pass, Reasoning Falls Short

The main experiment reveals a striking contrast.

On action appropriateness, the average score across all models reached 95.6% — an impressive result. But when examining whether models used correct, domain-specific reasoning, the gap widened dramatically: performance ranged from 25.6% to 76.9% across different models.

This means many models can "guess" the right answer without being able to explain why it's right. They may be relying on generalized intuition or broad safety heuristics rather than drawing on the specific norms of medicine, law, or other professional fields. For AI systems deployed in high-stakes professional contexts, this "knowing what but not why" state poses a real and serious risk — when facing complex edge cases near the boundaries of a rule, a model lacking sound reasoning is highly prone to giving wrong guidance.

Perceived Risk Becomes an Unexpected Trigger

The paper also includes a separate factorial experiment designed to isolate the effect of different variables on model behavior.

The results showed that explicitly naming the professional domain and changing the role the model was asked to play had minimal impact on model behavior. In other words, models don't rely heavily on these explicit signals to adjust their judgments.

The real problem emerged with the stakes variable. When the severity of a scenario was altered, the researchers observed serious but localized failures: in some cases, models changed their responses even when the underlying professional obligation had not changed at all.

Specifically, perceived severity appeared to act as a trigger for models to lean toward disclosure — a pattern observed in both honesty and confidentiality scenarios. That is, when a model "senses" that something is serious, it may breach confidentiality obligations it should not breach, even when doing so is not professionally justified. This reveals a dangerous vulnerability in how models make value judgments: they can be swayed by the emotional intensity of a scenario rather than by strict professional logic.

---

This phenomenon is known in AI alignment research as "context contamination" or "emotional hijacking" — the model's reasoning is distorted by emotionally charged or dramatic framing in the input, rather than being grounded in domain-specific norms. The root cause may lie in the distribution of training data: in human-written text, "high stakes" and "taking action (including disclosure)" frequently co-occur, causing models to learn a superficially correlated but normatively incorrect association. This connects directly to a core challenge in value alignment: getting models to distinguish between "what feels urgent intuitively" and "what constitutes a valid normative justification" requires not just more ethics training, but a deep understanding of professional reasoning logic itself.

---

Implications for AI Alignment Research

ContextAdapt delivers a clear and important message to the field of value alignment: evaluating alignment cannot stop at whether models follow abstract principles — it must also assess whether they apply those principles appropriately across different contexts.

This is a valuable corrective to current alignment evaluation methods. Many existing benchmarks operate at the level of "does the model agree with this ethical rule?" — but real-world professional decision-making is far more complex. It demands nuanced understanding of rules, exceptions, domain norms, and the weighing of competing considerations. ContextAdapt's cross-domain, cross-value design pushes evaluation to a level of granularity that better reflects real-world deployment.

For developers and organizations looking to deploy LLMs in professional settings like healthcare, law, or finance, this research sounds a clear warning: beneath a surface appearance of high accuracy, a model's underlying reasoning may be unreliable, and its judgments can be distorted by non-normative factors like perceived risk. Before genuinely trusting AI with high-stakes professional decisions, rigorous testing of context-adaptive alignment is not optional — it's essential.