More Than Just Style: SPACE Benchmark Reveals the Deep Impact of LLM Personality Tendencies

SPACE framework separates LLM personality from capability via controlled experiments, finding prompts can change style but not internal tendencies.
This article introduces the SPACE framework, which separates LLM personality tendencies from capabilities and instruction effects through controlled experiments. Comprising personality validation, capability assessment, and layered situational task datasets plus an evaluation protocol, SPACE determines personality's independent influence by comparing outputs under different guidance conditions. Core finding: prompts can change style but internal personality tendencies persist, resisting opposing instructions—explaining why system prompts underperform in complex tasks and highlighting personality-driven bias in model evaluation.
Large language models' behavior is shaped not only by explicit prompts and instructions but also by stable internal preferences resembling human personality tendencies. These preferences influence how models interpret situations and make decisions, yet in practical applications their effects are often entangled with model capabilities and instruction effectiveness, making them hard to isolate. A recent study titled "More Than Just Style: Benchmarking Personality Influence in Large Language Models" proposes the SPACE framework, which attempts to separate "personality" from "capability" and "instructions" through controlled experiments, yielding a noteworthy conclusion: personality tendencies are stubborn—prompts can change surface style but struggle to shake internal inclinations.

Why We Need Dedicated Measurement of LLM Personality Influence
We used to attribute differences in model outputs to whether prompts were well-written or whether the model itself was strong enough. But research points out that models harbor personality-like behavioral preferences internally, and these preferences persistently affect how they interpret tasks. The problem is that this influence is hard to distinguish from capability differences and instruction-following effects—a model may perform better because it is more capable, or simply because its "personality" happens to suit the current task.
The core idea of the SPACE framework is controlling variables. It places personality validation, capability assessment, and layered situational tasks under the same experimental protocol, comparing results under different guidance conditions to determine whether personality factors truly exert independent influence. This design turns "personality influence" from a vague intuitive judgment into a measurable, comparable variable.
LLM Personality typically refers to the stable behavioral tendencies models exhibit in interactions, similar to human personality traits such as extraversion, agreeableness, and conscientiousness. These tendencies are not determined by single prompts but stem from language patterns in pre-training data, preference alignment during fine-tuning, and inductive biases of model architecture. In psychology, personality traits are defined as cross-situationally stable behavioral patterns; correspondingly for LLMs, researchers measure consistent output preferences through standardized questionnaires (such as the Big Five personality inventory) or behavioral tasks. Unlike "style," personality tendencies affect how models interpret ambiguous situations and prioritize decisions, not just wording or tone. Therefore, distinguishing personality from capability requires controlling task difficulty and instruction strength—exactly the design starting point of the SPACE framework.
How the SPACE Framework Separates Personality from Capability
SPACE consists of three complementary datasets and a controlled evaluation protocol. The personality validation dataset confirms that models indeed exhibit stable personality tendencies; the capability assessment dataset quantifies models' actual performance across different tasks; and layered situational tasks simulate situations from simple to complex, observing models' decision patterns in near-real-world scenarios.
The key lies in controlled comparison. The research team set up different guidance conditions—such as applying instructions consistent with or contrary to internal tendencies—then observed output changes. If instructions could easily override personality tendencies, personality factors would not warrant separate attention; but experimental results showed the opposite. Training code and benchmark specifications have been made public on GitHub at https://github.com/Niklauseik/SPACE-training, facilitating reproduction and extension by subsequent research.
Controlled Experimental Design is the core method of causal inference, whose basic principle is to change only one independent variable while holding other conditions constant, thereby observing that variable's independent effect on the dependent variable. In LLM research, this means fixing model version, task inputs, and evaluation metrics, changing only guidance conditions (such as instructions consistent with or contrary to personality tendencies), then comparing output differences. The SPACE framework applies this paradigm to personality influence measurement: simulating different complexity levels through layered situational tasks and applying opposing guidance instructions in each situation—if output changes manifest only in surface style while decision patterns remain unchanged, personality tendencies are independent. This design effectively strips away capability differences (quantified through the capability assessment dataset) and instruction-following effects (through comparing instruction conditions), making personality influence a quantifiable variable.
Core Finding: Style Can Change, Tendencies Cannot
The most weighty conclusion from the experiments is: prompts can indeed change generation style, but models' internal personality tendencies are persistent and will not be easily overridden by opposing instructions. In other words, you can make a model "appear" more rigorous or gentler, but at the underlying level when interpreting situations and making decisions, it will still follow its original tendencies.
This explains a common confusion: why carefully designed system prompts sometimes have diminished effectiveness on complex tasks. Because instructions act on the surface, while personality tendencies act at a deeper level. Research also found that these latent tendencies have observable effects on model task performance, meaning that when evaluating models, one cannot look only at capability scores but must also consider biases introduced by personality factors.
System Prompt is the instruction used to set a model's role, behavioral boundaries, and output format when deploying an LLM, typically located at the very front of the conversation history with priority higher than user input. System prompts influence the model's generation distribution for subsequent content through attention mechanisms, but their scope is limited by the stability of internal representations. Research shows that system prompts can effectively regulate surface style (such as tone, level of detail, format) but struggle to change preference patterns formed in deep semantic understanding. This is because personality tendencies are already embedded in model parameters, manifesting as stable interpretation patterns for specific situations, while prompts only impose biases in the early layers of forward propagation and cannot reverse deep representations. Therefore, in complex tasks, system prompt effectiveness decays as task complexity increases, while personality tendencies' influence becomes relatively prominent.
Implications for Model Evaluation and Application
The significance of this research goes beyond the academic level. For those doing model evaluation, SPACE reminds us: capability comparisons between different models may incorporate interference from personality tendencies. A model scoring higher on a benchmark may not be "smarter"—it may just have a "more compatible personality." For application developers, if tasks are sensitive to decision style—such as risk assessment, customer service dialogue, content moderation—then model personality tendencies need to be included in selection considerations, rather than focusing only on parameters and benchmark scores.
The research team also points out that these characteristics deserve more attention and may bring new application directions. For example, can personality tendencies be actively leveraged to improve performance in specific scenarios? Or conversely, can personality biases be suppressed in tasks requiring neutral judgment? Answers to these questions are all built on the foundation of accurately measuring personality influence, and SPACE provides an operational starting point.
Final Thoughts
LLM personality tendencies are neither mysticism nor simple stylistic decoration. The SPACE framework brings them into the measurable domain through controlled experiments and delivers a clear signal: prompts can change style, but cannot change the underlying color. For researchers and developers concerned with model behavior, this is a variable that requires serious attention. With benchmarks and code made public, subsequent discussions around personality influence are expected to move from intuition to empirical evidence.
One-line Summary
SPACE framework separates LLM personality from capability through controlled experiments, finding prompts can change style but not internal tendencies.
Paragraph Summary
This article introduces the SPACE framework, designed to separately measure large language models' personality tendencies from capabilities and instruction effects through controlled experiments. SPACE consists of three datasets—personality validation, capability assessment, and layered situational tasks—plus an evaluation protocol, determining personality's independent influence by comparing output changes under different guidance conditions. The core finding: prompts can change generation style, but models' internal personality tendencies are persistent and hard to override by opposing instructions. This conclusion explains why system prompts have diminished effectiveness in complex tasks and suggests model evaluation must consider biases from personality factors. The research team has open-sourced training code and benchmark specifications, providing an operational starting point for subsequent empirical research.
Related articles

FBI Emergency Alert: How Autonomous AI Hacking Tools Breach Networks at Machine Speed
FBI emergency alert names open-source AI attack framework Artex, which can turn a plain-language sentence into a full intrusion chain and breach critical industry networks in under three hours. This article breaks down the three-stage attack mechanism, the reality of runaway open-source code, and the FBI's three hardening measures: SIEM intelligence ingestion, cloud least privilege, and legacy asset patching.

Gemini 4 Hydrogen Benchmark Scores Explained: Coding and AI Agent Performance
Full breakdown of Google Gemini 4 Hydrogen benchmarks: SWE-bench 13.5%, Terminal Bench 57%, OS World 77.6%, Automation Bench 76.76%, 1M token context, API from $1.5 per million input tokens, coding and AI agent capabilities analyzed.

FBI Warns About AI Hacking Tool Artex: Systems Breached in Three Hours
FBI warns about AI hacking tool Artex: describe targets in natural language for automated scanning and intrusion, with some attacks achieving full privileged access within three hours. It affected over 2,000 Microsoft 365 organizations and South Korean banks. Copies still circulate after the developer took it down in October; security fundamentals remain key.