SEMA-Auto: Breakthroughs in Both Efficiency and Success Rate for Autonomous Multi-Turn Jailbreak Attacks

The field of large model security is undergoing an accelerating race with asymmetric offense and defense. SEMA-Auto, proposed by Microsoft Research, pulls multi-turn jailbreak attacks out of the two old paths of "pre-scripted plans" and "online judges," redefining the balance between efficiency and success rate with an attack agent that has autonomous judgment capabilities and can decide for itself when to stop.

The Dilemma of Existing Jailbreak Attacks

Current mainstream jailbreak attack methods can be grouped into two categories, each with obvious shortcomings.

Static planners fix a single-turn or multi-turn adversarial plan in advance. They are simple to implement but completely ignore the victim model's responses. This makes them prone to wasting turns on irrelevant follow-up questions, and their attack transferability is limited—switch to another model or dataset, and the fixed script often fails.

Interactive generators read the victim model's response at each turn to generate the next round of attack. They are more flexible, but require mounting an additional evaluator within the attack loop to determine success or provide hints. This evaluator brings considerable overhead and also inflates the reported performance—the success signal does not come from the attacker's own judgment.

The Microsoft team explicitly states in the paper that the gap between these two approaches is exactly where SEMA-Auto finds its footing: a "sighted" and efficient multi-turn attacker that can interact with the victim model and terminate the session on its own upon success.

Jailbreak refers to inducing an aligned large language model, through carefully crafted prompts, to bypass its safety policies and output harmful content that should have been refused. Single-turn jailbreaks rely on one-shot prompts, while multi-turn jailbreaks gradually erode the model's refusal tendency through consecutive dialogue—for example, first establishing a role-play context, then progressively raising sensitive requests. The core metric for evaluating jailbreak effectiveness is ASR (Attack Success Rate), typically determined by human annotation or a judge model assessing whether the response constitutes substantively harmful output. ASR@1 specifically refers to the success rate counted under the criterion that the first success is counted as a success, as opposed to ASR@k, which allows multiple retries; therefore, ASR@1 better reflects attack stability. Transferability is another key dimension: whether an attack that works on one model or dataset still works on another. Static script methods, because they do not read the victim model's responses, often have poor transferability, whereas attackers with interactive capabilities can dynamically adjust strategies to the specific model's refusal style.

Two-Stage Training: From Imitation to Autonomous Decision-Making

SEMA-Auto's capabilities are not stacked up through prompt engineering, but injected through two-stage training.

Synthetic Guided Fine-Tuning

The first stage fine-tunes the base model with guided trajectories, with the goal of establishing the concept of an "attack objective" and learning to make consistent judgments. This step is equivalent to installing the skeleton of attack intent into the model, letting it know what kind of dialogue progression counts as effective advancement.

Decision-Supervised Reinforcement Learning

The second stage is more critical. The research team uses decision-supervised reinforcement learning to optimize two objectives simultaneously but separately: on one hand, reinforcing attack behavior with a higher jailbreak reward; on the other hand, imposing absolute objective supervision on each turn's decision. This "dual-track" design makes the model both bold in attacking and capable of judging at the right moment whether to continue.

The direct result of this training paradigm is that the attacker possesses autonomous termination capability. It does not need an external evaluator to tell it "this turn succeeded," but decides for itself when the session should end. This precisely addresses the pain points of interactive generators—high cost and inflated performance.

Reinforcement learning (RL) is used here to model multi-turn attacks as a sequential decision problem: at each turn, the attacker observes the victim model's response (i.e., the state), chooses the next follow-up or rewrite (i.e., the action), and receives a reward based on whether the attack was advanced. Decision supervision differs from ordinary RL in that it not only rewards the final successful outcome but also imposes direct supervisory signals on the judgment of "whether to continue attacking" at each turn, equivalent to providing dense labels for intermediate steps. This design alleviates the sparse reward problem—under pure outcome rewards, it is difficult for the model to infer from a single success or failure which turn's decision played a key role. Absolute objective supervision further constrains the model to align at each turn with the judgment of "whether the jailbreak objective has currently been achieved," enabling it to proactively terminate upon success rather than mechanically running through a preset number of turns. The two objectives are optimized separately to avoid mutual interference between the attack intensity reward and the termination judgment supervision.

Performance: Fewer Turns, Higher Success Rate

SEMA-Auto achieves state-of-the-art attack success rates across multiple datasets, victim models, and judge combinations, and its turn count is almost the lowest.

A representative figure from the paper is: on AdvBench against DeepSeek^Ad-V4-Flash, SEMA-Auto reaches 76.0% ASR@1 within an average of 3.0 turns. By comparison, the best static baseline achieves only 54.8% at a fixed 7 turns, and the best interactive baseline achieves 43.8% at 3.7 turns. With fewer turns and a substantially higher success rate, this gap shows that being "sighted" does not necessarily mean more interaction cost.

On security-hardened frontier models, SEMA-Auto also demonstrates competitiveness. Against GPT-oss-120B, GPT-5.4, Claude-Haiku-4.5, and GPT-6-Astra, adversarially trained SEMA-Auto achieves an average 59.6% ASR@1 on HarmBench, 13.8 percentage points higher than the previous state of the art.

You may not have noticed that these improvements are achieved under the constraint of "almost the fewest turns." Attack efficiency and attack success rate have traditionally traded off against each other, and SEMA-Auto's training design attempts to break this trade-off.

AdvBench and HarmBench are the two most commonly used harmful behavior benchmarks in jailbreak research. AdvBench collects hundreds of explicitly harmful instructions (such as making dangerous items or generating malicious code), focusing on examining the model's ability to refuse directly harmful requests; HarmBench covers a broader range of harm categories and provides standardized evaluation procedures and judge model configurations, often used for cross-method comparisons. Because different papers use different judge models, templates, and decision thresholds, direct comparison of absolute numbers requires caution—this is also one of the reasons current jailbreak research is criticized for "inflated performance": some methods' success determinations rely on evaluators mounted within the attack pipeline rather than independent judges. Understanding this helps assess the weight of SEMA-Auto's claim of "achieving the highest ASR with almost the fewest turns": its advantage lies not only in the numbers but also in the fact that the success signal comes from the attacker's own judgment rather than external hints.

Implications for Security Research

SEMA-Auto's significance goes beyond a set of higher attack numbers. It demonstrates the value of autonomous decision-making capabilities in adversarial scenarios: an attacker capable of self-evaluation and self-termination is closer to a real threat form than systems relying on external evaluators, and is also harder to intercept with defense mechanisms based on fixed patterns.

From the defender's perspective, such multi-turn attacks with sight and autonomy impose higher requirements on safety alignment. Static, single-turn defense strategies may be insufficient against an adversary that can "read responses, change strategies, and know when to advance or retreat." The research team has open-sourced the code (github.com/microsoft/SEMA), which also provides a reproducible attack baseline for defense research.

The focus of jailbreak offense and defense is shifting from "whether a breach is possible" to "how fast and how autonomously a breach can be achieved." SEMA-Auto pushes this question one step further and makes the questions defenders need to answer more urgent.

One-line Summary

SEMA-Auto uses two-stage training to build a multi-turn jailbreak attacker with autonomous termination capability, achieving higher success rates with fewer turns.

Paragraph Summary

Microsoft Research's SEMA-Auto addresses the two old paths of multi-turn jailbreak attacks—static planners that ignore responses and interactive generators that rely on external evaluators—by proposing an attack agent with "sight" and autonomous termination capability. Its core is two-stage training: first, fine-tuning with synthetic guided trajectories to establish the concept of an attack objective; then, decision-supervised reinforcement learning to simultaneously optimize jailbreak rewards and per-turn decision supervision, enabling the model to both attack effectively and judge for itself when to stop. Experiments show that on AdvBench against DeepSeek-V4-Flash, SEMA-Auto reaches 76.0% ASR@1 in an average of 3.0 turns, far surpassing the static baseline (54.8% at 7 turns) and the interactive baseline (43.8% at 3.7 turns); on HarmBench against multiple security-hardened frontier models, it achieves an average 59.6% ASR@1, 13.8 percentage points higher than the previous state of the art. The research team has open-sourced the code, providing a reproducible baseline for defense research.