A New AI Alignment Proposal: Don't Train the Next Generation Until You've Fixed the Current One

In the ongoing debate within AI safety circles about whether to pause frontier model development, a middle-ground approach is emerging: rather than a full stop, set a "minimum required pacing" for frontier AI progress. This proposal from the Alignment Forum lays out a clear and demanding standard — a lab should not train the next generation (more capable) model until it has adequately addressed all significant alignment problems in the current generation.

AI Alignment Pacing Proposal

The Core Argument: Fix It Before Moving On

The proposal rests on a straightforward analogy: you wouldn't rush to build a Mach 3 fighter jet before figuring out why your Mach 2 jet keeps crashing. The author pointedly observes that society wouldn't accept a "shrug and move on" attitude toward any other dangerous technology — so why should AI be any different?

Notably, the author takes care in a footnote to highlight a commonly overlooked standard: "scoring better than the previous generation on our metrics" does not equal adequately solving alignment. If a user identifies some previously unnoticed, strange, and alarming behavior after deployment, that's not just a model failure — it's a serious failure of the entire development process. In other words, the absence of evidence for a bad property is not the same as evidence of a good property.

The proposal's cleverness lies in its potential appeal to both optimists and pessimists about prosaic alignment. The fundamental divide between the two camps comes down to how hard it actually is to fix current alignment problems using existing techniques.

Optimists might believe a mild, coordinated slowdown is sufficient to fully address alignment at each generation, allowing development to continue — cautiously but steadily.
Pessimists might anticipate this would evolve into a prolonged, substantive pause — one that buys precious time to explore fundamentally new approaches to aligned AI.

Whichever camp you're in, this standard leads to the outcome you'd endorse. That's the political wisdom of proposing it as a "coordinated pacing" solution.

"Prosaic alignment" is an important conceptual branch within AI safety. It refers to using existing machine learning techniques — such as RLHF (Reinforcement Learning from Human Feedback), supervised fine-tuning, Constitutional AI, and similar methods — to solve alignment problems in current models, without waiting for breakthroughs in entirely new alignment theory. This contrasts with "fundamental alignment" research, which seeks to address deeper value alignment problems from a mathematical or theoretical standpoint. Optimists believe prosaic alignment techniques are already sufficient to handle the challenges posed by current models, and that with reasonable development pacing, each generation can be "fixed" before moving on. Pessimists argue that current engineering patches treat symptoms rather than causes — once truly dangerous capability emergence occurs, existing tools will likely prove inadequate, making it essential to buy time to develop more fundamental solutions. This divide directly determines whether the "minimum required pacing" proposal would play out as a gentle slowdown or a substantive long-term pause in practice.

What Does "Adequately" Solving Alignment Actually Mean?

The author readily acknowledges this is a vague, qualitative proposal, but sketches out a fairly rigorous procedural framework.

Red-Teaming Throughout

Multiple mutually independent alignment organizations review and red-team the latest model at several checkpoints throughout training, with full data access and the best available tools. The review criteria go beyond quantitative metrics — whether it's an anomalous score, a qualitative sense that something feels wrong, the presence of a bad property, or the inability to robustly demonstrate a good property, all of it must be taken seriously.

Alignment problems that appear early in training aren't necessarily fatal, but there must be a robust safety case explaining why subsequent training stages can resolve them — you can't just paper over them with optimistic assumptions.

"Red teaming" in the AI safety context refers to a dedicated team actively trying to surface harmful behaviors, jailbreak vulnerabilities, or alignment failures in a model — simulating real-world malicious or accidental use scenarios. It's analogous to penetration testing in traditional software security, but AI red-teaming faces additional challenges: the behavioral space of language models is nearly infinite and can't be exhaustively tested; models may only exhibit dangerous behaviors under specific triggering conditions; and evaluators themselves may have blind spots. The proposal's insistence that red-teaming organizations be "mutually independent" is precisely to prevent any single institution's blind spots or conflicting interests from causing something to slip through. The concept of a "safety case" comes from high-risk engineering domains (such as aviation and nuclear power) and requires a structured document explicitly arguing why a system is safe under specified conditions, along with the assumptions and evidence chain underlying that argument — this is far more auditable than a simple "passed testing" report.

Restricted Deployment and a Reset Mechanism

Only when alignment organizations are satisfied with the model does a new generation enter restricted deployment. Restricted deployment starts with a small group of trusted users performing tasks that don't feed back into model training, while alignment organizations continuously monitor for new issues.

If any significant alignment problem surfaces during restricted deployment, the process calls for a strict "reset":

Immediately pull the model;
Conduct a root cause analysis and investigate why training failed to catch the problem;
Find a principled way to fix the disease rather than the symptom;
Train the next iteration of the current generation accordingly, then start the whole process over.

Standing the Test of Time Before Moving On

A new generation of model is only cleared for broader deployment (internal or external) after it has "survived" a sufficiently long period of restricted deployment with no problems emerging. If significant alignment issues appear during broad deployment, the same reset process is triggered.

Only after a considerable deployment period with no alignment problems surfacing is a lab permitted to begin training the next generation. The author humbly adds: even then, he doesn't consider this "fully sufficient" — but he'd treat it as a minimum baseline.

An Unfinished Proposal

The author openly acknowledges this reads more like a heuristic draft than a ready-to-implement plan, and proactively lists several obvious gaps, inviting others to help fill them:

How do you define "a generation" of model? Within a single lab, or across multiple labs?
What if an intervention that fixes an alignment problem also significantly boosts capabilities?
How do you define and enforce a protocol in cases where predefined metrics aren't sufficient to navigate by?
How severe does an alignment problem need to be before it should halt a model? Here, labs face enormous pressure — they have a natural incentive to set this threshold very high.

That last point is especially sharp. Any mechanism that relies on lab self-regulation can't escape the problem of misaligned incentives: when commercial competition clashes with safety standards, who holds the line?

The "capability-alignment entanglement" problem is one of the thorniest challenges this proposal faces. Current mainstream alignment techniques (like RLHF) are often deeply intertwined with capability improvements — better instruction-following is simultaneously an alignment achievement and a capability gain; certain improvements that reduce hallucinations also enhance reasoning ability. This means "fixing alignment without improving capabilities" may be a condition that's technically impossible to strictly satisfy. On top of that, the definition of "a generation" is extremely fuzzy in today's fast-iterating industry — from GPT-4 to GPT-4o to various fine-tuned versions, capability jumps can happen at any update, not just at integer-generation boundaries. How to draw a "generational boundary" on a continuous capability evolution curve directly affects this proposal's operability, and neither academia nor industry has reached consensus on this.

A Closing Observation

The value of this proposal lies not in providing complete answers — it openly admits it can't — but in offering a "minimum shared framework" for parties across the debate to engage with. Compared to the radical call for a comprehensive pause and the status quo of unconstrained racing, "properly align the current generation first" occupies a middle ground that's intuitively harder to argue against and more likely to find acceptance across different camps.

Of course, the gap between a qualitative heuristic and a concrete, enforceable, auditable, internationally coordinated protocol remains enormous. But in an era of accelerating frontier model capabilities, establishing "fix it before moving forward" as a basic industry norm may be more realistic than debating whether we can slam the brakes entirely.