SJR Architecture: A New Paradigm for Multimodal Content Moderation That Decouples Content Understanding from Policy Learning

Content moderation systems have long faced a structural challenge: multimodal understanding and policy classification are deeply coupled. Every time a moderation policy changes, the entire model pipeline must be retrained — and because multimedia data is difficult to augment effectively, the problem of label scarcity remains unsolved. A new arXiv paper introduces the Summarize-Judge-Refine (SJR) architecture, which attempts to fundamentally restructure this workflow.

Why Traditional Content Moderation Is Hard to Scale

Traditional content moderation models typically cram both "understanding the content" and "deciding whether it violates policy" into a single end-to-end network. This design creates two critical problems.

The first is the high cost of policy changes. Platform moderation policies are dynamic — today a certain type of misleading advertising may be prohibited, tomorrow a new restriction on a different form of expression might be added. In a tightly coupled architecture, any policy update means retraining the entire multimodal pipeline, incurring enormous engineering and compute costs.

The second is label scarcity. Text data can be augmented through synonym replacement, back-translation, and similar techniques, but multimedia content like images and videos cannot be "meaningfully augmented" — you can't arbitrarily modify an advertising image and guarantee it still represents the same type of violation. This makes violating samples naturally rare, making cold-starting models extremely difficult.

The Core Idea of SJR: Natural Language as the Interface

SJR's key innovation is decoupling content understanding and policy judgment into two independent models, using natural language as the interface between them.

Specifically, the architecture consists of two components:

• Content Model: A multimodal model responsible for converting raw content — images, videos, etc. — into structured text summaries. It only cares about "what's in here," not "whether this is a violation."
• Policy Model: A text-only model that takes the summaries generated by the content model and performs classification based on specific policy definitions.

The benefits of this division of labor are straightforward: when policies change, only the text-based policy model needs to be adjusted or replaced, without touching the expensive multimodal content model. The understanding layer and the decision layer are completely isolated.

Co-Training and Data Augmentation in Text Space

SJR doesn't simply stitch two models together — it designs an iterative co-training loop.

The system uses GRPO (Group Relative Policy Optimization) to refine the content model, encouraging it to generate summaries that are better suited for policy judgment — that is, to produce "policy-relevant" summaries rather than generic descriptions. This step ensures that summaries don't omit details that are critical for detecting violations.

Even more cleverly, augmentation happens in text space. Since content has already been converted into text summaries, augmentation techniques that were previously infeasible for raw multimedia become viable. SJR generates adversarial summary variants to construct large numbers of training samples, enabling few-shot policy cold-starting — an augmentation path that raw multimedia data simply cannot provide.

GRPO (Group Relative Policy Optimization) is a reinforcement learning-based language model optimization method and a variant of RLHF (Reinforcement Learning from Human Feedback). Unlike traditional PPO (Proximal Policy Optimization), GRPO generates a group of candidate outputs for the same input and updates the model using relative rewards within the group rather than absolute reward signals, reducing dependence on a separate critic network and making training more stable. In SJR, GRPO is used to back-optimize the content model's summary generation behavior based on the policy model's classification results — if a generated summary causes the policy model to misclassify, that summary receives a lower relative reward, prompting the content model to generate descriptions more focused on policy-relevant details in the next round. This co-training approach creates an effective end-to-end feedback loop between the two decoupled models, without merging them into a single multimodal model.

Experimental Results: New Policies Can Launch with Zero Real Violation Samples

On the misleading advertisement detection task, SJR's performance is impressive.

Compared to a zero-shot chain-of-thought (CoT) baseline, SJR achieves a 23.6% relative improvement in F1 score for the "non-misleading" category, and outperforms end-to-end SFT, STaR/RFT, and RLFT across the board.

The most notable finding is a special variant tested in the paper — trained without any real violation samples, with all positive-class data synthetically generated. This variant's violation-class F1 score falls within just 0.2% relative difference of the model trained on full real data.

The significance of this result is that it demonstrates new policies can go live with zero real violation data. For platforms that need to rapidly respond to new types of violating content but lack historical labeled data, this is nearly an ideal cold-start solution.

SFT (Supervised Fine-Tuning), STaR/RFT (Self-Taught Reasoner / Reinforced Fine-Tuning), and RLFT (Reinforcement Learning Fine-Tuning) are the three baseline training paradigms the paper compares against. SFT is the most basic approach, directly maximizing the likelihood of correct labels on annotated samples. STaR/RFT introduces the idea of "generating reasoning chains first and then self-supervising," using the model's own correct reasoning paths as additional training data. RLFT directly applies reinforcement learning signals to fine-tune the entire model's output distribution. All three methods operate on end-to-end or tightly coupled pipelines, meaning policy changes still require retraining the whole system. SJR's across-the-board lead in this comparison demonstrates that the benefits of the decoupled architecture itself exceed what can be gained purely through optimization algorithm improvements.

Interpretability as a Structural Byproduct

SJR also delivers an additional benefit: every moderation decision is grounded in human-readable text summaries.

In many end-to-end black-box moderation systems, it's often difficult to explain why a model flagged a piece of content. In the SJR architecture, because the judgment itself is made based on a natural language summary, reviewers can directly inspect that summary and understand the basis for the decision. Interpretability is no longer an add-on module — it's a natural product of the architectural design.

Summary

SJR decouples content understanding from policy learning through a natural language interface, offering a new path for multimodal content moderation that balances flexibility, data efficiency, and interpretability. Its three-stage Summarize-Judge-Refine design addresses both the retraining cost of policy changes and the multimedia label scarcity bottleneck through text-space augmentation. In particular, the experimental result showing that new policies can be launched with zero real violation samples has considerable practical implications for content safety deployment.