PlayGround: A Progressive Layout Generation Framework Based on Rendering Feedback

When AI generates graphic designs, the most common problem isn't that it "looks bad" — it's that you "can't edit it." A generated poster that can't be edited has limited value for designers. Getting AI to arrange text and visual elements into a harmonious layout while preserving content and editability is the real prerequisite for entering the design workflow.

A recent study called PlayGround proposes a progressive layout generation framework. The core idea is to let the model "draw, look, and adjust as it goes," just like a human designer, rather than locking in a global plan from the start.

The Ceiling of Existing Approaches: Plans Are Fixed Before Execution

In recent years, design generation methods based on Multimodal Large Language Models (MLLMs) have attempted to simulate real design workflows — iterative generation and visual refinement. But these methods share a common limitation: high-level planning is determined before execution.

In other words, the model first decides "place the title first, then the hero image, then the logo in the bottom right corner," and then executes step by step. Once rendering begins, the model cannot adjust subsequent steps based on what's actually on the canvas. If the first group of elements leaves insufficient whitespace or creates a confusing hierarchy, subsequent operations can only push forward regardless.

This "plan-then-execute" paradigm is far removed from how designers actually work. Human designers dynamically decide what to place next, where, and how to adjust based on what's actually on the canvas.

Multimodal Large Language Models (MLLMs) refer to large language models capable of simultaneously processing and understanding multiple modalities such as text and images, e.g., GPT-4V, LLaVA, etc. Unlike pure-text LLMs, MLLMs can take images as part of the input, combining image features extracted by visual encoders with text tokens and feeding them into a Transformer for inference. In design generation tasks, MLLMs typically handle understanding user requirements, inferring design intent, and outputting layout plans or code. Their advantage lies in cross-modal semantic understanding, enabling them to generate reasonable layout structures from natural language descriptions. However, constrained by the autoregressive generation paradigm and context window, MLLMs are prone to cumulative errors when generating long sequences of fine-grained layouts, and struggle to backtrack and modify already-output content during generation — this is one of the technical roots of the "plans are fixed before execution" limitation.

PlayGround's Core Mechanism: A Progressive Design Loop

PlayGround's core is a progressive design loop, with four components working in concert:

Structure-Aware Retriever: Given an unordered set of initial elements, it first infers a rough layout structure, then retrieves structurally similar completed designs from a reference library. This is akin to a designer browsing reference cases before starting work.

Progressive Planner: This is the heart of the framework. Each round, it simultaneously examines the current rendered canvas and the remaining unplaced elements, deciding which functional group to form next and how that group should be composed. The key is that its decisions are based on "what the current canvas actually looks like," not a preset global plan.

Layout Generator: Based on the planner's instructions, it predicts editable layout attributes — position, size, alignment relationships, etc.

Visual Verifier: It evaluates the rendered candidate solutions, providing feedback when quality falls short and triggering refinement.

This loop executes repeatedly, handling one functional group at a time until all elements are reasonably placed. Throughout the process, the model always plans the next step based on "the already-rendered canvas," achieving truly rendering-driven planning.

Functional Group refers to a collection of elements that serve the same semantic or visual function in a design. For example, a poster's title, subtitle, and author information can form a "text information group," the hero visual and background color blocks can form a "visual subject group," while the logo, QR code, and copyright notice might form an "identity information group." The division of functional groups is not based solely on element type, but combines semantic roles, spatial proximity, and visual hierarchy. In PlayGround, each round the planner handles only one functional group, rather than individual elements or all elements, which both reduces the complexity of single-step decisions and ensures layout coordination among elements within a group. The granularity choice of functional groups directly affects generation quality and the number of iteration rounds: too fine a granularity leads to excessive rounds and reduced efficiency, while too coarse a granularity may lose the flexibility of progressive adjustment.

A Two-Stage Data Pipeline: Teaching the Planner to "Think Step by Step"

To equip the progressive planner with step-by-step decision-making ability, step-level supervision data is needed. However, annotations in existing design datasets typically only contain final layouts, without recording process information about "what was placed first, what came next, and why each step was done that way."

The research team built a two-stage data generation pipeline: first extracting functional groups from completed designs (i.e., which elements belong to the same group semantically and visually), then reconstructing round-by-round planning trajectories. This way, the planner can learn decision sequences similar to those of human designers — establishing the main structure first, then filling in details, and finally adjusting balance.

Step-level Supervision is relative to traditional outcome-level supervision. In layout generation tasks, outcome-level supervision only requires the model's final output layout to match the ground-truth design, without concern for the intermediate process; step-level supervision requires the model to make reasonable planning actions at each round of decision-making, such as "which functional group should be placed now," "which region of the canvas should this group occupy," and "how should elements within the group be aligned." This supervision signal is closer to the thought process of human designers and can guide the model to learn step-by-step reasoning rather than one-shot prediction. Obtaining step-level annotations is costly because real design files typically retain only final layer information and don't record creative order. Therefore, the research team reconstructed planning trajectories from completed designs through reverse engineering — essentially a weakly supervised or self-supervised data construction strategy that provides a learnable decision sequence for the planner in the absence of manual step-by-step annotations.

Experimental Validation: Surpassing Existing Methods on Two Benchmarks

On the Crello and LICA design layout datasets, PlayGround outperforms existing methods. These two benchmarks cover diverse design scenarios, testing the framework's generalization ability across different types of layout generation tasks.

The research team states that code and weights will be publicly released, which is a positive signal for teams hoping to build productized applications on this foundation.

Why "Progressive" Is Closer to Practical Use Than "One-Shot"

From a product perspective, PlayGround's value isn't in how beautiful the designs it generates are, but in the fact that its generated designs are editable — each element's layout attributes are explicitly predicted, and users can continue modifying the generated results.

More importantly, the progressive framework naturally supports human-AI collaboration. Designers can intervene at any round, adjust the current canvas, and the model will continue planning subsequent steps based on the modified canvas. This interaction mode is far more efficient than "regenerate if you're not satisfied with a version."

Of course, progressive also means more computation rounds and a more complex reasoning process. How to balance quality and efficiency is a challenge that subsequent engineering efforts will need to address.

Editable Layout refers to generated results that save each element's position, size, rotation angle, alignment relationships, and other attributes in a structured form, rather than a flat bitmap. This representation allows users to directly select, move, scale, or replace any element in design tools like Figma or Sketch without breaking other parts of the overall layout. In contrast, end-to-end image generation models (such as Stable Diffusion) directly output pixel-level images — while visually appealing, almost all elements are baked into pixels, and modifying one part often requires regenerating the entire image. PlayGround's layout generator explicitly predicts layout attributes, essentially outputting a "parseable design file," which allows it to seamlessly integrate into existing design workflows. This is also one of its core product values that distinguishes it from pure image generation approaches.

One-line Summary

PlayGround proposes a progressive layout generation framework that lets AI dynamically plan the next step based on the current rendered canvas, enabling editable and interactive design generation.

Paragraph Summary

PlayGround is a study addressing the "can't edit it" pain point in AI graphic design generation. Existing MLLM-based methods typically formulate a global plan before execution and cannot dynamically adjust based on rendering results. PlayGround proposes a progressive design loop in which four components — a structure-aware retriever, a progressive planner, a layout generator, and a visual verifier — work in concert. Each round, it decides the composition of the next functional group based on the current canvas and remaining elements, achieving rendering-driven planning. To train the planner, the team built a two-stage data pipeline that extracts functional groups and round-by-round planning trajectories from completed designs to form step-level supervision. On the Crello and LICA benchmarks, PlayGround outperforms existing methods, and code and weights will be publicly released. Its generated results are editable and support designers intervening at any round, bringing it closer to practical workflows.