AesCode
An aesthetic code generation method that uses a decoupled cross-modal reward mechanism, treating images as aesthetic guidance and ultimately outputting editable code, used to generate information-dense visual artifacts such as slides, posters and dashboards.
Timeline (last 90 days)
The paper "AesCode: Aesthetic Code Generation with Decoupled Cross-Modal Rewards" proposes a code generation paradigm that uses an image generated from the same prompt as aesthetic guidance but ultimately delivers code.
AesCode represents the goal as a design graph that describes the entire canvas and makes every attribute such as position, color scheme, and hierarchical relationships attributable to specific graph elements and relations.
AesCode uses a complementary cross-modal reward mechanism comprising two types of rewards: a deterministic verifier and a VLM judge.
The deterministic verifier is responsible for evaluating attributes that can be parsed from code and rendering, such as element alignment, size matching, and correct content presentation; such rewards are verifiable and reproducible.
The VLM judge uses a sample-level Visual Graph Rubric (VGR) tied to design graph elements and relations to judge aesthetic quality.
The two types of rewards are decoupled and then jointly optimized via reinforcement learning with per-channel normalization.
The researchers constructed a new benchmark that scores verifiable correctness and visual quality separately, where the visual score is highly consistent with human judgment.
Experiments found that reference images significantly improve the visual quality of code generated by existing models.
Ablation experiments show that AesCode benefits from reference images in two ways: both as model input and as a source of visual reward.
AesCode relies on generating a reference image from the same prompt, and the quality of this step affects the final result; the introduction of a VLM judge also means that training and evaluation costs are not low.
All Facts (10)
The paper "AesCode: Aesthetic Code Generation with Decoupled Cross-Modal Rewards" proposes a code generation paradigm that uses an image generated from the same prompt as aesthetic guidance but ultimately delivers code.
50%UnverifiedAesCode represents the goal as a design graph that describes the entire canvas and makes every attribute such as position, color scheme, and hierarchical relationships attributable to specific graph elements and relations.
50%UnverifiedAesCode uses a complementary cross-modal reward mechanism comprising two types of rewards: a deterministic verifier and a VLM judge.
50%UnverifiedThe deterministic verifier is responsible for evaluating attributes that can be parsed from code and rendering, such as element alignment, size matching, and correct content presentation; such rewards are verifiable and reproducible.
50%UnverifiedThe VLM judge uses a sample-level Visual Graph Rubric (VGR) tied to design graph elements and relations to judge aesthetic quality.
50%UnverifiedThe two types of rewards are decoupled and then jointly optimized via reinforcement learning with per-channel normalization.
50%UnverifiedThe researchers constructed a new benchmark that scores verifiable correctness and visual quality separately, where the visual score is highly consistent with human judgment.
50%UnverifiedExperiments found that reference images significantly improve the visual quality of code generated by existing models.
50%UnverifiedAblation experiments show that AesCode benefits from reference images in two ways: both as model input and as a source of visual reward.
50%UnverifiedAesCode relies on generating a reference image from the same prompt, and the quality of this step affects the final result; the introduction of a VLM judge also means that training and evaluation costs are not low.
50%