LongBanana: An Expert-Validated Benchmark for Long-Context Multi-Reference Image Synthesis

Multi-reference image synthesis is moving from single-image editing toward long-context tasks that involve "retrieving evidence—ruling out distractors—binding relationships—generating a coherent image." The recently released LongBanana benchmark aims to provide an auditable testbed for this new generation of agent-driven image generation systems.

Task definition: "assembling" one image from multiple reference images

LongBanana requires models to retrieve usable information from scattered visual evidence, reject distractor images irrelevant to the task, correctly bind entities and relationships, and ultimately synthesize a coherent image. What sets this apart from traditional text-to-image generation or single-reference image editing is the large number of reference images, the density of constraints, and the need for cross-image reasoning.

The benchmark contains 90 carefully designed tasks, organized under a 3×3 taxonomy. On average, each task includes about 32 reference images, 20 constraints, and 50 evaluation items, totaling 4,670 expert-specified atomic evaluation items. This high task density means models cannot pass by "happening to generate something that looks right."

Evaluation design: constraining generation results with entity-relationship graphs

The core innovation of LongBanana is its graph-grounded evaluation approach. Each task comes with reference images and an entity-relationship graph; during evaluation, obligations at the node, edge, and whole-graph levels are converted into checklists spanning three dimensions:

• Content Fidelity: whether the generated image contains the entities it should;
• Relational Binding: whether spatial and attribute relationships between entities are correct;
• Global Constraint Satisfaction: whether the entire image satisfies all constraint conditions.

This design shifts evaluation from vague holistic scoring to atomic judgments that can be verified item by item. On the expert annotation side, the benchmark includes a 15% double-annotated subset, with an exact agreement rate of 89.4% and a quadratic weighted kappa of 0.83, indicating that annotation quality is within an acceptable range.

Entity-Relationship Graph (ERG) is a representation method that structures image content into a graph structure: nodes represent entities in the image (such as objects, people, background elements), and edges represent spatial, attribute, or semantic relationships between entities (such as "on the left," "above," "color is red," "holding"). In LongBanana, each task is paired with such a graph, which serves as the "constraint blueprint" for the generation task. During evaluation, the system converts the nodes, edges, and whole-graph-level global obligations (such as "all entities do not overlap") into atomic check items that can be verified one by one, thereby transforming traditionally vague holistic similarity scoring into auditable discrete judgments based on graph structure. This design gives evaluation results greater interpretability and reproducibility, and also provides a unified measurement framework for comparing different generation systems.

LongBanana-Gen: the gains and boundaries of the agent framework

The accompanying LongBanana-Gen is an agent framework for long-horizon image generation. Experiments show that even a minimal agent harness can outperform direct generation; and after adopting the ReAct-Plan-Exec-Reflect (RPR) workflow, performance gains depend on the harness used—RPR brings benefits by strengthening evidence grounding and repair steps.

This result is noteworthy: agents are not a case of "add one and it works"—their effectiveness is strongly correlated with framework design. For developers, this means the upper bound of long-context image generation capability may depend more on how retrieval, planning, and repair processes are organized than on simply scaling up model size.

The ReAct-Plan-Exec-Reflect (RPR) workflow is an agent orchestration pattern for long-horizon generation tasks, whose name summarizes four core stages: Plan requires the agent to formulate a generation plan based on task constraints and reference images; Exec refers to calling the image generation model to produce intermediate or final images; Reflect lets the agent check the generation results against constraints and identify deviations; and ReAct emphasizes interleaving reasoning and action loops throughout planning and execution, that is, dynamically adjusting subsequent steps based on reflection results. Compared with a minimal harness, RPR improves generation quality through explicit evidence grounding and repair steps, but its gains are highly dependent on the specific implementation—if planning or reflection steps are poorly designed, the extra complexity may instead introduce noise. The significance of this framework lies in treating long-context image generation as a process requiring multi-step reasoning and iterative optimization, rather than a single forward generation.

Significance for the industry

LongBanana pushes "multi-reference image synthesis" from demonstrative cases toward quantifiable evaluation. Its 4,670 atomic evaluation items and auditable graph constraints provide a common goal for comparing different agentic image generation systems. Code and data are publicly available on the project page at https://long-banana.com/.

What is worth watching next: whether the benchmark can cover more open-ended real-world scenarios, and whether agent workflows beyond RPR can reproduce the gains under the same harness.

One-line Summary

LongBanana advances auditable evaluation of multi-reference image synthesis through graph-grounded evaluation and an agent framework.

Paragraph Summary

LongBanana is a new benchmark for long-context multi-reference image synthesis, requiring models to retrieve evidence from large numbers of reference images, rule out distractors, bind entity relationships, and generate coherent images. It contains 90 expert-designed tasks, each averaging about 32 reference images, 20 constraints, and 50 evaluation items, totaling 4,670 atomic evaluation items. Evaluation adopts a graph-grounded design, converting the nodes, edges, and global obligations of an entity-relationship graph into three types of checklists: content fidelity, relational binding, and global constraint satisfaction. Experiments with the accompanying LongBanana-Gen agent framework show that even a minimal harness already outperforms direct generation, while the RPR workflow can deliver harness-dependent gains by strengthening evidence grounding and repair. The benchmark provides an auditable common goal for comparing agentic image generation systems.