[KongchangAI]
ModelRaRe 框架

RaRe

A reasoning-aware relational representation learning framework that uses chain-of-thought rationales generated by multimodal large language models to construct and refine scene graph relation representations, improving generalization to rare and unseen relations in open-vocabulary scene graph generation.

Timeline (last 90 days)

Nov 11

The researchers propose RaRe, a reasoning-aware relational representation learning framework that converts chain-of-thought (CoT) rationales generated by multimodal large language models (MLLMs) into structured relational embeddings and directly optimizes the representation space derived from the rationales.

Unverified50%
Nov 11

Through self-supervised contrastive learning, RaRe aligns the CoT rationale embeddings generated by the MLLM with textualized relation descriptions, bringing the rationale representations close to the corresponding relation semantics in the semantic space.

Unverified50%
Nov 11

RaRe adopts a specificity-oriented reinforcement learning objective to refine rationale generation, encouraging the model to produce relation evidence that is specific to particular object pairs and semantically more discriminative.

Unverified50%
Nov 11

Unlike approaches that treat CoT merely as a reasoning trace, RaRe directly optimizes the representation space derived from the rationales to improve discriminability among relations.

Unverified50%
Nov 11

On the VG150 and PSG datasets, RaRe outperforms existing baselines in class-balanced recognition.

Unverified50%
Nov 11

The paper notes that the additional gains from learning vary across datasets and proposal settings, and that the effect is related to factors such as data distribution and proposal quality.

Unverified50%
Nov 11

The code for RaRe has been made public on the project page.

Unverified50%

All Facts (7)

Source Articles