LongCapHarness
A long-context reasoning enhancement framework that requires neither data synthesis nor post-training, guiding the model at inference time to invoke its own capabilities through a recursive self-improvement mechanism.
Timeline (last 90 days)
LongCapHarness is a framework that improves the long-context reasoning ability of large models through recursive self-improvement, without data synthesis or post-training.
LongCapHarness was applied on Llama3-8B and successfully transferred to Qwen2.5-3B.
On the HELMET and RULER benchmarks, LongCapHarness significantly outperforms recent data-synthesis-based baseline methods.
The estimated development cost of LongCapHarness is less than half of the training cost of NExtLong and PolicyLong.
The core of LongCapHarness is a recursive self-improvement loop that repeatedly revises the implementation of capability primitives, capability states and control logic on the development set.
Recursive self-improvement involves no post-training or gradient updates to the LLM; the model weights remain frozen, and each iteration requires only forward inference.
Capability primitives are the basic operation units required for long-context reasoning, such as information retrieval, cross-segment association and state tracking.
The effectiveness of LongCapHarness depends heavily on the quality and distribution of the development set, and the paper does not explain in detail how the development set is constructed or its scale.
The experiments of LongCapHarness are confined to the two benchmarks HELMET and RULER, and whether it is equally effective on a broader range of long-context tasks still needs verification.
All Facts (9)
LongCapHarness is a framework that improves the long-context reasoning ability of large models through recursive self-improvement, without data synthesis or post-training.
50%UnverifiedLongCapHarness was applied on Llama3-8B and successfully transferred to Qwen2.5-3B.
50%UnverifiedOn the HELMET and RULER benchmarks, LongCapHarness significantly outperforms recent data-synthesis-based baseline methods.
50%UnverifiedThe estimated development cost of LongCapHarness is less than half of the training cost of NExtLong and PolicyLong.
50%UnverifiedThe core of LongCapHarness is a recursive self-improvement loop that repeatedly revises the implementation of capability primitives, capability states and control logic on the development set.
50%UnverifiedRecursive self-improvement involves no post-training or gradient updates to the LLM; the model weights remain frozen, and each iteration requires only forward inference.
50%UnverifiedCapability primitives are the basic operation units required for long-context reasoning, such as information retrieval, cross-segment association and state tracking.
50%UnverifiedThe effectiveness of LongCapHarness depends heavily on the quality and distribution of the development set, and the paper does not explain in detail how the development set is constructed or its scale.
50%UnverifiedThe experiments of LongCapHarness are confined to the two benchmarks HELMET and RULER, and whether it is equally effective on a broader range of long-context tasks still needs verification.
50%