Unverified50% confidenceFactExact time
In the code output prediction task on the coding benchmark LiveCodeBench, the compositional generalization performance of the pretrained model Llama3-8B is affected by compositional equivalence.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
11/11/2026
First Seen
Valid until: 2/9/2027
Sources
Related Entities
Related Claims
UnverifiedLiveCodeBench Hard-level problems typically require multi-step reasoning, complex algorithm design, and handling of edge cases, placing extremely high demands on the model's one-shot code correctness rate.78% similarVerifiedLiveCodeBench continuously crawls new problems from competitive programming platforms such as LeetCode, Codeforces, and AtCoder to prevent models from gaming the benchmark, and its evaluation covers four dimensions: code generation, self-repair, code execution reasoning, and test output prediction.76% similarUnverifiedCode generation quality depends on model scale, training data quality, and context window size; current mainstream models are already quite reliable at completing a single line of code, but complex refactoring and long-range logical consistency still require human review.74% similarUnverifiedTraining reproducibility depends on fixing data versions and preprocessing scripts, complete records of random seeds and hyperparameter configurations, periodic saving of checkpoints, and version control of reward function code.73% similarUnverifiedThe training pipeline for modern code models typically includes a data deduplication step that detects and removes near-duplicate code snippets using techniques such as MinHash and SimHash.73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1158436API
curl https://kongchang.com/api/v1/knowledge/claims/1158436MCP
get_claim(id=1158436)