Unverified50% confidenceFactExact time
In 2018, Henderson et al. published a paper noting that the experimental results of many RL papers are highly sensitive to minor differences in random seeds, hyperparameters, and code implementations.
1
Sources
50%
Confidence
Long-term
Relevance
8/29/2026
First Seen
Sources
Related Entities
Related Claims
Unverified2018年 Henderson 等人的研究表明许多已发表的 RL 论文中的性能提升在更换随机种子后无法复现75% similarUnverified2023年多项研究证明，即使经过RLHF对齐的模型也难以完全抵抗精心设计的间接提示词注入72% similarUnverifiedIn 2021, NeurIPS conducted a famous experiment in which the same paper was submitted twice and received completely different review outcomes, revealing the high degree of randomness in the review process.67% similarUnverifiedWhen randomized experiments are not feasible, causal inference methods become critical, including Difference-in-Differences (DID), Regression Discontinuity Design (RDD), Instrumental Variables (IV), and Propensity Score Matching (PSM).66% similarUnverifiedA 2018 study by the Google Brain team showed that simply changing the random seed could cause performance fluctuations of several percentage points in certain models.65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/822100API
curl https://kongchang.com/api/v1/knowledge/claims/822100MCP
get_claim(id=822100)