Unverified50% confidenceFactExact time
Vectors produced by many pre-trained models are not evenly distributed in space but are instead clustered in a narrow cone-shaped region, a phenomenon known as anisotropy.
1
Sources
50%
Confidence
Long-term
Relevance
10/11/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedEthayarajh et al. revealed through empirical research in 2019 that word vectors in pre-trained models such as BERT and GPT are highly concentrated in a narrow cone-shaped region, exhibiting anisotropy.68% similarUnverifiedContext confusion produces narrow misalignment, where misalignment is concentrated in specific domains semantically close to the training data, rather than a broad degradation of the model's overall behavior.68% similarUnverifiedBehavior Cloning (BC) suffers from the covariate shift problem: during training the model only sees the state distribution of expert trajectories, and at deployment small errors can drive the robot into states not covered by the training set, leading to cascading failures.67% similarUnverifiedIf samples exceeding 32K tokens rarely appear in a model's pre-training corpus, the model's performance in that range tends to be unstable even when the context window is extended through positional encoding.67% similarUnverified域随机化的代价是早期训练信号更加嘈杂、收敛更慢，但换来策略在整个参数空间内均匀分布的泛化能力66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/995595API
curl https://kongchang.com/api/v1/knowledge/claims/995595MCP
get_claim(id=995595)