Unverified50% confidenceFactExact time
策略梯度定理表明期望回报对策略参数的梯度可表示为 ∇J(θ) = E[∇log π_θ(a|s) · A(s,a)]，其中A是优势函数
1
Sources
50%
Confidence
Long-term
Relevance
8/27/2026
First Seen
Sources
Related Entities
Related Claims
Unverified策略梯度定理证明梯度可表示为∇J(θ) = E[∇log π(a|s;θ) · Q^π(s,a)]，使用log-derivative trick将对概率分布的求导转化为对log概率的求导乘以采样值78% similarUnverified策略梯度的核心思想源自 REINFORCE 算法（Williams, 1992），通过对轨迹回报的蒙特卡洛估计来计算策略参数的梯度72% similarUnverified只有基于势函数（Potential-based）的奖励塑形才能保证最优策略不变，附加奖励形式为F(s,s')=γΦ(s')-Φ(s)时原始MDP和塑形后MDP的最优策略等价67% similarUnverified策略梯度方法本质上是on-policy的，因为梯度估计中的期望是关于当前策略分布的65% similarUnverifiedThe advantage of the two-tower model is that during inference, the vectors from the job tower can be precomputed offline, enabling millisecond-level response60% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/809956API
curl https://kongchang.com/api/v1/knowledge/claims/809956MCP
get_claim(id=809956)