POMDP
Partially Observable Markov Decision Process (POMDP) is a mathematical framework in reinforcement learning and decision theory that models sequential decision-making problems in which an agent cannot fully observe the underlying environment state. It is widely applied in domains such as robot navigation, game AI, and financial decision-making.
Core Facts
Timeline (last 90 days)
Even if the state space of the original POMDP is finite, its corresponding belief MDP has a continuous state space, and exact planning is infeasible in general.
The paper provides end-to-end finite-time error guarantees from the true POMDP value to the algorithm's estimated value.
The paper proves a finite-time bound on the gap between the particle belief MDP surrogate and the original POMDP as its core theoretical result.
Introducing risk-sensitivity by modifying the value function destroys the standard structure of the original problem, forcing researchers to design entirely new custom algorithms instead of reusing existing expectation-based planners.
Traditional online POMDP planners aim to optimize expected cumulative cost, and when the belief state assigns significant probability mass to high-cost states, purely pursuing the expectation optimum masks potentially dangerous states.
Clash Royale is a typical partially observable environment (POMDP), in which players cannot fully observe the opponent's hand, elixir accumulation, or card-playing intentions.
The Fog of War mechanic tests an AI's planning capability in a "Partially Observable Markov Decision Process" (POMDP) environment.
Partially observable environments correspond to Partially Observable Markov Decision Processes (POMDPs), which are computationally far more difficult than fully observable Markov Decision Processes (MDPs).
解决POMDP的经典方法是维护信念状态（belief state），但在高维空间中计算上不可行；现代深度强化学习通过LSTM或GRU等循环结构隐式维护历史信息
现代深度强化学习通过在策略网络中引入循环结构（如LSTM或GRU）来隐式维护历史信息以处理POMDP
5 more timeline events
All Facts (15)
Exactly solving POMDPs is computationally PSPACE-hard.
75%UnverifiedEven if the state space of the original POMDP is finite, its corresponding belief MDP has a continuous state space, and exact planning is infeasible in general.
50%UnverifiedThe paper proves a finite-time bound on the gap between the particle belief MDP surrogate and the original POMDP as its core theoretical result.
50%UnverifiedThe paper provides end-to-end finite-time error guarantees from the true POMDP value to the algorithm's estimated value.
50%UnverifiedTraditional online POMDP planners aim to optimize expected cumulative cost, and when the belief state assigns significant probability mass to high-cost states, purely pursuing the expectation optimum masks potentially dangerous states.
50%UnverifiedIntroducing risk-sensitivity by modifying the value function destroys the standard structure of the original problem, forcing researchers to design entirely new custom algorithms instead of reusing existing expectation-based planners.
50%UnverifiedClash Royale is a typical partially observable environment (POMDP), in which players cannot fully observe the opponent's hand, elixir accumulation, or card-playing intentions.
50%UnverifiedThe Fog of War mechanic tests an AI's planning capability in a "Partially Observable Markov Decision Process" (POMDP) environment.
50%UnverifiedPartially observable environments correspond to Partially Observable Markov Decision Processes (POMDPs), which are computationally far more difficult than fully observable Markov Decision Processes (MDPs).
50%Unverified解决POMDP的经典方法是维护信念状态（belief state），但在高维空间中计算上不可行；现代深度强化学习通过LSTM或GRU等循环结构隐式维护历史信息
50%Unverified现代深度强化学习通过在策略网络中引入循环结构（如LSTM或GRU）来隐式维护历史信息以处理POMDP
50%UnverifiedMDP假设当前状态包含做出最优决策所需的全部信息（马尔可夫性质），而POMDP中智能体无法直接观测完整的环境状态
50%UnverifiedA POMDP is defined by a 7-tuple (S, A, T, R, Ω, O, γ), in which the agent cannot directly observe the state and can only maintain a belief state through observation signals.
50%UnverifiedThe work of Williams and Young in 2007 established the theoretical foundation for POMDP-based dialogue management, demonstrating its superiority over rule-based or deterministic state-tracking systems in modeling uncertainty.
50%UnverifiedPOMDP（部分可观测马尔可夫决策过程）是MDP（马尔可夫决策过程）的推广形式，在标准MDP中智能体能完全观测环境状态，而POMDP承认智能体无法直接获知真实状态，只能通过带噪声的观测间接推断
50%