Bellman Language Model
A method that models the repeated decision-making of LLM agents as a Markov decision process and solves offline for a reusable policy. Its core procedure is decision-making after discretization, separating generative capability from decision optimization.
Timeline (last 90 days)
The Bellman Language Model (BLM) models repeated decisions as a Markov decision process (MDP), solves it once and reuses the solution, to improve the efficiency of LLM agents.
BLM's method is called "Discretize, Then Decide": starting from workflow logs or LLM agent run data, it reads each conversation as a discrete state and estimates probabilities and costs until no new states appear and the probabilities no longer change, then solves the MDP.
For the solved controller, actions with fixed content in a state are sent directly, while unseen states are handed back to the LLM for handling.
When the difficulty lies in writing content rather than choosing actions (for example, solving SymPy problems), few decisions can be resolved in advance, and BLM helps little.
BLM relies on the enumerability and stability of the decision space, and its advantage is limited for highly open or content-generation-intensive tasks.
All Facts (5)
The Bellman Language Model (BLM) models repeated decisions as a Markov decision process (MDP), solves it once and reuses the solution, to improve the efficiency of LLM agents.
50%UnverifiedBLM's method is called "Discretize, Then Decide": starting from workflow logs or LLM agent run data, it reads each conversation as a discrete state and estimates probabilities and costs until no new states appear and the probabilities no longer change, then solves the MDP.
50%UnverifiedFor the solved controller, actions with fixed content in a state are sent directly, while unseen states are handed back to the LLM for handling.
50%UnverifiedWhen the difficulty lies in writing content rather than choosing actions (for example, solving SymPy problems), few decisions can be resolved in advance, and BLM helps little.
50%UnverifiedBLM relies on the enumerability and stability of the decision space, and its advantage is limited for highly open or content-generation-intensive tasks.
50%