transition_matrix {pomdp} | R Documentation |
Converts the description of transition probabilities and observation
probabilities in a POMDP into a list of matrices. Individual values or parts of the matrices
can be more efficiently retrieved using the functions ending _prob
and _val
.
transition_matrix(x, episode = 1, action = NULL)
transition_prob(x, action, start.state, end.state, episode = 1)
observation_matrix(x, episode = 1, action = NULL)
observation_prob(x, action, end.state, observation, episode = 1)
reward_matrix(x, episode = 1, action = NULL, start.state = NULL)
reward_val(x, action, start.state, end.state, observation, episode = 1)
x |
A POMDP object. |
episode |
Episode used for time-dependent POMDPs (POMDP). |
action |
only return the matrix/value for a given action. |
start.state, end.state, observation |
name of the state or observation. |
See Details section in POMDP for details.
A list or a list of lists of matrices.
Michael Hahsler
Other POMDP:
POMDP()
,
plot_belief_space()
,
sample_belief_space()
,
simulate_POMDP()
,
solve_POMDP()
,
solve_SARSOP()
,
update_belief()
,
write_POMDP()
data("Tiger")
# List of |A| transition matrices. One per action in the from states x states
Tiger$transition_prob
transition_matrix(Tiger)
transition_prob(Tiger, action = "listen", start.state = "tiger-left")
# List of |A| observation matrices. One per action in the from states x observations
Tiger$observation_prob
observation_matrix(Tiger)
observation_prob(Tiger, action = "listen", end.state = "tiger-left")
# List of list of reward matrices. 1st level is action and second level is the
# start state in the form end state x observation
Tiger$reward
reward_matrix(Tiger)
reward_val(Tiger, action = "listen", start.state = "tiger")
# Visualize transition matrix for action 'open-left'
library("igraph")
g <- graph_from_adjacency_matrix(transition_matrix(Tiger)$"open-left", weighted = TRUE)
edge_attr(g, "label") <- edge_attr(g, "weight")
igraph.options("edge.curved" = TRUE)
plot(g, layout = layout_on_grid, main = "Transitions for action 'open=left'")
## Use a function for the Tiger transition model
trans <- function(action, end.state, start.state) {
## listen has an identity matrix
if(action == 'listen')
if(end.state == start.state) return(1)
else return(0)
# other actions have a uniform distribution
return(1/2)
}
Tiger$transition_prob <- trans
transition_matrix(Tiger)