Discrete Probability Models: Markov Chains

Suppose a bounty hunter is tracking a fugitive who moves between three cities: Bugtussle, Wobegon and Funland. They move independent of one another; the fugitive begins in Bugtussle; the bounty hunter begins in Wobegon. Each day the fugitive changes cities with the following probabilities: given that the fugitive is in Bugtussle, the probability of moving to Wobegon is .4, given that the fugitive is in Wobegon, the probability of moving to Bugtussle is .7, and given that the fugitive is in Funland, the probability of moving to Bugtussle is .1. The bounty hunter may or may not change cities with the following probabilities: given that the bounty hunter is in Bugtussle, the probability of staying in Bugtussle is .3, of moving to Wobegon is .2, given that the bounty hunter is in Wobegon, the probability of staying in Wobegon is .1, of moving to Bugtussel is .8, and given that the bounty hunter is in Funland, the probability of staying in Funland is .3, of moving to Wobegon is .2. Justify how the properties of the Markov Chain are satisfied. Identify the questions you hope to answer using this process. Construct appropriate one-step transition probability matrix, incidence matrix, state space graph, probability tree, and multi-step probability and incidence matrix. Communicate the information, results and implications of these probabilistic tools and techniques. Identify insights gained through these processes. Propose a new area of application or new questions raised as a result of the model.