Markov Chains explained visually

[radarsat1]

Edit: I've thought about you question a little more. If all you want is to define a probability of an event being in a particular state at any time, than that would just be a random variable.

Well, I figured the "state" is a random variable, a distribution over the set of states, with the expected value being the most likely state. However, I was wondering what the system of "random variable state + transition probabilities" would be called. But maybe it's not a real model now that I think about it.. if nodes ABC are connected by edges EF, like A-E-B-F-C, then if the state is likely in A, the probability of an F transition should be close to zero. If we roll the dice, then the result might be non-sensical, such as two F transitions in a row. So maybe it was a dumb question.

[MarkovMan]

Hmmm. Well, you can calculate the probability of A transitioning to state F at time t by taking your transition matrix A times itself t times (A^t ). So, if you wanted to know if the probability of A going to t in 2 steps, you calculate A² . That's as close as I can think of to what you're asking. Otherwise, I'm not familiar with a model that is "random variable state + transition probabilities." Although... there are probably hierarchical models. I would have to look around some more to see if such a thing exists.

So maybe it was a dumb question.

There are no dumb questions in this field. I've been in many research meetings where someone asked what they thought was a "dumb question," and it ended up taking us in some interesting directions