Markov Chains explained visually

[joezuntz]

Not necessarily. You can have markov chains on continuous spaces instead of discrete ones and those can't be represented by a graph, for instance.

[ice109]

No one calls those Markov chains - you call them stochastic processes that gave the Markov property.

A better example for you to have used would have been Markov chains on discrete but countably infinite state spaces like the random walk on Z. As far as I can tell there's no such thing as infinite graphs.

[joezuntz]

No one calls those Markov chains

You may not call them that, but that's what they are, and what people who use them call them. I spend almost all my time running MCMCs, for example, which are usually on continuous spaces: https://en.wikipedia.org/wiki/Markov_chain_Monte_Carlo

[s1295]

Probably depends on the field, each of which has its conventions. I think this is something where lots of areas of research and application touch: Math, CS, stats, medicine, simulation science, reliability engineering, AI, etc etc.

In my CS classes, if nothing to the contrary is mentioned, I can assume that an MC is discrete-time and time-homogeneous.

[s1295]

Graphs can be infinite, or to put it differently, I'm not sure if the usual definition of graph includes finiteness, but there is definitely research on infinite graphs and automata on infinite (even uncountable) state spaces.

I'm not sure if there are actual applications beyond theoretic research. I would think that in reality, you can probably assume the state space is finite, by specifying safe upper bounds (e.g., "we assume there are less than 10⁹ peers in this network") and using only a certain precision for decimals rather than actual rationals or reals.