ofxMarkovChain is an addon for openFrameworks that allows you to use Markov chains in a simple way.

A Markov chain is a mathematical system that includes a certain number of states. The transition from one state to another is determined by a probability. An interesting property of Markov chains is that the next transition only depends on the current state, thus making them memoryless. Here is a state diagram of a simple Markov chain with 2 states:

Markov chain

If the system is in the state , then the probability to stay in that state is and the probability to be in state is . Now, this system can be represented as a matrix:

You can notice that in order for the Markov chain to be coherent, the sum of the coefficients of each row of the matrix must be equal to .

An example provided with ofxMarkovChain is a random walker. The walker uses a Markov chain to choose in which direction to go next. The Markov chain has 4 states, each state corresponding to a direction: up, down, left or right.

Random walker