Steady state in continuous time markov chain

In a continuous time Markov chain, the vector of the stationary probabilities still exists and is independent of the initial distribution. This vector π is the following system solution:

markov9

The system is called balance equations.

Example

Two identical machines operate continuously unless broken. A repairman available as needed to repair the machines.

The repair time follows an exponential distribution with an average of 0.5 day. Once repaired, the time of use of a machine before its next break follows an exponential distribution of average of 1 day. We assume that these distributions are independent. Consider the random process defined in terms of the number of machines down.

Let us consider the random variable X(t’) describing the number of machines that fail at time t’. The states of the random variable are {0, 1, 2}. The repair time and the break time follow an exponential distribution so we are in the presence of a continuous time Markov chain. The repair time follows an exponential distribution with an average of 0.5 day. The repair rate is the opposite, ie 2 machines per day. Similarly, we deduce that the broken rate is 1 per day. At the moment when the two machines work we have a break rate = machine1 + machine2 = 2.

The states describe the number of broken machines. The two machines can not break at the same time so q02 = 0. The repairer only repairs one machine at a time so q20 = 0. The repair rate is 2 machines per day. The breaking rate of a machine is 1 machine per day, and 2 per day if both machines are running. Which gives us the following continuous state Markov chain:

proba40

If we take the balance equations, we have the following system:

markov10

Which gives for solution the vector (0.4, 0.4, 0.2). If one tries to calculate the average number of broken machines, it is enough to calculate the expectation since the states represent the number of machines broken: 0 * 0.4 + 1 * 0.4 + 2 * 0.2 = 0.8.