3.5.1 Continouse time Markov chain

\begin{align*} p_{ii}\left ( h\right ) & =1-v_{i}h+o\left ( h\right ) \\ p_{ij}\left ( h\right ) & =q_{ij}h+o\left ( h\right ) \ \ \ \ \ \ \ \ \ i\neq j \end{align*}

\(v_{i}\) is the parameter (rate) for the exponential distributed random variable which represents the time in that state. Hence The probability that system remains in state \(i\) for time larger than \(t\) is given by

\[ \Pr \left ( T_{i}>t\right ) =e^{-v_{i}t}\]

\(\circ )\) Jump probability \(Q_{ij}=\frac {q_{ij}}{v_{i}}\) for \(\ i\neq j\). This is the probability of going from state \(i\) to state \(j\) (once the process leaves state \(i\))

\(\circ )\) FOrward Komogolv equation

\(P^{\prime }\left ( t\right ) =P\left ( t\right ) Q\), let \(z\left ( t\right ) =z\left ( 0\right ) P\left ( t\right ) \), hence \(z^{\prime }\left ( t\right ) =z\left ( 0\right ) P^{\prime }\left ( t\right ) \,,\) hence \(z^{\prime }\left ( t\right ) =z\left ( 0\right ) P\left ( t\right ) Q\) therefore

\[ z^{\prime }\left ( t\right ) =z\left ( t\right ) Q \]

\(\circ )\) Balance equations

\[ \pi _{j}v_{j}={\displaystyle \sum \limits _{k\neq j}} q_{kj}\pi _{k}\]

This is ’flow out’ = ’flow in’.

This equation can also be obtaind more easily I think from \(\mathbf {\pi }Q=\mathbf {0}\) Where \(Q\) is the matrix made up from the \(q^{\prime }s\) and the \(v^{\prime }s\) on the diagonal. Just write then down, and at the end add \(\pi _{0}+\pi _{1}+\cdots =1\) to find \(\pi _{0}\)

l.614 — TeX4ht warning — \SaveEverypar’s: 2 at \begindocument and 4 \enddocument —