Markov chain
Markov property
Stochastic matrix
Semi-Markov process
Renewal theory
Continuous-time Markov process
Kolmogorov equations
Statistics
Markov processes
Markov models