thumb|A single realisation of three-dimensional [[Brownian motion for times 0 ≤ t ≤ 2. Brownian motion has the Markov property, as the displacement of the particle does not depend on its past displacements.]]

In probability theory and statistics, the Markov property is the memoryless property of a stochastic process, which means that its future evolution is independent of its history. It is named after the Russian mathematician Andrey Markov. The term strong Markov property is similar to the Markov property, except that the meaning of "present" is defined in terms of a random variable known as a stopping time.

The term Markov assumption is used to describe a model where the Markov property is assumed to hold, such as a hidden Markov model.

A Markov random field extends this property to two or more dimensions or to random variables defined for an interconnected network of items. An example of a model for such a field is the Ising model.

A discrete-time stochastic process satisfying the Markov property is known as a Markov chain.

Introduction

A stochastic process has the Markov property if the conditional probability distribution of future states of the process (conditional on both past and present values) depends only upon the present state; that is, given the present, the future does not depend on the past. A process with this property is said to be Markov or Markovian and known as a Markov process. Two famous classes of Markov process are the Markov chain and Brownian motion.

Note that there is a subtle, often overlooked and very important point that is often missed in the plain English statement of the definition: the statespace of the process is constant through time. The conditional description involves a fixed "bandwidth". For example, without this restriction we could augment any process to one which includes the complete history from a given initial condition and it would be made to be Markovian. But the state space would be of increasing dimensionality over time and does not meet the definition.

History

Definition

Let <math>(\Omega,\mathcal{F},P)</math> be a probability space with a filtration <math>(\mathcal{F}_s,\ s \in I)</math>, for some (totally ordered) index set <math>I</math>; and let <math>(S,\Sigma)</math> be a measurable space. An <math>(S,\Sigma)</math>-valued stochastic process <math>X=\{X_t:\Omega \to S\}_{t\in I}</math> adapted to the filtration is said to possess the Markov property if, for each <math>A \in \Sigma</math> and each <math>s,t\in I</math> with <math>s<t</math>,

:<math>P(X_t \in A \mid \mathcal{F}_s) = P(X_t \in A\mid X_s).</math>

In the case where <math>S</math> is a discrete set with the discrete sigma algebra and <math>I = \mathbb{N}</math>, this can be reformulated as follows:

:<math>P(X_{n+1}=x_{n+1}\mid X_n=x_n, \dots, X_1=x_1)=P(X_{n+1}=x_{n+1}\mid X_n=x_n) \text{ for all } n \in \mathbb{N}.</math>

In other words, the distribution of <math>X</math> at time <math>n+1</math> depend solely on the state of <math>X</math> at time <math>n</math> and is independent of the state of the process at any time previous to <math>n</math>, which corresponds precisely to the intuition described in the introduction.

If <math>I=[0,\infty)</math>, then <math>X</math> is called time-homogeneous if for all <math>t,s\geq 0</math> the weak Markov property holds:

:<math>P(X_{t+s}\in A\mid \mathcal F_s)=P(X_t\in A\mid X_0=x)|_{x=X_s}=:P^{X_s}(X_t\in A)</math>.

The newly introduced probability measure <math>P^x(X_t\in \cdot)</math>, <math>x\in S</math>, has the following intuition: It gives the probability that the process <math>X</math> lies in some set at time <math>t</math>, when it was started in <math>x</math> at time zero. The function <math>P_t(x,A):= P^x(X_t\in A)</math>, <math>(t,x,A)\in \R_+\times S\times \Sigma</math>, is also called the transition function of <math>X</math> and the collection <math>(P_t)_{t\geq 0}</math> its transition semigroup.

Alternative formulations

There exists multiple alternative formulations of the elementary Markov property described above. The following are all equivalent:

  • For all <math>t\geq 0</math> the <math>\sigma</math>-algebras <math>\mathcal F_t</math> and <math>\mathcal F_t' := \sigma(X_s:s\geq t)</math> are conditionally independent given <math>X_t</math>. In other words, for all <math>A\in \mathcal F_t</math>, <math>B\in\mathcal F_t'</math>:

<math>P(A\cap B\mid X_t)=P(A\mid X_t)P(B\mid X_t)</math>.

  • For all <math>t\geq 0</math>, <math>B\in\mathcal F_t'</math>:

<math>P(B\mid \mathcal F_t)=P(B\mid X_t)</math>.

  • For all <math>t\geq 0</math>, <math>A\in\mathcal F_t</math>:

<math>P(A\mid\mathcal F_t^')=P(A\mid X_t)</math>.

  • For all <math>t\geq 0</math> and <math>Y:\Omega\rightarrow \mathbb{R}</math> bounded and <math>\mathcal F_t'</math>-measurable

<math>\operatorname{E}[Y\mid\mathcal{F}_t]=\operatorname{E}[Y\mid X_t]</math>.

  • For all <math>t\geq s\geq 0</math> and <math>f:S\rightarrow \mathbb{R}</math> bounded and measurable

<math>\operatorname{E}[f(X_t)\mid\mathcal{F}_s]=\operatorname{E}[f(X_t)\mid X_s ]</math>.

  • For all <math>t\geq s\geq 0</math> and <math>f:S\rightarrow \mathbb{R}</math> continuous with compact support

<math>\operatorname{E}[f(X_t)\mid\mathcal{F}_s]=\operatorname{E}[f(X_t)\mid X_s ]</math>.

  • For all <math>0\leq s_1<...<s_n<s<t</math> and <math>f:S\rightarrow \mathbb{R}</math> continuous with compact support

<math>\operatorname{E}[f(X_t)\mid X_s,X_{s_n},...,X_{s_1}]=\operatorname{E}[f(X_t)\mid X_s]</math>.

If there exists a so-called shift-semigroup <math>(\theta_t)_{t\geq 0}</math>, i.e., functions <math>\theta_t:\Omega\to\Omega</math> such that

  1. <math>\theta_0=\mathrm{id}_\Omega</math>,
  2. <math>\theta_t\circ \theta_s = \theta_{t+s} \quad\forall s,t\geq 0 </math> (semigroup property),
  3. <math>X_t\circ\theta_s = X_{t+s} \quad\forall s,t\geq 0 </math>,

then the Markov property is equivalent to: The converse is in general not true.

The strong Markov property only leads to non-trivial results in continuous time (i.e., results which do not hold with merely the Markov property), as in the discrete case the strong and the elementary Markov property are equivalent.

<!-- Hide for now ... name Markov-type property seems doubtful

Markov Type Property

A stochastic process has a Markov-type property if the process's random variables determine a set of probabilities can be factored in a way that yields the Markov property. Useful in applied research, members of such classes defined by their mathematics or area of application are referred to as Markov random fields., and occur in many situations. The Ising model is a prototypical example. -->

Feller property

Although the strong Markov property is in general stronger than the elementary Markov property, it is fulfilled by Markov processes with sufficiently "nice" regularity properties.

A continuous time Markov process is said to have the Feller property, if its transition semigroup <math>(P_t)_{t\geq 0}</math> (see above) fulfills

Stochastic processes

Many prominent stochastic processes are Markov processes: The Brownian motion, the Brownian bridge, the stochastic exponential, the Ornstein-Uhlenbeck process and the Poisson process have the Markov property.

More generally, any semimartingale <math>X</math> with values in <math>\R^n</math> that is given by the stochastic differential equation

:<math>X_t = X_0 + \sum_{i=1}^d \Big[\int_0^t g_i(X_s)ds + \int_0^t f_i(X_s)dB_s^i \Big]</math>,

where <math>B=(B^1,...,B^d)</math> is a <math>d</math>-dimensional Brownian motion and <math>f_1,...,f_d,g_1,...,g_d:\R^n\to\R^n</math> are autonomous (i.e., they do not depend on time) Lipschitz functions, is time-homogeneous and has the strong Markov property. If<math>f_1,...,f_d,g_1,...,g_d</math> are not autonomous, then <math>X</math> still has the elementary Markov property.