Markov Chains / Processes  

In General > s.a. formulations of quantum mechanics.
* Idea: A process in which a system evolves through a sequence of steps in some set of possible states, the probability of it going to a certain state in the next step depending only on the state it is in (no memory); It is characterized by a transition matrix T such that Tij ≥ 0 for all i, j and ∑i Tij = 1 for all j.
* History: Introduced by Markov in 1906, who just wanted to show that independence was not needed for the law of large numbers; An example he considered was the alternation of consonants and vowels in Pushkin's Eugene Onegin, which he described as a two-state Markov chain; Soon Poincaré was studying Markov chains on finite groups to study card shuffling; Today they are in all applied sciences, from population biology to communication networks, diffusion models, or social mobility.
@ General references: Revuz 84; Norris 97 [II]; Brémaud 99; Baldi et al 02 [and exercises]; Borovkov & Hordijk AAP(04) [normed ergodicity]; Stroock 05; Lecomte et al JSP(07)cm/06 [thermodynamic formalism]; Rivas et al NJP(10)-a1006 [master equations]; van Casteren 10 [time-dependent strong Markov processes on Polish spaces].
@ Markov semigroups: Kolokoltsov JSP(07); Androulakis & Ziemke JMP(15)-a1406 [quantum Markov semigroups].
@ Non-linear: Frank PLA(08); Frank PhyA(09) [chaos].
@ Non-equilibrium: Lubashevsky et al PhyA(09) [superstatistical description].
@ Numerical simulations: Stewart 94; Berg 04 [Monte Carlo]; Brémaud 08; Diaconis BAMS(09).
@ Evolution, examples: Cufaro Petroni & Vigier IJTP(79) [at the speed of light, and the Klein-Gordon equation]; Albeverio & Høegh-Krohn RPMP(84) [fields]; Schächter FP(87); Ibison CSF(99)qp/01 [1+1 Dirac equation]; Turova JSP(03) [states = directed graphs]; Duchi & Schaeffer JCTA(05) [jumping particles, and Catalan numbers]; Lecomte et al PRL(05) [dynamic partition function, entropy]; Grone et al JPA(08) [reversible, coarse-graining of stochastic matrix]; Hou et al a0805 [and growing networks]; Eliazar JPA(12) [Poissonian steady states]; > s.a. gas [lattice gas].
@ Path-space maximum entropy: Pavon & Ticozzi JMP(10)-a0811; Lee & Pressé JChemP(12)-a1206 [and n-th order Markov process master equation].
@ Evolution, related topics: Costanza PhyA(11), PhyA(12) [derivation of deterministic evolution equations]; Cubitt et al PRL(12) [solving the embedding problem]; Jeknić-Dugić et al a1510 [dynamical emergence of time-coarse-grained Markovianity].
> Related topics: see Master Equation [including generalizations and non-Markovian dynamics]; noether theorem.
> Online resources: see MathWorld page; Ryan Ward's page; Wikipedia page.

Related Processes > s.a. Martingales; random process [walk].
@ Generalizations: Schreiber JSP(10)-a0905 [polygonal Markov fields]; > s.a. stochastic processes [non-Markovian].

Quantum Markov Processes > s.a. Adiabaticity; open systems.
@ General references: Dynkin 82; Ghirardi et al PRA(90); Marbeau & Gudder AIHP(90); Gudder & Schindler JMP(91); Accardi et al mp/04 [for spin chains]; Tay & Petrosky PRA(07)-a0705 [thermal symmetry]; Ibinson et al CMP(08) [robustness]; Leifer & Poulin AP(08) [quantum graphical models of belief propagation]; Patra & Brooke PRA(08)-a0808 [decoherence-free quantum information]; Gudder JMP(08); Kraus et al PRA(08)-a0810 [and entanglement production]; Faigle & Schönhuth a1011 [discrete]; Vacchini et al NJP(11)-a1106 [in quantum and classical systems]; Chruściński & Kossakowski JPB(12)-a1201 [Markovianity criteria]; Fannes & Wouters a1204 [fermionic]; Matsumoto a1212 [loss of memory and convergence]; > s.a. dissipative systems.
@ Measures of "Markovianity": Wolf et al PRL(08); Haikka et al PRA(11)-a1011; Alipour et al PRA(12)-a1203 [from quantum discord]; Haseli et al QIP-a1406; > s.a. Loschmidt Echo.
@ Generalized: Breuer & Vacchini PRL(08) [semi-Markov process]; Tarasov TMP(09)-a0909 [fractional]; Brown & Poulin a1206 [Quantum Markov networks, and Gibbs states of Hamiltonians].

main pageabbreviationsjournalscommentsother sitesacknowledgements
send feedback and suggestions to bombelli at – modified 15 jan 2016