Entropy

In General > s.a. chaos; entropy bounds; statistical mechanics; thermodynamics.
* Idea, in elementary thermodynamics: An extensive thermodynamical function of state which grows monotonically with E and is constant in a reversible adiabatic transformation; Defined, up to an additive constant, by dS = dQ/T.
* History: Concept introduced by Clausius in the 1860's; > s.a. history of physics.
* Idea, statistical interpretation: A measure of how spread out a classical distribution function or quantum density matrix is; Defined by S = –k tr ln , which becomes S = k ln N when corresponds to N equally probable microscopic states; The constant k is identified with Boltzmann's constant.
* And ignorance: Generally, one views entropy as a measure of our ignorance of the microscopic state of a system; This seems to make entropy a subjective thing which, for a given system, depends on how much we wanted to find out, and could be decreased if we just measured something more; This is so, but in practice it is not a real problem, because S ln N, N = number of states compatible with our observations, and the kind of measurements we could think of to decrease our subjective entropy might lower N by a factor of, say, 100; After taking the log, this becomes a very small amount to subtract from the previous entropy, and it makes no real difference; Therefore, in practice one doesn't need to introduce a definition of entropy different from the usual one (from R Sorkin, meeting 1.02.1985).
* And classical / quantum mechanics: In classical mechanics one assumes entropies are finite, but if one takes the → 0 limit of any quantum entropy expression, this diverges; So classical thermodynamics is not the classical limit of quantum statistical mechanics, it knows about quantum mechanics.

Information Theoretical (Shannon) Entropy > s.a. Brudno's Theorem; information; H Theorem; Landauer's Principle.
$ Def: The Shannon uncertainty; For a mixed state = _n p_n |n n|, where {|n} is a complete set of states,

S = –k _n p_n ln p_n .

It is equivalent to the Boltzmann-Gibbs definition of entropy under equilibrium conditions.
* Remark: Actually, we could use the log with any base, or any other convex function f(p_n/q_n), with q_n the equilibrium probabilities; For any such f the entropy would increase; Other properties like additivity put constraints on f.
* Remark: Then one bit of information corresponds to an entropy ln 2.
@ General references: Jaynes PR(57), PR(57) [maximum entropy principle]; Fahn FP(96) [and thermodynamics]; Fa JPA(98) [generalization]; Luo JPA(00) [proof of Wehrl's conjecture]; Chakrabarti & Chakrabarty IJMMS(05)qp [axiomatic]; Maroney qp/07 [Gibbs-Von Neumann, motivation]; Ladyman et al SHPMP(08).
@ Boltzmann entropy: Swendsen AJP(06) [for colloids, clarification]; Kalogeropoulos a0804 [variation wrt energy].
@ And correlations: Van Drie mp/00; Gu et al qp/06.
@ Wehrl information entropy: Miranowicz et al JPA(01)qp; Piatek & Leonski JPA(01) [entanglement and correlations].

Kolmogorov-Sinai Entropy
* Idea: The growth rate h of the phase space volume of a phase drop with time; By Liouville's theorem, for a Hamiltonian system we have h = 0 if there is no coarse-graining; If V is a coarse-grained phase space volume, we estimate V(t) = V₀ e^ht, and define

h:= lim_{V_0 to 0} lim_{t to infty} (1/t) ln V(t) .

* Remark: Notice that h is not actually an entropy (confusing name!), but the time derivative of the entropy S ln V; Related to the stability or instability (and random behavior) of the system.
* And Lyapunov exponents: Related by h = _i=1ⁿ _i [@ Pesin UMN 77].
@ References: see Klimek & Lesniewski AP(96) [non-commutative Connes-Størmer entropy]; Frigg BJPS(04) [and chaos].

Other Entropies > s.a. modified thermodynamics [non-extensive, relativistic, etc].
* Rényi entropy: S_q^R = (ln _i p_i^q)/(1–q).
* Non-extensive entropy: Applicable when microscopic interactions and memory are long-ranged, and extensive thermo-statistics fails; The Tsallis definition is S_q = k (1–_i p_i^q)/(q–1), with q > 1; The Boltzmann–Gibbs expression is recovered as q → 1; > s.a. chaos, cmb, stat mech.
* Metric entropy of families of metric spaces: The asymptotic behavior of covering numbers.
@ Classical mechanics: Brun & Hartle PRE(99)qp/98 [histories]; McLachlan & Ryland mp/02/JMP [algebraic].
@ Non-equilibrium: Holian PRA(86); Kandrup JMP(87); Martyushev et al JPA(07), Maes & Netocny JMP(07) [minimum entropy production].
@ Dynamical entropy: Connes et al CMP(87)*; Hudetz LMP(88); Benatti et al JPA(04) [and discrete chaos].
@ Algebraic entropy: Bellon & Viallet CMP(99) [discrete t systems].
@ Localization entropy: Schroer ht/01 [and area law].
@ Rényi entropy: Jizba & Arimitsu PhyA(04)cm/03 [and Tsallis' non-extensive S]; Masi PLA(05) [and Tsallis, common framework]; Harremoës PhyA(06) [operational]; Figueiredo et al PhyA(06) [statistical, and Tsallis]; Campisi & Bagci PLA(07) [and Tsallis ensemble]; Romera & Nagy PLA(08) [for atoms].
@ Tsallis entropy: Tsallis JSP(88); Suyari JPA(02); Furuichi et al JMP(04), Furuichi JMP(06) [properties]; Sattin PS(05) [interpretation ito incomplete knowledge]; Piasecki PhyA(06) [quasi-additivity]; Dukkipati et al PhyA(07) [measure-theoretic aspects]; Lukes-Gerakopoulos et al PhyA(08) [and weak chaos]; Wilc & Wlodarczyk PhyA(08) [interpretation]; > s.a. generalized statistical mechanics; uncertainty relations.
@ Related topics: Lubkin IJTP(87) [entropy of measurement]; Addison & Gray JPA(01) [extensivity]; Pérez-Madrid PhyA(04) [Gibbs entropy and irreversibility]; Edwards JSP(04) [granular or glassy systems]; Kaniadakis et al PhyA(04) [from deformed log's]; Souza & Tsallis PhyA(04) [concavity and stability]; Hansen JSP(07), Seiringer LMP(07)-a0704 [Wigner-Yanase entropy not subadditive].

Systems > s.a. black hole entropy; cell complex; gravitational thermodynamics; horizons; tilings.
@ In cosmology: Coule IJMPD(03) [brane cosmology]; Hosoya et al PRL(04)gq [information entropy]; Frampton et al a0801 [relative contributions]; > s.a. gr cosmology and string cosmology.
@ For matrices: Gadiyar et al JPA(03)mp/02 [orthogonal and Hadamard matrices].

References > s.a. quantum entropy.
@ II: Han et al cm/97, AJP(99) [tracing over degrees of freedom]; Styer AJP(00) [examples and meaning]; Lieb & Yngvason PT(00)apr [without thermodynamics].
@ General: Jaynes AJP(65) [Gibbs vs Boltzmann]; Wehrl RMP(78); Martin & England 81; Denbigh & Denbigh 85; in Sorkin PRL(86); Mackey RMP(89); Leff & Rex ed-90; Lavenda et al NCB(95); Dugdale 96 [II]; Leff AJP(96) [and energy sharing]; Levy m.GM/06 [mathematical interpretation]; Kozlov & Treshchev TMP(07) [coarse-grained]; Leff FP(07) [language and interpretation]; Scully & Scully 07 [Maxwell's demon, information, and the quantum].
@ Properties: Kay & Kay JPA(01)mp/98 [strong subadditivity and volume]; Touchette AJP(08) [(non)-concavity].
@ Subsystems: Hotta & Joichi PLA(99)cm [additivity]; Baumgartner JPA(02)mp/01.
@ Related topics: Hotke-Page & Page in(92) [and clock time]; Hall PRA(99)qp/98 [geometric, ensemble volume]; Narnhofer & Thirring Fiz(87) [relative entropy]; Leff AJP(99) [dimensionless]; Dover PhyA(04) [and scaling laws]; Mäkelä gq/06-GRF [and area law]; > s.a. Cellular Automaton, dissipation, Homogeneity.

Main page – Abbreviations – Journals – Comments – Other sites – Acknowledgements
Send feedback and suggestions to bombelli at olemiss.edu – Modified 8 jul 2008