Entropy

In General > s.a. chaos; entropy bounds; statistical mechanics; thermodynamics.
* Idea, in elementary thermodynamics: An extensive thermodynamical function of state which grows monotonically with E and is constant in a reversible adiabatic transformation; Defined, up to an additive constant, by dS = dQ/T.
* History: Concept introduced by Clausius in the 1860s; > s.a. history of physics.
* Idea, statistical interpretation: A measure of how spread out a classical distribution function or quantum density matrix is; Defined by S = –k tr ln , which becomes S = k ln N when corresponds to N equally probable microscopic states; The constant k is identified with Boltzmann's constant.
* And ignorance: Generally, one views entropy as a measure of our ignorance of the microscopic state of a system; This seems to make entropy a subjective thing which, for a given system, depends on how much we wanted to find out, and could be decreased if we just measured something more; This is so, but in practice it is not a real problem, because S ln N, N = number of states compatible with our observations, and the kind of measurements we could think of to decrease our subjective entropy might lower N by a factor of, say, 100; After taking the log, this becomes a very small amount to subtract from the previous entropy, and it makes no real difference; Therefore, in practice one doesn't need to introduce a definition of entropy different from the usual one (from R Sorkin, meeting 1.02.1985).
* And classical / quantum mechanics: In classical mechanics one assumes entropies are finite, but if one takes the → 0 limit of any quantum entropy expression, this diverges; So classical thermodynamics is not the classical limit of quantum statistical mechanics, it knows about quantum mechanics.
@ Articles, II: Han et al cm/97, AJP(99)jan [tracing over degrees of freedom]; Styer AJP(00)dec [examples and meaning]; Lieb & Yngvason PT(00)apr [without thermodynamics]; Styer AJP(08)oct [and evolution].
@ General references: Jaynes AJP(65)may [Gibbs vs Boltzmann]; Wehrl RMP(78); Martin & England 81; Denbigh & Denbigh 85; in Sorkin PRL(86); Mackey RMP(89); Leff & Rex ed-90; Lavenda et al NCB(95); Dugdale 96 [II]; Leff AJP(96)oct [and energy sharing]; Levy m.GM/06 [mathematical interpretation]; Kozlov & Treshchev TMP(07) [coarse-grained]; Leff FP(07) [language and interpretation]; Scully & Scully 07 [Maxwell's demon, information, and the quantum]; Topsoe a0807 [entropy and truth vs belief]; Grandy 08 [and probability].

Information Theoretical (Shannon) Entropy > s.a. Brudno's Theorem; information; H Theorem; Landauer's Principle.
$ Def: The Shannon uncertainty; For a mixed state = _n p_n |n n|, where {|n} is a complete set of states,

S = –k _n p_n ln p_n ;

It is equivalent to the Boltzmann-Gibbs definition of entropy under equilibrium conditions.
* Remark: Actually, we could use the log with any base, or any other convex function f(p_n/q_n), with q_n the equilibrium probabilities; For any such f the entropy would increase; Other properties like additivity put constraints on f.
* Remark: Then one bit of information corresponds to an entropy ln 2.
@ General references: Jaynes PR(57), PR(57) [maximum entropy principle]; Fahn FP(96) [and thermodynamics]; Fa JPA(98) [generalization]; Luo JPA(00) [proof of Wehrl's conjecture]; Chakrabarti & Chakrabarty IJMMS(05)qp [axiomatic]; Maroney qp/07 [Gibbs-Von Neumann, motivation]; Ladyman et al SHPMP(08).
@ Boltzmann entropy: Swendsen AJP(06)mar [for colloids, clarification]; Kalogeropoulos MPLB(08)-a0804 [variation wrt energy].
@ And correlations: Van Drie mp/00; Gu et al JPA(08)qp/06.
@ Wehrl information entropy: Miranowicz et al JPA(01)qp; Piatek & Leonski JPA(01) [entanglement and correlations].

Kolmogorov-Sinai Entropy
* Idea: The growth rate h of the phase-space volume of a phase drop with time; By Liouville's theorem, for a Hamiltonian system we have h = 0 if there is no coarse-graining; If V is a coarse-grained phase-space volume, we estimate V(t) = V₀ e^ht, and define

h:= lim_{V_0 to 0} lim_{t to infty} (1/t) ln V(t) .

* Remark: Notice that h is not actually an entropy (confusing name!), but the time derivative of the entropy S ln V; Related to the stability or instability (and random behavior) of the system.
* And Lyapunov exponents: Related by h = _i=1ⁿ _i [@ Pesin UMN 77].
@ References: see Klimek & Lesniewski AP(96) [non-commutative Connes-Størmer entropy]; Frigg BJPS(04) [and chaos].

Other Entropies > s.a. modified thermodynamics [relativistic]; non-equilibrium thermodynamics; non-extensive statistics [Tsallis entropy].
* Rényi entropy: S_q^R = (ln _i p_i^q)/(1–q); Used as a measure of the entanglement of a system with its environment.
* Metric entropy of families of metric spaces: The asymptotic behavior of covering numbers.
@ Classical mechanics: Brun & Hartle PRE(99)qp/98 [histories]; McLachlan & Ryland JMP(03)mp/02 [algebraic].
@ Dynamical entropy: Connes et al CMP(87)*; Hudetz LMP(88); Benatti et al JPA(04) [and discrete chaos].
@ Algebraic entropy: Bellon & Viallet CMP(99) [discrete-t systems]; Viallet IJGMP(08).
@ Localization entropy: Schroer ht/01 [and area law].
@ Rényi entropy: Harremoës PhyA(06) [operational]; Romera & Nagy PLA(08) [for atoms].
@ Related topics: Lubkin IJTP(87) [entropy of measurement]; Addison & Gray JPA(01) [extensivity]; Pérez-Madrid PhyA(04) [Gibbs entropy and irreversibility]; Edwards JSP(04) [granular or glassy systems]; Kaniadakis et al PhyA(04) [from deformed log's]; Souza & Tsallis PhyA(04) [concavity and stability]; Hansen JSP(07), Seiringer LMP(07)-a0704 [Wigner-Yanase entropy not subadditive].

Other References > s.a. quantum entropy.
@ And area law: Mäkelä gq/06-GRF; Masanes a0907 [for low-energy states].
@ Other properties: Kay & Kay JPA(01)mp/98 [strong subadditivity and volume]; Touchette AJP(08)jan [(non)-concavity].
@ Subsystems: Hotta & Joichi PLA(99)cm [additivity]; Baumgartner JPA(02)mp/01.
@ Related topics: Hotke-Page & Page in(92) [and clock time]; Hall PRA(99)qp/98 [geometric, ensemble volume]; Narnhofer & Thirring Fiz(87) [relative entropy]; Leff AJP(99)dec [dimensionless]; Dover PhyA(04) [and scaling laws]; Parrondo et al NJP(09) [entropy production and arrow of time]; > s.a. Cellular Automaton, dissipation, Homogeneity.
> Systems: see black-hole entropy; cell complex; gravitational thermodynamics; horizons; networks; thermodynamical systems; tilings.

main page – abbreviations – journals – comments – other sites – acknowledgements
send feedback and suggestions to bombelli at olemiss.edu – modified 3 nov 2009