Types of Entropies |

**Boltzmann Entropy**

* __Idea__: For a given macrostate, it is the statistical quantity *S* =* k*_{B} ln Ω, where *k*_{B} is the Boltzmann constant (which can be omitted to get a dimensionless entropy), and Ω the number of microstates compatible with the macrostate.

* __And ignorance__: Generally,
one views entropy as a measure of our ignorance of the microscopic state of
a system; This seems to make entropy a subjective
thing which, for a given system, depends on how much we wanted
to find out, and could be decreased if we just measured something more; This is so,
but in practice it is not a real problem, because *S* ~ ln Ω, and the kind of measurements
we could think of to decrease our subjective entropy might lower Ω by
a factor of, say, 100; After taking the log, this
becomes a very small amount to subtract from the previous entropy,
and it makes no real difference; Therefore, in practice one doesn't need to
introduce a definition of entropy different from the usual one (< RDS,
1.02.1985 meeting).

@ __General references__: Jaynes AJP(65)may [vs Gibbs entropy];
Kalogeropoulos MPLB(08)-a0804 [variation
with respect to energy].

@ __Examples__: Swendsen AJP(06)mar
[for colloids, clarification].

**Information Theoretical (Shannon) Entropy** > s.a. Brudno's
Theorem; information;
*H* Theorem; Landauer's Principle.

$ __Def__: The Shannon uncertainty; For a mixed state *ρ* = ∑_{n}* p*_{n} \(|n\rangle\langle n|\),
where {|*n*\(\rangle\)} is a complete set of states,

*S* = –*k*_{B} ∑_{n}* p*_{n} ln *p*_{n} ;

It is equivalent to the Boltzmann-Gibbs definition of entropy under equilibrium conditions, when
it corresponds to *N* equally probable microscopic states.

* __Remark__: Actually, we
could use the log with any base, or any other convex
function *f*(*p*_{n}/*q*_{n}),
with *q*_{n} the
equilibrium probabilities; For any such *f* the
entropy would increase; Other properties like additivity put constraints on *f*.

* __Remark__: Then one bit of
information corresponds to an entropy ln 2.

@ __General references__: Fahn FP(96) [and thermodynamics];
Fa JPA(98) [generalization];
Chakrabarti & Chakrabarty IJMMS(05)qp [axiomatic];
Maroney qp/07 [Gibbs-Von
Neumann, motivation]; Ladyman et al SHPMP(08); Wilde in(13)-a1106 [quantum Shannon theory]; Baccetti & Visser JSM(13)-a1212 [states with infinite entropy];
Weilenmann et al PRL(16)-a1501 [and thermodynamic entropy];
Anza & Vedral SRep(17)-a1509 [thermodynamical meaning, and thermal equilibrium].

@ __And correlations__: Van Drie mp/00;
Gu et al JPA(08)qp/06.

@ __Wehrl information entropy__: Miranowicz et al JPA(01)qp;
Piatek & Leonski
JPA(01)
[entanglement and correlations].

> __Online resources__: see Wikipedia page.

**Rényi Entropy** > s.a. statistical mechanical systems;
XY Chain.

* __Idea__: *S*_{q}^{R} = *k*_{B} (ln ∑_{i} *p*_{i}^{q})/(1–*q*);
Used as a measure of the entanglement of a system with its environment.

@ __References__:
Harremoës PhyA(06)
[operational]; Parvan & Biró PLA(10)
[Rényi statistics in canonical and microcanonical
ensembles]; Oikonomou PhyA(11) [multinomial coefficients method]; Baez a1102 [as free energy of Gibbs state]; Adesso et al PRL(12)-a1203 [and quantum information for Gaussian states]; Linden et al PRS(13) [Rényi entropic inequalities]; Müller-Lennert et al JMP(13)-a1306 [generalization]; Mosonyi & Ogawa CMP(15)-a1309 [operational interpretation]; Nishioka JHEP(14), Hama et al JHEP(14)-a1410 [supersymmetric]; Dong a1601 [quantum field theory, area law]; Bebiano et al a1706 [as basis for quantum thermodynamics].

@ __Rényi mutual information__: Schnitzer a1406 [for widely separated identical compound systems]; Hayashi & Tomamichel JMP(16)-a1408 [operational interpretation].

@ __For specific systems__: Romera & Nagy PLA(08)
[atoms]; Swingle et al PRB(13)-a1211 [Fermi gases and liquids]; Lee et al JHEP(15)-a1407, Lashkari PRL(14)-a1404 [conformal field theory]; Dowker a1512 [free scalar fields, charged Rényi entropies]; > s.a. dirac quantum field theory.

@ __Vs von Neumann entropy__: Fannes & Van Ryn JPA(12)-a1205 [for fermions]; Fannes a1310 [monotonicity of the von Neumann entropy].

**Other Entropies** > s.a. Kolmogorov-Sinai Entropy; non-extensive statistics [Tsallis entropy]; quantum
entropy and entanglement entropy [or geometric].

* __Relative entropy__: The relative entropy of a probability distribution *p* with respect to another probability distribution *q*; For discrete distributions, one definition is the Kullback-Leibler distance, *d*(*p*, *q*):= ∑_{i} *p*_{i} log_{2}(*p*_{i}/*q*_{i}); > s.a. MathWorld page; Wikipedia page.

* __Metric entropy of families
of metric spaces__: The asymptotic behavior
of covering numbers.

* __Shore-Johnson axioms__: Consistency conditions ensuring that probability distributions inferred from limited data by maximizing the entropy satisfy the multiplication rule of probability for independent events; They are satisfied by the Boltzmann-Gibbs form of the entropy, but not by non-additive entropies.

@ __Proposals__: Lubkin IJTP(87)
[entropy of measurement]; Kaniadakis et al PhyA(04)
[from deformed log's]; Petz a1009 [quasi-entropy]; Polkovnikov AP(11) [diagonal entropy]; Rastegin JSP(11)-a1106 [unified entropies]; Biró PhyA(13) [deformed entropy formulas for systems with finite reservoirs]; Kalogeropoulos a1705 [\(\delta\)-entropy].

@ __Classical mechanics__: Brun & Hartle PRE(99)qp/98 [histories];
McLachlan & Ryland JMP(03)mp/02
[algebraic].

@ __Relative entropy__: Narnhofer & Thirring Fiz(87); Baez & Fritz TAC-a1402 [Bayesian characterization]; Anshu et al a1404 [operational interpretation]; Lashkari PRL(14)-a1404 [in conformal field theory]; Rajagopal et al PRA(14) [Tsallis, and conditional entropy]; Leditzky a1611-PhD [and quantum information theory]; > s.a. entropy bound; entropy in quantum theory.

@ __Dynamical entropy__: Connes et al CMP(87)*;
Hudetz LMP(88);
Hudetz JMP(94)
[topological entropy]; Benatti et al JPA(04)
[and discrete chaos]; Segre a1004 [and deformation quantization].

@ __Algebraic entropy__: Bellon & Viallet CMP(99)
[discrete-time systems]; Viallet IJGMP(08).

@ __Localization entropy__: Schroer ht/01 [and area law].

@ __Related topics__: Addison & Gray JPA(01)
[extensivity]; Pérez-Madrid
PhyA(04)
[Gibbs entropy and irreversibility]; Edwards JSP(04)
[granular or glassy systems];
Souza & Tsallis
PhyA(04)
[concavity and stability]; Kay Ent(15)-a1504 [matter-gravity entanglement hypothesis and entropy of a closed system as a real and objective quantity]; > s.a. Coarse Graining; modified
thermodynamics [relativistic].

main page – abbreviations – journals – comments – other
sites – acknowledgements

send feedback and suggestions to bombelli at olemiss.edu – modified 23
jun
2017