Types of Entropies  

Boltzmann Entropy
* Idea: For a given macrostate, it is the statistical quantity S = kB ln Ω, where kB is the Boltzmann constant (it can be omitted to get a dimensionless entropy), and Ω the number of microstates compatible with the macrostate.
* And ignorance: Generally, one views entropy as a measure of our ignorance of the microscopic state of a system; This seems to make entropy a subjective thing which, for a given system, depends on how much we wanted to find out, and could be decreased if we just measured something more; This is so, but in practice it is not a real problem, because S ~ ln Ω, and the kind of measurements we could think of to decrease our subjective entropy might lower Ω by a factor of, say, 100; After taking the log, this becomes a very small amount to subtract from the previous entropy, and it makes no real difference; Therefore, in practice one doesn't need to introduce a definition of entropy different from the usual one (< RDS, 1.02.1985 meeting).
@ General references: Jaynes AJP(65)may [vs Gibbs entropy]; Kalogeropoulos MPLB(08)-a0804 [variation with respect to energy].
@ Examples: Swendsen AJP(06)mar [colloids, clarification]; Yoshida PRA(20)-a1909 [quantum field systems].

Information Theoretical (Shannon) Entropy > s.a. Brudno's Theorem; hamiltonian dynamics; information; H Theorem; Landauer's Principle.
$ Def: The Shannon uncertainty; For a mixed state ρ = ∑n pn \(|n\rangle\langle n|\), where {|n\(\rangle\)} is a complete set of states,

S = −kB ∑n pn ln pn ;

It is equivalent to the Boltzmann-Gibbs definition of entropy under equilibrium conditions, when it corresponds to N equally probable microscopic states.
* Remark: Actually, we could use the log with any base, or any other convex function f(pn/qn), with qn the equilibrium probabilities; For any such f the entropy would increase; Other properties like additivity put constraints on f.
* Remark: Then one bit of information corresponds to an entropy ln 2.
@ General references: Fahn FP(96) [and thermodynamics]; Fa JPA(98) [generalization]; Chakrabarti & Chakrabarty IJMMS(05)qp [axiomatic]; Maroney qp/07 [Gibbs-Von Neumann, motivation]; Ladyman et al SHPMP(08); Wilde in(13)-a1106 [quantum Shannon theory]; Baccetti & Visser JSM(13)-a1212 [states with infinite entropy]; Weilenmann et al PRL(16)-a1501 [and thermodynamic entropy]; Anza & Vedral SRep(17)-a1509 [thermodynamical meaning, and thermal equilibrium]; Carcassi et al EJP-a1912 [characterization].
@ And correlations: Van Drie mp/00; Gu et al JPA(08)qp/06.
@ Wehrl information entropy: Miranowicz et al JPA(01)qp; Piatek & Leonski JPA(01) [entanglement and correlations].
> Online resources: see Wikipedia page.

Rényi Entropy > s.a. entanglement entropy; statistical mechanical systems; XY Chain.
* Idea: For a mixed quantum state described by a density matrix ρ, it is defined as SnR = kB (ln ∑i pin) / (1−n), where the pi are the eigenstates of ρ (probabilities), n is an integer, and it is used as a measure of the entanglement of the system with its environment.
* Special cases: For n = 1 it gives the von Neumann entropy, and for n = 2 the mixedness of the system.
@ References: Harremoës PhyA(06) [operational]; Parvan & Biró PLA(10) [Rényi statistics in canonical and microcanonical ensembles]; Oikonomou PhyA(11) [multinomial coefficients method]; Baez a1102 [as free energy of Gibbs state]; Adesso et al PRL(12)-a1203 [and quantum information for Gaussian states]; Linden et al PRS(13) [Rényi entropic inequalities]; Mosonyi & Ogawa CMP(15)-a1309 [operational interpretation]; Dong nComm(16)-a1601 [quantum field theory, area law]; Bebiano et al a1706 [as basis for quantum thermodynamics]; Cresswell a1709 [entanglement timescale]; Bellomo et al SRep-a1710 [operational interpretation]; Kendall & Kempf a2004 [and interaction between systems].
@ Rényi mutual information: Schnitzer a1406 [for widely separated identical compound systems]; Hayashi & Tomamichel JMP(16)-a1408 [operational interpretation].
@ For specific systems: Romera & Nagy PLA(08) [atoms]; Swingle et al PRB(13)-a1211 [Fermi gases and liquids]; Lee et al JHEP(15)-a1407, Lashkari PRL(14)-a1404 [conformal field theory]; Dowker a1512 [free scalar fields, charged Rényi entropies]; Sugino & Korepin IJMPB(18)-a1806 [highly-entangled spin chains]; Olendski EJP(19)-a1811 [and Tsallis entropy, examples]; > s.a. dirac quantum field theory.
@ Vs von Neumann entropy: Fannes & Van Ryn JPA(12)-a1205 [for fermions]; Fannes a1310 [monotonicity of the von Neumann entropy].
@ Generalizations: Müller-Lennert et al JMP(13)-a1306; Nishioka JHEP(14), Hama et al JHEP(14)-a1410 [supersymmetric]; Johnson a1807; Mukhamedov QIP(19)-a1905 [on C*-algebras].

Other Entropies > s.a. Kolmogorov-Sinai Entropy; non-extensive statistics [Tsallis entropy]; quantum entropy and entanglement entropy [or geometric].
* Relative entropy: The relative entropy of a probability distribution p with respect to another probability distribution q; For discrete distributions, one definition is the Kullback-Leibler distance, \(d(p,q)\):= ∑i \(p_i \log_2(p_i/q_i)\); > s.a. MathWorld page; Wikipedia page.
* Metric entropy of families of metric spaces: The asymptotic behavior of covering numbers.
* Shore-Johnson axioms: Consistency conditions ensuring that probability distributions inferred from limited data by maximizing the entropy satisfy the multiplication rule of probability for independent events; They are satisfied by the Boltzmann-Gibbs form of the entropy, but not by non-additive entropies.
@ Proposals: Lubkin IJTP(87) [entropy of measurement]; Kaniadakis et al PhyA(04) [from deformed log's]; Petz a1009 [quasi-entropy]; Polkovnikov AP(11) [diagonal entropy]; Rastegin JSP(11)-a1106 [unified entropies]; Biró PhyA(13) [deformed entropy formulas for systems with finite reservoirs]; Kalogeropoulos AMP-a1705 [\(\delta\)-entropy]; Portesi et al EPJst(18)-a1802 [generalized (h, φ)-entropies].
@ Classical mechanics: Brun & Hartle PRE(99)qp/98 [histories]; McLachlan & Ryland JMP(03)mp/02 [algebraic].
@ Relative entropy: Narnhofer & Thirring Fiz(87); Baez & Fritz TAC-a1402 [Bayesian characterization]; Anshu et al IEEE(16)-a1404 [operational interpretation]; Lashkari PRL(14)-a1404 [in conformal field theory]; Rajagopal et al PRA(14) [Tsallis, and conditional entropy]; Leditzky a1611-PhD [and quantum information theory]; Kalogeropoulos PhyA-a1905 [relative q-entropy]; Dutta & Guha a1908 [Tsallis, modification for classical information theory]; > s.a. entropy bound; entropy in quantum theory; modified thermodynamics; non-equilibrium systems.
@ Dynamical entropy: Connes et al CMP(87)*; Hudetz LMP(88); Hudetz JMP(94) [topological entropy]; Benatti et al JPA(04) [and discrete chaos]; Segre a1004 [and deformation quantization].
@ Algebraic entropy: Bellon & Viallet CMP(99) [discrete-time systems]; Viallet IJGMP(08).
@ Localization entropy: Schroer ht/01 [and area law].
@ Related topics: Addison & Gray JPA(01) [extensivity]; Pérez-Madrid PhyA(04) [Gibbs entropy and irreversibility]; Edwards JSP(04) [granular or glassy systems]; Souza & Tsallis PhyA(04) [concavity and stability]; Malik & Lopez-Mobilia a2004 [based on the level of irreversibility of a process]; Mollabashi et al a2011 [pseudo-entropy]; > s.a. Coarse Graining; modified thermodynamics [relativistic]; Topos Theory.

main page – abbreviations – journals – comments – other sites – acknowledgements
send feedback and suggestions to bombelli at olemiss.edu – modified 8 feb 2021