*M. Vidyasagar*

- Published in print:
- 2014
- Published Online:
- October 2017
- ISBN:
- 9780691133157
- eISBN:
- 9781400850518
- Item type:
- chapter

- Publisher:
- Princeton University Press
- DOI:
- 10.23943/princeton/9780691133157.003.0002
- Subject:
- Mathematics, Probability / Statistics

This chapter provides an introduction to some elementary aspects of information theory, including entropy in its various forms. Entropy refers to the level of uncertainty associated with a random ...
More

This chapter provides an introduction to some elementary aspects of information theory, including entropy in its various forms. Entropy refers to the level of uncertainty associated with a random variable (or more precisely, the probability distribution of the random variable). When there are two or more random variables, it is worthwhile to study the conditional entropy of one random variable with respect to another. The last concept is relative entropy, also known as the Kullback–Leibler divergence, which measures the “disparity” between two probability distributions. The chapter first considers convex and concave functions before discussing the properties of the entropy function, conditional entropy, uniqueness of the entropy function, and the Kullback–Leibler divergence.Less

This chapter provides an introduction to some elementary aspects of information theory, including entropy in its various forms. Entropy refers to the level of uncertainty associated with a random variable (or more precisely, the probability distribution of the random variable). When there are two or more random variables, it is worthwhile to study the conditional entropy of one random variable with respect to another. The last concept is relative entropy, also known as the Kullback–Leibler divergence, which measures the “disparity” between two probability distributions. The chapter first considers convex and concave functions before discussing the properties of the entropy function, conditional entropy, uniqueness of the entropy function, and the Kullback–Leibler divergence.

*Joram Lindenstrauss, David Preiss, and Tiˇser Jaroslav*

- Published in print:
- 2012
- Published Online:
- October 2017
- ISBN:
- 9780691153551
- eISBN:
- 9781400842698
- Item type:
- chapter

- Publisher:
- Princeton University Press
- DOI:
- 10.23943/princeton/9780691153551.003.0003
- Subject:
- Mathematics, Analysis

This chapter shows how spaces with separable dual admit a Fréchet smooth norm. It first considers a criterion of the differentiability of continuous convex functions on Banach spaces before ...
More

This chapter shows how spaces with separable dual admit a Fréchet smooth norm. It first considers a criterion of the differentiability of continuous convex functions on Banach spaces before discussing Fréchet smooth and nonsmooth renormings and Fréchet differentiability of convex functions. It then describes the connection between porous sets and Fréchet differentiability, along with the set of points of Fréchet differentiability of maps between Banach spaces. It also examines the concept of separable determination, the relevance of the σ-porous sets for differentiability and proves the existence of a Fréchet smooth equivalent norm on a Banach space with separable dual. The chapter concludes by explaining how one can show that many differentiability type results hold in nonseparable spaces provided they hold in separable ones.Less

This chapter shows how spaces with separable dual admit a Fréchet smooth norm. It first considers a criterion of the differentiability of continuous convex functions on Banach spaces before discussing Fréchet smooth and nonsmooth renormings and Fréchet differentiability of convex functions. It then describes the connection between porous sets and Fréchet differentiability, along with the set of points of Fréchet differentiability of maps between Banach spaces. It also examines the concept of separable determination, the relevance of the σ-porous sets for differentiability and proves the existence of a Fréchet smooth equivalent norm on a Banach space with separable dual. The chapter concludes by explaining how one can show that many differentiability type results hold in nonseparable spaces provided they hold in separable ones.

*Andrew M. Steane*

- Published in print:
- 2016
- Published Online:
- January 2017
- ISBN:
- 9780198788560
- eISBN:
- 9780191830426
- Item type:
- chapter

- Publisher:
- Oxford University Press
- DOI:
- 10.1093/acprof:oso/9780198788560.003.0017
- Subject:
- Physics, Theoretical, Computational, and Statistical Physics

The treatment of out of equilibrium systems is introduced, and the conditions for stable equilibrium derived. The minimum energy principle is derived. The convex shape of the entropy function is ...
More

The treatment of out of equilibrium systems is introduced, and the conditions for stable equilibrium derived. The minimum energy principle is derived. The convex shape of the entropy function is explained, and Le Chatelier’s principle is given. The relation between entropy and phase change is explained. Free energy is considered afresh, as a suitable way of treating equilibrium conditions under various constraints. Availability is defined and used. The general concept of a system finding its equilibrium via the adjustment of internal parameters is described.Less

The treatment of out of equilibrium systems is introduced, and the conditions for stable equilibrium derived. The minimum energy principle is derived. The convex shape of the entropy function is explained, and Le Chatelier’s principle is given. The relation between entropy and phase change is explained. Free energy is considered afresh, as a suitable way of treating equilibrium conditions under various constraints. Availability is defined and used. The general concept of a system finding its equilibrium via the adjustment of internal parameters is described.