Search Results

Rényi divergence is a natural way to measure the rate of information flow in contexts like Bayesian updating. This chapter shows how Monte Carlo integration can be used to measure Rényi divergence when (as is often the case) only kernels of the relevant probability densities are available. The chapter further demonstrates that Rényi divergence is central to the convergence and efficiency of Monte Carlo integration procedures in which information flow is controlled. It uses this perspective to develop more flexible approaches to the controlled introduction of information; in the limited set of examples considered here, these alternatives enhance efficiency.

The core of data science is our fundamental understanding about data intelligence processes for transforming data to decisions. One aspect of this understanding is how to analyze the cost-benefit of data intelligence workflows. This work is built on the information-theoretic metric proposed by Chen and Golan for this purpose and several recent studies and applications of the metric. We present a set of extended interpretations of the metric by relating the metric to encryption, compression, model development, perception, cognition, languages, and media.

Mathematical tools necessary to the argument are presented and discussed. The focus is on concepts borrowed from the convex analysis and variational analysis literatures. The chapter starts by introducing the notions of a correspondence, upper hemi-continuity, and lower hemi-continuity. Superdifferential and subdifferential correspondences for real-valued functions are then introduced, and their essential properties and their role in characterizing global optima are surveyed. Convex sets are introduced and related to functional concavity (convexity). The relationship between functional concavity (convexity), superdifferentiability (subdifferentiability), and the existence of (one-sided) directional derivatives is examined. The theory of convex conjugates and essential conjugate duality results are discussed. Topics treated include Berge's Maximum Theorem, cyclical monotonicity of superdifferential (subdifferential) correspondences, concave (convex) conjugates and biconjugates, Fenchel's Inequality, the Fenchel-Rockafellar Conjugate Duality Theorem, support functions, superlinear functions, sublinear functions, the theory of infimal convolutions and supremal convolutions, and Fenchel's Duality Theorem.

Three generic economic optimization problems (expenditure (cost) minimization, revenue maximization, and profit maximization) are studied using the mathematical tools developed in Chapters 2 and 3. Conjugate duality results are developed for each. The resulting dual representations (E(q;y), R(p,x), and π‎(p,q)) are shown to characterize all of the economically relevant information in, respectively, V(y), Y(x), and Gr(≽(y)). The implications of different restrictions on ≽(y) for the dual representations are examined.

A computational theory of meaning tries to understand the phenomenon of meaning in terms of computation. Here we give an analysis in the context of Kolmogorov complexity. This theory measures the complexity of a data set in terms of the length of the smallest program that generates the data set on a universal computer. As a natural extension, the set of all programs that produce a data set on a computer can be interpreted as the set of meanings of the data set. We give an analysis of the Kolmogorov structure function and some other attempts to formulate a mathematical theory of meaning in terms of two-part optimal model selection. We show that such theories will always be context dependent: the invariance conditions that make Kolmogorov complexity a valid theory of measurement fail for this more general notion of meaning. One cause is the notion of polysemy: one data set (i.e., a string of symbols) can have different programs with no mutual information that compresses it. Another cause is the existence of recursive bijections between ℕ and ℕ2 for which the two-part code is always more efficient. This generates vacuous optimal two-part codes. We introduce a formal framework to study such contexts in the form of a theory that generalizes the concept of Turing machines to learning agents that have a memory and have access to each other’s functions in terms of a possible world semantics. In such a framework, the notions of randomness and informativeness become agent dependent. We show that such a rich framework explains many of the anomalies of the correct theory of algorithmic complexity. It also provides perspectives for, among other things, the study of cognitive and social processes. Finally, we sketch some application paradigms of the theory.

This chapter addresses the impact of product innovation on economic welfare, initially defined as the sum of consumer and producer surpluses. In a static framework, it is shown how product innovation can increase welfare via additions to consumer surplus and increased firm profits; and an estimate that the value of the increase for a typical product innovation might equal 2.5 per cent of the innovator’s revenue is reported from the literature. Problems with measuring welfare by the sum of consumer and producer surplus are raised, especially because of changes in the producers’ incentives to innovate. In an intertemporal framework, it is further shown that the optimal diffusion path could arise under either monopoly supply or competitive supply, depending on buyers’ price expectations formation processes. It is also argued that variety itself may generate welfare, and whether free markets would generate optimal variety is discussed. The literature suggests not.