Exponential error bounds achievable by universal coding and decoding are derived for frame-asynchronous discrete memoryless multiple access channels with two senders, via the method of subtypes, a refinement of the method of types. An empirical entropy decoder is employed. A key tool is an improved packing lemma, that overcomes the technical difficulty caused by codeword repetitions via an induction based new argument. The asymptotic form of the bounds admits numerical evaluation. This demonstrates that error exponents achievable by synchronous transmission can be superseded via controlled asynchronism, i.e. a deliberate shift of the codewords.

Minimization of the expectation E-P(X) of a random variable X over a family Gamma of plausible prior distributions P is addressed, when Gamma is a level set of some convex integral functional. As typical cases, Gamma may be an I-divergence ball or some other f-divergence ball or Bregman distance ball. Regarding localization of the infimum, we show that whether or not the minimum of E-P(X) subject to P is an element of Gamma is attained, the densities of the almost minimizing distributions cluster around an explicitly specified function that may have integral less than 1 if the minimum is not attained. If Gamma is an f -divergence ball of radius k, the minimum is either attained for any choice of k or it is not attained when k is less/larger than a critical value. A conjecture is formulated about extending this result beyond f -divergence balls

Communication over a discrete memoryless channel is addressed when codewords are transmitted in certain time intervals of arbitrary locations, at other times the channel outputs pure noise. The receiver has to locate and decode the codewords. Exponential error bounds are derived, jointly achievable via a semi-universal or universal decoder. Implications are discussed for the familiar model of communication under strong asynchronism when in exponentially long time only one codeword is transmitted.

Familiar approaches to risk and preferences involve minimizing the expectation E_IP(X) of a payoff function X over a family of plausible risk factor distributions IP . We consider determined by a bound on a convex integral functional of the density of IP , thus may be an I-divergence (relative entropy) ball or some other f-divergence ball or Bregman distance ball around a default distribution IP_0 . Using a Pythagorean identity we show that whether or not a worst case distribution exists (minimizing E_IP(X) subject to IP∈ ), the almost worst case distributions cluster around an explicitly specified, perhaps incomplete distribution. When is an f-divergence ball, a worst case distribution either exists for any radius, or it does/does not exist for radius less/larger than a critical value. It remains open how far the latter result extends beyond f-divergence balls.

Integral functionals based on convex normal integrands are minimized over convex constraint sets. Generalized minimizers exist under a boundedness condition. Sequences of the minimization problems are studied when the constraint sets are nested. The corresponding sequences of generalized minimizers are related to the minimization over limit convex sets. Martingale theorems and moment problems are discussed.

We propose to interpret distribution model risk as sensitivity of expected loss to changes in the risk factor distribution, and to measure the distribution model risk of a portfolio by the maximum expected loss over a set of plausible distributions defined in terms of some divergence from an estimated distribution. The divergence may be relative entropy or another f ‐divergence or Bregman distance. We use the theory of minimizing convex integral functionals under moment constraints to give formulae for the calculation of distribution model risk and to explicitly determine the worst case distribution from the set of plausible distributions. We also evaluate related risk measures describing divergence preferences.

Csiszr and Krner's book is widely regarded as a classic in the field of information theory, providing deep insights and expert treatment of the key theoretical issues. It includes in-depth coverage of the mathematics of reliable information transmission, both in two-terminal and multi-terminal network scenarios. Updated and considerably expanded, this new edition presents unique discussions of information theoretic secrecy and of zero-error information theory, including the deep connections of the latter with extremal combinatorics. The presentations of all core subjects are self contained, even the advanced topics, which helps readers to understand the important connections between seemingly different problems. Finally, 320 end-of-chapter problems, together with helpful solving hints, allow readers to develop a full command of the mathematical techniques. It is an ideal resource for graduate students and researchers in electrical and electronic engineering, computer science and applied mathematics.

The mathematical problem addressed is minimising the expectation of a random variable over a set of feasible distributions P ϵ Γ, given as a level set of a convex integral functional. As special cases, Γ may be an f-divergence or f-divergence ball or a Bregman ball around a default distribution. Our approach is motivated by geometric intuition and relies upon the theory of minimising convex integral functionals subject to moment constraints. One main result is that all “almost minimisers” P ϵ Γ belong to a small Bregman ball around a specified distribution or defective distribution P, equal to the strict minimiser if that exists but well defined also otherwise.

We propose to interpret distribution model risk as sensitivity of expected loss to changes in the risk factor distribution, and to measure the distribution model risk of a portfolio by the maximum expected loss over a set of plausible distributions defined in terms of some divergence from an estimated distribution. The divergence may be relative entropy, a Bregman distance, or an $f$-divergence. We give formulas for the calculation of distribution model risk and explicitly determine the worst case distribution from the set of plausible distributions. We also give formulas for the evaluation of divergence preferences describing ambiguity averse decision makers.

Shannon theoretic secret key generation by several parties is considered for models in which a secure noisy channel with multiple input and output terminals and a public noiseless channel of unlimited capacity are available for accomplishing this goal. The secret key is generated for a set A of terminals of the noisy channel, with the remaining terminals (if any) cooperating in this task through their public communication. Single-letter lower and upper bounds for secrecy capacities are obtained when secrecy is required from an eavesdropper that observes only the public communication and perhaps also a set of terminals disjoint from A. These bounds coincide in special cases, and the lower bounds are not tight in general. We also consider models in which different sets of terminals share multiple keys, one for terminals in each set with secrecy required from the eavesdropper as well as the remaining terminals in the other sets. Partial results include showing links among the associated secrecy capacity region for multiple keys, the transmission capacity region of the multiple access channel defined by the secure noisy channel, and achievable rates for a single secret key for all the terminals.

Integral functionals based on convex normal integrands are minimized subject to finitely many moment constraints. The integrands are assumed to be strictly convex but not autonomous or differentiable. The effective domain of the value function is described by a modification of the concept of convex core. The minimization is viewed as a primal problem and studied together with a dual one in the framework of convex duality. Main results assume a dual constraint qualification but dispense with the primal constraint qualification. Minimizers and generalized minimizers are explicitly described whenever the primal value is finite. Existence of a generalized dual solution is established whenever the dual value is finite. A generalized Pythagorean identity is presented using Bregman distance and a correction term. Results are applied to minimization of Bregman distances.

Contemporary techniques of data security are primarily based on computational complexity, typically on the infeasibility of inverting certain functions using currently available mathematical techniques and computing power. Among the various protocols based on such ideas, those approved by the cryptography community appear secure enough, but mathematical and technological progress may render them insecure in the future. Indeed, past experience suggests that this is likely to happen. Information-theoretic secrecy offers provable security even against an adversary with unlimited computing power. This chapter provides a glimpse into the substantial progress that has been made towards clarifying the theoretical possibilities in this direction. Practical applications are reasonably expected within a much shorter time than capacity-achieving coding techniques have followed Shannon's discovery of the noisy channel coding theorem. Two kinds of problems will be addressed: secure transmission over insecure channels and secret key generation taking advantage of public communication. After introducing necessary concepts and tools in Section 17.1, these problems will be treated in Sections 17.2 and 17.3. Let us emphasize that the mathematical models and techniques will be similar to those in previous chapters. These models, however, are now studied from a non-cooperative aspect: a major goal is to keep (at least) one party ignorant of (at least part of) the information exchanged.

Systematic model stress tests identify worst case risk factor distributions (models) satisfying some plausibility constraint. The expected loss in the worst case model provides a scenario-based risk measure which is coherent, robust under model uncertainty, and law-invariant. As plausibility constraint for risk factor distributions (models) we take a bound on relative entropy with respect to some estimated reference distribution. We determine explicitly the maximum expected loss within this admissibility set of models, as well as the worst case model. Practical implementations of this method do not require any numerical optimisation.

Csiszár and Körner's book is widely regarded as a classic in the field of information theory, providing deep insights and expert treatment of the key theoretical issues. It includes in-depth coverage of the mathematics of reliable information transmission, both in two-terminal and multi-terminal network scenarios. Updated and considerably expanded, this new edition presents unique discussions of information theoretic secrecy and of zero-error information theory, including the deep connections of the latter with extremal combinatorics. The presentations of all core subjects are self contained, even the advanced topics, which helps readers to understand the important connections between seemingly different problems. Finally, 320 end-of-chapter problems, together with helpful solving hints, allow readers to develop a full command of the mathematical techniques. It is an ideal resource for graduate students and researchers in electrical and electronic engineering, computer science and applied mathematics.

Csiszár and Körner's book is widely regarded as a classic in the field of information theory, providing deep insights and expert treatment of the key theoretical issues. It includes in-depth coverage of the mathematics of reliable information transmission, both in two-terminal and multi-terminal network scenarios. Updated and considerably expanded, this new edition presents unique discussions of information theoretic secrecy and of zero-error information theory, including the deep connections of the latter with extremal combinatorics. The presentations of all core subjects are self contained, even the advanced topics, which helps readers to understand the important connections between seemingly different problems. Finally, 320 end-of-chapter problems, together with helpful solving hints, allow readers to develop a full command of the mathematical techniques. It is an ideal resource for graduate students and researchers in electrical and electronic engineering, computer science and applied mathematics.