Search Results

This chapter introduces the basic mathematical formalism and discusses concepts used throughout the book. Among other things, it looks at decision problems vs optimization problems, Random Access Machines, big Oh notation, strings and graphs. It concludes by looking at the basics from computational complexity theory.

A decision-making problem is often formulated as the minimization or maximization of a multivariable function, an optimization problem. This chapter shows that the methods of statistical mechanics are useful to study some types of optimization problems including the number partitioning, the graph partitioning, the knapsack problem, and the satisfiability problem. All these problems are shown to be formulated and solved using the theory of spin glasses, in particular the replica method. Then, discussions are continued on the mathematical properties of simulated annealing, an approximate numerical method for generic optimization problems.

This chapter examines Samuelson's Le Chatelier principle of how the market responds to a change in parameters of demand and supply curves. It uses examples in demand theory, economic policy, and empirical research to illustrate the principle. It also stresses the flexibility of the principle to adapt to changing assumptions. Changes from the optimizing agents to equilibrium systems whose primary use is to provide a foundation for understanding multipliers are observed. The chapter evaluates the performance of the principle performed when confronted with local optimization problems as in production function settings, and in positive feedbacks systems as in gaming situations. The principle is found progressive in that it is able to capitalize on symmetric relations among substitutes and complements. In that regard, the principle has extended research into multiplier analysis, a research area that will continue into the 21st century.

This chapter discusses the transductive learning setting proposed by Vapnik where predictions are made only at a fixed number of known test points. Transductive support vector machines (TSVMs) implement the idea of transductive learning by including test points in the computation of the margin. This chapter provides some examples for why the margin on the test examples can provide useful prior information for learning, in particular for the problem of text classification. The resulting optimization problems, however, are difficult to solve. The chapter reviews exact and approximate optimization methods and discusses their properties. Finally, the chapter discusses connections to other related semi-supervised learning approaches such as co-training and methods based on graph cuts, which can be seen as solving variants of the TSVM optimization problem.

This chapter discusses an alternative approach that is based on a convex relaxation of the optimization problem associated with support vector machine transduction. The result is a semi-definite programming (SDP) problem which can be optimized in polynomial time, the solution of which is an approximation of the optimal labeling as well as a bound on the true optimum of the original transduction objective function. To further decrease the computational complexity, this chapter proposes an approximation that allows solving transduction problems of up to 1,000 unlabeled samples. Finally, the formulation is extended to more general settings of semi-supervised learning, where equivalence and inequivalence constraints are given on labels of some of the samples.

Dynamic optimization is widely used in many fields of economics, finance, and business management. Typically one searches for the optimal time path of one or several variables that maximizes the value of a specific objective function given certain constraints. While there exist some analytical solutions to deterministic dynamic optimization problems, things become much more complicated as soon as the environment in which we are searching for optimal decisions becomes uncertain. In such cases researchers typically rely on the technique of dynamic programming. This chapter introduces the principles of dynamic programming and provides a couple of solution algorithms that differ in accuracy, speed, and applicability. Chapters 8 to 11 show how to apply these dynamic programming techniques to various problems in macroeconomics and finance. To get things started we want to lay out the basic idea of dynamic programming and introduce the language that is typically used to describe it. The easiest way to do this is with a very simple example that we can solve both ‘by hand’ and with the dynamic programming technique. Let’s assume an agent owns a certain resource (say a cake or a mine) which has the size a0. In every period t = 0, 1, 2, . . . ,∞ the agent can decide how much to extract from this resource and consume, i.e. how much of the cake to eat or how many resources to extract from the mine.We denote his consumption in period t as ct. At each point in time the agent derives some utility from consumption which we express by the so-called instantaneous utility function u(ct). We furthermore assume that the agent’s utility is additively separable over time and that the agent is impatient, meaning that he derives more utility from consuming in period t than in any later period.We describe the extent of his impatience with the time discount factor 0 < β < 1.

This chapter describes an algorithm to assign weights to multiple graphs within graph-based semi-supervised learning. Both predicting class labels and searching for weights for combining multiple graphs are formulated into one convex optimization problem. The graph-combining method is applied to functional class prediction of yeast proteins. When compared with individual graphs, the combined graph with optimized weights performs significantly better than any single graph. When compared with the semi-definite programming-based support vector machine (SDP/SVM), it shows comparable accuracy in a remarkably short time. Compared with a combined graph with equal-valued weights, this method could select important graphs without loss of accuracy, which implies the desirable property of integration with selectivity.

This Chapter introduces terminal-agnostic gates, namely gates that do not distinguish between the traditional input or traditional output terminals. Memory is the physical property that allows the realization of these self-organizing gates. It also discusses the mathematical properties of these gates when assembled into circuits, and their possible hardware realization. It gives some examples of these circuits for the solution of factorization.

This chapter describes the use of time-reversal imaging techniques for optimal control of extended inclusions. It first considers the problem of reconstructing shape deformations of an extended elastic target before reconstructing the perturbations from boundary measurements of the displacement field. As for small-volume inclusions, direct imaging algorithms based on an asymptotic expansion for the perturbations in the data due to small shape deformations is introduced. The chapter also presents a concept equivalent to the polarization tensor for small-volume inclusions as well as the nonlinear optimization problem for reconstructing the shape of an extended inclusion from boundary displacement measurements. Finally, it develops iterative algorithms to address the nonlinearity of the problem.

This Chapter clarifies the meaning of memory as time non-locality. It shows that time non-locality induces spatial non-locality. It then introduces the notion of circuit elements with memory, memprocessors, and discusses the need of active elements in MemComputing. It finally highlights that MemComputing is not just ‘in-memory computing’.