Search Results

Generative Probabilistic Modeling: Understanding Causal Sensorimotor Integration

Sethu Vijayakumar, Timothy Hospedales, and Adrian Haith

in Sensory Cue Integration

This chapter argues that many aspects of human perception are best explained by adopting a modeling approach in which experimental subjects are assumed to possess a full generative probabilistic ... More

This chapter argues that many aspects of human perception are best explained by adopting a modeling approach in which experimental subjects are assumed to possess a full generative probabilistic model of the task they are faced with, and that they use this model to make inferences about their environment and act optimally given the information available to them. It applies this generative modeling framework in two diverse settings—concurrent sensory and motor adaptation, and multisensory oddity detection—and shows, in both cases, that the data are best described by a full generative modeling approach.Less

The Framework

Timothy J. O’Donnell and Noah D. Goodman

in Productivity and Reuse In Language: A Theory of Linguistic Computation and Storage

This chapter consists of four parts. The first section discusses the ideas behind the modeling framework adopted in the book---structured probabilistic generative models. The second section discusses ... More

This chapter consists of four parts. The first section discusses the ideas behind the modeling framework adopted in the book---structured probabilistic generative models. The second section discusses some theoretical and methodological issues in the interpretation of this approach to modeling. The third section develops the modelling framework from the perspective of probabilistic programming using the Church programming language. The Church formalization makes explicit the relationship between the model and two important technical ideas from computer science (and, specifically, the theory of programming languages). The final section of the chapter provides more discussion of the four classes of model evaluated in the book.Less

Models of network formation: A discussion of models of the formation of networks, particularly models of networks that grow by addition of vertices, such as the World Wide Web or citation networks

M. E. J. Newman

in Networks: An Introduction

Generative network models model the mechanisms by which networks are created. The idea behind models such as these is to explore hypothesized generative mechanisms to see what structures they ... More

Generative network models model the mechanisms by which networks are created. The idea behind models such as these is to explore hypothesized generative mechanisms to see what structures they produce. If the structures are similar to those of networks observed in the real world, this suggests — though does not prove — that similar generative mechanisms may be at work in the real networks. This chapter examines the best-known example of a generative network model: the ‘preferential attachment’ model for the growth of networks with power-law degree distributions. Exercises are provided at the end of the chapter.Less

Essentialism as a Generative Theory of Classification

Bob Rehder

in Causal Learning: Psychology, Philosophy, and Computation

Essentialism is the view that kinds are defined by underlying properties or characteristics (an essence) that is shared by all category members and by members of no other ... More

Essentialism is the view that kinds are defined by underlying properties or characteristics (an essence) that is shared by all category members and by members of no other categories and that are presumed to generate, or cause, perceptual features. Although unobservable, essential features can nonetheless affect classification by changing the evidence that observable features provide for category membership. This chapter proposes treating essentialized categories as a generative causal model and provides evidence for four phenomena that follow from this view: (a) classification as diagnostic reasoning; (b) classification as prospective reasoning; (c) boundary intensification; and (d) the effect of coherence on classification. The chapter also characterizes the development of conceptual knowledge in terms of an evolving set of causal models.Less

Semi-Supervised Text Classification Using EM

Nigam Kamal, McCallum Andrew, and Mitchell Tom

in Semi-Supervised Learning

This chapter explores the use of generative models for semi-supervised learning with labeled and unlabeled data in domains of text classification. The widely used naive Bayes classifier for ... More

This chapter explores the use of generative models for semi-supervised learning with labeled and unlabeled data in domains of text classification. The widely used naive Bayes classifier for supervised learning defines a mixture of multinomials mixture models. In some domains, model likelihood and classification accuracy are strongly correlated, despite the overly simplified generative model. Here, expectation-maximization finds more likely models and improved classification accuracy. In other domains, likelihood and accuracy are not well correlated with the naive Bayes model. Here, we can use a more expressive generative model that allows for multiple mixture components per class. This helps restore a moderate correlation between model likelihood and classification accuracy, and again, EM finds more accurate models. Finally, even with a well-correlated generative model, local maxima are a significant hindrance with EM. Here, the approach of deterministic annealing does provide much higher likelihood models, but often loses the correspondence with the class labels. When class label correspondence is easily corrected, high accuracy models result.Less

Top-Down Predictions Determine Perceptions

Martin V. Butz and Esther F. Kutter

in How the Mind Comes into Being: Introducing Cognitive Science from a Functional and Computational Perspective

While bottom-up visual processing is important, the brain integrates this information with top-down, generative expectations from very early on in the visual processing hierarchy. Indeed, our brain ... More

While bottom-up visual processing is important, the brain integrates this information with top-down, generative expectations from very early on in the visual processing hierarchy. Indeed, our brain should not be viewed as a classification system, but rather as a generative system, which perceives something by integrating sensory evidence with the available, learned, predictive knowledge about that thing. The involved generative models continuously produce expectations over time, across space, and from abstracted encodings to more concrete encodings. Bayesian information processing is the key to understand how information integration must work computationally – at least in approximation – also in the brain. Bayesian networks in the form of graphical models allow the modularization of information and the factorization of interactions, which can strongly improve the efficiency of generative models. The resulting generative models essentially produce state estimations in the form of probability densities, which are very well-suited to integrate multiple sources of information, including top-down and bottom-up ones. A hierarchical neural visual processing architecture illustrates this point even further. Finally, some well-known visual illusions are shown and the perceptions are explained by means of generative, information integrating, perceptual processes, which in all cases combine top-down prior knowledge and expectations about objects and environments with the available, bottom-up visual information.Less

Generative models

Thomas P. Trappenberg

in Fundamentals of Machine Learning

This chapter presents an introduction to the important topic of building generative models. These are models that are aimed to understand the variety of a class such as cars or trees. A generative ... More

This chapter presents an introduction to the important topic of building generative models. These are models that are aimed to understand the variety of a class such as cars or trees. A generative mode should be able to generate feature vectors for instances of the class they represent, and such models should therefore be able to characterize the class with all its variations. The subject is discussed both in a Bayesian and in a deep learning context, and also within a supervised and unsupervised context. This area is related to important algorithms such as k-means clustering, expectation maximization (EM), naïve Bayes, generative adversarial networks (GANs), and variational autoencoders (VAE).Less

Formalization of the Models and Inference

Timothy J. O’Donnell and Noah D. Goodman

in Productivity and Reuse In Language: A Theory of Linguistic Computation and Storage

This chapter the mathematical details of the models studied in this book. It also discusses the inference algorithms used for each of the models and various other issues of practical concern for the ... More

This chapter the mathematical details of the models studied in this book. It also discusses the inference algorithms used for each of the models and various other issues of practical concern for the simulations that we report later.Less

Learning Faster

Reza Shadmehr and Sandro Mussa-Ivaldi

in Biological Learning and Control: How the Brain Builds Representations, Predicts Events, and Makes Decisions

This chapter presents some useful ideas on how to encourage the process of learning. It illustrates that speeding up learning can occur if the learner becomes more sensitive to prediction errors. ... More

This chapter presents some useful ideas on how to encourage the process of learning. It illustrates that speeding up learning can occur if the learner becomes more sensitive to prediction errors. This chapter suggests that as the brain is presented with a prediction error, it tries to learn a generative model. It shows that the generative model’s trial-to-trial change is a forgetting rate. Presumably, biological learning is a process of state estimation, and a process in which the brain learns the structure of the generative model.Less

Probabilistic Semi-Supervised Clustering with Constraints

Basu Sugato, Bilenko Mikhail, Banerjee Arindam, and Mooney Raymond

in Semi-Supervised Learning

This chapter discusses a problem resulting from certain clustering tasks from which limited supervision is obtained in the form of pairwise constraints—semi-supervised clustering. Semi-supervised ... More

This chapter discusses a problem resulting from certain clustering tasks from which limited supervision is obtained in the form of pairwise constraints—semi-supervised clustering. Semi-supervised clustering is an instance of semi-supervised learning stemming from a traditional unsupervised learning setting. Several algorithms exist for enhancing clustering quality by using supervision in the form of constraints. These algorithms typically utilize the pairwise constraints to either modify the clustering objective function or to learn the clustering distortion measure. This chapter describes an approach that employs hidden Markov random fields (HMRFs) as a probabilistic generative model for semi-supervised clustering, thereby providing a principled framework for incorporating constraint-based supervision into prototype-based clustering. The HMRF-based model allows the use of a broad range of clustering distortion measures, including Bregman divergences and directional distance measures, making it applicable to a number of domains.Less

The Mechanisms and Benefits of a Future-Oriented Brain

Giovanni Pezzulo

in Seeing the Future: Theoretical Perspectives on Future-Oriented Mental Time Travel

The ubiquity of predictive processing in the brain suggests that it is functionally oriented toward the future. Mechanisms for predictive processing such as internal generative models can give rise ... More

The ubiquity of predictive processing in the brain suggests that it is functionally oriented toward the future. Mechanisms for predictive processing such as internal generative models can give rise to internally generated brain dynamics, thus permitting the brain to “detach,” at least partially, from the here and now of the current sensorimotor context. One example are internally generated sequences of neural activity in the rodent hippocampus, which can be produced and replayed in the absence of external cues, and have been linked to flexible decisions, planning, and memory functions. In this chapter the author considers the idea that other, more sophisticated kinds of detached cognition, such as counterfactual thinking and some forms of episodic simulation, might also be based on internally generated dynamics and use an internal model that originally supported predictive processing.Less

Major Contributions of Econophysics to Financial Economics

Franck Jovanovic and Christophe Schinckus

in Econophysics and Financial Economics: An Emerging Dialogue

Chapter 5 identifies what the potential contributions of econophysics to financial economics. First of all, the way econophysics models are or could be used in trading rooms is discussed. Then the ... More

Chapter 5 identifies what the potential contributions of econophysics to financial economics. First of all, the way econophysics models are or could be used in trading rooms is discussed. Then the theoretical contributions of these models are analyzed from the viewpoint of a financial economist. The rest of the chapter scrutinizes the recent developments in econophysics by presenting them as potential alternative solutions to existing issues in financial economics. With this purpose, two crucial issues are studied: on one hand, the generative models explaining the apparition of power laws in financial data, and, on the other hand, the development of new statistical tests for the validation of new tools capturing non-Gaussian uncertainty.Less

Continuous Dimensions and Distributional Learning

Vsevolod Kapatsinski

in Changing Minds Changing Tools: From Learning Theory to Language Acquisition to Language Change

This chapter describes the evidence for the existence of dimensions, focusing on the difference between the difficulty of attention shifts to a previously relevant vs. irrelevant dimension. It ... More

This chapter describes the evidence for the existence of dimensions, focusing on the difference between the difficulty of attention shifts to a previously relevant vs. irrelevant dimension. It discusses the representation of continuous dimensions in the associationist framework. including population coding and thermometer coding, as well as the idea that learning can adjust the breadth of adjustable receptive fields. In phonetics, continuous dimensions have been argued to be split into categories via distributional learning. This chapter reviews what we know about distributional learning and argues that it relies on several distinct learning mechanisms, including error-driven learning at two distinct levels and building a generative model of the speaker. The emergence of perceptual equivalence regions from error-driven learning is discussed, and implications for language change briefly noted with an iterated learning simulation.Less

Isaak Babel's Portrait of the Artist as a Former Jew

in The Jewish Persona in the European Imagination: A Case of Russian Literature

This chapter examines the place of the generative model of “the Jews” in Isaak Babel's artistic self-image through the close reading of his two stories, “Moi pervyi gus” and “Guy de Maupassant.” It ... More

This chapter examines the place of the generative model of “the Jews” in Isaak Babel's artistic self-image through the close reading of his two stories, “Moi pervyi gus” and “Guy de Maupassant.” It discusses Babel's depiction of “the Jews” as epitomizing the physical and spiritual decline of the European male, and considers his fictional and nonfictional writings as a coherent autobiographical narrative that presents in the symbolic language of a post-Christian culture a portrait of the artist as a former Jew. The chapter also argues that Babel's artistic affronts to Jewish and Christian sensibilities indicate his break from the traditional choices of the Russian-Jewish intelligentsia.Less

Models of network formation

Mark Newman

in Networks

This chapter describes models of the growth or formation of networks, with a particular focus on preferential attachment models. It starts with a discussion of the classic preferential attachment ... More

This chapter describes models of the growth or formation of networks, with a particular focus on preferential attachment models. It starts with a discussion of the classic preferential attachment model for citation networks introduced by Price, including a complete derivation of the degree distribution in the limit of large network size. Subsequent sections introduce the Barabasi-Albert model and various generalized preferential attachment models, including models with addition or removal of extra nodes or edges and models with nonlinear preferential attachment. Also discussed are node copying models and models in which networks are formed by optimization processes, such as delivery networks or airline networks.Less

Archetypal Elaboration

in The Jewish Persona in the European Imagination: A Case of Russian Literature

This chapter examines the archetypal description of the Jews in Christian imagination. It explains that the Jews are considered homologous to Satan and Eve by virtue of their embodiment of the “bad ... More

This chapter examines the archetypal description of the Jews in Christian imagination. It explains that the Jews are considered homologous to Satan and Eve by virtue of their embodiment of the “bad father” and the “bad mother.” The chapter analyzes the descriptive and narrative implications of these archetypal associations for the generative model of the Jews, and provides a summary of the invariant situations and motifs arising from the theological and archetypal structure of the “Jewish” image. It explains that these situations and motifs are the core narrative and descriptive features of the actor called “the Jews.”Less

The Helper

in The Jewish Persona in the European Imagination: A Case of Russian Literature

This chapter examines the role of the Jewish character of Iankel as a helper-figure in Nikolai Gogol's “Taras Bul'ba.” It explains the archetypal conception of “the Jews” as keepers of knowledge and ... More

This chapter examines the role of the Jewish character of Iankel as a helper-figure in Nikolai Gogol's “Taras Bul'ba.” It explains the archetypal conception of “the Jews” as keepers of knowledge and traffickers of information in this story and highlights their use of information not only a commodity but a weapon. The chapter discusses how Gogol conveyed the demonic nature of the “Jewish” Helper-figure based on the descriptive vocabulary furnished by the generative model of “the Jews,” and describes the alterity in the language spoken by Iankel and his kind in the story.Less