Search Results

This chapter looks at selection of mainly autonomic responses and their classical conditioning. Selection of approach or withdrawal, and their classical conditioning are also mentioned. It then goes on to describe a selection of fixed stimulus-response habits; and a selection of arbitrary behaviours to obtain goals, action-outcome learning, and emotional learning. The roles of the prefrontal cortex in decision-making and attention are described. The chapter then goes on to talk about neuroeconomics, reward magnitude, expected value, and expected utility; delay of reward, emotional choice, and rational choice; reward prediction error, temporal difference error, and choice; reciprocal altruism, strong reciprocity, generosity, and altruistic punishment; and dual routes to action, and decision-making.

The “environment-based hypothesis of prephenomenal unity” suggests that both phenomenal and prephenomenal unity are ultimately based on and predisposed by the virtual statistically based unity between the environmental stimuli’s natural and social statistics and the brain’s intrinsic activity: The more the resting state’s low-frequency oscillations shift towards and align themselves to the environmental (and bodily) stimuli’s statistical occurrences across the different discrete points in (physical) time and space, the stronger the degree of the environment–brain unity, and the more likely a high degree of prephenomenal and ultimately phenomenal unity and thus consciousness can be instantiated. Accordingly, a strong or high degree of environment–brain unity predisposes an increased probability of possible phenomenal unity and thus of consciousness. In contrast, a low or weak environment–brain unity decreases the probability of consciousness.

This chapter discusses the relevance of reinforcement learning (RL) to its hierarchical structure. It first reviews the fundamentals of RL, with a focus on temporal-difference learning in actor-critic models. Next, it discusses the scaling problem and the computational issues that stimulated the development of hierarchical reinforcement learning (HRL). The potential neuroscientific correlates of HRL are also described. The chapter also presents the results of some initial empirical tests and ends with directions for further research.

Delving further into development, adaptation, and learning, this chapter considers the potential of reward-oriented optimization of behavior. Reinforcement learning (RL) is motivated from the Rescorla–Wagner model in psychology and behaviorism. Next, a detailed introduction to RL in artificial systems is provided. It is shown when and how RL works, but also current shortcomings and challenges are discussed. In conclusion, the chapter emphasizes that behavioral optimization and reward-based behavioral adaptations can be well-accomplished with RL. However, to be able to solve more challenging planning problems and to enable flexible, goal-oriented behavior, hierarchically and modularly structured models about the environment are necessary. Such models then also enable the pursuance of abstract reasoning and of thoughts that are fully detached from the current environmental state. The challenge remains how such models may actually be learned and structured.

The quest to understand the relationship between neural activity and behavior has been ongoing for well over a hundred years. Although research based on the stimulus-and-response approach to behavior, advocated by behaviorists, flourished during the last century, this view does not, by design, account for unobservable variables (e.g., mental states). Putting aside this approach, modern cognitive science, cognitive neuroscience, neuroeconomics, and behavioral economics have sought to explain this connection computationally. One major hurdle lies in the fact that we lack even a simple model of cognitive function. This chapter sketches an application that connects neuromodulator function to decision making and the valuation that underlies it. The nature of this hypothesized connection offers a fruitful platform to understand some of the informational aspects of dopamine function in the brain and how it exposes many different ways of understanding motivated choice.

Learning belongs to a broad principle of biological design: adapt, match, learn, and forget. Organisms respond to changed demands by re-sculpting at all levels to prepare for what will most likely be next. Physical exercise thickens skin and strengthens muscle; mental practice combined with motor practice refines neural circuits and thereby skills. Circuits at early stages, e.g., V1, re-sculpt to match stable changes in bottom-up statistics. Circuits at later stages can, with extended training and practice, radically rewire. For example, an “object” area can re-wire to recognize and store written symbols and words – as it prunes back circuits that recognize and store objects. Because circuits occupy space in a skull of fixed volume, learning couples inescapably to forgetting. Efficient learning involves all the principles of neural design across all spatial and temporal scales: use chemistry; store only what is needed; store only for as long as needed; store and retrieve information without adding wire. Since learning requires practice, efficient design requires a teaching signal to tell the organism what behaviour to repeat. This is the dopamine reward signal that serves as a final common pathway for many learning systems across the whole brain. A similar circuit operates in insects.

The discussion here considers a much more common learning condition where an agent, such as a human or a robot, has to learn to make decisions in the environment from simple feedback. Such feedback is provided only after periods of actions in the form of reward or punishment without detailing which of the actions has contributed to the outcome. This type of learning scenario is called reinforcement learning. This learning problem is formalized in a Markov decision-making process with a variety of related algorithms. The second part of this chapter will use function approximators with neural networks which have made recent progress as deep reinforcement learning.

Chapter 3 introduces the use of mathematical models and modeling practices in contemporary biological and cognitive sciences. The familiar Lotka–Volterra model of predator–prey relations is used to explain these practices and show how they promote the extensions of predicates, including psychological predicates, into new and often unexpected domains. It presents two models of cognitive capacities that were developed to explain human behavioral data: Ratcliff’s drift-diffusion model of decision-making and Sutton and Barto’s temporal difference model of reinforcement learning. These are now used for fruit flies and neural populations. It also discusses contemporary and ongoing attempts to revise psychological concepts in response to empirical discovery.

This chapter on cognitive processes and behaviours introduces and describes core concepts of psychology, consumer research and behavioural economics that are relevant to consumer neuroscience. The overall emphasis of the chapter is on behavioural concepts and models, where a brief overview of functional neuroanatomy is provided when available and meaningful. Specifically, the following, partly overlapping models and concepts are discussed and put into the context of consumer neuroscience: memory and learning, arousal, attention and awareness, motivation, reward, decision-making, and cognitive processes, the somatic-marker hypothesis, theory of mind and reward-based learning. Despite some have these concepts have been known to psychologist for over a century, however, the presentation of topics generally emphasizes recent works and insights, where an attempt is being made to show how the different concepts interrelate.