Workshop 7: Systems Biology of Decision Making: Abstracts and Lecture Materials
Mate choice is the behavioural mechanism underlying sexual selection, and as such is of huge importance in evolutionary biology. Over the past 30 years research in behavioural ecology has provided lots of data about what traits animals prefer in their mates and why these preferences have evolved. Importantly, it is common for animals to attend to multiple different cues when making mating decisions because different cues may reveal different kinds of information about a potential mate. However, there has been much less attention given to the mechanisms of mate choice. How do animals integrate multiple sources of information about potential mates and compare the options available? Luckily similar problems have been considered in other areas of behavioural science, particularly studies of human decision making. One of my research aims for the past few years has been to try to bring together these different fields and ask whether the choice phenomena identified in human decision making also occur in animals faced with complex decisions. One general finding that is likely to be particularly important is that the choices we make are often dependent on the range of options on offer: our evaluations of options appear to be relative as opposed to absolute. In this talk I will describe some of the cognitive mechanisms that could be responsible for this context-dependency, and argue that on functional grounds similar mechanisms are likely to exist in animals. I will describe experiments from my lab on both human and bird mate choice designed to explore the possibility of context dependency. Finally, I will argue that context-dependency in mating decisions could have implications for the behaviour of animal groups that have not thus far been explored.
Pheromone trails are an important component of ant foraging behaviour. It has been known for a long time that ants lay pheromone trails to attract nest-mates to productive foraging sources, but it has recently become clear that at least some species use more complex pheromone signalling behaviour. We define positive and negative pheromones to be those laid after successful and unsuccessful foraging respectively, and attractive and repulsive pheromones to be those that encourage ants to follow a trail and deter them from it respectively. We might expect positive pheromones to be attractive, and negative pheromones to be repulsive. We address the following questions. Under what circumstances is it advantageous to use more than one pheromone? When are repulsive pheromones advantageous? When are long-term as well as short-term pheromones advantageous?
While first theoretical attempts have been made to study possible decision making mechanisms underlying choices between multiple alternatives, most behavioral and neurophysiological studies have so far focused on choices between two alternatives. The available data sets from choice experiments between multiple alternatives are largely not quantitative enough to address the underlying computational mechanism. The 2AFC version of the random dot motion direction discrimination task has been very helpful in advancing our understanding of the neural and computational mechanisms underlying choices between two alternatives.
Here we present behavioral data from a new version of the random dot motion discrimination task. Subjects are presented with a random dot stimulus containing up to three coherent motion components with different directions. They are asked to pick the dominant direction of motion out of three alternatives. The viewing duration is controlled by the subjects (reaction time task). Response times (RTs) and the subject's choice are measured. The advantage of this task is that it provides the experimenter with full control over the sensory evidence provided for each of the alternatives.
In the 2AFC version of the task both the behavioral as well as neurophysiological data recorded from the parietal cortex of monkeys performing the task could be explained by a computational model based on the idea of a race to threshold between two integrators accumulating the net sensory evidence for a particular choice. Here we show that the behavioral data (probabilities of particular choices as well as RT distributions) from our new 3-choice task are quite well explained by a computational model assuming a race to threshold between three integrators, one for each alternative. Each integrator accumulates the net sensory evidence for a particular alternative. The net sensory evidences are calculated as linear combinations of the activities of three relevant pools of sensory neurons with a positive weight assigned to the pool providing evidence for a particular choice and negative weights assigned to the pools providing evidence against a particular choice.
Previous 2AFC tasks have usually manipulated one stimulus strength parameter, resulting in monotonic relationships between stimulus strength and accuracy and between stimulus strength and mean RT. In these tasks, mean RT is usually also a monotonic function of accuracy. I will demonstrate how the multidimensional stimuli allow us to eliminate this tight coupling between speed and accuracy. Furthermore, in previous diffusion models of binary decision making it has often been assumed that the overall noise level in the decision circuitry stays constant and only the drift rate is affected by changing stimulus strength. I will highlight some aspects of our data set, which suggest that taking changes in the noise level of the sensory representation into account is essential for being able to explain our experimental findings.
Social insects exhibit both individual and collective intelligence. I will illustrate this distributed decision-making with house hunting in rock ants. Each worker, among the 100 or so in a colony, has less than 100,000 neurones (compared to 1011 neurones in humans) yet these ants employ the most sophisticated of all consumer strategies when choosing a new nest. Indeed, they can choose the best-of-N among alternative nests even though each has many different and important attributes. I will show how information can cascade through these social networks enabling colonies to benefit from both individual and collective intelligence. They use quorum sensing to conjure up collective intelligence and to fashion flexible speed accuracy trade-offs. When these ants have achieved a quorum in the new nest they switch from forward tandem running, from the old to the new nest, to social carrying. In addition, they often employ reverse tandem runs from the new nest back to the old one. During tandem runs one leader literally teaches one follower the route between the two nests. I will show how reverse tandem runs can expedite emigrations when these ants have used low quorums in emergencies. Tandem runs seem to be useful even when they do not progress all of the way to the target. I will describe the behaviour of both leaders and followers when tandems break up and show that followers without leaders use Lévy-like searches. I will also describe very recent work, using RFID tags on all workers, that shows how they can choose a better nest that is 9 times further away than a collinear poor one. Certain ants act as consensus breakers to re-route the whole colony. Such ants lead disproportionately more tandem runs first to the poor nest and then onto the more distant better one. Such switchers may have better access to the information needed to make well informed comparisons. Finally, I suspect that these tiny ants have larger behavioural repertoires than many if not most vertebrates. This begs the question; what are big brains for?
Many animals seek resources socially, that is under conditions where the pay-off of foraging strategies depends on the frequency of strategies present in the population. Here I present a brief overview of a number of social foraging decisions, many of which have yet to be studied empirically, focussing more on producer-scrounger decisions which have been the subject of several studies in my laboratory. As is typically done in behavioural ecology, I rely heavily on evolutionary modelling to derive predictions about behaviour. However, I will raise questions concerning what I call the "plasticity gambit", an implicit and untested (unfounded?) assumption that cognitive decision processes must ultimately provide individual foragers with the ability to make decisions in their everyday lives that match behaviour expected to have evolved over evolutionary time.
There is a long history of decision-making research in psychology and economics that has resulted in the development of formal models of behavior, which are inspired by behavioral data or the computational demands of a task. However, cognitive functions such as decision making cannot be completely understood on the basis of mathematical models and behavioral data alone; we have to investigate how mental (cognitive) and neuronal processes map onto each other. We investigate decision making in different domains. First, at the basis of a number of different decisions we are facing in everyday-life stands perceptual decision making. I will discuss how we may use stochastic diffusion models in combination with brain imaging to investigate perceptual decision making. Second, many of our decisions are influenced by the potential outcomes associated with different options. Specifically, I will discuss how dopaminergic neuromodulation (COMT-genotype, aging) affects reward-based decision making.
I will describe two modeling studies: of an individual stimulus identification task, and of a gambling task performed individually and with group feedback. In the first we model accuracy in a two-alternative forced-choice task with cued responses using an Onstein-Uhlenbeck reduction of a leaky competing accumulator model. This provides explicit predictions of psychometric functions, which we fit to data from monkeys performing a motion discrimination task. We then compute optimal (reward maximizing) strategies when reward magnitudes for the alternatives differ, and assess the abilities of individuals to achieve them. In the second study we extend a probabilistic model developed to account for individual choices on matching shoulders and rising optimum gambling tasks by Egelman, Montague, et al., to predict choice patterns in a group context. The model is equivalent to a drift diffusion (DD) process driven by the difference in predicted rewards for two alternatives, based on reward feedback and temporal difference learning. We couple multiple DD processes by incorporating influences from the choices and/or rewards earned by other players in the group. We fit this model to data from human subjects playing a two-armed bandit game, singly and in groups, and with various types of feedback on their own and other group members' rewards and choices.
The modeling work was done with Sam Feng (study 1) and Andrea Nedic, Jonathan Cohen and Deborah Prentice (study 2). Experimental data was collected and provided by Alan Rorie and Bill Newsome (study 1), and Damon Tomlin (study 2), who also advised on the modeling. Support was provided by AFOSR under the 2007 MURI Programs 15 and 16.
Decision-making dynamics and spatial dynamics in fish schools are naturally coupled when individuals only sense locally. As relative positions of fish change, neighbors come and go and the flow of information varies; changes in information flow affect the dynamics of consensus and this in turn influences collective motion. I will describe a study of consensus dynamics for a school using a model that extends coupled oscillator dynamics to include spatial dynamics and represents changing information flow with time-varying graphs. We examine the case of periodically time-varying information flow, motivated by fish schooling data of I. Couzin and A. Kao, in which killifish exhibit oscillatory speed profiles and nearest neighbors anti-synchronize the phases of their speed oscillations. We explore the possible advantages for speed and accuracy in consensus dynamics and for increased density of spatial sampling.
The problem of how to compromise between speed and accuracy in decision-making faces organisms at many levels of biological complexity. Recent models of decision-making in the vertebrate brain have shown how simple neural models can implement statistically optimal decision-making processes that are able to minimise decision time for any desired error rate. Such models consider mutually inhibitory populations of neurons. The activation threshold required of these populations can be varied to emphasise speed, or accuracy, of decision-making. There are some striking similarities between these models and models of the decision-making processes in social insect colonies when searching for a new nest site. In this talk I will present stochastic models of collective behaviour in social insects that can be directly compared with corresponding neural models, and show how they may also implement statistically optimal decision-making in a similar manner to neural circuits.
A generally useful decision mechanism must include a stage at which information from very disparate sources (i.e. sensory, reward, priors, etc) can be cast into a common neural currency for establishing the likelihood of one choice vs. the other. In monkeys, the lateral intraparietal area is thought to play such a role in oculomotor decisions, and past studies have indeed produced evidence for neural decision variables in LIP originating from sensory input (e.g. stimulus motion) or from economic value (reward size or probability). But no study has determined whether disparate signals are actually integrated in LIP, what the weights of the disparate signals are at the single cell level, and whether the relative weights at the neural level matches the relative weights inferred from behavior. Alan Rorie, a graduate student in my lab, and I addressed these questions by training monkeys on a choice paradigm in which sensory information about visual motion must be combined with information about reward size to make optimal choices. We show for the first time that sensory and reward signals are combined in single neuron activity in LIP, and that the relative magnitude of their influence at the population level matches their influence at the behavioral level. The data provide quantitative support for the notion that LIP activity reflects a high level of processing in which information from disparate sources are cast into a common neural currency for guiding oculomotor choices.
Systems biology of decision making focuses on understanding the structures, dynamics, and evolution of complex interconnected biological mechanisms that support decision making by individuals and social animal groups. In this talk, an experimentally validated model of the nest-site selection process of honey bee swarms is introduced. In this spatially distributed dynamical feedback process individual bee actions and bee-to-bee communications combine to produce an emergent "consensus" nest choice. The process has connections to neurobiological cognition systems, especially at the behavioral level: the swarm can effectively discriminate between different quality nest sites and eliminate from consideration relatively inferior distractor sites. Simulations indicate that individual-level bee decision-making mechanisms have been tuned by natural selection to provide a balance between the need for fast and accurate decisions at the group level. For more information see: http://www.ece.osu.edu/~passino/
Social insects have special interest in the study of decision-making, because they make choices both as individuals and as collectives. I will describe how these levels interact in nest-site selection by Temnothorax ants. Individual ants follow a complex behavioral algorithm that allows a group of poorly informed individuals to collectively choose the best site, without any central control. This algorithm is based on quality-dependent positive feedback and a threshold response, two features of general importance to collective decision-making systems. Quantitative tuning of this algorithm allows colonies to emphasize either the speed or accuracy of decision-making. The algorithm hinges on attainment of a nestmate quorum that triggers full commitment to a particular option. A simple model derived from the study of vertebrate decision-making may explain how the quorum can be detected by the integration of encounter rates between ants. Recent work suggests that the concept of rationality may reveal more about the interaction of group and individual levels. Decision-makers are irrational when they fail to attach consistent values to options. For individuals, irrationality can reliably be obtained through several experimental paradigms involving options with multiple, conflicting attributes. Faced with one of these paradigms, Temnothorax colonies nonetheless show consistent preferences, suggesting that collective decision-making algorithms may filter out the effects of irrationality at the level of individual workers.
I will talk about 2-3 topics in simple decision making. I will first briefly introduce the diffusion model for simple decision making and note that it can account for correct and error RT distributions as well as accuracy. Then I will show that an EEG measure indexes trial by trial variability in drift rate in the model. I will then show interpretations of the effects of age and sleep deprivation on performance in terms of an analysis based on the model. If time permits I will show how the model can uncover the main individual differences on speed of processing in simple two choice tasks such as recognition memory and word identification.
Movements of inanimate objects explained by external forces referred to as causes. In contrast, many movements of humans are described as actions directed toward a goal for a reason. A purposeful action (a wink) is distinguished from a mere event (a blink) by reference to some intelligible plan because actions are performed to achieve a goal. In other words, actions have reasons ("I did it for..."), but events just have causes ("It happened because..."). Reasons for actions are explanations in terms of purposes, goals and beliefs. But if all actions are really caused by just neurons firing and muscles contracting, then how can there be any reasons for actions? This presentation will seek to articulate how intentional reasons can be reconciled with neural causes. The answer will emphasize the many-to-one mapping of neural activity onto behavior and cognition so that if a given action can arise from different brain states, then the relationship of the behavior to an intention holds in virtue of the content of the representation of the intention and not its neural realization as such. Thus, a movement can be called an intentional action if and only if it originates from a cognitive state with meaningful content, and this content defines the cognitive state's causal influence. But this analysis depends on whether the brain knows what it means to do. In fact, recent cognitive neuroscience research has described particular brain circuits that register errors and success. Such signals can be used to adjust behavior and provide the basis for distinguishing "I did" from "it happened" which is just what is needed to feel like we are acting with freedom and responsible power.
Over last two decades, significant progress has been made toward understanding the mechanistic basis of decision making, both in individual animals and in animal groups. I will review one of the systems where the strongest progress has been made toward understanding group decision making: nest-site choice in honeybees. We will see that the bees' decision-making process has many remarkable consistencies with the picture that has emerged from neurobiological studies of individual decision making. In both cases, the process of decision-making involve building a sensory representation of the external world, then transforming this sensory representation into a decision, and finally implementing the chose course of action. Furthermore, in both cases, the decision-making process is essentially a race between competing accumulations of evidence in support of the various alternatives, with the choice determined by which accumulation first reaches the threshold level of evidence needed for a response. These and other similarities between neuronal and social systems of decision making point to principles in the "design" of distributed decision-making systems, and these will be discussed.
Experimental studies show that animals have strong preferences for immediacy. Specifically, experimental data suggest that animals make systematic errors by over-valuing small but immediate benefits, and under-valuing larger later consequences. Why would natural selection favor a mechanism that makes such a glaring error? One hypothesis is that these are not errors at all, because delay reduces or discounts the fitness value of benefits. A second hypothesis is that the 'impulsive' rules revealed by these experiments seldom lead to errors in natural situations. I call this the 'ecological rationality' hypothesis. My laboratory has developed this idea via experiments that compare the standard 'self-control' preparation to a more naturalistic choice preparation that attempts to mimic patch exploitation decisions. I review several of these experiments. We consistently find that subjects achieve higher levels of performance (e.g. higher long term intake rates) in our more naturalistic 'patch' situation, but many simple hypotheses (including the short-term rate model) fail to explain this difference. Finally, I will discuss four general problems in decision-making and how they emerge my groups studies of impulsivity. These are: 1) can we specify the relationship between learned and evolved solutions to behavioral problems; 2) how can we meaningfully integrate mechanistic and functional approaches to decision-making; 3) can we build a predictive theory of bounded rationality or is this fundamentally a post hoc approach; and 4) are animal decision mechanisms special tools for specific tasks or general mechanisms that apply across the broad, and how would we know?
A number of neurocomputational models have been proposed to account for the algorithm used by the brain to make decisions when faced with ambiguous information. In this talk I examine shared and diverging assumptions that these models make in accounting for choice patterns. For example, I examine the need to assume an integration over noisy samples during stimulus presentation, the existence of competition (or lateral inhibition) between choice alternatives and the assumption that a response criterion is used even in non-speeded judgments (making the impact of early evidence more strongly weighted). Preliminary data on choice tasks that manipulate the time-course of the information flow will be presented examining the models' predictions. Finally, I discuss how these models can be used to account for judgments of confidence, and for the fact that observers have the metacognitive ability to judge incorrect responses as less confident.
Work done in collaboration with Eddy J. Davelaar.
