• Etzel JA, Gazzola V, Keysers C, Testing Simulation Theory with Cross-Modal Multivariate Classification of fMRI Data. PlosOne, 2008, 3, Issue 11, e3690.
  Critical to the idea of simulation is the fact that the pattern of activity in premotor and parietal areas is similar between action execution and action perception. Here we show that if we train a pattern classifier to distinguish the sound of hand and mouth actions based on either premotor (BA44/6) or posterior parietal activity, it can tell hand and mouth actions apart while participants execute these two classes of actions. This is therefore the first demonstration that we do not only activate the same brain areas during action execution and perception, but that the pattern of activity is indeed similar enough to support simulation theories.
  
• Etzel JA, Gazzola V, Keysers C, Testing Simulation Theory with Cross-Modal Multivariate Classification of fMRI Data. PlosOne, 2008, 3, Issue 11, e3690.
  Critical to the idea of simulation is the fact that the pattern of activity in premotor and parietal areas is similar between action execution and action perception. Here we show that if we train a pattern classifier to distinguish the sound of hand and mouth actions based on either premotor (BA44/6) or posterior parietal activity, it can tell hand and mouth actions apart while participants execute these two classes of actions. This is therefore the first demonstration that we do not only activate the same brain areas during action execution and perception, but that the pattern of activity is indeed similar enough to support simulation theories.
• Christian Keysers and Luciano Fadiga, The mirror neuron system: New frontiers, Social Neuroscience, 2008, 3, 193-198 .
  As an introduction to a special issue of Social Neuroscience, we review briefly what we know about the mirror system and point out what we believe to be some of the new frontiers of the study of neuroscience: What does the mirror system code? How is the mirror system embedded into the mosaic of circuits that compose our brain? How does the mirror system contribute to communication, language and social interaction? Can the principle of mirror neurons be extended to emotions, sensations and thoughts? Papers using a wide range of methods, including single cell recordings, fMRI, TMS, EEG and psychophysics, collected in the remainder of that special issue, start to give us some impressive answers.
  
• Jabbi M, Bastiaansen J, Keysers C(2008), A Common Anterior Insula Representation of Disgust Observation, Experience and Imagination Shows Divergent Functional Connectivity Pathways, PlosOne,3,issue 8,e2939 .
  Similar brain regions are involved when we imagine, observe and execute an action. Is the same true for emotions? Here, the same subjects were scanned while they (a) experience, (b) view someone else experiencing and (c) imagine experiencing gustatory emotions (through script-driven imagery). Capitalizing on the fact that disgust is repeatedly inducible within the scanner environment, we scanned the same participantswhile they (a) view actors taste the content of a cup and look disgusted (b) tasted unpleasant bitter liquids to induce disgust, and (c) read and imagine scenarios involving disgust and their neutral counterparts. To reduce habituation, we inter-mixed trials of positive emotions in all three scanning experiments. We found voxels in the anterior Insula and adjacent frontal operculum to be involved in all three modalities of disgust, suggesting that simulation in the context of social perception and mental imagery of disgust share a common neural substrates. Using effective connectivity, this shared region however was found to be embedded in distinct functional circuits during the three modalities, suggesting why observing, imagining and experiencing an emotion feels so different.
  
• Thioux M, Gazzola V,Keysers C(2008), Action Understanding: How, What and Why, Current Biology 18:10 R431-R434.
  This study shows that our action perception system is highly integrated. Visual areas and regions of the mirror neuron system computing “how”and “what”are likely to be as important as mentalizing areas, for understanding “why”. Also a split between “how”and “what” between visual areas and mirror neuron system is artificial. All mirror neurons need visual input and different mirror neurons represent both “how”and “what”.
  
• van der Gaag C, Minderaa R, Keysers C(2007), Facial expressions: What the mirror neuron system can and cannot tell us, Social Neuroscience 2:179-222 .
  Here we show that observing facial expressions recruits a circuit composed of temporal, parietal and inferior frontal regions that are also active while subjects execute similar facial expressions. The inferior frontal and temporal nodes of this putative mirror system were selectively more active during the viewing of emotional facial expressions than other control stimuli including moving patterns and neutral facial expressions. Such activations were found in 3 separate experiments, including passive viewing, an emotion discrimination and imitation task, suggesting that the putative mirror system for facial expressions is spontaneously active. We failed to find brain areas that reliably discriminate between different facial expressions during both observation and execution.
• Gazzola V, van der Worp H, Mulder T, Wicker B, Rizzolatti G, Keysers C(2007) Aplasics born without hands mirror the goal of hand actions with their feet. Current Biology 17:1235-1240.
  While it is generally assumed that the sight of actions activates corresponding motor representations in the observer, what 'corresponding' really means remained unclear. Here we show that subjects born without hands and arms activate a combination of effector independent motor representations and motor representations of their foot actions while observing the hand actions of other individuals. These activations are as strong as those of control subjects, suggesting that the mirror system transforms the goal of hand actions into motor representation with similar goals despite differences in effectors. This finding may help understand why non-human primates with mirror systems do not imitate the way in which an action is performed despite their ability to learn to achieve a goal through observation. By showing that mirroring is thus a subjective interpretation of the actions of others in terms of our own, personal motor programs even if these deviate significantly from those observed, we both show the remarkable flexibility of the mirror system and the fact that rather than being an accurate mirror of other people's behaviour, the mirror system performs an interpretation, that may also introduce the potential for misinterpretation. Supplemental Material
• Keysers C and Gazzola V(2007) Integrating simulation and theory of mind: from self to social cognition. Trends in Cognitive Sciences.
  In this article we propose how simulation and ToM could work together based on the idea that regions in the medial prefrontal cortex that introspect our own states could reflect about the states of others by introspecting the simulated states of others induced by mirror like systems
• van der Gaag C, Minderaa R, Keysers C(2007), The BOLD signal in the amygdala does not differentiate between dynamic facial expressions, Social Cognitive and Addective Neuroscience .
  In this article we show that the amygdala responds similarly to movies of happy, fearful, disgusted and neutral facial expressions, suggesting that fear selectivity in the amygdala might be an artifact of using static images in previous studies
• Gazzola V, Rizzolatti G, Wicker B, Keysers C (2007), The Anthropomorphic Brain: the mirror neuron system responds to human and robotic actions, NeuroImage .
  In this article we show that the mirror system responds also to the actions of robots. This suggests that the mirror system matches the goal of an action and not its kinematic details
• Jabbi M., Swart M., Keysers C (2007), Empathy for positive and negative emotions in the gustatory cortex, Neuroimage 34:1744-53.
  In this article we show that activations in the anterior insula during the observation of the emotions of others correlate with empathy, and that the insula responds both to the sight of positive and negative emotions
• Gazzola V, Aziz-Zadeh L, Keysers C (2006), Empathy and the somatotopic auditory mirror system in humans, Current Biology 16:1824-9.
  In this article we show that action programs in the premotor cortex become activated both by the vision and the sound of the actions of others in a somatotopical matter. Most importantly, we show a link between empathy and the mirror system: more empathic individuals activate their mirror system more while listening to the actions of others.
• Keysers C, Gazzola V (2006), Towards a unifying neural theory of social cognition, Progress in Brain Research.
  In this article we provide a framework in which we combine our data on the observation/listening of actions, sensations and emotions with patient lesion data to propose a parsimonious hypothesis on the neural basis of social cognition. We examine the responses of that system to non-human agents and discuss the relationship between ToM and simulation.
• Gallese V, Keysers C, Rizzolatti G (2004), A unifying view of the basis of social cognition, Trends in Cognitive Sciences 8: 396-403.
  In this article we provide a unifying neural hypothesis on how individuals understand the actions and emotions of others. Our main claim is that the fundamental mechanism at the basis of the experiential understanding of others' actions is the activation of the mirror neuron system. A similar mechanism, but involving the activation of viscero-motor centers, underlies the experiential understanding of the emotions of others.
• Keysers C, Wicker B, Gazzola V, Anton JL, Fogassi L, Gallese V (2004), A touching sight: SII/PV activation during the observation and experience of touch, Neuron 42: 335-346.Watching the movie scene in which a tarantula crawls on James Bond's chest can make us literally shiver-as if the spider crawled on our own chest. What neural mechanisms are responsible for this "tactile empathy"? The observation of the actions of others activates the premotor cortex normally involved in the execution of the same actions. If a similar mechanism applies to the sight of touch, movies depicting touch should automatically activate the somatosensory cortex of the observer. Here we found using fMRI that the secondary but not the primary somatosensory cortex is activated both when the participants were touched and when they observed someone or something else getting touched by objects. The neural mechanisms enabling our own sensation of touch may therefore be a window also to our understanding of touch.
• Keysers C, Perrett DI (2004), Demystifying social cognition: a Hebbian perspective, Trends in Cognitive Sciences 8: 501-507.
  For humans and monkeys, understanding the actions of others is central to survival. Here we review the physiological properties of three cortical areas involved in this capacity: the STS, PF and F5. Based on the anatomical connections of these areas, and the Hebbian learning rule, we propose a simple but powerful account of how the monkey brain can learn to understand the actions of others by associating them with self-produced actions, at the same time discriminating its own actions from those of others. As this system appears also to exist in man, this network model can provide a framework for understanding human social perception.
• Keysers C, Kohler E, Umilta MA, Nanetti L, Fogassi L, Gallese V (2003), Audiovisual mirror neurons and action recognition, Experimental Brain Research 153: 628-636.Many object-related actions can be recognized both by their sound and by their vision. Here we describe a population of neurons in the ventral premotor cortex of the monkey that discharge both when the animal performs a specific action and when it hears or sees the same action performed by another individual. These 'audiovisual mirror neurons' therefore represent actions independently of whether these actions are performed, heard or seen. The magnitude of auditory and visual responses did not differ significantly in half the neurons. A neurometric analysis revealed that based on the response of these neurons, two actions could be discriminated with 97% accuracy.
• Wicker B, Keysers C, Plailly J, Royet JP, Gallese V, Rizzolatti G (2003), Both of us disgusted in My Insula: The common neural basis of seeing and feeling disgust, Neuron 40: 655-664.
  What neural mechanism underlies the capacity to understand the emotions of others? Does this mechanism involve brain areas normally involved in experiencing the same emotion? We performed an fMRI study in which participants inhaled odorants producing a strong feeling of disgust. The same participants observed video clips showing the emotional facial expression of disgust. Observing such faces and feeling disgust activated the same sites in the anterior insula and to a lesser extent in the anterior cingulate cortex. Thus, as observing hand actions activates the observer's motor representation of that action, observing an emotion activates the neural representation of that emotion. This finding provides a unifying mechanism for understanding the behaviors of others.
• Kohler E, Keysers C, Umilta MA, Fogassi L, Gallese V, Rizzolatti G (2002), Hearing sounds, understanding actions: Action representation in mirror neurons, Science 297: 846-848.Many object-related actions can be recognized by their sound. We found neurons in monkey premotor cortex that discharge when the animal performs a specific action and when it hears the related sound. Most of the neurons also discharge when the monkey observes the same action. These audiovisual mirror neurons code actions independently of whether these actions are performed, heard, or seen. This discovery in the monkey homolog of Broca's area might shed light on the origin of language: audiovisual mirror neurons code abstract contents - the meaning of actions - and have the auditory access typical of human language to these contents.
• Gallese V, Keysers C (2001), Mirror neurons: A sensorimotor representation system, Behavioral and Brain Sciences 24: 983.
  Positing the importance of sensorimotor contingencies for perception is by no means denying the presence and importance of representations. Using the evidence of mirror neurons we will show the intrinsic relationship between action control and representation within the logic of forward models.
• Umilta MA, Kohler E, Gallese V, Fogassi L, Fadiga L, Keysers C, Rizzolatti G (2001), I know what you are doing: A neurophysiological study, Neuron 31: 155-165.
  In the ventral premotor cortex of the macaque monkey, there are neurons that discharge both during the execution of hand actions and during the observation of the same actions made by others (mirror neurons). In the present study, we show that a subset of mirror neurons becomes active during action presentation and also when the final part of the action, crucial in triggering the response in full vision, is hidden and can therefore only be inferred. This implies that the motor representation of an action performed by others can be internally generated in the observer's premotor cortex, even when a visual description of the action is lacking. The present findings support the hypothesis that mirror neuron activation could be at the basis of action recognition.

Key publications on The rapid visual processing of biological stimuli:

• Keysers C, Xiao DK, Foldiak P, Perrett DI (2005), Out of sight but not out of mind: The neurophysiology of iconic memory in the superior temporal sulcus, Cognitive Neuropsychology 22: 316-332.
  Iconic memory, the short-lasting visual memory of a briefly flashed stimulus, is an important component of most models of visual perception. Here we investigate what physiological mechanisms underlie this capacity by showing rapid serial visual presentation (RSVP) sequences with and without interstimulus gaps to human observers and macaque monkeys. For gaps of up to 93 ms between consecutive images, human observers and neurones in the temporal cortex of macaque monkeys were found to continue processing a stimulus as if it was still present on the screen. The continued firing of neurotics in temporal cortex may therefore underlie iconic memory. Based on these findings, a neurophysiological vision of iconic memory is presented.
• Edwards R, Xiao DK, Keysers C, Foldiak P, Perrett D (2003), Color sensitivity of cells responsive to complex stimuli in the temporal cortex, Journal of Neurophysiology 90: 1245-1256.
  The inferotemporal (IT) cortex of the monkey lies at the head of the ventral visual pathway and is known to mediate object recognition and discrimination. It is often assumed that color plays a minor role in the recognition of objects and faces because discrimination remains highly accurate with black-and-white images. Furthermore it has been suggested that for rapid presentation and reaction tasks, object classification may be based on a first wave of feedforward visual information, which is coarse and achromatic. The fine detail and color information follows later, allowing similar stimuli to be discriminated. To allow these theories to be tested, this study investigates whether the presence of color affects the response of IT neurons to complex stimuli, such as faces, and whether color information is delayed with respect to information about stimulus form in these cells. Color, achromatic, and false-color versions of effective stimuli were presented using a rapid serial visual presentation paradigm, and responses recorded from single cells in IT of the adult monkey. Achromatic images were found to evoke significantly reduced responses compared with color images in the majority of neurons (70%) tested. Differential activity for achromatic and colored stimuli was evident from response onset with no evidence to support the hypothesis that information about object color is delayed with respect to object form. A negative correlation ( P < 0.01) was found between cell latency and color sensitivity, with the most color-sensitive cells tending to respond earliest. The results of this study suggest a strong role for color in familiar object recognition and provide no evidence to support the idea of a first wave of form processing in the ventral stream based on purely achromatic information.
• Foldiak P, Xiao DK, Keysers C, Edwards R, Perrett DI (2003), Rapid serial visual presentation for the determination of neural selectivity in area STSa, Roots of Visual Awareness 144: 107-116.
  We show that rapid serial visual presentation (RSVP) in combination with a progressive reduction of the stimulus set is an efficient method for describing the selectivity properties of high-level cortical neurons in single-cell electrophysiological recording experiments. Rapid presentation allows the experimental testing of a significantly larger number of stimuli, which can reduce the subjectivity of the results due to stimulus selection and the lack of sufficient control stimuli. We prove the reliability of the rapid presentation and stimulus reduction methods by repeated experiments and the comparison of different testing conditions. Our results from neurons in area STSa of the macaque temporal cortex provide a well-controlled confirmation for the existence of a population of cells that respond selectively to stimuli containing faces. View tuning properties measured using this method also confirmed earlier results. In addition, we found a population of cells that respond reliably to complex non-face stimuli, though their tuning properties are not obvious.
• Keysers C, Perrett DI (2002), Visual masking and RSVP reveal neural competition, Trends in Cognitive Sciences 6: 120-125.
  A test visual stimulus is harder to recognize when another stimulus is presented in close temporal vicinity; presenting stimuli in close spatial vicinity of a test stimulus reduces its visibility; presenting a stimulus to one eye can render invisible another stimulus presented to the other eye; and perceiving one interpretation of an ambiguous image prevents the simultaneous perception of other visual interpretations. A single, neurophysiological theory, which may be called 'neural competition' might explain all these phenomena: when two alternative neural visual representations co-exist in the brain,they compete against each other.
• Baker CI, Keysers C, Jellema T, Wicker B, Perrett DI (2001), Neuronal representation of disappearing and hidden objects in temporal cortex of the macaque, Experimental Brain Research 140: 375-381.
  Neurons in the anterior regions of the banks of the superior temporal sulcus (STSa) of the macaque monkey respond to the sight of biologically significant stimuli such as faces, bodies and their motion. In this study the responses of STSa neurons were recorded during the gradual occlusion of the experimenter and other mobile objects behind screens at distances of 0.5-4 in from the monkeys. The experimenter or other object remained out of sight for 3-15 s before emerging back in to view. We describe a population of neurons (n=33) showing increased activity during the occlusion of objects that was maintained for up to 11 s following complete occlusion (when only the occluder itself was visible). This increase in activity was selective for the position of the occlusion within the testing room. Many neurons showed little or no change in activity prior to occlusion when the object or experimenter was completely in view. By coding for the presence and location of recently occluded objects, these responses may contribute to the perceptual capacity for object permanence.
• Keysers C, Xiao DK, Foldiak P, Perrett DI (2001), The speed of sight, Journal of Cognitive Neuroscience 13: 90-101.
  Macaque monkeys were presented with continuous rapid serial visual presentation (RSVP) sequences of unrelated naturalistic images at rates of 14-222 msec/image, while neurons that responded selectively to complex patterns (e.g., faces) were recorded in temporal cortex. Stimulus selectivity was preserved for 65% of these neurons even at surprisingly fast presentation rates (14 msec/image or 72 images/sec). Five human subjects were asked to detect or remember images under equivalent conditions, Their performance in both tasks was above chance at all rates (14-111 msec/image). The performance of single neurons was comparable to that of humans anti responded in a similar way to changes in presentation rate, The implications for the role of temporal cortex cells in perception are discussed.

Dissertations of the Social Brain Lab