• PhD Students:
Nikola Valchev, Cui Fang
• Researchers:
Valeria Gazzola, Marc Thioux
• PostDocs:
Leonardo Cerliani
• Collaborators:
Christiaan van der Gaag

• Others:
Annerose Engel, Mbemba Jabbi (NIH,Bethesda), Filippo Migliorati

• Keysers C, Kaas J, Gazzola V."Somatosensation in Social Perception." Nature Reviews Neuroscience. 2010 Jun;11(6):417-28.
  The discovery of mirror neurons in motor areas of the brain has led many to assume that our ability to understand other people’s behaviour partially relies on vicarious activations of motor cortices. This review focuses the limelight of social neuroscience on a different set of brain regions: the somatosensory cortices. These have anatomical connections that enable them to have a role in visual and auditory social perception. Studies that measure brain activity while participants witness the sensations, actions and somatic pain of others consistently show vicarious activation in the somatosensory cortices. Neuroscientists are starting to understand how the brain adds a somatosensory dimension to our perception of other people.
• Schippers MB, Roebroeck A, Renken R, Nanetti L, Keysers C."Mapping the Information flow from one brain to another during gestural communication". Proc Natl Acad Sci U S A. 2010 May 18;107(20):9388-93.
  Both the putative mirror neuron system (pMNS) and the ventral medial prefrontal cortex (vmPFC) are deemed important for social interaction: the pMNS because it “resonates” with the actions of others, the vmPFC because it is involved in mentalizing. The resonance property of the pMNS has never been investigated and classical fMRI experiments have only investigated whether pMNS regions augment their activity when an action is seen or executed. Here we directly explore whether such resonance occurs during continuous streams of actions. We let participants play the game of charades while we measured brain activity of both gesturer and guesser. We then applied a method to localize directed influences between the brains of the participants: between-brain Granger-causality mapping. Results show that a guesser's brain activity in regions involved in mentalizing and mirroring echoes the temporal structure of a gesturer's brain activity. This provides evidence for resonance theories and indicates a fine-grained temporal interplay between regions involved in motor planning and regions involved in thinking about the mental states of others. This method enables experiments to be more ecologically valid by providing the opportunity to leave social interaction unconstrained. This would allow us to tap into the neural substrates of social deficits such as autism spectrum disorder.
• Keysers C., Gazzola V. "Social Neuroscience: Mirror Neurons recorded in Humans".Current Biology, 2010,20:8.
  New single-cell recordings show that humans do have mirror neurons, and in more brain regions than previously suspected. Some action–execution neurons were seen to be inhibited during observation, possibly preventing imitation and helping self/other discrimination.
• Keysers C."Mirror Neurons. Where can I find out more?"Current Biology, 2009, 19:21.
  An interview and a quick guide about mirror neurons.
• Keysers C, Gazzola V."Expanding the mirror: vicarious activity for actions, emotions and sensations".Current Opinions in Neurobiology, 2009, 19:1-6.
  The aim of this review is to highlight two developments of the last years about the neural basis of empathy. First, mirror neurons for actions not only exist in the premotor cortex or in monkeys, but also other brain regions and species have similar properties. Second vicarious activity can also be measured for the emotions and sensations of others. Although we still need to empirically explore the function and development of these vicarious activations, we should stop thinking of vicarious brain activity as a peculiar property of the premotor cortex: instead it seems to be a very common phenomenon which leads social stimuli to recruit a wide range of seemingly private neural systems.
• Perret DI, Xiao D, Barraclough NE, Keysers C, Oram MW."Seeing the future: Natural image sequences produce "anticipatory" neuronal activity and bias perceptual report". Q J Exp Psychol (Colchester). 2009 Nov;62(11):2081-104.
  While most of our work investigates the contributions of simulation to perception, this paper relates human perception to the functioning of cells in the temporal cortex that are engaged in high-level pattern processing. After a review of historical developments concerning the functional organization of cells processing faces and the selectivity for faces in cell responses., this paper focusses on the comparison of perception and cell responses to images of faces presented in sequences of unrelated images. Specifically the cell function and perception in circumstances where meaningful patterns occur momentarily in the context of a naturally or unnaturally changing visual environment. Experience of visual sequences allows anticipation, yet one sensory stimulus also “masks” perception and neural processing of subsequent stimuli. Forward masking has unrecognized benefits for perception because it can transform neuronal activity to make it predictive during natural change.
  
• Schippers MB, Gazzola V, Goebel R, Keysers C."Playing Charades in the fMRI: Are Mirror and/or Mentalizing Areas Involved in Gestural Communication?".PlosOne, 2009, 4, Issue 8.
  Spoken language plays an important role in communication between human individuals. Manual gestures however often aid the semantic interpretation of the spoken message, and gestures may have played a central role in the earlier evolution of communication. Here the social game of charades has been used to investigate the neural basis of gestural communication by having participants produce and interpret meaningful gestures while their brain activity was measured using functional magnetic resonance imaging.
  
• Kokal I, Gazzola V, Keysers C."Acting together in and beyond the mirror neuron system".Neuroimage. 2009 Oct 1;47(4):2046-56.
  Moving a set dinner table often takes two people, and doing so without spilling the glasses requires the close coordination of the two agents' actions. The mirror neuron system may be the key neural locus of such coordination. Instead, such coordination recruits two separable sets of areas: one that could translate between motor and visual codes and one that could integrate these information to achieve common goals. This finding shows that although the putative mirror neuron system can play a critical role in joint actions by translating both agents' actions into a common code, the flexible remapping of our own actions with those of others required during joint actions seems to be performed outside of the putative mirror neuron system.
  
• Nanetti L, Cerliani L, Gazzola V, Renken R, Keysers C."Group analyses of connectivity-based cortical percellation using repeated k-means clustering". Neuroimage. 2009 Oct 1;47(4):1666-77. K-means clustering is a popular tool for connectivity-based cortical segmentation using Diffusion Weighted Imaging (DWI) data. Here we show that the output of the algorithm depends on the initial placement of starting points, and that different sets of starting points therefore could lead to different solutions. We found that in both brain regions, repeatedly applying k-means often leads to a variety of rather different cortical based parcellations, that ~256 k-means repetitions are needed to accurately estimate the distribution of possible solutions. We propose a method to employ the variability of clustering solutions to assess the reliability with which certain voxels can be attributed to a particular cluster. In addition, we show that the proportion of voxels that can be attributed significantly to either cluster in the SMA and preSMA is relatively higher than in the insula and discuss how this difference may relate to differences in the anatomy of these regions.
  
• Bastiaansen JACJ, Thioux M, Keysers C."Evidence for mirror systems in emotions", Philosophical Transactions of the Royal Society, Biological Sciences, 2009, 364, 2391-2404.
  Why do we feel tears well up when we see a loved one cry? Why do we wince when we see other people hurt themselves? Seeing the emotions of others also recruits regions involved in experiencing similar emotions, although there does not seem to be a reliable mapping of particular emotions onto particular brain regions. Instead, emotion simulation seems to involve a mosaic of affective, motor and somatosensory components. Recent experimental evidence suggests that motor simulation may be a trigger for the simulation of associated feeling states. This mosaic of simulations may be necessary for generating the compelling insights we have into the feelings of others. Through their integration with, and modulation by, higher cognitive functions, they could be at the core of important social functions, including empathy, mind reading and social learning.
  
• Joset A. Etzel, Valeria Gazzola, Christian Keysers."An Introduction to Anatomical ROI-based fMRI Classification Analysis", Brain Research, 1282 (2009)114-125.
  Modern cognitive neuroscience often thinks at the interface between anatomy and function, hypothesizing that one structure is important for a task while another is not. A flexible and sensitive way to test such hypotheses is to evaluate the pattern of activity in the specific structures using multivariate classification techniques. These methods consider the activation patterns across groups of voxels, and so are consistent with current theories of how information is encoded in the brain: that the pattern of activity in brain areas is more important than the activity of single neurons or voxels. This paper is an introduction to applying classification methods to functional magnetic resonance imaging (fMRI) data, particularly for region of interest (ROI) based hypotheses.
  
• Onur O, Walter H, Schlaepfer Th, Rehme A, Schmidt C, Keysers C, Maier W, Hurlemann R. "Noradrenergic ehancement of amygdala responses to fear", Social Cognitive and Affective Neuroscience Advance Access, February 25,2009.
  Multiple lines of evidence implicate the basolateral amygdala (BLA) and the noradrenergic (norepinephrine, NE) system in responding to stressful stimuli such as fear signals, suggesting hyperfunction of both in the development of stress-related pathologies including anxiety disorders. However, no causative link between elevated NE neurotransmission and BLA hyperresponsiveness to fear signals has been established to date in humans. To determine whether or not increased noradrenergic tone enhances BLA responses to fear signals, we used functional magnetic resonance imaging (fMRI) and a strategy of pharmacologically potentiating NE neurotransmission in healthy volunteers. 18 subjects were scanned two times on a facial emotion paradigm and given either a single-dose placebo or 4 mg of the selective NE reuptake inhibitor reboxetine 2 h prior to an fMRI session. We found that reboxetine induced an amygdala response bias towards fear signals that did not exist at placebo baseline. This pharmacological effect was probabilistically mapped to the BLA. Extrapolation of our data to conditions of traumatic stress suggests that disinhibited endogenous NE signaling could serve as a crucial etiological contributor to post-traumatic stress disorder (PTSD) by eliciting exaggerated BLA responses to fear signals.
  
• Mbemba Jabbi, Christian Keysers."Inferior Frontal Gyrus Activity Triggers Anterior Insula Response to Emotional Facial Expressions", Emotion 2008, Vol 8, no 6, 775-780.
  The observation of movies of facial expressions of others has been shown to recruit similar areas involved in experiencing one’s own emotions: the inferior frontal gyrus (IFG), the anterior insula and adjacent frontal operculum (IFO). The causal link between activity in these 2 regions, associated with motor and emotional simulation, respectively, has remained unknown. Here using psychophysiological interaction and Granger Causality Modeling, we show that activity in the IFO is causally triggered by activity in the IFG, and that this effective connectivity is specific to the IFG. These findings shed new light on the intricate relationship between motor and affective components of emotional empathy.
  
• Del Giudice M, Manera V, Keysers C. "Programmed to learn? The ontogeny of mirror neurons". Developmental Science, 2008, DOI:10.1111/j1467-7687.2008.00783.x.
  We add a new element to the idea that mirror neurons result from Hebbian learning by suggesting that the infant's perceptual-motor system is optimized to provide the brain with the correct input for Hebbian learning. We review evidence that infants have a marked visual preference for hands, show cyclic movement patterns with a frequency that could be in the optimal range for enhanced Hebbian learning, and show synchronized theta EEG in mirror cortical areas during self- observation of grasping. These conditions allow mirror neurons for manual actions to develop quickly and reliably through experiential canalization. This hypothesis provides a plausible pathway for the emergence of mirror neurons that integrates learning with genetic pre-programming, suggesting research on the link between synaptic processes and behaviour in ontogeny.
  
• Kukolja j, Schlapfer TE, Keysers C, Klingmuller D, Maier W, Fink G, Hurlemann R. "Modeling a Negative Response Bias in the Human Amygdala by Noradrenergic-Clucocorticoid Interactions". Journal of Neuroscience, November 26, 2008- 28(48):12868-12876.
  An emerging theme in the neuroscience of emotion is the question how acute stress shapes and distorts social behavior. Acute stress leads to increased levels of norepinephrine (NE) and cortisol (CORT), and seem to influence amygdala responses to social-emotional stimuli. Here we show a causative role of NE-CORT interaction on the way the brain processes the emotions of others: the amygdala, which is not selective for negative emotions under placebo conditions, does respond most to negative emotions while under the influence of NE and CORT.
  
• Valeria Gazzola and Christian Keysers, "The Observation and Execution of Actions Share Motor and Somatosensory Voxels in all tested Subjects: Single Subject Analyses of Unsmoothed fMRI Data". Cerebral Cortex, Oxford Journals, November 19, 2008 .
  This study provides a more detailed description of the location and reliability of shared voxels (sVx), underlining the importance of somatosensory cortices in motor simulation, and proposes a model that extends the original idea of the mirror neuron system to include forward and inverse internal models and somatosensory simulation, distinguishing the MNS from a more general concept of shared Voxels.
  
• Christian Keysers & Luciano Fadiga (Editors, 2008), Special Issue: The Mirror Neuron System. Social Neuroscience, 3/4 .
  This special issue of the journal Social Neuroscience explores some of the most exciting questions surrounding the mirror system: What does the mirror system code? Where is it located? How does it develop? How is the mirror system embedded into the mosaic of circuits that compose our brain? How does the mirror system contribute to communication, language and social interaction? Can the principle of mirror neurons be extended to emotions, sensations and thoughts?
  
• Etzel JA, Gazzola V, Keysers C, "Testing Simulation Theory with Cross-Modal Multivariate Classification of fMRI Data". PlosOne, 2008, 3, Issue 11, e3690.
  Critical to the idea of simulation is the fact that the pattern of activity in premotor and parietal areas is similar between action execution and action perception. Here we show that if we train a pattern classifier to distinguish the sound of hand and mouth actions based on either premotor (BA44/6) or posterior parietal activity, it can tell hand and mouth actions apart while participants execute these two classes of actions. This is therefore the first demonstration that we do not only activate the same brain areas during action execution and perception, but that the pattern of activity is indeed similar enough to support simulation theories.
  
• Jabbi M, Bastiaansen J, Keysers C. "A Common Anterior Insula Representation of Disgust Observation, Experience and Imagination Shows Divergent Functional Connectivity Pathways". PlosOne, 2008, 3, Issue 8, e2939.
  Humans can achieve vivid emotional feeling states in the absence of actual emotional encounters in a myriad of ways. Here we show that the Insula and adjacent frontal operculum, which we have shown previously to respond to the sight and experience of disgust, also becomes active while reading a disgusting story, showing how imagination and text can be plugged into our mirror system for emotions.
  
• Christian Keysers & Luciano Fadiga, "The mirror system: New frontiers". Social Neuroscience, 2008, 3, 193-198.
  As an introduction to a special issue of Social Neuroscience, we review briefly what we know about the mirror system and point out what we believe to be some of the new frontiers of the study of neuroscience: What does the mirror system code? How is the mirror system embedded into the mosaic of circuits that compose our brain? How does the mirror system contribute to communication, language and social interaction? Can the principle of mirror neurons be extended to emotions, sensations and thoughts? Papers using a wide range of methods, including single cell recordings, fMRI, TMS, EEG and psychophysics, collected in the remainder of that special issue, start to give us some impressive answers.
  
• Thioux M, Gazzola V, Keysers C(2008), "Action Understanding: How, What and Why", Current Biology 18:10 R431-R434 .
  This study shows that our action perception system is highly integrated. Visual areas and regions of the mirror neuron system computing “how”and “what”are likely to be as important as mentalizing areas, for understanding “why”. Also a split between “how”and “what” between visual areas and mirror neuron system is artificial. All mirror neurons need visual input and different mirror neurons represent both “how”and “what”.
  
• van der Gaag C, Minderaa R, Keysers C(2007), "Facial expressions: What the mirror neuron system can and cannot tell us", Social Neuroscience 2:179-222 .
  Here we show that observing facial expressions recruits a circuit composed of temporal, parietal and inferior frontal regions that are also active while subjects execute similar facial expressions. The inferior frontal and temporal nodes of this putative mirror system were selectively more active during the viewing of emotional facial expressions than other control stimuli including moving patterns and neutral facial expressions. Such activations were found in 3 separate experiments, including passive viewing, an emotion discrimination and imitation task, suggesting that the putative mirror system for facial expressions is spontaneously active. We failed to find brain areas that reliably discriminate between different facial expressions during both observation and execution.
• Gazzola V, van der Worp H, Mulder T, Wicker B, Rizzolatti G, Keysers C(2007) "Aplasics born without hands mirror the goal of hand actions with their feet", Current Biology 17:1235-1240.
  While it is generally assumed that the sight of actions activates corresponding motor representations in the observer, what 'corresponding' really means remained unclear. Here we show that subjects born without hands and arms activate a combination of effector independent motor representations and motor representations of their foot actions while observing the hand actions of other individuals. These activations are as strong as those of control subjects, suggesting that the mirror system transforms the goal of hand actions into motor representation with similar goals despite differences in effectors. This finding may help understand why non-human primates with mirror systems do not imitate the way in which an action is performed despite their ability to learn to achieve a goal through observation. By showing that mirroring is thus a subjective interpretation of the actions of others in terms of our own, personal motor programs even if these deviate significantly from those observed, we both show the remarkable flexibility of the mirror system and the fact that rather than being an accurate mirror of other people's behaviour, the mirror system performs an interpretation, that may also introduce the potential for misinterpretation. Supplemental Material
• Keysers C and Gazzola V(2007) "Integrating simulation and theory of mind: from self to social cognition". Trends in Cognitive Sciences.
  In this article we propose how simulation and ToM could work together based on the idea that regions in the medial prefrontal cortex that introspect our own states could reflect about the states of others by introspecting the simulated states of others induced by mirror like systems
• van der Gaag C, Minderaa R, Keysers C(2007), "The BOLD signal in the amygdala does not differentiate between dynamic facial expressions", Social Cognitive and Addective Neuroscience .
  In this article we show that the amygdala responds similarly to movies of happy, fearful, disgusted and neutral facial expressions, suggesting that fear selectivity in the amygdala might be an artifact of using static images in previous studies
• Gazzola V, Rizzolatti G, Wicker B, Keysers C (2007), "The Anthropomorphic Brain: the mirror neuron system responds to human and robotic actions", NeuroImage .
  In this article we show that the mirror system responds also to the actions of robots. This suggests that the mirror system matches the goal of an action and not its kinematic details
• Jabbi M., Swart M., Keysers C (2007), "Empathy for positive and negative emotions in the gustatory cortex", Neuroimage 34:1744-53.
  In this article we show that activations in the anterior insula during the observation of the emotions of others correlate with empathy, and that the insula responds both to the sight of positive and negative emotions
• Gazzola V, Aziz-Zadeh L, Keysers C (2006), "Empathy and the somatotopic auditory mirror system in humans", Current Biology 16:1824-9.
  In this article we show that action programs in the premotor cortex become activated both by the vision and the sound of the actions of others in a somatotopical matter. Most importantly, we show a link between empathy and the mirror system: more empathic individuals activate their mirror system more while listening to the actions of others.
• Keysers C, Gazzola V (2006), "Towards a unifying neural theory of social cognition", Progress in Brain Research.
  In this article we provide a framework in which we combine our data on the observation/listening of actions, sensations and emotions with patient lesion data to propose a parsimonious hypothesis on the neural basis of social cognition. We examine the responses of that system to non-human agents and discuss the relationship between ToM and simulation.
• Gallese V, Keysers C, Rizzolatti G (2004), "A unifying view of the basis of social cognition", Trends in Cognitive Sciences 8: 396-403.
  In this article we provide a unifying neural hypothesis on how individuals understand the actions and emotions of others. Our main claim is that the fundamental mechanism at the basis of the experiential understanding of others' actions is the activation of the mirror neuron system. A similar mechanism, but involving the activation of viscero-motor centers, underlies the experiential understanding of the emotions of others.
• Keysers C, Wicker B, Gazzola V, Anton JL, Fogassi L, Gallese V (2004), "A touching sight: SII/PV activation during the observation and experience of touch", Neuron 42: 335-346.Watching the movie scene in which a tarantula crawls on James Bond's chest can make us literally shiver-as if the spider crawled on our own chest. What neural mechanisms are responsible for this "tactile empathy"? The observation of the actions of others activates the premotor cortex normally involved in the execution of the same actions. If a similar mechanism applies to the sight of touch, movies depicting touch should automatically activate the somatosensory cortex of the observer. Here we found using fMRI that the secondary but not the primary somatosensory cortex is activated both when the participants were touched and when they observed someone or something else getting touched by objects. The neural mechanisms enabling our own sensation of touch may therefore be a window also to our understanding of touch.
• Keysers C, Perrett DI (2004), "Demystifying social cognition: a Hebbian perspective", Trends in Cognitive Sciences 8: 501-507.
  For humans and monkeys, understanding the actions of others is central to survival. Here we review the physiological properties of three cortical areas involved in this capacity: the STS, PF and F5. Based on the anatomical connections of these areas, and the Hebbian learning rule, we propose a simple but powerful account of how the monkey brain can learn to understand the actions of others by associating them with self-produced actions, at the same time discriminating its own actions from those of others. As this system appears also to exist in man, this network model can provide a framework for understanding human social perception.
• Keysers C, Kohler E, Umilta MA, Nanetti L, Fogassi L, Gallese V (2003),"Audiovisual mirror neurons and action recognition", Experimental Brain Research 153: 628-636.Many object-related actions can be recognized both by their sound and by their vision. Here we describe a population of neurons in the ventral premotor cortex of the monkey that discharge both when the animal performs a specific action and when it hears or sees the same action performed by another individual. These 'audiovisual mirror neurons' therefore represent actions independently of whether these actions are performed, heard or seen. The magnitude of auditory and visual responses did not differ significantly in half the neurons. A neurometric analysis revealed that based on the response of these neurons, two actions could be discriminated with 97% accuracy.
• Wicker B, Keysers C, Plailly J, Royet JP, Gallese V, Rizzolatti G (2003), "Both of us disgusted in My Insula: The common neural basis of seeing and feeling disgust", Neuron 40: 655-664.
  What neural mechanism underlies the capacity to understand the emotions of others? Does this mechanism involve brain areas normally involved in experiencing the same emotion? We performed an fMRI study in which participants inhaled odorants producing a strong feeling of disgust. The same participants observed video clips showing the emotional facial expression of disgust. Observing such faces and feeling disgust activated the same sites in the anterior insula and to a lesser extent in the anterior cingulate cortex. Thus, as observing hand actions activates the observer's motor representation of that action, observing an emotion activates the neural representation of that emotion. This finding provides a unifying mechanism for understanding the behaviors of others.
• Kohler E, Keysers C, Umilta MA, Fogassi L, Gallese V, Rizzolatti G (2002), "Hearing sounds, understanding actions: Action representation in mirror neurons", Science 297: 846-848.Many object-related actions can be recognized by their sound. We found neurons in monkey premotor cortex that discharge when the animal performs a specific action and when it hears the related sound. Most of the neurons also discharge when the monkey observes the same action. These audiovisual mirror neurons code actions independently of whether these actions are performed, heard, or seen. This discovery in the monkey homolog of Broca's area might shed light on the origin of language: audiovisual mirror neurons code abstract contents - the meaning of actions - and have the auditory access typical of human language to these contents.
• Gallese V, Keysers C (2001), "Mirror neurons: A sensorimotor representation system", Behavioral and Brain Sciences 24: 983.
  Positing the importance of sensorimotor contingencies for perception is by no means denying the presence and importance of representations. Using the evidence of mirror neurons we will show the intrinsic relationship between action control and representation within the logic of forward models.
• Umilta MA, Kohler E, Gallese V, Fogassi L, Fadiga L, Keysers C, Rizzolatti G (2001), "I know what you are doing: A neurophysiological study", Neuron 31: 155-165.
  In the ventral premotor cortex of the macaque monkey, there are neurons that discharge both during the execution of hand actions and during the observation of the same actions made by others (mirror neurons). In the present study, we show that a subset of mirror neurons becomes active during action presentation and also when the final part of the action, crucial in triggering the response in full vision, is hidden and can therefore only be inferred. This implies that the motor representation of an action performed by others can be internally generated in the observer's premotor cortex, even when a visual description of the action is lacking. The present findings support the hypothesis that mirror neuron activation could be at the basis of action recognition.

Key publications on The rapid visual processing of biological stimuli:

• Keysers C, Xiao DK, Foldiak P, Perrett DI (2005), "Out of sight but not out of mind: The neurophysiology of iconic memory in the superior temporal sulcus", Cognitive Neuropsychology 22: 316-332.
  Iconic memory, the short-lasting visual memory of a briefly flashed stimulus, is an important component of most models of visual perception. Here we investigate what physiological mechanisms underlie this capacity by showing rapid serial visual presentation (RSVP) sequences with and without interstimulus gaps to human observers and macaque monkeys. For gaps of up to 93 ms between consecutive images, human observers and neurones in the temporal cortex of macaque monkeys were found to continue processing a stimulus as if it was still present on the screen. The continued firing of neurotics in temporal cortex may therefore underlie iconic memory. Based on these findings, a neurophysiological vision of iconic memory is presented.
• Edwards R, Xiao DK, Keysers C, Foldiak P, Perrett D (2003), "Color sensitivity of cells responsive to complex stimuli in the temporal cortex", Journal of Neurophysiology 90: 1245-1256.
  The inferotemporal (IT) cortex of the monkey lies at the head of the ventral visual pathway and is known to mediate object recognition and discrimination. It is often assumed that color plays a minor role in the recognition of objects and faces because discrimination remains highly accurate with black-and-white images. Furthermore it has been suggested that for rapid presentation and reaction tasks, object classification may be based on a first wave of feedforward visual information, which is coarse and achromatic. The fine detail and color information follows later, allowing similar stimuli to be discriminated. To allow these theories to be tested, this study investigates whether the presence of color affects the response of IT neurons to complex stimuli, such as faces, and whether color information is delayed with respect to information about stimulus form in these cells. Color, achromatic, and false-color versions of effective stimuli were presented using a rapid serial visual presentation paradigm, and responses recorded from single cells in IT of the adult monkey. Achromatic images were found to evoke significantly reduced responses compared with color images in the majority of neurons (70%) tested. Differential activity for achromatic and colored stimuli was evident from response onset with no evidence to support the hypothesis that information about object color is delayed with respect to object form. A negative correlation ( P < 0.01) was found between cell latency and color sensitivity, with the most color-sensitive cells tending to respond earliest. The results of this study suggest a strong role for color in familiar object recognition and provide no evidence to support the idea of a first wave of form processing in the ventral stream based on purely achromatic information.
• Foldiak P, Xiao DK, Keysers C, Edwards R, Perrett DI (2003), "Rapid serial visual presentation for the determination of neural selectivity in area STSa", Roots of Visual Awareness 144: 107-116.
  We show that rapid serial visual presentation (RSVP) in combination with a progressive reduction of the stimulus set is an efficient method for describing the selectivity properties of high-level cortical neurons in single-cell electrophysiological recording experiments. Rapid presentation allows the experimental testing of a significantly larger number of stimuli, which can reduce the subjectivity of the results due to stimulus selection and the lack of sufficient control stimuli. We prove the reliability of the rapid presentation and stimulus reduction methods by repeated experiments and the comparison of different testing conditions. Our results from neurons in area STSa of the macaque temporal cortex provide a well-controlled confirmation for the existence of a population of cells that respond selectively to stimuli containing faces. View tuning properties measured using this method also confirmed earlier results. In addition, we found a population of cells that respond reliably to complex non-face stimuli, though their tuning properties are not obvious.
• Keysers C, Perrett DI (2002), "Visual masking and RSVP reveal neural competition", Trends in Cognitive Sciences 6: 120-125.
  A test visual stimulus is harder to recognize when another stimulus is presented in close temporal vicinity; presenting stimuli in close spatial vicinity of a test stimulus reduces its visibility; presenting a stimulus to one eye can render invisible another stimulus presented to the other eye; and perceiving one interpretation of an ambiguous image prevents the simultaneous perception of other visual interpretations. A single, neurophysiological theory, which may be called 'neural competition' might explain all these phenomena: when two alternative neural visual representations co-exist in the brain,they compete against each other.
• Baker CI, Keysers C, Jellema T, Wicker B, Perrett DI (2001), "Neuronal representation of disappearing and hidden objects in temporal cortex of the macaque", Experimental Brain Research 140: 375-381.
  Neurons in the anterior regions of the banks of the superior temporal sulcus (STSa) of the macaque monkey respond to the sight of biologically significant stimuli such as faces, bodies and their motion. In this study the responses of STSa neurons were recorded during the gradual occlusion of the experimenter and other mobile objects behind screens at distances of 0.5-4 in from the monkeys. The experimenter or other object remained out of sight for 3-15 s before emerging back in to view. We describe a population of neurons (n=33) showing increased activity during the occlusion of objects that was maintained for up to 11 s following complete occlusion (when only the occluder itself was visible). This increase in activity was selective for the position of the occlusion within the testing room. Many neurons showed little or no change in activity prior to occlusion when the object or experimenter was completely in view. By coding for the presence and location of recently occluded objects, these responses may contribute to the perceptual capacity for object permanence.
• Keysers C, Xiao DK, Foldiak P, Perrett DI (2001), "The speed of sight", Journal of Cognitive Neuroscience 13: 90-101.
  Macaque monkeys were presented with continuous rapid serial visual presentation (RSVP) sequences of unrelated naturalistic images at rates of 14-222 msec/image, while neurons that responded selectively to complex patterns (e.g., faces) were recorded in temporal cortex. Stimulus selectivity was preserved for 65% of these neurons even at surprisingly fast presentation rates (14 msec/image or 72 images/sec). Five human subjects were asked to detect or remember images under equivalent conditions, Their performance in both tasks was above chance at all rates (14-111 msec/image). The performance of single neurons was comparable to that of humans anti responded in a similar way to changes in presentation rate, The implications for the role of temporal cortex cells in perception are discussed.

Dissertations of the Social Brain Lab