NeuroImage 59 (2012) 872879

Contents lists available at ScienceDirect

NeuroImage
j o u r n a l h o m e p a g e : w w w. e l s ev i e r. c o m / l o c a t e / y n i m g

Reading the mind's eye: Online detection of visuo-spatial working memory and
visual imagery in the inferior temporal lobe
Carlos M. Hamam a, b, c,, Juan R. Vidal a, b, c, Toms Ossandn a, b, c, Karim Jerbi a, b, c, Sarang S. Dalal a, b, c,
Lorella Minotti d, e, f, Olivier Bertrand a, b, c, Philippe Kahane d, e, f, Jean-Philippe Lachaux a, b, c
a
INSERM U1028, Lyon Neuroscience Research Center, Brain Dynamics and Cognition Team, Lyon, France
b
CNRS UMR5292, Lyon Neuroscience Research Center, Brain Dynamics and Cognition Team, Lyon, France
c
University Lyon 1, Lyon, France
d
Neurology Department, Grenoble University Hospital, Grenoble, France
e
INSERM U836, Grenoble Neuroscience Institute, Grenoble, France
f
University Joseph Fourier, Grenoble, France

a r t i c l e i n f o a b s t r a c t

Article history: Several brain regions involved in visual perception have been shown to also participate in non-sensory
Received 10 April 2011 cognitive processes of visual representations. Here we studied the role of ventral visual pathway areas in
Revised 25 July 2011 visual imagery and working memory. We analyzed intracerebral EEG recordings from the left inferior
Accepted 28 July 2011
temporal lobe of an epileptic patient during working memory tasks and mental imagery. We found that high
Available online 3 August 2011
frequency gamma-band activity (50150 Hz) in the inferior temporal gyrus (ITG) increased with memory
Keywords:
load only during visuo-spatial, but not verbal, working memory. Using a real-time set-up to measure and
Visual mental imagery visualize gamma-band activity online BrainTV we found a systematic activity increase in ITG when the
Working memory patient was visualizing a letter (visual imagery), but not during perception of letters. In contrast, only 7 mm
Word form area more medially, neurons located in the fusiform gyrus exhibited a complete opposite pattern, responding
Intracerebral EEG recordings during verbal working memory retention and letter presentation, but not during imagery or visuo-spatial
Electrocorticography working memory maintenance. Talairach coordinates indicate that the fusiform contact site corresponds to
Gamma-band activity the word form area, suggesting that this region has a role not only in processing letter-strings, but also in
Brain-computer interface
working memory retention of verbal information. We conclude that neural networks supporting imagination
Epilepsy
of a visual element are not necessarily the same as those underlying perception of that element. Additionally,
we present evidence that gamma-band activity in the inferior temporal lobe, can be used as a direct measure
of the efciency of top-down attentional control over visual areas with implications for the development of
novel brain-computer interfaces. Finally, by just reading gamma-band activity in these two recording sites, it
is possible to determine, accurately and in real-time, whether a given memory content is verbal or visuo-
spatial.
2011 Elsevier Inc. All rights reserved.

Introduction
The denition of certain brain regions as perceptual, motor or
associative, has been based mostly upon functional criteria. For
example, regions of the human brain showing selective responses to
visual stimuli would usually be labeled as visual areas. However, the
human visual system is known to be involved not only in perception
of environmental stimuli, but also in mental visual imagery (Kreiman
Abbreviations: BCI, brain-computer interface; MVIS, visuo-spatial working memory
task; MVEB, verbal working memory task; ITG, inferior temporal gyrus; TMS,
transcranial magnetic stimulation; WFA, word form area.
Corresponding author at: INSERM U1028, CNRS UMR5292, Lyon Neuroscience
Research Center, Brain Dynamics and Cognition Team, 95, Bd. Pinel, Centre Hospitalier
Le Vinatier (Bt. 452), F-69500, Bron, France. Fax: + 33 4 72 13 89 01.
E-mail address: carlos.hamame@inserm.fr (C.M. Hamam).
et al., 2002; Le Bihan et al., 1993) and in working memory retention of
visual information (Awh and Jonides, 2001; Haxby et al., 1994;
Serences et al., 2009).
Visual imagery refers to the maintenance of a stable conscious visual
representation independent of visual inputs (Moulton and Kosslyn,
2009), very much like the maintenance of a visual event in working
memory, a theoretical construct that refers to memory stores and
executive processes supporting our capacity to keep in mind
information no longer available in the environment (Baddeley, 1992;
Baddeley and Hitch, 1974). Indeed, behavioral studies have shown that
increasing working memory load impairs the capacity of subjects to
mentally create and/or manipulate visual images (Atkinson and Shiffrin,
1968; Baddeley and Andrade, 2000; Gyselinck et al., 2007), suggesting
that visual imagery and visual working memory share common
cognitive and neural resources. These ndings are in agreement with
the sensory recruitment hypothesis, which states that information to

1053-8119/$ see front matter 2011 Elsevier Inc. All rights reserved.
doi:10.1016/j.neuroimage.2011.07.087
 C.M. Hamam et al. / NeuroImage 59 (2012) 872879
be recalled is temporarily stored in the sustained activity of brain
regions involved in perception of that information (Awh and Jonides,
2001; Serences et al., 2009). In fact, the disruption of neural activity in
the early visual cortex with single TMS pulses affects performance in
both visual imagery and short-term memory tasks (Cattaneo et al.,
2009). Moreover, apart from the sensory recruitment hypothesis,
several authors have proposed that visual imagery and visuo-spatial
working memory might rely on the same large-scale network
connecting the frontal executive system with occipito-temporal visual
areas, possibly via the parietal lobes (Cabeza and Nyberg, 2000;
Ranganath and D'Esposito, 2005; Zimmer, 2008). Nevertheless, how
and to what extent visual brain regions participate in imagery and
working memory remains unknown.
In the present study, we focus primarily on the neural implemen-
tation of visual imagery and working memory maintenance in the
inferior temporal lobe. We hypothesized that visual areas which activity
increases with memory-load should also be active during visual
imagery. We tested this hypothesis in an epileptic patient implanted
with intracerebral electrodes for clinical reasons. We recorded the
patient during a simple visuo-spatial working memory task and
searched for cortical sites active during memory maintenance. We
then monitored the activity of those sites online and found an
instantaneous and systematic increase in gamma-band neural activity
(50150 Hz) when the patient voluntarily engaged in visual imagery.
Methods
Participant
Subject was a right-handed 16 year old female undergoing
resective neurosurgical treatment for drug-resistant epilepsy. Neuro-
psychological examination was normal, with intellectual abilities and
memory in the normal range. A thickening of the right superior
temporal sulcus was suspected, though repeated MRI proved
inconclusive. 18-FDG-PET demonstrated a right posterior temporo-
perisylvian hypometabolism. Interictal EEG mainly showed right
temporal slow waves. In order to precisely identify the epileptic focus,
the patient was stereotactically implanted with 15 multi-lead depth-
EEG electrodes. The implantation strategy favored the posterior
temporal cortex of both sides with 9 electrodes on the right side,
and 6 electrodes on the left, chosen solely by clinical criterion. The
patient was electrophysiologically monitored for a three-week period
by a team of clinicians at Grenoble University Hospital (see Kahane et
al., 2004 for method description). During this period, parents and
subject provided informed consent for participation in the study.
Research was approved by the Regional Ethical Committee (CPP Sud-
Est V). After clinical examination, the epileptogenic area proved to be
right temporo-basal. Fig. 1 depicts the patient's MRI showing location
of the electrode contacts of interest for this study, f6 and f8.
Intracerebral recordings
A multi-channel video-EEG acquisition and monitoring system
(Micromed, Treviso, Italy) was used to record neural activity from 15
semi-rigid uni-linear electrode arrays. Each array comprises from 6 to
15 contact sites separated by 3.5 mm (Dixi, Besanon, France)
bilaterally spanning temporal and occipital lobes. The intracerebral
EEG signal was acquired at a 512 Hz sampling rate and bandpass
ltered between 0.1 and 200 Hz. A contact site placed in white matter
was chosen as monopolar reference for the acquisition. However, and
in order to increase spatial resolution, a bipolar montage (i.e.,
difference between adjacent contacts) was applied for online analysis
and display. After real-time exploration of gamma-band response
(50150 Hz) in different cognitive and behavioral conditions (Jerbi et
al., 2009b; Lachaux et al., 2007a), a left fusiform (f6) and an inferior
temporal gyrus (ITG) (f8) recording sites were selected for further
873
Fig. 1. Patient preoperative MRI and electrode positions. A) Coronal and B) horizontal
view of MRI showing recording sites f6 and f8. C) Sagital, view in the preoperative MRI
highlighting localization of contact site f6 in the fusiform gyrus, Tailarach coordinates:
42, 55, 11. D) As in C, but highlighting localization of contact site f8 in the
inferior temporal gyrus, Tailarach coordinates: 49, 55, 11.
study, due to their selective response to load manipulations during
working memory tasks. Activity in these recording sites was veried
to be free from epileptiform activity, eye movements, and electro-
myographic artifacts. Structural MRIs were acquired before implan-
tation, post-implantation (i.e., with electrodes in place) and after
resective surgery. The post-implantation MRI was analyzed with
custom Matlab (Mathworks, Natick, MA, USA) routines to precisely
localize electrodes and contact sites. Talairach and Tournoux (1988)
atlas was used to identify and report recorded brain regions as
standard coordinates.
Ofine time-frequency analysis
Time courses of gamma-band (50150 Hz) amplitude (i.e.,
envelope or band-limited power) were calculated ofine for the
electrophysiological signal acquired during visuo-spatial and verbal
working memory tasks (see below for an explanation of procedures).
Continuous data was rst band-pass ltered between 50 and 150 Hz
in steps of 10 Hz to obtain 10 consecutive frequency bands from 50
60 Hz to 140150 Hz. Afterward, the envelope of each of these band-
passed ltered signals was calculated using the standard Hilbert
transform (c.f. Le Van Quyen et al., 2001). To normalize across
frequency bands, amplitude values for each 10 Hz band were divided
by their mean throughout the entire session and then multiplied by
100. The resulting values are thus presented as a percentage of
increase or decrease in activity relative to the entire recording period.
Finally, the 10 envelopes were averaged across frequency yielding a
single time-series for broad-high-gamma (50150 Hz).
Event-related modulations of gamma envelope were obtained by
averaging its amplitude across trials for a given experimental
condition. For visualization purposes, envelopes were smoothed
using a sliding-window moving average of 250 ms and then displayed
relative to a common pre-stimulus baseline of 400 to 100 ms.
All spectral analyses, display and related statistical tests were
performed using Matlab custom routines.
Real-time visualization of gamma-band activity
A real-time visualization system, dubbed BrainTV (detailed descrip-
tion in Lachaux et al., 2007a, 2007b), was used for online analysis and
874 C.M. Hamam et al. / NeuroImage 59 (2012) 872879
display of instantaneous power variations in gamma activity. Signal
amplitude was displayed as an increase or decrease (activation score) in
spectral power relative to baseline. The only difference between online
and ofine time-frequency analysis was that normalization across
frequency bands was done considering the last two minutes of
recording for online computation, while in the ofine analysis the
entire recording period was preferred. A computer screen was used to
display the gamma-band envelope measured during the last ten
seconds of recording (refreshed every 250 ms).
Experimental procedures
Visual working memory task (MVIS)
A delayed matched to sample task was used to evaluate brain
dynamics during retention of different loads of visuo-spatial infor-
mation. Each of the 48 trials started with the presentation of a
1500 ms central xation cross followed by a 4-by-4 grid with two,
four or six dots (top-right panel in Fig. 2). This 1500 ms probe-
stimulus was replaced by an empty grid during a retention period of
3000 ms. The patient was instructed to remember the spatial
arrangement of the dots on the screen until sample presentation.
The sample consisted of a dot shortly (1500 ms) presented on one of
the 16 spaces of the grid. The subject was required to respond if the
sample dot was also present in the probe-stimulus pressing one of
two buttons in a game-pad with her right index nger. Memory load
was manipulated by changing the number of dot positions to retain
two for low, four for medium, and six for high load. Additionally, 15
trials consisted of a control condition where one of the dots in the
probe was highlighted meaning that it was the only position to retain.
All conditions were randomly assigned across trials.
Verbal working memory task (MVEB)
A very similar delayed matched to sample task was used to
evaluate verbal working memory. The only difference was that the
probe consisted of a horizontal, central and linear array of two, four, or
six letters (top-right panel in Fig. 3). The participant was asked to
Fig. 2. Gamma-band activity during visuo-spatial working memory task. Superior right corner of the gure depicts an example of probe stimulus. Each row represents a single-trial of
gamma activity during sample, maintenance and probe periods recorded from electrode A) f8 at the ITG and B) f6 at the fusiform gyrus. Gamma amplitude during a 2-second
window centered in the maintenance period from recording sites C) f8 and D) f6.
remember the identity of the letters and respond after the retention
period if the presented sample matched the probe. The number of
trials per condition was the same as in MVIS.
Visual imagery
The BrainTV environment (Lachaux et al., 2007a, 2007b) allows
the experimenter to quickly test the reactivity of any given brain
region as the participant engages in simple cognitive procedures, such
as mentally exploring her own house, or shifting her attention
between internal representations and external sensory inputs. In the
example of interest for this study, the participant was initially shown
a blank sheet of paper and was asked to mentally visualize letters
drawn on it, as vividly as possible (see Supplementary Movie). The
reactivity of the ITG was then tested while the participant was asked
to visually imagine real-life objects. The quality of the mental imagery
was assessed by interviewing the patient about how she performed
the task, what was she actually imagining and how difcult it was for
her to follow the instructions.
Statistical analysis
The Wilcoxon signed rank test was performed to reveal signicant
differences in gamma band activity across memory loads and control
condition in both working memory tasks. Alpha value for single
comparison was 0.01, while the Bonferroni correction was applied for
multiple comparisons yielding a more conservative alpha of 0.005.
Results
The maintenance period in the visuo-spatial working memory task
was characterized by increased gamma activity for high loads in the
inferior temporal gyrus, but not in the fusiform gyrus. More specically,
a recording site located in the ITG (T3, electrode contact f8, Talairach
coordinates: 49, 55, 11) showed a strong increase in gamma-
band energy after presentation of the sample array, which was sustained
throughout the 3-second maintenance interval in medium (load4) and
 C.M. Hamam et al. / NeuroImage 59 (2012) 872879 875

Fig. 3. Gamma-band activity during verbal working memory task. Superior right corner of the gure depicts an example of probe stimulus. Each row represents a single-trial of
gamma activity during task from recording site A) f8 at the ITG and B) f6 at the fusiform gyrus. Gamma amplitude during a 2-second window centered in the maintenance period
from recording sites C) f8 and D) f6.

high (load6) load conditions (Fig. 2). In contrast, the effect was absent in stimulation, we decided to test the reactivity of the recorded visual areas
either control or low load (load2) conditions. When measuring the while the patient was asked to produce vivid images of single letters
gamma-band energy specically within the maintenance period (a 2 while xating on a white sheet of paper (Supplementary Movie 1).
second window centered in that interval), we found that all trials in the The result was a systematic and unambiguous energy increase of
high load condition (12 out of 12) and all but one (11 out of 12) in the gamma-band activity sustained throughout the imagery effort at ITG,
medium load condition had energy values higher than the highest value but not at the fusiform site (Fig. 6). Interestingly, the same effect was
measured in the same interval during the control (High-loadN Control, found when the patient was instructed to imagine other more complex
Medium-loadN Control, p b 0.0001). In contrast, we found no such effect real-life objects. Therefore, the effect may have been not specic to
in the adjacent more medial fusiform gyrus (f6, Talairach coordinates: letters, but related to a visual imagery effort.
42, 55, 11).
An opposite pattern of response was found during the verbal Discussion
working memory task (Fig. 3). There was no increase in gamma-band
energy during memory maintenance or encoding at ITG (f8). In Our guiding intuition was that visual mental imagery and visuo-
contrast, the fusiform site (f6) revealed an increased gamma activity spatial working memory shared a neural substrate. Indeed, we show
with verbal memory load (medium load N control, high load N control, direct evidence that neural populations of human ITG are selectively
p b 0.002). Here again, if we take the gamma energy measured during active during both visuo-spatial working memory and visual mental
the maintenance period, from the fusiform gyrus, we found that all imagery. ITG gamma amplitude modulations during MVIS and visual
values from the high load condition were above the maximal value
observed in the control. In summary, high load could be distinguished
from the control condition on a single-trial basis in ITG during MVIS,
and in the fusiform during MVEB. Furthermore, a comparison
between maintenance values measured in ITG during MVEB and
MVIS revealed that a very strong dissociation between memory types
(Fig. 4): in the high load condition, 11 out of 12 values were higher
than the highest maintenance value measured in the MVEB condition.
In the medium load condition, this proportion was 10/12. This
corresponds to a 92% (resp. 83%) dissociation between the MVEB and
MVIS condition, a direct measure of the ability to distinguish between
verbal and visuo-spatial working memory in ITG.
Inferior temporal and fusiform gyri differed also markedly in their
gamma-band responses to visual stimulation (Fig. 5). In the fusiform
gyrus, we found that letter-strings (pseudo-words and consonant
strings), but not pictures of scenes or faces, induces a strong increase
Fig. 4. Amplitude of gamma activity from the ITG dissociates between visuospatial
of activity. In contrast, we failed to nd any gamma response to letter- (blue) and verbal (red) content of working memory maintenance. Gamma amplitude is
strings, faces or natural scenes at the ITG recording site. Since ITG displayed as a function of difculty in the working memory tasks, i.e. control, low,
responded to visuo-spatial memory maintenance, but not to visual medium and high load conditions.
876 C.M. Hamam et al. / NeuroImage 59 (2012) 872879

Fig. 5. Inferior temporal (red) and fusiform (blue) gyrus selective visual responses to faces, natural scenes, words or pseudowords. Note that while ITG does not exhibit a particularly
strong response to the selected categories, the recording site located in the fusiform gyrus showed a largely selective visual response to written-like stimuli.

imagery were sufciently clear to be readily detected in real-time.
Furthermore, we found that ITG did not respond during visual
stimulation or verbal working memory. This dissociation is in fact
sufciently clear to decode the type of material being maintained in
working memory (verbal vs. visuo-spatial) on a single trial basis. The
implications of these results regarding the neurophysiology and
cognitive models of visual imagery and working memory, together
with possible novel BCIs are discussed below.
A common neural substrate for imagery and working memory maintenance
in the inferior temporal lobe
Our ndings provide novel insights into the neurophysiology of
mental imagery and working memory. Given that both phenomena
can be described as the voluntary manipulation of a mental visual
representation (Ranganath and D'Esposito, 2005), they might at least
partially share some of their neural mechanisms as well. For instance,
it has been observed that both working memory (Howard et al., 2003;
Mainy et al., 2007; Tallon-Baudry et al., 2005) and visual imagery
(Kreiman et al., 2002) modulate neural activity in visual areas of the
medial temporal lobe. Moreover, there is also a body of evidence for a
distributed fronto-temporal network sustaining mental imagery
(Ishai et al., 2000; Zimmer, 2008) and thus revealing further
functional and anatomical overlap. Here we show for the rst time
direct electrophysiological evidence that visuo-spatial working
memory and visual imagery activate the same neural population in
the ventral visual pathway. Indeed, the voluntary stabilization of a
visual mental image can be seen as functionally equivalent to its
maintenance on the visual sketchpad of working memory. This also
ts well with the patient's ofine report that she performed our visuo-

Fig. 6. ITG (left, red) selectively increases gamma activity amplitude during visual imagery, while fusiform gyrus (right, blue) remains around baseline level. Letters depicted next to each
plot represent the tokens that subject was instructed to imagine.
 C.M. Hamam et al. / NeuroImage 59 (2012) 872879
spatial working memory task by mentally stabilizing the dot-array
picture.
While memory-related activity in the visual cortex is a fairly
common observation, it is most often found in regions responding
also to external visual inputs. Our recording site in the ITG did not reveal
any visual response when the patient was passively presented with
letter strings, faces or landscapes (Fig. 5). This observation could almost
by itself dene that brain region of the ITG as non-visual. Yet, it was
active during the visual presentation of dot arrays during the working
memory task. Our interpretation is that this brain region selectively
responds to task-relevant visual stimuli and is almost exclusively driven
by a top-down process. Stimuli shown passively did not require such a
process, thus yielding no clear visual response. In other words, our
results suggest that there are regions in the ventral visual pathway that
do not respond to visual stimuli, unless those stimuli are task-relevant.
This means that passive visual stimulation activate some and not all of
the areas in the visual pathway. Further research could conrm if there
is a top-down/bottom-up functional organization of lateral and medial
portions of the ventral visual pathway accounting for this nding.
A subtle anatomical dissociation between verbal and visuo-spatial working
memory networks in the inferior temporal lobe
The observed dissociation between verbal and visuo-spatial working
memory in the inferior temporal lobe is consistent with the classical
multicomponent model rst posed by Baddeley and Hitch (1974).
According to that model (see also Baddeley, 1992, 2010), visual
information is temporarily sustained in a visuo-spatial sketch-pad,
which is functionally dissociated from the phonological loop that
momentarily stores verbal information and usually involves covert
speech. Further research on the role of attention, have shown that
subjects who can recall a larger number of objects from a visuo-spatial
array are also more efcient in ltering-out irrelevant information
(Conway et al., 2001; Olesen et al., 2007; Vogel et al., 2005; Zanto and
Gazzaley, 2009), suggesting that attentional processes may determine
storage capability. Similar results have been obtained with verbal
material only when precluding covert-speech rehearsal during the
maintenance period. This suggests that rehearsal prevention forces
subjects to rely on more attention-demanding strategies (Cowan, 2008;
Cowan and Morey, 2006; Morey and Cowan, 2005). In contrast to the
multicomponent model of working memory, those ndings have been
interpreted as pointing to a modality-independent capability that does
not depend on pre-dened stores or buffers (Cowan, 2008; Cowan and
Morey, 2007).
Here, we found that gamma-band activity in the ITG proportion-
ally increases with visuo-spatial working memory load, suggesting
that ITG participates of the proposed visual sketchpad. Similarly, in a
previous study by our team, intracerebral EEG were used to localize a
set of perisylvian cortical areas involved in the phonological loop
(Mainy et al., 2007; Vidal et al., 2010). Regardless of its modality,
maintenance of information in working memory most likely involves
some sort of mental imagery. Indeed, we have shown that ITG had a
sustained gamma response not only during visuo-spatial working
memory, but also during visual imagery. Accordingly, a shared neural
substrate between working memory and imagery may very well
explain the pattern of gamma-band activity that we found in the ITG.
We have also shown that in the adjacent fusiform gyrus, there was an
increase in gamma amplitude during presentation, maintenance, but
not imagery of verbal information (letter strings). Interestingly, this
recording site was located (Talairach coordinates: 42, 55, 11)
very closely if not inside the visual word-form area (WFA), a part of
the visual cortex that is selectively involved in processing letter-
strings (Cohen et al., 2000; Dehaene and Cohen, 2011; Mainy et al.,
2008; Price and Devlin, 2011). The fact that the WFA increases its
activity during verbal working memory, but not during letter imagery,
suggests that its role in the phonological loop is not to create or
877
sustain a mental image of a letter, but to identify or code a visual
element as a letter. Since in our verbal working memory task it was
necessary to remember a given string of letters, rehearsal of the entire
sequence would have to involve the repeated identication of its
components, thus yielding a sustained activation of the WFA.
In other words, since ITG was involved in both visuo-spatial
working memory and visual imagery, we propose that the strategy of
maintenance during the visuo-spatial task was the actual retention of
an imagined dot-array. On the other hand, the WFA showed a steady
and increased GBA during verbal working memory, but not during
visual imagery. Therefore, we propose that during the verbal task,
successful maintenance of the letter-string was accomplished not
through visual imagery, but by means of covert speech or verbal
rehearsal, which could explain the observed recurrent activity in the
WFA during memory maintenance, but not during imagery.
Although the multicomponent model of working memory has
been one of the most fruitful theoretical constructs in cognitive
neuroscience, a number of critics have raised, specially regarding how
xed the proposed storages are (Cowan and Morey, 2006), the role of
attention (Awh and Jonides, 2001; Cowan and Morey, 2006) and the
possibility of having a modality-independent capacity for maintaining
a given item in working memory (Cowan and Morey, 2006). Our
results do not represent conclusive evidence regarding the exposed
issues of the multicomponent model, specially because of our limited
sample size (one patient). Even so, we think our study shows that
models of working memory could greatly benet from taking into
account the complex relationship between working memory, reten-
tion strategy and the activity in perceptual areas.
Intracerebral high-frequency gamma
We have found that high and broad gamma activity (50150 Hz)
from the inferior temporal lobe is actively involved in working
memory and imagery. It has been proposed that neural synchroniza-
tion in the gamma-band (3060 Hz) is a fundamental physiological
mechanism for attention and working memory (Jensen et al., 2007).
One inuential hypothesis states that gamma oscillations can organize
neural activity in time by modulating spike probability during
different phases of its cycle (Fries et al., 2007). However, our data is
not sufciently precise to demonstrate that our broad-band activity
(50150 Hz) corresponds to that synchronization process. An alter-
native view is that high-frequency broad-band gamma may be the
trace of an increase of neural ring rate (Dalal et al., 2011; Manning et
al., 2009; Miller, 2010). In either case, we have found a reliable neural
marker of visual imagery and visuo-spatial memory maintenance in
the inferior temporal lobe and more evidence for a functional role of
high frequency gamma in working memory and the visual system.
Visual imagery and BCI
Brain Computer Interfaces (BCI) are widely known for transforming
thoughts into actions. BCI systems take a recorded brain signal, which
can be voluntarily modulated by motor intention (e.g. Daly and
Wolpaw, 2008; Hochberg et al., 2006; Jerbi et al., 2009a; Kim et al.,
2008; Leuthardt et al., 2004; Schalk et al., 2007), attention (van Gerven
and Jensen, 2009; van Gerven et al., 2009; Vansteensel et al., 2010) or
imagery, and use it to control a computer or robotic device (Birbaumer
et al., 2008; Jerbi et al., 2010). Real-time recognition of a neural signature
of visual mental imagery serves as critical proof of concept for
implementing BCIs controlled by visual thoughts. An advantageous
aspect of such device would be that visual mental imagery does not
require training and can be exercised by patients as a natural cognitive
strategy for controlling a computer or a robotic machine. The next step,
for visual imagery to provide an alternative/supplementary approach to
existing invasive BCI techniques that use motor-related activity, would
be to close the loop between user and operated device. For the time
878 C.M. Hamam et al. / NeuroImage 59 (2012) 872879
being, this can only be achieved invasively using depth electrodes
implanted into the infero-temporal cortex. As much as it is more
desirable to avoid surgery risks by using non-invasive methods, there is
increasing agreement that intracranial recordings of small neural
populations provide the most stable, clean, and effective input signal
for BCI (Moran, 2010). However, since data was acquired from a single
patient and fully operational BCI testing (closed loop between brain and
computer) was not performed, we are not able to currently deliver the
application. Nonetheless, we claim that the presented evidence points to
the feasibility of BCIs controlled by visual imagery and that further BCI
development could largely benet from researching how motor-related
signals can be complemented, or even replaced, by visual imagery-
related activity from the inferior temporal lobe.
Conclusions
We found selective increases in gamma-band (50150 Hz) activity
of the ITG during visual imagery and visuo-spatial working memory
maintenance. Measures from this and the adjacent fusiform gyrus
revealed a clear distinction between areas involved in the phonolog-
ical loop or visual sketchpad, suggesting a novel role of the WFA in
working memory. Moreover, our results suggest that lateral and
medial portions of the inferior temporal visual cortex may selectively
respond to top-down or bottom-up modulations of activity in the
gamma range. Finally, we found that neural activity related to
voluntary cognitive operations, such as visual imagery and working
memory maintenance, can be recorded in real-time, in sites which
do not respond to visual stimulation, thus opening a promising
possibility for novel BCIs.
Supplementary materials related to this article can be found online
at doi:10.1016/j.neuroimage.2011.07.087.
Acknowledgments
We would like to thank the patient who participated in the study
and the invaluable support of the staff from Grenoble University
Hospital Epilepsy Unit. Funding was provided by: Fondation Fyssen to
C.M.H., Fellowship from Ministre de l'Education Nationale et la
Recherche (France) to T.O., Fondation pour la Recherche Medicale
(FRM), the Marie Curie Fellowship (FP7 Grant PIIF-GA-2008-221097)
to S.S.D., BrainSync FP7 European Project (Grant HEALTH-F2-2008-
200728), ANR OPENVIBE2 to J.R.V. and J.P.L.
References
Atkinson, R.C., Shiffrin, R.M., 1968. Human memory: a proposed system and its control
processes. In: Spence, K.W., Spence, J.T. (Eds.), The Psychology of Learning and
Motivation: Advances in Research and Theory, vol. 2. Academic Press, New York,
pp. 89195.
Awh, E., Jonides, J., 2001. Overlapping mechanisms of attention and spatial working
memory. Trends Cogn. Sci. 5, 119126.
Baddeley, A.D., 1992. Working memory. Science 255, 556559.
Baddeley, A., 2010. Working memory. Curr. Biol. 20, R136R140.
Baddeley, A.D., Andrade, J., 2000. Working memory and the vividness of imagery. J. Exp.
Psychol. Gen. 129, 126145.
Baddeley, A.D., Hitch, G., 1974. Working memory. In: Bower, G.H. (Ed.), The psychology
of learning and motivation: Advances in research and theory, vol. 8. Academic
Press, New York, pp. 4789.
Birbaumer, N., Murguialday, A.R., Cohen, L., 2008. Brain-computer interface in paralysis.
Curr. Opin. Neurol. 21, 634638.
Cabeza, R., Nyberg, L., 2000. Imaging cognition II: an empirical review of 275 PET and
fMRI studies. J. Cogn. Neurosci. 12, 147.
Cattaneo, Z., Vecchi, T., Pascual-Leone, A., Silvanto, J., 2009. Contrasting early visual
cortical activation states causally involved in visual imagery and short-term
memory. Eur. J. Neurosci. 30, 13931400.
Cohen, L., Dehaene, S., Naccache, L., Lehricy, S., Dehaene-Lambertz, G., Hnaff, M.A.,
Michel, F., 2000. The visual word form area: spatial and temporal characterization
of reading in normal subjects and posterior split-brain patients. Brain 123,
291307.
Conway, A.R.A., Cowan, N., Bunting, M.F., 2001. The cocktail party phenomenon
revisited: the importance of working memory capacity. Psychon. Bull. Rev. 8,
331335.
Cowan, N., 2008. What are the differences between long-term, short-term, and working
memory? Prog. Brain Res. 169, 323338.
Cowan, C., Morey, C.C., 2006. Visual working memory depends on attentional ltering.
Trends Cogn. Sci. 10, 139141.
Cowan, N., Morey, C.C., 2007. How can dual-task working memory retention limits be
investigated? Psychol. Sci. 18, 686688.
Dalal, S.S., Vidal, J.R., Hamam, C.M., Ossandon, T., Bertrand, O., Lachaux, J.P., Jerbi, K.,
2011. Spanning the rich spectrum of the human brain: slow waves to gamma and
beyond. Brain Struct. Funct. 216, 7784.
Daly, J.J., Wolpaw, J.R., 2008. Brain-computer interfaces in neurological rehabilitation.
Lancet Neurol. 7, 10321043.
Dehaene, S., Cohen, L., 2011. The unique role of the visual word form area in reading.
Trends Cogn. Sci. 15, 254262.
Fries, P., Nikolic, D., Singer, W., 2007. The gamma cycle. Trends Neurosci. 30, 309316.
Gyselinck, V., De Beni, R., Pazzaglia, F., Meneghetti, C., Mondoloni, A., 2007. Working
memory components and imagery instructions in the elaboration of a spatial
mental model. Psychol. Res. 71, 373382.
Haxby, J.V., Horwitz, B., Ungerleider, L.G., Maisog, J.M., Pietrini, P., Grady, C.L., 1994. The
functional organization of human extrastriate cortex: a PET-rCBF study of selective
attention to faces and locations. J. Neurosci. 14, 63366353.
Hochberg, L.R., Serruya, M.D., Friehs, G.M., Mukand, J.A., Saleh, M., Caplan, A.H., Branner,
A., Chen, D., Penn, R.D., Donoghue, J.P., 2006. Neuronal ensemble control of
prosthetic devices by a human with tetraplegia. Nature 442, 164171.
Howard, M.W., Rizzuto, D.S., Caplan, J.B., Madsen, J.R., Lisman, J., Aschenbrenner-
Scheibe, R.A., Schulze-Bonhage, A., Kahana, M.J., 2003. gamma oscillations correlate
with working memory load in humans. Cereb. Cortex 13, 13691374.
Ishai, A., Ungerleider, L.G., Haxby, J.V., 2000. Distributed neural systems for the
generation of visual images. Neuron 28, 979990.
Jensen, O., Kaiser, J., Lachaux, J.P., 2007. Human gamma-frequency oscillations
associated with attention and memory. Trends Neurosci. 30, 317324.
Jerbi, K., Freyermuth, S., Minotti, L., Kahane, P., Berthoz, A., Lachaux, J.P., 2009a.
Watching BrainTV and playing brain ball: exploring novel BCI strategies using real-
time analysis of human intracranial data. Int. Rev. Neurobiol. 86, 159168.
Jerbi, K., Ossandon, T., Hamam, C.M., Senova, S., Dalal, S.S., Jung, J., Minotti, L., Bertrand,
O., Berthoz, A., Kahane, P., Lachaux, J.P., 2009b. Task-related brain dynamics from an
intracerebral perspective: Review and implications for surface EEG and MEG. Hum.
Brain Mapp. 30, 17581771.
Jerbi, K., Vidal, J.R., Mattout, J., Maby, E., Bauchet-Lecaignard, F., Ossandon, T.,
Hamam, C.M., Dalal, S.S., Lachaux, J.P., Leahy, R.M., Baillet, S., Garnero, L.,
Delpuech, C., Bertrand, O., 2010. Inferring hand movement kinematics with MEG
and EEG: from brain-machine interfaces to motor rehabilitation. IRBM Biomed.
Eng. Res. 32, 818.
Kahane, P., Minotti, L., Hoffmann, D., Lachaux, J.P., Ryvlin, P., 2004. Invasive EEG in the
denition of the seizure onset zone: depth electrodes. In: Rosenow, F., Lders, H.O.
(Eds.), Presurgical assessment of the epilepsies with clinical neurophysiology and
functional imaging. : Handbook of Clinical Neurophysiology, vol.3. Elsevier BV,
Amsterdam, pp. 109133.
Kim, S.P., Simeral, J.D., Hochberg, L.R., Donoghue, J.P., Black, M.J., 2008. Neural control of
computer cursor velocity by decoding motor cortical spiking activity in humans
with tetraplegia. J. Neural Eng. 5, 455476.
Kreiman, G., Koch, C., Fried, I., 2002. Imagery neurons in the human brain. Nature 408,
357361.
Lachaux, J.P., Jerbi, K., Bertrand, O., Minotti, L., Hoffmann, D., Schoendorff, B., Kahane, P.,
2007a. A blueprint for real-time functional mapping via human intracranial
recordings. PLoS One 10, e1094.
Lachaux, J.P., Jerbi, K., Bertrand, O., Minotti, L., Hoffmann, D., Schoendorff, B., Kahane, P.,
2007b. BrainTV: a novel approach for online mapping of human brain functions.
Biol. Res. 40, 401413.
Le Bihan, D., Turner, R., Zefro, T.A., Cunod, C.A., Jezzard, P., Bonnerot, V., 1993.
Activation of human primary visual cortex during visual recall: a magnetic
resonance study. PNAS 90, 1180211805.
Le Van Quyen, M., Foucher, J., Lachaux, J.P., Rodriguez, E., Lutz, A., Martinerie, J., Varela,
F.J., 2001. Comparison of Hilbert transform and wavelet methods for the analysis of
neuronal synchrony. J. Neurosci. Methods 111, 8398.
Leuthardt, E.C., Schalk, G., Wolpaw, J.R., Ojemann, J.G., Moran, D.W., 2004. A brain-
computer interface using electrocorticographic signals in humans. J. Neural Eng. 1,
6371.
Mainy, N., Kahane, P., Minotti, L., Hoffmann, D., Bertrand, O., Lachaux, J.P., 2007. Neural
correlates of consolidation in working memory. Hum. Brain Mapp. 28, 183193.
Mainy, N., Jung, J., Baciu, M., Kahane, P., Schoendorff, B., Minotti, L., Hoffmann, D.,
Bertrand, O., Lachaux, J.P., 2008. Cortical dynamics of word recognition. Hum. Brain
Mapp. 29, 12151230.
Manning, J.R., Jacobs, J., Fried, I., Kahana, M.J., 2009. Broadband shifts in local eld
potential power spectra are correlated with single-neuron spiking in humans.
J. Neurosci. 29, 1361313620.
Miller, K.J., 2010. Broadband spectral change: evidence for a macroscale correlate of
population ring rate? J. Neurosci. 30, 64776479.
Moran, D., 2010. Evolution of brain-computer interface: action potentials, local eld
potentials and electrocorticograms. Curr. Opin. Neurobiol. 20, 741745.
Morey, C.C., Cowan, N., 2005. When do visual and verbal memories conict? The
importance of working-memory load retrieval. J. Exp. Psychol. Learn. Mem. Cogn.
31, 703713.
Moulton, S.T., Kosslyn, S.M., 2009. Imagining predictions: mental imagery as mental
emulation. Philos. Trans. R. Soc. Lond. B Biol. Sci. 364, 12731280.
Olesen, P.J., Macoveanu, J., Tegnr, J., Klinberg, T., 2007. Brain activity related to working
memory and distraction in children and adults. Cereb. Cortex 17, 10471054.
 C.M. Hamam et al. / NeuroImage 59 (2012) 872879
Price, C.J., Devlin, J.D., 2011. The interactive account of ventral occipitotemporal
contributions to reading. Trends Cogn. Sci. 15, 246253.
Ranganath, C., D'Esposito, M., 2005. Directing the mind's eye: prefrontal, inferior and
medial temporal mechanisms for visual working memory. Curr. Opin. Neurobiol. 15,
175182.
Schalk, G., Kubnek, J., Miller, K.J., Anderson, N.R., Leuthardt, E.C., Ojemann, J.G.,
Limbrick, D., Moran, D., Gerhardt, L.A., Wolpaw, J.R., 2007. Decoding two-
dimensional movement trajectories using electrocorticographic signals in humans.
J. Neural Eng. 4, 264275.
Serences, J.T., Ester, E.F., Vogel, E.K., Awh, E., 2009. Stimulus-specic delay activity in
human primary visual cortex. Psychol. Sci. 20, 207214.
Talairach, J., Tournoux, P., 1988. Co-planar stereotaxic atlas of the human brain:
3-dimensional proportional system - an approach to cerebral imaging. Thieme
Medical Publishers, New York.
Tallon-Baudry, C., Bertrand, O., Hnaff, M.A., Isnard, J., Fischer, C., 2005. Attention
modulates gamma-band oscillations differently in the human lateral occipital
cortex and fusiform gyrus. Cereb. Cortex 15, 654662.
879
van Gerven, M., Jensen, O., 2009. Attention modulations of posterior alpha as a control signal
for two-dimensional braincomputer interfaces. J. Neurosci. Methods 179, 7884.
van Gerven, M., Bahramisharif, A., Heskes, T., Jensen, O., 2009. Selecting features for
BCI control based on a covert spatial attention paradigm. Neural Netw. 22,
12711277.
Vansteensel, M.J., Hermes, D., Aarnoutse, E.J., Bleichner, M.G., Schalk, G., van Rijen, P.C.,
Leijten, F.S.S., Ramsey, N.F., 2010. Braincomputer interfacing based on cognitive
control. Ann. Neurol. 67, 809816.
Vidal, J.R., Ossandon, T., Jerbi, K., Dalal, S.S., Minotti, L., Ryvlin, P., Kahane, P., Lachaux, J.P.,
2010. Category-specic visual responses: an intracranial study comparing gamma,
beta, alpha, and ERP response selectivity. Front. Hum. Neurosci. 4, A195.
Vogel, E.K., McCollough, A.W., Machizawa, M.G., 2005. Neural measures reveal
individual differences controlling access to working memory. Nature 438, 500503.
Zanto, T.P., Gazzaley, A., 2009. Neural suppression of irrelevant information underlies
optimal performance. J. Neurosci. 29, 30593066.
Zimmer, H.D., 2008. Visual and spatial working memory: from boxes to networks.
Neurosci. Behav. Rev. 32, 13731395.