Start Submission Become a Reviewer

Reading: Rapid Extraction of Emotion Regularities from Complex Scenes in the Human Brain

Download

A- A+
Alt. Display

Original research report

Rapid Extraction of Emotion Regularities from Complex Scenes in the Human Brain

Authors:

Antonio Schettino,

Department of Experimental-Clinical and Health Psychology, Ghent University, 9000 Ghent, BE; Institute for Globally Distributed Open Research and Education (IGDORE), ID
About Antonio
I use high density scalp EEG to study the modulatory effect of emotion on proactive processes leading to visual scene recognition. I am also interested in the relationship between emotion and attention mechanisms.
X close

Christopher Gundlach,

Institute of Psychology, University of Leipzig, Leipzig, DE
X close

Matthias M. Müller

Institute of Psychology, University of Leipzig, Leipzig, DE
X close

Abstract

Adaptive behavior requires the rapid extraction of behaviorally relevant information in the environment, with particular emphasis on emotional cues. However, the speed of emotional feature extraction from complex visual environments is largely undetermined. Here we use objective electrophysiological recordings in combination with frequency tagging to demonstrate that the extraction of emotional information from neutral, pleasant, or unpleasant naturalistic scenes can be completed at a presentation speed of 167 ms (i.e., 6 Hz) under high perceptual load. Emotional compared to neutral pictures evoked enhanced electrophysiological responses with distinct topographical activation patterns originating from different neural sources. Cortical facilitation in early visual cortex was also more pronounced for scenes with pleasant compared to unpleasant or neutral content, suggesting a positivity offset mechanism dominating under conditions of rapid scene processing. These results significantly advance our knowledge of complex scene processing in demonstrating rapid integrative content identification, particularly for emotional cues relevant for adaptive behavior in complex environments.
How to Cite: Schettino, A., Gundlach, C., & Müller, M. M. (2019). Rapid Extraction of Emotion Regularities from Complex Scenes in the Human Brain. Collabra: Psychology, 5(1), 20. DOI: http://doi.org/10.1525/collabra.226
121
Views
25
Downloads
Altmetric
  Published on 07 May 2019
 Accepted on 02 Apr 2019            Submitted on 09 Jan 2019

Introduction

The prioritization of emotional stimuli is pivotal for fast behavioral reactions, for example in the case of threat or danger when the decision for fight-or-flight needs to be made almost instantaneously in order to survive. Appetitive and defensive neural systems, evolved to ensure survival and the continuation of the species (Lang & Bradley, 2010), allow the organism to appraise environmental stimuli on the basis of common motivational parameters, such as valence and arousal (Cacioppo & Berntson, 1994; Cacioppo & Gardner, 1999; Öhman, Hamm, & Hugdahl, 2000; Russell, 1980). In the cognitive electrophysiological literature, many studies have consistently reported enhanced electrical brain activity in response to emotional compared to neutral visual scenes (Olofsson, Nordin, Sequeira, & Polich, 2008), starting at around ~200 ms post-stimulus onset (Costa et al., 2014; Junghöfer, Bradley, Elbert, & Lang, 2001; Schupp, Junghöfer, Weike, & Hamm, 2003b) and presumably subtended by attention-dependent enhancement of sensory input strength (Desimone & Duncan, 1995; Reynolds & Heeger, 2009).

While electrophysiological enhancement is typically interpreted as a consequence of the high relevance of valence and arousal features for adaptive behavior and survival (Frijda, 2016; Lang & Bradley, 2010), the speed at which these emotional cues are extracted from complex visual scenes is still unresolved. Most electrophysiological studies in the affective neuroscience literature have displayed visual stimuli on screen for a relatively long time – e.g., from 333 milliseconds (Schupp, Junghöfer, Weike, & Hamm, 2003a) up to 6 seconds (Hajcak, Dunning, & Foti, 2009) –, a methodological choice that gives the visual system ample opportunity to thoroughly process semantic content but, on the other hand, likely overestimates the time required for initial emotional cue extraction. One obvious solution is to shorten the presentation time of each individual stimulus, thus creating a train of identical images in rapid succession (rapid serial visual presentation; RSVP). Such a presentation typically elicits steady-state visually evoked potentials (SSVEPs), oscillatory posterior brain responses considered a continuous marker of stimulus processing in early visual cortex (Norcia, Appelbaum, Ales, Cottereau, & Rossion, 2015; Regan, 1977; Vialatte, Maurice, Dauwels, & Cichocki, 2010; Wieser, Miskovic, & Keil, 2016). Previous studies (A. Keil et al., 2003, 2009, 2008) used this technique by presenting identical images ten times per second (i.e., 10 Hz) and found larger SSVEP amplitudes for emotional compared to neutral scenes, suggesting once again attention-dependent sensory enhancement for emotional material (Kim, Grabowecky, Paller, Muthu, & Suzuki, 2007; M. M. Müller & Hübner, 2002; M. M. Müller, Malinowski, Gruber, & Hillyard, 2003). However, flickering the exact same picture at a specific frequency over the course of a trial makes it difficult to assess whether the observed increased occipital activity is driven by the extraction of the emotional content from each individual picture onset or a consequence of the integration of emotional information across the entire trial. To overcome this limitation, one can present a different image at every cycle (Alonso-Prieto, Belle, Liu-Shuang, Norcia, & Rossion, 2013; Bekhtereva & Müller, 2015; Retter & Rossion, 2016), with the important corollary that each image serves as a forward mask for the subsequent stimulus and as a backward mask for the preceding one (Keysers & Perrett, 2002). For instance, Bekhtereva and Müller (2015) employed an RSVP paradigm in which different images were presented at each cycle, while the emotional content of the stimuli switched only once per trial (e.g., from streams of neutral to unpleasant scenes and vice versa). Emotion-dependent amplitude modulations of occipito-temporal SSVEPs were observed when each stimulus was presented every ~167 ms (i.e., 6 Hz), whereas no statistical difference was reported at a faster presentation rate (i.e., ~67 ms/image, 15 Hz). This is consistent with previous findings showing that the visual system is capable of extracting the emotional content of complex scenes from images presented as briefly as 80 ms in a backward-masking paradigm (e.g., Codispoti, Mazzetti, & Bradley, 2009).

In the present study, we investigated whether the extraction of emotional features from each picture in the stream can be achieved in situations of greater variability. In other words, is the visual system able to reliably extract emotional information from rapidly presented complex images whose semantic content varies in a quick and unpredictable way? Consider this approach similar to “stress tests” performed on computer hardware or software to assess robustness and error handling under a heavy perceptual load, in order to ensure stability in normal environments. Capitalizing on the results of Bekhtereva and Müller (2015), we flickered different images every ~167 ms (i.e., 6 Hz). Pleasant, unpleasant, or neutral content was presented in a pseudorandom fashion, to exclude the possibility that cortical facilitation could be based on the extraction of general valence rather than the processing of each individual image. The key manipulation was that, within each picture stream, a neutral, pleasant, or unpleasant picture was regularly presented once every three stimuli, thereby creating a semantic regularity at 2 Hz. In addition, we controlled for the influence of low-level visual properties by using the very same stimulation protocol with scrambled versions of the test images (see Materials and Methods for details). Given our previous findings, we hypothesized that, if the visual system were able to extract the general emotional content from images presented as briefly as ~167 ms, the regularity of emotional content within the presented stream of images should lead to a discernable neural signal of this extraction. Specifically, we predicted increased SSVEP amplitudes at 2 Hz only for original (non-scrambled) pictures when the regularity is conveyed by emotional compared to neutral scenes.

Results

Participants were required to keep fixation on a central cross and passively view an RSVP stream of pictures flickering at 6 Hz. A different picture was presented at each cycle, corresponding to ~167 ms presentation time per individual image. In the irregular condition, neutral, pleasant, and unpleasant scenes alternated in pseudorandom order. In the original regular conditions, a neutral, pleasant, or unpleasant picture was systematically presented once every 3 stimuli during the stream, i.e., at 2 Hz. This regularity was strengthened by two additional methodological constraints: (i) we never used the same picture more than once within a trial; (ii) the order of the non-regular fillers (i.e., in between cycles of regular pictures) was pseudorandomized (e.g., unpleasant – pleasant – neutral – unpleasant – neutral – pleasant – unpleasant – …), thus creating completely unpredictable sequences. Scrambled regular conditions were additionally used to control whether any SSVEP amplitude enhancement at 2 Hz could be ascribed to the extraction of low-level features (e.g., color, contrast, or spatial frequency content) (see Figure 1). Scrambled images were preferred as control condition over picture inversion because identification times for upright and inverted scenes has been found to be similar (Rieger, Köchy, Schalk, Grüschow, & Heinze, 2008; Rousselet, Macé, & Fabre-Thorpe, 2003; Vuong, Hof, Bülthoff, & Thornton, 2006), presumably because the main constituents of inverted scenes can quickly be identified through some “flipping compensation process” (J. E. Murray, 1997) matching the current object with memory templates (De Caro & Reeves, 2000).

Figure 1 

Stimuli and procedure. Participants were presented with neutral, pleasant, and unpleasant scenes from the IAPS (Lang et al., 2008) and EmoPics (Wessa et al., 2010) databases (not shown here for copyright reasons; examples are taken from Google Images and published under Creative Commons licenses). In some trials, a neutral, pleasant, or unpleasant scene was regularly presented once every 3 stimuli, thus creating a predictable semantic regularity within the visual stream. The same presentation was used with scrambled pictures, to verify that the steady-state response elicited by such regularity would not exclusively be driven by low-level visual properties.

Increased signal-to-noise ratio for original regular scenes

A necessary first step was to verify that 2 Hz amplitude during the presentation of original scenes was reliably larger than noise, i.e., amplitude at neighboring frequencies (Ding, Sperling, & Srinivasan, 2006; Gundlach & Müller, 2013; Sutoyo & Srinivasan, 2009). Therefore, we calculated the signal-to-noise ratio (SNR) of each condition by dividing amplitude at 2 Hz by the averaged amplitude at the following frequencies (in Hz): 1.00, 1.14, 1.29, 1.43, 1.57, 1.71, 1.86, 2.14, 2.29, 2.43, 2.57, 2.71, 2.86, 3.00. SNR values close to 1 would indicate close similarity between signal and noise. As can be seen in Table 1, SNR to original regular trials was consistently above 1, whereas SNR to irregular and scrambled regular pictures was close to (or below) 1.

Table 1

Signal-to-noise ratio (SNR) and SSVEP amplitude values (in μV) at 2 Hz.

condition SNR amplitude

irregular 0.99 (0.10) 0.17 (0.02)
original regular neutral 1.85 (0.19) 0.33 (0.03)
original regular pleasant 2.46 (0.34) 0.46 (0.04)
original regular unpleasant 1.70 (0.14) 0.27 (0.02)
scrambled regular neutral 0.65 (0.05) 0.12 (0.01)
scrambled regular pleasant 0.98 (0.12) 0.19 (0.01)
scrambled regular unpleasant 1.00 (0.10) 0.18 (0.01)

Note: 20% trimmed means and standard errors (in parentheses).

One-tailed one-sample Wilcoxon signed-rank tests confirmed that the SSVEP signal at 2 Hz was statistically different from noise in response to regular original neutral (Z = 329, p = 3.99 × 10–5, Hedges’ g = 0.94, CI95% [0.58, 1.28]), original pleasant (Z = 345, p = 1.46 × 10–6, g = 1.11, CI95% [0.80, 1.45]), and original unpleasant scenes (Z = 340, p = 4.92 × 10–6, g = 0.97, CI95% [0.69, 1.26]). No statistically significant differences were observed for irregular (Z = 168, p = .999, g = –0.03, CI95% [–0.43, 0.37]), regular scrambled neutral (Z = 14, p = .999, g = –1.27, CI95% [–1.95, –0.63]; here noise is higher than signal, hence the large effect size but non-significant p-value), scrambled pleasant (Z = 178, p = .999, g = 0.14, CI95% [–0.25, 0.50]), or scrambled unpleasant conditions (Z = 177, p = .999, g = 0.12, CI95% [–0.29, 0.44]). Complementary one-tailed Bayesian t-tests against 1 (Rouder, Speckman, Sun, Morey, & Iverson, 2009) confirmed that the observed SNR was better explained by the alternative hypothesis (H1) for regular original neutral (BF10 = 1,182.36 ± 0.00%), original pleasant (BF10 = 9,662.62 ± 0.00%), and original unpleasant conditions (BF10 = 1,609.18 ± 0.00%), whereas the null hypothesis (H0) ought to be preferred for SNR in response to irregular (BF10 = 0.19 ± 0.00%), regular scrambled neutral (BF10 = 0.02 ± 0.00%), scrambled pleasant (BF10 = 0.40 ± 0.03%), and scrambled unpleasant conditions (BF10 = 0.37 ± 0.00%) (see Table 2). For a specification of H1 and H0, see Materials and Methods.

Table 2

Bayes Factors (BF10) and percentage of proportional errors (% pe) for each model relative to the null. The dependent variable is SSVEP amplitude values at 2 Hz.

comparison condition r = 1 r = .707 r = .5

BF10 % pe BF10 % pe BF10 % pe

2 Hz amplitude irregular 0.14 ±0.01 0.19 ±0.00 0.25 ±0.01
regular neutral 1,203.13 ±0.00 1,182.36 ±0.00 1,053.77 ±0.00
regular pleasant 10,404.17 ±0.00 9,662.62 ±0.00 8,182.60 ±0.00
vs. regular unpleasant 1,652.19 ±0.00 1,609.17 ±0.00 1,422.10 ±0.00
noise scrambled regular neutral 0.02 ±0.00 0.02 ±0.00 0.01 ±0.00
scrambled regular pleasant 0.30 ±0.00 0.40 ±0.03 0.51 ±0.00
scrambled regular unpleasant 0.27 ±0.02 0.37 ±0.00 0.48 ±0.00

scrambled neutral 115,028.63 ±0.00 100,128.86 ±0.00 80,480.92 ±0.00
vs. pleasant 6,601.58 ±0.00 6,096.65 ±0.00 5,138.10 ±0.00
original unpleasant 62.41 ±0.00 65.88 ±0.00 63.12 ±0.00

2 Hz amplitude, main effect emotion 10.04 ±0.01 13.45 ±0.00 15.66 ±0.01

pairwise neutral vs. pleasant 13.55 ±0.00 15.12 ±0.00 15.43 ±0.00
comparisons neutral vs. unpleasant 0.26 ±0.00 0.35 ±0.03 0.45 ±0.01
(original) unpleasant vs. pleasant 224.47 ±0.00 227.19 ±0.00 208.40 ±0.00

A direct comparison of SNR in regular original and scrambled conditions would provide compelling evidence that the SSVEP signal at 2 Hz is reliably reflecting cognitive processes not exclusively related to the extraction of low-level visual properties of the stimuli. Two-tailed paired-sample Wilcoxon signed-rank tests confirmed larger SNR in response to original vs. scrambled neutral (Z = 348, p = 4.47 × 10–7, g = 1.37, CI95% [0.98, 1.76]), pleasant (Z = 343, p = 1.49 × 10–6, g = 1.13, CI95% [0.76, 1.48]), and unpleasant scenes (Z = 316, p = 1.26 × 10–4, g = 0.76, CI95% [0.26, 1.12]). Bayesian t-tests corroborated the NHST results: the difference between original and scrambled conditions was more likely under H1 for neutral (BF10 = 100,128.86 ± 0.00%), pleasant (BF10 = 6,096.65 ± 0.00%), and unpleasant scenes (BF10 = 65.88 ± 0.00%) (see Table 2).

These results show that the 2 Hz signal in response to original regular pictures was reliably larger than the signal in response to irregular or scrambled pictures and likely reflected post-perceptual brain processes. As 2 Hz signals for irregular and scrambled conditions did not reliably differ from noise, they were not further analyzed.

Amplitude modulations depend on emotional content

Our main prediction was that SSVEP amplitude at 2 Hz should be larger in response to emotional relative to neutral scenes. A one-way robust repeated measures ANOVA (rbANOVARM) (Field, Miles, & Field, 2012; Field & Wilcox, 2017) revealed a significant main effect of emotion (F1.68, 25.13 = 17.00, p = 4.79 × 10–5, ξ = .51, CI95% [.14, .82]). A complementary Bayesian ANOVARM (Rouder, Engelhardt, McCabe, & Morey, 2016; Rouder, Morey, Speckman, & Province, 2012; Rouder, Morey, Verhagen, Swagman, & Wagenmakers, 2017) confirmed that these amplitude values were more likely to be explained by the model with the main effect of emotion compared to the null model (BF10 = 13.45 ± 0.01%). We proceeded to qualify the direction of this difference by means of two-tailed paired-sample Wilcoxon signed-rank tests. Amplitude in response to pleasant scenes was reliably larger compared to neutral (Z = 60, p = .005, g = –0.64, CI95% [–1.08, –0.15]) and unpleasant scenes (Z = 43, p = .001, g = –0.87, CI95% [–1.36, –0.45]). The difference between neutral and unpleasant conditions was not statistically significant (Z = 219, p = .280, g = 0.20, CI95% [–0.20, 0.62]). Bayesian t-tests confirmed that H1 should be preferred when comparing pleasant to neutral (BF10 = 15.12 ± 0.00%) and unpleasant conditions (BF10 = 227.19 ± 0.00%), whereas the difference between neutral and unpleasant conditions leaned towards H0 (BF10 = 0.35 ± 0.03%) (see Table 2 and Figure 2).

Figure 2 

Results of the 2 Hz signals. (A) Grand average FFT-amplitude spectra derived from EEG signals at each participant’s best four-electrode cluster for the 2 Hz signal for the different experimental conditions (blue: neutral; green: pleasant; red: unpleasant). The shaded areas around the means indicate 95% confidence intervals. The electrode cluster selected for the analysis is highlighted in the topography. (B) Amplitude values of the 2 Hz signals for each participant (single dots) and experimental condition. Mean amplitude values are marked by horizontal black lines and 95% Bayesian highest density interval (HDI) represented as white boxes. **: p < .01; n.s.: not significant.

We also found differences in the topographical distribution of the 2 Hz signal for the different emotion conditions (Lehmann & Skrandies, 1984; Michel & Murray, 2012; M. M. Murray, Brunet, & Michel, 2008). While there was a significant dissimilarity between 2 Hz topographies for regularly presented neutral and pleasant images (Global Map Dissimilarity, GMD = 0.638, p < .001) as well as neutral and unpleasant images (GMD = 0.587, p = .033), there was no significant dissimilarity between regularly presented unpleasant and pleasant images (GMD = 0.357, p = .317). These results point towards statistically similar topographic representations of the 2 Hz signal when emotional images are presented regularly, which is different from that of regularly presented neutral images (see Figure 3A).

Figure 3 

Results of the 2 Hz signals. (A) Grand Mean topographical representation of the 2 Hz signals for different emotion conditions. (B) Results of the source localization. Significant source amplitude modulations are shown for the factor emotion (whole brain family-wise-error-corrected, pvoxel < .001, pcluster < .05) in different axial slices projected to a standard brain. Numbers above slices represent MNI coordinates of axial slice. Effect estimates are shown for each cluster in bar graphs. Error bars represent SEM.

This pattern was mirrored in the source estimates of the 2 Hz signals for the different emotional conditions (Friston et al., 2008; Litvak et al., 2011). In two clusters, source power estimates differed significantly as a function of emotional content of the regularly presented images (pvoxel < .001, pcluster < .05, whole-brain FWE-corrected; see Figure 3B). In one cluster, comprising of areas in right inferior temporal gyrus and fusiform gyrus, source power estimates were highest when neutral images were presented regularly while lower for emotional images (MNI coordinates of peak voxel: x = 52, y = –44, z = –26, Fpeakvoxel = 39.86, 704 voxels in cluster). Source power estimates in a second cluster – including more anterior areas in right fusiform and right inferior temporal gyrus, as well as right hippocampal regions and parts of the right amygdala (MNI coordinates of peak voxel: x = 38, y = –10, z = –30, Fpeakvoxel = 38.17, 534 voxels in cluster) – were higher when emotional images were presented regularly as compared to regularly presented neutral images.

Discussion

The results of the “perceptual stress test” presented here demonstrate that the human brain is able to extract, within 170 ms post-stimulus onset, diagnostic cues from complex naturalistic scenes. Importantly, we show that the extracted information is not limited to basic semantic features, e.g., the presence of living or non-living objects (Grill-Spector, Kushnir, Hendler, & Malach, 2000; Johnson & Olshausen, 2003; Liu, Agam, Madsen, & Kreiman, 2009; Thorpe, Fize, & Marlot, 1996; VanRullen & Thorpe, 2001a), but encompasses motivationally relevant cues such as emotional valence and arousal (Lang & Bradley, 2010). The continuous stream of rapidly changing stimuli is known to produce clear masking effects, thereby making visual information available only for the presentation time of each image (Alonso-Prieto et al., 2013; Bekhtereva & Müller, 2015; Retter & Rossion, 2016), which was still sufficient for the extraction of emotional cues even during continuous presentation over 7 seconds. Of note, our stimulation protocol was different from recent studies that used regular presentations of faces, body parts, or houses within a stream of natural objects (Jacques, Retter, & Rossion, 2016; Retter & Rossion, 2016; Rossion, Torfs, Jacques, & Liu-Shuang, 2015) or expressive faces in a stream of neutral faces (Dzhelyova, Jacques, & Rossion, 2017), in that it conveyed the regularity using the same stimulus type as the fillers, i.e., naturalistic scenes matched with respect to apparent contrast, subjective and objective complexity, and proportion of living/non- living objects (see Supplementary Materials for details). This strategy allowed us to circumvent several caveats associated with the exclusive use of faces to elicit regularity in the visual system. For instance, faces are perceptually simpler (M. S. Keil, 2008; VanRullen, 2006), over-trained (Bukach, Gauthier, & Tarr, 2006; Gauthier & Nelson, 2001; Tanaka, 2001; Tarr & Gauthier, 2000), and more salient – because they typically convey important social signals (Calder & Young, 2005; Frith, 2009; Said, Haxby, & Todorov, 2011) – compared to non-face objects or naturalistic scenes. Even more relevant for the current study, emotional cue extraction occurs earlier for faces than scenes (Bekhtereva, Craddock, & Müller, 2015) and may be subserved by partially segregated neural circuits (Britton, Taylor, Sudheimer, & Liberzon, 2006; Haxby, Hoffman, & Gobbini, 2000). Finally, this study complements existing literature suggesting rapid extraction of emotional information after short stimulus presentation (e.g., Codispoti et al., 2009) by focusing on early visual cortex activity instead of late electrophysiological components, which are the by-product of several perceptual and cognitive processes (Hajcak, MacNamara, & Olvet, 2010).

Pleasant information is prioritized in early visual cortex

Nonparametric and Bayes factor analyses converged to show a robust 2 Hz SSVEP response to the regularly presented neutral, unpleasant, and pleasant pictures, but not their scrambled versions. This result excludes the possibility that our 2 Hz response for original scenes could solely be ascribed to the rapid extraction of low-level features such as spatial frequency, color, or contrast, which were matched between original and scrambled pictures.

Crucially, our paradigm highlighted robust amplitude differences between emotional conditions, with pleasant scenes eliciting larger 2 Hz amplitude compared to unpleasant and neutral pictures. These findings are consistent with a wealth of studies showing privileged processing of emotional visual stimuli (Carretié, 2014; Frijda, 2016; Lang & Bradley, 2010; Pessoa, 2008; Pourtois, Schettino, & Vuilleumier, 2013; Vuilleumier, 2005). Surprisingly, 2 Hz activity was specifically enhanced for pleasant scenes, suggesting preferential attentional capture for intrinsically hedonic stimuli. One plausible post-hoc explanation could be that, during such an effortless task (passive viewing) that would eventually lead to a reward (i.e., monetary compensation for participation), observers would be able to engage their cognitive resources towards a thorough exploration of the RSVP streams and focus on the rewarding information conveyed by pleasant scenes. This positivity offset, as termed within the Evaluative Space Model framework (Cacioppo, Gardner, & Berntson, 1997; Ito, Cacioppo, & Lang, 1998; Norris, Gollan, Berntson, & Cacioppo, 2010), refers to a stronger motivation to approach than avoid unfamiliar but non-threatening contexts. From an evolutionary perspective, this activation function enables organisms to explore novel environments, with the ultimate goal to find additional sources of nourishment and protection as well as occasions for mating and reproduction (Cacioppo & Berntson, 1994; Cacioppo & Gardner, 1999). Research has shown that positivity offset (as well as negativity bias) can be generalized across different kinds of stimuli (Norris, Larsen, Crawford, & Cacioppo, 2011), is temporally stable and trait-like consistent (Ito & Cacioppo, 2005), and can be used as a theoretical construct to interpret serotonergic function in healthy and clinical populations (Ashare, Norris, Wileyto, Cacioppo, & Strasser, 2013; Carver, Johnson, & Joormann, 2009; Gollan et al., 2016).

During our task, positivity offset may have outweighed negativity bias because of the absence of proximal environmental danger which, in turn, promoted exploration (Ito & Cacioppo, 2005; Schettino, Loeys, Bossi, & Pourtois, 2012; Schettino, Loeys, Delplanque, & Pourtois, 2011; Schettino, Loeys, & Pourtois, 2013). Furthermore, participants were probably inclined to thoroughly examine images carrying intrinsically rewarding information also because these pictures likely matched their motivational dispositions (Byrne & Eysenck, 1993; Niedenthal, Halberstadt, & Setterlund, 1997; Niedenthal & Setterlund, 1994; Raila, Scholl, & Gruber, 2015). Broadly speaking, positivity offset processes could underpin attentional biases towards pleasant stimuli frequently observed in healthy people (Pool, Brosch, Delplanque, & Sander, 2016; Sennwald et al., 2016). Intriguingly, positive stimuli seem to have their major impact during the initial stages of visual processing (Pool et al., 2016), when attentional shifts are more likely to be driven by stimulus properties rather than top-down goals (Theeuwes, 1994). Thus, pleasant images might show strongest attentional capture – and, consequently, largest electrophysiological effects – when briefly presented, like in our RSVP streams.

As we have shown in previous work (Bekhtereva & Müller, 2015), the modulation of SSVEPs for emotional as compared to neutral, rapidly presented visual images requires sufficient processing time for emotional cue extraction in the time range of the EPN (Junghöfer et al., 2001; Schupp, Flaisch, Stockburger, & Junghöfer, 2006), thus potentially linking SSVEP modulations and the EPN. Intriguingly, the EPN seems to be modulated by stimulus valence in a similar manner, showing a larger deflection for pleasant as compared to unpleasant stimuli (Schupp, Junghöfer, Weike, & Hamm, 2004). Only recently we could show that the SSVEP amplitude modulation by the emotional content of the image stream differed for various stimulation frequencies and that these differences may be explained by a model that posits the SSVEP as a superposition of ERPs (Bekhtereva, Pritschmann, Keil, & Müller, 2018). It is thus tempting, but highly speculative, to link the modulations of SSVEP found here to known ERP modulations of the EPN by the emotional picture content.

Is the 2 Hz response a consequence of stimulus predictability?

Given the regular presentation of pictures of the same valence category, one could wonder whether the observed 2 Hz SSVEP response is a consequence of predictability rather than perceptual categorization. Indeed, recent studies have shown that temporal expectations could aid visual facilitation of regularly presented items (Breska & Deouell, 2014; Cravo, Haddad, Claessens, & Baldo, 2013), and that perceptual expectations may influence recognition processes at different stages (Carlson, Grol, & Verstraten, 2006; Ploran, Tremel, Nelson, & Wheeler, 2011; Summerfield & de Lange, 2014; Summerfield & Egner, 2009) depending on the emotional connotation of the stimuli (Barrett & Bar, 2009). Temporal expectations might indeed have had an impact on previous studies (Peyk, Schupp, Keil, Elbert, & Junghöfer, 2009) reporting emotional cue extraction up to 12 Hz (e.g., 83 ms per image), because of the presentation of simple and predictable stimulus sequences (e.g., pleasant – neutral – pleasant – neutral – …). Nonetheless, we consider the influence of temporal and perceptual expectations highly unlikely in our paradigm, for several reasons. First, in the regular conditions the pseudorandom stimulus presentation created completely unpredictable sequences (see Results section). Second, the regularity was always elicited by different pictures, which had little in common except emotional valence. Third, participants were not aware of any regularities in the RSVP stream, as post-experiment verbal reports confirmed. Fourth, a recent paper (Quek & Rossion, 2017) provided additional evidence that category-specific responses in RSVP streams are unlikely to be solely influenced by temporal expectations.

Distinct patterns of cortical activation for emotional vs. neutral content

The reported electrophysiological responses to emotional and neutral scenes also differed in the topographical distribution and estimated cortical sources of the 2 Hz SSVEP signal. Neutral images elicited higher activity in right inferotemporal and fusiform gyri, whereas more anterior portions of the right fusiform and temporal gyri as well as hippocampal regions responded more strongly to emotional scenes. Of note, no statistical differences were observed between pleasant and unpleasant scenes, presumably because these stimuli recruit the same neural circuits while modulating their intensity. Overall, these results point towards a general involvement of scene-selective cortical regions in the lateral occipital cortex (Grill-Spector, Kourtzi, & Kanwisher, 2001; Nasr et al., 2011). Emotional scenes seem to elicit enhanced activity in anterior occipitotemporal areas (Bradley et al., 2003; Sabatinelli et al., 2011) and even the amygdala, a subcortical structure classically implicated in emotion detection and recognition (Adolphs, 2002; LeDoux, 2007; Phelps & LeDoux, 2005; Pourtois et al., 2013; Vuilleumier, 2005). However, we prefer not to provide a strong interpretation of these findings, not only because of the exploratory nature of our analyses but, most importantly, due to the characteristics of the electrophysiological signal recorded on the scalp as well as the poor spatial resolution of source localization algorithms without individual co-registration (Belardinelli, Ortiz, Barnes, Noppeney, & Preissl, 2012; Grech et al., 2008; López, Litvak, Espinosa, Friston, & Barnes, 2014; López, Penny, Espinosa, & Barnes, 2012). Future studies using imaging techniques suited to precisely pinpoint activity in specific brain areas could provide more compelling evidence of the recruitment of deep brain structures in this experimental paradigm.

Conclusions

The present study significantly advances our knowledge of complex scene processing by showing that the human brain can rapidly extract motivationally relevant information within a perceptually challenging temporal succession of pictures. Despite limited visibility of each individual scene by forward and backward masking, emotional cues could quickly be extracted within the first 167 ms post-stimulus onset. Greater neural facilitation for pleasant compared to neutral and unpleasant images may be due to a positivity bias that could dominate under such rapid categorical demands in non-threatening contexts. Whether the presentation time of 167 ms is the lowest limit for semantic categorization of complex naturalistic scenes is the subject of future studies but, on the basis of current models of object processing (Fabre-Thorpe, Delorme, Marlot, & Thorpe, 2001; Grill-Spector et al., 2000; Johnson & Olshausen, 2003; VanRullen & Thorpe, 2001b) as well as other studies employing a similar paradigm (Alonso-Prieto et al., 2013; Bekhtereva & Müller, 2015), presentation frequencies around 6 Hz seem to be the upper bound for efficient semantic categorization.

Materials and Methods

Participants

Twenty-six Caucasian individuals (13M/13F, 25 right-handed; median age 24 years, range 19–38) were recruited from the student population of the University of Leipzig and among the general public. An equal number of male and female participants was planned to avoid possible gender-specific differences in emotion reaction and evaluation (Lithari et al., 2010; Proverbio, Adorni, Zani, & Trestianu, 2009; Sabatinelli, Flaisch, Bradley, Fitzsimmons, & Lang, 2004). All volunteers were German speaking, had normal or corrected-to-normal vision, and reported no history of neurological or psychiatric disorders. This sample size was chosen based on available time and economic resources (no statistical a priori power analysis was conducted).

The experimental protocol was approved by the ethics committee of the University of Leipzig (ethical approval #415/17-ek). The study was conducted in accordance with the guidelines of the ethics committee of the University of Leipzig and the Code of Ethics of the World Medical Association. All volunteers gave written informed consent prior to participation. At the end of the experiment, they were fully debriefed and received 12 € (6 € per hour).

Stimuli

One-hundred and fifty pictures were selected from the IAPS (Lang, Bradley, & Cuthbert, 2008) and EmoPics (Wessa et al., 2010) databases, equally divided into three emotion categories – neutral, unpleasant, and pleasant – according to their normative valence and arousal ratings. These stimuli were resized to 419 × 314 pixels (to minimize eye movements) and comparable with respect to a number of low-level visual properties (for a complete description, see the Supplementary Materials). A separate set of scrambled images was additionally created: each original picture was modified by applying a spatial discrete Fourier transform to each RGB-color channel, replacing the phase spectrum with random values, and reconstructing the image by applying an inverse Fourier transform. This procedure disrupts picture content while keeping color, contrast, and spatial frequency content intact (Hindi Attar, Andersen, & Müller, 2010; Hindi Attar & Müller, 2012; M. M. Müller, Andersen, & Hindi Attar, 2011; Schettino, Keil, Porcu, & Müller, 2016). In sum, a total of 300 pictures were used, 50 for each stimulus type: (i) original neutral; (ii) original pleasant; (iii) original unpleasant; (iv) scrambled neutral; (v) scrambled pleasant; (vi) scrambled unpleasant. An example of each stimulus type is provided in Figure 1.

Procedure

Upon arrival at the laboratory, participants signed the informed consent, had EEG sensors placed on the scalp, and were seated in a dimly lit Faraday cage at approximately 80 cm from a 19” CRT monitor (Samsung Samtron 98PDF (L) L, 1024 × 768 pixels screen resolution, 16-bit color, 60 Hz refresh rate) connected to a PC running Matlab v7.5.0 (The Mathworks, Inc, Natick, MA) and the Cogent toolbox (v1.32; http://www.vislab.ucl.ac.uk/cogent.php).

The experimenter ensured that the task was understood by providing verbal and written instructions. After a practice session with 14 trial pictures (not included in the experimental picture set), the main experiment started. On each trial, a central white fixation cross (1° × 1° degrees of visual angle) was presented on a gray background for 250 ms. The stimuli (10.5° × 7.9°) were subsequently presented in the center of the screen – time-locked to the refresh rate of the monitor – as an RSVP stream flickering at 6 Hz (i.e., ~167 ms on-presentation time per image). Participants were asked to simply focus on the content of each picture. In the main experimental conditions, a neutral, pleasant, or unpleasant picture (regular neutral, regular pleasant, and regular unpleasant conditions, respectively) was regularly presented once every 3 stimuli during the stream, i.e., at 2 Hz. The second and third images of each image triplet was randomly drawn from the two remaining emotional content categories, so that for each triplet each emotional content category was presented once. Please note that, to create such regularity, we never used the same picture more than once within a stream: every 2 Hz cycle contained a different scene, and the only common feature of all images within the other cycles was the emotional content. To verify that the hypothesized enhancement of the electrophysiological signal was not due to the influence of low-level features (e.g., color, contrast, or spatial frequency content), the same regular presentation scheme was employed for scrambled images (scrambled regular neutral, scrambled regular pleasant, and scrambled regular unpleasant conditions). In the last condition (irregular), the original pictures were presented randomly, i.e., no regularity at 2 Hz was imposed (see Figure 1 for examples). Each trial lasted 7,000 ms, with 42 pictures shown at the driving frequency of 6 Hz and 14 presentations at 2 Hz in the regular conditions. During the inter-trial interval, randomly varying between 1,500 and 2,000 ms, a white ‘X’ signaled that participants could blink. The experiment consisted of 490 trials subdivided in 14 blocks (i.e., 35 trials per block), with 70 trials per condition.

To ensure that our pre-selected pictures were processed by participants in accordance with our categorization, valence and arousal ratings were collected at the end of the main task via the Self-Assessment-Manikin (Bradley & Lang, 1994), ranging from 1 (low arousal – unpleasant valence) to 9 (high arousal – pleasant valence) (see results in the Supplementary Materials). Responses were given on the numeric pad of a standard QWERTZ keyboard connected via USB.

EEG recording and pre-processing

Electroencephalographic activity (EEG) was recorded with an ActiveTwo amplifier (BioSemi, Inc., The Netherlands) at a sampling rate of 512 Hz. Sixty-four Ag/AgCl electrodes were fitted into an elastic cap, following the international 10/10 system (Oostenveld & Praamstra, 2001) except electrodes T7 and T8, which were moved in position I1 and I2 to increase spatial resolution at occipital sites. The common mode sense (CMS) active electrode and the driven right leg (DRL) passive electrode were used as reference and ground electrodes, respectively. Horizontal and vertical electrooculogram (EOGs) were monitored using four facial bipolar electrodes placed on the outer canthi of each eye and in the inferior and superior areas of the left orbit.

Data preprocessing was performed offline with custom MATLAB scripts and functions included in EEGLAB v14.0.0b (Delorme & Makeig, 2004), FASTER v1.2.3b (Nolan, Whelan, & Reilly, 2010), and SPM12 (http://www.fil.ion.ucl.ac.uk/spm/software/spm12/) toolboxes. After subtracting the mean value of the waveform (DC offset), the continuous EEG data were epoched between 0 and 7,000 ms, corresponding to the beginning and end of the picture streams, respectively. After referencing to Cz, FASTER functions were used to label data exceeding a z-score of ±3 standard deviations as contaminated by artifacts (for details, see our script at https://osf.io/vb8a4/). Noisy channels were interpolated via a spherical spline procedure (Perrin, Pernier, Bertrand, & Echallier, 1989). Epochs containing artifacts and/or more than 12 interpolated channels were discarded. After preprocessing, the average number of interpolated channels was 3.12 (SD = 1.53, range 0–7) and the mean percentage of rejected epochs was 5.81% (SD = 3.81, range 1.02–15.51). After re-referencing to the average amplitude of all scalp electrodes, 7 grand-averages were computed: (i) irregular (number of averaged trials: M = 65.54, SD = 2.77); (ii) regular neutral (M = 64.85, SD = 2.77); (iii) regular pleasant (M = 65.69, SD = 2.38); (iv) regular unpleasant (M = 65.92, SD = 2.68); (v) scrambled regular neutral (M = 65.08, SD = 3.07); (vi) scrambled regular pleasant (M = 65.65, SD = 2.48); (vii) scrambled regular unpleasant (M = 66.23, SD = 2.52).

Confirmatory analysis: Amplitude of SSVEP signal

Amplitude at 6 Hz was analyzed only to confirm that the main stimulation frequency elicited a robust entrained signal in the brain. The results can be found in the Supplementary Materials.

With respect to 2 Hz, electrodes with maximum SSVEP amplitudes were identified by calculating isocontour voltage maps based on grand-averaged data collapsed across all conditions. As shown in Figure 2, activity was mainly localized at right occipito-temporal channels (e.g., PO10, PO8, P8). To account for inter-individual variations in topographical SSVEP amplitude distributions, we identified and averaged activity from the four electrodes displaying, for each participant, the largest frequency-specific amplitude. After removing linear trends, we extracted SSVEP amplitude at 2 Hz from each individual electrode cluster, separately for each condition (averaged across trials). Fast Fourier Transforms on the EEG signal in a time window from 500 ms (to exclude the typically strong phasic visual evoked response to picture onset) to 7,000 ms after stimulus onset was applied, and amplitudes were obtained by extracting the absolute values of the resulting complex Fourier coefficients.

Given that the data in some conditions violated the assumption of normality (verified via Shapiro-Wilk and Anderson-Darling tests), we compared SSVEP amplitudes across conditions using robust nonparametric methods (Field et al., 2012; Field & Wilcox, 2017). When comparing more than two conditions, we employed repeated measures ANOVAs on 20% trimmed means (rbANOVARM), accompanied by an explanatory measure of effect size ξ and its respective 95% bootstrapped confidence intervals (5,000 samples). One-sample and paired-sample comparisons were conducted via Wilcoxon signed-rank test, complemented by bootstrapped Hedges’ g (and respective 95% confidence intervals) as a measure of effect size (Fritz, Morris, & Richler, 2012; Lakens, 2013). The significance level for all tests was set at p = .05, corrected for multiple comparisons via Bonferroni-Holm procedure (Holm, 1979).

Given the problems inherent in accepting the null hypothesis with classical frequentist procedures (Kruschke, 2010; Morey, Romeijn, & Rouder, 2016; Rouder, Morey, Verhagen, Province, & Wagenmakers, 2016; Wagenmakers, 2007), we additionally calculated Bayes Factors (Jeffreys, 1961; Kass & Raftery, 1995). With Bayesian ANOVAs, Bayes Factors (BF10) were estimated – using Markov-chain Monte Carlo sampling (100,000 iterations) – to quantify the evidence in favor of each model of interest relative to the null model (Rouder, Engelhardt, et al., 2016; Rouder et al., 2012, 2017). With respect to Bayesian t-tests, BF10 were calculated to estimate the degree of evidence in favor of a model assuming differences between two specified conditions relative to a model assuming no differences (Rouder et al., 2009). For all analyses, the null hypothesis was specified as a point-null prior placed on standardized effect size (i.e., δ = 0) whereas, for the alternative hypothesis, Jeffrey-Zellner-Siow (JZS) priors were used, i.e., a folded Cauchy distribution centered around δ = 0 with various scaling factors (r = 1, r = 0.707, r = 0.5). This sensitivity analysis is useful to verify the robustness of the results regardless of changes in the prior (Schönbrodt, Wagenmakers, Zehetleitner, & Perugini, 2017). The BF10 with JSZ prior Cauchy(0, 0.707) is reported in the main text, but the reader is referred to the corresponding tables for details.

Exploratory analysis: Topographic distribution of SSVEP signal

To quantify the observed differences in the topographic distribution of SSVEP amplitude values between conditions of interest, we used the Global Map Dissimilarity index (GMD). Mean topographic distributions for each condition were first normalized by the Global Field Power (GFP), a time-dependent measure of the electric field strength that reflects the amount of synchronized activity across all electrodes (Michel & Murray, 2012). The dissimilarity between any two normalized topographies was then calculated as the square root of the averaged differences in amplitudes between these two topographies at each channel (Lehmann & Skrandies, 1984; M. M. Murray et al., 2008). The resulting GMD – ranging from 0 (topographic homogeneity) to 2 (maximal topographic inversion) – was then statistically evaluated with a permutation approach. By comparing the actual GMD with the distribution of permutation-based chance GMDs, a Monte Carlo based p-value was calculated. Specifically, topographies across the two compared conditions were treated as if they belonged to the same distribution. The resulting p-value represents the likelihood of getting a GMD as large as the one observed purely by chance (i.e., by randomly assigning the condition labels to the topographies). For a p-value < .05, one would assume the observed GMD to be due to the fact that topographies for the conditions of interest belong to two different distributions rather than one, i.e., to be significantly different from each other.

Exploratory analysis: Source localization of SSVEP signal

Sources of SSVEPs were modelled using source reconstruction algorithms implemented in SPM12. A standardized forward model was constructed using a template cortical mesh with 8,196 vertices co-registered to standard EEG positions. The lead field for the forward model was computed using the three-shell BEM EEG head model implemented in SPM12. The preprocessed EEG data were filtered with a 1.5–2.5 Hz Hamming windowed-sinc FIR filter (filter order 4096), and trials were averaged for each condition. Sources were estimated using multiple sparse priors (Friston et al., 2008). For each participant and condition, three-dimensional source power estimates were extracted. These source power images could then be statistically analyzed on a group level using conventional SPM t-tests and regression statistics. Statistical parametric maps were thresholded at p < .001 on a voxel level and corrected on the cluster level using random field theory (pcluster < .05, whole-brain FWE corrected).

To estimate potential source power differences for the processing of images of different emotional quality, the source power images for the 2 Hz signal of the regularly presented original images were tested with a one-way repeated measures ANOVA (ANOVARM) with factor emotion (neutral, pleasant, unpleasant). The model comprised only regular conditions because 2 Hz signals for these conditions were different from noise (see Results section). For an estimation of the effects, the averaged beta coefficients of the effects of interest from the ANOVARM model were extracted for each significant cluster with the help of the MarsBaR toolbox (Brett, Anton, Valabregue, & Poline, 2002). To relate results to cytoarchitectonic references, the SPM anatomy toolbox (Eickhoff et al., 2005) was used whenever possible and images for publication were created using MRIcron (Rorden & Brett, 2000).

Software

Data visualization and statistical analyses were performed using R v3.4.3 (R Core Team, 2017) via RStudio v1.1.419 (RStudio Team, 2015). We used the following packages (and their respective dependencies):

Amplitude spectra and EEG topographies were created by adapting the scripts by Dr. Matt Craddock (https://github.com/craddm/eegUtils). Topographic and source estimation analyses were carried out in Matlab v7.5.0 (The Mathworks, Inc, Natick, MA).

Data Accessibility Statement

Raw and pre-processed data, materials, and analysis scripts are available on https://osf.io/9dcsm/.

Additional File

The additional file for this article can be found as follows:

Text S1

Rapid extraction of emotion regularities from complex scenes in the human brain. DOI: https://doi.org/10.1525/collabra.226.s1

Acknowledgements

We would like to thank Renate Zahn for her help with data collection.

Funding Information

This work is supported by grants from Deutsche Forschungsgemeinschaft (MU972/22-1, MU972/22-1/2) and Ghent University (BOF14/PDO/123). The funding sources had no involvement in the study design; collection, analysis, and interpretation of data; writing of the report; and decision to submit the article for publication.

Competing Interests

The authors have no competing interests to declare.

Author Contributions

AS, CG, and MMM conceived the study. AS and CG programmed the experimental paradigm. AS and CG analyzed the data. AS, CG, and MMM contributed reagents/materials/tools. AS wrote the main manuscript text. AS, CG, and MMM reviewed and critically revised the manuscript.

References

  1. Adolphs, R. (2002). Neural systems for recognizing emotion. Current Opinion in Neurobiology, 12(2), 169–177. DOI: https://doi.org/10.1016/S0959-4388(02)00301-X 

  2. Akima, H., & Gebhardt, A. (2016). akima: Interpolation of Irregularly and Regularly Spaced Data. Retrieved from https://CRAN.R-project.org/package=akima 

  3. Alonso-Prieto, E., Belle, G. V., Liu-Shuang, J., Norcia, A. M., & Rossion, B. (2013). The 6 Hz fundamental stimulation frequency rate for individual face discrimination in the right occipito-temporal cortex. Neuropsychologia, 51(13), 2863–2875. DOI: https://doi.org/10.1016/j.neuropsychologia.2013.08.018 

  4. Arnold, J. B. (2017). ggthemes: Extra Themes, Scales and Geoms for “ggplot2.” Retrieved from https://CRAN.R-project.org/package=ggthemes 

  5. Ashare, R. L., Norris, C. J., Wileyto, E. P., Cacioppo, J. T., & Strasser, A. A. (2013). Individual differences in positivity offset and negativity bias: Gender-specific associations with two serotonin receptor genes. Personality and Individual Differences, 55(5), 469–473. DOI: https://doi.org/10.1016/j.paid.2013.04.009 

  6. Auguie, B. (2017). gridExtra: Miscellaneous Functions for “Grid” Graphics. Retrieved from https://CRAN.R-project.org/package=gridExtra 

  7. Bache, S. M., & Wickham, H. (2014). magrittr: A Forward-Pipe Operator for R. Retrieved from https://CRAN.R-project.org/package=magrittr 

  8. Barrett, L. F., & Bar, M. (2009). See it with feeling: affective predictions during object perception. Philosophical Transactions of the Royal Society B: Biological Sciences, 364(1521), 1325–1334. DOI: https://doi.org/10.1098/rstb.2008.0312 

  9. Bekhtereva, V., Craddock, M., & Müller, M. M. (2015). Attentional bias to affective faces and complex IAPS images in early visual cortex follows emotional cue extraction. NeuroImage, 112, 254–266. DOI: https://doi.org/10.1016/j.neuroimage.2015.03.052 

  10. Bekhtereva, V., & Müller, M. M. (2015). Affective facilitation of early visual cortex during rapid picture presentation at 6 and 15 Hz. Social Cognitive and Affective Neuroscience, 10(12), 1623–33. DOI: https://doi.org/10.1093/scan/nsv058 

  11. Bekhtereva, V., Pritschmann, R., Keil, A., & Müller, M. M. (2018). The neural signature of extracting emotional content from rapid visual streams at multiple presentation rates: A cross-laboratory study. Psychophysiology, 55(12), e13222. DOI: https://doi.org/10.1111/psyp.13222 

  12. Belardinelli, P., Ortiz, E., Barnes, G., Noppeney, U., & Preissl, H. (2012). Source reconstruction accuracy of MEG and EEG Bayesian inversion approaches. PloS One. DOI: https://doi.org/10.1371/journal.pone.0051985 

  13. Bradley, M. M., & Lang, P. J. (1994). Measuring Emotion: The Self-Assessment Mannequin and the Semantic Differential. Journal of Behavior Therapy and Experimental Psychiatry, 25(1), 49–59. DOI: https://doi.org/10.1016/0005-7916(94)90063-9 

  14. Bradley, M. M., Sabatinelli, D., Lang, P. J., Fitzsimmons, J. R., King, W., & Desai, P. (2003). Activation of the Visual Cortex in Motivated Attention. 117(2), 369–380. DOI: https://doi.org/10.1037/0735-7044.117.2.369 

  15. Breska, A., & Deouell, L. Y. (2014). Automatic bias of temporal expectations following temporally regular input independently of high-level temporal expectation. Journal of Cognitive Neuroscience, 26(7), 1555–1571. DOI: https://doi.org/10.1162/jocn_a_00564 

  16. Brett, M., Anton, J.-L., Valabregue, R., & Poline, J.-B. (2002, June 2). Region of interest analysis using an SPM toolbox [abstract]. Presented at the 8th International Conference on Functional Mapping of the Human Brain, Sendai, Japan. 

  17. Britton, J. C., Taylor, S. F., Sudheimer, K. D., & Liberzon, I. (2006). Facial expressions and complex IAPS pictures: common and differential networks. NeuroImage, 31(2), 906–919. DOI: https://doi.org/10.1016/j.neuroimage.2005.12.050 

  18. Bukach, C. M., Gauthier, I., & Tarr, M. J. (2006). Beyond faces and modularity: the power of an expertise framework. Trends in Cognitive Sciences, 10(4), 159–166. DOI: https://doi.org/10.1016/j.tics.2006.02.004 

  19. Byrne, A., & Eysenck, M. W. (1993). Individual differences in positive and negative interpretive biases. Personality and Individual Differences, 14(6), 849–851. DOI: https://doi.org/10.1016/0191-8869(93)90100-H 

  20. Cacioppo, J. T., & Berntson, G. G. (1994). Relationship between Attitudes and Evaluative Space: A Critical Review, with Emphasis on the Separability of Positive and Negative Substrates. Psychological Bulletin, 115(3), 401–423. DOI: https://doi.org/10.1037/0033-2909.115.3.401 

  21. Cacioppo, J. T., & Gardner, W. L. (1999). Emotion. Annual Review of Psychology, 50, 191–214. DOI: https://doi.org/10.1146/annurev.psych.50.1.191 

  22. Cacioppo, J. T., Gardner, W. L., & Berntson, G. G. (1997). Beyond Bipolar Conceptualizations and Measures: The Case of Attitudes and Evaluative Space. Personality and Social Psychology Review, 1(1), 3–25. DOI: https://doi.org/10.1207/s15327957pspr0101_2 

  23. Calder, A. J., & Young, A. W. (2005). Understanding the recognition of facial identity and facial expression. Nature Reviews. Neuroscience, 6(8), 641–651. DOI: https://doi.org/10.1038/nrn1724 

  24. Carlson, T., Grol, M. J., & Verstraten, F. A. J. (2006). Dynamics of visual recognition revealed by fMRI. Neuroimage, 32(2), 892–905. DOI: https://doi.org/10.1016/j.neuroimage.2006.03.059 

  25. Carretié, L. (2014). Exogenous (automatic) attention to emotional stimuli: a review. Cognitive, Affective & Behavioral Neuroscience, 14(4), 1228–1258. DOI: https://doi.org/10.3758/s13415-014-0270-2 

  26. Carver, C. S., Johnson, S. L., & Joormann, J. (2009). Two-Mode Models of Self-Regulation as a Tool for Conceptualizing Effects of the Serotonin System in Normal Behavior and Diverse Disorders. Current Directions in Psychological Science, 18(4), 195–199. DOI: https://doi.org/10.1111/j.1467-8721.2009.01635.x 

  27. Codispoti, M., Mazzetti, M., & Bradley, M. M. (2009). Unmasking emotion: Exposure duration and emotional engagement. Psychophysiology, 46(4), 731–738. DOI: https://doi.org/10.1111/j.1469-8986.2009.00804.x 

  28. Costa, T., Cauda, F., Crini, M., Tatu, M.-K., Celeghin, A., de Gelder, B., & Tamietto, M. (2014). Temporal and spatial neural dynamics in the perception of basic emotions from complex scenes. Social Cognitive and Affective Neuroscience, 9(11), 1690–1703. DOI: https://doi.org/10.1093/scan/nst164 

  29. Cravo, A. M., Haddad, H., Claessens, P. M. E., & Baldo, M. V. C. (2013). Bias and learning in temporal binding: intervals between actions and outcomes are compressed by prior bias. Consciousness and Cognition, 22(4), 1174–1180. DOI: https://doi.org/10.1016/j.concog.2013.08.001 

  30. De Caro, S. A., & Reeves, A. (2000). Rotating objects to determine orientation, not identity: evidence from a backward-masking/dual-task procedure. Perception & Psychophysics, 62(7), 1356–1366. DOI: https://doi.org/10.3758/BF03212138 

  31. Delorme, A., & Makeig, S. (2004). EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. Journal of Neuroscience Methods, 134(1), 9–21. DOI: https://doi.org/10.1016/j.jneumeth.2003.10.009 

  32. Desimone, R., & Duncan, J. (1995). Neural Mechanisms of Selective Visual Attention. Annual Review of Neuroscience, 18, 193–222. DOI: https://doi.org/10.1146/annurev.ne.18.030195.001205 

  33. Ding, J., Sperling, G., & Srinivasan, R. (2006). Attentional modulation of SSVEP power depends on the network tagged by the flicker frequency. Cerebral Cortex, 16(7), 1016–1029. DOI: https://doi.org/10.1093/cercor/bhj044 

  34. Dzhelyova, M., Jacques, C., & Rossion, B. (2017). At a Single Glance: Fast Periodic Visual Stimulation Uncovers the Spatio-Temporal Dynamics of Brief Facial Expression Changes in the Human Brain. Cerebral Cortex, 27(8), 4106–4123. DOI: https://doi.org/10.1093/cercor/bhw223 

  35. Eickhoff, S. B., Stephan, K. E., Mohlberg, H., Grefkes, C., Fink, G. R., Amunts, K., & Zilles, K. (2005). A new SPM toolbox for combining probabilistic cytoarchitectonic maps and functional imaging data. NeuroImage, 25(4), 1325–1335. DOI: https://doi.org/10.1016/j.neuroimage.2004.12.034 

  36. Fabre-Thorpe, M., Delorme, A., Marlot, C., & Thorpe, S. (2001). A limit to the speed of processing in ultra-rapid visual categorization of novel natural scenes. Journal of Cognitive Neuroscience, 13(2), 171–180. DOI: https://doi.org/10.1162/089892901564234 

  37. Field, A. P., Miles, J. N. V., & Field, Z. C. (2012). Discovering statistics using R: and sex and drugs and rock “n” roll. London: Sage Publications Ltd. 

  38. Field, A. P., & Wilcox, R. R. (2017). Robust statistical methods: A primer for clinical psychology and experimental psychopathology researchers. Behaviour Research and Therapy, 98, 19–38. DOI: https://doi.org/10.1016/j.brat.2017.05.013 

  39. Frijda, N. H. (2016). The evolutionary emergence of what we call “emotions.” Cognition & Emotion, 30(4), 609–620. DOI: https://doi.org/10.1080/02699931.2016.1145106 

  40. Friston, K., Harrison, L., Daunizeau, J., Kiebel, S., Phillips, C., Trujillo-Barreto, N., Mattout, J., et al. (2008). Multiple sparse priors for the M/EEG inverse problem. NeuroImage, 39(3), 1104–1120. DOI: https://doi.org/10.1016/j.neuroimage.2007.09.048 

  41. Frith, C. (2009). Role of facial expressions in social interactions. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 364(1535), 3453–3458. DOI: https://doi.org/10.1098/rstb.2009.0142 

  42. Fritz, C. O., Morris, P. E., & Richler, J. J. (2012). Effect size estimates: current use, calculations, and interpretation. Journal of Experimental Psychology. General, 141(1), 2–18. DOI: https://doi.org/10.1037/a0024338 

  43. Gauthier, I., & Nelson, C. A. (2001). The development of face expertise. Current Opinion in Neurobiology, 11(2), 219–224. DOI: https://doi.org/10.1016/S0959-4388(00)00200-2 

  44. Gerlanc, D., & Kirby, K. (2015). bootES: Bootstrap Effect Sizes. Retrieved from https://CRAN.R-project.org/package=bootES 

  45. Gollan, J. K., Hoxha, D., Hunnicutt-Ferguson, K., Norris, C. J., Rosebrock, L., Sankin, L., & Cacioppo, J. (2016). Twice the Negativity Bias and Half the Positivity Offset: Evaluative Responses to Emotional Information in Depression. Journal of Behavior Therapy and Experimental Psychiatry, 52, 166–170. DOI: https://doi.org/10.1016/j.jbtep.2015.09.005 

  46. Grech, R., Cassar, T., Muscat, J., Camilleri, K., Fabri, S., Zervakis, M., Vanrumste, B., et al. (2008). Review on solving the inverse problem in EEG source analysis. Journal of NeuroEngineering and Rehabilitation, 5(1), 25. DOI: https://doi.org/10.1186/1743-0003-5-25 

  47. Grill-Spector, K., Kourtzi, Z., & Kanwisher, N. (2001). The lateral occipital complex and its role in object recognition. Vision Research, 41(10–11), 1409–1422. DOI: https://doi.org/10.1016/S0042-6989(01)00073-6 

  48. Grill-Spector, K., Kushnir, T., Hendler, T., & Malach, R. (2000). The dynamics of object-selective activation correlate with recognition performance in humans. Nature Neuroscience, 3(8), 837–843. DOI: https://doi.org/10.1038/77754 

  49. Gross, J., & Ligges, U. (2015). nortest: Tests for Normality. Retrieved from https://CRAN.R-project.org/package=nortest 

  50. Gundlach, C., & Müller, M. M. (2013). Perception of illusory contours forms intermodulation responses of steady state visual evoked potentials as a neural signature of spatial integration. Biological Psychology, 94(1), 55–60. DOI: https://doi.org/10.1016/j.biopsycho.2013.04.014 

  51. Hajcak, G., Dunning, J. P., & Foti, D. (2009). Motivated and controlled attention to emotion: time-course of the late positive potential. Clinical Neurophysiology: Official Journal of the International Federation of Clinical Neurophysiology, 120(3), 505–510. DOI: https://doi.org/10.1016/j.clinph.2008.11.028 

  52. Hajcak, G., MacNamara, A., & Olvet, D. M. (2010). Event-Related Potentials, Emotion, and Emotion Regulation: An Integrative Review. Developmental Neuropsychology, 35(2), 129–155. DOI: https://doi.org/10.1080/87565640903526504 

  53. Haxby, J. V., Hoffman, E. A., & Gobbini, M. I. (2000). The distributed human neural system for face perception. Trends in Cognitive Sciences, 4(6), 223–233. DOI: https://doi.org/10.1016/S1364-6613(00)01482-0 

  54. Hindi Attar, C., Andersen, S. K., & Müller, M. M. (2010). Time course of affective bias in visual attention: convergent evidence from steady-state visual evoked potentials and behavioral data. Neuroimage, 53(4), 1326–33. DOI: https://doi.org/10.1016/j.neuroimage.2010.06.074 

  55. Hindi Attar, C., & Müller, M. M. (2012). Selective attention to task-irrelevant emotional distractors is unaffected by the perceptual load associated with a foreground task. PloS One, 7(5), e37186. DOI: https://doi.org/10.1371/journal.pone.0037186 

  56. Holm, S. (1979). A Simple Sequentially Rejective Multiple Test Procedure. Scandinavian Journal of Statistics, 6(2), 65–70. 

  57. Hope, R. M. (2013). Rmisc: Ryan Miscellaneous. Retrieved from https://CRAN.R-project.org/package=Rmisc 

  58. Ito, T. A., & Cacioppo, J. T. (2005). Variations on a human universal: Individual differences in positivity offset and negativity bias. Cognition & Emotion, 19(1), 1–26. DOI: https://doi.org/10.1080/02699930441000120 

  59. Ito, T. A., Cacioppo, J. T., & Lang, P. J. (1998). Eliciting affect using the international affective picture system: Trajectories through evaluative space. Personality and Social Psychology Bulletin, 24(8), 855–879. DOI: https://doi.org/10.1177/0146167298248006 

  60. Jacques, C., Retter, T. L., & Rossion, B. (2016). A single glance at natural face images generate larger and qualitatively different category-selective spatio-temporal signatures than other ecologically-relevant categories in the human brain. NeuroImage, 137, 21–33. DOI: https://doi.org/10.1016/j.neuroimage.2016.04.045 

  61. Jeffreys, H. (1961). Theory of probability. Oxford, England: Oxford University Press. 

  62. Johnson, J. S., & Olshausen, B. A. (2003). Timecourse of neural signatures of object recognition. 499–512. DOI: https://doi.org/10.1167/3.7.4 

  63. Junghöfer, M., Bradley, M. M., Elbert, T. R., & Lang, P. J. (2001). Fleeting images: A new look at early emotion discrimination. Psychophysiology, 38(2), 175–178. DOI: https://doi.org/10.1111/1469-8986.3820175 

  64. Kass, R. E., & Raftery, A. E. (1995). Bayes Factors. Journal of the American Statistical Association, 90(430), 773–795. DOI: https://doi.org/10.1080/01621459.1995.10476572 

  65. Keil, A., Gruber, T., Müller, M. M., Moratti, S., Stolarova, M., Bradley, M. M., & Lang, P. J. (2003). Early modulation of visual perception by emotional arousal: Evidence from steady-state visual evoked brain potentials. Cognitive, Affective & Behavioral Neuroscience, 3(3), 195–206. DOI: https://doi.org/10.3758/CABN.3.3.195 

  66. Keil, A., Sabatinelli, D., Ding, M., Lang, P. J., Ihssen, N., & Heim, S. (2009). Re-entrant projections modulate visual cortex in affective perception: evidence from Granger causality analysis. Human Brain Mapping, 30(2), 532–40. DOI: https://doi.org/10.1002/hbm.20521 

  67. Keil, A., Smith, J. C., Wangelin, B. C., Sabatinelli, D., Bradley, M. M., & Lang, P. J. (2008). Electrocortical and electrodermal responses covary as a function of emotional arousal: a single-trial analysis. Psychophysiology, 45(4), 516–23. DOI: https://doi.org/10.1111/j.1469-8986.2008.00667.x 

  68. Keil, M. S. (2008). Does face image statistics predict a preferred spatial frequency for human face processing? Proceedings. Biological Sciences, 275(1647), 2095–2100. DOI: https://doi.org/10.1098/rspb.2008.0486 

  69. Keysers, C., & Perrett, D. I. (2002). Visual masking and RSVP reveal neural competition. Trends in Cognitive Sciences, 6(3), 120–125. DOI: https://doi.org/10.1016/S1364-6613(00)01852-0 

  70. Kim, Y. J., Grabowecky, M., Paller, K. A., Muthu, K., & Suzuki, S. (2007). Attention induces synchronization-based response gain in steady-state visual evoked potentials. Nature Neuroscience, 10(1), 117–125. DOI: https://doi.org/10.1038/nn1821 

  71. Kruschke, J. K. (2010). What to believe: Bayesian methods for data analysis. Trends in Cognitive Sciences, 14(7), 293–300. DOI: https://doi.org/10.1016/j.tics.2010.05.001 

  72. Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4(November), 863. DOI: https://doi.org/10.3389/fpsyg.2013.00863 

  73. Lang, P. J., & Bradley, M. M. (2010). Emotion and the motivational brain. Biological Psychology, 84(3), 437–450. DOI: https://doi.org/10.1016/j.biopsycho.2009.10.007 

  74. Lang, P. J., Bradley, M. M., & Cuthbert, B. N. (2008). International affective picture system (IAPS): Affective ratings of pictures and instruction manual. Technical Report A-8. Gainesville, FL: University of Florida. 

  75. LeDoux, J. (2007). The amygdala. Current Biology, 17(20), R868–R874. DOI: https://doi.org/10.1016/j.cub.2007.08.005 

  76. Lehmann, D., & Skrandies, W. (1984). Spatial analysis of evoked potentials in man–a review. Progress in Neurobiology, 23(3), 227–250. DOI: https://doi.org/10.1016/0301-0082(84)90003-0 

  77. Lithari, C., Frantzidis, C. A., Papadelis, C., Vivas, A. B., Klados, M. A., Kourtidou-Papadeli, C., Bamidis, P. D., et al. (2010). Are Females More Responsive to Emotional Stimuli? A Neurophysiological Study Across Arousal and Valence Dimensions. Brain Topography, 23(1), 27–40. DOI: https://doi.org/10.1007/s10548-009-0130-5 

  78. Litvak, V., Mattout, J., Kiebel, S., Phillips, C., Henson, R., Kilner, J., Friston, K., et al. (2011). EEG and MEG data analysis in SPM8. Computational Intelligence and Neuroscience, 2011, 852961. DOI: https://doi.org/10.1155/2011/852961 

  79. Liu, H., Agam, Y., Madsen, J. R., & Kreiman, G. (2009). Timing, timing, timing: fast decoding of object information from intracranial field potentials in human visual cortex. Neuron, 62(2), 281–290. DOI: https://doi.org/10.1016/j.neuron.2009.02.025 

  80. López, J. D., Litvak, V., Espinosa, J. J., Friston, K., & Barnes, G. R. (2014). Algorithmic procedures for Bayesian MEG/EEG source reconstruction in SPM. NeuroImage, 84, 476–487. DOI: https://doi.org/10.1016/j.neuroimage.2013.09.002 

  81. López, J. D., Penny, W. D., Espinosa, J. J., & Barnes, G. R. (2012). A general Bayesian treatment for MEG source reconstruction incorporating lead field uncertainty. NeuroImage, 60(2), 1194–1204. DOI: https://doi.org/10.1016/j.neuroimage.2012.01.077 

  82. Mair, P., Schönbrodt, F., & Wilcox, R. (2017). WRS2: Wilcox robust estimation and testing. Retrieved from https://cran.r-project.org/web/packages/WRS2/ 

  83. Michel, C. M., & Murray, M. M. (2012). Towards the utilization of EEG as a brain imaging tool. Neuroimage, 61(2), 371–385. DOI: https://doi.org/10.1016/j.neuroimage.2011.12.039 

  84. Morey, R. D., Romeijn, J.-W., & Rouder, J. N. (2016). The philosophy of Bayes factors and the quantification of statistical evidence. Journal of Mathematical Psychology, 72, 6–18. DOI: https://doi.org/10.1016/j.jmp.2015.11.001 

  85. Morey, R. D., & Rouder, J. N. (2015). BayesFactor: Computation of Bayes factors for common designs. Retrieved from https://CRAN.R-project.org/package=BayesFactor 

  86. Müller, K. (2017). here: A Simpler Way to Find Your Files. Retrieved from https://CRAN.R-project.org/package=here 

  87. Müller, M. M., Andersen, S. K., & Hindi Attar, C. (2011). Attentional bias to briefly presented emotional distractors follows a slow time course in visual cortex. Journal of Neuroscience, 31(44), 15914–8. DOI: https://doi.org/10.1523/JNEUROSCI.1997-11.2011 

  88. Müller, M. M., & Hübner, R. (2002). Can the spotlight of attention be shaped like a doughnut? Evidence from steady-state visual evoked potentials. Psychological Science, 13(2), 119–124. DOI: https://doi.org/10.1111/1467-9280.00422 

  89. Müller, M. M., Malinowski, P., Gruber, T., & Hillyard, S. A. (2003). Sustained division of the attentional spotlight. Nature, 424(6946), 309–12. DOI: https://doi.org/10.1038/nature01812 

  90. Murray, J. E. (1997). Flipping and spinning: spatial transformation procedures in the identification of rotated natural objects. Memory & Cognition, 25(1), 96–105. DOI: https://doi.org/10.3758/BF03197287 

  91. Murray, M. M., Brunet, D., & Michel, C. M. (2008). Topographic ERP analyses: A step-by-step tutorial review. Brain Topography, 20(4), 249–264. DOI: https://doi.org/10.1007/s10548-008-0054-5 

  92. Nasr, S., Liu, N., Devaney, K. J., Yue, X., Rajimehr, R., Ungerleider, L. G., & Tootell, R. B. H. (2011). Scene-selective cortical regions in human and nonhuman primates. The Journal of Neuroscience, 31(39), 13771–13785. DOI: https://doi.org/10.1523/JNEUROSCI.2792-11.2011 

  93. Niedenthal, P. M., Halberstadt, J. B., & Setterlund, M. B. (1997). Being Happy and Seeing “Happy”: Emotional State Mediates Visual Word Recognition. Cognition & Emotion, 11, 403–432. DOI: https://doi.org/10.1080/026999397379863 

  94. Niedenthal, P. M., & Setterlund, M. B. (1994). Emotion Congruence in Perception. Personality and Social Psychology Bulletin, 20(4), 401–411. DOI: https://doi.org/10.1177/0146167294204007 

  95. Nolan, H., Whelan, R., & Reilly, R. B. (2010). FASTER: Fully Automated Statistical Thresholding for EEG artifact Rejection. Journal of Neuroscience Methods, 192(1), 152–62. DOI: https://doi.org/10.1016/j.jneumeth.2010.07.015 

  96. Norcia, A. M., Appelbaum, L. G., Ales, J. M., Cottereau, B. R., & Rossion, B. (2015). The steady-state visual evoked potential in vision research: A review. Journal of Vision, 15(6), 4. DOI: https://doi.org/10.1167/15.6.4 

  97. Norris, C. J., Gollan, J., Berntson, G. G., & Cacioppo, J. T. (2010). The current status of research on the structure of evaluative space. Biological Psychology, 84(3), 422–436. DOI: https://doi.org/10.1016/j.biopsycho.2010.03.011 

  98. Norris, C. J., Larsen, J. T., Crawford, L. E., & Cacioppo, J. T. (2011). Better (or worse) for some than others: Individual differences in the positivity offset and negativity bias. Journal of Research in Personality, 45(1), 100–111. DOI: https://doi.org/10.1016/j.jrp.2010.12.001 

  99. Öhman, A., Hamm, A., & Hugdahl, K. (2000). Cognition and the autonomic nervous system: Orienting, anticipation, and conditioning. In J. T. Cacioppo, L. Tassinary, & G. G. Berntson (Eds.), Handbook of Psychophysiology (pp. 533–575). New York: Cambridge University Press. 

  100. Olofsson, J. K., Nordin, S., Sequeira, H., & Polich, J. (2008). Affective picture processing: An integrative review of ERP findings. Biological Psychology, 77(3), 247–265. DOI: https://doi.org/10.1016/j.biopsycho.2007.11.006 

  101. Oostenveld, R., & Praamstra, P. (2001). The five percent electrode system for high-resolution EEG and ERP measurements. Clinical Neurophysiology, 112(4), 713–719. DOI: https://doi.org/10.1016/S1388-2457(00)00527-7 

  102. Perrin, F., Pernier, J., Bertrand, O., & Echallier, J. F. (1989). Spherical Splines for Scalp Potential and Current-Density Mapping. Electroencephalography and Clinical Neurophysiology, 72(2), 184–187. DOI: https://doi.org/10.1016/0013-4694(89)90180-6 

  103. Pessoa, L. (2008). On the relationship between emotion and cognition. Nature Reviews Neuroscience, 9(2), 148–158. DOI: https://doi.org/10.1038/nrn2317 

  104. Peters, G. (2017). userfriendlyscience: Quantitative analysis made accessible. Retrieved from http://userfriendlyscience.com 

  105. Peyk, P., Schupp, H. T., Keil, A., Elbert, T., & Junghöfer, M. (2009). Parallel processing of affective visual stimuli. Psychophysiology, 46(1), 200–208. DOI: https://doi.org/10.1111/j.1469-8986.2008.00755.x 

  106. Phelps, E. A., & LeDoux, J. E. (2005). Contributions of the amygdala to emotion processing: from animal models to human behavior. Neuron, 48(2), 175–187. DOI: https://doi.org/10.1016/j.neuron.2005.09.025 

  107. Phillips, N. (2017). yarrr: A Companion to the e-Book “YaRrr!: The Pirate’s Guide to R.” Retrieved from https://CRAN.R-project.org/package=yarrr 

  108. Ploran, E. J., Tremel, J. J., Nelson, S. M., & Wheeler, M. E. (2011). High Quality but Limited Quantity Perceptual Evidence Produces Neural Accumulation in Frontal and Parietal Cortex. Cerebral Cortex, 21(11), 2650–2662. DOI: https://doi.org/10.1093/cercor/bhr055 

  109. Pool, E., Brosch, T., Delplanque, S., & Sander, D. (2016). Attentional bias for positive emotional stimuli: A meta-analytic investigation. Psychological Bulletin, 142(1), 79–106. DOI: https://doi.org/10.1037/bul0000026 

  110. Pourtois, G., Schettino, A., & Vuilleumier, P. (2013). Brain mechanisms for emotional influences on perception and attention: what is magic and what is not. Biological Psychology, 92(3), 492–512. DOI: https://doi.org/10.1016/j.biopsycho.2012.02.007 

  111. Proverbio, A. M., Adorni, R., Zani, A., & Trestianu, L. (2009). Sex differences in the brain response to affective scenes with or without humans. Neuropsychologia, 47(12), 2374–2388. DOI: https://doi.org/10.1016/j.neuropsychologia.2008.10.030 

  112. Quek, G. L., & Rossion, B. (2017). Category-selective human brain processes elicited in fast periodic visual stimulation streams are immune to temporal predictability. Neuropsychologia, 104, 182–200. DOI: https://doi.org/10.1016/j.neuropsychologia.2017.08.010 

  113. Raila, H., Scholl, B. J., & Gruber, J. (2015). Seeing the world through rose-colored glasses: People who are happy and satisfied with life preferentially attend to positive stimuli. Emotion, 15(4), 449–462. DOI: https://doi.org/10.1037/emo0000049 

  114. R Core Team. (2017). R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing. 

  115. Regan, D. (1977). Steady-state evoked potentials. Journal of the Optical Society of America, 67(11), 1475–89. DOI: https://doi.org/10.1364/JOSA.67.001475 

  116. Retter, T. L., & Rossion, B. (2016). Uncovering the neural magnitude and spatio-temporal dynamics of natural image categorization in a fast visual stream. Neuropsychologia, 91, 9–28. DOI: https://doi.org/10.1016/j.neuropsychologia.2016.07.028 

  117. Reynolds, J. H., & Heeger, D. J. (2009). The Normalization Model of Attention. Neuron, 61(2), 168–185. DOI: https://doi.org/10.1016/j.neuron.2009.01.002 

  118. Rieger, J. W., Köchy, N., Schalk, F., Grüschow, M., & Heinze, H.-J. (2008). Speed limits: orientation and semantic context interactions constrain natural scene discrimination dynamics. Journal of Experimental Psychology. Human Perception and Performance, 34(1), 56–76. DOI: https://doi.org/10.1037/0096-1523.34.1.56 

  119. Rinker, T., & Kurkiewicz, D. (n.d.). pacman: Package Management for R. Retrieved from http://github.com/trinker/pacman 

  120. Robinson, D. (2017). broom: Convert Statistical Analysis Objects into Tidy Data Frames. Retrieved from https://CRAN.R-project.org/package=broom 

  121. Rorden, C., & Brett, M. (2000). Stereotaxic display of brain lesions. Behavioural Neurology, 12(4), 191–200. DOI: https://doi.org/10.1155/2000/421719 

  122. Rossion, B., Torfs, K., Jacques, C., & Liu-Shuang, J. (2015). Fast periodic presentation of natural images reveals a robust face-selective electrophysiological response in the human brain. Journal of Vision, 15(1), 15.1.18. DOI: https://doi.org/10.1167/15.1.18 

  123. Rouder, J. N., Engelhardt, C. R., McCabe, S., & Morey, R. D. (2016). Model comparison in ANOVA. Psychonomic Bulletin & Review, 23(6), 1779–1786. DOI: https://doi.org/10.3758/s13423-016-1026-5 

  124. Rouder, J. N., Morey, R. D., Speckman, P. L., & Province, J. M. (2012). Default Bayes factors for ANOVA designs. Journal of Mathematical Psychology, 56(5), 356–374. DOI: https://doi.org/10.1016/j.jmp.2012.08.001 

  125. Rouder, J. N., Morey, R. D., Verhagen, J., Province, J. M., & Wagenmakers, E.-J. (2016). Is There a Free Lunch in Inference? Topics in Cognitive Science, 8(3), 520–547. DOI: https://doi.org/10.1111/tops.12214 

  126. Rouder, J. N., Morey, R. D., Verhagen, J., Swagman, A. R., & Wagenmakers, E.-J. (2017). Bayesian analysis of factorial designs. Psychological Methods, 22(2), 304–321. DOI: https://doi.org/10.1037/met0000057 

  127. Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009). Bayesian t tests for accepting and rejecting the null hypothesis. Psychonomic Bulletin & Review, 16(2), 225–237. DOI: https://doi.org/10.3758/PBR.16.2.225 

  128. Rousselet, G. A., Macé, M. J.-M., & Fabre-Thorpe, M. (2003). Is it an animal? Is it a human face? Fast processing in upright and inverted natural scenes. Journal of Vision, 3(6), 440–455. DOI: https://doi.org/10.1167/3.6.5 

  129. RStudio Team. (2015). RStudio: Integrated Development for R. Retrieved from http://www.rstudio.com/ 

  130. Russell, J. A. (1980). A Circumplex Model of Affect. Journal of Personality and Social Psychology, 39(6), 1161–1178. DOI: https://doi.org/10.1037/h0077714 

  131. Sabatinelli, D., Flaisch, T., Bradley, M. M., Fitzsimmons, J. R., & Lang, P. J. (2004). Affective picture perception: gender differences in visual cortex? Neuroreport, 15(7), 1109–1112. DOI: https://doi.org/10.1097/00001756-200405190-00005 

  132. Sabatinelli, D., Fortune, E. E., Li, Q., Siddiqui, A., Krafft, C., Oliver, W. T., Jeffries, J., et al. (2011). Emotional perception: meta-analyses of face and natural scene processing. Neuroimage, 54(3), 2524–33. DOI: https://doi.org/10.1016/j.neuroimage.2010.10.011 

  133. Said, C. P., Haxby, J. V., & Todorov, A. (2011). Brain systems for assessing the affective value of faces. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 366(1571), 1660–1670. DOI: https://doi.org/10.1098/rstb.2010.0351 

  134. Schettino, A., Keil, A., Porcu, E., & Müller, M. M. (2016). Shedding light on emotional perception: Interaction of brightness and semantic content in extrastriate visual cortex. NeuroImage, 133, 341–353. DOI: https://doi.org/10.1016/j.neuroimage.2016.03.020 

  135. Schettino, A., Loeys, T., Bossi, M., & Pourtois, G. (2012). Valence-specific modulation in the accumulation of perceptual evidence prior to visual scene recognition. PloS One, 7(5), e38064. DOI: https://doi.org/10.1371/journal.pone.0038064 

  136. Schettino, A., Loeys, T., Delplanque, S., & Pourtois, G. (2011). Brain dynamics of upstream perceptual processes leading to visual object recognition: a high density ERP topographic mapping study. NeuroImage, 55(3), 1227–1241. DOI: https://doi.org/10.1016/j.neuroimage.2011.01.009 

  137. Schettino, A., Loeys, T., & Pourtois, G. (2013). Multiple synergistic effects of emotion and memory on proactive processes leading to scene recognition. NeuroImage, 81, 81–95. DOI: https://doi.org/10.1016/j.neuroimage.2013.04.115 

  138. Schönbrodt, F. D., Wagenmakers, E.-J., Zehetleitner, M., & Perugini, M. (2017). Sequential hypothesis testing with Bayes factors: Efficiently testing mean differences. Psychological Methods, 22(2), 322–339. DOI: https://doi.org/10.1037/met0000061 

  139. Schupp, H. T., Flaisch, T., Stockburger, J., & Junghöfer, M. (2006). Emotion and attention: event-related brain potential studies. Progress in Brain Research, 156, 31–51. DOI: https://doi.org/10.1016/S0079-6123(06)56002-9 

  140. Schupp, H. T., Junghöfer, M., Weike, A. I., & Hamm, A. O. (2003a). Attention and emotion: an ERP analysis of facilitated emotional stimulus processing. Neuroreport, 14(8), 1107–1110. DOI: https://doi.org/10.1097/00001756-200306110-00002 

  141. Schupp, H. T., Junghöfer, M., Weike, A. I., & Hamm, A. O. (2003b). Emotional facilitation of sensory processing in the visual cortex. Psychological Science, 14(1), 7–13. DOI: https://doi.org/10.1111/1467-9280.01411 

  142. Schupp, H. T., Junghöfer, M., Weike, A. I., & Hamm, A. O. (2004). The selective processing of briefly presented affective pictures: An ERP analysis. Psychophysiology, 41(3), 441–449. DOI: https://doi.org/10.1111/j.1469-8986.2004.00174.x 

  143. Sennwald, V., Pool, E., Brosch, T., Delplanque, S., Bianchi-Demicheli, F., & Sander, D. (2016). Emotional attention for erotic stimuli: Cognitive and brain mechanisms. The Journal of Comparative Neurology, 524(8), 1668–1675. DOI: https://doi.org/10.1002/cne.23859 

  144. Summerfield, C., & de Lange, F. P. (2014). Expectation in perceptual decision making: neural and computational mechanisms. Nature Reviews Neuroscience, 15(11), 745–756. DOI: https://doi.org/10.1038/nrn3838 

  145. Summerfield, C., & Egner, T. (2009). Expectation (and attention) in visual cognition. Trends in Cognitive Sciences, 13(9), 403–409. DOI: https://doi.org/10.1016/j.tics.2009.06.003 

  146. Sutoyo, D., & Srinivasan, R. (2009). Nonlinear SSVEP responses are sensitive to the perceptual binding of visual hemifields during conventional “eye” rivalry and interocular “percept” rivalry. Brain Research, 1251, 245–255. DOI: https://doi.org/10.1016/j.brainres.2008.09.086 

  147. Tanaka, J. W. (2001). The entry point of face recognition: evidence for face expertise. Journal of Experimental Psychology. General, 130(3), 534–543. DOI: https://doi.org/10.1037/0096-3445.130.3.534 

  148. Tarr, M. J., & Gauthier, I. (2000). FFA: a flexible fusiform area for subordinate-level visual processing automatized by expertise. Nature Neuroscience, 3(8), 764–769. DOI: https://doi.org/10.1038/77666 

  149. Theeuwes, J. (1994). Endogenous and Exogenous Control of Visual Selection. Perception, 23(4), 429–440. DOI: https://doi.org/10.1068/p230429 

  150. Thorpe, S., Fize, D., & Marlot, C. (1996). Speed of processing in the human visual system. Nature, 381(6582), 520–522. DOI: https://doi.org/10.1038/381520a0 

  151. VanRullen, R. (2006). On second glance: still no high-level pop-out effect for faces. Vision Research, 46(18), 3017–3027. DOI: https://doi.org/10.1016/j.visres.2005.07.009 

  152. VanRullen, R., & Thorpe, S. J. (2001a). Is it a bird? Is it a plane? Ultra-rapid visual categorisation of natural and artifactual objects. Perception, 30(6), 655–668. DOI: https://doi.org/10.1068/p3029 

  153. VanRullen, R., & Thorpe, S. J. (2001b). The time course of visual processing: From early perception to decision-making. Journal of Cognitive Neuroscience, 13(4), 454–461. DOI: https://doi.org/10.1162/08989290152001880 

  154. Vialatte, F.-B., Maurice, M., Dauwels, J., & Cichocki, A. (2010). Steady-state visually evoked potentials: focus on essential paradigms and future perspectives. Progress in Neurobiology, 90(4), 418–38. DOI: https://doi.org/10.1016/j.pneurobio.2009.11.005 

  155. Vuilleumier, P. (2005). How brains beware: neural mechanisms of emotional attention. Trends in Cognitive Sciences, 9(12), 585–594. DOI: https://doi.org/10.1016/j.tics.2005.10.011 

  156. Vuong, Q. C., Hof, A. F., Bülthoff, H. H., & Thornton, I. M. (2006). An advantage for detecting dynamic targets in natural scenes. Journal of Vision, 6(1), 87–96. DOI: https://doi.org/10.1167/6.1.8 

  157. Wagenmakers, E.-J. (2007). A practical solution to the pervasive problems of p values. Psychonomic Bulletin & Review, 14(5), 779–804. DOI: https://doi.org/10.3758/BF03194105 

  158. Wessa, M., Kanske, P., Neumeister, P., Bode, K., Heissler, J., & Schönfelder, S. (2010). EmoPics: Subjektive und psychophysiologische Evaluationen neuen Bildmaterials für die klinisch-biopsychologische Forschung. Zeitschrift Für Klinische Psychologie Und Psychotherapie, 39(1/11), 77. 

  159. Wickham, H. (2009). ggplot2: Elegant Graphics for Data Analysis. Retrieved from http://ggplot2.org. DOI: https://doi.org/10.1007/978-0-387-98141-3 

  160. Wickham, H. (2017a). scales: Scale Functions for Visualization. Retrieved from https://CRAN.R-project.org/package=scales 

  161. Wickham, H. (2017b). tidyverse: Easily Install and Load the “Tidyverse.” Retrieved from https://CRAN.R-project.org/package=tidyverse 

  162. Wieser, M. J., Miskovic, V., & Keil, A. (2016). Steady-state visual evoked potentials as a research tool in social affective neuroscience. Psychophysiology, 53(12), 1763–1775. DOI: https://doi.org/10.1111/psyp.12768 

  163. Wood, S. N. (2017). Generalized Additive Models: An Introduction with R (2nd ed.). Chapman and Hall/CRC. DOI: https://doi.org/10.1201/9781315370279 

  164. Xie, Y. (2018). knitr: A General-Purpose Package for Dynamic Report Generation in R. Retrieved from https://cran.r-project.org/web/packages/knitr/index.html 

  165. Zhu, H. (2018). kableExtra: Construct Complex Table with “kable” and Pipe Syntax. Retrieved from https://CRAN.R-project.org/package=kableExtra 

Peer Review Comments

The author(s) of this paper chose the Open Review option, and the peer review comments are available at: http://doi.org/10.1525/collabra.226.pr

comments powered by Disqus