For over thirty years, cognitive scientists have examined the fact that people often re-use syntactic constructions across sentences. This tendency, alternately called syntactic priming, structural priming, or structural persistence (see Pickering & Ferreira, 2008), was noted in studies reported by Levelt and Kelter (1982) and Weiner and Labov (1983), and became established as an important psycholinguistic phenomenon through Kay Bock’s (1986; 1989) pioneering work. In a typical experiment (e.g., Bock, 1986; Pickering & Branigan, 1998), structural priming is induced through the use of prime sentences. Participants are asked to produce or comprehend prime sentences with a given syntactic form, such as the double object construction (DO; Michael sent Meghan a postcard). Subsequent to this, participants generate a target sentence that allows for the possibility of employing the syntactic form used in the prime sentence (in this case, a double object: Tim handed Heather the book) or using an alternate form (in this case, a prepositional object [PO] form: Tim handed the book to Heather). Structural priming is the finding that participants show increased odds of generating a DO target sentence following a DO prime, and an increased odds of generating a PO target sentence following a PO prime.

Structural priming is a robust phenomenon in language use (see Mahowald, James, Futrell, & Gibson, 2016, and Pickering & Ferreira, 2008, for reviews). It occurs in language production (e.g., Bock, 1986; Pickering & Branigan, 1998), language comprehension (e.g., Branigan, Pickering, & McLean, 2005; Kaschak & Glenberg, 2004), and between comprehension and production (e.g., Bock, Dell, Chang, & Onishi, 2007). It occurs in spoken language (e.g., Bock, 1986; Bock & Griffin, 2000), written language (e.g., Kaschak, Kutta & Coyle, 2014; Pickering & Branigan, 1998), and between spoken and written language (e.g., Cleland & Pickering, 2006). Structural priming can be captured in the lab (e.g., Bock, 1986; Branigan et al., 2005) as well as in naturally occurring speech (e.g., Gries, 2005; Weiner & Labov, 1983). Structural priming has also been discussed for the role that it might play in language acquisition (e.g., Brooks & Tomasello, 1999; Huttenlocher, Vasilyeva, & Shimpi, 2004; McDonough & Mackey, 2008), language users’ adaptations to features of their linguistic environment (e.g., Jaeger & Snider, 2013; Kaschak & Glenberg, 2004), and language change (e.g., Bock & Griffin, 2000). Because structural priming is such a common and far-reaching element of language use, understanding the mechanisms that produce structural repetition may be quite useful in constraining our theories of language use.

In the past ten years, a consensus approach to structural priming has emerged (e.g., Chang, Dell, & Bock, 2006; Hartsuiker, Bernolet, Schoonbaert, Speybroeck, & Vanderelst, 2008; Reitter, Keller, & Moore, 2011). The basic structural priming effect is taken to reflect ongoing implicit learning in the language processing systems (e.g., Chang et al., 2006; Ferreira, Bock, Wilson, & Cohen, 2008; Jaeger & Snider, 2013). As one example of the implicit learning account, Chang et al., (2006) demonstrate that error-based learning in connectionist models can simulate a wide range of phenomena observed in studies of structural priming. Structural priming effects are amplified in cases where lexical items, such as verbs, are re-used in prime and target sentences (e.g., Hartsuiker et al., 2008; Scheepers, Raffray & Myachykov, 2017). This finding, which has been dubbed the lexical boost (e.g., Hartsuiker et al., 2008; Pickering & Branigan, 1998), is taken to reflect the influence of short-lived explicit memory processes. Repeating words across prime and target sentences raises the odds of the prime being recalled when the target is produced, thus raising the odds of the target sentence and the prime sentence having the same syntactic form. Thus, the consensus approach to structural priming is what we might call the two-mechanism account – the basic priming effect arises from implicit learning processes, and the temporary increase in priming strength seen when words are repeated is the result of explicit memory processes. The two-mechanism account has been modeled by Reitter et al. (2011), and discussed by Chang et al. (2006), Hartsuiker et al. (2008), and Scheepers et al. (2017).

The empirical case for the two-mechanism account is strong. The claim that structural priming is driven by implicit learning processes rests largely on three pieces of evidence. First, structural priming effects can last a long time. Structural priming remains intact with (at least) 10 filler items separating the prime and target (Bock & Griffin, 2000; Bock et al., 2007; but see Bernolet, Collina & Hartsuiker, 2016, for evidence that structural priming is not always so long-lasting). The accumulation of structural priming within an experiment can last for at least one week (e.g., Kaschak, Kutta, & Coyle, 2014; Kaschak, Kutta & Schatschneider, 2011; Kutta & Kaschak, 2012). Second, structural priming is preserved in anterograde amnesiacs (Ferreira et al., 2008), who show impaired explicit memory performance but intact procedural (implicit) learning performance. Thus, structural priming can be observed in the absence of explicit memory for the sentences in the experiments (e.g., Bock et al., 2007; Ferreira et al., 2008). Third, several studies report an inverse relationship between construction frequency and the magnitude of priming seen for those constructions (e.g., Jaeger & Snider, 2013; Kaschak et al., 2011). For example, the DO construction has a higher frequency than the PO construction for speakers of American English (e.g., Bock & Griffin, 2000). The less-frequent PO construction shows stronger priming compared to the more-frequent DO construction (e.g., Kaschak et al., 2011). This inverse frequency effect falls out naturally from implicit learning approaches to structural priming (e.g., Chang et al., 2006; Jaeger & Snider, 2013): structures that are more frequent are less “surprising” to the language system (in Jaeger & Snider’s, 2013, terms), or produce less prediction error in the system (in Chang et al., 2006’s, terms), and therefore produce weaker priming effects.

The effect of explicit memory processes on structural priming is demonstrated in two ways. First, in contrast to the long-lived structural priming reported by Bock and Griffin (2000), the lexical boost is short-lived, disappearing when even a small number of sentences separate the prime and target sentences (e.g., Hartsuiker et al., 2008). The lexical boost, therefore, seems to indicate that short-lived explicit memory for the prime sentence can affect the production of the target sentence (though see Bernolet et al., 2016, for an argument that all prime-target priming is boosted by explicit memory, both in cases where the prime and target sentences utilize the same lexical items and cases where they do not). Second, modeling efforts such as those of Chang et al. (2006) and Reitter et al. (2011) demonstrate that a single-mechanism, implicit learning account is unable to explain both structural priming and the lexical boost to that priming. It is noteworthy that the case for explicit memory in structural priming is considerably weaker than the case for implicit learning mechanisms. Whereas a number of hallmarks of implicit learning effects have been demonstrated in structural priming experiments (e.g., priming is long-lasting, shows inverse frequency effects, and so on; see Kaschak et al., 2014, for a discussion), factors known to affect explicit memory have not been extensively explored in structural priming paradigms (but see Bernolet et al., 2016, and Kutta & Kaschak, 2012, for examples).

The experiments reported here represent an effort to examine further the influence of implicit learning and explicit memory on structural priming. We begin with an initial set of two experiments that had the same design as Kaschak, Kutta, and Jones (2011). Participants completed several sentence stems. In the first phase of the experiment, participants completed a set of target stems that allowed both DO and PO continuations (e.g., Heather sent…). This allowed us to establish a base rate of DO and PO production for our participants. Participants then completed a series of prime stems that only allowed either a DO (e.g., Michael gave Meghan…) or PO (e.g., Michael gave the violin…) continuation. The prime stems permitted us to bias participants 100% toward the DO or PO construction. Finally, participants completed another set of target stems to assess how much the completion of the prime stems moved their rates of DO and PO production.

Kaschak, Kutta, and Jones (2011) demonstrated an inverse frequency effect. The base rates of production were slanted toward the DO construction (~ 60% DO completions), establishing the DO as the more frequently used construction in our participant population. After the cumulative priming manipulation (where the priming accumulates over many prime stems), participants biased toward the DO construction showed little change in their rate of DO productions, whereas participants biased toward the PO construction showed a substantive decrease in the rates of DO production. That is, there was a priming effect for the less-frequent PO construction, but not for the more-frequent DO construction.

The first set of experiments reported here represents an effort to replicate and extend Kaschak, Kutta, and Jones’ (2011) finding of an inverse frequency effect. One of the experiments is a straight replication of the original study using the same subject population. It was expected (and found) that the results of this replication would largely mirror those of the original study. The other experiment had essentially the same design as Kaschak, Kutta, and Jones’ (2011) experiment, but drew on a different participant pool – we used Amazon’s Mechanical Turk to collect data from a sample of speakers of British English. We chose to sample this population because corpus studies suggest that speakers of British English do not have the same balance of DO and PO productions as speakers of American English do (e.g., Gries, 2005; Gries & Stefanowich, 2004). Because speakers of British English do not have a DO bias (and, may have a PO bias), we expected that these participants would not show the same inverse frequency effects as the speakers of American English used in our other studies. If the British English speakers showed no bias toward the DO or PO at Pre-bias, we expected to see priming effects for both constructions in the experiment; if the British English speakers demonstrated a bias toward the PO at Pre-bias, we expected to see a different sort of inverse frequency effect – no priming from the more-frequent PO, and a priming effect from the less-frequent DO.

Based on the results of this initial set of experiments, we conducted a second set of experiments in an effort to further explore the role that explicit memory might play in long-term, cumulative structural priming. We delay outlining the rationale for these studies until after discussing the outcome of our first two experiments.

Experiment Set 1: Inverse Frequency Effect

The inverse frequency effect (infrequently occurring structures produce stronger priming compared to more-frequently occurring structures) has been taken as a central piece of evidence for the claim that implicit learning drives structural priming (e.g., Jaeger & Snider, 2013; Kaschak, Kutta & Jones, 2011; Reitter et al., 2011). As described earlier, Kaschak, Kutta, and Jones (2011) used a written stem completion task to demonstrate such an effect using the (more frequent) DO construction and the (less frequent) PO construction. We explore this effect in two experiments. The first is a replication of Kaschak, Kutta, and Jones’ (2011) result using the same participant population as in the original study. The second is a replication of these two experiments using a different population of language users – in this case, speakers of British English. Our goal in doing so is to test the robustness of the inverse frequency effect in a group of speakers that show different base rates of DO and PO production.

Experiment 1: Replication of Kaschak, Kutta, and Jones (2011)

Method

Participants. 88 undergraduate psychology students from Florida State University. The students participated in exchange for partial course credit. Participants were only excluded from the final data set if they failed to produce any usable responses in either the Pre-bias or Post-bias phases. In this case, no participants were excluded from the analysis.

Materials. Items were constructed to fulfill the pre-test/post-test design. Twelve sentence (target) stems were constructed (e.g., The policeman handed…) for the pre-Bias and post-Bias phases. These 12 target stems contained only a noun phrase and verb, which allowed the participants to complete the sentence as either a DO or PO construction. The target stems were split into two sets of six (A and B), with one set appearing in the Pre-bias phase of the study, and the other appearing in the Post-bias phase. For the Bias phase, fourteen pairs of prime stems were constructed. One member of each pair elicited the DO construction (The captain handed the old sailor…), and the other member elicited the PO construction (The captain handed the travel log…). Ninety-eight filler sentence stems were created to obscure the purpose of the experiment. These fillers appeared across all three phases of the experiment. The fillers were a mix of transitive and intransitive constructions, and could not easily be completed as a DO or PO construction. The critical items for all experiments are presented in Appendix A.

Procedure. In all experiments reported here, participants began by completing an informed consent form that was approved by the Institutional Review Board of Florida State University. Participants were randomly assigned to either the DO-bias or PO-bias condition. They were told that they would see a series of sentence stems appear on the computer. For each sentence stem, they were to provide a completion with the first thing that came to mind. Out of a total of 124 trials, one participant would complete six target stems in the Pre-bias phase of the study, 14 prime stems (all DO primes in the DO Bias condition or all PO primes in the PO Bias condition) in the Bias phase of the study, and six additional target stems in the Post-bias phase of the study. The target stems were split into two sets of six (A and B). Of the 88 participants, 46 saw set A in the Pre-test phase, and set B in the Post-bias phase; the other 42 participants saw the sets in the opposite order. Three filler items separated each critical (prime or target) stem. The items were presented in random order, with the constraint that certain critical items appeared in the Pre-bias and Post-bias phases (in keeping with the counterbalancing described above), and items appeared only once in the experiment.

Scoring. The stem completions for all experiments in this paper were scored using the following procedure. Prime stem completions were scored as a DO if the completion began with a noun phrase that contained the direct object of the verb. They were scored as a PO if the completion began with a prepositional phrase using the word “to” and contained the indirect object of the verb.

Target stem completions were scored as a DO if they contained first a noun phrase containing the indirect object of the verb and then a noun phrase containing the direct object of the verb. Target stem completions were scored as a PO if they contained a noun phrase and a prepositional phrase using the word “to.” The noun phrase had to contain the direct object of the verb, and the prepositional phrase had to contain the indirect object of the verb.

Completions that did not meet these requirements were scored as other. This included completions containing a verb particle (e.g., “The teacher handed the test back to the students”), or completions that were nonreversible (i.e., a PO completion needed to have an analogous DO counterpart; for example, “The teacher gave it to the student” is a grammatical PO completion, but “The teacher gave the student it” is not a valid DO completion).

Scoring of the 1056 trials in this experiment showed 564 DO responses, 284 PO responses, and 208 other responses.

Design and Analysis. The stem completions were analyzed according to the following procedure. First, in order to assess whether the proportion of “other” completions differed across conditions, we submitted the stem completions to a mixed logit analysis (DV = stem completions, with “other” coded as 1, and “DO or PO” coded as 0) using the same statistical design as the main analysis of the data for the experiment in question. The mean proportion of “other” completions for each experiment is presented in Appendix B. Statistical analysis of the “other” completions revealed no statistically reliable effects, except in three cases: Experiment 1 showed a main effect of Bias condition, Experiment 3 showed a main effect of Time, and Experiment 4 showed main effects of Memory Instructions and Time (see Appendix B). As these effects did not replicate across experiments (note that the effects in Experiments 3 and 4 may be due to the fact that items were not counterbalanced across time) and do not seem to be tied to the particular manipulations in our experiments, we do not interpret them further.

The primary analysis for each experiment was the analysis of the target responses. For these analyses, we excluded trials on which participants produced an “other” response. Doing so resulted in a binary dependent variable, with DO completions scored as 1 and PO completions scored as 0. We analyzed this variable using mixed models logistic regression to predict the log odds of producing a DO target completion. All models reported in this paper include participants and items as crossed random factors with random intercepts. The model for Experiment 1 also included Bias condition (.5 = DO bias, –.5 = PO bias), Time (.5 = Post-bias, –.5 = Pre-bias) and the Time × Bias condition interaction as predictors. Except where noted, the regression models for all analyses reported were run with the full possible complement of random slopes across participants and items. The mixed logit models were run using the student version of the statistical software HLM 7 (Raudenbush, Bryk, Cheong, Congdon & du Toit, 2011). Throughout the paper, we present the results of the mixed models regression for each analysis in table form. In order to keep the results sections focused, we only discuss the effects of major importance in the text itself.

The data from all experiments reported in this paper, as well as the data from Kaschak et al. (2011), can be found at https://osf.io/jtmxx/.

Results and Discussion

The mixed logit model predicting the log odds of producing a DO target completion is presented in Table 1, along with the raw and predicted means from the model. The critical result of this experiment is the significant Time × Bias condition interaction (p = .035). We followed up on the significant interaction by conducting separate mixed models analyses looking at the effect of Time (coded 0 for Pre-bias, and 1 for Post-bias) for the DO-biased and PO-biased participants. For the PO-biased participants, there was a significant effect of Time (p = .01) (see Table 1), with participants producing fewer DO completions in the Post-bias phase than in the Pre-bias phase. For the DO-biased participants, there was no effect of Time (p = .96), indicating that the rate of DO completions did not change from Pre-bias to Post-bias. The results of this experiment parallel those of Kaschak et al. (2011), and provide a replication of the inverse frequency effect: there is a cumulative priming effect for the lower-frequency PO construction, and no detectable cumulative priming effect for the higher-frequency DO construction.

Table 1

Mixed Logit Analysis of Experiment 1 with Raw and Estimated Means.

Mixed Logit Model

Predictor Coefficient SE t-value p-value

Intercept .830 .329 2.523 .012
Bias condition –.297 .320 –927 .356
Time –.379 .259 –1.463 .144
Time × Bias condition .793 .370 2.146 .035
Analysis conditional on DO

Intercept .622 .404 1.538 .125
Time .013 .276 .046 .963
Analysis conditional on PO

Intercept 1.456 .489 2.977 .003
Time –.858 .333 –2.575 .010
Raw and Estimated Means (Standard Deviations in Parentheses)

Raw Means Estimated Means

Pre-bias Post-bias Pre-bias Post-bias

DO Bias .65(.28) .64(.25) .66 .67
PO Bias .74(.27) .62(.33) .80 .64

Experiment 2 was designed as a further test of the robustness of the inverse frequency effect. Corpus studies (e.g., Gries, 2005; Gries & Stefanowich, 2004), as well as the results of studies from other labs (e.g., Branigan, Pickering & Cleland, 2000), suggest that speakers of British English have a different base rate of DO and PO production than the speakers in the participant pool from which our participants were drawn – depending on the source of the data, speakers of British English appear to have either a bias toward the PO construction, or no strong preference for the DO or PO1. Replicating Experiment 1 with speakers of British English, therefore, provides an opportunity to test a prediction about the implicit learning account of the inverse frequency effect – in testing speakers with a bias toward the PO construction, the inverse frequency effect from Experiment 1 should be reversed. Specifically, the now-higher-frequency PO construction should show no cumulative priming effect, and the now-lower-frequency DO construction should show a cumulative priming effect. Alternately, if there is no bias toward the DO or PO construction, one would expect to see cumulative priming effects for both constructions.

Experiment 2: Speakers of British English

Method

Participants. 97 users of Amazon’s Mechanical Turk participated in Experiment 2. They were paid 10 cents for their participation. We specified that participants should be speakers of British English. Participants were excluded based on the following criteria: 1) failing to produce any usable responses in either the Pre-bias or Post-bias phases (2 participants), 2) failing to report their native language in a post-experiment questionnaire or reporting that English was not their native language (7 participants), or 3) failing to report having grown up as a native speaker of British English (13 participants). This left 75 participants in the final data set.

Materials. In order to shorten the task for our Mechanical Turk users, eight target stems were taken from the set used in Experiment 1. There were four target stems present in the Pre-bias and Post-bias phases, respectively. The target stems were split into two sets of four (A and B). Of the 75 participants included in the final analysis, 42 saw set A in the Pre-test phase, and set B in the Post-bias phase; the other 33 participants saw the sets in the opposite order. The items were presented in a fixed order within both sets. Seven pairs of prime stems were also selected from the ones used in the Bias phase of Experiment 1. Forty-two filler stems from Experiment 1 were used in this study.

Procedure. The procedure was the same as Experiment 1, with the exception that a) critical stems (prime and target stems) were separated by 2 or 3 filler items in this Experiment (rather than being always separated by 3, as in Experiment 1), b) the items were presented in a fixed order (as noted above), and c) the number of prime and target stems was reduced in this experiment.

Scoring. Responses were scored as in Experiment 1. Scoring of the 600 trials in this experiment showed 186 DO responses, 223 PO responses, and 191 other responses.

Design and Analysis. The data were analyzed in the same manner as in Experiment 1.

Results and Discussion

The mixed logit model predicting the log odds of producing a DO target completion is presented in Table 2. Replicating both Kaschak et al. (2011) and the current Experiment 1, the Time × Bias condition interaction was statistically significant (p = .011). We followed up the significant Time × Bias condition interaction by conducting separate mixed models analyses examining the effect of Time (coded as 0 for Pre-bias, and 1 for Post-bias) for the DO-biased and PO-biased participants. For the DO-biased participants, the main effect of Time was not significant (p = .87). For the PO-Biased participants, the main effect of Time was significant (p = .002). Although participants displayed no obvious preference for the DO or PO construction in the Pre-bias phase of this experiment (there was roughly a 50/50 split of DO and PO completions; see Table 2), we observed a pattern similar to what was observed in Experiment 1, and in Kaschak, Kutta, and Jones (2011) – no cumulative priming effect for the DO construction, and a cumulative priming effect for the PO construction. Before moving on to a broader discussion of the implications of these results, we report a set of analyses using the combined data from all three of these experiments (Kaschak, Kutta & Jones, 2011, plus Experiments 1 and 2 from this paper).

Table 2

Mixed Logit Analysis of Experiment 2 with Raw and Estimated Means.

Mixed Logit Model

Predictor Coefficient SE t-value p-value

Intercept –.288 .348 –.825 .410
Bias condition .616 .353 1.745 .085
Time –.578 .240 –2.407 .017
Time × Bias condition 1.293 .498 2.597 .011
Analysis conditional on DO

Intercept –.020 .292 –.068 .946
Time .055 .338 .163 .871
Analysis conditional on PO

Intercept .004 .478 .008 .993
Time –1.166 .364 –3.207 .002
Raw and Estimated Means (Standard Deviations in Parentheses)

Raw Means Estimated Means

Pre-bias Post-bias Pre-bias Post-bias

DO Bias .50(.36) .51(.39) .50 .51
PO Bias .51(.40) .30(.32) .50 .23

Analysis Across Experiment Set 1

The results of Experiments 1 and 2 parallel those of Kaschak, Kutta, and Jones (2011) in showing a reliable cumulative priming effect for the PO construction, but no such effect for the DO construction. The consistency of this result is noteworthy in that it appears regardless of the initial Pre-bias frequency of DO production. The same pattern of results is observed whether participants’ Pre-bias data shows a clear preference for the DO construction (Kaschak, Kutta, and Jones, 2011; the current Experiment 1) or not (the current Experiment 2). These results suggest that the effect reported by Kaschak et al. (2011) may not be best characterized as an “inverse frequency effect,” but rather a priming difference for the DO and PO construction that is not related to broad measures of the frequency of those structures. In order to shore up this claim, we conducted an analysis combining the data from all three of these experiments.

We conducted three sets of analyses. First, to demonstrate that the Pre-bias rate of DO production was different across experiments, we analyzed the data from the Pre-bias phase of this first set of experiments (i.e., Kaschak et al., 2011, Experiment 1, & Experiment 2). We conducted a mixed models logistic regression analysis with two predictors coding specific contrasts between the three experiments. One contrast compared Experiment 2 (British English speakers) to the combination of Experiment 1 and Kaschak et al. (2011). The contrast was designed to show that the baseline rate of DO production was different in Experiment 2 than in the other experiments. The other contrast compared Experiment 1 to Kaschak et al. (2011). The results of this analysis are presented in Table 3. Both contrasts were significant (Experiment 2 vs. others: p = .035; Experiment 1 vs. Kaschak et al., 2011: p = .015). Thus, the base rates of DO production were reliably different across experiments.

Table 3

Analysis Across Experiment Set 1.

Time 1 Analysis

Predictor Coefficient SE t-value p-value

Intercept .535 .307 1.742 .082
Exp. 2 vs. others .184 .087 2.120 .035
Exp. 1 vs. Kaschak et al. –.282 .115 –2.441 .015
Overall Analysis

Intercept .343 .296 1.156 .248
Bias condition .279 .211 1.323 .187
Exp. 2 vs. others –.228 .078 –2.921 .004
Exp. 1 vs. Kaschak et al .282 .112 2.520 .012
Bias × Exp 2 vs. others .179 .150 1.193 .234
Bias × Exp. 1 vs. Kaschak et al. –.398 .234 –1.704 .090
Time –.415 .150 –2.765 < .001
Time × Bias condition 1.185 .301 3.942 < .001
Time × Exp. 2 vs. others –.079 .131 –.605 .546
Time × Exp. 1 vs. Kaschak et al. –.001 .126 –.073 .942
Time × Bias × Exp. 2 vs. others .173 .194 .894 .372
Time × Bias × Exp. 1 vs Kaschak et al. –.147 .284 –.520 .604
Analysis conditional on DO

Intercept .455 .310 1.467 .143
Time .101 .174 .584 .559
Analysis conditional on PO

Intercept .782 .408 1.915 .056
Time –.973 .234 –4.153 < .001

We next conducted an analysis to determine whether the critical Time × Bias condition interaction was different across the three experiments. The model and results for this analysis are presented in Table 3, and the means on which the analysis is based are presented in Figure 1. The Time × Bias condition interaction was significant in this analysis (p < .001), but there were no higher-order interactions involving the Time × Bias pattern. Thus, it appears that cumulative priming (as indexed by the Time × Bias condition interaction) does not differ across experiments.

Figure 1 

Proportion of DO Completions for Kaschak et al., 2011, Experiment 1, and Experiment 2.

Finally, we followed up the significant Time × Bias condition interaction by separately examining the effect of Time (coded as 0 for Pre-bias, and 1 for Post-bias) for the DO-biased and PO-biased participants. As in the individual experiments, the effect of Time was significant for the PO-biased participants (p < .001), with participants producing fewer DO responses in the Post-bias phase compared to the Pre-bias phase. The main effect of Time was not significant for the DO-biased participants (p = .56).

Interim Discussion

The results of Kaschak, Kutta, and Jones (2011), the two experiments presented here, and the combined analysis of all of the experiments points to the conclusion that biasing participants toward the DO construction does not lead to a strong cumulative priming effect, but biasing participants toward the PO construction does. This pattern seems to be independent of the initial base rates of DO production observed in the experiments – whether the base rate revealed a bias toward the DO (as in Experiment 1, with a base rate of ~ 70% DO productions) or not (as in Experiment 2, with a base rate of ~ 50% DO productions), the same general pattern was observed.

We acknowledge that there are potential weaknesses to this claim. First, the Pre-bias phase of the experiments was comparatively short (4 – 6 target items), and our estimates of the base rates of DO production therefore rest on a comparatively small sample of language production. This concern is partly mitigated by the results of Experiment 3b (see Table 5), where the base rates for our Mechanical Turk samples are comparable regardless of whether the participants completed 6 or 12 target stems, and the base rate for our FSU sample (with 12 target stems) is comparable to the base rates in our other studies (with 6 target stems). A second potential concern with our experiments is that the materials consisted of a limited set of verbs and stimulus items. Individual verbs and items have their own structural biases (see Kaschak, 2007, Experiment 2, for an example), and therefore the particular items selected for the experiment may also affect our estimates of the participants’ base rates of DO production. Nonetheless, if our results are taken at face value, they are potentially problematic for the claim that the main pattern of data observed in these studies reflects an inverse frequency effect that is driven by implicit learning. Indeed, our results suggest that the Time × Bias condition interaction that we consistently observe may not be driven by frequency differences between the DO and PO, but by some other factor. The first set of experiments do not point to a clear candidate for what this factor might be, but one hypothesis is that the lexical marking of the PO construction (i.e., it always contains the preposition to) makes the construction stand out in the experiment, thereby increasing the strength of the priming that is observed (as in other observations of a lexical boost). If this hypothesis is correct, it would suggest that explicit memory effects might play a role in cumulative structural priming.

The above results and speculation led us to explore the possibility that explicit memory effects may affect the long-range accumulation of structural priming. There were several reasons for this. First, much of the evidence against the effects of explicit memory on structural priming comes from experiments in which the memory constraints of the task at hand have not been directly manipulated (e.g., Ferreira et al., 2008). We thought it would be prudent to perform such a manipulation before eliminating the possibility of explicit memory factors affecting performance on our task. Second, the literature on implicit learning has featured a long-standing debate over whether the learning observed in implicit learning paradigms represents implicit or explicit knowledge (e.g., Perruchet & Pacteau, 1990; Perruchet & Vinter, 2002). It is beyond the scope of the current paper to provide a full airing of the issues at hand, but we think it is sufficient to note that it is likely the case that there are few “pure” implicit or explicit learning and memory tasks (see Jacoby, 1991, for a discussion) – including the cumulative priming paradigm that is used in the current set of experiments.

A third reason to consider the possibility that explicit memory may affect the accumulation of structural priming can be found in observations from our lab. Kaschak and Borreggine (2008) discuss the possibility that the presence of stronger cumulative priming for one particular verb in their Experiment 2 may have been the result of the participants explicitly noting the form of the verb used in the experiment (lent as the past tense of lend, which is an unusual choice for the students in our participant pool). Furthermore, unpublished studies from our lab suggest that verb-based learning of structural biases, such as that reported in Coyle and Kaschak (2008), is stronger when the participants notice the verb that is being manipulated. This is true whether the participants notice the verb on their own, or whether the verb is made to stand out from the other experimental items (e.g., by being the only word presented in ALL CAPS during the experiment). In both of these cases, the fact that participants are explicitly noting something about the critical sentences in the experiment may increase the odds that they will be reminded of an earlier sentence in the experiment when producing a target item. Reminding of this sort is the explanation typically given for how explicit memory may affect structural priming (e.g., Hartsuiker et al., 2008), and is consistent with our earlier speculation about the role that the lexical marking of the PO may play in explaining the results of Experiment Set 1. Finally, Bernolet et al. (2016) present data suggesting that the influence of explicit memory on structural priming not be limited to cases where lexical items overlap between primes and targets. Based on these factors, we conducted a second series of experiments to examine the possible influence of explicit memory on the long-term accumulation of structural priming.

Experiment Set 2: Explicit Effects on Cumulative Structural Priming

The second set of experiments explores the possibility that explicit memory for previously encountered sentences can affect structural priming over a longer range of time (or intervening items) than has previously been observed (i.e., for longer than the short-term lexical boost; Hartsuiker et al., 2008). We did this by conducting experiments that replicate the design of the first set of experiments, with the addition of the “looking back” manipulation of Jacoby and Wahlheim (2013). While completing the sentence stems, participants were told to assess whether each stem a) was a repeat of the immediately preceding item (one-back condition) or b) had previously appeared at any point in the experiment (n-back condition). Jacoby and Wahlheim (2013; Jacoby, Wahlheim & Kelley, 2015) demonstrated that looking back in this way increased the odds of being reminded about items encountered earlier in the experiment, particularly in the case of the n-back condition. As discussed in the memory literature (e.g., Hintzman, 2011; Jacoby & Wahlheim, 2013), remindings are connected to a range of outcomes, including improved memory performance. This type of running recognition task has been used as the cover task in other structural priming experiments (e.g., Ferreira et al., 2008; Bock & Griffin, 2000). However, the previous studies did not manipulate the nature of the memory task, and therefore did not assess whether different memory conditions might affect the strength of the observed priming effects.

We chose to use this sort of “looking back” task as a vehicle for uncovering explicit memory influences on structural priming for a couple of reasons. First, the one-back and n-back conditions have been shown to reliably induce remindings of earlier items from the experiment, particularly in the case of the n-back condition (Jacoby & Wahlheim, 2013). The addition of this feature to our experiments creates a condition in which we are likely to see more within-experiment reminding than likely occurs in the paradigm that was used in the first set of experiments. As discussed in many places, remindings of earlier sentences while producing a target sentence is the putative mechanism through which explicit memory can affect structural priming (e.g., Bernolet et al., 2016; Hartsuiker et al., 2008). We can therefore compare the results of the second set of experiments to those of the first set of experiments to assess whether looking back (and ostensibly being reminded) increases the strength of the observed cumulative priming effects. If explicit memory affects long-range cumulative priming, this should be the case. The effects of looking back should be strongest in the n-back condition, which elicits more remindings than the one-back condition. The effects of looking back should also be stronger for the DO construction than the PO construction. There are two reasons for this. First, as discussed earlier, it is possible that the repeated lexical item in the PO construction (the preposition to) is already leading to some degree of reminding (and stronger priming) in our paradigm; as such, the “looking back” manipulation may not greatly increase the reminding (and priming) for the PO. Second, the lack of cumulative priming effect for the DO may suggest that this construction is not typically associated with much reminding during the experiment; as such, the “looking back” manipulation may increase the odds of a reminding (and therefore increase priming) for this construction.

The second reason why we chose to use the “looking back” instructions for these experiments it that, as shown by Ferreira et al. (2008), and Bock and Griffin (2000), the “looking back” manipulation can be incorporated into a structural priming paradigm relatively unobtrusively. Thus, we were able to change the memory dynamics of the Kaschak, Kutta, and Jones (2011) paradigm with minimal change to the overall structure of the task.

Experiment 3a: One-back vs. N-back in Cumulative Structural Priming

This experiment used the same paradigm as Experiment 1, with the addition of the one-back and n-back memory instructions.

Method

Participants. 143 undergraduate psychology students participated in Experiment 3a for partial course credit. Participants were only excluded if they did not correctly follow the instructions of the experiment or failed to produce any usable responses in either the Pre-or Post-bias phases. Thirteen participants were excluded using these criteria. An additional 3 participants were excluded for failing to complete the entire experiment. This left 127 participants in the final data set.

Materials. The materials were the same as in Experiment 1, except that 10 filler items were replaced by “repeat” items (i.e., the spot that these items held in the experiment was used to repeat another filler item for the sake of the memory manipulation).

Procedure. The procedure was essentially the same as in Experiment 1. Participants were told that they would see a series of sentence stems on the computer screen, that they should complete each with the first thought that occurred to them, and try to complete each sentence with more than one-word responses. The key difference in this experiment was the addition of the Memory condition (i.e., one-back and n-back). Each trial began with the presentation of the stem to be completed. Once the participant completed and entered the stem, another screen came up and probed them with the one-back or n-back question (Is this sentence stem the same as the sentence stem immediately before?, or, Is this sentence old or new?). Once the participant indicated their response by pressing the “0” key for no, or the “1” key for yes., the next trial was initiated. In addition, in order to simplify the scoring of the memory judgments and the recency judgments taken after the main experiment (see below), we presented the critical items (prime and target stems) in a single fixed order for all participants. There were 10 repeated items in the experiment. All of the repeated items were filler items.

After completing the Post-bias phase of the experiment, all of the participants completed a recency test. This test consisted of thirty questions that asked participants to indicate which of two sentences in the experiment appeared first. These sentences could either be fillers, primes, or targets. The two sentences were always of the same kind (e.g., two fillers, or two primes) and always from the same phase of the experiment.2

Scoring. Responses were scored as in Experiment 1. Scoring of the 1524 trials in this experiment showed 768 DO responses, 407 PO responses, and 349 other responses.

Design and Analysis. The data were analyzed with mixed models logistic regression, as in Experiment Set 1. Participants and items were crossed random factors (with random intercepts) in this analysis, and the model included the full complement of random slopes. The mixed logit model tested included the Time × Bias condition × Memory instruction interaction, with the full complement of two-way interactions and main effects as predictors. The predictors were coded as follows: Bias condition (DO bias =.5, PO bias =–.5), Time (Pre-bias =–.5, Post-bias =.5), and Memory instruction (one-back =–.5, n-back =.5). One important difference in the analysis of Experiment 3a (compared to Experiments 1 and 2) is that Time is a between-items factor due to the fixed order of the items across the Pre- and Post-bias phases.

Results and Discussion

The mixed logit model predicting the log odds of producing a DO target completion for this experiment, along with the raw and estimated means from this sample, is presented in Table 4. As in the first set of experiments, the Time × Bias condition interaction was significant(p <.001). However, the three-way Time × Bias condition × Memory interaction was not significant (p = .947), indicating that the memory manipulation did not have a marked impact on the Time × Bias condition interaction that is characteristic of the experiments in this literature.

Table 4

Mixed Logit Analysis of Experiment 3a with Raw and Estimated Means.

Mixed Logit Model

Predictor Coefficient SE t-value p-value

Intercept .795 .287 2.770 .006
Bias condition 1.054 .316 3.337 .001
Memory instructions .779 .304 2.562 .012
Time × Bias condition 1.293 .498 2.597 .011
Bias × Memory –.393 .669 –.588 .558
Time –.021 .517 –.040 .969
Time × Bias 2.163 .389 5.556 < .001
Time × Memory .426 .350 1.217 .224
Time × Bias × Memory –.059 .886 –.067 .947
Analysis conditional on DO

Intercept .797 .409 1.948 .052
Time 1.025 .538 1.906 .086
Analysis conditional on PO

Intercept .830 .410 2.022 .044
Time –1.052 .524 –2.005 .073
Analysis of Memory Instruction Effect (Pre-bias)

Intercept .485 .431 1.124 .262
Memory instruction .578 .317 1.822 .071
Raw and Estimated Means – N-back Condition (Standard Deviations in Parentheses)

Raw Means Estimated Means

Pre-bias Post-bias Pre-bias Post-bias

DO Bias .73(.36) .85(.25) .73 .90
PO Bias .71(.30) .56(.35) .77 .58
Raw and Estimated Means – One-back Condition (Standard Deviations in Parentheses)

Raw Means Estimated Means

Pre-bias Post-bias Pre-bias Post-bias

DO Bias .61(.33) .75(.30) .65 .81
PO Bias .60(.35) .34(.34) .61 .29

As in the previous experiments, separate mixed models regressions were used to examine the effects of Time (coded as 0 for Pre-bias, and 1 for Post-bias) on the DO-biased and PO-biased participants. The Pre-bias to Post-bias difference was not quite significant for either the PO-biased participants (p = .073) or the DO-biased participants (p = .086). Although it was not quite significant, the Pre-bias to Post-bias difference for the DO-biased participants was somewhat larger than it was in the first set of experiments (a difference of ~ 13%, as opposed to the 0 – 2% differences seen in experiments such as those reported with Experiment Set 1).

One unexpected feature of our data is that the rates of DO production in the Pre-bias phase of the experiment appear to be somewhat higher in the n-back condition than in the one-back condition. We explored this observation by examining the effect of the Memory manipulation in the pre-Bias phase of the experiment (see Table 4). We did this with a mixed models logistic regression coding the one-back condition as 0 and the n-back condition as 1. The difference in rates of DO production between the one-back and n-back conditions was not quite significant (p = .071).

There were several noteworthy findings from Experiment 3a. First, the “looking back” addition to the cumulative priming paradigm did not appear to greatly strengthen the cumulative priming effect that was observed for the DO construction. Although the shift from Pre-bias to Post-bias in the DO bias condition is larger than has been observed in other studies of this sort, it was not statistically significant. In addition, the cumulative priming effect in the PO bias condition appears to be a bit weaker (at least statistically) than it was in the first set of experiments. Second, the magnitude of the cumulative priming effects did not appear to be affected by the manipulation of the one-back vs. n-back memory instruction. It may be the case that any looking back within the experiment encourages remindings of earlier sentences, a point to which we will return later. Finally, although the one-back vs. n-back manipulation did not affect the cumulative priming effects that were observed, the manipulation appeared to have a weak effect on the initial base rates of DO production: n-back participants generated DO responses more frequently than participants in the one-back condition. As this effect was not predicted and was not quite significant at the p < .05 level, it is possible that it may simply reflect natural variation in rates of DO production across participants (note how the base rate of DO productions differ across the studies in Experiment Set 1). However, because this observation was associated with the introduction of a novel manipulation within our paradigm, and because the design of the current experiment may have been less-than-optimal for observing the effect (the relatively small number of target stems in the Pre-bias phase may create too much variability in the estimates of the base rates of DO production to see a smaller effect), we thought it would be worthwhile to replicate the observation in an effort to determine whether the n-back effect on the base rates of DO production is genuine.

In sum, although the results of Experiment 3a hint at the possibility that the “looking back” manipulation may have an influence on cumulative structural priming, it was necessary to replicate and extend these findings to assess their generality.

Experiment 3b: Replication of the N-back Effect on Base Rates of DO Production

In order to assess whether the unexpected increase in the base rate of DO productions in the n-back condition was a genuine effect, we attempted to replicate the effect in a series of experiments that only collected Pre-bias data – that is, we collected base rates of DO production with the one-back and n-back instructions, and did not do the subsequent biasing of participants toward the DO or PO construction (as in the previous experiments). This effort did not turn out to be as straightforward as we initially expected, and as such we ended up conducting several attempts to replicate this finding. A summary of our efforts is as follows. We began by collecting data from a Mechanical Turk sample, which showed no evidence of a difference in the base rates of DO production between the one-back and n-back conditions. The overall base rate of DO completions was much lower than in the Experiment 3a sample (around 50% in the MTurk sample, as opposed to the base rates > 60% seen in Experiment 3a), so we collected another set of data from the FSU participant pool from which the Experiment 3a sample was drawn to see whether the effect depends on the base rate of DO productions. The base rate of DO production in this new sample was over 60% (as in Experiment 3a), but the n-back effect was still not observed. We subsequently collected an additional MTurk sample, which failed to show the n-back effect seen in Experiment 3a.

Method

Participants. In total, there were two samples of Mechanical Turk users collected and one sample of FSU undergraduates. The first sample was 109 Mechanical Turk users, who participated for monetary compensation. The second Mechanical Turk sample was comprised of 144 users that participated for monetary compensation. Finally, 132 undergraduate psychology students participated in Experiment 3b for partial course credit. Participants were excluded if they did not correctly follow the instructions of the experiment, failed to produce any usable responses in the experiment, or if they self-identified as a non-native English speaker. Across all three samples, a total of ten participants were excluded based on these criteria. This left 106 participants in the first Mechanical Turk sample, 140 participants in the second Mechanical Turk sample, and 132 participants in the FSU sample.

Materials. The first Mechanical Turk sample completed the same six items as in the pre-Bias phase of Experiment 1. The second Mechanical Turk sample and the FSU sample completed all twelve items from the pre- and post-Bias phases of Experiment 1. The filler items from the previous experiments were used in this study as well.

Procedure. The procedure and instructions were the same as in Experiment 3a except that two of the three samples of participants were selected from the Mechanical Turk population, and we only ran Pre-bias conditions in these experiments.

Scoring. Responses were scored as in Experiment 1. Scoring of the 3900 trials across the three runs of this experiment showed 1651 DO responses, 1337 PO responses, and 912 other responses.

Design and Analysis. The analysis of these replication attempts had participants and items as crossed random factors, and Memory instruction (one-back = 0; n-back = 1) as a predictor of the log odds of producing a DO target completion.

Results and Discussion

The mixed logit models predicting the log odds of producing a DO target completion for each of the three replication attempts are presented in Table 5. The effect of Memory instruction was not significant for the first or second Mechanical Turk samples, or for the FSU sample (p =.77, p =.29, and p =.45, respectively). Our replication effort paints a clear picture: the potential n-back effect that emerged in Experiment 3a was not reproduced in any of our follow-up studies. As such, it most likely reflects natural variation in the base rates of production for the DO construction across participants.

Table 5

Mixed Logit Analysis of Experiment 3b with Raw and Estimated Means.

Mixed Logit Model

Mechanical Turk Sample 1

Predictor Coefficient SE t-value p-value

Intercept .008 .590 .013 .990
Memory instruction .109 .343 .319 .750
FSU Sample

Intercept .578 .379 1.525 .127
Memory instruction .261 .342 .765 .446
Mechanical Turk Sample 2

Intercept .079 .367 .215 .830
Memory instruction –.312 .314 –.991 .323
Raw and Estimated Means (Standard Deviations in Parentheses)

Raw Means Estimated Means

Pre-bias Post-bias Pre-bias Post-bias

Mechanical Turk 1 .51(.32) .51(.35) .50 .53
FSU .61(.31) .65(.31) .64 .70
Mechanical Turk 2 .51(.31) .46(.32) .52 .44

Experiment 4: Introducing the “Looking Back” Manipulation After the Pre-bias Phase

Experiment 3a provided a hint that the “looking back” component added to Kaschak et al., (2011) paradigm may have the effect of boosting the cumulative priming effect for the DO construction – while not significant, the cumulative priming effect for the DO construction was numerically larger than in the first set of experiments. Experiment 4 was conducted with the aim to replicate and modestly extend this result. The experiment had the same design as Experiment 3a, except that the “looking back” instructions were not introduced until after the end of the Pre-bias phase of the study. This design was used to remedy a potential weakness in Experiment 3a – when participants were “looking back” through the Pre-bias phase, they may have experienced remindings of sentences using the construction opposite the one toward which they would subsequently be biased. For example, a participant may be reminded of an example of a PO construction in the Pre-bias phase, but would subsequently be biased toward the DO construction. Because remindings increase memory strength (Jacoby & Wahlheim, 2013; Jacoby, Wahlheim & Kelley, 2015), this may have worked against the cumulative priming effect. By limiting the “looking back” instructions to the latter part of the experiment – that is, by letting the participants go through the Pre-bias phase with no expectation that they would need to remember the sentences that they encountered – we hoped to reduce the potential effect that the Pre-bias phase would have on the cumulative priming effects that were observed.

Method

Participants. 185 undergraduate psychology students participated in Experiment 4 for partial course credit. Participants were only excluded if they did not correctly follow the instructions of the experiment or failed to produce any usable responses in either the Pre-or Post-bias phases. No participants were excluded based on these criteria. One participant was excluded for failing to complete the entire experiment, leaving 184 participants in the final data set.

Materials. The materials were the same as in Experiment 3a.

Procedure. The procedure was the same as Experiment 3a, except that participants did not start answering the one-back or n-back questions until the beginning of the Bias phase.

Scoring. Responses were scored as in Experiment 1. Scoring of the 2208 trials in this experiment showed 1023 DO responses, 758 PO responses, and 427 other responses.

Design and Analysis. The data were analyzed as in Experiment 3a.

Results and Discussion

The mixed logit model predicting the log odds of producing a DO target completion is presented in Table 6. The results closely echo those of Experiment 3a: whereas the Time × Bias condition interaction was significant (p < .001), the Time × Bias condition × Memory interaction was not (p = .597). Follow-up analyses examining the effect of Time (Pre-bias coded as 0, Post-bias coded as 1) were conducted separately for each Bias condition. The pattern of means resembled the results of the previous experiments, but the main effect of Time was not significant for either the DO-biased participants (p = .31) or the PO-biased participants (p = .19).

Table 6

Mixed Logit Analysis of Experiment 4 with Raw and Estimated Means.

Mixed Logit Model

Predictor Coefficient SE t-value p-value

Intercept .312 .307 1.017 .309
Bias condition .403 .261 1.544 .124
Memory instructions .152 .252 .607 .545
Bias × Memory –.091 .499 –.182 .856
Time –.124 .579 –.215 .836
Time × Bias 1.463 .322 4.539 < .001
Time × Memory .099 .291 .341 .733
Time × Bias × Memory .300 .568 .528 .597
Analysis conditional on DO

Intercept .211 .406 .521 .603
Time .580 .546 1.063 .313
Analysis conditional on PO

Intercept .539 .464 1.163 .245
Time –.868 .628 –1.383 .197
Raw and Estimated Means – N-back Condition (Standard Deviations in Parentheses)

Raw Means Estimated Means

Pre-bias Post-bias Pre-bias Post-bias

DO Bias .55(.34) .68(.30) .55 .72
PO Bias .62(.30) .48(.37) .66 .44
Raw and Estimated Means – One-back Condition (Standard Deviations in Parentheses)

Raw Means Estimated Means

Pre-bias Post-bias Pre-bias Post-bias

DO Bias .57(.31) .64(.30) .55 .67
PO Bias .59(.32) .45(.36) .60 .40

The results of this experiment largely parallel those of Experiment 3a. Although the pattern in the means was consistent with the existence of cumulative priming for both the DO- and PO-biased participants (for DO-biased participants, more DOs produced in the Post-bias phase than in the Pre-bias phase; for PO-biased participants, fewer DOs produced in the Post-bias phase than in the Pre-bias phase), this effect was not statistically reliable for either Bias condition. Taken together with Experiment 3a, these results might hint at the following possibility: looking back over the sentences produced in the experiment has the impact of slightly increasing the cumulative priming effect for the DO construction, and slightly weakening the effect for the PO construction.

The observation that cumulative priming for the DO appears to be stronger, and cumulative priming for the PO appears to be weaker, in these experiments might be explained by the overall frequency advantage that the DO construction enjoys in these experiments. For example, the more-frequently-occurring DO construction might be more likely to be subject to a reminding, and therefore boost the strength of the DO construction (and weaken the strength of the PO construction) within the experiment. An alternative explanation for this pattern might be that the memory demands of the task created a situation in which participants had fewer cognitive resources to handle the stem completion task, and therefore were more likely to select the more-frequent DO (strengthening DO priming) more often than the less-frequent PO (weakening PO priming). These speculations come with an important caveat – Experiments 3a, 3b, and 4 all presented the critical items in a fixed order, such that the items presented at Pre-bias are different from the items presented Post-bias. Due to this design feature, the possibility exists that item-based effects might explain why the cumulative priming effect for the DO construction is numerically stronger than in the first set of experiments, and why the cumulative priming effect for the PO construction is statistically weaker than in the first set of experiments. We can partially rule this possibility out by looking at the results of Experiment 3b, where two of the replication efforts involved presenting all 12 critical items in the same fixed order as they appeared in Experiments 3a and 4. These data show no evidence that the Pre-bias items have a different general DO bias than the Post-bias items (58% DO completions for the Pre-bias items, 57% DO completions for the Post-bias items). Nonetheless, we decided that the best way to settle this issue would be to conduct one further experiment. This experiment replicates Experiment 4, except that we counterbalanced the presentation of items so that across participants the same set of items appeared in the Pre-bias and Post-bias phases of the experiment.

Experiment 5: Replication of Experiment 4, with Counterbalanced Items

Method

Participants. 132 undergraduate psychology students participated in Experiment 5 for partial course credit. Participants were only excluded if they did not correctly follow the instructions of the experiment or failed to produce any usable responses in either the Pre-bias or Post-bias phases. In this experiment, twelve participants were excluded based on these criteria, leaving a total of 120 participants in the final data set.

Materials. The materials were the same as in Experiment 4.

Procedure. The procedure was the same as in Experiment 4, except that the order of the items was counterbalanced so that across participants the same critical items appeared in the Pre- and Post-bias phases of the experiment.

Scoring. Responses were scored as in Experiment 1. Scoring of the 1440 trials in this experiment showed 700 DO responses, 464 PO responses, and 276 other responses.

Design and Analysis. The data were analyzed as in Experiment 4.

Results and Discussion

The mixed logit model predicting the log odds of producing a DO target completion is presented in Table 7.

Table 7

Mixed Logit Analysis of Experiment 5 with Raw and Estimated Means.

Mixed Logit Model

Predictor Coefficient SE t-value p-value

Intercept .517 .264 1.957 .051
Bias condition .492 .306 1.607 .111
Memory instructions .254 .293 .865 .389
Bias × Memory –.139 .610 –.229 .819
Time –.463 .195 –2.371 .018
Time × Bias 1.667 .371 4.490 < .001
Time × Memory .093 .367 .253 .800
Time × Bias × Memory .120 .889 .135 .893
Analysis conditional on DO

Intercept .564 .329 1.71 .088
Time .339 .224 1.511 .131
Analysis conditional on PO

Intercept .845 .325 2.603 .010
Time –1.232 .274 –4.503 < .001
Raw and Estimated Means – N-back Condition (Standard Deviations in Parentheses)

Raw Means Estimated Means

Pre-bias Post-bias Pre-bias Post-bias

DO Bias .62(.34) .69(.34) .65 .75
PO Bias .71(.32) .46(.35) .74 .45
Raw and Estimated Means – One-back Condition (Standard Deviations in Parentheses)

Raw Means Estimated Means

Pre-bias Post-bias Pre-bias Post-bias

DO Bias .62(.30) .65(.30) .63 .69
PO Bias .62(.32) .41(.37) .68 .37

The results replicate the outcome of Experiments 3a and 4. The Time × Bias condition interaction was significant (p < .001), but the three-way Time × Bias condition × Memory instruction interaction was not (p = .89). The Time × Bias condition interaction was followed up with separate mixed models analyses examining the effect of Time for the DO-biased and PO-biased participants (Pre-bias coded as 0, Post-bias coded as 1). The effect of Time was significant for the PO-biased participants (p < .001), but not for the DO-biased participants (p = .13).

There are three things to note about the results of this experiment. First, the major result of Experiments 3a and 4 were replicated – whereas the standard Time × Bias condition interaction was significant, the Time × Bias condition × Memory instruction interaction was not. Second, the cumulative priming effect for the PO construction was quite strong in Experiment 5, suggesting that the “looking back” instructions do not reliably weaken this effect. Third, the cumulative priming effect for the DO-biased participants was again numerically larger than what was observed in the first set of experiments, yet was weaker than the effect observed in Experiment 3a. The finding that the cumulative priming effect for the DO is generally small (and not statistically significant) in the individual experiments creates some ambiguity in interpreting our results – the “looking back” component of the task may strengthen the DO priming effect (note that the effect in each of these three experiments is numerically larger than it is in the first set of experiments), but the statistical evidence for this claim is weak. As a further examination of this issue, we conducted an analysis that combined the results from Experiments 3a, 4, and 5. If there is a genuine cumulative priming effect for the DO construction, it might emerge statistically in this more powerful combined analysis.

Analysis across Experiment Set 2

We began our combined analysis of Experiment Set 2 by doing a mixed logit analysis that included Time (Pre-bias coded as –.5, Post-bias coded as .5), Bias condition (DO bias coded as .5, PO bias coded as –.5), Instructions (one-back = –.5; n-back = .5) and Experiment (coded with two contrasts: Experiment 3 vs. Experiments 4 and 5; and, Experiment 4 vs. Experiment 5), and the interactions that follow from these variables as predictors. We had some difficulty getting a model with this full design (with the full complement of random slopes) to run without producing errors, so we eliminated some of the highest-order interactions from the analysis (see Table 8 for model and results, and Figures 2 and 3 for the means on which the analysis is based).

Figure 2 

Proportion of DO Completions for the N-back Condition in Experiments 3a, 4, and 5.

Figure 3 

Proportion of DO Completions for the One-back Conditions in Experiments 3a, 4, and 5.

Table 8

Analysis Across Experiment Set 2.

Overall Analysis

Predictor Coefficient SE t-value p-value

Intercept .567 .266 2.136 .033
Bias condition .679 .175 3.886 < .001
Memory instructions .410 .167 2.453 .015
Exp. 3 vs. others –.122 .062 –1.981 .048
Exp. 4 vs. Exp. 5 –.083 .107 –.773 .440
Bias × Memory –.320 .354 –.905 .366
Bias × Exp. 3 vs. others –.213 .123 –1.732 .084
Bias × Exp. vs. Exp. 4 –.039 .222 –.174 .862
Time –.443 .191 –2.313 .021
Time × Bias 1.816 .225 8.069 < .001
Time × Bias 1.816 .225 8.069 < .001
Time × Memory .240 .208 1.153 .250
Time × Exp. 3 vs. others –.035 .076 –.466 .641
Time × Exp. 4 vs. Exp. 5 –.015 .151 –.102 .909
Time × Bias × Memory .261 .469 .556 .579
Time × Bias × Exp. 3 vs. others –.180 .152 –1.184 .237
Time × Bias × Exp. 4 vs. Exp. 5 –.059 .279 –.213 .832
Analysis conditional on DO

Intercept .633 .322 1.967 .049
Time .351 .184 1.906 .057
Analysis conditional on PO

Intercept .724 .318 2.274 .023
Time –1.050 .199 –5.281 < .001

The results of the combined analysis echo those of the individual experiments. The only significant interaction was the Time × Bias condition interaction (p < .001). As in Experiment Set 1, this result suggests that the Time × Bias condition pattern that we observed is similar across all three experiments. We followed up the significant Time × Bias condition interaction by analyzing the effect of Time separately for the PO- and DO-biased participants. For the PO-biased participants, the main effect of Time was significant (p < .001). For the DO-biased participants, the main effect of Time was not quite significant (p = .057). If the memory instructions did strengthen the cumulative priming effect for the DO, the effect appears to be weak even in this more powerful combined analysis.

Analysis across Experiment Sets 1 and 2

The results of Experiment Set 2 hint at the possibility that our memory instructions may have boosted the strength of the cumulative priming effect in these experiments, particularly in the case of the DO construction. We, therefore, conducted a set of analyses to determine whether the patterns observed in Experiment Set 2 differ from those observed in Experiment Set 1. The analysis of the aggregated data sets included Time (Pre-bias coded as –.5, Post bias coded as .5), Bias condition (DO bias coded as .5, PO bias coded as –.5), and Memory instruction (–.5 = no memory instruction; Experiment Set 1; .5 = memory instructions; Experiment Set 2) and the interactions that follow from these variables as predictors (see Table 9 for model and results). The question of main interest in this analysis is whether the Time × Bias condition interaction pattern is different in Experiment Set 1 than it is in Experiment Set 2.

Table 9

Analysis Across Experiments Sets 1 and 2.

Mixed Logit Model

Predictor Coefficient SE t-value p-value

Intercept .468 .255 1.837 .066
Bias condition .410 .146 2.806 .005
Memory instruction .102 .227 .449 .654
Bias × Memory .338 .266 1.271 .204
Time –.404 .128 –3.171 .002
Time × Bias 1.382 .240 5.678 < .001
Time × Memory .054 .184 .292 .770
Time × Bias × Memory .531 .336 1.578 .115
Analysis conditional on DO

Intercept .665 .243 2.737 .006
Memory instruction .283 .252 1.124 .262
Time .222 .150 1.481 .139
Time × Memory .252 .219 1.151 .251
Analysis conditional on PO

Intercept .245 .280 .879 .380
Memory instruction –.095 .267 –.356 .722
Time –1.046 .178 –5.879 < .001
Time × Memory –.105 .293 –.358 .720

The results of this analysis appear in Table 9. Whereas the Time × Bias condition interaction was significant(p < .001), the Time × Bias condition × Memory instruction interaction was not (p = .115). This result suggests that the Time × Bias condition pattern (i.e., the cumulative priming effect) was similar across experiment sets. We followed up the significant Time × Bias condition by examining performance for the DO- and PO-biased participants separately. Although the three-way Time × Bias condition × Memory instruction interaction was not significant, we included Time (Pre-bias = –.5; Post-bias = .5), Memory instruction (No instruction = –.5; Instruction = .5), and the Time × Memory instruction interaction as predictors in these analyses to provide another opportunity for differences in cumulative priming to emerge between experiment sets. The results of these analyses are presented in Table 9. For the PO-biased participants, the main effect of Time was significant(p < .001), but the effect of Memory instruction and the Time × Memory instruction interaction were not. For the DO-biased participants, the effect of Time was not significant (p = .139). The effect of Memory instruction and the Time × Memory instruction interaction were also not significant. This outcome argues that the cumulative priming effects observed in our studies do not differ across experiment sets, and suggest that the “looking back” instructions employed in Experiment Set 2 did not greatly affect the patterns of cumulative priming observed throughout these experiments.

Analysis Based on Individual Base Rates of DO Production

The experiments reported in this paper show a consistent pattern of strong cumulative priming for the PO construction, and weaker cumulative priming for the DO construction (see also Kaschak et al., 2011; Jaeger & Snider, 2013). Contrary to earlier reports that characterized this pattern as an inverse frequency effect (e.g., Kaschak et al., 2011), our data suggest that the difference in priming is not related to the base rates of DO production found in the Pre-bias phase of the experiment. In our discussion of Experiment Set 1, we speculated that the lexical marking of the PO construction (i.e., the use of the preposition to) might have made the construction stand out in our experiment. This type of explicit memory effect might strengthen the priming observed for the construction. The results of Experiment Set 2, however, do not strongly support the idea that explicit memory effects are operative in boosting the strength of cumulative structural priming (with the caveat that this conclusion is tied to one particular method for boosting explicit memory for the sentences in the experiments).

A hypothesis that we have not yet explored is that the differences in priming for the DO and PO construction may be related to production frequencies of individual speakers. As Jaeger and Snider (2013; Fine & Jaeger, 2013) point out, average production frequencies (such as corpus estimates of DO and PO frequency, or the base rates that we derive from the Pre-bias phases of our experiments) may not appropriately capture the linguistic experience and expectations of individual speakers. They further note that implicit learning accounts of structural priming suggest that nuanced information about the prior experience of individual speakers should be more valuable in predicting subsequent production performance (such as the effects that our Bias phases have on DO and PO production) than population-or sample-wide estimates of an individual’s production experience. We examined this claim in a further set of analyses conducted on the combined data from Experiment Sets 1 and 2.

Table 10 displays the number of participants (out of the 707 participants included in the analysis across all experiments) with a given proportion of DO completions in the Pre-bias phase of the experiment. Although the overall bias toward the DO seen in our Pre-bias phases is reflected in these data (59% of participants have a base rate > = .60), the data also show that there is a lot of variability in the Pre-bias base rates – including a fair number of participants with a base rate of 0 for the DO construction. Our first step in exploring the relationship between the participants’ Pre-bias base rates and the cumulative priming effects was to conduct analyses on the participants that had either 0%, 50%, or 100% DO completions in the Pre-bias phase of the experiment (that is, on participants with an extreme bias toward either the DO or PO, or participants with no bias toward either construction)3. Since the participants in each group were selected so that they had the same rate of DO production in the Pre-bias phase, the analysis only looked at their performance in the Post-bias phase. Mixed models logistic regression analysis was used to predict the log odds of a DO target completion in the Post-bias phase of the experiment, with Bias condition, Memory instruction and the Bias × Memory interaction as predictors.

Table 10

Number of Participants with Different Pre-bias Proportion of DO Completions Across Experiment Sets 1 and 2.

Proportion N

0 73
.17 13
.20 22
.25 33
.33 42
.40 29
.5 81
.6 42
.67 65
.75 40
.8 45
.83 38
1.00 184

The results of our analysis of the Post-bias data are presented in Table 11. The effect of Bias condition was significant for the 0% DO (p = .006), 50% DO (p = .003) and 100% DO (p = .001) participants. One-sample t-tests were used to compare the means for each Bias condition in each participant group against the Pre-bias proportion of DO completions (0 for the 0% DO participants, .5 for the 50% DO participants, and 1.0 for the 100% DO participants). All of these tests were significant (see Table 11). For the 100% DO participants, the fact that the DO Bias average was different than 1.0 suggests that there is some non-trivial regression to the mean in the Post-bias phase for these participants – it is unlikely that they never produce PO sentences (despite their Pre-bias base rate), and this is reflected in their Post-bias data. At the same time, participants in the DO Bias condition do not move from their initial base rate as much as participants in the PO Bias condition, demonstrating cumulative priming for the PO construction. For the 0% DO participants, we again see this sort of regression to the mean (as evidenced by the participants in the PO Bias condition who move away from 0% DO production). Importantly, the fact that participants in the DO Bias condition move away from their Pre-bias baseline more than the PO Bias participants demonstrates cumulative priming for the DO construction. For the 50% DO participants, we observe cumulative priming for both the PO and DO constructions, as participants in the DO Bias and PO Bias conditions both moved significantly from their initial rate of DO production (and toward the construction for which they were biased).

Table 11

Analysis of Post-bias Rates of DO Production for Participants with 0%, 50% or 100% DO Completions in the Pre-bias Phase, With Raw and Estimated Means.

0% DO (Pre-bias)

Predictor Coefficient SE t-value p-value

Intercept –1.287 .266 –4.835 < .001
Bias condition 1.221 .429 2.850 .006
Memory instruction –.142 .441 –.323 .748
Bias × Memory .367 1.001 .366 .715
50% DO (Pre-bias)

Intercept –.037 .267 –.137 .891
Bias condition 1.088 .351 3.103 .003
Memory instruction .345 .410 .843 .402
Bias × Memory .621 .766 .811 .420
100% DO (Pre-bias)

Intercept 1.516 .301 5.034 <.001
Bias condition 1.079 .324 3.335 .001
Memory instruction .099 .386 .258 .797
Bias × Memory .601 .651 .923 .357
Raw and Estimated Means (With Standard Deviations in Parentheses)

Raw Means Estimated Means

DO Bias PO Bias DO Bias PO Bias

0% DO .37(.29)* .15(.28)* .34 .13
50% DO .64(.29)* .38(.34)* .63 .36
100% DO .84(.27)* .67(.32)* .89 .73

Note: Means marked with an asterisk (*) are significantly different from the Pre-bias proportion of DO completions for that participant groups (0, .50, and 1.00, respectively).

Differences in the cumulative priming effects for the DO and PO construction based on participants’ Pre-bias rates of DO production can also be demonstrated using a broader sample of our participants. We ran a set of mixed models regressions identical to those used to conduct the analysis across all experiments (see Table 9), except that for one analysis we only included participants with Pre-bias rates of DO production < 50% (see Table 12), and for the other we included only participants with a Pre-bias rate of DO production > 50% (see Table 13). Both sets of participants show a significant Time × Bias condition interaction (p’s < .001), but the shape of these interactions is opposite. For the < 50% group (Table 12), there is an effect of Time for the DO Bias participants (p < .001), but no effect of Time for the PO Bias participants (p = .62). For the > 50% group (Table 13), there is an effect of time for the PO Bias participants (p < .001), but not for the DO bias participants (p = .23). In other words, participants show a cumulative priming effect for the construction that is lower-frequency for them (as judged by the Pre-bias base rates of production), but not for the construction that is higher-frequency for them.

Table 12

Analysis Across Experiment Sets 1 and 2 for Participants with Baseline PO Bias.

Mixed Logit Model

Predictor Coefficient SE t-value p-value

Intercept –1.225 .251 –4.873 < .001
Bias condition .662 .202 3.277 .001
Memory instruction .043 .223 .194 .847
Bias × Memory .288 .383 .751 .454
Time .540 .208 2.594 .010
Time × Bias 1.124 .324 3.471 < .001
Time × Memory .103 .385 .268 .789
Time × Bias × Memory .915 .652 1.403 .162
Analysis conditional on DO

Intercept –.770 .232 –3.322 < .001
Time 1.166 .276 4.227 < .001
Analysis conditional on PO

Intercept –1.594 .310 –5.140 < .001
Time –.148 .298 –.497 .619
Raw and Estimated Means for Time × Bias Condition Interaction (With Standard Deviations in Parentheses)

Raw Means Estimated Means

Pre-bias Post-bias Pre-bias Post-bias

DO Bias .19(.15) .44(.30) .19 .41
PO Bias .19(.15) .20(.28) .18 .17

Table 13

Analysis Across Experiment Sets 1 and 2 for Participants with Baseline DO Bias.

Mixed Logit Model

Predictor Coefficient SE t-value p-value

Intercept 1.474 .276 5.340 < .001
Bias condition .492 .137 3.584 < .001
Memory instruction .154 .256 .601 .548
Bias × Memory .053 .300 .178 .859
Time –.980 .166 –5.911 < .001
Time × Bias 1.339 .379 3.535 < .001
Time × Memory –.022 .312 –.072 .942
Time × Bias × Memory .564 .479 1.179 .239
Analysis conditional on DO

Intercept 1.716 .275 6.233 < .001
Time –.244 .205 –1.189 .235
Analysis conditional on PO

Intercept 1.206 .285 4.227 < .001
Time –1.579 .265 –5.968 < .001
Raw and Estimated Means for Time × Bias Condition Interaction (With Standard Deviations in Parentheses)

Raw Means Estimated Means

DO Bias PO Bias DO Bias PO Bias

DO Bias .84(.16) .79(.26) .87 .83
PO Bias .85(.15) .58(.33) .89 .60

Our analyses that take participants’ individual base rates of DO production into account shed a different light on the nature of the cumulative priming that is observed for the DO and PO constructions. Whereas previous explorations of this phenomenon based on “average” estimates of the participants’ disposition toward the DO and PO (e.g., Kaschak et al., 2011) report no significant cumulative priming for the DO construction, the current set of analyses suggests that the claim that cumulative priming does not occur for the DO is incorrect – the analyses reported in Table 12 and 13 suggest that cumulative priming can be observed for the DO construction in participants whose base rates of DO production indicate a bias toward the PO construction, or no bias toward either construction. This conclusion is consistent with Jaeger and Snider’s (2013) claim that adaptations in language production should be affected by individual speakers’ prior language experience and experience within a given context, rather than by more global-level information about the average language use of a group of individuals.

General Discussion

We undertook this project in an effort to examine the influence of implicit and explicit memory processes on the accumulation of structural priming across sentences. Previous work has suggested that whereas cumulative priming can be observed for the PO construction, it is typically not observed for the DO construction (e.g., Kaschak et al., 2011; Reitter et al., 2011). The difference in cumulative priming for the DO and PO constructions has typically been explained through mechanisms of implicit learning. Lower-frequency structures (such as the PO) lead to more prediction error (e.g., Chang et al., 2006) or belief updating (e.g., Jaeger & Snider, 2013) in the language processing system, and therefore prime more strongly than higher frequency structures (such as the DO). When frequency is viewed in terms of average levels of DO production for a group of participants (as in Kaschak et al., 2011), the implicit learning account is ruled out by our Experiment 2 (in which there was no Pre-bias preference for the DO, but no hint of DO priming was observed). When frequency is viewed in terms of individual rates of DO production, however, the patterns observed in our data are consistent with the implicit learning account. Participants who have a preference for the DO in the Pre-bias phase show the pattern traditionally seen in these experiments: cumulative priming for the PO, but not for the DO. Participants who have a preference for the PO in the Pre-bias phase show the opposite pattern: cumulative priming for the DO, but not for the PO. Thus, it appears that the inverse frequency effect that is taken as evidence for implicit learning accounts of structural priming should be evaluated based on an assessment of individual-level structural preferences.

Jaeger and Snider (2013; Fine & Jaeger, 2013) present an extensive discussion of implicit learning-based accounts of structural priming, and argue that the predictions of these accounts should rest on nuanced, contextual information about individual language users, rather than population-level estimates of an individual’s linguistic experience. Our data add weight to this claim. When viewed from the level of the whole sample, implicit learning accounts would appear to be unable to simultaneously explain a) the fact that participants do not seem to learn from repeated exposure to the DO construction, and b) the fact that this appears to be true regardless of the Pre-bias rates of DO production. When viewed on the individual level, (a) and (b) might be explained by the fact that our samples tend to have more DO-biased participants than PO-biased participants (i.e., the number of participants who would ostensibly show a cumulative priming effect for the DO is smaller than the number who would not show the effect).

Whereas we see our data as supporting an implicit learning approach to cumulative structural priming, we advise a certain degree of caution in interpreting the outcome of our individual base-rate analyses. These analyses were done in an exploratory manner, taking advantage of the fact that the combined sample across all of our experiments yielded enough participants at each base rate level of DO production that the data could be analyzed in a meaningful way. The Pre-bias phase of these experiments represents a comparatively small sample of language use, and it is therefore expected that our base rates will contain a degree of error. These observations should be confirmed with a set of studies that assesses base rates of production with a larger sample of language use, as this will allow for a stronger test of the ways that individual linguistic performance affects subsequent patterns of cumulative priming.

In addition to examining the role of implicit learning in driving cumulative structural priming, we also considered the role that explicit memory mechanisms might play in this priming. In our discussion of Experiment 2, we speculated that the reason why participants showed priming for the PO construction, but not the DO construction, is that the lexical marker to leads to an explicit memory boost for the PO. The results of our individual level analyses argue against this speculation: participants who favor the PO construction in the Pre-bias phase of the experiments show priming for the DO construction, but not the PO construction. This pattern cannot be explained by the lexical marker hypothesis.

In our second set of experiments, we considered the possibility that increasing the odds of participants being explicitly reminded of earlier sentences in the experiment would increase the strength of cumulative structural priming (particularly for the DO construction). The results of our analysis across experiments, and the results of our analyses taking individual base rates of production into account, suggest that looking back – that is, creating conditions under which participants would explicitly remember sentences from earlier in the experiment – does not greatly affect the nature of cumulative structural priming. A potential issue with our conclusion about the effect of long-range remindings on structural priming is that although we presume that our n-back and one-back instructions increased the odds of participants being reminded of sentences encountered earlier in the experiment (as compared to what might occur in Experiments 1 and 2, with no memory instruction), we did not directly measure what participants recalled when responding on any given trial. Thus, unlike the memory experiments on which our paradigm was based (e.g., Jacoby, Wahlheim & Kelley, 2015), we do not have information about which trials elicited a specific reminding of previous sentences, how many times a reminding occurred, and what the content of those remindings were. We did not solicit this information from participants while they were completing the experiment, as we wished to keep the method of Experiments 3 – 5 as similar as possible to that of Experiments 1 – 2. In addition, we were concerned about the potential effect that the elicited production of “reminded” sentences would have on our bias manipulations. As these were initial studies in this area of research, it seemed prudent to begin with a sequence of studies that stick as closely as possible to the original studies that we were replicating. A more detailed examination of when participants are reminded of earlier sentences, and what is recalled when such remindings occur, was viewed as a necessary follow-up project should these initial results suggest that remindings affect cumulative structural priming.

Although the results from Experiment Set 2, and the analysis across experiment sets, suggest that cumulative priming is not greatly affected by the “looking back” manipulations, it is possible that a more detailed examination of the memory dynamics in this paradigm might be fruitful. For example, just as remindings affect performance on memory tasks on an item-by-item basis – only items for which a reminding occurs show improved memory (e.g., Jacoby et al., 2015) – it may be the case that the effect of remindings need to be observed on an individual-item basis in experiments such as ours, rather than on a global task level (as was done in the current set of studies). If this is the case, gathering specific information about when remindings occurred during the task, and what was remembered during those remindings, would be necessary to more strongly assess the possibility that explicit memory affects long-term cumulative structural priming.

One goal of this paper was to assess the status of the two-mechanism account of structural priming. The data presented here is consistent with this account. Long-range cumulative structural priming can be explained through mechanisms of implicit learning, and does not seem to be greatly affected by explicit memory effects. Keeping in mind the shortcomings of our “looking back” paradigm (as discussed above), and the fact that this is but one of many explicit memory manipulations that could have been done, it may be premature to completely rule out the possibility of explicit memory affecting long-range structural priming. Nonetheless, our results indicate that an adequate account of cumulative structural priming can be made based on mechanisms of implicit learning.

Data Accessibility Statement

The stimuli used in these experiments is presented in Appendix A. The data from the experiments reported here can be found at https://osf.io/jtmxx/.

Additional Files

The additional files for this article can be found as follows:

Appendix A

Critical Sentence Stems Used in Experiments 1–5. DOI: https://doi.org/10.1525/collabra.59.s1

Appendix B

Mean Proportion of “Other” Responses for Experiments 1–5 (Standard Deviations in Parentheses). DOI: https://doi.org/10.1525/collabra.59.s2