Start Submission Become a Reviewer

Reading: A Perspective on the Relevance and Public Reception of Psychological Science

Download

A- A+
Alt. Display

Perspective/Opinion

A Perspective on the Relevance and Public Reception of Psychological Science

Author:

Jonathon McPhetres

University of Rochester, Department of Clinical and Social Science in Psychology, Rochester, NY, US; University of Regina, CA; Massachusetts Institute of Technology, US
About Jonathon
I'm a social scientist studying what people think about science and scientists.
X close

Abstract

In this short commentary, data from the website Reddit is used to examine how people receive social psychological research. The data show that people care greatly about research dealing with humans: links tagged as psychology, social sciences, and health are upvoted more than other categories on Reddit. Within the category of psychology, articles were coded based on the topic of research. Articles dealing generally with social psychological topics are among the highest in number and upvotes on the subreddit r/Science. Many posts were upvoted tens of thousands of times. However, upvotes on Reddit are unrelated to scientific publishing metrics (e.g., impact factor, journal rankings, and citations), suggesting a disconnect between what psychologists and Redditors may see as relevant. These findings also highlight some points for reflection. For example, psychologists may benefit from thinking about the purpose, goals, and beneficiaries of the research they pursue. Additionally, the level of attention that some psychological research receives has implications for transparent research practices. Researchers have a responsibility to ensure that findings are reported accurately and transparently because, whether scientists like it or not, people care about psychological research, they share it, and use it in their lives.
Subject: Psychology
How to Cite: McPhetres, J. (2019). A Perspective on the Relevance and Public Reception of Psychological Science. Collabra: Psychology, 5(1), 34. DOI: http://doi.org/10.1525/collabra.223
163
Views
19
Downloads
Altmetric
  Published on 30 Jul 2019
 Accepted on 03 Jul 2019            Submitted on 07 Jan 2019

As scientists, it can be easy to lock ourselves inside the “science bubble” and ignore how the general public perceives our work. While some researchers may see the importance of communicating findings to the public, others may argue that “research is for researchers” or even that psychological research is too complicated to communicate to the public. Despite the variety of opinions on the matter, the decision is ultimately up to members of the public: pay attention to research if one finds it interesting and valuable or ignore the research if not. The question is, then, how does the public receive psychological research? Indeed, past researchers have asked similar questions: is psychological research relevant (Giner-sorolla et al., 2018; Lilienfeld, 2012; Silverman, 1971)? Are psychologists asking the kinds of questions that are important and interesting to non-psychologists (Sherif, 1970)?

One way to examine the level of interest in psychological research is to examine how people engage with and share research on social media websites. Reddit is a free social media website with a large worldwide audience (500+ million subscribers). Reddit is the second-most used site for news and the 18th most-trafficked website (Alexa, n.d.) and is considered to be one of the most influential websites on the internet (Shaer, 2012). With such a large audience, this website provides an excellent window into online trends and the interests of the public.

One subsection of this website (called a “subreddit”) is called r/Science, which allows users to engage with the most recent scientific research and media coverage. All users are subscribed by default to this section. Anyone can post content, comment, “upvote” content they like, and “downvote” content they do not like. Further, posts are tagged by the poster as belonging to various scientific disciplines. However, the title of the post is often not the title of the published paper nor is the journal name required to be advertised. Further, posts to r/Science, must be peer-reviewed research (no summaries, secondhand reviews, pop articles, etc), the research must be less than 6 months old, and the subreddit is heavily moderated to enforce these rules. Thus, such a metric is a straightforward method of assessing whether psychological research is interesting to the average user and, qualitatively, what kinds of research topics are of most interest.

For the purposes of this investigation, data were downloaded from Reddit which include posts to r/Science from all of 2017 (data were downloaded in mid 2018, so 2017 was the most recent complete year). The graph below (Figure 1) depicts the mean upvotes per discipline. Posts tagged with “Psychology” have the highest mean upvotes (M = 2,759, SE = 302.95) followed closely by those tagged with “Social Science” (M = 2,602, SE = 447.34) and “Health” (M = 2,525, SE = 253.35). Clearly, Redditors are interested in research which deals with humans and psychology is a clear example of this.

Breaking down the data even further, I had four research assistants categorize each psychology post into a broad sub-discipline (social, developmental, clinical, or neuroscience) based on the topic of the research as indicated by title of the post. The coders disagreed on 28% of the titles; for these 252 titles, an independent coder reviewed the disagreements and served as the tie-breaker. Figure 2 shows the distributions of posts according to the four categories. Those categorized as social psychology were the most in number followed by clinical, developmental, and finally neuroscience, which received fewer than 1,000 upvotes.

Figure 1 

Mean upvotes by discipline tag on r/Science.

Note: Error bars represent +/– 1 SE.

Figure 2 

Distribution of Reddit upvotes by psychology sub-discipline.

Note: Each dot represents the number of upvotes accrued for a given article; the left half depicts the distribution of posts with fewer than 1,000 upvotes because this was the maximum for the neuroscience category; the right half depicts the distribution for the full range of upvotes.

The next goal was to examine the “Social Psychology” posts more closely. However, the post itself does not share the same name as the article, nor does it always link directly to the article (it may link to a press release). By following the link from the post on r/Science and searching the press releases, the journal in which each article was published was identified. This allowed for the journal-level metrics to be recorded, including H-index, SJR score, impact factor, and SciMago rankings (SCImago, n.d.). In addition, the number of citations from Google Scholar and the Altmetric score for each article were also recorded at the request of reviewers.

The first thing to notice about these data are that the posts with the highest number of upvotes came from a variety of journals (see Table 1), few of which are considered the highest-ranking journals in social psychology. Looking at the top 10 most upvoted posts offers a glimpse into the kinds of research Redditors find most interesting. One thing to notice about these data is that the top 10 most upvoted posts received between 37,000 and 74,000 upvotes: psychological research is hardly going unnoticed.

Table 1

Top 10 most upvoted articles/posts within social psychology.

Upvotes Journal Authors Post Title

74,636 Journal of Applied Research in Memory and Cognition Taylor, Wood, & Garry, (2017) In a new study, Americans disproportionately chose the years of their own youth as the countrys greatest years no matter how old they were now. This finding is the latest involving a phenomenon known as the reminiscence bump.
57,898 Journal of Experimental Social Psychology; Perspectives on Psychological Science Frimer, Skitka, & Motyl, (2017); Ditto et al., (2017) Two new psychology studies show that liberals are just as prone to confirmation bias as conservatives.
54,090 Frontiers in Psychology Hoehl, Hellmer, Johansson & Gredebäck, (2017) Snakes and spiders evoke fear and disgust in many people. There has been debate about whether this aversion is innate or learnt. Scientists discovered that babies as young as six months old feel stressed when seeing these creatures, long before they could have learnt this reaction.
52,571 Sex Roles Robinson, Anderson & White, (2017) Thanks to a decline in homophobia, straight men are now better able to embrace “bromances” with other men – which the study describes as being “more emotionally intimate, physically demonstrative, and based upon unrivaled trust and cohesion compared to their other friendships”.
48,250 Journal of Experimental Social Psychology Cowgill, Rios, & Simpson, (2017) Study: Atheists behave more fairly toward Christians than Christians behave toward atheists.
46,692 Personality and Individual Differences Weiser et al., (2017) Study of 550 college students who had used or were using Tinder finds about one in five participants admit to talking with a person on the dating app while in an exclusive romantic relationship.
44,505 Consciousness and Cognition LaFrance & Cuttler, (2017) A new study suggests that marijuana users are more creative, but it’s probably not because of their marijuana use.
43,125 Social Psychological and Personality Science Zitek & Jordan, (2017) People with a greater sense of entitlement are less likely to follow instructions than less entitled people are, because they view the instructions as an unfair imposition on them, finds new research in the journal Social Psychological and Personality Science.
39,140 Proceedings of the Royal Society B: Biological Sciences Brienza & Grossman, (2017) A study has found that people with the lowest social class scores, those with less income, less education, and more worries about money scored about twice as high on the wise reasoning scale as those in the highest social class.
37,583 Personality and Individual Differences Ståhl & van Prooijen, (2018) A person is more effective at analyzing fake news and conspiracy theories if they have a tendency for analytical thinking, which provides consistent protection against conspiratorial thinking and other irrational beliefs, but only if it was accompanied by a belief in the value of critical thinking.

While some of the most upvoted posts may belong to “top tier” journals, other higher-ranking journals frequently had their articles posted on Reddit even if these were not the most upvoted posts. Looking at the journals which had the most posts (see Figure 3) we see that journal status does not necessarily indicate public interest. The Journal of Experimental Social Psychology (IF = 2.87) had some of the most upvoted articles while Psychological Science (IF = 6.13) and Journal of Personality and Social Psychology (IF = 5.73), for example, had more posts with lower mean upvotes within a tighter range.

Figure 3 

Mean and range of upvotes for psychology journals with the most posts on Reddit.

Note: Mean upvotes by journal is depicted in the top plot and error bars indicate +/– 1 SE; the distribution of votes is depicted in the bottom plot with each dot representing the number of upvotes accrued by a given article; JESP = Journal of Experimental Psychology; PAID = Personality and Individual Differences; SPPS = Social Psychological and Personality Science; JPSP = Journal of Personality and Social Psychology; PSCI = Psychological Science; SCREP = Scientific Reports; PLOS = PLOS One; PSPB = Personality and Social Psychology Bulletin; PNAS = Proceedings of the National Academy of Sciences; NHB = Nature Human Behaviour.

Examining this data at the journal level changes the information we can glean. Not only are we asking what kinds of psychological research Reddit users find interesting, but we can now see whether the questions of interest and relevance to this audience are in any way related to metrics used by social psychologists. That is, are metrics like journal ranking and impact factor a good indicator of what research receives upvotes? It appears they are not.

For all posts categorized as belonging to social psychology, correlations between Reddit upvotes and the journal-level and article-level metrics were examined. Given the skew of upvotes, I also examined the natural-log of upvotes. In either case, Reddit upvotes were simply not correlated with impact factor or journal rankings (see Table 2). Please note that the n varies across correlations for each metric because not all metrics are available for all journals. For example, not all journals in the dataset have a ranking in the SciMago “social psychology” category and not all journals have published impact factors (SCImago, n.d.).

Table 2

Correlations between Reddit upvotes and metrics for all social psychology articles.

1 2 3 4 5 6 7 8 9 10

1 Reddit Upvotes
2 Reddit Upvotes (natural log) .66*
3 Journal Impact Factor –.06   –.06  
4 ournal Rank: Overall .01   –.04   –.46**
5 Journal Rank: Social Psyc. –.05   –.06   –.57** .99**
6 Journal Rank: Psychology .08   .04   –.66** .99** 1.00*  
7 Journal H-Index .02   –.03   –.11** .79** .76** .67**
8 Journal SJR Score –.06   –.06   .85** –.54** –.70** –.73** –.23**
9 Journal cites/3 years .01   .00   –.05     .03     –.68** .00     –.08     –.11*
10 Article citations –.01   .01   .10*   –.09*   –.13     –.12*   –.04     .15* .00
11 Article Altmetric Score .03   .10* .17** –.16** –1.0     –.09     –.08     .16* .06 .07

Note: These correlations were derived from all posts categorized as belonging to Social Psychology; not all metrics were available for all articles, so the n varies; ** p < .01, * p < .05.

Metrics such as impact factor and journal rankings are often factored into decisions for hiring, tenure promotions, and grants (but, see Abambres, Ribeira, Sousa, & Lantsoght, 2018, for an argument as to why impact factors are poor metrics of researcher performance). Thus, these results further suggest that the kind of research that makes it into top journals is not necessarily the same research that is upvoted on Reddit.

Importantly, this data contains information about many articles from the same journal, so it’s also important to check for interdependence. Thus, I attempted to construct a multi-level model to identify what proportion of variance in upvotes can be accounted for by the journal. However, there was almost no variance at the journal-level (ICC = .005), which means that the reliability of measurements within each journal are highly inconsistent. This suggests that a mixed model accounting for both article and journal level variance is not needed to understand the patterns in these data.

What do these results mean? First, these data show that Redditors care deeply about research which deals with humans—namely psychological research, and especially social psychological research: many posts were upvoted tens of thousands of times. For comparison, the other most upvoted r/Science categories were Social Science, Health, Engineering, and Environment. The first, tenth, and twentieth number of upvotes are displayed in Table 3. As you can see, social psychology is not too far ahead nor too far behind these other categories. Thus, one take-home message is that Redditors care about psychological research.

Table 3

Range of most upvoted articles in psychology as compared to other discipline tags.

Position Social Psychology Social Science Health Engineering Environment

1st 74,636 110,135 66,313 49,488 72,542
10th 37,583 43,269 27,367 27,990 35,862
20th 25,886 25,031 5,381 7,422 26,991

However, it’s also worth noting that posts in r/Science are still far from being the most upvoted posts on the entire Reddit website. For example, the most upvoted post of all time on Reddit received 349,000 upvotes (Reddit, n.d.) and Reddit posts from other subreddits frequently exceed 100,000 upvotes (Pereira, 2018).

The reasons why some articles are shared on Reddit and others are not is not a question that can be answered definitively with the present data. However, examining the ten most upvoted articles in Social Psychology (Table 1) suggests that users of r/Science appreciate descriptive research and research that offers up practical findings which describe something interesting about ourselves—for example, politics, religion, dating, personality, identity, etc.

In contrast, one can also speculate about the kinds of research which people find less interesting. For example, past research suggests that over time psychology has become more complicated, esoteric, and long-winded (Witte & Brandt, 2011), as well as overly theoretical and less descriptive (Rozin, 2001), and increasingly complex (Quiñones-Vidal, José López-García, Peñaranda-Ortega, & Tortosa-Gil, 2004; Sherman, Buddie, Dragan, End, & Finney, 1999). Compared to the descriptive and practical findings noted in Table 1, this combination of characteristics seems less likely to attract the average reader even though it may gain praise from scientists (Reis & Stiller, 1992).

Second, these data point to a disconnect between the metrics of evaluation used by scientists and general audiences. These data show that the research people find interesting on Reddit is not necessarily associated with scientific publishing metrics. Other research has reported that the public attention an article receives (e.g. Altmetric score) is not a good predictor of its future citation count (Thelwall & Nevill, 2018) which further suggests a disconnect between the metrics used by science and readers. There are a few possible explanations for this disconnect which can be explored in future research (but are not testable with the present data). For example, one could speculate that scientists may be concerned with theory and lay people may be concerned with application. Perhaps scientists want something to build their new findings on and lay people want findings that apply in the present moment. Indeed, the articles shown in Table 1 are more practical rather than theoretical. Another possibility is that maybe scientists see a different goal and a different audience for their science. Again, these possibilities are speculative but could be examined in further experimental studies. Whatever the reason, I think these data suggest that people find at least some psychological research interesting and perhaps “relevant” despite concerns otherwise.

Of course, the data presented here are not meant to be prescriptive; that is, I would not recommend that scientists go on Reddit to identify the areas that lay people find interesting and start conducting their research accordingly. Perhaps this would be a great way to decide on the topic for a popular science book, instead. However, the pursuit of flashy and surprising findings is one factor leading to large-scale non-replicability (Collaboration, 2015; Camerer et al., 2018), possibly because the qualities of risky and surprising findings differ from the kinds of research yielding expected (or “boring”) results (Fiedler, 2017).

Instead of planning out research based on what will be “popular,” perhaps we can reflect on who should be the beneficiaries of our science and whether our science is having the desired effect. As psychology is the field aiming to explain human behaviors (Rad, Martingano, & Ginges, 2018), should we be producing findings that can apply to people’s life in the present? From this data, it appears that psychology as a whole does fulfill that role (to some extent) whether scientists like it or not. Perhaps scientists should be trying to identify the areas of research which are most impactful and beneficial in everyday life. Research identifying these areas of study would be most welcome.

Therefore, these findings have some important implications for issues around the replicability and integrity of our research. People will pay attention to our findings, they will share them online, they will talk about them, and they will use them in their lives. We (scientists) then have a responsibility to make sure that our published findings are accurate and hold up to scrutiny. We have a responsibility to make sure that we share data, we are transparent in our analyses, and that we minimize the reporting of false-positives. We have a responsibility to make sure that we are communicating our results accurately because whether we like it or not, our research does not just go to researchers.

The take-home message is that there both is and isn’t a “crisis of relevance” (Giner-Sorolla, 2019). In one sense, there may be a crisis because the kinds of research that lay people find interesting is not necessarily the kind of research that gets cited or published in high-impact journals; this disconnect may be concerning. However, people (at least users of Reddit) clearly find some psychological research interesting enough if it is tied for the highest mean upvotes on r/Science. I hope that we, psychologists, can continue to garner interest and that we can continue to ask the right questions. At the same time, I hope the field can continue to improve the methods we use and the way we report our findings because people are watching eagerly.

Data Accessibility Statement

Data described herein can be accessed at: https://osf.io/bca4v/?view_only=6e3a3f379c1f4a86bbdec04fbe69746d.

Acknowledgements

Special thanks to Viet Duong and Phu Pham who accessed the data for me; special thanks to Thuy-vy Nguyen who provided comments on the drafts of this manuscript.

Competing Interests

The author has no competing interests to declare.

References

  1. Abambres, M., Ribeiro, T., Sousa, A., & Lantsoght, E. (2018). Research Counts, Not the Journal. OSF Preprints. DOI: https://doi.org/10.2139/ssrn.3368683 

  2. Alexa: Actionable Analytics for the Web. (2014). Analytics for Reddit.com. Retrieved on April 20th, 2019. http://www.alexa.com/siteinfo/Reddit.com 

  3. Brienza, J. P., & Grossmann, I. (2017). Social class and wise reasoning about interpersonal conflicts across regions, persons and situations. Proceedings of the Royal Society B: Biological Sciences, 284(1869). DOI: https://doi.org/10.1098/rspb.2017.1870 

  4. Camerer, C. F., Dreber, A., Holzmeister, F., Ho, T. H., Huber, J., Johannesson, M., Altmejd, A., et al. (2018). Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nature Human Behaviour, 2(9), 637. DOI: https://doi.org/10.1038/s41562-018-0399-z 

  5. Cowgill, C. M., Rios, K., & Simpson, A. (2017). Generous heathens? Reputational concerns and atheists’ behavior toward Christians in economic games. Journal of Experimental Social Psychology, 73, 169–179. DOI: https://doi.org/10.1016/j.jesp.2017.06.015 

  6. Ditto, P. H., Liu, B. S., Clark, C. J., Wojcik, S. P., Chen, E. E., Grady, R. H., & Zinger, J. F., et al. (2018). At least bias is bipartisan: A meta-analytic comparison of partisan bias in liberals and conservatives. Perspectives on Psychological Science. DOI: https://doi.org/10.1177/1745691617746796 

  7. Frimer, J. A., Skitka, L. J., & Motyl, M. (2017). Liberals and conservatives are similarly motivated to avoid exposure to one another’s opinions. Journal of Experimental Social Psychology, 72, 1–12. DOI: https://doi.org/10.1016/j.jesp.2017.04.003 

  8. Giner-Sorolla, R. (2019). From crisis of evidence to a “crisis” of relevance? Incentive-based answers for social psychology’s perennial relevance worries. European Review of Social Psychology, 30(1), 1–38. DOI: https://doi.org/10.31234/osf.io/nua9s 

  9. Hoehl, S., Hellmer, K., Johansson, M., & Gredebäck, G. (2017). Itsy bitsy spider…: infants react with increased arousal to spiders and snakes. Frontiers in psychology, 8, 1710. DOI: https://doi.org/10.3389/fpsyg.2017.01710 

  10. LaFrance, E. M., & Cuttler, C. (2017). Inspired by Mary Jane? Mechanisms underlying enhanced creativity in cannabis users. Consciousness and cognition, 56, 68–76. DOI: https://doi.org/10.1016/j.concog.2017.10.009 

  11. Lilienfeld, S. O. (2012). Public skepticism of psychology: Why many people perceive the study of human behavior as unscientific. The American Psychologist, 67, 111–129. DOI: https://doi.org/10.1037/a0023963 

  12. Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251). DOI: https://doi.org/10.1126/science.aac4716 

  13. Pereira, A. (2018). These were the most upvoted Reddit posts this year. Retrieved May 30th, 2019 from www.SFchronicle.com 

  14. Quiñones-Vidal, E., Loźpez-García, J. J., Peñaranda-Ortega, M., & Tortosa-Gil, F. (2004). The nature of social and personality psychology as reflected in JPSP, 1965–2000. Journal of Personality and Social Psychology, 86(3), 435. DOI: https://doi.org/10.1037/0022-3514.86.3.435 

  15. Rad, M. S., Martingano, A. J., & Ginges, J. (2018). Toward a psychology of Homo sapiens: Making psychological science more representative of the human population. Proceedings of the National Academy of Sciences, 115(45), 11401–11405. DOI: https://doi.org/10.1073/pnas.1721165115 

  16. Reddit (n.d.). The senate. Upvote this so that people see it when they Google “The Senate.” Retrieved May 20th, 2019 from: https://www.reddit.com/r/movies/comments/62sjuh/the_senate_upvote_this_so_that_people_see_it_when/?st=JIPKDMZC&sh=6fcba263 

  17. Robinson, S., Anderson, E., & White, A. (2018). The bromance: Undergraduate male friendships and the expansion of contemporary homosocial boundaries. Sex Roles, 78(1–2), 94–106. DOI: https://doi.org/10.1007/s11199-017-0768-5 

  18. Rozin, P. (2001). Social psychology and science: Some lessons from Solomon Asch. Personality and Social Psychology Review, 5(1), 2–14. DOI: https://doi.org/10.1207/S15327957PSPR0501_1 

  19. SCImago (n.d.). SJR — SCImago Journal & Country Rank [Portal]. Retrieved April 20th, 2019 from http://www.scimagojr.com 

  20. Shaer, M. (2012, July 8). “Reddit in the Flesh.” New York Magazine. Retrieved from http://nymag.com/news/features/Reddit-2012-7/ 

  21. Sherif, M. (1970). On the relevance of social psychology. American Psychologist, 25, 144–156. DOI: https://doi.org/10.1037/h0029429 

  22. Silverman, I. (1971). Crisis in social psychology: The relevance of relevance. American Psychologist, 26, 583. DOI: https://doi.org/10.1037/h0031445 

  23. Ståhl, T., & Van Prooijen, J. W. (2018). Epistemic rationality: Skepticism toward unfounded beliefs requires sufficient cognitive ability and motivation to be rational. Personality and Individual Differences, 122, 155–163. DOI: https://doi.org/10.1016/j.paid.2017.10.026 

  24. Taylor, R. J., Burton-Wood, C. G., & Garry, M. (2017). America was great when nationally relevant events occurred and when Americans were young. Journal of Applied Research in Memory and Cognition, 6(4), 425–433. DOI: https://doi.org/10.1016/j.jarmac.2017.05.003 

  25. Weiser, D. A., Niehuis, S., Flora, J., Punyanunt-Carter, N. M., Arias, V. S., & Baird, R. H. (2018). Swiping right: Sociosexuality, intentions to engage in infidelity, and infidelity experiences on Tinder. Personality and Individual Differences, 133, 29–33. DOI: https://doi.org/10.1016/j.paid.2017.10.025 

  26. Witte, E. H., & Brandt, V. (2011). Social psychological research: The comparison of four journals. Retrieved from http://psydok.sulb.uni-saarland.de/volltexte/2011/2743 

  27. Zitek, E. M., & Jordan, A. H. (2017). Psychological Entitlement Predicts Failure to Follow Instructions. Social Psychological and Personality Science. DOI: https://doi.org/10.1177/1948550617729885 

Peer Review Comments

The author(s) of this paper chose the Open Review option, and the peer review comments are available at: http://doi.org/10.1525/collabra.223.pr

comments powered by Disqus