- All eBooks & Audiobooks
- Academic eBook Collection
- Home Grown eBook Collection
- Off-Campus Access
- Literature Resource Center
- Opposing Viewpoints
- ProQuest Central
- Course Guides
- Citing Sources
- Library Research
- Websites by Topic
- Research Tutorials
- Use the Catalog
- Use Databases
- Use Films on Demand
- Use Home Grown eBooks
- Use NC LIVE
- Evaluating Sources
- Primary vs. Secondary
- Scholarly vs. Popular
- Make an Appointment
- Writing Tools
- Annotated Bibliographies
- Summaries, Reviews & Critiques
- Writing Center
How to Write Article Summaries, Reviews & Critiques
- Writing an article SUMMARY
- Writing an article REVIEW
Writing an article CRITIQUE
- Citing Sources This link opens in a new window
- About RCC Library
Email: [email protected]
Like us on Facebook
Links on this guide may go to external web sites not connected with Randolph Community College. Their inclusion is not an endorsement by Randolph Community College and the College is not responsible for the accuracy of their content or the security of their site.
A critique asks you to evaluate an article and the author’s argument. You will need to look critically at what the author is claiming, evaluate the research methods, and look for possible problems with, or applications of, the researcher’s claims.
Give an overview of the author’s main points and how the author supports those points. Explain what the author found and describe the process they used to arrive at this conclusion.
Interpret the information from the article:
- Does the author review previous studies? Is current and relevant research used?
- What type of research was used – empirical studies, anecdotal material, or personal observations?
- Was the sample too small to generalize from?
- Was the participant group lacking in diversity (race, gender, age, education, socioeconomic status, etc.)
- For instance, volunteers gathered at a health food store might have different attitudes about nutrition than the population at large.
- How useful does this work seem to you? How does the author suggest the findings could be applied and how do you believe they could be applied?
- How could the study have been improved in your opinion?
- Does the author appear to have any biases (related to gender, race, class, or politics)?
- Is the writing clear and easy to follow? Does the author’s tone add to or detract from the article?
- How useful are the visuals (such as tables, charts, maps, photographs) included, if any? How do they help to illustrate the argument? Are they confusing or hard to read?
- What further research might be conducted on this subject?
Try to synthesize the pieces of your critique to emphasize your own main points about the author’s work, relating the researcher’s work to your own knowledge or to topics being discussed in your course.
From the Center for Academic Excellence (opens in a new window), University of Saint Joseph Connecticut
All links open in a new window.
Writing an Article Critique (from The University of Arizona Global Campus Writing Center)
How to Critique an Article (from Essaypro.com)
How to Write an Article Critique (from EliteEditing.com.au)
How to Write an Article Critique Like a Pro (from Citetotal.com)
- << Previous: Writing an article REVIEW
- Next: Citing Sources >>
- Last Updated: Aug 16, 2023 11:47 AM
- URL: https://libguides.randolph.edu/summaries
IOE Writing Centre
- Writing a Critical Review
Writing a Critique
A critique (or critical review) is not to be mistaken for a literature review. A 'critical review', or 'critique', is a complete type of text (or genre), discussing one particular article or book in detail. In some instances, you may be asked to write a critique of two or three articles (e.g. a comparative critical review). In contrast, a 'literature review', which also needs to be 'critical', is a part of a larger type of text, such as a chapter of your dissertation.
Most importantly: Read your article / book as many times as possible, as this will make the critical review much easier.
1. Read and take notes 2. Organising your writing 3. Summary 4. Evaluation 5. Linguistic features of a critical review 6. Summary language 7. Evaluation language 8. Conclusion language 9. Example extracts from a critical review 10. Further resources
Read and Take Notes
To improve your reading confidence and efficiency, visit our pages on reading.
Further reading: Read Confidently
After you are familiar with the text, make notes on some of the following questions. Choose the questions which seem suitable:
- What kind of article is it (for example does it present data or does it present purely theoretical arguments)?
- What is the main area under discussion?
- What are the main findings?
- What are the stated limitations?
- Where does the author's data and evidence come from? Are they appropriate / sufficient?
- What are the main issues raised by the author?
- What questions are raised?
- How well are these questions addressed?
- What are the major points/interpretations made by the author in terms of the issues raised?
- Is the text balanced? Is it fair / biased?
- Does the author contradict herself?
- How does all this relate to other literature on this topic?
- How does all this relate to your own experience, ideas and views?
- What else has this author written? Do these build / complement this text?
- (Optional) Has anyone else reviewed this article? What did they say? Do I agree with them?
^ Back to top
Organising your writing
You first need to summarise the text that you have read. One reason to summarise the text is that the reader may not have read the text. In your summary, you will
- focus on points within the article that you think are interesting
- summarise the author(s) main ideas or argument
- explain how these ideas / argument have been constructed. (For example, is the author basing her arguments on data that they have collected? Are the main ideas / argument purely theoretical?)
In your summary you might answer the following questions: Why is this topic important? Where can this text be located? For example, does it address policy studies? What other prominent authors also write about this?
Evaluation is the most important part in a critical review.
Use the literature to support your views. You may also use your knowledge of conducting research, and your own experience. Evaluation can be explicit or implicit.
Explicit evaluation involves stating directly (explicitly) how you intend to evaluate the text. e.g. "I will review this article by focusing on the following questions. First, I will examine the extent to which the authors contribute to current thought on Second Language Acquisition (SLA) pedagogy. After that, I will analyse whether the authors' propositions are feasible within overseas SLA classrooms."
Implicit evaluation is less direct. The following section on Linguistic Features of Writing a Critical Review contains language that evaluates the text. A difficult part of evaluation of a published text (and a professional author) is how to do this as a student. There is nothing wrong with making your position as a student explicit and incorporating it into your evaluation. Examples of how you might do this can be found in the section on Linguistic Features of Writing a Critical Review. You need to remember to locate and analyse the author's argument when you are writing your critical review. For example, you need to locate the authors' view of classroom pedagogy as presented in the book / article and not present a critique of views of classroom pedagogy in general.
Linguistic features of a critical review
The following examples come from published critical reviews. Some of them have been adapted for student use.
- This article / book is divided into two / three parts. First...
- While the title might suggest...
- The tone appears to be...
- Title is the first / second volume in the series Title, edited by...The books / articles in this series address...
- The second / third claim is based on...
- The author challenges the notion that...
- The author tries to find a more middle ground / make more modest claims...
- The article / book begins with a short historical overview of...
- Numerous authors have recently suggested that...(see Author, Year; Author, Year). Author would also be once such author. With his / her argument that...
- To refer to title as a...is not to say that it is...
- This book / article is aimed at... This intended readership...
- The author's book / article examines the...To do this, the author first...
- The author develops / suggests a theoretical / pedagogical model to…
- This book / article positions itself firmly within the field of...
- The author in a series of subtle arguments, indicates that he / she...
- The argument is therefore...
- The author asks "..."
- With a purely critical / postmodern take on...
- Topic, as the author points out, can be viewed as...
- In this recent contribution to the field of...this British author...
- As a leading author in the field of...
- This book / article nicely contributes to the field of...and complements other work by this author...
- The second / third part of...provides / questions / asks the reader...
- Title is intended to encourage students / researchers to...
- The approach taken by the author provides the opportunity to examine...in a qualitative / quantitative research framework that nicely complements...
- The author notes / claims that state support / a focus on pedagogy / the adoption of...remains vital if...
- According to Author (Year) teaching towards examinations is not as effective as it is in other areas of the curriculum. This is because, as Author (Year) claims that examinations have undue status within the curriculum.
- According to Author (Year)…is not as effective in some areas of the curriculum / syllabus as others. Therefore the author believes that this is a reason for some school's…
- This argument is not entirely convincing, as...furthermore it commodifies / rationalises the...
- Over the last five / ten years the view of...has increasingly been viewed as 'complicated' (see Author, Year; Author, Year).
- However, through trying to integrate...with...the author...
- There are difficulties with such a position.
- Inevitably, several crucial questions are left unanswered / glossed over by this insightful / timely / interesting / stimulating book / article. Why should...
- It might have been more relevant for the author to have written this book / article as...
- This article / book is not without disappointment from those who would view...as...
- This chosen framework enlightens / clouds...
- This analysis intends to be...but falls a little short as...
- The authors rightly conclude that if...
- A detailed, well-written and rigorous account of...
- As a Korean student I feel that this article / book very clearly illustrates...
- The beginning of...provides an informative overview into...
- The tables / figures do little to help / greatly help the reader...
- The reaction by scholars who take a...approach might not be so favourable (e.g. Author, Year).
- This explanation has a few weaknesses that other researchers have pointed out (see Author, Year; Author, Year). The first is...
- On the other hand, the author wisely suggests / proposes that...By combining these two dimensions...
- The author's brief introduction to...may leave the intended reader confused as it fails to properly...
- Despite my inability to...I was greatly interested in...
- Even where this reader / I disagree(s), the author's effort to...
- The author thus combines...with...to argue...which seems quite improbable for a number of reasons. First...
- Perhaps this aversion to...would explain the author's reluctance to...
- As a second language student from ...I find it slightly ironic that such an anglo-centric view is...
- The reader is rewarded with...
- Less convincing is the broad-sweeping generalisation that...
- There is no denying the author's subject knowledge nor his / her...
- The author's prose is dense and littered with unnecessary jargon...
- The author's critique of...might seem harsh but is well supported within the literature (see Author, Year; Author, Year; Author, Year). Aligning herself with the author, Author (Year) states that...
- As it stands, the central focus of Title is well / poorly supported by its empirical findings...
- Given the hesitation to generalise to...the limitation of...does not seem problematic...
- For instance, the term...is never properly defined and the reader left to guess as to whether...
- Furthermore, to label...as...inadvertently misguides...
- In addition, this research proves to be timely / especially significant to... as recent government policy / proposals has / have been enacted to...
- On this well researched / documented basis the author emphasises / proposes that...
- Nonetheless, other research / scholarship / data tend to counter / contradict this possible trend / assumption...(see Author, Year; Author, Year).
- Without entering into detail of the..., it should be stated that Title should be read by...others will see little value in...
- As experimental conditions were not used in the study the word 'significant' misleads the reader.
- The article / book becomes repetitious in its assertion that...
- The thread of the author's argument becomes lost in an overuse of empirical data...
- Almost every argument presented in the final section is largely derivative, providing little to say about...
- She / he does not seem to take into consideration; however, that there are fundamental differences in the conditions of…
- As Author (Year) points out, however, it seems to be necessary to look at…
- This suggest that having low…does not necessarily indicate that…is ineffective.
- Therefore, the suggestion made by Author (Year)…is difficult to support.
- When considering all the data presented…it is not clear that the low scores of some students, indeed, reflects…
- Overall this article / book is an analytical look at...which within the field of...is often overlooked.
- Despite its problems, Title offers valuable theoretical insights / interesting examples / a contribution to pedagogy and a starting point for students / researchers of...with an interest in...
- This detailed and rigorously argued...
- This first / second volume / book / article by...with an interest in...is highly informative...
Example extracts from a critical review
If you have been told your writing is not critical enough, it probably means that your writing treats the knowledge claims as if they are true, well supported, and applicable in the context you are writing about. This may not always be the case.
In these two examples, the extracts refer to the same section of text. In each example, the section that refers to a source has been highlighted in bold. The note below the example then explains how the writer has used the source material.
There is a strong positive effect on students, both educationally and emotionally, when the instructors try to learn to say students' names without making pronunciation errors (Kiang, 2004).
Use of source material in example a:
This is a simple paraphrase with no critical comment. It looks like the writer agrees with Kiang. (This is not a good example for critical writing, as the writer has not made any critical comment).
Kiang (2004) gives various examples to support his claim that "the positive emotional and educational impact on students is clear" (p.210) when instructors try to pronounce students' names in the correct way. He quotes one student, Nguyet, as saying that he "felt surprised and happy" (p.211) when the tutor said his name clearly . The emotional effect claimed by Kiang is illustrated in quotes such as these, although the educational impact is supported more indirectly through the chapter. Overall, he provides more examples of students being negatively affected by incorrect pronunciation, and it is difficult to find examples within the text of a positive educational impact as such.
Use of source material in example b:
The writer describes Kiang's (2004) claim and the examples which he uses to try to support it. The writer then comments that the examples do not seem balanced and may not be enough to support the claims fully. This is a better example of writing which expresses criticality.
^Back to top
You may also be interested in our page on criticality, which covers criticality in general, and includes more critical reading questions.
Further reading: Read and Write Critically
We recommend that you do not search for other university guidelines on critical reviews. This is because the expectations may be different at other institutions. Ask your tutor for more guidance or examples if you have further questions.
IOE Writing Centre Online
Self-access resources from the Academic Writing Centre at the UCL Institute of Education.
Anonymous Suggestions Box
Information for Staff
Academic Writing Centre
Academic Writing Centre, UCL Institute of Education [email protected] Twitter: @AWC_IOE Skype: awc.ioe
SPH Writing Support Services
- Appointment System
- ESL Conversation Group
- Thesis/Dissertation Writing Group
- Career Writing
- Citing Sources
- Critiquing Research Articles
- Project Planning for the Beginner This link opens in a new window
- Grant Writing
- Publishing in the Sciences
- Systematic Review Overview
- Systematic Review Resources This link opens in a new window
- Writing Across Borders / Writing Across the Curriculum
- Conducting an article critique for a quantitative research study: Perspectives for doctoral students and other novice readers (Vance et al.)
- Critique Process (Boswell & Cannon)
- The experience of critiquing published research: Learning from the student and researcher perspective (Knowles & Gray)
- A guide to critiquing a research paper. Methodological appraisal of a paper on nurses in abortion care (Lipp & Fothergill)
- Step-by-step guide to critiquing research. Part 1: Quantitative research (Coughlan et al.)
- Step-by-step guide to critiquing research. Part 2: Qualitative research (Coughlan et al.)
- Critiquing Research Articles (Flinders University)
- Framework for How to Read and Critique a Research Study (American Nurses Association)
- How to Critique a Journal Article (UIS)
- How to Critique a Research Paper (University of Michigan)
- How to Write an Article Critique
- Research Article Critique Form
- Writing a Critique or Review of a Research Article (University of Calgary)
- The Critique Process: Reviewing and Critiquing Research
- Writing a Critique
- << Previous: Citing Sources
- Next: Project Planning for the Beginner >>
- Last Updated: Feb 12, 2024 1:11 PM
- URL: https://libguides.sph.uth.tmc.edu/writing_support_services
- Spartanburg Community College Library
- SCC Research Guides
- Writing a Critique
- Parts of a Critique
Depending on the source you are critiquing, your critique may not follow this exact format below. However, in general, critiques will be formatted in a similar way.
- The name of the source or event
- What kind of source it is (book, film, lecture, etc.)
- The name of the author or the speaker
- The author or speaker's experience/expertise on the topic
- The main argument in the source (or the thesis statement of the source)
- The intended (target) audience for the source or event
- The purpose of the source or the event
- Did the author/speaker well-support their thesis statement?
- Did the author use any interest supports (stories, humor, examples, interactions, personal experience, etc.). Were they effective?
- What kind of evidence did the author/speaker use in the source (statistics, facts, quotations, surveys, studies, interviews, expert opinions). Are these resources credible/reliable? Did the evidence add to or contradict the author/speaker's argument?
- Did the source have quality content (avoiding fillers, presented newsworthy information, kept audiences interested)?
- Did the source use any visual aids (PowerPoint, images, artwork, etc.). Did the visual aids match or enhance what the author/speaker was discussing? Were the visual aids clearly organized, spell-checked, and included citations?
- Did the speaker move well through different topics?
- If the source was a live event or a recording, was the speaker energetic? Did they talk to the crowd or did they look at their notes too much? Were you able to hear and understand the speaker?
- If you're critiquing a film, were the film techniques used effective?
- What was your overall impression of the source?
- Would you recommend this source to others? Why or why not?
- What are your final thoughts about the source?
- Sample Critique Paper Check out a sample critique essay of an event a student attended.
- How to Critique for a Live Performance (WOW) This worksheet will provide an outline of how to write a critique for the Wonders of Writing (WOW) event at SCC.
- How to Critique a Live/Zoom Presentation (Informational Presentation) This worksheet will show the outline for writing a critique for a live or a Zoom informational presentation.
- << Previous: Writing a Critique
- Next: Recommended Videos >>
- Recommended Videos
- Recommended Websites
Questions? Ask a Librarian
- Last Updated: Jan 17, 2024 10:29 AM
- URL: https://libguides.sccsc.edu/writingacritique
Giles Campus | 864.592.4764 | Toll Free 866.542.2779 | Contact Us
Copyright © 2024 Spartanburg Community College. All rights reserved.
Info for Library Staff | Guide Search
Return to SCC Website
Making sense of research: A guide for critiquing a paper
- 1 School of Nursing, Griffith University, Meadowbrook, Queensland.
- PMID: 16114192
- DOI: 10.5172/conu.14.1.38
Learning how to critique research articles is one of the fundamental skills of scholarship in any discipline. The range, quantity and quality of publications available today via print, electronic and Internet databases means it has become essential to equip students and practitioners with the prerequisites to judge the integrity and usefulness of published research. Finding, understanding and critiquing quality articles can be a difficult process. This article sets out some helpful indicators to assist the novice to make sense of research.
- Data Interpretation, Statistical
- Research Design
- Review Literature as Topic
How to Critique a Research Paper
Published 16 October, 2023
You might be asked by your teachers and professors in college to critique an article or research paper to reveal its authenticity through criticism. A research paper critique considers both negative and positive points of the research paper before giving any statement about the Research Paper. Students having no idea about how to critique a research paper can take the help of the following tips given for the research paper critique. My Research Topics experts give their guidance to critique a research paper to the students.
Useful tips and examples are cited below for the students to understand the process of critiquing a research paper with expertise. Make sure that you are not skipping these tips as professional research paper writers of My Research Topics have suggested these guiding tips to the students.
Read the Entire Research Paper before Start critiquing it
If you have been assigned with writing a research paper critique make sure that you have finished the primary reading of the research paper. It is very important to read the research paper carefully till the end It will help the students to find out certain useful leads to a critique of a research paper. Also, research topics for the critique also become easy to find. You can also take the help to critique a research paper from the professional research paper helpers of the My Research Topics. So do not show a casual approach a write high-quality research paper on the deadline assigned to you.
Note down some important arguments that are discussed in the paper
While reading the research paper it is the responsibility of the students to note down important points of the research paper. These important points include the arguments that the writer of the research paper has risen through his writing the paper. It will help the students to start critiquing their research paperwork. Students can pin these points on the paper and then while finding the research leads these arguments could serve this purpose easily. Help from the certified and professional research paper helpers could be taken by the students. High qualities help in critiquing arguments of the research paper is given to the students and that is to all the time.
Read Also: Discourse Analysis Research Methodology
Do research on the authenticity of these arguments and resources used
The next step in the process of article critique is to check the authenticity of the arguments that are written by the writers of a research paper. This authenticity could be checked only through performing research on the resources used and how authentic they were. If research is done by using authentic resources and by applying a correct research methodology authenticity of the research paper could be checked. Help in checking the research paper’s authenticity to the students is given by Myresearchtopics .com experts to the students. So make sure that you are taking this help to check whether the given arguments of the research paper are authentic or not.
Write the validity of the solutions given for these arguments
The next step in writing the research paper article is to check the validity of the solutions that research paper writers have given to the arguments of the topic. These solutions must be practical in nature which could be applied to real life. There are many solutions that the research paper contains that are of no use when it comes to applying them on the real grounds. Such solutions must be disclosed by the critique while critiquing a research paper. Help to critique the research paper solutions is given by the My Research Topics Experts to the students.
So if you are assigned such a task take the help of experts for this task. There is no value in the solutions given for the troubles discussed related to the research paper topics if they are not useful in the real part of life. A research paper critique must keep this point in mind before giving an unbiased research paper critique to the professors. A high score could be obtained by following this philosophy.
Give your personal critical analysis at the end
When all the above tasks are completed by the students the final step is to give an overall personal criticism on the research paper. It is what the critique personally feels about the research paper. But it does not mean that anything could be said about the research paper as every statement has to be substantiated with valid reasons and examples. So this is the most difficult task that students do in their personal critical analysis while writing a research paper critique. Seek the help of the My Research Topics experts to complete research paper critique assignments on time with these experts and submit them before the deadline.
Read Also: Thematic Analysis for research
Conclude your article critique
Finally, conclude your research paper critiquing at the end by giving a research conclusion at the end. You can give the overall review of the entire critique of the research paper that you have written so far. Both negative and positive aspects of the research paper need to be discussed at this stage. Do not include new points at this part of the time. You have to show the success of your article critique at this point in time. You have the option to seek help from the My Research Topics Experts as well. High-quality help in writing research paper critiques is given to the students by these experts.
Stuck During Your Dissertation
Our top dissertation writing experts are waiting 24/7 to assist you with your university project,from critical literature reviews to a complete PhD dissertation.
Other Related Guides
- Research Project Questions
- Types of Validity in Research – Explained With Examples
- Schizophrenia Sample Research Paper
- Quantitative Research Methods – Definitive Guide
- Research Paper On Homelessness For College Students
- How to Study for Biology Final Examination
- Textual Analysis in Research / Methods of Analyzing Text
A Guide to Start Research Process – Introduction, Procedure and Tips
Research findings – objectives , importance and techniques.
- Topic Sentences in Research Paper – Meaning, Parts, Importance, Procedure and Techniques
Recent Research Guides for 2023
Get 15% off your first order with My Research Topics
Connect with a professional writer within minutes by placing your first order. No matter the subject, difficulty, academic level or document type, our writers have the skills to complete it.
My Research Topics is provides assistance since 2004 to Research Students Globally. We help PhD, Psyd, MD, Mphil, Undergrad, High school, College, Masters students to compete their research paper & Dissertations. Our Step by step mentorship helps students to understand the research paper making process.
Research Topics & Ideas
- Sociological Research Paper Topics & Ideas For Students 2023
- Nurses Research Paper Topics & Ideas 2023
- Nursing Capstone Project Research Topics & Ideas 2023
- Unique Research Paper Topics & Ideas For Students 2023
- Teaching Research Paper Topics & Ideas 2023
- Literary Research Paper Topics & Ideas 2023
- Nursing Ethics Research Topics & Ideas 2023
Disclaimer: The Reference papers provided by the Myresearchtopics.com serve as model and sample papers for students and are not to be submitted as it is. These papers are intended to be used for reference and research purposes only.
A guide for critique of research articles
Following is the list of criteria to evaluate (critique) a research article. Please note that you should first summarize the paper and then evaluate different parts of it.
Most of the evaluation section should be devoted to evaluation of internal validity of the conclusions. Please add at the end a section entitled ''changes in the design/procedures if I want to replicate this study." Attach a copy of the original article to your paper.
Click here to see a an example (this is how you start) of a research critique.
Click here to see the original article.
The following list is a guide for you to organize your evaluation. It is recommended to organize your evaluation in this order. This is a long list of questions. You don’t have to address all questions. However, you should address highlighted questions . Some questions may not be relevant to your article.
1. Is there a statement of the problem?
2. Is the problem “researchable”? That is, can it be investigated through the collection and analysis of data?
3. Is background information on the problem presented?
4. Is the educational significance of the problem discussed?
5. Does the problem statement indicate the variables of interest and the specific relationship between those variables which are investigated? When necessary, are variables directly or operationally defined?
Review of Related Literature
1. Is the review comprehensive?
2. Are all cited references relevant to the problem under investigation?
3. Are most of the sources primary, i.e., are there only a few or no secondary sources?
4. Have the references been critically analyzed and the results of various studies compared and contrasted, i.e., is the review more than a series of abstracts or annotations?
5. Does the review conclude with a brief summary of the literature and its implications for the problem investigated?
6. Do the implications discussed form an empirical or theoretical rationale for the hypotheses which follow?
1. Are specific questions to be answered listed or specific hypotheses to be tested stated?
2. Does each hypothesis state an expected relationship or difference?
3. If necessary, are variables directly or operationally defined?
4. Is each hypothesis testable?
1. Are the size and major characteristics of the population studied described?
2. If a sample was selected, is the method of selecting the sample clearly described?
3. Is the method of sample selection described one that is likely to result in a representative, unbiased sample?
4. Did the researcher avoid the use of volunteers?
5. Are the size and major characteristics of the sample described?
6. Does the sample size meet the suggested guideline for minimum sample size appropriate for the method of research represented?
1. Is the rationale given for the selection of the instruments (or measurements) used?
2. Is each instrument described in terms of purpose and content?
3. Are the instruments appropriate for measuring the intended variables?
4. Is evidence presented that indicates that each instrument is appropriate for the sample under study?
5. Is instrument validity discussed and coefficients given if appropriate?
6. Is reliability discussed in terms of type and size of reliability coefficients?
7. If appropriate, are subtest reliabilities given?
8. If an instrument was developed specifically for the study, are the procedures involved in its development and validation described?
9. If an instrument was developed specifically for the study, are administration, scoring or tabulating, and interpretation procedures fully described?
Design and Procedure
1. Is the design appropriate for answering the questions or testing the hypotheses of the study?
2. Are the procedures described in sufficient detail to permit them to be replicated by another researcher?
3. If a pilot study was conducted, are its execution and results described as well as its impact on the subsequent study?
4. Are the control procedures described?
5. Did the researcher discuss or account for any potentially confounding variables that he or she was unable to control for?
1. Are appropriate descriptive or inferential statistics presented?
2. Was the probability level, α, at which the results of the tests of significance were evaluated,
specified in advance of the data analyses?
3. If parametric tests were used, is there evidence that the researcher avoided violating the
required assumptions for parametric tests?
4. Are the tests of significance described appropriate, given the hypotheses and design of the
5. Was every hypothesis tested?
6. Are the tests of significance interpreted using the appropriate degrees of freedom?
7. Are the results clearly presented?
8. Are the tables and figures (if any) well organized and easy to understand?
9. Are the data in each table and figure described in the text?
Discussion (Conclusions and Recommendation)
1. Is each result discussed in terms of the original hypothesis to which it relates?
2. Is each result discussed in terms of its agreement or disagreement with previous results
obtained by other researchers in other studies?
3. Are generalizations consistent with the results?
4. Are the possible effects of uncontrolled variables on the results discussed?
5. Are theoretical and practical implications of the findings discussed?
6. Are recommendations for future action made?
7. Are the suggestions for future action based on practical significance or on statistical
significance only, i.e., has the author avoided confusing practical and statistical
8. Are recommendations for future research made?
Additional general questions to be answered in your critique.
1. What is (are) the research question(s) (or hypothesis)?
2. Describe the sample used in this study.
3. Describe the reliability and validity of all the instruments used.
4. What type of research is this? Explain.
5. How was the data analyzed?
6. What is (are) the major finding(s)?
- Bipolar Disorder
- Therapy Center
- When To See a Therapist
- Types of Therapy
- Best Online Therapy
- Best Couples Therapy
- Best Family Therapy
- Managing Stress
- Sleep and Dreaming
- Understanding Emotions
- Healthy Relationships
- Student Resources
- Personality Types
- Verywell Mind Insights
- 2023 Verywell Mind 25
- Mental Health in the Classroom
- Editorial Process
- Meet Our Review Board
- Crisis Support
How to Write an Article Critique
Tips for Writing a Psychology Critique Paper
Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."
Emily is a board-certified science editor who has worked with top digital publishing brands like Voices for Biodiversity, Study.com, GoodTherapy, Vox, and Verywell.
Cultura RM / Gu Cultura / Getty Images
- Steps for Writing a Critique
Evaluating the Article
- How to Write It
- Helpful Tips
An article critique involves critically analyzing a written work to assess its strengths and flaws. If you need to write an article critique, you will need to describe the article, analyze its contents, interpret its meaning, and make an overall assessment of the importance of the work.
Critique papers require students to conduct a critical analysis of another piece of writing, often a book, journal article, or essay . No matter your major, you will probably be expected to write a critique paper at some point.
For psychology students, critiquing a professional paper is a great way to learn more about psychology articles, writing, and the research process itself. Students will analyze how researchers conduct experiments, interpret results, and discuss the impact of the results.
At a Glance
An article critique involves making a critical assessment of a single work. This is often an article, but it might also be a book or other written source. It summarizes the contents of the article and then evaluates both the strengths and weaknesses of the piece. Knowing how to write an article critique can help you learn how to evaluate sources with a discerning eye.
Steps for Writing an Effective Article Critique
While these tips are designed to help students write a psychology critique paper, many of the same principles apply to writing article critiques in other subject areas.
Your first step should always be a thorough read-through of the material you will be analyzing and critiquing. It needs to be more than just a casual skim read. It should be in-depth with an eye toward key elements.
To write an article critique, you should:
- Read the article , noting your first impressions, questions, thoughts, and observations
- Describe the contents of the article in your own words, focusing on the main themes or ideas
- Interpret the meaning of the article and its overall importance
- Critically evaluate the contents of the article, including any strong points as well as potential weaknesses
The following guidelines can help you assess the article you are reading and make better sense of the material.
Read the Introduction Section of the Article
Start by reading the introduction . Think about how this part of the article sets up the main body and how it helps you get a background on the topic.
- Is the hypothesis clearly stated?
- Is the necessary background information and previous research described in the introduction?
In addition to answering these basic questions, note other information provided in the introduction and any questions you have.
Read the Methods Section of the Article
Is the study procedure clearly outlined in the methods section ? Can you determine which variables the researchers are measuring?
Remember to jot down questions and thoughts that come to mind as you are reading. Once you have finished reading the paper, you can then refer back to your initial questions and see which ones remain unanswered.
Read the Results Section of the Article
Are all tables and graphs clearly labeled in the results section ? Do researchers provide enough statistical information? Did the researchers collect all of the data needed to measure the variables in question?
Make a note of any questions or information that does not seem to make sense. You can refer back to these questions later as you are writing your final critique.
Read the Discussion Section of the Article
Experts suggest that it is helpful to take notes while reading through sections of the paper you are evaluating. Ask yourself key questions:
- How do the researchers interpret the results of the study?
- Did the results support their hypothesis?
- Do the conclusions drawn by the researchers seem reasonable?
The discussion section offers students an excellent opportunity to take a position. If you agree with the researcher's conclusions, explain why. If you feel the researchers are incorrect or off-base, point out problems with the conclusions and suggest alternative explanations.
Another alternative is to point out questions the researchers failed to answer in the discussion section.
Begin Writing Your Own Critique of the Paper
Once you have read the article, compile your notes and develop an outline that you can follow as you write your psychology critique paper. Here's a guide that will walk you through how to structure your critique paper.
Begin your paper by describing the journal article and authors you are critiquing. Provide the main hypothesis (or thesis) of the paper. Explain why you think the information is relevant.
The final part of your introduction should include your thesis statement. Your thesis statement is the main idea of your critique. Your thesis should briefly sum up the main points of your critique.
Provide a brief summary of the article. Outline the main points, results, and discussion.
When describing the study or paper, experts suggest that you include a summary of the questions being addressed, study participants, interventions, comparisons, outcomes, and study design.
Don't get bogged down by your summary. This section should highlight the main points of the article you are critiquing. Don't feel obligated to summarize each little detail of the main paper. Focus on giving the reader an overall idea of the article's content.
In this section, you will provide your critique of the article. Describe any problems you had with the author's premise, methods, or conclusions. You might focus your critique on problems with the author's argument, presentation, information, and alternatives that have been overlooked.
When evaluating a study, summarize the main findings—including the strength of evidence for each main outcome—and consider their relevance to key demographic groups.
Organize your paper carefully. Be careful not to jump around from one argument to the next. Arguing one point at a time ensures that your paper flows well and is easy to read.
Your critique paper should end with an overview of the article's argument, your conclusions, and your reactions.
More Tips When Writing an Article Critique
- As you are editing your paper, utilize a style guide published by the American Psychological Association, such as the official Publication Manual of the American Psychological Association .
- Reading scientific articles can be challenging at first. Remember that this is a skill that takes time to learn but that your skills will become stronger the more that you read.
- Take a rough draft of your paper to your school's writing lab for additional feedback and use your university library's resources.
What This Means For You
Being able to write a solid article critique is a useful academic skill. While it can be challenging, start by breaking down the sections of the paper, noting your initial thoughts and questions. Then structure your own critique so that you present a summary followed by your evaluation. In your critique, include the strengths and the weaknesses of the article.
Archibald D, Martimianakis MA. Writing, reading, and critiquing reviews . Can Med Educ J . 2021;12(3):1-7. doi:10.36834/cmej.72945
Pautasso M. Ten simple rules for writing a literature review . PLoS Comput Biol . 2013;9(7):e1003149. doi:10.1371/journal.pcbi.1003149
Gülpınar Ö, Güçlü AG. How to write a review article? Turk J Urol . 2013;39(Suppl 1):44–48. doi:10.5152/tud.2013.054
Erol A. Basics of writing review articles . Noro Psikiyatr Ars . 2022;59(1):1-2. doi:10.29399/npa.28093
American Psychological Association. Publication Manual of the American Psychological Association (7th ed.). Washington DC: The American Psychological Association; 2019.
By Kendra Cherry, MSEd Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."
You are using an outdated browser
Unfortunately Ausmed.com does not support your browser. Please upgrade your browser to continue.
How to Critique a Research Article
Published: 01 October 2023
Let's briefly examine some basic pointers on how to perform a literature review.
If you've managed to get your hands on peer-reviewed articles, then you may wonder why it is necessary for you to perform your own article critique. Surely the article will be of good quality if it has made it through the peer-review process?
Unfortunately, this is not always the case.
Publication bias can occur when editors only accept manuscripts that have a bearing on the direction of their own research, or reject manuscripts with negative findings. Additionally, not all peer reviewers have expert knowledge on certain subject matters , which can introduce bias and sometimes a conflict of interest.
Performing your own critical analysis of an article allows you to consider its value to you and to your workplace.
Critical evaluation is defined as a systematic way of considering the truthfulness of a piece of research, its results and how relevant and applicable they are.
How to Critique
It can be a little overwhelming trying to critique an article when you're not sure where to start. Considering the article under the following headings may be of some use:
Title of Study/Research
You may be a better judge of this after reading the article, but the title should succinctly reflect the content of the work, stimulating readers' interest.
Three to six keywords that encapsulate the main topics of the research will have been drawn from the body of the article.
This should include:
- Evidence of a literature review that is relevant and recent, critically appraising other works rather than merely describing them
- Background information on the study to orientate the reader to the problem
- Hypothesis or aims of the study
- Rationale for the study that justifies its need, i.e. to explore an un-investigated gap in the literature.
Materials and Methods
Similar to a recipe, the description of materials and methods will allow others to replicate the study elsewhere if needed. It should both contain and justify the exact specifications of selection criteria, sample size, response rate and any statistics used. This will demonstrate how the study is capable of achieving its aims. Things to consider in this section are:
- What sort of sampling technique and size was used?
- What proportion of the eligible sample participated? (e.g. '553 responded to a survey sent to 750 medical technologists'
- Were all eligible groups sampled? (e.g. was the survey sent only in English?)
- What were the strengths and weaknesses of the study?
- Were there threats to the reliability and validity of the study, and were these controlled for?
- Were there any obvious biases?
- If a trial was undertaken, was it randomised, case-controlled, blinded or double-blinded?
Results should be statistically analysed and presented in a way that an average reader of the journal will understand. Graphs and tables should be clear and promote clarity of the text. Consider whether:
- There were any major omissions in the results, which could indicate bias
- Percentages have been used to disguise small sample sizes
- The data generated is consistent with the data collected.
Negative results are just as relevant as research that produces positive results (but, as mentioned previously, may be omitted in publication due to editorial bias).
This should show insight into the meaning and significance of the research findings. It should not introduce any new material but should address how the aims of the study have been met. The discussion should use previous research work and theoretical concepts as the context in which the new study can be interpreted. Any limitations of the study, including bias, should be clearly presented. You will need to evaluate whether the author has clearly interpreted the results of the study, or whether the results could be interpreted another way.
These should be clearly stated and will only be valid if the study was reliable, valid and used a representative sample size. There may also be recommendations for further research.
These should be relevant to the study, be up-to-date, and should provide a comprehensive list of citations within the text.
Undertaking a critique of a research article may seem challenging at first, but will help you to evaluate whether the article has relevance to your own practice and workplace. Reading a single article can act as a springboard into researching the topic more widely, and aids in ensuring your nursing practice remains current and is supported by existing literature.
- Marshall, G 2005, ‘Critiquing a Research Article’, Radiography , vol. 11, no. 1, viewed 2 October 2023, https://www.radiographyonline.com/article/S1078-8174(04)00119-1/fulltext
Sarah Vogel View profile
Help and feedback, publications.
Ausmed Education is a Trusted Information Partner of Healthdirect Australia. Verify here .
Writing a critique involves more than pointing out mistakes. It involves conducting a systematic analysis of a scholarly article or book and then writing a fair and reasonable description of its strengths and weaknesses. Several scholarly journals have published guides for critiquing other people’s work in their academic area. Search for a “manuscript reviewer guide” in your own discipline to guide your analysis of the content. Use this handout as an orientation to the audience and purpose of different types of critiques and to the linguistic strategies appropriate to all of them.
Types of critique
Article or book review assignment in an academic class.
Text: Article or book that has already been published Audience: Professors Purpose:
- to demonstrate your skills for close reading and analysis
- to show that you understand key concepts in your field
- to learn how to review a manuscript for your future professional work
Published book review
Text: Book that has already been published Audience: Disciplinary colleagues Purpose:
- to describe the book’s contents
- to summarize the book’s strengths and weaknesses
- to provide a reliable recommendation to read (or not read) the book
Text: Manuscript that has been submitted but has not been published yet Audience: Journal editor and manuscript authors Purpose:
- to provide the editor with an evaluation of the manuscript
- to recommend to the editor that the article be published, revised, or rejected
- to provide the authors with constructive feedback and reasonable suggestions for revision
Language strategies for critiquing
For each type of critique, it’s important to state your praise, criticism, and suggestions politely, but with the appropriate level of strength. The following language structures should help you achieve this challenging task.
Offering Praise and Criticism
A strategy called “hedging” will help you express praise or criticism with varying levels of strength. It will also help you express varying levels of certainty in your own assertions. Grammatical structures used for hedging include:
Modal verbs Using modal verbs (could, can, may, might, etc.) allows you to soften an absolute statement. Compare:
This text is inappropriate for graduate students who are new to the field. This text may be inappropriate for graduate students who are new to the field.
Qualifying adjectives and adverbs Using qualifying adjectives and adverbs (possible, likely, possibly, somewhat, etc.) allows you to introduce a level of probability into your comments. Compare:
Readers will find the theoretical model difficult to understand. Some readers will find the theoretical model difficult to understand. Some readers will probably find the theoretical model somewhat difficult to understand completely.
Note: You can see from the last example that too many qualifiers makes the idea sound undesirably weak.
Tentative verbs Using tentative verbs (seems, indicates, suggests, etc.) also allows you to soften an absolute statement. Compare:
This omission shows that the authors are not aware of the current literature. This omission indicates that the authors are not aware of the current literature. This omission seems to suggest that the authors are not aware of the current literature.
Whether you are critiquing a published or unpublished text, you are expected to point out problems and suggest solutions. If you are critiquing an unpublished manuscript, the author can use your suggestions to revise. Your suggestions have the potential to become real actions. If you are critiquing a published text, the author cannot revise, so your suggestions are purely hypothetical. These two situations require slightly different grammar.
Unpublished manuscripts: “would be X if they did Y” Reviewers commonly point out weakness by pointing toward improvement. For instance, if the problem is “unclear methodology,” reviewers may write that “the methodology would be more clear if …” plus a suggestion. If the author can use the suggestions to revise, the grammar is “X would be better if the authors did Y” (would be + simple past suggestion).
The tables would be clearer if the authors highlighted the key results. The discussion would be more persuasive if the authors accounted for the discrepancies in the data.
Published manuscripts: “would have been X if they had done Y” If the authors cannot revise based on your suggestions, use the past unreal conditional form “X would have been better if the authors had done Y” (would have been + past perfect suggestion).
The tables would have been clearer if the authors had highlighted key results. The discussion would have been more persuasive if the authors had accounted for discrepancies in the data.
Note: For more information on conditional structures, see our Conditionals handout .
Make a Gift
An official website of the United States government
The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.
The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.
- Account settings
- Advanced Search
- Journal List
- v.25(3); 2014 Oct
Peer Review in Scientific Publications: Benefits, Critiques, & A Survival Guide
1 Clinical Biochemistry, Department of Pediatric Laboratory Medicine, The Hospital for Sick Children, University of Toronto, Toronto, Ontario, Canada
2 Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, Canada
3 Chair, Communications and Publications Division (CPD), International Federation for Sick Clinical Chemistry (IFCC), Milan, Italy
The authors declare no conflicts of interest regarding publication of this article.
Peer review has been defined as a process of subjecting an author’s scholarly work, research or ideas to the scrutiny of others who are experts in the same field. It functions to encourage authors to meet the accepted high standards of their discipline and to control the dissemination of research data to ensure that unwarranted claims, unacceptable interpretations or personal views are not published without prior expert review. Despite its wide-spread use by most journals, the peer review process has also been widely criticised due to the slowness of the process to publish new findings and due to perceived bias by the editors and/or reviewers. Within the scientific community, peer review has become an essential component of the academic writing process. It helps ensure that papers published in scientific journals answer meaningful research questions and draw accurate conclusions based on professionally executed experimentation. Submission of low quality manuscripts has become increasingly prevalent, and peer review acts as a filter to prevent this work from reaching the scientific community. The major advantage of a peer review process is that peer-reviewed articles provide a trusted form of scientific communication. Since scientific knowledge is cumulative and builds on itself, this trust is particularly important. Despite the positive impacts of peer review, critics argue that the peer review process stifles innovation in experimentation, and acts as a poor screen against plagiarism. Despite its downfalls, there has not yet been a foolproof system developed to take the place of peer review, however, researchers have been looking into electronic means of improving the peer review process. Unfortunately, the recent explosion in online only/electronic journals has led to mass publication of a large number of scientific articles with little or no peer review. This poses significant risk to advances in scientific knowledge and its future potential. The current article summarizes the peer review process, highlights the pros and cons associated with different types of peer review, and describes new methods for improving peer review.
WHAT IS PEER REVIEW AND WHAT IS ITS PURPOSE?
Peer Review is defined as “a process of subjecting an author’s scholarly work, research or ideas to the scrutiny of others who are experts in the same field” ( 1 ). Peer review is intended to serve two primary purposes. Firstly, it acts as a filter to ensure that only high quality research is published, especially in reputable journals, by determining the validity, significance and originality of the study. Secondly, peer review is intended to improve the quality of manuscripts that are deemed suitable for publication. Peer reviewers provide suggestions to authors on how to improve the quality of their manuscripts, and also identify any errors that need correcting before publication.
HISTORY OF PEER REVIEW
The concept of peer review was developed long before the scholarly journal. In fact, the peer review process is thought to have been used as a method of evaluating written work since ancient Greece ( 2 ). The peer review process was first described by a physician named Ishaq bin Ali al-Rahwi of Syria, who lived from 854-931 CE, in his book Ethics of the Physician ( 2 ). There, he stated that physicians must take notes describing the state of their patients’ medical conditions upon each visit. Following treatment, the notes were scrutinized by a local medical council to determine whether the physician had met the required standards of medical care. If the medical council deemed that the appropriate standards were not met, the physician in question could receive a lawsuit from the maltreated patient ( 2 ).
The invention of the printing press in 1453 allowed written documents to be distributed to the general public ( 3 ). At this time, it became more important to regulate the quality of the written material that became publicly available, and editing by peers increased in prevalence. In 1620, Francis Bacon wrote the work Novum Organum, where he described what eventually became known as the first universal method for generating and assessing new science ( 3 ). His work was instrumental in shaping the Scientific Method ( 3 ). In 1665, the French Journal des sçavans and the English Philosophical Transactions of the Royal Society were the first scientific journals to systematically publish research results ( 4 ). Philosophical Transactions of the Royal Society is thought to be the first journal to formalize the peer review process in 1665 ( 5 ), however, it is important to note that peer review was initially introduced to help editors decide which manuscripts to publish in their journals, and at that time it did not serve to ensure the validity of the research ( 6 ). It did not take long for the peer review process to evolve, and shortly thereafter papers were distributed to reviewers with the intent of authenticating the integrity of the research study before publication. The Royal Society of Edinburgh adhered to the following peer review process, published in their Medical Essays and Observations in 1731: “Memoirs sent by correspondence are distributed according to the subject matter to those members who are most versed in these matters. The report of their identity is not known to the author.” ( 7 ). The Royal Society of London adopted this review procedure in 1752 and developed the “Committee on Papers” to review manuscripts before they were published in Philosophical Transactions ( 6 ).
Peer review in the systematized and institutionalized form has developed immensely since the Second World War, at least partly due to the large increase in scientific research during this period ( 7 ). It is now used not only to ensure that a scientific manuscript is experimentally and ethically sound, but also to determine which papers sufficiently meet the journal’s standards of quality and originality before publication. Peer review is now standard practice by most credible scientific journals, and is an essential part of determining the credibility and quality of work submitted.
IMPACT OF THE PEER REVIEW PROCESS
Peer review has become the foundation of the scholarly publication system because it effectively subjects an author’s work to the scrutiny of other experts in the field. Thus, it encourages authors to strive to produce high quality research that will advance the field. Peer review also supports and maintains integrity and authenticity in the advancement of science. A scientific hypothesis or statement is generally not accepted by the academic community unless it has been published in a peer-reviewed journal ( 8 ). The Institute for Scientific Information ( ISI ) only considers journals that are peer-reviewed as candidates to receive Impact Factors. Peer review is a well-established process which has been a formal part of scientific communication for over 300 years.
OVERVIEW OF THE PEER REVIEW PROCESS
The peer review process begins when a scientist completes a research study and writes a manuscript that describes the purpose, experimental design, results, and conclusions of the study. The scientist then submits this paper to a suitable journal that specializes in a relevant research field, a step referred to as pre-submission. The editors of the journal will review the paper to ensure that the subject matter is in line with that of the journal, and that it fits with the editorial platform. Very few papers pass this initial evaluation. If the journal editors feel the paper sufficiently meets these requirements and is written by a credible source, they will send the paper to accomplished researchers in the field for a formal peer review. Peer reviewers are also known as referees (this process is summarized in Figure 1 ). The role of the editor is to select the most appropriate manuscripts for the journal, and to implement and monitor the peer review process. Editors must ensure that peer reviews are conducted fairly, and in an effective and timely manner. They must also ensure that there are no conflicts of interest involved in the peer review process.
Overview of the review process
When a reviewer is provided with a paper, he or she reads it carefully and scrutinizes it to evaluate the validity of the science, the quality of the experimental design, and the appropriateness of the methods used. The reviewer also assesses the significance of the research, and judges whether the work will contribute to advancement in the field by evaluating the importance of the findings, and determining the originality of the research. Additionally, reviewers identify any scientific errors and references that are missing or incorrect. Peer reviewers give recommendations to the editor regarding whether the paper should be accepted, rejected, or improved before publication in the journal. The editor will mediate author-referee discussion in order to clarify the priority of certain referee requests, suggest areas that can be strengthened, and overrule reviewer recommendations that are beyond the study’s scope ( 9 ). If the paper is accepted, as per suggestion by the peer reviewer, the paper goes into the production stage, where it is tweaked and formatted by the editors, and finally published in the scientific journal. An overview of the review process is presented in Figure 1 .
WHO CONDUCTS REVIEWS?
Peer reviews are conducted by scientific experts with specialized knowledge on the content of the manuscript, as well as by scientists with a more general knowledge base. Peer reviewers can be anyone who has competence and expertise in the subject areas that the journal covers. Reviewers can range from young and up-and-coming researchers to old masters in the field. Often, the young reviewers are the most responsive and deliver the best quality reviews, though this is not always the case. On average, a reviewer will conduct approximately eight reviews per year, according to a study on peer review by the Publishing Research Consortium (PRC) ( 7 ). Journals will often have a pool of reviewers with diverse backgrounds to allow for many different perspectives. They will also keep a rather large reviewer bank, so that reviewers do not get burnt out, overwhelmed or time constrained from reviewing multiple articles simultaneously.
WHY DO REVIEWERS REVIEW?
Referees are typically not paid to conduct peer reviews and the process takes considerable effort, so the question is raised as to what incentive referees have to review at all. Some feel an academic duty to perform reviews, and are of the mentality that if their peers are expected to review their papers, then they should review the work of their peers as well. Reviewers may also have personal contacts with editors, and may want to assist as much as possible. Others review to keep up-to-date with the latest developments in their field, and reading new scientific papers is an effective way to do so. Some scientists use peer review as an opportunity to advance their own research as it stimulates new ideas and allows them to read about new experimental techniques. Other reviewers are keen on building associations with prestigious journals and editors and becoming part of their community, as sometimes reviewers who show dedication to the journal are later hired as editors. Some scientists see peer review as a chance to become aware of the latest research before their peers, and thus be first to develop new insights from the material. Finally, in terms of career development, peer reviewing can be desirable as it is often noted on one’s resume or CV. Many institutions consider a researcher’s involvement in peer review when assessing their performance for promotions ( 11 ). Peer reviewing can also be an effective way for a scientist to show their superiors that they are committed to their scientific field ( 5 ).
ARE REVIEWERS KEEN TO REVIEW?
A 2009 international survey of 4000 peer reviewers conducted by the charity Sense About Science at the British Science Festival at the University of Surrey, found that 90% of reviewers were keen to peer review ( 12 ). One third of respondents to the survey said they were happy to review up to five papers per year, and an additional one third of respondents were happy to review up to ten.
HOW LONG DOES IT TAKE TO REVIEW ONE PAPER?
On average, it takes approximately six hours to review one paper ( 12 ), however, this number may vary greatly depending on the content of the paper and the nature of the peer reviewer. One in every 100 participants in the “Sense About Science” survey claims to have taken more than 100 hours to review their last paper ( 12 ).
HOW TO DETERMINE IF A JOURNAL IS PEER REVIEWED
Ulrichsweb is a directory that provides information on over 300,000 periodicals, including information regarding which journals are peer reviewed ( 13 ). After logging into the system using an institutional login (eg. from the University of Toronto), search terms, journal titles or ISSN numbers can be entered into the search bar. The database provides the title, publisher, and country of origin of the journal, and indicates whether the journal is still actively publishing. The black book symbol (labelled ‘refereed’) reveals that the journal is peer reviewed.
THE EVALUATION CRITERIA FOR PEER REVIEW OF SCIENTIFIC PAPERS
As previously mentioned, when a reviewer receives a scientific manuscript, he/she will first determine if the subject matter is well suited for the content of the journal. The reviewer will then consider whether the research question is important and original, a process which may be aided by a literature scan of review articles.
Scientific papers submitted for peer review usually follow a specific structure that begins with the title, followed by the abstract, introduction, methodology, results, discussion, conclusions, and references. The title must be descriptive and include the concept and organism investigated, and potentially the variable manipulated and the systems used in the study. The peer reviewer evaluates if the title is descriptive enough, and ensures that it is clear and concise. A study by the National Association of Realtors (NAR) published by the Oxford University Press in 2006 indicated that the title of a manuscript plays a significant role in determining reader interest, as 72% of respondents said they could usually judge whether an article will be of interest to them based on the title and the author, while 13% of respondents claimed to always be able to do so ( 14 ).
The abstract is a summary of the paper, which briefly mentions the background or purpose, methods, key results, and major conclusions of the study. The peer reviewer assesses whether the abstract is sufficiently informative and if the content of the abstract is consistent with the rest of the paper. The NAR study indicated that 40% of respondents could determine whether an article would be of interest to them based on the abstract alone 60-80% of the time, while 32% could judge an article based on the abstract 80-100% of the time ( 14 ). This demonstrates that the abstract alone is often used to assess the value of an article.
The introduction of a scientific paper presents the research question in the context of what is already known about the topic, in order to identify why the question being studied is of interest to the scientific community, and what gap in knowledge the study aims to fill ( 15 ). The introduction identifies the study’s purpose and scope, briefly describes the general methods of investigation, and outlines the hypothesis and predictions ( 15 ). The peer reviewer determines whether the introduction provides sufficient background information on the research topic, and ensures that the research question and hypothesis are clearly identifiable.
The methods section describes the experimental procedures, and explains why each experiment was conducted. The methods section also includes the equipment and reagents used in the investigation. The methods section should be detailed enough that it can be used it to repeat the experiment ( 15 ). Methods are written in the past tense and in the active voice. The peer reviewer assesses whether the appropriate methods were used to answer the research question, and if they were written with sufficient detail. If information is missing from the methods section, it is the peer reviewer’s job to identify what details need to be added.
The results section is where the outcomes of the experiment and trends in the data are explained without judgement, bias or interpretation ( 15 ). This section can include statistical tests performed on the data, as well as figures and tables in addition to the text. The peer reviewer ensures that the results are described with sufficient detail, and determines their credibility. Reviewers also confirm that the text is consistent with the information presented in tables and figures, and that all figures and tables included are important and relevant ( 15 ). The peer reviewer will also make sure that table and figure captions are appropriate both contextually and in length, and that tables and figures present the data accurately.
The discussion section is where the data is analyzed. Here, the results are interpreted and related to past studies ( 15 ). The discussion describes the meaning and significance of the results in terms of the research question and hypothesis, and states whether the hypothesis was supported or rejected. This section may also provide possible explanations for unusual results and suggestions for future research ( 15 ). The discussion should end with a conclusions section that summarizes the major findings of the investigation. The peer reviewer determines whether the discussion is clear and focused, and whether the conclusions are an appropriate interpretation of the results. Reviewers also ensure that the discussion addresses the limitations of the study, any anomalies in the results, the relationship of the study to previous research, and the theoretical implications and practical applications of the study.
The references are found at the end of the paper, and list all of the information sources cited in the text to describe the background, methods, and/or interpret results. Depending on the citation method used, the references are listed in alphabetical order according to author last name, or numbered according to the order in which they appear in the paper. The peer reviewer ensures that references are used appropriately, cited accurately, formatted correctly, and that none are missing.
Finally, the peer reviewer determines whether the paper is clearly written and if the content seems logical. After thoroughly reading through the entire manuscript, they determine whether it meets the journal’s standards for publication,
and whether it falls within the top 25% of papers in its field ( 16 ) to determine priority for publication. An overview of what a peer reviewer looks for when evaluating a manuscript, in order of importance, is presented in Figure 2 .
How a peer review evaluates a manuscript
To increase the chance of success in the peer review process, the author must ensure that the paper fully complies with the journal guidelines before submission. The author must also be open to criticism and suggested revisions, and learn from mistakes made in previous submissions.
ADVANTAGES AND DISADVANTAGES OF THE DIFFERENT TYPES OF PEER REVIEW
The peer review process is generally conducted in one of three ways: open review, single-blind review, or double-blind review. In an open review, both the author of the paper and the peer reviewer know one another’s identity. Alternatively, in single-blind review, the reviewer’s identity is kept private, but the author’s identity is revealed to the reviewer. In double-blind review, the identities of both the reviewer and author are kept anonymous. Open peer review is advantageous in that it prevents the reviewer from leaving malicious comments, being careless, or procrastinating completion of the review ( 2 ). It encourages reviewers to be open and honest without being disrespectful. Open reviewing also discourages plagiarism amongst authors ( 2 ). On the other hand, open peer review can also prevent reviewers from being honest for fear of developing bad rapport with the author. The reviewer may withhold or tone down their criticisms in order to be polite ( 2 ). This is especially true when younger reviewers are given a more esteemed author’s work, in which case the reviewer may be hesitant to provide criticism for fear that it will damper their relationship with a superior ( 2 ). According to the Sense About Science survey, editors find that completely open reviewing decreases the number of people willing to participate, and leads to reviews of little value ( 12 ). In the aforementioned study by the PRC, only 23% of authors surveyed had experience with open peer review ( 7 ).
Single-blind peer review is by far the most common. In the PRC study, 85% of authors surveyed had experience with single-blind peer review ( 7 ). This method is advantageous as the reviewer is more likely to provide honest feedback when their identity is concealed ( 2 ). This allows the reviewer to make independent decisions without the influence of the author ( 2 ). The main disadvantage of reviewer anonymity, however, is that reviewers who receive manuscripts on subjects similar to their own research may be tempted to delay completing the review in order to publish their own data first ( 2 ).
Double-blind peer review is advantageous as it prevents the reviewer from being biased against the author based on their country of origin or previous work ( 2 ). This allows the paper to be judged based on the quality of the content, rather than the reputation of the author. The Sense About Science survey indicates that 76% of researchers think double-blind peer review is a good idea ( 12 ), and the PRC survey indicates that 45% of authors have had experience with double-blind peer review ( 7 ). The disadvantage of double-blind peer review is that, especially in niche areas of research, it can sometimes be easy for the reviewer to determine the identity of the author based on writing style, subject matter or self-citation, and thus, impart bias ( 2 ).
Masking the author’s identity from peer reviewers, as is the case in double-blind review, is generally thought to minimize bias and maintain review quality. A study by Justice et al. in 1998 investigated whether masking author identity affected the quality of the review ( 17 ). One hundred and eighteen manuscripts were randomized; 26 were peer reviewed as normal, and 92 were moved into the ‘intervention’ arm, where editor quality assessments were completed for 77 manuscripts and author quality assessments were completed for 40 manuscripts ( 17 ). There was no perceived difference in quality between the masked and unmasked reviews. Additionally, the masking itself was often unsuccessful, especially with well-known authors ( 17 ). However, a previous study conducted by McNutt et al. had different results ( 18 ). In this case, blinding was successful 73% of the time, and they found that when author identity was masked, the quality of review was slightly higher ( 18 ). Although Justice et al. argued that this difference was too small to be consequential, their study targeted only biomedical journals, and the results cannot be generalized to journals of a different subject matter ( 17 ). Additionally, there were problems masking the identities of well-known authors, introducing a flaw in the methods. Regardless, Justice et al. concluded that masking author identity from reviewers may not improve review quality ( 17 ).
In addition to open, single-blind and double-blind peer review, there are two experimental forms of peer review. In some cases, following publication, papers may be subjected to post-publication peer review. As many papers are now published online, the scientific community has the opportunity to comment on these papers, engage in online discussions and post a formal review. For example, online publishers PLOS and BioMed Central have enabled scientists to post comments on published papers if they are registered users of the site ( 10 ). Philica is another journal launched with this experimental form of peer review. Only 8% of authors surveyed in the PRC study had experience with post-publication review ( 7 ). Another experimental form of peer review called Dynamic Peer Review has also emerged. Dynamic peer review is conducted on websites such as Naboj, which allow scientists to conduct peer reviews on articles in the preprint media ( 19 ). The peer review is conducted on repositories and is a continuous process, which allows the public to see both the article and the reviews as the article is being developed ( 19 ). Dynamic peer review helps prevent plagiarism as the scientific community will already be familiar with the work before the peer reviewed version appears in print ( 19 ). Dynamic review also reduces the time lag between manuscript submission and publishing. An example of a preprint server is the ‘arXiv’ developed by Paul Ginsparg in 1991, which is used primarily by physicists ( 19 ). These alternative forms of peer review are still un-established and experimental. Traditional peer review is time-tested and still highly utilized. All methods of peer review have their advantages and deficiencies, and all are prone to error.
PEER REVIEW OF OPEN ACCESS JOURNALS
Open access (OA) journals are becoming increasingly popular as they allow the potential for widespread distribution of publications in a timely manner ( 20 ). Nevertheless, there can be issues regarding the peer review process of open access journals. In a study published in Science in 2013, John Bohannon submitted 304 slightly different versions of a fictional scientific paper (written by a fake author, working out of a non-existent institution) to a selected group of OA journals. This study was performed in order to determine whether papers submitted to OA journals are properly reviewed before publication in comparison to subscription-based journals. The journals in this study were selected from the Directory of Open Access Journals (DOAJ) and Biall’s List, a list of journals which are potentially predatory, and all required a fee for publishing ( 21 ). Of the 304 journals, 157 accepted a fake paper, suggesting that acceptance was based on financial interest rather than the quality of article itself, while 98 journals promptly rejected the fakes ( 21 ). Although this study highlights useful information on the problems associated with lower quality publishers that do not have an effective peer review system in place, the article also generalizes the study results to all OA journals, which can be detrimental to the general perception of OA journals. There were two limitations of the study that made it impossible to accurately determine the relationship between peer review and OA journals: 1) there was no control group (subscription-based journals), and 2) the fake papers were sent to a non-randomized selection of journals, resulting in bias.
JOURNAL ACCEPTANCE RATES
Based on a recent survey, the average acceptance rate for papers submitted to scientific journals is about 50% ( 7 ). Twenty percent of the submitted manuscripts that are not accepted are rejected prior to review, and 30% are rejected following review ( 7 ). Of the 50% accepted, 41% are accepted with the condition of revision, while only 9% are accepted without the request for revision ( 7 ).
SATISFACTION WITH THE PEER REVIEW SYSTEM
Based on a recent survey by the PRC, 64% of academics are satisfied with the current system of peer review, and only 12% claimed to be ‘dissatisfied’ ( 7 ). The large majority, 85%, agreed with the statement that ‘scientific communication is greatly helped by peer review’ ( 7 ). There was a similarly high level of support (83%) for the idea that peer review ‘provides control in scientific communication’ ( 7 ).
HOW TO PEER REVIEW EFFECTIVELY
The following are ten tips on how to be an effective peer reviewer as indicated by Brian Lucey, an expert on the subject ( 22 ):
1) Be professional
Peer review is a mutual responsibility among fellow scientists, and scientists are expected, as part of the academic community, to take part in peer review. If one is to expect others to review their work, they should commit to reviewing the work of others as well, and put effort into it.
2) Be pleasant
If the paper is of low quality, suggest that it be rejected, but do not leave ad hominem comments. There is no benefit to being ruthless.
3) Read the invite
When emailing a scientist to ask them to conduct a peer review, the majority of journals will provide a link to either accept or reject. Do not respond to the email, respond to the link.
4) Be helpful
Suggest how the authors can overcome the shortcomings in their paper. A review should guide the author on what is good and what needs work from the reviewer’s perspective.
5) Be scientific
The peer reviewer plays the role of a scientific peer, not an editor for proofreading or decision-making. Don’t fill a review with comments on editorial and typographic issues. Instead, focus on adding value with scientific knowledge and commenting on the credibility of the research conducted and conclusions drawn. If the paper has a lot of typographical errors, suggest that it be professionally proof edited as part of the review.
6) Be timely
Stick to the timeline given when conducting a peer review. Editors track who is reviewing what and when and will know if someone is late on completing a review. It is important to be timely both out of respect for the journal and the author, as well as to not develop a reputation of being late for review deadlines.
7) Be realistic
The peer reviewer must be realistic about the work presented, the changes they suggest and their role. Peer reviewers may set the bar too high for the paper they are editing by proposing changes that are too ambitious and editors must override them.
8) Be empathetic
Ensure that the review is scientific, helpful and courteous. Be sensitive and respectful with word choice and tone in a review.
Remember that both specialists and generalists can provide valuable insight when peer reviewing. Editors will try to get both specialised and general reviewers for any particular paper to allow for different perspectives. If someone is asked to review, the editor has determined they have a valid and useful role to play, even if the paper is not in their area of expertise.
10) Be organised
A review requires structure and logical flow. A reviewer should proofread their review before submitting it for structural, grammatical and spelling errors as well as for clarity. Most publishers provide short guides on structuring a peer review on their website. Begin with an overview of the proposed improvements; then provide feedback on the paper structure, the quality of data sources and methods of investigation used, the logical flow of argument, and the validity of conclusions drawn. Then provide feedback on style, voice and lexical concerns, with suggestions on how to improve.
In addition, the American Physiology Society (APS) recommends in its Peer Review 101 Handout that peer reviewers should put themselves in both the editor’s and author’s shoes to ensure that they provide what both the editor and the author need and expect ( 11 ). To please the editor, the reviewer should ensure that the peer review is completed on time, and that it provides clear explanations to back up recommendations. To be helpful to the author, the reviewer must ensure that their feedback is constructive. It is suggested that the reviewer take time to think about the paper; they should read it once, wait at least a day, and then re-read it before writing the review ( 11 ). The APS also suggests that Graduate students and researchers pay attention to how peer reviewers edit their work, as well as to what edits they find helpful, in order to learn how to peer review effectively ( 11 ). Additionally, it is suggested that Graduate students practice reviewing by editing their peers’ papers and asking a faculty member for feedback on their efforts. It is recommended that young scientists offer to peer review as often as possible in order to become skilled at the process ( 11 ). The majority of students, fellows and trainees do not get formal training in peer review, but rather learn by observing their mentors. According to the APS, one acquires experience through networking and referrals, and should therefore try to strengthen relationships with journal editors by offering to review manuscripts ( 11 ). The APS also suggests that experienced reviewers provide constructive feedback to students and junior colleagues on their peer review efforts, and encourages them to peer review to demonstrate the importance of this process in improving science ( 11 ).
The peer reviewer should only comment on areas of the manuscript that they are knowledgeable about ( 23 ). If there is any section of the manuscript they feel they are not qualified to review, they should mention this in their comments and not provide further feedback on that section. The peer reviewer is not permitted to share any part of the manuscript with a colleague (even if they may be more knowledgeable in the subject matter) without first obtaining permission from the editor ( 23 ). If a peer reviewer comes across something they are unsure of in the paper, they can consult the literature to try and gain insight. It is important for scientists to remember that if a paper can be improved by the expertise of one of their colleagues, the journal must be informed of the colleague’s help, and approval must be obtained for their colleague to read the protected document. Additionally, the colleague must be identified in the confidential comments to the editor, in order to ensure that he/she is appropriately credited for any contributions ( 23 ). It is the job of the reviewer to make sure that the colleague assisting is aware of the confidentiality of the peer review process ( 23 ). Once the review is complete, the manuscript must be destroyed and cannot be saved electronically by the reviewers ( 23 ).
COMMON ERRORS IN SCIENTIFIC PAPERS
When performing a peer review, there are some common scientific errors to look out for. Most of these errors are violations of logic and common sense: these may include contradicting statements, unwarranted conclusions, suggestion of causation when there is only support for correlation, inappropriate extrapolation, circular reasoning, or pursuit of a trivial question ( 24 ). It is also common for authors to suggest that two variables are different because the effects of one variable are statistically significant while the effects of the other variable are not, rather than directly comparing the two variables ( 24 ). Authors sometimes oversee a confounding variable and do not control for it, or forget to include important details on how their experiments were controlled or the physical state of the organisms studied ( 24 ). Another common fault is the author’s failure to define terms or use words with precision, as these practices can mislead readers ( 24 ). Jargon and/or misused terms can be a serious problem in papers. Inaccurate statements about specific citations are also a common occurrence ( 24 ). Additionally, many studies produce knowledge that can be applied to areas of science outside the scope of the original study, therefore it is better for reviewers to look at the novelty of the idea, conclusions, data, and methodology, rather than scrutinize whether or not the paper answered the specific question at hand ( 24 ). Although it is important to recognize these points, when performing a review it is generally better practice for the peer reviewer to not focus on a checklist of things that could be wrong, but rather carefully identify the problems specific to each paper and continuously ask themselves if anything is missing ( 24 ). An extremely detailed description of how to conduct peer review effectively is presented in the paper How I Review an Original Scientific Article written by Frederic G. Hoppin, Jr. It can be accessed through the American Physiological Society website under the Peer Review Resources section.
CRITICISM OF PEER REVIEW
A major criticism of peer review is that there is little evidence that the process actually works, that it is actually an effective screen for good quality scientific work, and that it actually improves the quality of scientific literature. As a 2002 study published in the Journal of the American Medical Association concluded, ‘Editorial peer review, although widely used, is largely untested and its effects are uncertain’ ( 25 ). Critics also argue that peer review is not effective at detecting errors. Highlighting this point, an experiment by Godlee et al. published in the British Medical Journal (BMJ) inserted eight deliberate errors into a paper that was nearly ready for publication, and then sent the paper to 420 potential reviewers ( 7 ). Of the 420 reviewers that received the paper, 221 (53%) responded, the average number of errors spotted by reviewers was two, no reviewer spotted more than five errors, and 35 reviewers (16%) did not spot any.
Another criticism of peer review is that the process is not conducted thoroughly by scientific conferences with the goal of obtaining large numbers of submitted papers. Such conferences often accept any paper sent in, regardless of its credibility or the prevalence of errors, because the more papers they accept, the more money they can make from author registration fees ( 26 ). This misconduct was exposed in 2014 by three MIT graduate students by the names of Jeremy Stribling, Dan Aguayo and Maxwell Krohn, who developed a simple computer program called SCIgen that generates nonsense papers and presents them as scientific papers ( 26 ). Subsequently, a nonsense SCIgen paper submitted to a conference was promptly accepted. Nature recently reported that French researcher Cyril Labbé discovered that sixteen SCIgen nonsense papers had been used by the German academic publisher Springer ( 26 ). Over 100 nonsense papers generated by SCIgen were published by the US Institute of Electrical and Electronic Engineers (IEEE) ( 26 ). Both organisations have been working to remove the papers. Labbé developed a program to detect SCIgen papers and has made it freely available to ensure publishers and conference organizers do not accept nonsense work in the future. It is available at this link: http://scigendetect.on.imag.fr/main.php ( 26 ).
Additionally, peer review is often criticized for being unable to accurately detect plagiarism. However, many believe that detecting plagiarism cannot practically be included as a component of peer review. As explained by Alice Tuff, development manager at Sense About Science, ‘The vast majority of authors and reviewers think peer review should detect plagiarism (81%) but only a minority (38%) think it is capable. The academic time involved in detecting plagiarism through peer review would cause the system to grind to a halt’ ( 27 ). Publishing house Elsevier began developing electronic plagiarism tools with the help of journal editors in 2009 to help improve this issue ( 27 ).
It has also been argued that peer review has lowered research quality by limiting creativity amongst researchers. Proponents of this view claim that peer review has repressed scientists from pursuing innovative research ideas and bold research questions that have the potential to make major advances and paradigm shifts in the field, as they believe that this work will likely be rejected by their peers upon review ( 28 ). Indeed, in some cases peer review may result in rejection of innovative research, as some studies may not seem particularly strong initially, yet may be capable of yielding very interesting and useful developments when examined under different circumstances, or in the light of new information ( 28 ). Scientists that do not believe in peer review argue that the process stifles the development of ingenious ideas, and thus the release of fresh knowledge and new developments into the scientific community.
Another issue that peer review is criticized for, is that there are a limited number of people that are competent to conduct peer review compared to the vast number of papers that need reviewing. An enormous number of papers published (1.3 million papers in 23,750 journals in 2006), but the number of competent peer reviewers available could not have reviewed them all ( 29 ). Thus, people who lack the required expertise to analyze the quality of a research paper are conducting reviews, and weak papers are being accepted as a result. It is now possible to publish any paper in an obscure journal that claims to be peer-reviewed, though the paper or journal itself could be substandard ( 29 ). On a similar note, the US National Library of Medicine indexes 39 journals that specialize in alternative medicine, and though they all identify themselves as “peer-reviewed”, they rarely publish any high quality research ( 29 ). This highlights the fact that peer review of more controversial or specialized work is typically performed by people who are interested and hold similar views or opinions as the author, which can cause bias in their review. For instance, a paper on homeopathy is likely to be reviewed by fellow practicing homeopaths, and thus is likely to be accepted as credible, though other scientists may find the paper to be nonsense ( 29 ). In some cases, papers are initially published, but their credibility is challenged at a later date and they are subsequently retracted. Retraction Watch is a website dedicated to revealing papers that have been retracted after publishing, potentially due to improper peer review ( 30 ).
Additionally, despite its many positive outcomes, peer review is also criticized for being a delay to the dissemination of new knowledge into the scientific community, and as an unpaid-activity that takes scientists’ time away from activities that they would otherwise prioritize, such as research and teaching, for which they are paid ( 31 ). As described by Eva Amsen, Outreach Director for F1000Research, peer review was originally developed as a means of helping editors choose which papers to publish when journals had to limit the number of papers they could print in one issue ( 32 ). However, nowadays most journals are available online, either exclusively or in addition to print, and many journals have very limited printing runs ( 32 ). Since there are no longer page limits to journals, any good work can and should be published. Consequently, being selective for the purpose of saving space in a journal is no longer a valid excuse that peer reviewers can use to reject a paper ( 32 ). However, some reviewers have used this excuse when they have personal ulterior motives, such as getting their own research published first.
RECENT INITIATIVES TOWARDS IMPROVING PEER REVIEW
F1000Research was launched in January 2013 by Faculty of 1000 as an open access journal that immediately publishes papers (after an initial check to ensure that the paper is in fact produced by a scientist and has not been plagiarised), and then conducts transparent post-publication peer review ( 32 ). F1000Research aims to prevent delays in new science reaching the academic community that are caused by prolonged publication times ( 32 ). It also aims to make peer reviewing more fair by eliminating any anonymity, which prevents reviewers from delaying the completion of a review so they can publish their own similar work first ( 32 ). F1000Research offers completely open peer review, where everything is published, including the name of the reviewers, their review reports, and the editorial decision letters ( 32 ).
PeerJ was founded by Jason Hoyt and Peter Binfield in June 2012 as an open access, peer reviewed scholarly journal for the Biological and Medical Sciences ( 33 ). PeerJ selects articles to publish based only on scientific and methodological soundness, not on subjective determinants of ‘impact ’, ‘novelty’ or ‘interest’ ( 34 ). It works on a “lifetime publishing plan” model which charges scientists for publishing plans that give them lifetime rights to publish with PeerJ, rather than charging them per publication ( 34 ). PeerJ also encourages open peer review, and authors are given the option to post the full peer review history of their submission with their published article ( 34 ). PeerJ also offers a pre-print review service called PeerJ Pre-prints, in which paper drafts are reviewed before being sent to PeerJ to publish ( 34 ).
Rubriq is an independent peer review service designed by Shashi Mudunuri and Keith Collier to improve the peer review system ( 35 ). Rubriq is intended to decrease redundancy in the peer review process so that the time lost in redundant reviewing can be put back into research ( 35 ). According to Keith Collier, over 15 million hours are lost each year to redundant peer review, as papers get rejected from one journal and are subsequently submitted to a less prestigious journal where they are reviewed again ( 35 ). Authors often have to submit their manuscript to multiple journals, and are often rejected multiple times before they find the right match. This process could take months or even years ( 35 ). Rubriq makes peer review portable in order to help authors choose the journal that is best suited for their manuscript from the beginning, thus reducing the time before their paper is published ( 35 ). Rubriq operates under an author-pay model, in which the author pays a fee and their manuscript undergoes double-blind peer review by three expert academic reviewers using a standardized scorecard ( 35 ). The majority of the author’s fee goes towards a reviewer honorarium ( 35 ). The papers are also screened for plagiarism using iThenticate ( 35 ). Once the manuscript has been reviewed by the three experts, the most appropriate journal for submission is determined based on the topic and quality of the paper ( 35 ). The paper is returned to the author in 1-2 weeks with the Rubriq Report ( 35 ). The author can then submit their paper to the suggested journal with the Rubriq Report attached. The Rubriq Report will give the journal editors a much stronger incentive to consider the paper as it shows that three experts have recommended the paper to them ( 35 ). Rubriq also has its benefits for reviewers; the Rubriq scorecard gives structure to the peer review process, and thus makes it consistent and efficient, which decreases time and stress for the reviewer. Reviewers also receive feedback on their reviews and most significantly, they are compensated for their time ( 35 ). Journals also benefit, as they receive pre-screened papers, reducing the number of papers sent to their own reviewers, which often end up rejected ( 35 ). This can reduce reviewer fatigue, and allow only higher-quality articles to be sent to their peer reviewers ( 35 ).
According to Eva Amsen, peer review and scientific publishing are moving in a new direction, in which all papers will be posted online, and a post-publication peer review will take place that is independent of specific journal criteria and solely focused on improving paper quality ( 32 ). Journals will then choose papers that they find relevant based on the peer reviews and publish those papers as a collection ( 32 ). In this process, peer review and individual journals are uncoupled ( 32 ). In Keith Collier’s opinion, post-publication peer review is likely to become more prevalent as a complement to pre-publication peer review, but not as a replacement ( 35 ). Post-publication peer review will not serve to identify errors and fraud but will provide an additional measurement of impact ( 35 ). Collier also believes that as journals and publishers consolidate into larger systems, there will be stronger potential for “cascading” and shared peer review ( 35 ).
Peer review has become fundamental in assisting editors in selecting credible, high quality, novel and interesting research papers to publish in scientific journals and to ensure the correction of any errors or issues present in submitted papers. Though the peer review process still has some flaws and deficiencies, a more suitable screening method for scientific papers has not yet been proposed or developed. Researchers have begun and must continue to look for means of addressing the current issues with peer review to ensure that it is a full-proof system that ensures only quality research papers are released into the scientific community.
- Queen's University Library
- Research Guides
How to Critique an Article (Psychology)
- The introduction is a justification for why the study was conducted.
- By the end of the introduction you should have a very good idea of what the researchers are going to study, and be convinced that the study is absolutely necessary to advance the field.
- The justification should be a combination of improving on previous research and good theoretical reasons and practical reasons for why the study is important.
- If the authors are talking about a controversial issue, are they presenting both sides in a reasonable way? Is their choice of one side over the other based on hard evidence?
- Do you understand what their hypotheses are e.g. what they expect to find?
- It is not good enough just to say that the study has not been done before. There are plenty of topics that have not been scientifically researched before but that doesn't mean that they should be. For example, I doubt that anyone has ever looked at the correlation between favorite color of Skittles and personality, but that doesn't mean that it should be researched unless there is a good theoretical reason for why we would expect a relationship and a good reason to think that knowing the relationship would advance our understanding of personality in some meaningful way.
- Last Updated: Nov 5, 2021 9:46 AM
- Subjects: Psychology
- View all journals
- Explore content
- About the journal
- Publish with us
- Sign up for alerts
- 07 February 2024
Fake research papers flagged by analysing authorship trends
- Dalmeet Singh Chawla
You can also search for this author in PubMed Google Scholar
A new method searches the scholarly literature for trends in authorship that indicate paper-mill activity. Credit: Zoonar GmbH/Alamy
A research-technology firm has developed a new approach to help identify journal articles that originate from paper mills — companies that churn out fake or poor-quality studies and sell authorships.
The technique, described in a preprint posted on arXiv last month 1 , uses factors such as the combination of a paper’s authors to flag suspicious studies. Its developers at London-based firm Digital Science say it can help to identify cases in which researchers might have bought their way onto a paper.
Science’s fake-paper problem: high-profile effort will tackle paper mills
Previous efforts to detect the products of paper mills have tended to focus on analysing the content of the manuscripts. One online tool, for example, searches papers for tortured phrases — strange alternative turns of phrase for existing terminology produced by software designed to avoid plagiarism detection. Another tool, being piloted by the International Association of Scientific, Technical, and Medical Publishers (STM), flags when identical manuscripts are submitted to several journals or publishers at the same time.
An approach that instead analyses the relationships between authors could be valuable as paper mills become better at producing convincing text , says Hylke Koers, chief information officer at the STM, who is based in Utrecht, the Netherlands. “This is the kind of signal that is much more difficult to work around or outcompete by clever use of generative AI.”
Paper mills are a growing problem for publishers — according to one estimate, around 2% of all published papers in 2022 resembled studies produced by paper mills — and in recent years publishers have stepped up efforts to tackle them .
As well as being of poor quality, often containing made-up data and nonsensical text, the articles that paper mills churn out are frequently padded with researchers who buy authorship on manuscripts already accepted for publication . Some paper mills claim to have brokered tens of thousands of authorships — including in journals that are indexed in respected databases, such as Web of Science and Scopus.
This can create unusual patterns of co-authorship and networks of researchers that are different from those in legitimate research, says Simon Porter, vice-president for research futures at Digital Science.
Multimillion-dollar trade in paper authorships alarms publishers
Under normal circumstances, “you would expect to find behaviour where a young researcher is publishing with their supervisor, and starts to branch out a little later and publish with other people”, Porter says. “You can see an evolution; it’s not a random network.”
This is not the case with paper-mill works. The technology that Porter developed, together with Leslie McIntosh, vice-president for research integrity at Digital Science, searches for trends that indicate paper-mill activity. These include co-author networks composed of early-career researchers who suddenly have a spike in publications, and papers featuring several authors who have no publication history or a collection of collaborators who are unlikely to have worked together, such as authors from several locations or unrelated disciplines.
When they compared the new technique’s results with those of the Problematic Paper Screener , a tool that searches for tortured phrases and other red flags, Porter and McIntosh identified a significant overlap. Around 10% of authors were directly flagged by both tools, their study found, and 72% of authors in the ‘author networks’ data set can be linked through co-authorship to those in the ‘tortured phrases’ data set.
Although paper mills have quickly evolved so that fewer papers with tortured phrases are being published, Porter thinks the companies will find it difficult to circumvent flagging by these tools while keeping their current business model.
Digital Science has posted the code underlying the technique online , and Porter says that publishers could begin using it straight away.
Joris Van Rossum, programme director at STM Solutions in Amsterdam, says his organization will consider adding the new technology to the STM Integrity Hub — a collection of resources and tools designed to help publishers to detect fraudulent papers.
AI intensifies fight against ‘paper mills’ that churn out fake research
Chris Graf, research-integrity director at Springer Nature in London, says that obstacles remain, particularly in distinguishing between researchers who share a name and weeding out authors who are flagged erroneously. “We have found that there can be some challenges with data consistency in this context that mean this is not straightforward,” Graf says. “Very brilliant young researchers with a low cluster coefficient could show up as false positives, which is clearly far from ideal.” But he adds: “Having said that, we are exploring a lot of different options, and nothing is off the table.” ( Nature ’s news team is independent of Springer Nature, its publisher.)
Anna Abalkina, a sociologist at the Free University of Berlin who has been tracking paper-mill studies for years, says it’s a good idea to scrutinize author networks. “Paper mills definitely do have collaboration anomalies,” she says.
Abalkina warns, however, that our knowledge of paper mills’ business models and processes is limited. It is also difficult to prove that a published study is definitely the product of a paper mill, she notes, which makes it hard to use that as a reason for retraction.
Ultimately, “it’s going to take every trick in the book to be able to provide a convincing filter for paper mills”, Porter says. “It won’t just be one technique.”
Porter, S. J. & McIntosh, L. D. Preprint at arXiv https://doi.org/10.48550/arXiv.2401.04022 (2024).
Reprints and permissions
- Scientific community
How journals are fighting back against a wave of questionable images
News 12 FEB 24
‘Obviously ChatGPT’ — how reviewers accused me of scientific fraud
Career Column 05 FEB 24
What’s needed to rebuild Saudi Arabia’s research reputation
Correspondence 30 JAN 24
Largest post-pandemic survey finds trust in scientists is high
News 14 FEB 24
Build global collaborations to protect marine migration routes
Correspondence 13 FEB 24
Deep-sea mining opponents: there’s no free lunch when it comes to clean energy
Open-access publishing: citation advantage is unproven
China conducts first nationwide review of retractions and research misconduct
Attending Physician (m/f/d)
The Institute of Transfusion Medicine – Transfusion Centre headed by Univ.-Prof. Dr. med. Daniela S. Krause is hiring:
Mainz, Rheinland-Pfalz (DE)
University of Mainz
NIH Cancer Postdoc Fellowship
Train with world-renowned cancer researchers at NIH? Consider joining the Center for Cancer Research (CCR) at the National Cancer Institute
Bethesda, Maryland (US)
NIH National Cancer Institute (NCI)
Postdoctoral Fellow (PhD)
Houston, Texas (US)
Baylor College of Medicine (BCM)
Professional-Track Faculty Positions Available!
VGTI is seeking professional-track faculty candidates with demonstrated potential for creative collaborations in infectious disease.
Vaccine & Gene Therapy Institute, Oregon Health & Science University
Postdoctoral Fellow in Cancer Biology
Postdoctoral position in cancer biology is available to carry out projects focused on studying the effects of small molecules in cancer.
Ann Arbor, Michigan
Jolanta Grembecka, PhD, Professor
Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.
- Explore articles by subject
- Guide to authors
- Editorial policies