Detail of a painting depicting the landscape of New Mexico with mountains in the distance

Explore millions of high-quality primary sources and images from around the world, including artworks, maps, photographs, and more.

Explore migration issues through a variety of media types

  • Part of The Streets are Talking: Public Forms of Creative Expression from Around the World
  • Part of The Journal of Economic Perspectives, Vol. 34, No. 1 (Winter 2020)
  • Part of Cato Institute (Aug. 3, 2021)
  • Part of University of California Press
  • Part of Open: Smithsonian National Museum of African American History & Culture
  • Part of Indiana Journal of Global Legal Studies, Vol. 19, No. 1 (Winter 2012)
  • Part of R Street Institute (Nov. 1, 2020)
  • Part of Leuven University Press
  • Part of UN Secretary-General Papers: Ban Ki-moon (2007-2016)
  • Part of Perspectives on Terrorism, Vol. 12, No. 4 (August 2018)
  • Part of Leveraging Lives: Serbia and Illegal Tunisian Migration to Europe, Carnegie Endowment for International Peace (Mar. 1, 2023)
  • Part of UCL Press

Harness the power of visual materials—explore more than 3 million images now on JSTOR.

Enhance your scholarly research with underground newspapers, magazines, and journals.

Explore collections in the arts, sciences, and literature from the world’s leading museums, archives, and scholars.

What is Database Search?

Harvard Library licenses hundreds of online databases, giving you access to academic and news articles, books, journals, primary sources, streaming media, and much more.

The contents of these databases are only partially included in HOLLIS. To make sure you're really seeing everything, you need to search in multiple places. Use Database Search to identify and connect to the best databases for your topic.

In addition to digital content, you will find specialized search engines used in specific scholarly domains.

Related Services & Tools

U.S. flag

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

  • Publications
  • Account settings

PubMed Central (PMC) Home Page

PubMed Central ® (PMC) is a free full-text archive of biomedical and life sciences journal literature at the U.S. National Institutes of Health's National Library of Medicine (NIH/NLM)

Discover a digital archive of scholarly articles, spanning centuries of scientific research.

Learn how to find and read articles of interest to you.

Collections

Browse the PMC Journal List or learn about some of PMC's unique collections.

For Authors

Navigate the PMC submission methods to comply with a funder mandate, expand access, and ensure preservation.

For Publishers

Learn about deposit options for journals and publishers and the PMC selection process.

For Developers

Find tools for bulk download, text mining, and other machine analysis.

9.7 MILLION articles are archived in PMC.

Content provided in part by:, full participation journals.

Journals deposit the complete contents of each issue or volume.

NIH Portfolio Journals

Journals deposit all NIH-funded articles as defined by the NIH Public Access Policy.

Selective Deposit Programs

Publisher deposits a subset of articles from a collection of journals.

Dec. 15, 2023

Update on pubreader format.

The PubReader format was added to PMC in 2012 to make it easier to read full text articles on tablet, mobile, and oth…

Aug. 30, 2023

Update on access to coronavirus-related articles in pubmed central (pmc) covid-19 collection after end of public health emergency.

Early in the COVID-19 pandemic, the National Library of Medicine (NLM) collaborated with publishers and scholarly soci…

Second Phase of the NIH Preprint Pilot Launched

Example of the preprint banner that displays on preprints in PMC

The second phase launched in January 2023 and expands the scope of the Pilot to include preprints resulting from all NIH-funded research.

U.S. flag

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

  • Publications
  • Account settings

Trending Articles

  • Pan-cancer proteogenomics characterization of tumor immunity. Petralia F, et al. Cell. 2024. PMID: 38359819
  • Plasma proteomic profiles predict future dementia in healthy adults. Guo Y, et al. Nat Aging. 2024. PMID: 38347190
  • Hepatic glycogenesis antagonizes lipogenesis by blocking S1P via UDPG. Chen J, et al. Science. 2024. PMID: 38359126
  • Deep whole-genome analysis of 494 hepatocellular carcinomas. Chen L, et al. Nature. 2024. PMID: 38355797
  • 30-day Morbidity and Mortality after Cholecystectomy for Benign Gallbladder Disease (AMBROSE): A Prospective, International Collaborative Cohort Study. Wong GYM, et al. Ann Surg. 2024. PMID: 38348652

Latest Literature

  • Am Heart J (2)
  • Am J Clin Nutr (1)
  • Ann Oncol (3)
  • Arch Phys Med Rehabil (3)
  • Gastroenterology (1)
  • J Am Acad Dermatol (3)
  • Lancet (11)
  • Metabolism (1)
  • Nat Commun (1)
  • Sci Rep (50)

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Reference management. Clean and simple.

Academic Databases

ERIC research database: complete tutorial

ERIC research database: complete tutorial

The ERIC database is the premier education literature database for scholarly research. This guide covers search types and strategies, filters, and full text options.

How to search online databases

How to efficiently search online databases for academic research

Academic research isn't difficult if you know where and how to search for scholarly articles and research papers. Here's how to do it.

How to use Google scholar: the ultimate guide

How to use Google Scholar: the ultimate guide

Google Scholar is the number one academic search engine. Our detailed guide covers best practices for basic and advanced search strategies in Google Scholar.

Is Google Scholar a database or search engine

Is Google Scholar a database or search engine? [Update 2024]

Google Scholar is the number one free resource to discover scientific literature, but is it an academic database or a search engine?

best research databases

The best academic research databases [Update 2024]

Your research is stuck and you need to find new sources? Take a look at our compilation of academic research databases: Scopus, Web of Science, PubMed, ERIC, JSTOR, DOAJ, Science Direct, and IEEE Xplore.

academic search engines

The best academic search engines [Update 2024]

Your research is stuck, and you need to find new sources. Take a look at our compilation of free academic search engines: ✓ Google Scholar ✓ BASE ✓ CORE ✓ Science.gov

The best research databases for computer science

The best research databases for computer science [Update 2024]

The top 4 research databases specifically dedicated to computer science:✓ ACM Digital Library ✓ IEEE Xplore ✓ dbpl ✓ Springer LNCS

Health and medicine

The best research databases for healthcare and medicine [Update 2024]

We have compiled the top list of research databases for healthcare, medicine, and biomedical research: PubMed, EMBASE, PMC, and Cochrane Library.

Article type icon

21 Legit Research Databases for Free Journal Articles in 2022

#scribendiinc

Written by  Scribendi

Has this ever happened to you? While looking for websites for research, you come across a research paper site that claims to connect academics to a peer-reviewed article database for free.

Intrigued, you search for keywords related to your topic, only to discover that you must pay a hefty subscription fee to access the service. After the umpteenth time being duped, you begin to wonder if there's even such a thing as free journal articles .

Subscription fees and paywalls are often the bane of students and academics, especially those at small institutions who don't provide access to many free article directories and repositories.

Whether you're working on an undergraduate paper, a PhD dissertation, or a medical research study, we want to help you find tools to locate and access the information you need to produce well-researched, compelling, and innovative work.

Below, we discuss why peer-reviewed articles are superior and list out the best free article databases to use in 2022.

Download Our Free Research Database Roundup PDF

Why peer-reviewed scholarly journal articles are more authoritative.

Peer-Reviewed Articles

Determining what sources are reliable can be challenging. Peer-reviewed scholarly journal articles are the gold standard in academic research. Reputable academic journals have a rigorous peer-review process.

The peer review process provides accountability to the academic community, as well as to the content of the article. The peer review process involves qualified experts in a specific (often very specific) field performing a review of an article's methods and findings to determine things like quality and credibility.

Peer-reviewed articles can be found in peer-reviewed article databases and research databases, and if you know that a database of journals is reliable, that can offer reassurances about the reliability of a free article. Peer review is often double blind, meaning that the author removes all identifying information and, likewise, does not know the identity of the reviewers. This helps reviewers maintain objectivity and impartiality so as to judge an article based on its merit.

Where to Find Peer-Reviewed Articles

Peer-reviewed articles can be found in a variety of research databases. Below is a list of some of the major databases you can use to find peer-reviewed articles and other sources in disciplines spanning the humanities, sciences, and social sciences.

What Are Open Access Journals?

An open access (OA) journal is a journal whose content can be accessed without payment. This provides scholars, students, and researchers with free journal articles . OA journals use alternate methods of funding to cover publication costs so that articles can be published without having to pass those publication costs on to the reader.

Open Access Journals

Some of these funding models include standard funding methods like advertising, public funding, and author payment models, where the author pays a fee in order to publish in the journal. There are OA journals that have non-peer-reviewed academic content, as well as journals that focus on dissertations, theses, and papers from conferences, but the main focus of OA is peer-reviewed scholarly journal articles.

The internet has certainly made it easier to access research articles and other scholarly publications without needing access to a university library, and OA takes another step in that direction by removing financial barriers to academic content.

Choosing Wisely

Features of legitimate oa journals.

 There are things to look out for when trying to decide if a free publication journal is legitimate:

Mission statement —The mission statement for an OA journal should be available on their website.

Publication history —Is the journal well established? How long has it been available?

Editorial board —Who are the members of the editorial board, and what are their credentials?

Indexing —Can the journal be found in a reliable database?

Peer review —What is the peer review process? Does the journal allow enough time in the process for a reliable assessment of quality?

Impact factor —What is the average number of times the journal is cited over a two-year period?

Features of Illegitimate OA Journals

There are predatory publications that take advantage of the OA format, and they are something to be wary of. Here are some things to look out for:

Contact information —Is contact information provided? Can it be verified?

Turnaround —If the journal makes dubious claims about the amount of time from submission to publication, it is likely unreliable.

Editorial board —Much like determining legitimacy, looking at the editorial board and their credentials can help determine illegitimacy.

Indexing —Can the journal be found in any scholarly databases?

Peer review —Is there a statement about the peer review process? Does it fit what you know about peer review?

How to Find Scholarly Articles

Identify keywords.

Keywords are included in an article by the author. Keywords are an excellent way to find content relevant to your research topic or area of interest. In academic searches, much like you would on a search engine, you can use keywords to navigate through what is available to find exactly what you're looking for.

Authors provide keywords that will help you easily find their article when researching a related topic, often including general terms to accommodate broader searches, as well as some more specific terms for those with a narrower scope. Keywords can be used individually or in combination to refine your scholarly article search.

Narrow Down Results

Sometimes, search results can be overwhelming, and searching for free articles on a journal database is no exception, but there are multiple ways to narrow down your results. A good place to start is discipline.

What category does your topic fall into (psychology, architecture, machine learning, etc.)? You can also narrow down your search with a year range if you're looking for articles that are more recent.

A Boolean search can be incredibly helpful. This entails including terms like AND between two keywords in your search if you need both keywords to be in your results (or, if you are looking to exclude certain keywords, to exclude these words from the results).

Consider Different Avenues

If you're not having luck using keywords in your search for free articles, you may still be able to find what you're looking for by changing your tactics. Casting a wider net sometimes yields positive results, so it may be helpful to try searching by subject if keywords aren't getting you anywhere.

You can search for a specific publisher to see if they have OA publications in the academic journal database. And, if you know more precisely what you're looking for, you can search for the title of the article or the author's name.

The Top 21 Free Online Journal and Research Databases

Navigating OA journals, research article databases, and academic websites trying to find high-quality sources for your research can really make your head spin. What constitutes a reliable database? What is a useful resource for your discipline and research topic? How can you find and access full-text, peer-reviewed articles?

Fortunately, we're here to help. Having covered some of the ins and outs of peer review, OA journals, and how to search for articles, we have compiled a list of the top 21 free online journals and the best research databases. This list of databases is a great resource to help you navigate the wide world of academic research.

These databases provide a variety of free sources, from abstracts and citations to full-text, peer-reviewed OA journals. With databases covering specific areas of research and interdisciplinary databases that provide a variety of material, these are some of our favorite free databases, and they're totally legit!

CORE is a multidisciplinary aggregator of OA research. CORE has the largest collection of OA articles available. It allows users to search more than 219 million OA articles. While most of these link to the full-text article on the original publisher's site, or to a PDF available for download, five million records are hosted directly on CORE.

CORE's mission statement is a simple and straightforward commitment to offering OA articles to anyone, anywhere in the world. They also host communities that are available for researchers to join and an ambassador community to enhance their services globally. In addition to a straightforward keyword search, CORE offers advanced search options to filter results by publication type, year, language, journal, repository, and author.

CORE's user interface is easy to use and navigate. Search results can be sorted based on relevance or recency, and you can search for relevant content directly from the results screen.

Collection: 219,537,133 OA articles

Other Services: Additional services are available from CORE, with extras that are geared toward researchers, repositories, and businesses. There are tools for accessing raw data, including an API that provides direct access to data, datasets that are available for download, and FastSync for syncing data content from the CORE database.

CORE has a recommender plug-in that suggests relevant OA content in the database while conducting a search and a discovery feature that helps you discover OA versions of paywalled articles. Other features include tools for managing content, such as a dashboard for managing repository output and the Repository Edition service to enhance discoverability.

Good Source of Peer-Reviewed Articles: Yes

Advanced Search Options: Language, author, journal, publisher, repository, DOI, year

2. ScienceOpen

Functioning as a research and publishing network, ScienceOpen offers OA to more than 74 million articles in all areas of science. Although you do need to register to view the full text of articles, registration is free. The advanced search function is highly detailed, allowing you to find exactly the research you're looking for.

The Berlin- and Boston-based company was founded in 2013 to "facilitate open and public communications between academics and to allow ideas to be judged on their merit, regardless of where they come from." Search results can be exported for easy integration with reference management systems.

You can also bookmark articles for later research. There are extensive networking options, including your Science Open profile, a forum for interacting with other researchers, the ability to track your usage and citations, and an interactive bibliography. Users have the ability to review articles and provide their knowledge and insight within the community.

Collection: 74,560,631

Other Services: None

Advanced Search Options:  Content type, source, author, journal, discipline

3. Directory of Open Access Journals

A multidisciplinary, community-curated directory, the Directory of Open Access Journals (DOAJ) gives researchers access to high-quality peer-reviewed journals. It has archived more than two million articles from 17,193 journals, allowing you to either browse by subject or search by keyword.

The site was launched in 2003 with the aim of increasing the visibility of OA scholarly journals online. Content on the site covers subjects from science, to law, to fine arts, and everything in between. DOAJ has a commitment to "increase the visibility, accessibility, reputation, usage and impact of quality, peer-reviewed, OA scholarly research journals globally, regardless of discipline, geography or language."

Information about the journal is available with each search result. Abstracts are also available in a collapsible format directly from the search screen. The scholarly article website is somewhat simple, but it is easy to navigate. There are 16 principles of transparency and best practices in scholarly publishing that clearly outline DOAJ policies and standards.

Collection: 6,817,242

Advanced Search Options:  Subject, journal, year

4. Education Resources Information Center

The Education Resources Information Center (ERIC) of the Institution of Education Sciences allows you to search by topic for material related to the field of education. Links lead to other sites, where you may have to purchase the information, but you can search for full-text articles only. You can also search only peer-reviewed sources.

The service primarily indexes journals, gray literature (such as technical reports, white papers, and government documents), and books. All sources of material on ERIC go through a formal review process prior to being indexed. ERIC's selection policy is available as a PDF on their website.

The ERIC website has an extensive FAQ section to address user questions. This includes categories like general questions, peer review, and ERIC content. There are also tips for advanced searches, as well as general guidance on the best way to search the database. ERIC is an excellent database for content specific to education.

Collection: 1,292,897

Advanced Search Options: Boolean

5. arXiv e-Print Archive

The arXiv e-Print Archive is run by Cornell University Library and curated by volunteer moderators, and it now offers OA to more than one million e-prints.

There are advisory committees for all eight subjects available on the database. With a stated commitment to an "emphasis on openness, collaboration, and scholarship," the arXiv e-Print Archive is an excellent STEM resource.

The interface is not as user-friendly as some of the other databases available, and the website hosts a blog to provide news and updates, but it is otherwise a straightforward math and science resource. There are simple and advanced search options, and, in addition to conducting searches for specific topics and articles, users can browse content by subject. The arXiv e-Print Archive clearly states that they do not peer review the e-prints in the database.

Collection: 1,983,891

Good Source of Peer-Reviewed Articles: No

Advanced Search Options:  Subject, date, title, author, abstract, DOI

6. Social Science Research Network

The Social Science Research Network (SSRN) is a collection of papers from the social sciences community. It is a highly interdisciplinary platform used to search for scholarly articles related to 67 social science topics. SSRN has a variety of research networks for the various topics available through the free scholarly database.

The site offers more than 700,000 abstracts and more than 600,000 full-text papers. There is not yet a specific option to search for only full-text articles, but, because most of the papers on the site are free access, it's not often that you encounter a paywall. There is currently no option to search for only peer-reviewed articles.

You must become a member to use the services, but registration is free and enables you to interact with other scholars around the world. SSRN is "passionately committed to increasing inclusion, diversity and equity in scholarly research," and they encourage and discuss the use of inclusive language in scholarship whenever possible.

Collection: 1,058,739 abstracts; 915,452 articles

Advanced Search Options: Term, author, date, network

7. Public Library of Science

Public Library of Science (PLOS) is a big player in the world of OA science. Publishing 12 OA journals, the nonprofit organization is committed to facilitating openness in academic research. According to the site, "all PLOS content is at the highest possible level of OA, meaning that scientific articles are immediately and freely available to anyone, anywhere."

PLOS outlines four fundamental goals that guide the organization: break boundaries, empower researchers, redefine quality, and open science. All PLOS journals are peer-reviewed, and all 12 journals uphold rigorous ethical standards for research, publication, and scientific reporting.

PLOS does not offer advanced search options. Content is organized by topic into research communities that users can browse through, in addition to options to search for both articles and journals. The PLOS website also has resources for peer reviewers, including guidance on becoming a reviewer and on how to best participate in the peer review process.

Collection: 12 journals

Advanced Search Options: None

8. OpenDOAR

OpenDOAR, or the Directory of Open Access Repositories, is a comprehensive resource for finding free OA journals and articles. Using Google Custom Search, OpenDOAR combs through OA repositories around the world and returns relevant research in all disciplines.

The repositories it searches through are assessed and categorized by OpenDOAR staff to ensure they meet quality standards. Inclusion criteria for the database include requirements for OA content, global access, and categorically appropriate content, in addition to various other quality assurance measures. OpenDOAR has metadata, data, content, preservation, and submission policies for repositories, in addition to two OA policy statements regarding minimum and optimum recommendations.

This database allows users to browse and search repositories, which can then be selected, and articles and data can be accessed from the repository directly. As a repository database, much of the content on the site is geared toward the support of repositories and OA standards.

Collection: 5,768 repositories

Other Services: OpenDOAR offers a variety of additional services. Given the nature of the platform, services are primarily aimed at repositories and institutions, and there is a marked focus on OA in general. Sherpa services are OA archiving tools for authors and institutions.

They also offer various resources for OA support and compliance regarding standards and policies. The publication router matches publications and publishers with appropriate repositories.

There are also services and resources from JISC for repositories for cost management, discoverability, research impact, and interoperability, including ORCID consortium membership information. Additionally, a repository self-assessment tool is available for members.

Advanced Search Options:  Name, organization name, repository type, software name, content type, subject, country, region

9. Bielefeld Academic Search Engine

The Bielefeld Academic Search Engine (BASE) is operated by the Bielefeld University Library in Germany, and it offers more than 240 million documents from more than 8,000 sources. Sixty percent of its content is OA, and you can filter your search accordingly.

BASE has rigorous inclusion requirements for content providers regarding quality and relevance, and they maintain a list of content providers for the sake of transparency, which can be easily found on their website. BASE has a fairly elegant interface. Search results can be organized by author, title, or date.

From the search results, items can be selected and exported, added to favorites, emailed, and searched in Google Scholar. There are basic and advanced search features, with the advanced search offering numerous options for refining search criteria. There is also a feature on the website that saves recent searches without additional steps from the user.

Collection: 276,019,066 documents; 9,286 content providers

Advanced Search Options:  Author, subject, year, content provider, language, document type, access, terms of reuse

Research Databases

10. Digital Library of the Commons Repository

Run by Indiana University, the Digital Library of the Commons (DLC) Repository is a multidisciplinary journal repository that allows users to access thousands of free and OA articles from around the world. You can browse by document type, date, author, title, and more or search for keywords relevant to your topic.

DCL also offers the Comprehensive Bibliography of the Commons, an image database, and a keyword thesaurus for enhanced search parameters. The repository includes books, book chapters, conference papers, journal articles, surveys, theses and dissertations, and working papers. DCL advanced search features drop-down menus of search types with built-in Boolean search options.

Searches can be sorted by relevance, title, date, or submission date in ascending or descending order. Abstracts are included in selected search results, with access to full texts available, and citations can be exported from the same page. Additionally, the image database search includes tips for better search results.

Collection: 10,784

Advanced Search Options:  Author, date, title, subject, sector, region, conference

11. CIA World Factbook

The CIA World Factbook is a little different from the other resources on this list in that it is not an online journal directory or repository. It is, however, a useful free online research database for academics in a variety of disciplines.

All the information is free to access, and it provides facts about every country in the world, which are organized by category and include information about history, geography, transportation, and much more. The World Factbook can be searched by country or region, and there is also information about the world’s oceans.

This site contains resources related to the CIA as an organization rather than being a scientific journal database specifically. The site has a user interface that is easy to navigate. The site also provides a section for updates regarding changes to what information is available and how it is organized, making it easier to interact with the information you are searching for.

Collection: 266 countries

12. Paperity

Paperity boasts its status as the "first multidisciplinary aggregator of OA journals and papers." Their focus is on helping you avoid paywalls while connecting you to authoritative research. In addition to providing readers with easy access to thousands of journals, Paperity seeks to help authors reach their audiences and help journals increase their exposure to boost readership.

Paperity has journal articles for every discipline, and the database offers more than a dozen advanced search options, including the length of the paper and the number of authors. There is even an option to include, exclude, or exclusively search gray papers.

Paperity is available for mobile, with both a mobile site and the Paperity Reader, an app that is available for both Android and Apple users. The database is also available on social media. You can interact with Paperity via Twitter and Facebook, and links to their social media are available on their homepage, including their Twitter feed.

Collection: 8,837,396

Advanced Search Options: Title, abstract, journal title, journal ISSN, publisher, year of publication, number of characters, number of authors, DOI, author, affiliation, language, country, region, continent, gray papers

13. dblp Computer Science Bibliography

The dblp Computer Science Bibliography is an online index of major computer science publications. dblp was founded in 1993, though until 2010 it was a university-specific database at the University of Trier in Germany. It is currently maintained by the Schloss Dagstuhl – Leibniz Center for Informatics.

Although it provides access to both OA articles and those behind a paywall, you can limit your search to only OA articles. The site indexes more than three million publications, making it an invaluable resource in the world of computer science. dblp entries are color-coded based on the type of item.

dblp has an extensive FAQ section, so questions that might arise about topics like the database itself, navigating the website, or the data on dblp, in addition to several other topics, are likely to be answered. The website also hosts a blog and has a section devoted to website statistics.

Collection: 5,884,702

14. EconBiz

EconBiz is a great resource for economic and business studies. A service of the Leibniz Information Centre for Economics, it offers access to full texts online, with the option of searching for OA material only. Their literature search is performed across multiple international databases.

EconBiz has an incredibly useful research skills section, with resources such as Guided Walk, a service to help students and researchers navigate searches, evaluate sources, and correctly cite references; the Research Guide EconDesk, a help desk to answer specific questions and provide advice to aid in literature searches; and the Academic Career Kit for what they refer to as Early Career Researchers.

Other helpful resources include personal literature lists, a calendar of events for relevant calls for papers, conferences, and workshops, and an economics terminology thesaurus to help in finding keywords for searches. To stay up-to-date with EconBiz, you can sign up for their newsletter.

Collection: 1,075,219

Advanced Search Options:  Title, subject, author, institution, ISBN/ISSN, journal, publisher, language, OA only

15. BioMed Central

BioMed Central provides OA research from more than 300 peer-reviewed journals. While originally focused on resources related to the physical sciences, math, and engineering, BioMed Central has branched out to include journals that cover a broader range of disciplines, with the aim of providing a single platform that provides OA articles for a variety of research needs. You can browse these journals by subject or title, or you can search all articles for your required keyword.

BioMed Central has a commitment to peer-reviewed sources and to the peer review process itself, continually seeking to help and improve the peer review process. They're "committed to maintaining high standards through full and stringent peer review." They publish the journal Research Integrity and Peer Review , which publishes research on the subject.

Additionally, the website includes resources to assist and support editors as part of their commitment to providing high-quality, peer-reviewed OA articles.

Collection: 507,212

Other Services: BMC administers the International Standard Randomised Controlled Trial Number (ISRCTN) registry. While initially designed for registering clinical trials, since its creation in 2000, the registry has broadened its scope to include other health studies as well.

The registry is recognized by the International Committee of Medical Journal Editors, as well as the World Health Organization (WHO), and it meets the requirements established by the WHO International Clinical Trials Registry Platform.

The study records included in the registry are all searchable and free to access. The ISRCTN registry "supports transparency in clinical research, helps reduce selective reporting of results and ensures an unbiased and complete evidence base."

Advanced Search Options:  Author, title, journal, list

A multidisciplinary search engine, JURN provides links to various scholarly websites, articles, and journals that are free to access or OA. Covering the fields of the arts, humanities, business, law, nature, science, and medicine, JURN has indexed almost 5,000 repositories to help you find exactly what you're looking for.

Search features are enhanced by Google, but searches are filtered through their index of repositories. JURN seeks to reach a wide audience, with their search engine tailored to researchers from "university lecturers and students seeking a strong search tool for OA content" and "advanced and ambitious students, age 14-18" to "amateur historians and biographers" and "unemployed and retired lecturers."

That being said, JURN is very upfront about its limitations. They admit to not being a good resource for educational studies, social studies, or psychology, and conference archives are generally not included due to frequently unstable URLs.

Collection: 5,064 indexed journals

Other Services: JURN has a browser add-on called UserScript. This add-on allows users to integrate the JURN database directly into Google Search. When performing a search through Google, the add-on creates a link that sends the search directly to JURN CSE. JURN CSE is a search service that is hosted by Google.

Clicking the link from the Google Search bar will run your search through the JURN database from the Google homepage. There is also an interface for a DuckDuckGo search box; while this search engine has an emphasis on user privacy, for smaller sites that may be indexed by JURN, DuckDuckGo may not provide the same depth of results.

Advanced Search Options:  Google search modifiers

Dryad is a digital repository of curated, OA scientific research data. Launched in 2009, it is run by a not-for-profit membership organization, with a community of institutional and publisher members for whom their services have been designed. Members include institutions such as Stanford, UCLA, and Yale, as well as publishers like Oxford University Press and Wiley.

Dryad aims to "promote a world where research data is openly available, integrated with the scholarly literature, and routinely reused to create knowledge." It is free to access for the search and discovery of data. Their user experience is geared toward easy self-depositing, supports Creative Commons licensing, and provides DOIs for all their content.

Note that there is a publishing charge associated if you wish to publish your data in Dryad. When searching datasets, they are accompanied by author information and abstracts for the associated studies, and citation information is provided for easy attribution.

Collection: 44,458

Advanced Search Options: No

Run by the British Library, the E-Theses Online Service (EThOS) allows you to search over 500,000 doctoral theses in a variety of disciplines. All of the doctoral theses available on EThOS have been awarded by higher education institutions in the United Kingdom.

Although some full texts are behind paywalls, you can limit your search to items available for immediate download, either directly through EThOS or through an institution's website. More than half of the records in the database provide access to full-text theses.

EThOS notes that they do not hold all records for all institutions, but they strive to index as many doctoral theses as possible, and the database is constantly expanding, with approximately 3,000 new records added and 2,000 new full-text theses available every month. The availability of full-text theses is dependent on multiple factors, including their availability in the institutional repository and the level of repository development.

Collection: 500,000+

Advanced Search Options:  Abstract, author's first name, author's last name, awarding body, current institution, EThOS ID, year, language, qualifications, research supervisor, sponsor/funder, keyword, title

PubMed is a research platform well-known in the fields of science and medicine. It was created and developed by the National Center for Biotechnology Information (NCBI) at the National Library of Medicine (NLM). It has been available since 1996 and offers access to "more than 33 million citations for biomedical literature from MEDLINE, life science journals, and online books."

While PubMed does not provide full-text articles directly, and many full-text articles may be behind paywalls or require subscriptions to access them, when articles are available from free sources, such as through PubMed Central (PMC), those links are provided with the citations and abstracts that PubMed does provide.

PMC, which was established in 2000 by the NLM, is a free full-text archive that includes more than 6,000,000 records. PubMed records link directly to corresponding PMC results. PMC content is provided by publishers and other content owners, digitization projects, and authors directly.

Collection: 33,000,000+

Advanced Search Options: Author's first name, author's last name, identifier, corporation, date completed, date created, date entered, date modified, date published, MeSH, book, conflict of interest statement, EC/RN number, editor, filter, grant number, page number, pharmacological action, volume, publication type, publisher, secondary source ID, text, title, abstract, transliterated title

20. Semantic Scholar

A unique and easy-to-use resource, Semantic Scholar defines itself not just as a research database but also as a "search and discovery tool." Semantic Scholar harnesses the power of artificial intelligence to efficiently sort through millions of science-related papers based on your search terms.

Through this singular application of machine learning, Semantic Scholar expands search results to include topic overviews based on your search terms, with the option to create an alert for or further explore the topic. It also provides links to related topics.

In addition, search results produce "TLDR" summaries in order to provide concise overviews of articles and enhance your research by helping you to navigate quickly and easily through the available literature to find the most relevant information. According to the site, although some articles are behind paywalls, "the data [they] have for those articles is limited," so you can expect to receive mostly full-text results.

Collection: 203,379,033

Other Services: Semantic Scholar supports multiple popular browsers. Content can be accessed through both mobile and desktop versions of Firefox, Microsoft Edge, Google Chrome, Apple Safari, and Opera.

Additionally, Semantic Scholar provides browser extensions for both Chrome and Firefox, so AI-powered scholarly search results are never more than a click away. The mobile interface includes an option for Semantic Swipe, a new way of interacting with your research results.

There are also beta features that can be accessed as part of the Beta Program, which will provide you with features that are being actively developed and require user feedback for further improvement.

Advanced Search Options: Field of study, date range, publication type, author, journal, conference, PDF

Zenodo, powered by the European Organization for Nuclear Research (CERN), was launched in 2013. Taking its name from Zenodotus, the first librarian of the ancient library of Alexandria, Zenodo is a tool "built and developed by researchers, to ensure that everyone can join in open science." Zenodo accepts all research from every discipline in any file format.

However, Zenodo also curates uploads and promotes peer-reviewed material that is available through OA. A DOI is assigned to everything that is uploaded to Zenodo, making research easily findable and citable. You can sort by keyword, title, journal, and more and download OA documents directly from the site.

While there are closed access and restricted access items in the database, the vast majority of research is OA material. Search results can be filtered by access type, making it easy to view the free articles available in the database.

Collection: 2,220,000+

Advanced Search Options:  Access, file type, keywords

Check out our roundup of free research databases as a handy one-page PDF.

How to find peer-reviewed articles.

There are a lot of free scholarly articles available from various sources. The internet is a big place. So how do you go about finding peer-reviewed articles when conducting your research? It's important to make sure you are using reputable sources.

The first source of the article is the person or people who wrote it. Checking out the author can give you some initial insight into how much you can trust what you’re reading. Looking into the publication information of your sources can also indicate whether the article is reliable.

Aspects of the article, such as subject and audience, tone, and format, are other things you can look at when evaluating whether the article you're using is valid, reputable, peer-reviewed material. So, let's break that down into various components so you can assess your research to ensure that you're using quality articles and conducting solid research.

Check the Author

Peer-reviewed articles are written by experts or scholars with experience in the field or discipline they're writing about. The research in a peer-reviewed article has to pass a rigorous evaluation process, so it’s a foregone conclusion that the author(s) of a peer-reviewed article should have experience or training related to that research.

When evaluating an article, take a look at the author’s information. What credentials does the author have to indicate that their research has scholarly weight behind it? Finding out what type of degree the author has—and what that degree is in—can provide insight into what kind of authority the author is on the subject.

Something else that might lend credence to the author’s scholarly role is their professional affiliation. A look at what organization or institution they are affiliated with can tell you a lot about their experience or expertise. Where were they trained, and who is verifying their research?

Identify Subject and Audience

The ultimate goal of a study is to answer a question. Scholarly articles are also written for scholarly audiences, especially articles that have gone through the peer review process. This means that the author is trying to reach experts, researchers, academics, and students in the field or topic the research is based on.

Think about the question the author is trying to answer by conducting this research, why, and for whom. What is the subject of the article? What question has it set out to answer? What is the purpose of finding the information? Is the purpose of the article of importance to other scholars? Is it original content?

Research should also be approached analytically. Is the methodology sound? Is the author using an analytical approach to evaluate the data that they have obtained? Are the conclusions they've reached substantiated by their data and analysis? Answering these questions can reveal a lot about the article’s validity.

Format Matters

Reliable articles from peer-reviewed sources have certain format elements to be aware of. The first is an abstract. An abstract is a short summary or overview of the article. Does the article have an abstract? It's unlikely that you're reading a peer-reviewed article if it doesn’t. Peer-reviewed journals will also have a word count range. If an article seems far too short or incredibly long, that may be reason to doubt it.

Another feature of reliable articles is the sections the information is divided into. Peer-reviewed research articles will have clear, concise sections that appropriately organize the information. This might include a literature review, methodology, and results in the case of research articles and a conclusion.

One of the most important sections is the references or bibliography. This is where the researcher lists all the sources of their information. A peer-reviewed source will have a comprehensive reference section.

An article that has been written to reach an academic community will have an academic tone. The language that is used, and the way this language is used, is important to consider. If the article is riddled with grammatical errors, confusing syntax, and casual language, it almost definitely didn't make it through the peer review process.

Also consider the use of terminology. Every discipline is going to have standard terminology or jargon that can be used and understood by other academics in the discipline. The language in a peer-reviewed article is going to reflect that.

If the author is going out of their way to explain simple terms, or terms that are standard to the field or discipline, it's unlikely that the article has been peer reviewed, as this is something that the author would be asked to address during the review process.

Publication

The source of the article will be a very good indicator of the likelihood that it was peer reviewed. Where was the article published? Was it published alongside other academic articles in the same discipline? Is it a legitimate and reputable scholarly publication?

A trade publication or newspaper might be legitimate or reputable, but it is not a scholarly source, and it will not have been subject to the peer review process. Scholarly journals are the best resource for peer-reviewed articles, but it's important to remember that not all scholarly journals are peer reviewed.

It’s helpful to look at a scholarly source’s website, as peer-reviewed journals will have a clear indication of the peer review process. University libraries, institutional repositories, and reliable databases (and you now might have a list of some legit ones) can also help provide insight into whether an article comes from a peer-reviewed journal.

Free Online Journal

Common Research Mistakes to Avoid

Research is a lot of work. Even with high standards and good intentions, it’s easy to make mistakes. Perhaps you searched for access to scientific journals for free and found the perfect peer-reviewed sources, but you forgot to document everything, and your references are a mess. Or, you only searched for free online articles and missed out on a ground-breaking study that was behind a paywall.

Whether your research is for a degree or to get published or to satisfy your own inquisitive nature, or all of the above, you want all that work to produce quality results. You want your research to be thorough and accurate.

To have any hope of contributing to the literature on your research topic, your results need to be high quality. You might not be able to avoid every potential mistake, but here are some that are both common and easy to avoid.

Sticking to One Source

One of the hallmarks of good research is a healthy reference section. Using a variety of sources gives you a better answer to your question. Even if all of the literature is in agreement, looking at various aspects of the topic may provide you with an entirely different picture than you would have if you looked at your research question from only one angle.

Not Documenting Every Fact

As you conduct your research, do yourself a favor and write everything down. Everything you include in your paper or article that you got from another source is going to need to be added to your references and cited.

It's important, especially if your aim is to conduct ethical, high-quality research, that all of your research has proper attribution. If you don't document as you go, you could end up making a lot of work for yourself if the information you don’t write down is something that later, as you write your paper, you really need.

Using Outdated Materials

Academia is an ever-changing landscape. What was true in your academic discipline or area of research ten years ago may have since been disproven. If fifteen studies have come out since the article that you're using was published, it's more than a little likely that you're going to be basing your research on flawed or dated information.

If the information you're basing your research on isn’t as up-to-date as possible, your research won't be of quality or able to stand up to any amount of scrutiny. You don’t want all of your hard work to be for naught.

Relying Solely on Open Access Journals

OA is a great resource for conducting academic research. There are high-quality journal articles available through OA, and that can be very helpful for your research. But, just because you have access to free articles, that doesn't mean that there's nothing to be found behind a paywall.

Just as dismissing high-quality peer-reviewed articles because they are OA would be limiting, not exploring any paid content at all is equally short-sighted. If you're seeking to conduct thorough and comprehensive research, exploring all of your options for quality sources is going to be to your benefit.

Digging Too Deep or Not Deep Enough

Research is an art form, and it involves a delicate balance of information. If you conduct your research using only broad search terms, you won't be able to answer your research question well, or you'll find that your research provides information that is closely related to your topic but, ultimately, your findings are vague and unsubstantiated.

On the other hand, if you delve deeply into your research topic with specific searches and turn up too many sources, you might have a lot of information that is adjacent to your topic but without focus and perhaps not entirely relevant. It's important to answer your research question concisely but thoroughly.

Different Types of Scholarly Articles

Different types of scholarly articles have different purposes. An original research article, also called an empirical article, is the product of a study or an experiment. This type of article seeks to answer a question or fill a gap in the existing literature.

Research articles will have a methodology, results, and a discussion of the findings of the experiment or research and typically a conclusion.

Review articles overview the current literature and research and provide a summary of what the existing research indicates or has concluded. This type of study will have a section for the literature review, as well as a discussion of the findings of that review. Review articles will have a particularly extensive reference or bibliography section.

Theoretical articles draw on existing literature to create new theories or conclusions, or look at current theories from a different perspective, to contribute to the foundational knowledge of the field of study.

10 Tips for Navigating Journal Databases

Use the right academic journal database for your search, be that interdisciplinary or specific to your field. Or both!

If it’s an option, set the search results to return only peer-reviewed sources.

Start by using search terms that are relevant to your topic without being overly specific.

Try synonyms, especially if your keywords aren’t returning the desired results.

Scholarly Journal Articles

Even if you’ve found some good articles, try searching using different terms.

Explore the advanced search features of the database(s).

Learn to use Booleans (AND, OR, NOT) to expand or narrow your results.

Once you’ve gotten some good results from a more general search, try narrowing your search.

Read through abstracts when trying to find articles relevant to your research.

Keep track of your research and use citation tools. It’ll make life easier when it comes time to compile your references.

7 Frequently Asked Questions

1. how do i get articles for free.

Free articles can be found through free online academic journals, OA databases, or other databases that include OA journals and articles. These resources allow you to access free papers online so you can conduct your research without getting stuck behind a paywall.

Academics don’t receive payment for the articles they contribute to journals. There are often, in fact, publication fees that scholars pay in order to publish. This is one of the funding structures that allows OA journals to provide free content so that you don’t have to pay fees or subscription costs to access journal articles.

2. How Do I Find Journal Articles?

Journal articles can be found in databases and institutional repositories that can be accessed at university libraries. However, online research databases that contain OA articles are the best resource for getting free access to journal articles that are available online.

Peer-reviewed journal articles are the best to use for academic research, and there are a number of databases where you can find peer-reviewed OA journal articles. Once you've found a useful article, you can look through the references for the articles the author used to conduct their research, and you can then search online databases for those articles, too.

3. How Do I Find Peer-Reviewed Articles?

Peer-reviewed articles can be found in reputable scholarly peer-reviewed journals. High-quality journals and journal articles can be found online using academic search engines and free research databases. These resources are excellent for finding OA articles, including peer-reviewed articles.

OA articles are articles that can be accessed for free. While some scholarly search engines and databases include articles that aren't peer reviewed, there are also some that provide only peer-reviewed articles, and databases that include non-peer-reviewed articles often have advanced search features that enable you to select “peer review only.” The database will return results that are exclusively peer-reviewed content.

4. What Are Research Databases?

A research database is a list of journals, articles, datasets, and/or abstracts that allows you to easily search for scholarly and academic resources and conduct research online. There are databases that are interdisciplinary and cover a variety of topics.

For example, Paperity might be a great resource for a chemist as well as a linguist, and there are databases that are more specific to a certain field. So, while ERIC might be one of the best educational databases available for OA content, it's not going to be one of the best databases for finding research in the field of microbiology.

5. How Do I Find Scholarly Articles for Specific Fields?

There are interdisciplinary research databases that provide articles in a variety of fields, as well as research databases that provide articles that cater to specific disciplines. Additionally, a journal repository or index can be a helpful resource for finding articles in a specific field.

When searching an interdisciplinary database, there are frequently advanced search features that allow you to narrow the search results down so that they are specific to your field. Selecting “psychology” in the advanced search features will return psychology journal articles in your search results. You can also try databases that are specific to your field.

If you're searching for law journal articles, many law reviews are OA. If you don’t know of any databases specific to history, visiting a journal repository or index and searching “history academic journals” can return a list of journals specific to history and provide you with a place to begin your research.

6. Are Peer-Reviewed Articles Really More Legitimate?

The short answer is yes, peer-reviewed articles are more legitimate resources for academic research. The peer review process provides legitimacy, as it is a rigorous review of the content of an article that is performed by scholars and academics who are experts in their field of study. The review provides an evaluation of the quality and credibility of the article.

Non-peer-reviewed articles are not subject to a review process and do not undergo the same level of scrutiny. This means that non-peer-reviewed articles are unlikely, or at least not as likely, to meet the same standards that peer-reviewed articles do.

7. Are Free Article Directories Legitimate?

Yes! As with anything, some databases are going to be better for certain requirements than others. But, a scholarly article database being free is not a reason in itself to question its legitimacy.

Free scholarly article databases can provide access to abstracts, scholarly article websites, journal repositories, and high-quality peer-reviewed journal articles. The internet has a lot of information, and it's often challenging to figure out what information is reliable. 

Research databases and article directories are great resources to help you conduct your research. Our list of the best research paper websites is sure to provide you with sources that are totally legit.

Get Professional Academic Editing

Hire an expert academic editor , or get a free sample, about the author.

Scribendi Editing and Proofreading

Scribendi's in-house editors work with writers from all over the globe to perfect their writing. They know that no piece of writing is complete without a professional edit, and they love to see a good piece of writing transformed into a great one. Scribendi's in-house editors are unrivaled in both experience and education, having collectively edited millions of words and obtained numerous degrees. They love consuming caffeinated beverages, reading books of various genres, and relaxing in quiet, dimly lit spaces.

Have You Read?

"The Complete Beginner's Guide to Academic Writing"

Related Posts

How to Write a Research Proposal

How to Write a Research Proposal

How to Write a Scientific Paper

How to Write a Scientific Paper

How to Write a Thesis or Dissertation

How to Write a Thesis or Dissertation

Upload your file(s) so we can calculate your word count, or enter your word count manually.

We will also recommend a service based on the file(s) you upload.

English is not my first language. I need English editing and proofreading so that I sound like a native speaker.

I need to have my journal article, dissertation, or term paper edited and proofread, or I need help with an admissions essay or proposal.

I have a novel, manuscript, play, or ebook. I need editing, copy editing, proofreading, a critique of my work, or a query package.

I need editing and proofreading for my white papers, reports, manuals, press releases, marketing materials, and other business documents.

I need to have my essay, project, assignment, or term paper edited and proofread.

I want to sound professional and to get hired. I have a resume, letter, email, or personal document that I need to have edited and proofread.

 Prices include your personal % discount.

 Prices include % sales tax ( ).

research paper of database

Internet Archive Scholar logo (vaporwave)

Librarians/Admins

  • EBSCOhost Collection Manager
  • EBSCO Experience Manager
  • EBSCO Connect
  • Start your research
  • EBSCO Mobile App

Clinical Decisions Users

  • DynaMed Decisions
  • Dynamic Health
  • Waiting Rooms
  • NoveList Blog

Free Databases

EBSCO provides free research databases covering a variety of subjects for students, researchers and librarians.

Exploring Race in Society

This free research database offers essential content covering important issues related to race in society today. Essays, articles, reports and other reliable sources provide an in-depth look at the history of race and provide critical context for learning more about topics associated with race, ethnicity, diversity and inclusiveness.

EBSCO Open Dissertations

EBSCO Open Dissertations is a collaboration between EBSCO and BiblioLabs to increase traffic and discoverability of ETD research. You can join the movement and add your theses and dissertations to the database, making them freely available to researchers everywhere.

GreenFILE is a free research database covering all aspects of human impact to the environment. Its collection of scholarly, government and general-interest titles includes content on global warming, green building, pollution, sustainable agriculture, renewable energy, recycling, and more. 

Library, Information Science and Technology Abstracts

Library, Information Science & Technology Abstracts (LISTA) is a free research database for library and information science studies. LISTA provides indexing and abstracting for hundreds of key journals, books, research reports. It is EBSCO's intention to provide access to this resource on a continual basis.

Teacher Reference Center

A complimentary research database for teachers,  Teacher Reference Center  (TRC) provides indexing and abstracts for more than 230 peer-reviewed journals.

European Views of the Americas: 1493 to 1750

European Views of the Americas: 1493 to 1750 is a free archive of indexed publications related to the Americas and written in Europe before 1750. It includes thousands of valuable primary source records covering the history of European exploration as well as portrayals of Native American peoples.

Recommended Reading

Top 10 Support Resources for K-12 Schools on EBSCO Connect

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • View all journals
  • Explore content
  • About the journal
  • Publish with us
  • Sign up for alerts
  • 12 February 2024

China conducts first nationwide review of retractions and research misconduct

  • Smriti Mallapaty

You can also search for this author in PubMed   Google Scholar

Technicians wearing full PPE work in a lab

The reputation of Chinese science has been "adversely affected" by the number of retractions in recent years, according to a government notice. Credit: Qilai Shen/Bloomberg/Getty

Chinese universities are days away from the deadline to complete a nationwide audit of retracted research papers and probe of research misconduct. By 15 February, universities must submit to the government a comprehensive list of all academic articles retracted from English- and Chinese-language journals in the past three years. They need to clarify why the papers were retracted and investigate cases involving misconduct, according to a 20 November notice from the Ministry of Education’s Department of Science, Technology and Informatization.

The government launched the nationwide self-review in response to Hindawi, a London-based subsidiary of the publisher Wiley, retracting a large number of papers by Chinese authors. These retractions, along with those from other publishers, “have adversely affected our country’s academic reputation and academic environment”, the notice states.

A Nature analysis shows that last year, Hindawi issued more than 9,600 retractions, of which the vast majority — about 8,200 — had a co-author in China. Nearly 14,000 retraction notices, of which some three-quarters involved a Chinese co-author, were issued by all publishers in 2023.

This is “the first time we’ve seen such a national operation on retraction investigations”, says Xiaotian Chen, a library and information scientist at Bradley University in Peoria, Illinois, who has studied retractions and research misconduct in China. Previous investigations have largely been carried out on a case-by-case basis — but this time, all institutions have to conduct their investigations simultaneously, says Chen.

Tight deadline

The ministry’s notice set off a chain of alerts, cascading to individual university departments. Bulletins posted on university websites required researchers to submit their retractions by a range of dates, mostly in January — leaving time for universities to collate and present the data.

Although the alerts included lists of retractions that the ministry or the universities were aware of, they also called for unlisted retractions to be added.

research paper of database

More than 10,000 research papers were retracted in 2023 — a new record

According to Nature ’s analysis, which includes only English-language journals, more than 17,000 retraction notices for papers published by Chinese co-authors have been issued since 1 January 2021, which is the start of the period of review specified in the notice. The analysis, an update of one conducted in December , used the Retraction Watch database, augmented with retraction notices collated from the Dimensions database, and involved assistance from Guillaume Cabanac, a computer scientist at the University of Toulouse in France. It is unclear whether the official lists contain the same number of retracted papers.

Regardless, the timing to submit the information will be tight, says Shu Fei, a bibliometrics scientist at Hangzhou Dianzi University in China. The ministry gave universities less than three months to complete their self-review — and this was cut shorter by the academic winter break, which typically starts in mid-January and concludes after the Chinese New Year, which fell this year on 10 February.

“The timing is not good,” he says. Shu expects that universities are most likely to submit only a preliminary report of their researchers’ retracted papers included on the official lists.

But Wang Fei, who studies research-integrity policy at Dalian University of Technology in China, says that because the ministry has set a deadline, universities will work hard to submit their findings on time.

Researchers with retracted papers will have to explain whether the retraction was owing to misconduct, such as image manipulation, or an honest mistake, such as authors identifying errors in their own work, says Chen: “In other words, they may have to defend themselves.” Universities then must investigate and penalize misconduct. If a researcher fails to declare their retracted paper and it is later uncovered, they will be punished, according to the ministry notice. The cost of not reporting is high, says Chen. “This is a very serious measure.”

It is not known what form punishment might take, but in 2021, China’s National Health Commission posted the results of its investigations into a batch of retracted papers. Punishments included salary cuts, withdrawal of bonuses, demotions and timed suspensions from applying for research grants and rewards.

The notice states explicitly that the first corresponding author of a paper is responsible for submitting the response. This requirement will largely address the problem of researchers shirking responsibility for collaborative work, says Li Tang, a science- and innovation-policy researcher at Fudan University in Shanghai, China. The notice also emphasizes due process, says Tang. Researchers alleged to have committed misconduct have a right to appeal during the investigation.

The notice is a good approach for addressing misconduct, says Wang. Previous efforts by the Chinese government have stopped at issuing new research-integrity guidelines that were poorly implemented, she says. And when government bodies did launch self-investigations of published literature, they were narrower in scope and lacked clear objectives. This time, the target is clear — retractions — and the scope is broad, involving the entire university research community, she says.

“Cultivating research integrity takes time, but China is on the right track,” says Tang.

It is not clear what the ministry will do with the flurry of submissions. Wang says that, because the retraction notices are already freely available, publicizing the collated lists and underlying reasons for retraction could be useful. She hopes that a similar review will be conducted every year “to put more pressure” on authors and universities to monitor research integrity.

What happens next will reveal how seriously the ministry regards research misconduct, says Shu. He suggests that, if the ministry does not take further action after the Chinese New Year, the notice could be an attempt to respond to the reputational damage caused by the mass retractions last year.

The ministry did not respond to Nature ’s questions about the misconduct investigation.

Chen says that, regardless of what the ministry does with the information, the reporting process itself will help to curb misconduct because it is “embarrassing to the people in the report”.

But it might primarily affect researchers publishing in English-language journals. Retraction notices in Chinese-language journals are rare.

doi: https://doi.org/10.1038/d41586-024-00397-x

Data analysis by Richard Van Noorden.

Reprints and permissions

Related Articles

research paper of database

  • Scientific community
  • Research management

Structural biology for researchers with low vision

Structural biology for researchers with low vision

Career Column 19 FEB 24

Just 5 women have won a top maths prize in the past 90 years

Just 5 women have won a top maths prize in the past 90 years

News 16 FEB 24

Largest post-pandemic survey finds trust in scientists is high

Largest post-pandemic survey finds trust in scientists is high

News 14 FEB 24

Could roving researchers help address the challenge of taking parental leave?

Could roving researchers help address the challenge of taking parental leave?

Career Feature 07 FEB 24

Best practice for LGBTQ+ data collection by STEM organizations

Correspondence 06 FEB 24

Open science — embrace it before it’s too late

Open science — embrace it before it’s too late

Editorial 06 FEB 24

Open-access publishing: citation advantage is unproven

Correspondence 13 FEB 24

How journals are fighting back against a wave of questionable images

How journals are fighting back against a wave of questionable images

News 12 FEB 24

COVID’s preprint bump set to have lasting effect on research publishing

COVID’s preprint bump set to have lasting effect on research publishing

Nature Index 09 FEB 24

Professor Helminthology

The Department of Biomedical Sciences is opening a new research unit in the field of pathogen-host-vector interactions. We are looking for a Professor

Antwerp (BE)

Institute of Tropical Medicine

research paper of database

Postdoctoral Research Associate at the RTG Chemical Biology of Ion Channels (Chembion)

An ambitious postdoctoral research associate with a fitting research idea to be included in our multidisciplinary team.

Münster, Nordrhein-Westfalen (DE)

University of Münster

research paper of database

Training Support Specialist, China

Purpose of the role:       This role is responsible for maintaining and developing client relationships for all contracted customers to Nature Mast...

Shanghai (CN)

Springer Nature Ltd

research paper of database

Artificial Intelligence and Data Science Faculty Positions in the SOE at the Westlake University

We are dedicated to achieving influential innovations in theories and applications of these research fields.

Yungu, Hangzhou, Zhejiang, China

Westlake University

research paper of database

Faculty Positions in School of Engineering, Westlake University

Tenured or tenure-track faculty positions in all ranks. We seek candidates with research interests in multiple areas.

School of Engineering, Westlake University

research paper of database

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Quick links

  • Explore articles by subject
  • Guide to authors
  • Editorial policies
  • Systematic review
  • Open access
  • Published: 19 February 2024

‘It depends’: what 86 systematic reviews tell us about what strategies to use to support the use of research in clinical practice

  • Annette Boaz   ORCID: orcid.org/0000-0003-0557-1294 1 ,
  • Juan Baeza 2 ,
  • Alec Fraser   ORCID: orcid.org/0000-0003-1121-1551 2 &
  • Erik Persson 3  

Implementation Science volume  19 , Article number:  15 ( 2024 ) Cite this article

Metrics details

The gap between research findings and clinical practice is well documented and a range of strategies have been developed to support the implementation of research into clinical practice. The objective of this study was to update and extend two previous reviews of systematic reviews of strategies designed to implement research evidence into clinical practice.

We developed a comprehensive systematic literature search strategy based on the terms used in the previous reviews to identify studies that looked explicitly at interventions designed to turn research evidence into practice. The search was performed in June 2022 in four electronic databases: Medline, Embase, Cochrane and Epistemonikos. We searched from January 2010 up to June 2022 and applied no language restrictions. Two independent reviewers appraised the quality of included studies using a quality assessment checklist. To reduce the risk of bias, papers were excluded following discussion between all members of the team. Data were synthesised using descriptive and narrative techniques to identify themes and patterns linked to intervention strategies, targeted behaviours, study settings and study outcomes.

We identified 32 reviews conducted between 2010 and 2022. The reviews are mainly of multi-faceted interventions ( n  = 20) although there are reviews focusing on single strategies (ICT, educational, reminders, local opinion leaders, audit and feedback, social media and toolkits). The majority of reviews report strategies achieving small impacts (normally on processes of care). There is much less evidence that these strategies have shifted patient outcomes. Furthermore, a lot of nuance lies behind these headline findings, and this is increasingly commented upon in the reviews themselves.

Combined with the two previous reviews, 86 systematic reviews of strategies to increase the implementation of research into clinical practice have been identified. We need to shift the emphasis away from isolating individual and multi-faceted interventions to better understanding and building more situated, relational and organisational capability to support the use of research in clinical practice. This will involve drawing on a wider range of research perspectives (including social science) in primary studies and diversifying the types of synthesis undertaken to include approaches such as realist synthesis which facilitate exploration of the context in which strategies are employed.

Peer Review reports

Contribution to the literature

Considerable time and money is invested in implementing and evaluating strategies to increase the implementation of research into clinical practice.

The growing body of evidence is not providing the anticipated clear lessons to support improved implementation.

Instead what is needed is better understanding and building more situated, relational and organisational capability to support the use of research in clinical practice.

This would involve a more central role in implementation science for a wider range of perspectives, especially from the social, economic, political and behavioural sciences and for greater use of different types of synthesis, such as realist synthesis.

Introduction

The gap between research findings and clinical practice is well documented and a range of interventions has been developed to increase the implementation of research into clinical practice [ 1 , 2 ]. In recent years researchers have worked to improve the consistency in the ways in which these interventions (often called strategies) are described to support their evaluation. One notable development has been the emergence of Implementation Science as a field focusing explicitly on “the scientific study of methods to promote the systematic uptake of research findings and other evidence-based practices into routine practice” ([ 3 ] p. 1). The work of implementation science focuses on closing, or at least narrowing, the gap between research and practice. One contribution has been to map existing interventions, identifying 73 discreet strategies to support research implementation [ 4 ] which have been grouped into 9 clusters [ 5 ]. The authors note that they have not considered the evidence of effectiveness of the individual strategies and that a next step is to understand better which strategies perform best in which combinations and for what purposes [ 4 ]. Other authors have noted that there is also scope to learn more from other related fields of study such as policy implementation [ 6 ] and to draw on methods designed to support the evaluation of complex interventions [ 7 ].

The increase in activity designed to support the implementation of research into practice and improvements in reporting provided the impetus for an update of a review of systematic reviews of the effectiveness of interventions designed to support the use of research in clinical practice [ 8 ] which was itself an update of the review conducted by Grimshaw and colleagues in 2001. The 2001 review [ 9 ] identified 41 reviews considering a range of strategies including educational interventions, audit and feedback, computerised decision support to financial incentives and combined interventions. The authors concluded that all the interventions had the potential to promote the uptake of evidence in practice, although no one intervention seemed to be more effective than the others in all settings. They concluded that combined interventions were more likely to be effective than single interventions. The 2011 review identified a further 13 systematic reviews containing 313 discrete primary studies. Consistent with the previous review, four main strategy types were identified: audit and feedback; computerised decision support; opinion leaders; and multi-faceted interventions (MFIs). Nine of the reviews reported on MFIs. The review highlighted the small effects of single interventions such as audit and feedback, computerised decision support and opinion leaders. MFIs claimed an improvement in effectiveness over single interventions, although effect sizes remained small to moderate and this improvement in effectiveness relating to MFIs has been questioned in a subsequent review [ 10 ]. In updating the review, we anticipated a larger pool of reviews and an opportunity to consolidate learning from more recent systematic reviews of interventions.

This review updates and extends our previous review of systematic reviews of interventions designed to implement research evidence into clinical practice. To identify potentially relevant peer-reviewed research papers, we developed a comprehensive systematic literature search strategy based on the terms used in the Grimshaw et al. [ 9 ] and Boaz, Baeza and Fraser [ 8 ] overview articles. To ensure optimal retrieval, our search strategy was refined with support from an expert university librarian, considering the ongoing improvements in the development of search filters for systematic reviews since our first review [ 11 ]. We also wanted to include technology-related terms (e.g. apps, algorithms, machine learning, artificial intelligence) to find studies that explored interventions based on the use of technological innovations as mechanistic tools for increasing the use of evidence into practice (see Additional file 1 : Appendix A for full search strategy).

The search was performed in June 2022 in the following electronic databases: Medline, Embase, Cochrane and Epistemonikos. We searched for articles published since the 2011 review. We searched from January 2010 up to June 2022 and applied no language restrictions. Reference lists of relevant papers were also examined.

We uploaded the results using EPPI-Reviewer, a web-based tool that facilitated semi-automation of the screening process and removal of duplicate studies. We made particular use of a priority screening function to reduce screening workload and avoid ‘data deluge’ [ 12 ]. Through machine learning, one reviewer screened a smaller number of records ( n  = 1200) to train the software to predict whether a given record was more likely to be relevant or irrelevant, thus pulling the relevant studies towards the beginning of the screening process. This automation did not replace manual work but helped the reviewer to identify eligible studies more quickly. During the selection process, we included studies that looked explicitly at interventions designed to turn research evidence into practice. Studies were included if they met the following pre-determined inclusion criteria:

The study was a systematic review

Search terms were included

Focused on the implementation of research evidence into practice

The methodological quality of the included studies was assessed as part of the review

Study populations included healthcare providers and patients. The EPOC taxonomy [ 13 ] was used to categorise the strategies. The EPOC taxonomy has four domains: delivery arrangements, financial arrangements, governance arrangements and implementation strategies. The implementation strategies domain includes 20 strategies targeted at healthcare workers. Numerous EPOC strategies were assessed in the review including educational strategies, local opinion leaders, reminders, ICT-focused approaches and audit and feedback. Some strategies that did not fit easily within the EPOC categories were also included. These were social media strategies and toolkits, and multi-faceted interventions (MFIs) (see Table  2 ). Some systematic reviews included comparisons of different interventions while other reviews compared one type of intervention against a control group. Outcomes related to improvements in health care processes or patient well-being. Numerous individual study types (RCT, CCT, BA, ITS) were included within the systematic reviews.

We excluded papers that:

Focused on changing patient rather than provider behaviour

Had no demonstrable outcomes

Made unclear or no reference to research evidence

The last of these criteria was sometimes difficult to judge, and there was considerable discussion amongst the research team as to whether the link between research evidence and practice was sufficiently explicit in the interventions analysed. As we discussed in the previous review [ 8 ] in the field of healthcare, the principle of evidence-based practice is widely acknowledged and tools to change behaviour such as guidelines are often seen to be an implicit codification of evidence, despite the fact that this is not always the case.

Reviewers employed a two-stage process to select papers for inclusion. First, all titles and abstracts were screened by one reviewer to determine whether the study met the inclusion criteria. Two papers [ 14 , 15 ] were identified that fell just before the 2010 cut-off. As they were not identified in the searches for the first review [ 8 ] they were included and progressed to assessment. Each paper was rated as include, exclude or maybe. The full texts of 111 relevant papers were assessed independently by at least two authors. To reduce the risk of bias, papers were excluded following discussion between all members of the team. 32 papers met the inclusion criteria and proceeded to data extraction. The study selection procedure is documented in a PRISMA literature flow diagram (see Fig.  1 ). We were able to include French, Spanish and Portuguese papers in the selection reflecting the language skills in the study team, but none of the papers identified met the inclusion criteria. Other non- English language papers were excluded.

figure 1

PRISMA flow diagram. Source: authors

One reviewer extracted data on strategy type, number of included studies, local, target population, effectiveness and scope of impact from the included studies. Two reviewers then independently read each paper and noted key findings and broad themes of interest which were then discussed amongst the wider authorial team. Two independent reviewers appraised the quality of included studies using a Quality Assessment Checklist based on Oxman and Guyatt [ 16 ] and Francke et al. [ 17 ]. Each study was rated a quality score ranging from 1 (extensive flaws) to 7 (minimal flaws) (see Additional file 2 : Appendix B). All disagreements were resolved through discussion. Studies were not excluded in this updated overview based on methodological quality as we aimed to reflect the full extent of current research into this topic.

The extracted data were synthesised using descriptive and narrative techniques to identify themes and patterns in the data linked to intervention strategies, targeted behaviours, study settings and study outcomes.

Thirty-two studies were included in the systematic review. Table 1. provides a detailed overview of the included systematic reviews comprising reference, strategy type, quality score, number of included studies, local, target population, effectiveness and scope of impact (see Table  1. at the end of the manuscript). Overall, the quality of the studies was high. Twenty-three studies scored 7, six studies scored 6, one study scored 5, one study scored 4 and one study scored 3. The primary focus of the review was on reviews of effectiveness studies, but a small number of reviews did include data from a wider range of methods including qualitative studies which added to the analysis in the papers [ 18 , 19 , 20 , 21 ]. The majority of reviews report strategies achieving small impacts (normally on processes of care). There is much less evidence that these strategies have shifted patient outcomes. In this section, we discuss the different EPOC-defined implementation strategies in turn. Interestingly, we found only two ‘new’ approaches in this review that did not fit into the existing EPOC approaches. These are a review focused on the use of social media and a review considering toolkits. In addition to single interventions, we also discuss multi-faceted interventions. These were the most common intervention approach overall. A summary is provided in Table  2 .

Educational strategies

The overview identified three systematic reviews focusing on educational strategies. Grudniewicz et al. [ 22 ] explored the effectiveness of printed educational materials on primary care physician knowledge, behaviour and patient outcomes and concluded they were not effective in any of these aspects. Koota, Kääriäinen and Melender [ 23 ] focused on educational interventions promoting evidence-based practice among emergency room/accident and emergency nurses and found that interventions involving face-to-face contact led to significant or highly significant effects on patient benefits and emergency nurses’ knowledge, skills and behaviour. Interventions using written self-directed learning materials also led to significant improvements in nurses’ knowledge of evidence-based practice. Although the quality of the studies was high, the review primarily included small studies with low response rates, and many of them relied on self-assessed outcomes; consequently, the strength of the evidence for these outcomes is modest. Wu et al. [ 20 ] questioned if educational interventions aimed at nurses to support the implementation of evidence-based practice improve patient outcomes. Although based on evaluation projects and qualitative data, their results also suggest that positive changes on patient outcomes can be made following the implementation of specific evidence-based approaches (or projects). The differing positive outcomes for educational strategies aimed at nurses might indicate that the target audience is important.

Local opinion leaders

Flodgren et al. [ 24 ] was the only systemic review focusing solely on opinion leaders. The review found that local opinion leaders alone, or in combination with other interventions, can be effective in promoting evidence‐based practice, but this varies both within and between studies and the effect on patient outcomes is uncertain. The review found that, overall, any intervention involving opinion leaders probably improves healthcare professionals’ compliance with evidence-based practice but varies within and across studies. However, how opinion leaders had an impact could not be determined because of insufficient details were provided, illustrating that reporting specific details in published studies is important if diffusion of effective methods of increasing evidence-based practice is to be spread across a system. The usefulness of this review is questionable because it cannot provide evidence of what is an effective opinion leader, whether teams of opinion leaders or a single opinion leader are most effective, or the most effective methods used by opinion leaders.

Pantoja et al. [ 26 ] was the only systemic review focusing solely on manually generated reminders delivered on paper included in the overview. The review explored how these affected professional practice and patient outcomes. The review concluded that manually generated reminders delivered on paper as a single intervention probably led to small to moderate increases in adherence to clinical recommendations, and they could be used as a single quality improvement intervention. However, the authors indicated that this intervention would make little or no difference to patient outcomes. The authors state that such a low-tech intervention may be useful in low- and middle-income countries where paper records are more likely to be the norm.

ICT-focused approaches

The three ICT-focused reviews [ 14 , 27 , 28 ] showed mixed results. Jamal, McKenzie and Clark [ 14 ] explored the impact of health information technology on the quality of medical and health care. They examined the impact of electronic health record, computerised provider order-entry, or decision support system. This showed a positive improvement in adherence to evidence-based guidelines but not to patient outcomes. The number of studies included in the review was low and so a conclusive recommendation could not be reached based on this review. Similarly, Brown et al. [ 28 ] found that technology-enabled knowledge translation interventions may improve knowledge of health professionals, but all eight studies raised concerns of bias. The De Angelis et al. [ 27 ] review was more promising, reporting that ICT can be a good way of disseminating clinical practice guidelines but conclude that it is unclear which type of ICT method is the most effective.

Audit and feedback

Sykes, McAnuff and Kolehmainen [ 29 ] examined whether audit and feedback were effective in dementia care and concluded that it remains unclear which ingredients of audit and feedback are successful as the reviewed papers illustrated large variations in the effectiveness of interventions using audit and feedback.

Non-EPOC listed strategies: social media, toolkits

There were two new (non-EPOC listed) intervention types identified in this review compared to the 2011 review — fewer than anticipated. We categorised a third — ‘care bundles’ [ 36 ] as a multi-faceted intervention due to its description in practice and a fourth — ‘Technology Enhanced Knowledge Transfer’ [ 28 ] was classified as an ICT-focused approach. The first new strategy was identified in Bhatt et al.’s [ 30 ] systematic review of the use of social media for the dissemination of clinical practice guidelines. They reported that the use of social media resulted in a significant improvement in knowledge and compliance with evidence-based guidelines compared with more traditional methods. They noted that a wide selection of different healthcare professionals and patients engaged with this type of social media and its global reach may be significant for low- and middle-income countries. This review was also noteworthy for developing a simple stepwise method for using social media for the dissemination of clinical practice guidelines. However, it is debatable whether social media can be classified as an intervention or just a different way of delivering an intervention. For example, the review discussed involving opinion leaders and patient advocates through social media. However, this was a small review that included only five studies, so further research in this new area is needed. Yamada et al. [ 31 ] draw on 39 studies to explore the application of toolkits, 18 of which had toolkits embedded within larger KT interventions, and 21 of which evaluated toolkits as standalone interventions. The individual component strategies of the toolkits were highly variable though the authors suggest that they align most closely with educational strategies. The authors conclude that toolkits as either standalone strategies or as part of MFIs hold some promise for facilitating evidence use in practice but caution that the quality of many of the primary studies included is considered weak limiting these findings.

Multi-faceted interventions

The majority of the systematic reviews ( n  = 20) reported on more than one intervention type. Some of these systematic reviews focus exclusively on multi-faceted interventions, whilst others compare different single or combined interventions aimed at achieving similar outcomes in particular settings. While these two approaches are often described in a similar way, they are actually quite distinct from each other as the former report how multiple strategies may be strategically combined in pursuance of an agreed goal, whilst the latter report how different strategies may be incidentally used in sometimes contrasting settings in the pursuance of similar goals. Ariyo et al. [ 35 ] helpfully summarise five key elements often found in effective MFI strategies in LMICs — but which may also be transferrable to HICs. First, effective MFIs encourage a multi-disciplinary approach acknowledging the roles played by different professional groups to collectively incorporate evidence-informed practice. Second, they utilise leadership drawing on a wide set of clinical and non-clinical actors including managers and even government officials. Third, multiple types of educational practices are utilised — including input from patients as stakeholders in some cases. Fourth, protocols, checklists and bundles are used — most effectively when local ownership is encouraged. Finally, most MFIs included an emphasis on monitoring and evaluation [ 35 ]. In contrast, other studies offer little information about the nature of the different MFI components of included studies which makes it difficult to extrapolate much learning from them in relation to why or how MFIs might affect practice (e.g. [ 28 , 38 ]). Ultimately, context matters, which some review authors argue makes it difficult to say with real certainty whether single or MFI strategies are superior (e.g. [ 21 , 27 ]). Taking all the systematic reviews together we may conclude that MFIs appear to be more likely to generate positive results than single interventions (e.g. [ 34 , 45 ]) though other reviews should make us cautious (e.g. [ 32 , 43 ]).

While multi-faceted interventions still seem to be more effective than single-strategy interventions, there were important distinctions between how the results of reviews of MFIs are interpreted in this review as compared to the previous reviews [ 8 , 9 ], reflecting greater nuance and debate in the literature. This was particularly noticeable where the effectiveness of MFIs was compared to single strategies, reflecting developments widely discussed in previous studies [ 10 ]. We found that most systematic reviews are bounded by their clinical, professional, spatial, system, or setting criteria and often seek to draw out implications for the implementation of evidence in their areas of specific interest (such as nursing or acute care). Frequently this means combining all relevant studies to explore the respective foci of each systematic review. Therefore, most reviews we categorised as MFIs actually include highly variable numbers and combinations of intervention strategies and highly heterogeneous original study designs. This makes statistical analyses of the type used by Squires et al. [ 10 ] on the three reviews in their paper not possible. Further, it also makes extrapolating findings and commenting on broad themes complex and difficult. This may suggest that future research should shift its focus from merely examining ‘what works’ to ‘what works where and what works for whom’ — perhaps pointing to the value of realist approaches to these complex review topics [ 48 , 49 ] and other more theory-informed approaches [ 50 ].

Some reviews have a relatively small number of studies (i.e. fewer than 10) and the authors are often understandably reluctant to engage with wider debates about the implications of their findings. Other larger studies do engage in deeper discussions about internal comparisons of findings across included studies and also contextualise these in wider debates. Some of the most informative studies (e.g. [ 35 , 40 ]) move beyond EPOC categories and contextualise MFIs within wider systems thinking and implementation theory. This distinction between MFIs and single interventions can actually be very useful as it offers lessons about the contexts in which individual interventions might have bounded effectiveness (i.e. educational interventions for individual change). Taken as a whole, this may also then help in terms of how and when to conjoin single interventions into effective MFIs.

In the two previous reviews, a consistent finding was that MFIs were more effective than single interventions [ 8 , 9 ]. However, like Squires et al. [ 10 ] this overview is more equivocal on this important issue. There are four points which may help account for the differences in findings in this regard. Firstly, the diversity of the systematic reviews in terms of clinical topic or setting is an important factor. Secondly, there is heterogeneity of the studies within the included systematic reviews themselves. Thirdly, there is a lack of consistency with regards to the definition and strategies included within of MFIs. Finally, there are epistemological differences across the papers and the reviews. This means that the results that are presented depend on the methods used to measure, report, and synthesise them. For instance, some reviews highlight that education strategies can be useful to improve provider understanding — but without wider organisational or system-level change, they may struggle to deliver sustained transformation [ 19 , 44 ].

It is also worth highlighting the importance of the theory of change underlying the different interventions. Where authors of the systematic reviews draw on theory, there is space to discuss/explain findings. We note a distinction between theoretical and atheoretical systematic review discussion sections. Atheoretical reviews tend to present acontextual findings (for instance, one study found very positive results for one intervention, and this gets highlighted in the abstract) whilst theoretically informed reviews attempt to contextualise and explain patterns within the included studies. Theory-informed systematic reviews seem more likely to offer more profound and useful insights (see [ 19 , 35 , 40 , 43 , 45 ]). We find that the most insightful systematic reviews of MFIs engage in theoretical generalisation — they attempt to go beyond the data of individual studies and discuss the wider implications of the findings of the studies within their reviews drawing on implementation theory. At the same time, they highlight the active role of context and the wider relational and system-wide issues linked to implementation. It is these types of investigations that can help providers further develop evidence-based practice.

This overview has identified a small, but insightful set of papers that interrogate and help theorise why, how, for whom, and in which circumstances it might be the case that MFIs are superior (see [ 19 , 35 , 40 ] once more). At the level of this overview — and in most of the systematic reviews included — it appears to be the case that MFIs struggle with the question of attribution. In addition, there are other important elements that are often unmeasured, or unreported (e.g. costs of the intervention — see [ 40 ]). Finally, the stronger systematic reviews [ 19 , 35 , 40 , 43 , 45 ] engage with systems issues, human agency and context [ 18 ] in a way that was not evident in the systematic reviews identified in the previous reviews [ 8 , 9 ]. The earlier reviews lacked any theory of change that might explain why MFIs might be more effective than single ones — whereas now some systematic reviews do this, which enables them to conclude that sometimes single interventions can still be more effective.

As Nilsen et al. ([ 6 ] p. 7) note ‘Study findings concerning the effectiveness of various approaches are continuously synthesized and assembled in systematic reviews’. We may have gone as far as we can in understanding the implementation of evidence through systematic reviews of single and multi-faceted interventions and the next step would be to conduct more research exploring the complex and situated nature of evidence used in clinical practice and by particular professional groups. This would further build on the nuanced discussion and conclusion sections in a subset of the papers we reviewed. This might also support the field to move away from isolating individual implementation strategies [ 6 ] to explore the complex processes involving a range of actors with differing capacities [ 51 ] working in diverse organisational cultures. Taxonomies of implementation strategies do not fully account for the complex process of implementation, which involves a range of different actors with different capacities and skills across multiple system levels. There is plenty of work to build on, particularly in the social sciences, which currently sits at the margins of debates about evidence implementation (see for example, Normalisation Process Theory [ 52 ]).

There are several changes that we have identified in this overview of systematic reviews in comparison to the review we published in 2011 [ 8 ]. A consistent and welcome finding is that the overall quality of the systematic reviews themselves appears to have improved between the two reviews, although this is not reflected upon in the papers. This is exhibited through better, clearer reporting mechanisms in relation to the mechanics of the reviews, alongside a greater attention to, and deeper description of, how potential biases in included papers are discussed. Additionally, there is an increased, but still limited, inclusion of original studies conducted in low- and middle-income countries as opposed to just high-income countries. Importantly, we found that many of these systematic reviews are attuned to, and comment upon the contextual distinctions of pursuing evidence-informed interventions in health care settings in different economic settings. Furthermore, systematic reviews included in this updated article cover a wider set of clinical specialities (both within and beyond hospital settings) and have a focus on a wider set of healthcare professions — discussing both similarities, differences and inter-professional challenges faced therein, compared to the earlier reviews. These wider ranges of studies highlight that a particular intervention or group of interventions may work well for one professional group but be ineffective for another. This diversity of study settings allows us to consider the important role context (in its many forms) plays on implementing evidence into practice. Examining the complex and varied context of health care will help us address what Nilsen et al. ([ 6 ] p. 1) described as, ‘society’s health problems [that] require research-based knowledge acted on by healthcare practitioners together with implementation of political measures from governmental agencies’. This will help us shift implementation science to move, ‘beyond a success or failure perspective towards improved analysis of variables that could explain the impact of the implementation process’ ([ 6 ] p. 2).

This review brings together 32 papers considering individual and multi-faceted interventions designed to support the use of evidence in clinical practice. The majority of reviews report strategies achieving small impacts (normally on processes of care). There is much less evidence that these strategies have shifted patient outcomes. Combined with the two previous reviews, 86 systematic reviews of strategies to increase the implementation of research into clinical practice have been conducted. As a whole, this substantial body of knowledge struggles to tell us more about the use of individual and MFIs than: ‘it depends’. To really move forwards in addressing the gap between research evidence and practice, we may need to shift the emphasis away from isolating individual and multi-faceted interventions to better understanding and building more situated, relational and organisational capability to support the use of research in clinical practice. This will involve drawing on a wider range of perspectives, especially from the social, economic, political and behavioural sciences in primary studies and diversifying the types of synthesis undertaken to include approaches such as realist synthesis which facilitate exploration of the context in which strategies are employed. Harvey et al. [ 53 ] suggest that when context is likely to be critical to implementation success there are a range of primary research approaches (participatory research, realist evaluation, developmental evaluation, ethnography, quality/ rapid cycle improvement) that are likely to be appropriate and insightful. While these approaches often form part of implementation studies in the form of process evaluations, they are usually relatively small scale in relation to implementation research as a whole. As a result, the findings often do not make it into the subsequent systematic reviews. This review provides further evidence that we need to bring qualitative approaches in from the periphery to play a central role in many implementation studies and subsequent evidence syntheses. It would be helpful for systematic reviews, at the very least, to include more detail about the interventions and their implementation in terms of how and why they worked.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Before and after study

Controlled clinical trial

Effective Practice and Organisation of Care

High-income countries

Information and Communications Technology

Interrupted time series

Knowledge translation

Low- and middle-income countries

Randomised controlled trial

Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients’ care. Lancet. 2003;362:1225–30. https://doi.org/10.1016/S0140-6736(03)14546-1 .

Article   PubMed   Google Scholar  

Green LA, Seifert CM. Translation of research into practice: why we can’t “just do it.” J Am Board Fam Pract. 2005;18:541–5. https://doi.org/10.3122/jabfm.18.6.541 .

Eccles MP, Mittman BS. Welcome to Implementation Science. Implement Sci. 2006;1:1–3. https://doi.org/10.1186/1748-5908-1-1 .

Article   PubMed Central   Google Scholar  

Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:2–14. https://doi.org/10.1186/s13012-015-0209-1 .

Article   Google Scholar  

Waltz TJ, Powell BJ, Matthieu MM, Damschroder LJ, et al. Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: results from the Expert Recommendations for Implementing Change (ERIC) study. Implement Sci. 2015;10:1–8. https://doi.org/10.1186/s13012-015-0295-0 .

Nilsen P, Ståhl C, Roback K, et al. Never the twain shall meet? - a comparison of implementation science and policy implementation research. Implementation Sci. 2013;8:2–12. https://doi.org/10.1186/1748-5908-8-63 .

Rycroft-Malone J, Seers K, Eldh AC, et al. A realist process evaluation within the Facilitating Implementation of Research Evidence (FIRE) cluster randomised controlled international trial: an exemplar. Implementation Sci. 2018;13:1–15. https://doi.org/10.1186/s13012-018-0811-0 .

Boaz A, Baeza J, Fraser A, European Implementation Score Collaborative Group (EIS). Effective implementation of research into practice: an overview of systematic reviews of the health literature. BMC Res Notes. 2011;4:212. https://doi.org/10.1186/1756-0500-4-212 .

Article   PubMed   PubMed Central   Google Scholar  

Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L, et al. Changing provider behavior – an overview of systematic reviews of interventions. Med Care. 2001;39 8Suppl 2:II2–45.

Google Scholar  

Squires JE, Sullivan K, Eccles MP, et al. Are multifaceted interventions more effective than single-component interventions in changing health-care professionals’ behaviours? An overview of systematic reviews. Implement Sci. 2014;9:1–22. https://doi.org/10.1186/s13012-014-0152-6 .

Salvador-Oliván JA, Marco-Cuenca G, Arquero-Avilés R. Development of an efficient search filter to retrieve systematic reviews from PubMed. J Med Libr Assoc. 2021;109:561–74. https://doi.org/10.5195/jmla.2021.1223 .

Thomas JM. Diffusion of innovation in systematic review methodology: why is study selection not yet assisted by automation? OA Evid Based Med. 2013;1:1–6.

Effective Practice and Organisation of Care (EPOC). The EPOC taxonomy of health systems interventions. EPOC Resources for review authors. Oslo: Norwegian Knowledge Centre for the Health Services; 2016. epoc.cochrane.org/epoc-taxonomy . Accessed 9 Oct 2023.

Jamal A, McKenzie K, Clark M. The impact of health information technology on the quality of medical and health care: a systematic review. Health Inf Manag. 2009;38:26–37. https://doi.org/10.1177/183335830903800305 .

Menon A, Korner-Bitensky N, Kastner M, et al. Strategies for rehabilitation professionals to move evidence-based knowledge into practice: a systematic review. J Rehabil Med. 2009;41:1024–32. https://doi.org/10.2340/16501977-0451 .

Oxman AD, Guyatt GH. Validation of an index of the quality of review articles. J Clin Epidemiol. 1991;44:1271–8. https://doi.org/10.1016/0895-4356(91)90160-b .

Article   CAS   PubMed   Google Scholar  

Francke AL, Smit MC, de Veer AJ, et al. Factors influencing the implementation of clinical guidelines for health care professionals: a systematic meta-review. BMC Med Inform Decis Mak. 2008;8:1–11. https://doi.org/10.1186/1472-6947-8-38 .

Jones CA, Roop SC, Pohar SL, et al. Translating knowledge in rehabilitation: systematic review. Phys Ther. 2015;95:663–77. https://doi.org/10.2522/ptj.20130512 .

Scott D, Albrecht L, O’Leary K, Ball GDC, et al. Systematic review of knowledge translation strategies in the allied health professions. Implement Sci. 2012;7:1–17. https://doi.org/10.1186/1748-5908-7-70 .

Wu Y, Brettle A, Zhou C, Ou J, et al. Do educational interventions aimed at nurses to support the implementation of evidence-based practice improve patient outcomes? A systematic review. Nurse Educ Today. 2018;70:109–14. https://doi.org/10.1016/j.nedt.2018.08.026 .

Yost J, Ganann R, Thompson D, Aloweni F, et al. The effectiveness of knowledge translation interventions for promoting evidence-informed decision-making among nurses in tertiary care: a systematic review and meta-analysis. Implement Sci. 2015;10:1–15. https://doi.org/10.1186/s13012-015-0286-1 .

Grudniewicz A, Kealy R, Rodseth RN, Hamid J, et al. What is the effectiveness of printed educational materials on primary care physician knowledge, behaviour, and patient outcomes: a systematic review and meta-analyses. Implement Sci. 2015;10:2–12. https://doi.org/10.1186/s13012-015-0347-5 .

Koota E, Kääriäinen M, Melender HL. Educational interventions promoting evidence-based practice among emergency nurses: a systematic review. Int Emerg Nurs. 2018;41:51–8. https://doi.org/10.1016/j.ienj.2018.06.004 .

Flodgren G, O’Brien MA, Parmelli E, et al. Local opinion leaders: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2019. https://doi.org/10.1002/14651858.CD000125.pub5 .

Arditi C, Rège-Walther M, Durieux P, et al. Computer-generated reminders delivered on paper to healthcare professionals: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2017. https://doi.org/10.1002/14651858.CD001175.pub4 .

Pantoja T, Grimshaw JM, Colomer N, et al. Manually-generated reminders delivered on paper: effects on professional practice and patient outcomes. Cochrane Database Syst Rev. 2019. https://doi.org/10.1002/14651858.CD001174.pub4 .

De Angelis G, Davies B, King J, McEwan J, et al. Information and communication technologies for the dissemination of clinical practice guidelines to health professionals: a systematic review. JMIR Med Educ. 2016;2:e16. https://doi.org/10.2196/mededu.6288 .

Brown A, Barnes C, Byaruhanga J, McLaughlin M, et al. Effectiveness of technology-enabled knowledge translation strategies in improving the use of research in public health: systematic review. J Med Internet Res. 2020;22:e17274. https://doi.org/10.2196/17274 .

Sykes MJ, McAnuff J, Kolehmainen N. When is audit and feedback effective in dementia care? A systematic review. Int J Nurs Stud. 2018;79:27–35. https://doi.org/10.1016/j.ijnurstu.2017.10.013 .

Bhatt NR, Czarniecki SW, Borgmann H, et al. A systematic review of the use of social media for dissemination of clinical practice guidelines. Eur Urol Focus. 2021;7:1195–204. https://doi.org/10.1016/j.euf.2020.10.008 .

Yamada J, Shorkey A, Barwick M, Widger K, et al. The effectiveness of toolkits as knowledge translation strategies for integrating evidence into clinical care: a systematic review. BMJ Open. 2015;5:e006808. https://doi.org/10.1136/bmjopen-2014-006808 .

Afari-Asiedu S, Abdulai MA, Tostmann A, et al. Interventions to improve dispensing of antibiotics at the community level in low and middle income countries: a systematic review. J Glob Antimicrob Resist. 2022;29:259–74. https://doi.org/10.1016/j.jgar.2022.03.009 .

Boonacker CW, Hoes AW, Dikhoff MJ, Schilder AG, et al. Interventions in health care professionals to improve treatment in children with upper respiratory tract infections. Int J Pediatr Otorhinolaryngol. 2010;74:1113–21. https://doi.org/10.1016/j.ijporl.2010.07.008 .

Al Zoubi FM, Menon A, Mayo NE, et al. The effectiveness of interventions designed to increase the uptake of clinical practice guidelines and best practices among musculoskeletal professionals: a systematic review. BMC Health Serv Res. 2018;18:2–11. https://doi.org/10.1186/s12913-018-3253-0 .

Ariyo P, Zayed B, Riese V, Anton B, et al. Implementation strategies to reduce surgical site infections: a systematic review. Infect Control Hosp Epidemiol. 2019;3:287–300. https://doi.org/10.1017/ice.2018.355 .

Borgert MJ, Goossens A, Dongelmans DA. What are effective strategies for the implementation of care bundles on ICUs: a systematic review. Implement Sci. 2015;10:1–11. https://doi.org/10.1186/s13012-015-0306-1 .

Cahill LS, Carey LM, Lannin NA, et al. Implementation interventions to promote the uptake of evidence-based practices in stroke rehabilitation. Cochrane Database Syst Rev. 2020. https://doi.org/10.1002/14651858.CD012575.pub2 .

Pedersen ER, Rubenstein L, Kandrack R, Danz M, et al. Elusive search for effective provider interventions: a systematic review of provider interventions to increase adherence to evidence-based treatment for depression. Implement Sci. 2018;13:1–30. https://doi.org/10.1186/s13012-018-0788-8 .

Jenkins HJ, Hancock MJ, French SD, Maher CG, et al. Effectiveness of interventions designed to reduce the use of imaging for low-back pain: a systematic review. CMAJ. 2015;187:401–8. https://doi.org/10.1503/cmaj.141183 .

Bennett S, Laver K, MacAndrew M, Beattie E, et al. Implementation of evidence-based, non-pharmacological interventions addressing behavior and psychological symptoms of dementia: a systematic review focused on implementation strategies. Int Psychogeriatr. 2021;33:947–75. https://doi.org/10.1017/S1041610220001702 .

Noonan VK, Wolfe DL, Thorogood NP, et al. Knowledge translation and implementation in spinal cord injury: a systematic review. Spinal Cord. 2014;52:578–87. https://doi.org/10.1038/sc.2014.62 .

Albrecht L, Archibald M, Snelgrove-Clarke E, et al. Systematic review of knowledge translation strategies to promote research uptake in child health settings. J Pediatr Nurs. 2016;31:235–54. https://doi.org/10.1016/j.pedn.2015.12.002 .

Campbell A, Louie-Poon S, Slater L, et al. Knowledge translation strategies used by healthcare professionals in child health settings: an updated systematic review. J Pediatr Nurs. 2019;47:114–20. https://doi.org/10.1016/j.pedn.2019.04.026 .

Bird ML, Miller T, Connell LA, et al. Moving stroke rehabilitation evidence into practice: a systematic review of randomized controlled trials. Clin Rehabil. 2019;33:1586–95. https://doi.org/10.1177/0269215519847253 .

Goorts K, Dizon J, Milanese S. The effectiveness of implementation strategies for promoting evidence informed interventions in allied healthcare: a systematic review. BMC Health Serv Res. 2021;21:1–11. https://doi.org/10.1186/s12913-021-06190-0 .

Zadro JR, O’Keeffe M, Allison JL, Lembke KA, et al. Effectiveness of implementation strategies to improve adherence of physical therapist treatment choices to clinical practice guidelines for musculoskeletal conditions: systematic review. Phys Ther. 2020;100:1516–41. https://doi.org/10.1093/ptj/pzaa101 .

Van der Veer SN, Jager KJ, Nache AM, et al. Translating knowledge on best practice into improving quality of RRT care: a systematic review of implementation strategies. Kidney Int. 2011;80:1021–34. https://doi.org/10.1038/ki.2011.222 .

Pawson R, Greenhalgh T, Harvey G, et al. Realist review–a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005;10Suppl 1:21–34. https://doi.org/10.1258/1355819054308530 .

Rycroft-Malone J, McCormack B, Hutchinson AM, et al. Realist synthesis: illustrating the method for implementation research. Implementation Sci. 2012;7:1–10. https://doi.org/10.1186/1748-5908-7-33 .

Johnson MJ, May CR. Promoting professional behaviour change in healthcare: what interventions work, and why? A theory-led overview of systematic reviews. BMJ Open. 2015;5:e008592. https://doi.org/10.1136/bmjopen-2015-008592 .

Metz A, Jensen T, Farley A, Boaz A, et al. Is implementation research out of step with implementation practice? Pathways to effective implementation support over the last decade. Implement Res Pract. 2022;3:1–11. https://doi.org/10.1177/26334895221105585 .

May CR, Finch TL, Cornford J, Exley C, et al. Integrating telecare for chronic disease management in the community: What needs to be done? BMC Health Serv Res. 2011;11:1–11. https://doi.org/10.1186/1472-6963-11-131 .

Harvey G, Rycroft-Malone J, Seers K, Wilson P, et al. Connecting the science and practice of implementation – applying the lens of context to inform study design in implementation research. Front Health Serv. 2023;3:1–15. https://doi.org/10.3389/frhs.2023.1162762 .

Download references

Acknowledgements

The authors would like to thank Professor Kathryn Oliver for her support in the planning the review, Professor Steve Hanney for reading and commenting on the final manuscript and the staff at LSHTM library for their support in planning and conducting the literature search.

This study was supported by LSHTM’s Research England QR strategic priorities funding allocation and the National Institute for Health and Care Research (NIHR) Applied Research Collaboration South London (NIHR ARC South London) at King’s College Hospital NHS Foundation Trust. Grant number NIHR200152. The views expressed are those of the author(s) and not necessarily those of the NIHR, the Department of Health and Social Care or Research England.

Author information

Authors and affiliations.

Health and Social Care Workforce Research Unit, The Policy Institute, King’s College London, Virginia Woolf Building, 22 Kingsway, London, WC2B 6LE, UK

Annette Boaz

King’s Business School, King’s College London, 30 Aldwych, London, WC2B 4BG, UK

Juan Baeza & Alec Fraser

Federal University of Santa Catarina (UFSC), Campus Universitário Reitor João Davi Ferreira Lima, Florianópolis, SC, 88.040-900, Brazil

Erik Persson

You can also search for this author in PubMed   Google Scholar

Contributions

AB led the conceptual development and structure of the manuscript. EP conducted the searches and data extraction. All authors contributed to screening and quality appraisal. EP and AF wrote the first draft of the methods section. AB, JB and AF performed result synthesis and contributed to the analyses. AB wrote the first draft of the manuscript and incorporated feedback and revisions from all other authors. All authors revised and approved the final manuscript.

Corresponding author

Correspondence to Annette Boaz .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: appendix a., additional file 2: appendix b., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Boaz, A., Baeza, J., Fraser, A. et al. ‘It depends’: what 86 systematic reviews tell us about what strategies to use to support the use of research in clinical practice. Implementation Sci 19 , 15 (2024). https://doi.org/10.1186/s13012-024-01337-z

Download citation

Received : 01 November 2023

Accepted : 05 January 2024

Published : 19 February 2024

DOI : https://doi.org/10.1186/s13012-024-01337-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Implementation
  • Interventions
  • Clinical practice
  • Research evidence
  • Multi-faceted

Implementation Science

ISSN: 1748-5908

  • Submission enquiries: Access here and click Contact Us
  • General enquiries: [email protected]

research paper of database

Suggestions or feedback?

MIT News | Massachusetts Institute of Technology

  • Machine learning
  • Social justice
  • Black holes
  • Classes and programs

Departments

  • Aeronautics and Astronautics
  • Brain and Cognitive Sciences
  • Architecture
  • Political Science
  • Mechanical Engineering

Centers, Labs, & Programs

  • Abdul Latif Jameel Poverty Action Lab (J-PAL)
  • Picower Institute for Learning and Memory
  • Lincoln Laboratory
  • School of Architecture + Planning
  • School of Engineering
  • School of Humanities, Arts, and Social Sciences
  • Sloan School of Management
  • School of Science
  • MIT Schwarzman College of Computing

MIT researchers remotely map crops, field by field

Press contact :, media download.

Four Google Street View photos show rice, cassava, sugarcane, and maize fields.

*Terms of Use:

Images for download on the MIT News office website are made available to non-commercial entities, press and the general public under a Creative Commons Attribution Non-Commercial No Derivatives license . You may not alter the images provided, other than to crop them to size. A credit line must be used when reproducing images; if one is not provided below, credit the images to "MIT."

Four Google Street View photos show rice, cassava, sugarcane, and maize fields.

Previous image Next image

Crop maps help scientists and policymakers track global food supplies and estimate how they might shift with climate change and growing populations. But getting accurate maps of the types of crops that are grown from farm to farm often requires on-the-ground surveys that only a handful of countries have the resources to maintain.

Now, MIT engineers have developed a method to quickly and accurately label and map crop types without requiring in-person assessments of every single farm. The team’s method uses a combination of Google Street View images, machine learning, and satellite data to automatically determine the crops grown throughout a region, from one fraction of an acre to the next. 

The researchers used the technique to automatically generate the first nationwide crop map of Thailand — a smallholder country where small, independent farms make up the predominant form of agriculture. The team created a border-to-border map of Thailand’s four major crops — rice, cassava, sugarcane, and maize — and determined which of the four types was grown, at every 10 meters, and without gaps, across the entire country. The resulting map achieved an accuracy of 93 percent, which the researchers say is comparable to on-the-ground mapping efforts in high-income, big-farm countries.

The team is applying their mapping technique to other countries such as India, where small farms sustain most of the population but the type of crops grown from farm to farm has historically been poorly recorded.

“It’s a longstanding gap in knowledge about what is grown around the world,” says Sherrie Wang, the d’Arbeloff Career Development Assistant Professor in MIT’s Department of Mechanical Engineering, and the Institute for Data, Systems, and Society (IDSS). “The final goal is to understand agricultural outcomes like yield, and how to farm more sustainably. One of the key preliminary steps is to map what is even being grown — the more granularly you can map, the more questions you can answer.”

Wang, along with MIT graduate student Jordi Laguarta Soler and Thomas Friedel of the agtech company PEAT GmbH, will present a paper detailing their mapping method later this month at the AAAI Conference on Artificial Intelligence.

Ground truth

Smallholder farms are often run by a single family or farmer, who subsist on the crops and livestock that they raise. It’s estimated that smallholder farms support two-thirds of the world’s rural population and produce 80 percent of the world’s food. Keeping tabs on what is grown and where is essential to tracking and forecasting food supplies around the world. But the majority of these small farms are in low to middle-income countries, where few resources are devoted to keeping track of individual farms’ crop types and yields.

Crop mapping efforts are mainly carried out in high-income regions such as the United States and Europe, where government agricultural agencies oversee crop surveys and send assessors to farms to label crops from field to field. These “ground truth” labels are then fed into machine-learning models that make connections between the ground labels of actual crops and satellite signals of the same fields. They then label and map wider swaths of farmland that assessors don’t cover but that satellites automatically do.

“What’s lacking in low- and middle-income countries is this ground label that we can associate with satellite signals,” Laguarta Soler says. “Getting these ground truths to train a model in the first place has been limited in most of the world.”

The team realized that, while many developing countries do not have the resources to maintain crop surveys, they could potentially use another source of ground data: roadside imagery, captured by services such as Google Street View and Mapillary, which send cars throughout a region to take continuous 360-degree images with dashcams and rooftop cameras.

In recent years, such services have been able to access low- and middle-income countries. While the goal of these services is not specifically to capture images of crops, the MIT team saw that they could search the roadside images to identify crops.

Cropped image

In their new study, the researchers worked with Google Street View (GSV) images taken throughout Thailand — a country that the service has recently imaged fairly thoroughly, and which consists predominantly of smallholder farms.

Starting with over 200,000 GSV images randomly sampled across Thailand, the team filtered out images that depicted buildings, trees, and general vegetation. About 81,000 images were crop-related. They set aside 2,000 of these, which they sent to an agronomist, who determined and labeled each crop type by eye. They then trained a convolutional neural network to automatically generate crop labels for the other 79,000 images, using various training methods, including iNaturalist — a web-based crowdsourced  biodiversity database, and GPT-4V, a “multimodal large language model” that enables a user to input an image and ask the model to identify what the image is depicting. For each of the 81,000 images, the model generated a label of one of four crops that the image was likely depicting — rice, maize, sugarcane, or cassava.

The researchers then paired each labeled image with the corresponding satellite data taken of the same location throughout a single growing season. These satellite data include measurements across multiple wavelengths, such as a location’s greenness and its reflectivity (which can be a sign of water). 

“Each type of crop has a certain signature across these different bands, which changes throughout a growing season,” Laguarta Soler notes.

The team trained a second model to make associations between a location’s satellite data and its corresponding crop label. They then used this model to process satellite data taken of the rest of the country, where crop labels were not generated or available. From the associations that the model learned, it then assigned crop labels across Thailand, generating a country-wide map of crop types, at a resolution of 10 square meters.

This first-of-its-kind crop map included locations corresponding to the 2,000 GSV images that the researchers originally set aside, that were labeled by arborists. These human-labeled images were used to validate the map’s labels, and when the team looked to see whether the map’s labels matched the expert, “gold standard” labels, it did so 93 percent of the time.

“In the U.S., we’re also looking at over 90 percent accuracy, whereas with previous work in India, we’ve only seen 75 percent because ground labels are limited,” Wang says. “Now we can create these labels in a cheap and automated way.”

The researchers are moving to map crops across India, where roadside images via Google Street View and other services have recently become available.

“There are over 150 million smallholder farmers in India,” Wang says. “India is covered in agriculture, almost wall-to-wall farms, but very small farms, and historically it’s been very difficult to create maps of India because there are very sparse ground labels.”

The team is working to generate crop maps in India, which could be used to inform policies having to do with assessing and bolstering yields, as global temperatures and populations rise.

“What would be interesting would be to create these maps over time,” Wang says. “Then you could start to see trends, and we can try to relate those things to anything like changes in climate and policies.”

Share this news article on:

Related links.

  • Sherrie Wang
  • Institute for Data, Systems, and Society
  • Department of Mechanical Engineering

Related Topics

  • Agriculture
  • Computer modeling
  • Computer vision
  • Developing countries
  • Environment
  • Mechanical engineering

Related Articles

Collage of eleven new faculty member's headshots, arranged in two rows

School of Engineering welcomes new faculty

Landscape of a peat bog under a blue sky. In the foreground, several islands of peat are surrounded by water.

Satellite-based method measures carbon in peat bogs

Three women, researchers from the GEAR Lab, stand on a dirt road in a field in Jordan holding laptops.

Smart irrigation technology covers “more crop per drop”

The village has about 20 huts that form a large a ring around an empty, brown, circular area. Lots of trees are around the village.

Ancient Amazonians intentionally created fertile “dark earth”

Aerial view of an abandoned agricultural terrace in France

3 Questions: Can disused croplands help mitigate climate change?

Previous item Next item

More MIT News

Three layers show a glob of glue, shiny circular metal bits, and the colorful blue computer chip. Pink lasers go through the chip and hit the circular metal bits and bounce back. A lock icon is to the right.

This tiny, tamper-proof ID tag can authenticate almost anything

Read full story →

An aerial view shows trees and sports stadiums and parking lots. The areas are measured with a road measuring “921.97ft” and a parking lot measuring, “614,232.74 ft squared.”

Stitch3D is powering a new wave of 3D data collaboration

Headshots of Thea Keith-Lucas, Sarah Johnson, and Natalie Hill

MIT course aids social connection, better relationships, and happiness

Illustration of U shaped vaccine moledules with tails attach to three-lobed albumin molecules. The background is an image of a oval shaped lymph node.

Hitchhiking cancer vaccine makes progress in the clinic

Portrait photo of Leon Sandler standing in the foyer of an MIT building

A passion for innovation and education

Graphic of glowing moleculars being touched by an electrical charge against black background

With just a little electricity, MIT researchers boost common catalytic reactions

  • More news on MIT News homepage →

Massachusetts Institute of Technology 77 Massachusetts Avenue, Cambridge, MA, USA

  • Map (opens in new window)
  • Events (opens in new window)
  • People (opens in new window)
  • Careers (opens in new window)
  • Accessibility
  • Social Media Hub
  • MIT on Facebook
  • MIT on YouTube
  • MIT on Instagram

Help | Advanced Search

Computer Science > Computation and Language

Title: large language models: a survey.

Abstract: Large Language Models (LLMs) have drawn a lot of attention due to their strong performance on a wide range of natural language tasks, since the release of ChatGPT in November 2022. LLMs' ability of general-purpose language understanding and generation is acquired by training billions of model's parameters on massive amounts of text data, as predicted by scaling laws \cite{kaplan2020scaling,hoffmann2022training}. The research area of LLMs, while very recent, is evolving rapidly in many different ways. In this paper, we review some of the most prominent LLMs, including three popular LLM families (GPT, LLaMA, PaLM), and discuss their characteristics, contributions and limitations. We also give an overview of techniques developed to build, and augment LLMs. We then survey popular datasets prepared for LLM training, fine-tuning, and evaluation, review widely used LLM evaluation metrics, and compare the performance of several popular LLMs on a set of representative benchmarks. Finally, we conclude the paper by discussing open challenges and future research directions.

Submission history

Access paper:.

  • Download PDF
  • HTML (experimental)
  • Other Formats

license icon

References & Citations

  • Google Scholar
  • Semantic Scholar

BibTeX formatted citation

BibSonomy logo

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

  • Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

IMAGES

  1. (PDF) Impact of database management in modern world

    research paper of database

  2. Sample research paper example

    research paper of database

  3. Science Research Paper Example / Examples Of Science Research Papers

    research paper of database

  4. 3

    research paper of database

  5. (PDF) Database System Concepts, Implementations and Organizations-A

    research paper of database

  6. Writing a Databases Research Paper

    research paper of database

VIDEO

  1. Introduction to Temporal Databases

  2. XML Databases

  3. Binary Conversion

  4. Ph.D

  5. Free database to search articles for your thesis

  6. Previous year question paper of database management system #btech #ptu #cse #dbms

COMMENTS

  1. The best academic research databases [Update 2024]

    The top list of academic research databases Content: 1. Scopus 2. Web of Science 3. PubMed 4. ERIC 5. IEEE Xplore 6. ScienceDirect 7. Directory of Open Access Journals (DOAJ) 8. JSTOR Frequently Asked Questions about academic research databases Related Articles

  2. List of academic databases and search engines

    This article contains a representative list of notable databases and search engines useful in an academic setting for finding and accessing articles in academic journals, institutional repositories, archives, or other collections of scientific and other articles.

  3. 19024 PDFs

    Jan 2024 Explore the latest full-text research PDFs, articles, conference papers, preprints and more on DATABASE MANAGEMENT SYSTEMS. Find methods information, sources, references or conduct a...

  4. JSTOR Home

    Harness the power of visual materials—explore more than 3 million images now on JSTOR. Enhance your scholarly research with underground newspapers, magazines, and journals. Explore collections in the arts, sciences, and literature from the world's leading museums, archives, and scholars. JSTOR is a digital library of academic journals ...

  5. PDF Database management system performance comparisons: A systematic

    arXiv:2301.01095v3 [cs.PF] 27 Nov 2023 has been tested, (ii) how performance has been recommended to be tested, (iii) how the performance comparison results should be interpreted, (iv) what other aspects besides performance should be considered, and (v) what other avenues might be fruitful for DBMS performance testing.

  6. Advances in database systems education: Methods, tools, curricula, and

    Researchers have designed a variety of tools for making different concepts of introductory database course more interesting and easier to teach and learn interactively (Brusilovsky et al., 2010) either using visual support (Nagataki et al., 2013 ), or with the help of gamification (Fisher & Khine, 2006 ).

  7. (PDF) Database System: Concepts and Design

    MEDLINE, DrugBank, Protein Data Bank (PDB) and Binding Database (BD) are some of the examples of databases (Gunjal 2003). 54 J. Reang et al. • User-friendly: Database delivers easy to learn...

  8. Google Scholar

    Google Scholar provides a simple way to broadly search for scholarly literature. Search across a wide variety of disciplines and sources: articles, theses, books, abstracts and court opinions.

  9. Search NCBI databases

    Search all biomedical databases provided by the National Center for Biotechnology Information (NCBI), an agency of the U.S. National Library of Medicine at the NIH ... of a new online tool, the Dataset Catalog. The tool is designed to help improve the discoverability and reuse of research data by making it easier for users to find and connect ...

  10. (PDF) A Literature Review on Evolving Database

    A Literature Review on Evolving Database March 2017 International Journal of Computer Applications 162 (9):35-41 DOI: Authors: Shagufta Praveen Glocal University Umesh Chandra Banda University of...

  11. Relational data paradigms: What do we learn by taking the materiality

    Borgman CL (2012) The conundrum of sharing research data. Journal of the American Society for Information Science and Technology 63(6): 1-40. Crossref. Google Scholar. ... Lupton D (2015) Personal Data Practices in the Age of Lively Data. Scholarly Paper, 28 July. Rochester, NY: Social Science Research Network. Available at: ...

  12. Databases

    A database is one or more sets of data, for example numbers, characters and images, bundled together with software that enables data to be added, removed or retrieved. Databases can be used to ...

  13. Database Search

    A catalog to find the specialized search engine that has what you need—identifying and connecting to the best databases for your research topic. What is Database Search? Harvard Library licenses hundreds of online databases, giving you access to academic and news articles, books, journals, primary sources, streaming media, and much more.

  14. Home

    PubMed Central ® (PMC) is a free full-text archive of biomedical and life sciences journal literature at the U.S. National Institutes of Health's National Library of Medicine (NIH/NLM) About PMC Discover a digital archive of scholarly articles, spanning centuries of scientific research. User Guide

  15. PubMed

    PubMed® comprises more than 36 million citations for biomedical literature from MEDLINE, life science journals, and online books. Citations may include links to full text content from PubMed Central and publisher web sites.

  16. The Ultimate Guide to Using Databases for Paper Writing

    Introduction. Writing a research paper can be a daunting task, especially when it comes to finding reliable and relevant sources. With the abundance of information available online, it's crucial to utilize databases that curate high-quality content.

  17. Academic research: how to search online databases [8 steps ...

    Use the campus network to access research databases. 2. Find databases that are specifically related to your topic. 3. Set up the search parameters within a database to be as narrow as possible. 4. Ask a librarian for help. 5. Slowly expand your search to get additional results.

  18. Academic Databases

    The best academic research databases [Update 2024] Your research is stuck and you need to find new sources? Take a look at our compilation of academic research databases: Scopus, Web of Science, PubMed, ERIC, JSTOR, DOAJ, Science Direct, and IEEE Xplore.

  19. 21 Legit Research Databases for Free Journal Articles in 2022

    While looking for websites for research, you come across a research paper site that claims to connect academics to a peer-reviewed article database for free. Intrigued, you search for keywords related to your topic, only to discover that you must pay a hefty subscription fee to access the service.

  20. 23 Research Databases for Professional and Academic Use

    Scopus Scopus is a database that features literature for a large variety of disciplines. It offers some services for free, but full access to the database requires a subscription. A unique feature of Scopus is that it also ranks journals and authors by their h-index, which tracks how many users cite the specific resource.

  21. arXiv.org e-Print archive

    arXiv is a free distribution service and an open-access archive for nearly 2.4 million scholarly articles in the fields of physics, mathematics, computer science, quantitative biology, quantitative finance, statistics, electrical engineering and systems science, and economics.

  22. Internet Archive Scholar

    Search Millions of Research Papers This fulltext search index includes over 35 million research articles and other scholarly documents preserved in the Internet Archive. The collection spans from digitized copies of eighteenth century journals through the latest Open Access conference proceedings and preprints crawled from the World Wide Web.

  23. Free Research Databases from EBSCO

    GreenFILE is a free research database covering all aspects of human impact to the environment. Its collection of scholarly, government and general-interest titles includes content on global warming, green building, pollution, sustainable agriculture, renewable energy, recycling, and more. Access now

  24. New global area estimates for coral reefs from high-resolution mapping

    This paper acts as a scientific reference for the mapping framework and summary statistics for the Allen Coral Atlas. In summary, for shallow tropical reefs (−30° to 30° latitude) at 5-m-pixel resolution, we present: (1) their global extent and distribution, (2) the distribution of 11 geomorphic classes, and (3) the distribution of 6 benthic classes (Figure 1).

  25. China conducts first nationwide review of retractions and research

    Chinese universities are days away from the deadline to complete a nationwide audit of retracted research papers and probe of research misconduct. By 15 February, universities must submit to the ...

  26. 'It depends': what 86 systematic reviews tell us about what strategies

    To reduce the risk of bias, papers were excluded following discussion between all members of the team. Data were synthesised using descriptive and narrative techniques to identify themes and patterns linked to intervention strategies, targeted behaviours, study settings and study outcomes. We identified 32 reviews conducted between 2010 and 2022.

  27. MIT researchers remotely map crops, field by field

    A new method can remotely map crop types in low- or middle-income countries where agricultural data are sparse. The maps will help scientists and policymakers track global food supplies and estimate how they might shift with climate change and growing populations. ... Paper. Paper: "Combining Deep Learning and Street View Imagery to Map ...

  28. [2402.06196] Large Language Models: A Survey

    The research area of LLMs, while very recent, is evolving rapidly in many different ways. In this paper, we review some of the most prominent LLMs, including three popular LLM families (GPT, LLaMA, PaLM), and discuss their characteristics, contributions and limitations. We also give an overview of techniques developed to build, and augment LLMs.

  29. Search

    Discover the world's scientific knowledge With 160+ million publication pages, 25+ million researchers and 1+ million questions, this is where everyone can access science You can use AND, OR, NOT,...