literature review table examples

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• Write for Us
• BMJ Journals

You are here

• Volume 24, Issue 2
• Five tips for developing useful literature summary tables for writing review articles
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0003-0157-5319 Ahtisham Younas 1 , 2 ,
• http://orcid.org/0000-0002-7839-8130 Parveen Ali 3 , 4
• 1 Memorial University of Newfoundland , St John's , Newfoundland , Canada
• 2 Swat College of Nursing , Pakistan
• 3 School of Nursing and Midwifery , University of Sheffield , Sheffield , South Yorkshire , UK
• 4 Sheffield University Interpersonal Violence Research Group , Sheffield University , Sheffield , UK
• Correspondence to Ahtisham Younas, Memorial University of Newfoundland, St John's, NL A1C 5C4, Canada; ay6133{at}mun.ca

https://doi.org/10.1136/ebnurs-2021-103417

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

Literature reviews offer a critical synthesis of empirical and theoretical literature to assess the strength of evidence, develop guidelines for practice and policymaking, and identify areas for future research. 1 It is often essential and usually the first task in any research endeavour, particularly in masters or doctoral level education. For effective data extraction and rigorous synthesis in reviews, the use of literature summary tables is of utmost importance. A literature summary table provides a synopsis of an included article. It succinctly presents its purpose, methods, findings and other relevant information pertinent to the review. The aim of developing these literature summary tables is to provide the reader with the information at one glance. Since there are multiple types of reviews (eg, systematic, integrative, scoping, critical and mixed methods) with distinct purposes and techniques, 2 there could be various approaches for developing literature summary tables making it a complex task specialty for the novice researchers or reviewers. Here, we offer five tips for authors of the review articles, relevant to all types of reviews, for creating useful and relevant literature summary tables. We also provide examples from our published reviews to illustrate how useful literature summary tables can be developed and what sort of information should be provided.

Tip 1: provide detailed information about frameworks and methods

• Download figure
• Open in new tab
• Download powerpoint

Tabular literature summaries from a scoping review. Source: Rasheed et al . 3

The provision of information about conceptual and theoretical frameworks and methods is useful for several reasons. First, in quantitative (reviews synthesising the results of quantitative studies) and mixed reviews (reviews synthesising the results of both qualitative and quantitative studies to address a mixed review question), it allows the readers to assess the congruence of the core findings and methods with the adapted framework and tested assumptions. In qualitative reviews (reviews synthesising results of qualitative studies), this information is beneficial for readers to recognise the underlying philosophical and paradigmatic stance of the authors of the included articles. For example, imagine the authors of an article, included in a review, used phenomenological inquiry for their research. In that case, the review authors and the readers of the review need to know what kind of (transcendental or hermeneutic) philosophical stance guided the inquiry. Review authors should, therefore, include the philosophical stance in their literature summary for the particular article. Second, information about frameworks and methods enables review authors and readers to judge the quality of the research, which allows for discerning the strengths and limitations of the article. For example, if authors of an included article intended to develop a new scale and test its psychometric properties. To achieve this aim, they used a convenience sample of 150 participants and performed exploratory (EFA) and confirmatory factor analysis (CFA) on the same sample. Such an approach would indicate a flawed methodology because EFA and CFA should not be conducted on the same sample. The review authors must include this information in their summary table. Omitting this information from a summary could lead to the inclusion of a flawed article in the review, thereby jeopardising the review’s rigour.

Tip 2: include strengths and limitations for each article

Critical appraisal of individual articles included in a review is crucial for increasing the rigour of the review. Despite using various templates for critical appraisal, authors often do not provide detailed information about each reviewed article’s strengths and limitations. Merely noting the quality score based on standardised critical appraisal templates is not adequate because the readers should be able to identify the reasons for assigning a weak or moderate rating. Many recent critical appraisal checklists (eg, Mixed Methods Appraisal Tool) discourage review authors from assigning a quality score and recommend noting the main strengths and limitations of included studies. It is also vital that methodological and conceptual limitations and strengths of the articles included in the review are provided because not all review articles include empirical research papers. Rather some review synthesises the theoretical aspects of articles. Providing information about conceptual limitations is also important for readers to judge the quality of foundations of the research. For example, if you included a mixed-methods study in the review, reporting the methodological and conceptual limitations about ‘integration’ is critical for evaluating the study’s strength. Suppose the authors only collected qualitative and quantitative data and did not state the intent and timing of integration. In that case, the strength of the study is weak. Integration only occurred at the levels of data collection. However, integration may not have occurred at the analysis, interpretation and reporting levels.

Tip 3: write conceptual contribution of each reviewed article

While reading and evaluating review papers, we have observed that many review authors only provide core results of the article included in a review and do not explain the conceptual contribution offered by the included article. We refer to conceptual contribution as a description of how the article’s key results contribute towards the development of potential codes, themes or subthemes, or emerging patterns that are reported as the review findings. For example, the authors of a review article noted that one of the research articles included in their review demonstrated the usefulness of case studies and reflective logs as strategies for fostering compassion in nursing students. The conceptual contribution of this research article could be that experiential learning is one way to teach compassion to nursing students, as supported by case studies and reflective logs. This conceptual contribution of the article should be mentioned in the literature summary table. Delineating each reviewed article’s conceptual contribution is particularly beneficial in qualitative reviews, mixed-methods reviews, and critical reviews that often focus on developing models and describing or explaining various phenomena. Figure 2 offers an example of a literature summary table. 4

Tabular literature summaries from a critical review. Source: Younas and Maddigan. 4

Tip 4: compose potential themes from each article during summary writing

While developing literature summary tables, many authors use themes or subthemes reported in the given articles as the key results of their own review. Such an approach prevents the review authors from understanding the article’s conceptual contribution, developing rigorous synthesis and drawing reasonable interpretations of results from an individual article. Ultimately, it affects the generation of novel review findings. For example, one of the articles about women’s healthcare-seeking behaviours in developing countries reported a theme ‘social-cultural determinants of health as precursors of delays’. Instead of using this theme as one of the review findings, the reviewers should read and interpret beyond the given description in an article, compare and contrast themes, findings from one article with findings and themes from another article to find similarities and differences and to understand and explain bigger picture for their readers. Therefore, while developing literature summary tables, think twice before using the predeveloped themes. Including your themes in the summary tables (see figure 1 ) demonstrates to the readers that a robust method of data extraction and synthesis has been followed.

Tip 5: create your personalised template for literature summaries

Often templates are available for data extraction and development of literature summary tables. The available templates may be in the form of a table, chart or a structured framework that extracts some essential information about every article. The commonly used information may include authors, purpose, methods, key results and quality scores. While extracting all relevant information is important, such templates should be tailored to meet the needs of the individuals’ review. For example, for a review about the effectiveness of healthcare interventions, a literature summary table must include information about the intervention, its type, content timing, duration, setting, effectiveness, negative consequences, and receivers and implementers’ experiences of its usage. Similarly, literature summary tables for articles included in a meta-synthesis must include information about the participants’ characteristics, research context and conceptual contribution of each reviewed article so as to help the reader make an informed decision about the usefulness or lack of usefulness of the individual article in the review and the whole review.

In conclusion, narrative or systematic reviews are almost always conducted as a part of any educational project (thesis or dissertation) or academic or clinical research. Literature reviews are the foundation of research on a given topic. Robust and high-quality reviews play an instrumental role in guiding research, practice and policymaking. However, the quality of reviews is also contingent on rigorous data extraction and synthesis, which require developing literature summaries. We have outlined five tips that could enhance the quality of the data extraction and synthesis process by developing useful literature summaries.

• Aromataris E ,
• Rasheed SP ,

Twitter @Ahtisham04, @parveenazamali

Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

Competing interests None declared.

Patient consent for publication Not required.

Provenance and peer review Not commissioned; externally peer reviewed.

Read the full text or download the PDF:

Welcome to the new OASIS website! We have academic skills, library skills, math and statistics support, and writing resources all together in one new home.

• Walden University
• Faculty Portal

Common Assignments: Literature Review Matrix

Literature review matrix.

As you read and evaluate your literature there are several different ways to organize your research. Courtesy of Dr. Gary Burkholder in the School of Psychology, these sample matrices are one option to help organize your articles. These documents allow you to compile details about your sources, such as the foundational theories, methodologies, and conclusions; begin to note similarities among the authors; and retrieve citation information for easy insertion within a document.

You can review the sample matrixes to see a completed form or download the blank matrix for your own use.

• Literature Review Matrix 1 This PDF file provides a sample literature review matrix.
• Literature Review Matrix 2 This PDF file provides a sample literature review matrix.
• Literature Review Matrix Template (Word)
• Literature Review Matrix Template (Excel)

Related Resources

Didn't find what you need? Email us at [email protected] .

• Previous Page: Commentary Versus Opinion
• Next Page: Professional Development Plans (PDPs)
• Office of Student Disability Services

Walden Resources

Departments.

• Academic Residencies
• Academic Skills
• Career Planning and Development
• Customer Care Team
• Field Experience
• Military Services
• Student Success Advising
• Writing Skills

Centers and Offices

• Center for Social Change
• Office of Academic Support and Instructional Services
• Office of Degree Acceleration
• Office of Research and Doctoral Services
• Office of Student Affairs

Student Resources

• Doctoral Writing Assessment
• Form & Style Review
• Quick Answers
• ScholarWorks
• SKIL Courses and Workshops
• Walden Bookstore
• Walden Catalog & Student Handbook
• Student Safety/Title IX
• Legal & Consumer Information
• Website Terms and Conditions
• Cookie Policy
• Accessibility
• Accreditation
• State Authorization
• Net Price Calculator
• Cost of Attendance
• Contact Walden

Walden University is a member of Adtalem Global Education, Inc. www.adtalem.com Walden University is certified to operate by SCHEV © 2024 Walden University LLC. All rights reserved.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates, and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

• Quick Run-through
• Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and its scholarly context
• Develop a theoretical framework and methodology for your research
• Position your work in relation to other researchers and theorists
• Show how your research addresses a gap or contributes to a debate
• Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models, and methods?
• Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

Prevent plagiarism. Run a free check.

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

• Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically evaluate: mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarize yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved August 26, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

What is a Literature Review? How to Write It (with Examples)

A literature review is a critical analysis and synthesis of existing research on a particular topic. It provides an overview of the current state of knowledge, identifies gaps, and highlights key findings in the literature. 1 The purpose of a literature review is to situate your own research within the context of existing scholarship, demonstrating your understanding of the topic and showing how your work contributes to the ongoing conversation in the field. Learning how to write a literature review is a critical tool for successful research. Your ability to summarize and synthesize prior research pertaining to a certain topic demonstrates your grasp on the topic of study, and assists in the learning process. 

Table of Contents

• What is the purpose of literature review? 
• a. Habitat Loss and Species Extinction: 
• b. Range Shifts and Phenological Changes: 
• c. Ocean Acidification and Coral Reefs: 
• d. Adaptive Strategies and Conservation Efforts: 

How to write a good literature review 

• Choose a Topic and Define the Research Question: 
• Decide on the Scope of Your Review: 
• Select Databases for Searches: 
• Conduct Searches and Keep Track: 
• Review the Literature: 
• Organize and Write Your Literature Review: 
• How to write a literature review faster with Paperpal? 
• Frequently asked questions 

What is a literature review?

A well-conducted literature review demonstrates the researcher’s familiarity with the existing literature, establishes the context for their own research, and contributes to scholarly conversations on the topic. One of the purposes of a literature review is also to help researchers avoid duplicating previous work and ensure that their research is informed by and builds upon the existing body of knowledge.

What is the purpose of literature review?

A literature review serves several important purposes within academic and research contexts. Here are some key objectives and functions of a literature review: 2  

1. Contextualizing the Research Problem: The literature review provides a background and context for the research problem under investigation. It helps to situate the study within the existing body of knowledge. 

2. Identifying Gaps in Knowledge: By identifying gaps, contradictions, or areas requiring further research, the researcher can shape the research question and justify the significance of the study. This is crucial for ensuring that the new research contributes something novel to the field. 

Find academic papers related to your research topic faster. Try Research on Paperpal  

3. Understanding Theoretical and Conceptual Frameworks: Literature reviews help researchers gain an understanding of the theoretical and conceptual frameworks used in previous studies. This aids in the development of a theoretical framework for the current research. 

4. Providing Methodological Insights: Another purpose of literature reviews is that it allows researchers to learn about the methodologies employed in previous studies. This can help in choosing appropriate research methods for the current study and avoiding pitfalls that others may have encountered. 

5. Establishing Credibility: A well-conducted literature review demonstrates the researcher’s familiarity with existing scholarship, establishing their credibility and expertise in the field. It also helps in building a solid foundation for the new research. 

6. Informing Hypotheses or Research Questions: The literature review guides the formulation of hypotheses or research questions by highlighting relevant findings and areas of uncertainty in existing literature. 

Literature review example

Let’s delve deeper with a literature review example: Let’s say your literature review is about the impact of climate change on biodiversity. You might format your literature review into sections such as the effects of climate change on habitat loss and species extinction, phenological changes, and marine biodiversity. Each section would then summarize and analyze relevant studies in those areas, highlighting key findings and identifying gaps in the research. The review would conclude by emphasizing the need for further research on specific aspects of the relationship between climate change and biodiversity. The following literature review template provides a glimpse into the recommended literature review structure and content, demonstrating how research findings are organized around specific themes within a broader topic. 

Literature Review on Climate Change Impacts on Biodiversity:

Climate change is a global phenomenon with far-reaching consequences, including significant impacts on biodiversity. This literature review synthesizes key findings from various studies: 

a. Habitat Loss and Species Extinction:

Climate change-induced alterations in temperature and precipitation patterns contribute to habitat loss, affecting numerous species (Thomas et al., 2004). The review discusses how these changes increase the risk of extinction, particularly for species with specific habitat requirements. 

b. Range Shifts and Phenological Changes:

Observations of range shifts and changes in the timing of biological events (phenology) are documented in response to changing climatic conditions (Parmesan & Yohe, 2003). These shifts affect ecosystems and may lead to mismatches between species and their resources. 

c. Ocean Acidification and Coral Reefs:

The review explores the impact of climate change on marine biodiversity, emphasizing ocean acidification’s threat to coral reefs (Hoegh-Guldberg et al., 2007). Changes in pH levels negatively affect coral calcification, disrupting the delicate balance of marine ecosystems. 

d. Adaptive Strategies and Conservation Efforts:

Recognizing the urgency of the situation, the literature review discusses various adaptive strategies adopted by species and conservation efforts aimed at mitigating the impacts of climate change on biodiversity (Hannah et al., 2007). It emphasizes the importance of interdisciplinary approaches for effective conservation planning. 

Strengthen your literature review with factual insights. Try Research on Paperpal for free!    

Writing a literature review involves summarizing and synthesizing existing research on a particular topic. A good literature review format should include the following elements. 

Introduction: The introduction sets the stage for your literature review, providing context and introducing the main focus of your review. 

• Opening Statement: Begin with a general statement about the broader topic and its significance in the field. 
• Scope and Purpose: Clearly define the scope of your literature review. Explain the specific research question or objective you aim to address. 
• Organizational Framework: Briefly outline the structure of your literature review, indicating how you will categorize and discuss the existing research. 
• Significance of the Study: Highlight why your literature review is important and how it contributes to the understanding of the chosen topic. 
• Thesis Statement: Conclude the introduction with a concise thesis statement that outlines the main argument or perspective you will develop in the body of the literature review. 

Body: The body of the literature review is where you provide a comprehensive analysis of existing literature, grouping studies based on themes, methodologies, or other relevant criteria. 

• Organize by Theme or Concept: Group studies that share common themes, concepts, or methodologies. Discuss each theme or concept in detail, summarizing key findings and identifying gaps or areas of disagreement. 
• Critical Analysis: Evaluate the strengths and weaknesses of each study. Discuss the methodologies used, the quality of evidence, and the overall contribution of each work to the understanding of the topic. 
• Synthesis of Findings: Synthesize the information from different studies to highlight trends, patterns, or areas of consensus in the literature. 
• Identification of Gaps: Discuss any gaps or limitations in the existing research and explain how your review contributes to filling these gaps. 
• Transition between Sections: Provide smooth transitions between different themes or concepts to maintain the flow of your literature review. 

Write and Cite as you go with Paperpal Research. Start now for free.   

Conclusion: The conclusion of your literature review should summarize the main findings, highlight the contributions of the review, and suggest avenues for future research. 

• Summary of Key Findings: Recap the main findings from the literature and restate how they contribute to your research question or objective. 
• Contributions to the Field: Discuss the overall contribution of your literature review to the existing knowledge in the field. 
• Implications and Applications: Explore the practical implications of the findings and suggest how they might impact future research or practice. 
• Recommendations for Future Research: Identify areas that require further investigation and propose potential directions for future research in the field. 
• Final Thoughts: Conclude with a final reflection on the importance of your literature review and its relevance to the broader academic community. 

Conducting a literature review

Conducting a literature review is an essential step in research that involves reviewing and analyzing existing literature on a specific topic. It’s important to know how to do a literature review effectively, so here are the steps to follow: 1  

Choose a Topic and Define the Research Question:

• Select a topic that is relevant to your field of study. 
• Clearly define your research question or objective. Determine what specific aspect of the topic do you want to explore? 

Decide on the Scope of Your Review:

• Determine the timeframe for your literature review. Are you focusing on recent developments, or do you want a historical overview? 
• Consider the geographical scope. Is your review global, or are you focusing on a specific region? 
• Define the inclusion and exclusion criteria. What types of sources will you include? Are there specific types of studies or publications you will exclude? 

Select Databases for Searches:

• Identify relevant databases for your field. Examples include PubMed, IEEE Xplore, Scopus, Web of Science, and Google Scholar. 
• Consider searching in library catalogs, institutional repositories, and specialized databases related to your topic. 

Conduct Searches and Keep Track:

• Develop a systematic search strategy using keywords, Boolean operators (AND, OR, NOT), and other search techniques. 
• Record and document your search strategy for transparency and replicability. 
• Keep track of the articles, including publication details, abstracts, and links. Use citation management tools like EndNote, Zotero, or Mendeley to organize your references. 

Review the Literature:

• Evaluate the relevance and quality of each source. Consider the methodology, sample size, and results of studies. 
• Organize the literature by themes or key concepts. Identify patterns, trends, and gaps in the existing research. 
• Summarize key findings and arguments from each source. Compare and contrast different perspectives. 
• Identify areas where there is a consensus in the literature and where there are conflicting opinions. 
• Provide critical analysis and synthesis of the literature. What are the strengths and weaknesses of existing research? 

Organize and Write Your Literature Review:

• Literature review outline should be based on themes, chronological order, or methodological approaches. 
• Write a clear and coherent narrative that synthesizes the information gathered. 
• Use proper citations for each source and ensure consistency in your citation style (APA, MLA, Chicago, etc.). 
• Conclude your literature review by summarizing key findings, identifying gaps, and suggesting areas for future research. 

Whether you’re exploring a new research field or finding new angles to develop an existing topic, sifting through hundreds of papers can take more time than you have to spare. But what if you could find science-backed insights with verified citations in seconds? That’s the power of Paperpal’s new Research feature!  

How to write a literature review faster with Paperpal?

Paperpal, an AI writing assistant, integrates powerful academic search capabilities within its writing platform. With the Research feature, you get 100% factual insights, with citations backed by 250M+ verified research articles, directly within your writing interface with the option to save relevant references in your Citation Library. By eliminating the need to switch tabs to find answers to all your research questions, Paperpal saves time and helps you stay focused on your writing.   

Here’s how to use the Research feature:  

• Ask a question: Get started with a new document on paperpal.com. Click on the “Research” feature and type your question in plain English. Paperpal will scour over 250 million research articles, including conference papers and preprints, to provide you with accurate insights and citations. 
• Review and Save: Paperpal summarizes the information, while citing sources and listing relevant reads. You can quickly scan the results to identify relevant references and save these directly to your built-in citations library for later access. 
• Cite with Confidence: Paperpal makes it easy to incorporate relevant citations and references into your writing, ensuring your arguments are well-supported by credible sources. This translates to a polished, well-researched literature review. 

The literature review sample and detailed advice on writing and conducting a review will help you produce a well-structured report. But remember that a good literature review is an ongoing process, and it may be necessary to revisit and update it as your research progresses. By combining effortless research with an easy citation process, Paperpal Research streamlines the literature review process and empowers you to write faster and with more confidence. Try Paperpal Research now and see for yourself.  

Frequently asked questions

A literature review is a critical and comprehensive analysis of existing literature (published and unpublished works) on a specific topic or research question and provides a synthesis of the current state of knowledge in a particular field. A well-conducted literature review is crucial for researchers to build upon existing knowledge, avoid duplication of efforts, and contribute to the advancement of their field. It also helps researchers situate their work within a broader context and facilitates the development of a sound theoretical and conceptual framework for their studies.

Literature review is a crucial component of research writing, providing a solid background for a research paper’s investigation. The aim is to keep professionals up to date by providing an understanding of ongoing developments within a specific field, including research methods, and experimental techniques used in that field, and present that knowledge in the form of a written report. Also, the depth and breadth of the literature review emphasizes the credibility of the scholar in his or her field.  

Before writing a literature review, it’s essential to undertake several preparatory steps to ensure that your review is well-researched, organized, and focused. This includes choosing a topic of general interest to you and doing exploratory research on that topic, writing an annotated bibliography, and noting major points, especially those that relate to the position you have taken on the topic. 

Literature reviews and academic research papers are essential components of scholarly work but serve different purposes within the academic realm. 3 A literature review aims to provide a foundation for understanding the current state of research on a particular topic, identify gaps or controversies, and lay the groundwork for future research. Therefore, it draws heavily from existing academic sources, including books, journal articles, and other scholarly publications. In contrast, an academic research paper aims to present new knowledge, contribute to the academic discourse, and advance the understanding of a specific research question. Therefore, it involves a mix of existing literature (in the introduction and literature review sections) and original data or findings obtained through research methods. 

Literature reviews are essential components of academic and research papers, and various strategies can be employed to conduct them effectively. If you want to know how to write a literature review for a research paper, here are four common approaches that are often used by researchers.  Chronological Review: This strategy involves organizing the literature based on the chronological order of publication. It helps to trace the development of a topic over time, showing how ideas, theories, and research have evolved.  Thematic Review: Thematic reviews focus on identifying and analyzing themes or topics that cut across different studies. Instead of organizing the literature chronologically, it is grouped by key themes or concepts, allowing for a comprehensive exploration of various aspects of the topic.  Methodological Review: This strategy involves organizing the literature based on the research methods employed in different studies. It helps to highlight the strengths and weaknesses of various methodologies and allows the reader to evaluate the reliability and validity of the research findings.  Theoretical Review: A theoretical review examines the literature based on the theoretical frameworks used in different studies. This approach helps to identify the key theories that have been applied to the topic and assess their contributions to the understanding of the subject.  It’s important to note that these strategies are not mutually exclusive, and a literature review may combine elements of more than one approach. The choice of strategy depends on the research question, the nature of the literature available, and the goals of the review. Additionally, other strategies, such as integrative reviews or systematic reviews, may be employed depending on the specific requirements of the research.

The literature review format can vary depending on the specific publication guidelines. However, there are some common elements and structures that are often followed. Here is a general guideline for the format of a literature review:  Introduction:   Provide an overview of the topic.  Define the scope and purpose of the literature review.  State the research question or objective.  Body:   Organize the literature by themes, concepts, or chronology.  Critically analyze and evaluate each source.  Discuss the strengths and weaknesses of the studies.  Highlight any methodological limitations or biases.  Identify patterns, connections, or contradictions in the existing research.  Conclusion:   Summarize the key points discussed in the literature review.  Highlight the research gap.  Address the research question or objective stated in the introduction.  Highlight the contributions of the review and suggest directions for future research.

Both annotated bibliographies and literature reviews involve the examination of scholarly sources. While annotated bibliographies focus on individual sources with brief annotations, literature reviews provide a more in-depth, integrated, and comprehensive analysis of existing literature on a specific topic. The key differences are as follows: 

References 

• Denney, A. S., & Tewksbury, R. (2013). How to write a literature review.  Journal of criminal justice education ,  24 (2), 218-234. 
• Pan, M. L. (2016).  Preparing literature reviews: Qualitative and quantitative approaches . Taylor & Francis. 
• Cantero, C. (2019). How to write a literature review.  San José State University Writing Center . 

Paperpal is an AI writing assistant that help academics write better, faster with real-time suggestions for in-depth language and grammar correction. Trained on millions of research manuscripts enhanced by professional academic editors, Paperpal delivers human precision at machine speed.  

Try it for free or upgrade to  Paperpal Prime , which unlocks unlimited access to premium features like academic translation, paraphrasing, contextual synonyms, consistency checks and more. It’s like always having a professional academic editor by your side! Go beyond limitations and experience the future of academic writing.  Get Paperpal Prime now at just US$19 a month!

Related Reads:

• Empirical Research: A Comprehensive Guide for Academics 
• How to Write a Scientific Paper in 10 Steps 
• How Long Should a Chapter Be?
• How to Use Paperpal to Generate Emails & Cover Letters?

6 Tips for Post-Doc Researchers to Take Their Career to the Next Level

Self-plagiarism in research: what it is and how to avoid it, you may also like, academic integrity vs academic dishonesty: types & examples, dissertation printing and binding | types & comparison , what is a dissertation preface definition and examples , the ai revolution: authors’ role in upholding academic..., the future of academia: how ai tools are..., how to write a research proposal: (with examples..., how to write your research paper in apa..., how to choose a dissertation topic, how to write a phd research proposal, how to write an academic paragraph (step-by-step guide).

• Research Guides
• Vanderbilt University Libraries
• Peabody Library

Literature Reviews

Literature review table.

• Steps to Success: The Literature Review Process
• Literature Reviews Webinar Recording
• Writing Like an Academic
• Publishing in Academic Journals
• Managing Citations This link opens in a new window
• Literature Review Table Template Peabody Librarians have created a sample literature review table to use to organize your research. Feel free to download this file and use or adapt as needed.
• << Previous: Literature Reviews Webinar Recording
• Next: Writing Like an Academic >>
• Last Updated: Jul 12, 2024 12:55 PM
• URL: https://researchguides.library.vanderbilt.edu/peabody/litreviews

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• CAREER FEATURE
• 04 December 2020
• Correction 09 December 2020

How to write a superb literature review

Andy Tay is a freelance writer based in Singapore.

You can also search for this author in PubMed   Google Scholar

Credit: Getty

Literature reviews are important resources for scientists. They provide historical context for a field while offering opinions on its future trajectory. Creating them can provide inspiration for one’s own research, as well as some practice in writing. But few scientists are trained in how to write a review — or in what constitutes an excellent one. Even picking the appropriate software to use can be an involved decision (see ‘Tools and techniques’). So Nature asked editors and working scientists with well-cited reviews for their tips.

WENTING ZHAO: Be focused and avoid jargon

Assistant professor of chemical and biomedical engineering, Nanyang Technological University, Singapore.

When I was a research student, review writing improved my understanding of the history of my field. I also learnt about unmet challenges in the field that triggered ideas.

For example, while writing my first review 1 as a PhD student, I was frustrated by how poorly we understood how cells actively sense, interact with and adapt to nanoparticles used in drug delivery. This experience motivated me to study how the surface properties of nanoparticles can be modified to enhance biological sensing. When I transitioned to my postdoctoral research, this question led me to discover the role of cell-membrane curvature, which led to publications and my current research focus. I wouldn’t have started in this area without writing that review.

Collection: Careers toolkit

A common problem for students writing their first reviews is being overly ambitious. When I wrote mine, I imagined producing a comprehensive summary of every single type of nanomaterial used in biological applications. It ended up becoming a colossal piece of work, with too many papers discussed and without a clear way to categorize them. We published the work in the end, but decided to limit the discussion strictly to nanoparticles for biological sensing, rather than covering how different nanomaterials are used in biology.

My advice to students is to accept that a review is unlike a textbook: it should offer a more focused discussion, and it’s OK to skip some topics so that you do not distract your readers. Students should also consider editorial deadlines, especially for invited reviews: make sure that the review’s scope is not so extensive that it delays the writing.

A good review should also avoid jargon and explain the basic concepts for someone who is new to the field. Although I trained as an engineer, I’m interested in biology, and my research is about developing nanomaterials to manipulate proteins at the cell membrane and how this can affect ageing and cancer. As an ‘outsider’, the reviews that I find most useful for these biological topics are those that speak to me in accessible scientific language.

Bozhi Tian likes to get a variety of perspectives into a review. Credit: Aleksander Prominski

BOZHI TIAN: Have a process and develop your style

Associate professor of chemistry, University of Chicago, Illinois.

In my lab, we start by asking: what is the purpose of this review? My reasons for writing one can include the chance to contribute insights to the scientific community and identify opportunities for my research. I also see review writing as a way to train early-career researchers in soft skills such as project management and leadership. This is especially true for lead authors, because they will learn to work with their co-authors to integrate the various sections into a piece with smooth transitions and no overlaps.

After we have identified the need and purpose of a review article, I will form a team from the researchers in my lab. I try to include students with different areas of expertise, because it is useful to get a variety of perspectives. For example, in the review ‘An atlas of nano-enabled neural interfaces’ 2 , we had authors with backgrounds in biophysics, neuroengineering, neurobiology and materials sciences focusing on different sections of the review.

After this, I will discuss an outline with my team. We go through multiple iterations to make sure that we have scanned the literature sufficiently and do not repeat discussions that have appeared in other reviews. It is also important that the outline is not decided by me alone: students often have fresh ideas that they can bring to the table. Once this is done, we proceed with the writing.

I often remind my students to imagine themselves as ‘artists of science’ and encourage them to develop how they write and present information. Adding more words isn’t always the best way: for example, I enjoy using tables to summarize research progress and suggest future research trajectories. I’ve also considered including short videos in our review papers to highlight key aspects of the work. I think this can increase readership and accessibility because these videos can be easily shared on social-media platforms.

ANKITA ANIRBAN: Timeliness and figures make a huge difference

Editor, Nature Reviews Physics .

One of my roles as a journal editor is to evaluate proposals for reviews. The best proposals are timely and clearly explain why readers should pay attention to the proposed topic.

It is not enough for a review to be a summary of the latest growth in the literature: the most interesting reviews instead provide a discussion about disagreements in the field.

Careers Collection: Publishing

Scientists often centre the story of their primary research papers around their figures — but when it comes to reviews, figures often take a secondary role. In my opinion, review figures are more important than most people think. One of my favourite review-style articles 3 presents a plot bringing together data from multiple research papers (many of which directly contradict each other). This is then used to identify broad trends and suggest underlying mechanisms that could explain all of the different conclusions.

An important role of a review article is to introduce researchers to a field. For this, schematic figures can be useful to illustrate the science being discussed, in much the same way as the first slide of a talk should. That is why, at Nature Reviews, we have in-house illustrators to assist authors. However, simplicity is key, and even without support from professional illustrators, researchers can still make use of many free drawing tools to enhance the value of their review figures.

Yoojin Choi recommends that researchers be open to critiques when writing reviews. Credit: Yoojin Choi

YOOJIN CHOI: Stay updated and be open to suggestions

Research assistant professor, Korea Advanced Institute of Science and Technology, Daejeon.

I started writing the review ‘Biosynthesis of inorganic nanomaterials using microbial cells and bacteriophages’ 4 as a PhD student in 2018. It took me one year to write the first draft because I was working on the review alongside my PhD research and mostly on my own, with support from my adviser. It took a further year to complete the processes of peer review, revision and publication. During this time, many new papers and even competing reviews were published. To provide the most up-to-date and original review, I had to stay abreast of the literature. In my case, I made use of Google Scholar, which I set to send me daily updates of relevant literature based on key words.

Through my review-writing process, I also learnt to be more open to critiques to enhance the value and increase the readership of my work. Initially, my review was focused only on using microbial cells such as bacteria to produce nanomaterials, which was the subject of my PhD research. Bacteria such as these are known as biofactories: that is, organisms that produce biological material which can be modified to produce useful materials, such as magnetic nanoparticles for drug-delivery purposes.

Synchronized editing: the future of collaborative writing

However, when the first peer-review report came back, all three reviewers suggested expanding the review to cover another type of biofactory: bacteriophages. These are essentially viruses that infect bacteria, and they can also produce nanomaterials.

The feedback eventually led me to include a discussion of the differences between the various biofactories (bacteriophages, bacteria, fungi and microalgae) and their advantages and disadvantages. This turned out to be a great addition because it made the review more comprehensive.

Writing the review also led me to an idea about using nanomaterial-modified microorganisms to produce chemicals, which I’m still researching now.

PAULA MARTIN-GONZALEZ: Make good use of technology

PhD student, University of Cambridge, UK.

Just before the coronavirus lockdown, my PhD adviser and I decided to write a literature review discussing the integration of medical imaging with genomics to improve ovarian cancer management.

As I was researching the review, I noticed a trend in which some papers were consistently being cited by many other papers in the field. It was clear to me that those papers must be important, but as a new member of the field of integrated cancer biology, it was difficult to immediately find and read all of these ‘seminal papers’.

That was when I decided to code a small application to make my literature research more efficient. Using my code, users can enter a query, such as ‘ovarian cancer, computer tomography, radiomics’, and the application searches for all relevant literature archived in databases such as PubMed that feature these key words.

The code then identifies the relevant papers and creates a citation graph of all the references cited in the results of the search. The software highlights papers that have many citation relationships with other papers in the search, and could therefore be called seminal papers.

My code has substantially improved how I organize papers and has informed me of key publications and discoveries in my research field: something that would have taken more time and experience in the field otherwise. After I shared my code on GitHub, I received feedback that it can be daunting for researchers who are not used to coding. Consequently, I am hoping to build a more user-friendly interface in a form of a web page, akin to PubMed or Google Scholar, where users can simply input their queries to generate citation graphs.

Tools and techniques

Most reference managers on the market offer similar capabilities when it comes to providing a Microsoft Word plug-in and producing different citation styles. But depending on your working preferences, some might be more suitable than others.

Reference managers

Here is a comparison of the more popular collaborative writing tools, but there are other options, including Fidus Writer, Manuscript.io, Authorea and Stencila.

Collaborative writing tools

*Markdown and LaTex are code-based formatting languages favoured by physicists, mathematicians and computer scientists who code on a regular basis, and less popular in other disciplines such as biology and chemistry.

doi: https://doi.org/10.1038/d41586-020-03422-x

Interviews have been edited for length and clarity.

Updates & Corrections

Correction 09 December 2020 : An earlier version of the tables in this article included some incorrect details about the programs Zotero, Endnote and Manubot. These have now been corrected.

Hsing, I.-M., Xu, Y. & Zhao, W. Electroanalysis 19 , 755–768 (2007).

Article   Google Scholar  

Ledesma, H. A. et al. Nature Nanotechnol. 14 , 645–657 (2019).

Article   PubMed   Google Scholar  

Brahlek, M., Koirala, N., Bansal, N. & Oh, S. Solid State Commun. 215–216 , 54–62 (2015).

Choi, Y. & Lee, S. Y. Nature Rev. Chem . https://doi.org/10.1038/s41570-020-00221-w (2020).

Download references

Related Articles

• Research management

How to harness AI’s potential in research — responsibly and ethically

Career Feature 23 AUG 24

Partners in drug discovery: how to collaborate with non-governmental organizations

Time to refocus for South Korean science

Nature Index 21 AUG 24

South Korean science on the global stage

How South Korea can build better gender diversity into research

The citation black market: schemes selling fake references alarm scientists

News 20 AUG 24

Lecturer/Senior Lecturer at the Dyson School of Design Engineering

About the role: Do you want to change the world for the better, and do you believe this can be done through research and education? If so, grab you...

South Kensington, London (Greater) (GB)

Imperial College London (ICL)

Faculty Positions in Westlake University

Founded in 2018, Westlake University is a new type of non-profit research-oriented university in Hangzhou, China, supported by public a...

Hangzhou, Zhejiang, China

Westlake University

Call for Global Talents, Recruitment Information of Nankai University

Nankai University welcomes global outstanding talents to join for common development.

Tianjin, China

Nankai University

Director for Signature Research Programme in Cancer and Stem Cell Biology, DukeNUS Medical School &

The successful candidate will demonstrate their ability of growing, leading and mentoring faculty as well as developing and operationalising strategy.

Singapore (SG)

DukeNUS Medical School

Scientist / Postdoc (m/f/d): Analysis of Microscopic BIOMedical Images (AMBIOM)

A new project area in the institute is the development of artificial intelligence (AI)

Dortmund, Nordrhein-Westfalen (DE)

Leibniz-Institut für Analytische Wissenschaften – ISAS – e.V.

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Reference management. Clean and simple.

What is a literature review? [with examples]

What is a literature review?

The purpose of a literature review, how to write a literature review, the format of a literature review, general formatting rules, the length of a literature review, literature review examples, frequently asked questions about literature reviews, related articles.

A literature review is an assessment of the sources in a chosen topic of research.

In a literature review, you’re expected to report on the existing scholarly conversation, without adding new contributions.

If you are currently writing one, you've come to the right place. In the following paragraphs, we will explain:

• the objective of a literature review
• how to write a literature review
• the basic format of a literature review

Tip: It’s not always mandatory to add a literature review in a paper. Theses and dissertations often include them, whereas research papers may not. Make sure to consult with your instructor for exact requirements.

The four main objectives of a literature review are:

• Studying the references of your research area
• Summarizing the main arguments
• Identifying current gaps, stances, and issues
• Presenting all of the above in a text

Ultimately, the main goal of a literature review is to provide the researcher with sufficient knowledge about the topic in question so that they can eventually make an intervention.

The format of a literature review is fairly standard. It includes an:

• introduction that briefly introduces the main topic
• body that includes the main discussion of the key arguments
• conclusion that highlights the gaps and issues of the literature

➡️ Take a look at our guide on how to write a literature review to learn more about how to structure a literature review.

First of all, a literature review should have its own labeled section. You should indicate clearly in the table of contents where the literature can be found, and you should label this section as “Literature Review.”

➡️ For more information on writing a thesis, visit our guide on how to structure a thesis .

There is no set amount of words for a literature review, so the length depends on the research. If you are working with a large amount of sources, it will be long. If your paper does not depend entirely on references, it will be short.

Take a look at these three theses featuring great literature reviews:

• School-Based Speech-Language Pathologist's Perceptions of Sensory Food Aversions in Children [ PDF , see page 20]
• Who's Writing What We Read: Authorship in Criminological Research [ PDF , see page 4]
• A Phenomenological Study of the Lived Experience of Online Instructors of Theological Reflection at Christian Institutions Accredited by the Association of Theological Schools [ PDF , see page 56]

Literature reviews are most commonly found in theses and dissertations. However, you find them in research papers as well.

There is no set amount of words for a literature review, so the length depends on the research. If you are working with a large amount of sources, then it will be long. If your paper does not depend entirely on references, then it will be short.

No. A literature review should have its own independent section. You should indicate clearly in the table of contents where the literature review can be found, and label this section as “Literature Review.”

The main goal of a literature review is to provide the researcher with sufficient knowledge about the topic in question so that they can eventually make an intervention.

How To Structure Your Literature Review

3 options to help structure your chapter.

By: Amy Rommelspacher (PhD) | Reviewer: Dr Eunice Rautenbach | November 2020 (Updated May 2023)

Writing the literature review chapter can seem pretty daunting when you’re piecing together your dissertation or thesis. As  we’ve discussed before , a good literature review needs to achieve a few very important objectives – it should:

• Demonstrate your knowledge of the research topic
• Identify the gaps in the literature and show how your research links to these
• Provide the foundation for your conceptual framework (if you have one)
• Inform your own  methodology and research design

To achieve this, your literature review needs a well-thought-out structure . Get the structure of your literature review chapter wrong and you’ll struggle to achieve these objectives. Don’t worry though – in this post, we’ll look at how to structure your literature review for maximum impact (and marks!).

But wait – is this the right time?

Deciding on the structure of your literature review should come towards the end of the literature review process – after you have collected and digested the literature, but before you start writing the chapter. 

In other words, you need to first develop a rich understanding of the literature before you even attempt to map out a structure. There’s no use trying to develop a structure before you’ve fully wrapped your head around the existing research.

Equally importantly, you need to have a structure in place before you start writing , or your literature review will most likely end up a rambling, disjointed mess. 

Importantly, don’t feel that once you’ve defined a structure you can’t iterate on it. It’s perfectly natural to adjust as you engage in the writing process. As we’ve discussed before , writing is a way of developing your thinking, so it’s quite common for your thinking to change – and therefore, for your chapter structure to change – as you write. 

Need a helping hand?

Like any other chapter in your thesis or dissertation, your literature review needs to have a clear, logical structure. At a minimum, it should have three essential components – an  introduction , a  body   and a  conclusion . 

Let’s take a closer look at each of these.

1: The Introduction Section

Just like any good introduction, the introduction section of your literature review should introduce the purpose and layout (organisation) of the chapter. In other words, your introduction needs to give the reader a taste of what’s to come, and how you’re going to lay that out. Essentially, you should provide the reader with a high-level roadmap of your chapter to give them a taste of the journey that lies ahead.

Here’s an example of the layout visualised in a literature review introduction:

Your introduction should also outline your topic (including any tricky terminology or jargon) and provide an explanation of the scope of your literature review – in other words, what you  will   and  won’t   be covering (the delimitations ). This helps ringfence your review and achieve a clear focus . The clearer and narrower your focus, the deeper you can dive into the topic (which is typically where the magic lies). 

Depending on the nature of your project, you could also present your stance or point of view at this stage. In other words, after grappling with the literature you’ll have an opinion about what the trends and concerns are in the field as well as what’s lacking. The introduction section can then present these ideas so that it is clear to examiners that you’re aware of how your research connects with existing knowledge .

2: The Body Section

The body of your literature review is the centre of your work. This is where you’ll present, analyse, evaluate and synthesise the existing research. In other words, this is where you’re going to earn (or lose) the most marks. Therefore, it’s important to carefully think about how you will organise your discussion to present it in a clear way. 

The body of your literature review should do just as the description of this chapter suggests. It should “review” the literature – in other words, identify, analyse, and synthesise it. So, when thinking about structuring your literature review, you need to think about which structural approach will provide the best “review” for your specific type of research and objectives (we’ll get to this shortly).

There are (broadly speaking)  three options  for organising your literature review.

Option 1: Chronological (according to date)

Organising the literature chronologically is one of the simplest ways to structure your literature review. You start with what was published first and work your way through the literature until you reach the work published most recently. Pretty straightforward.

The benefit of this option is that it makes it easy to discuss the developments and debates in the field as they emerged over time. Organising your literature chronologically also allows you to highlight how specific articles or pieces of work might have changed the course of the field – in other words, which research has had the most impact . Therefore, this approach is very useful when your research is aimed at understanding how the topic has unfolded over time and is often used by scholars in the field of history. That said, this approach can be utilised by anyone that wants to explore change over time .

For example , if a student of politics is investigating how the understanding of democracy has evolved over time, they could use the chronological approach to provide a narrative that demonstrates how this understanding has changed through the ages.

Here are some questions you can ask yourself to help you structure your literature review chronologically.

• What is the earliest literature published relating to this topic?
• How has the field changed over time? Why?
• What are the most recent discoveries/theories?

In some ways, chronology plays a part whichever way you decide to structure your literature review, because you will always, to a certain extent, be analysing how the literature has developed. However, with the chronological approach, the emphasis is very firmly on how the discussion has evolved over time , as opposed to how all the literature links together (which we’ll discuss next ).

Option 2: Thematic (grouped by theme)

The thematic approach to structuring a literature review means organising your literature by theme or category – for example, by independent variables (i.e. factors that have an impact on a specific outcome).

As you’ve been collecting and synthesising literature , you’ll likely have started seeing some themes or patterns emerging. You can then use these themes or patterns as a structure for your body discussion. The thematic approach is the most common approach and is useful for structuring literature reviews in most fields.

For example, if you were researching which factors contributed towards people trusting an organisation, you might find themes such as consumers’ perceptions of an organisation’s competence, benevolence and integrity. Structuring your literature review thematically would mean structuring your literature review’s body section to discuss each of these themes, one section at a time.

Here are some questions to ask yourself when structuring your literature review by themes:

• Are there any patterns that have come to light in the literature?
• What are the central themes and categories used by the researchers?
• Do I have enough evidence of these themes?

PS – you can see an example of a thematically structured literature review in our literature review sample walkthrough video here.

Option 3: Methodological

The methodological option is a way of structuring your literature review by the research methodologies used . In other words, organising your discussion based on the angle from which each piece of research was approached – for example, qualitative , quantitative or mixed  methodologies.

Structuring your literature review by methodology can be useful if you are drawing research from a variety of disciplines and are critiquing different methodologies. The point of this approach is to question  how  existing research has been conducted, as opposed to  what  the conclusions and/or findings the research were.

For example, a sociologist might centre their research around critiquing specific fieldwork practices. Their literature review will then be a summary of the fieldwork methodologies used by different studies.

Here are some questions you can ask yourself when structuring your literature review according to methodology:

• Which methodologies have been utilised in this field?
• Which methodology is the most popular (and why)?
• What are the strengths and weaknesses of the various methodologies?
• How can the existing methodologies inform my own methodology?

3: The Conclusion Section

Once you’ve completed the body section of your literature review using one of the structural approaches we discussed above, you’ll need to “wrap up” your literature review and pull all the pieces together to set the direction for the rest of your dissertation or thesis.

The conclusion is where you’ll present the key findings of your literature review. In this section, you should emphasise the research that is especially important to your research questions and highlight the gaps that exist in the literature. Based on this, you need to make it clear what you will add to the literature – in other words, justify your own research by showing how it will help fill one or more of the gaps you just identified.

Last but not least, if it’s your intention to develop a conceptual framework for your dissertation or thesis, the conclusion section is a good place to present this.

Example: Thematically Structured Review

In the video below, we unpack a literature review chapter so that you can see an example of a thematically structure review in practice.

Let’s Recap

In this article, we’ve  discussed how to structure your literature review for maximum impact. Here’s a quick recap of what  you need to keep in mind when deciding on your literature review structure:

• Just like other chapters, your literature review needs a clear introduction , body and conclusion .
• The introduction section should provide an overview of what you will discuss in your literature review.
• The body section of your literature review can be organised by chronology , theme or methodology . The right structural approach depends on what you’re trying to achieve with your research.
• The conclusion section should draw together the key findings of your literature review and link them to your research questions.

If you’re ready to get started, be sure to download our free literature review template to fast-track your chapter outline.

Psst… there’s more!

This post is an extract from our bestselling short course, Literature Review Bootcamp . If you want to work smart, you don't want to miss this .

28 Comments

Great work. This is exactly what I was looking for and helps a lot together with your previous post on literature review. One last thing is missing: a link to a great literature chapter of an journal article (maybe with comments of the different sections in this review chapter). Do you know any great literature review chapters?

I agree with you Marin… A great piece

I agree with Marin. This would be quite helpful if you annotate a nicely structured literature from previously published research articles.

Awesome article for my research.

I thank you immensely for this wonderful guide

It is indeed thought and supportive work for the futurist researcher and students

Very educative and good time to get guide. Thank you

Great work, very insightful. Thank you.

Thanks for this wonderful presentation. My question is that do I put all the variables into a single conceptual framework or each hypothesis will have it own conceptual framework?

Thank you very much, very helpful

This is very educative and precise . Thank you very much for dropping this kind of write up .

Pheeww, so damn helpful, thank you for this informative piece.

I’m doing a research project topic ; stool analysis for parasitic worm (enteric) worm, how do I structure it, thanks.

comprehensive explanation. Help us by pasting the URL of some good “literature review” for better understanding.

great piece. thanks for the awesome explanation. it is really worth sharing. I have a little question, if anyone can help me out, which of the options in the body of literature can be best fit if you are writing an architectural thesis that deals with design?

I am doing a research on nanofluids how can l structure it?

Beautifully clear.nThank you!

Lucid! Thankyou!

Brilliant work, well understood, many thanks

I like how this was so clear with simple language 😊😊 thank you so much 😊 for these information 😊

Insightful. I was struggling to come up with a sensible literature review but this has been really helpful. Thank you!

You have given thought-provoking information about the review of the literature.

Thank you. It has made my own research better and to impart your work to students I teach

I learnt a lot from this teaching. It’s a great piece.

I am doing research on EFL teacher motivation for his/her job. How Can I structure it? Is there any detailed template, additional to this?

You are so cool! I do not think I’ve read through something like this before. So nice to find somebody with some genuine thoughts on this issue. Seriously.. thank you for starting this up. This site is one thing that is required on the internet, someone with a little originality!

I’m asked to do conceptual, theoretical and empirical literature, and i just don’t know how to structure it

Asking questions are actually fastidious thing if you are not understanding anything fully, but this article presents good understanding yet.

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

Literature Review Basics

• What is a Literature Review?
• Synthesizing Research
• Using Research & Synthesis Tables
• Additional Resources

About the Research and Synthesis Tables

Research Tables and Synthesis Tables are useful tools for organizing and analyzing your research as you assemble your literature review. They represent two different parts of the review process: assembling relevant information and synthesizing it. Use a Research table to compile the main info you need about the items you find in your research -- it's a great thing to have on hand as you take notes on what you read! Then, once you've assembled your research, use the Synthesis table to start charting the similarities/differences and major themes among your collected items.

We've included an Excel file with templates for you to use below; the examples pictured on this page are snapshots from that file.

• Research and Synthesis Table Templates This Excel workbook includes simple templates for creating research tables and synthesis tables. Feel free to download and use!

Using the Research Table

This is an example of a  research table,  in which you provide a basic description of the most important features of the studies, articles, and other items you discover in your research. The table identifies each item according to its author/date of publication, its purpose or thesis, what type of work it is (systematic review, clinical trial, etc.), the level of evidence it represents (which tells you a lot about its impact on the field of study), and its major findings. Your job, when you assemble this information, is to develop a snapshot of what the research shows about the topic of your research question and assess its value (both for the purpose of your work and for general knowledge in the field).

Think of your work on the research table as the foundational step for your analysis of the literature, in which you assemble the information you'll be analyzing and lay the groundwork for thinking about what it means and how it can be used.

Using the Synthesis Table

This is an example of a  synthesis table  or  synthesis matrix , in which you organize and analyze your research by listing each source and indicating whether a given finding or result occurred in a particular study or article ( each row lists an individual source, and each finding has its own column, in which X = yes, blank = no). You can also add or alter the columns to look for shared study populations, sort by level of evidence or source type, etc. The key here is to use the table to provide a simple representation of what the research has found (or not found, as the case may be). Think of a synthesis table as a tool for making comparisons, identifying trends, and locating gaps in the literature.

How do I know which findings to use, or how many to include?  Your research question tells you which findings are of interest in your research, so work from your research question to decide what needs to go in each Finding header, and how many findings are necessary. The number is up to you; again, you can alter this table by adding or deleting columns to match what you're actually looking for in your analysis. You should also, of course, be guided by what's actually present in the material your research turns up!

• << Previous: Synthesizing Research
• Next: Additional Resources >>
• Last Updated: Sep 26, 2023 12:06 PM
• URL: https://usi.libguides.com/literature-review-basics

• Reserve a study room
• Library Account
• Undergraduate Students
• Graduate Students
• Faculty & Staff

How to Conduct a Literature Review (Health Sciences and Beyond)

• What is a Literature Review?
• Developing a Research Question
• Selection Criteria
• Database Search
• Documenting Your Search

Review Matrix

• Reference Management

Using a spreadsheet or table to organize the key elements (e.g. subjects, methodologies, results) of articles/books you plan to use in your literature review can be helpful. This is called a review matrix.

When you create a review matrix, the first few columns should include (1) the authors, title, journal, (2) publication year, and (3) purpose of the paper. The remaining columns should identify important aspects of each study such as methodology and findings.

Click on the image below to view a sample review matrix.

You can also download this template as a Microsoft Excel file .

The information on this page is from the book below. The 5th edition is available online through VCU Libraries.

• << Previous: Documenting Your Search
• Next: Reference Management >>
• Last Updated: Mar 15, 2024 12:22 PM
• URL: https://guides.library.vcu.edu/health-sciences-lit-review

The Sheridan Libraries

• Write a Literature Review
• Sheridan Libraries
• Evaluate This link opens in a new window

What Will You Do Differently?

Please help your librarians by filling out this two-minute survey of today's class session..

Professor, this one's for you .

Introduction

Literature reviews take time. here is some general information to know before you start.  .

•  VIDEO -- This video is a great overview of the entire process.  (2020; North Carolina State University Libraries) --The transcript is included --This is for everyone; ignore the mention of "graduate students" --9.5 minutes, and every second is important  
• OVERVIEW -- Read this page from Purdue's OWL. It's not long, and gives some tips to fill in what you just learned from the video.  
• NOT A RESEARCH ARTICLE -- A literature review follows a different style, format, and structure from a research article.  

• Next: Evaluate >>
• Last Updated: Jul 30, 2024 1:42 PM
• URL: https://guides.library.jhu.edu/lit-review

Literature Review Example

A literature review is a summary of the existing knowledge and research on a particular subject. by identifying gaps in the literature, it provides a foundation for future research. as such, it’s a crucial first step in any research project..

What is a literature review?

A literature review serves several purposes:

• identifies knowledge gaps
• evaluates the quality of existing research
• provides a foundation for newly presented research

Looking at existing examples of literature reviews is beneficial to get a clear understanding of what they entail. Find examples of a literature review by using an academic search engine (e.g. Google Scholar). As a starting point, search for your keyword or topic along with the term "literature review".

Example of literature review

Identify the research question or topic, making it as narrow as possible. In this example of a literature review, we review the anxiolytic (anti-anxiety) activity of Piper methysticum , or Kava .

Let's walk through the steps in the process with this literature review example.

Define the research question

First, identify the research question or topic, making it as narrow as possible. In this literature review example, we're examining the effects of urbanization on the migration of birds.

Search for relevant literature

Searching for relevant studies is arguably the most important aspect of the literature review.

Start by identifying keywords and phrases related to the topic and use them to search academic journals and databases ( Google Scholar , BASE , PubMed , etc.). For our example, you might start with "the effects of urbanization on bird migration", but after researching the field, discover that other terms like "avian migration" and "avian populations" are more commonly used.

Search for your keywords in Litmaps to find some initial articles to explore the field from. You can then use Litmaps to find additinal sources and curate a whole library of literature on your topic.

Search for your keywords in Litmaps, and select a starting article. This will return a visualization containing suggestions for relevant articles on your literature review topic. Review these to start curating your library.

Evaluate the sources

Evaluate the relevance and quality of the sources found by reading abstracts of the most relevant articles. Additionally, consider the publication venue, year of publication and other salient measures to identify the reliability and relevance of the source.

Read and analyze the sources

Take notes on the key findings, methodologies, and theoretical frameworks used in the studies.

Use a research-friendly note-taking software, like Obsidian , that provide #tags to keep track of key concepts.

Organize the literature

Organize the literature according to themes, subtopics, or categories, which will help outline the layout of the literature review.

Tag keywords using a tool like Obsidian to help organize papers into subtopics for the review.

Write the literature review

Summarize and synthesize the findings from the sources analyzed. Start with an introduction that defines the research question, followed by the themes, subtopics, or categories identified. After that, provide a discussion or conclusion that addresses any gaps in the literature to motivate future research. Lastly, edit and revise your review to ensure it is well-structured, clear, and concise. The example below is from a review paper, which includes a table comparing the different sources evaluated. Such tables can be useful if you are conducting a comprehensive review.

If you're conducting a comprehensive review, you can include a table of sources reviewed in your process, like the one above from this publication .

Cite and reference the sources

Lastly, cite and reference the sources used in the literature review. Consider any referencing style requirements of the institution or journal you're submitting to. APA is the most common. However, you may need to familiarize yourself with other citation styles such as MLA, Chicago, or MHRA depending on your venue. See the image below for a literature review example APA of references. To cite references you've saved in Litmaps, you can move your saved articles from Litmaps to a reference manager (i.e. Zotero, Mendeley, EndNote, etc.) and then export their bibliography from there. Here's how to export articles from Litmaps.

Use a reference manager tool like Zotero to easily export which makes them easy to manage, like in this APA literature review example.

A successful literature review tells a brief story about the topic at hand and leaves the reader a clear notion of what has been covered. Most importantly, a literature review addresses any gaps in the field and frames newly presented research. Understand the key steps and look at literature review examples in order to create a high quality review.

Header image Forest & Kim Starr, used under Creative Commons BY 3.0

• UWF Libraries

Literature Review: Conducting & Writing

• Sample Literature Reviews
• Steps for Conducting a Lit Review
• Finding "The Literature"
• Organizing/Writing
• APA Style This link opens in a new window
• Chicago: Notes Bibliography This link opens in a new window
• MLA Style This link opens in a new window

Sample Lit Reviews from Communication Arts

Have an exemplary literature review.

• Literature Review Sample 1
• Literature Review Sample 2
• Literature Review Sample 3

Have you written a stellar literature review you care to share for teaching purposes?

Are you an instructor who has received an exemplary literature review and have permission from the student to post?

Please contact Britt McGowan at [email protected] for inclusion in this guide. All disciplines welcome and encouraged.

• << Previous: MLA Style
• Next: Get Help! >>
• Last Updated: Aug 8, 2024 11:00 AM
• URL: https://libguides.uwf.edu/litreview

Nursing and Allied Health: Building a Summary Table or Synthesis Matrix

• Nursing Library Services
• Library Support for Nursing Accreditation
• Academic Research Libraries (ACRL) Information Literacy Standards for Science
• ACRL Guidelines for Distance Learning
• AACN Resources: Information Literacy
• The Essentials: Core Competencies for Professional Nursing Education
• Accreditation Commision for Education
• ACEN Learning Resources | Definitions & Standards
• Tutoring & Paper Review: College of Nursing & Allied Health Students
• Technology and Collaboration
• Affordable Learning Louisiana
• NIH Virtual Reality Funding
• Assistive Technology Resources
• Web Accessibility Initiative WC3
• Healthcare Links for Persons With Disabilities
• Accessibility
• Accessibility Tools: Videos
• Braile Institute: Blind and Low Vision Web Accessibility
• Braile Institute: Assistive Technology for the Visually Impaired This link opens in a new window
• Mental Health Resources for Students & Faculty
• Student Activities & Organizations
• Anatomage FAQ's
• APA Reference Worksheet With Examples This link opens in a new window
• APA Style and Grammar Guidelines, 7.0
• What's New in APA Guide, 7.0 This link opens in a new window
• APA Instructional Aids for Professors/Instructors
• APA Handouts & Guides This link opens in a new window
• Sample Papers for Students and Instructors
• Academic Writer Tutorial: Basics This link opens in a new window
• APA Styling Your Paper
• When to cite or not cite a database
• Video: A Step-By-Step Guide for APA Style Student Papers
• Video: Citing Works in Text Using APA 7.0 This link opens in a new window
• Video: Creating References Using APA Guide, 7.0 This link opens in a new window
• Journal Article Reporting Standards
• Digital Information Literacy
• Tips Sheet: Copyright Essentials for Higher Education This link opens in a new window
• Citing National Patient Safety Goals (Joint Commission)
• Citing Agency for Healthcare Research and Quality (AHRQ) in APA format
• Best Nursing Apps
• Writing an Abstract for Your PILT or Special Project
• PILT Poster Presentation Archive
• Academic Success
• Healthcare Career Salaries
• Industry Statistics: Healthcare
• Nursing Organizations
• Radiology Organizations
• Controlled Medical Vocabularies
• COVID-19 Current Vaccines (Medscape)
• COVID-19 Current and Candidate Vaccine Landscape and Tracker (WHO)
• COVID-19: Clinician Care Guidance (CDC.gov) This link opens in a new window
• COVID -19 Contract Tracing & Patient Privacy (Cornell) This link opens in a new window
• COVID-19: Coronavirus Dashboard (Johns Hopkins Epidemiology)
• COVID-19: Coronavirus Guidelines (Up-To-Date) This link opens in a new window
• COVID-19: Critical Care Evidence (Cochrane) This link opens in a new window
• COVID-19: Diagnosis & Treatment (JAMA) This link opens in a new window
• COVID-19: Free Video Access (Jove) This link opens in a new window
• COVID-19: Healthcare Hub (Elsevier) This link opens in a new window
• COVID-19: Keeping Up With A Moving Target (Johns Hopkins Nursing Videos)
• COVID-19: LitCovid Daily Update (NLM)
• COVID 19 - Long Term Health Effects Stemming From COVID-19
• COVID-19: Louisiana Department of Health
• COVID-19: Novel Coronavirus Information Center (Elsevier) This link opens in a new window
• COVID-19 Nursing Resources (Medscape)
• COVID-19: Red Book - Recent Pediatric Updates (AAP)
• COVID-19: Washing Your Hands Thoroughly (NHS)
• COVID-19: Well-Being Initiative (ANF)
• Properly Putting On Your Facemask & Getting a Good Seal (Dr. Scheiner) This link opens in a new window
• Creating Personal Accounts
• Creating a CINAHL MyFolder Account
• Creating a PubMed | MyNCBI Personal Account
• Creating a ProQuest Nursing & Allied Health Premium | MyResearch Folder
• Creating an OVID MyWorkspace Personal Account
• Mobile APPS | CINAHL for Apple and Android Mobile Devices
• My Circulation Login
• Interlibrary Loan Personal Account
• Data Visualization Products
• International Classification of Diseases (ICD)
• Diagnostic and Statistical Manual of Mental Disorders | DSM-5-TR
• [C1] Infections | Infectious Diseases
• [C04] Neoplasms
• [C05] Musculoskeletal Diseases
• [C06] Digestive System Diseases
• [C07] Stomatognathic Diseases
• [C08] Respiratory Tract Diseases
• [C09] Otorhinolaryngologic Diseases
• [C10] Nervous System Diseases
• [C11] Eye Diseases
• [C12] Urogenital Diseases
• [C14] Cardiovascular Diseases
• [C15] Hemic and Lymphatic Diseases
• [C16] Congenital, Hereditary, and Neonatal Diseases and Abnormalities
• [C17] Skin and Connective Tissue Diseases
• [C18] Nutritional and Metabolic Diseases
• [C19] Endocrine System Diseases
• [C20] Immune System Diseases
• [C21] Disorders of Environmental Origin
• [C22] Animal Diseases [Zoonotic diseases]
• [C23] Pathological Conditions, Signs and Symptoms
• [C24] Occupational Diseases
• [C25] Chemically-Induced Disorders
• [C26] Wounds and Injuries
• WHO Drug Information [Relative to Diseases]
• NDDK Patient Education Tool Kit
• Clinical Tools & Patient Education
• NDDK Resources on Medline Plus
• NDDK Open Research
• Dietary Guidelines for Americans
• Physical Activity Guidelines for Americans
• Move Your Way Community Resources
• National Youth Sports Strategy
• President’s Council on Sports, Fitness & Nutrition
• White House Conference on Hunger, Nutrition, and Health
• Equitable Long-Term Recovery and Resilience
• National Health Observances
• Finding Clinical Trials
• NIH News in Health
• Dosage Calculations & Pharmacology
• PICO - EBM Video This link opens in a new window
• PICO Slides
• Fillable CONSAH Pico Form
• Evidence Based Practice for Nursing
• Evidence-Based Nursing: -Step 2
• Evidence Appraisal - Step 3
• Evidence Application - Step 4
• Outcome Evaluation - Step 5
• Evidence Translation - Step 6
• Google Advanced Search for EBM
• Nursing Research Methods
• Faculty Book Request
• Proctor Request Form
• Peer Reviewed Literature: Assessment Goals
• Full Text Finder
• EBSCO e-Books Nursing 2021
• EBSCO eBooks Anesthesia
• EBSCO eBooks Radiology Science & Allied Health
• EBSCO eBooks: Writing About Nursing and Allied Health
• Alzheimers and Dementia
• Statistics on Aging
• CDC Bibliography: Alzheimers & Aging This link opens in a new window
• Health Conditions
• Health Behaviors
• Populations
• Settings and Systems
• Social Determinants of Health
• ILL Interlibrary Loan
• Gestational Diabetes and Fast Foods (MeSH)
• Mobile Resources
• Nursing Theory
• Psychiatric Nursing Journals
• Display, sort and; navigate
• Similar articles
• Cite, save and share
• Citations in PubMed
• All About PubMed Filters
• PubMed Quick Tours and Tutorials
• Evidence Based Practice Tutorial (PubMed) This link opens in a new window
• Developing a Clinical Question This link opens in a new window
• Using PubMed to Find Relevant Articles This link opens in a new window
• Next Steps This link opens in a new window
• Scenario (practice) This link opens in a new window
• Radiology Books and e-Books
• History of Radiology & Radiography
• Radiology: Anatomage This link opens in a new window
• Radiology Anatomy Atlas Viewer
• Advanced Radiographic Research
• Diagnostic Imaging Selected Articles
• Faculty and Administrative Resources
• Radiology Tech and MRI Salaries (Bureau of Labor Statistics)
• Radiology Technician Demand by State
• Review Tools for Graduate Students
• Training & Videos
• Register for an Online Meeting
• Joining an Online Meeting
• Training Videos & Search Examples
• Training Survey
• Sources for Health Statistics
• Ebola and Infectious Diseases
• Nursing Sites
• Writing Research Papers

Building a Summary Table or Synthesis Matrix

Quick Links | Nursing & Allied Health

• Library Home
• Campus Computer Labs
• Circulation Policies
• Catalog Search (books/ebooks)
• Contact Your Librarian For Research Assistance             (link sends email)
• Database Directory >> Nursing & Allied Health
• Faculty Materials Request Form
• Holidays & University Closures
• Interlibrary Loan Request Form
• Journals Full Text Finder
• LibGuides (Online Help)
• Review My Paper**
• Shreveport Proctor Request Form
• Shreveport Main Phone:  (318) 677-3007
• Shreveport Librarian Phone:  (318) 677-3013
• Student Help Desk :  Phone: (318) 357-6696  [email protected]
• The 20-20-20 Rule
• Training Videos

** Submit your research paper for grammar review

   GOOGLE MAP  |  LINKED

• Simplifying Synthesis | Download the Article PDF Copy
• Writing a Literature Review and Using a Synthesis Matrix

What a Summary Table or Synthesis Matrix looks like

Use the "Literature Review Matrix Template" as a guideline to help you sort through your thoughts, note important points and think through the similarities and differences: 

You are organizing the review by ideas and not by sources .  The literature review is not just a summary of the already published works.  Your synthesis should show how various articles are linked. 

A summary table is also called a synthesis matrix.  The table helps you organize and compare information for your systematic review, scholarly report, dissertation or thesis

Synthesis Matrix.

A summary table is also called a synthesis matrix . A summary table helps you record the main points of each source and document how sources relate to each other. After summarizing and evaluating your sources, arrange them in a matrix to help you see how they relate to each other, and apply to each of your themes or variables.

Faculty who typically guide students find it challenging to help students learn how to synthesize material (Blondy, Blakesless, Scheffer, Rubenfeld, Cronin, & Luster-Turner, 2016; Kearney, 2015) .  Writers  can easily summarize material but seem to struggle to adequately synthesize knowledge about their topic and express that in their writing. So, whether you are writing a student papers, dissertations, or scholarly report it is necessary to learn a few tips and tricks to organize your ideas.

Building a summary table and developing solid synthesis skills is important for nurses, nurse practitioners, and allied health researchers.  Quality evidence-based practice initiatives and nursing care and medicine are based on understanding and evaluating the resources and research available, identifying gaps, and building a strong foundation for future work.

Good synthesis is about putting the data gathered, references read, and literature analyzed together in a new way that shows connections and relationships. ( Shellenbarger, 2016 ). The Merriam-Webster dictionary defines synthesis as something that is made by combining different things or the composition or combination of parts or elements so as to form a whole (Synthesis, n.d.).  

In other words, building a summary table or synthesis matrix  involves taking information from a variety of sources, evaluating that information and forming new ideas or insights in an original way.  This can be a new and potentially challenging experience for students and researchers who are used to just repeating what is already in the literature.

Visit Our Libraries

   Interlibrary Loan    |    Shreveport Education Center Library   |    Eugene P. Watson Memorial Library    |     NSU Leesville Library 

Cammie G. Henry Research Center    |   Prince Music Media Library

• << Previous: Best Nursing Apps
• Next: Writing an Abstract for Your PILT or Special Project >>
• Last Updated: Jul 29, 2024 2:38 PM
• URL: https://libguides.nsula.edu/NursingandAlliedHealth

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

• An introductory paragraph that explains what your working topic and thesis is
• A forecast of key topics or texts that will appear in the review
• Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
• Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically Evaluate: Mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

• Summarize the key findings you have taken from the literature and emphasize their significance
• Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

• Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
• Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
• Qualitative versus quantitative research
• Empirical versus theoretical scholarship
• Divide the research by sociological, historical, or cultural sources
• Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

• It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
• Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
• Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
• Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

15 Literature Review Examples

Chris Drew (PhD)

Dr. Chris Drew is the founder of the Helpful Professor. He holds a PhD in education and has published over 20 articles in scholarly journals. He is the former editor of the Journal of Learning Development in Higher Education. [Image Descriptor: Photo of Chris]

Learn about our Editorial Process

Literature reviews are a necessary step in a research process and often required when writing your research proposal . They involve gathering, analyzing, and evaluating existing knowledge about a topic in order to find gaps in the literature where future studies will be needed.

Ideally, once you have completed your literature review, you will be able to identify how your research project can build upon and extend existing knowledge in your area of study.

Generally, for my undergraduate research students, I recommend a narrative review, where themes can be generated in order for the students to develop sufficient understanding of the topic so they can build upon the themes using unique methods or novel research questions.

If you’re in the process of writing a literature review, I have developed a literature review template for you to use – it’s a huge time-saver and walks you through how to write a literature review step-by-step:

Get your time-saving templates here to write your own literature review.

Literature Review Examples

For the following types of literature review, I present an explanation and overview of the type, followed by links to some real-life literature reviews on the topics.

1. Narrative Review Examples

Also known as a traditional literature review, the narrative review provides a broad overview of the studies done on a particular topic.

It often includes both qualitative and quantitative studies and may cover a wide range of years.

The narrative review’s purpose is to identify commonalities, gaps, and contradictions in the literature .

I recommend to my students that they should gather their studies together, take notes on each study, then try to group them by themes that form the basis for the review (see my step-by-step instructions at the end of the article).

Example Study

Title: Communication in healthcare: a narrative review of the literature and practical recommendations

Citation: Vermeir, P., Vandijck, D., Degroote, S., Peleman, R., Verhaeghe, R., Mortier, E., … & Vogelaers, D. (2015). Communication in healthcare: a narrative review of the literature and practical recommendations. International journal of clinical practice , 69 (11), 1257-1267.

Source: https://onlinelibrary.wiley.com/doi/pdf/10.1111/ijcp.12686  

Overview: This narrative review analyzed themes emerging from 69 articles about communication in healthcare contexts. Five key themes were found in the literature: poor communication can lead to various negative outcomes, discontinuity of care, compromise of patient safety, patient dissatisfaction, and inefficient use of resources. After presenting the key themes, the authors recommend that practitioners need to approach healthcare communication in a more structured way, such as by ensuring there is a clear understanding of who is in charge of ensuring effective communication in clinical settings.

Other Examples

• Burnout in United States Healthcare Professionals: A Narrative Review (Reith, 2018) – read here
• Examining the Presence, Consequences, and Reduction of Implicit Bias in Health Care: A Narrative Review (Zestcott, Blair & Stone, 2016) – read here
• A Narrative Review of School-Based Physical Activity for Enhancing Cognition and Learning (Mavilidi et al., 2018) – read here
• A narrative review on burnout experienced by medical students and residents (Dyrbye & Shanafelt, 2015) – read here

2. Systematic Review Examples

This type of literature review is more structured and rigorous than a narrative review. It involves a detailed and comprehensive plan and search strategy derived from a set of specified research questions.

The key way you’d know a systematic review compared to a narrative review is in the methodology: the systematic review will likely have a very clear criteria for how the studies were collected, and clear explanations of exclusion/inclusion criteria. 

The goal is to gather the maximum amount of valid literature on the topic, filter out invalid or low-quality reviews, and minimize bias. Ideally, this will provide more reliable findings, leading to higher-quality conclusions and recommendations for further research.

You may note from the examples below that the ‘method’ sections in systematic reviews tend to be much more explicit, often noting rigid inclusion/exclusion criteria and exact keywords used in searches.

Title: The importance of food naturalness for consumers: Results of a systematic review  

Citation: Roman, S., Sánchez-Siles, L. M., & Siegrist, M. (2017). The importance of food naturalness for consumers: Results of a systematic review. Trends in food science & technology , 67 , 44-57.

Source: https://www.sciencedirect.com/science/article/pii/S092422441730122X  

Overview: This systematic review included 72 studies of food naturalness to explore trends in the literature about its importance for consumers. Keywords used in the data search included: food, naturalness, natural content, and natural ingredients. Studies were included if they examined consumers’ preference for food naturalness and contained empirical data. The authors found that the literature lacks clarity about how naturalness is defined and measured, but also found that food consumption is significantly influenced by perceived naturalness of goods.

• A systematic review of research on online teaching and learning from 2009 to 2018 (Martin, Sun & Westine, 2020) – read here
• Where Is Current Research on Blockchain Technology? (Yli-Huumo et al., 2016) – read here
• Universities—industry collaboration: A systematic review (Ankrah & Al-Tabbaa, 2015) – read here
• Internet of Things Applications: A Systematic Review (Asghari, Rahmani & Javadi, 2019) – read here

3. Meta-analysis

This is a type of systematic review that uses statistical methods to combine and summarize the results of several studies.

Due to its robust methodology, a meta-analysis is often considered the ‘gold standard’ of secondary research , as it provides a more precise estimate of a treatment effect than any individual study contributing to the pooled analysis.

Furthermore, by aggregating data from a range of studies, a meta-analysis can identify patterns, disagreements, or other interesting relationships that may have been hidden in individual studies.

This helps to enhance the generalizability of findings, making the conclusions drawn from a meta-analysis particularly powerful and informative for policy and practice.

Title: Cholesterol and Alzheimer’s Disease Risk: A Meta-Meta-Analysis

Citation: Sáiz-Vazquez, O., Puente-Martínez, A., Ubillos-Landa, S., Pacheco-Bonrostro, J., & Santabárbara, J. (2020). Cholesterol and Alzheimer’s disease risk: a meta-meta-analysis. Brain sciences, 10(6), 386.

Source: https://doi.org/10.3390/brainsci10060386  

O verview: This study examines the relationship between cholesterol and Alzheimer’s disease (AD). Researchers conducted a systematic search of meta-analyses and reviewed several databases, collecting 100 primary studies and five meta-analyses to analyze the connection between cholesterol and Alzheimer’s disease. They find that the literature compellingly demonstrates that low-density lipoprotein cholesterol (LDL-C) levels significantly influence the development of Alzheimer’s disease.

• The power of feedback revisited: A meta-analysis of educational feedback research (Wisniewski, Zierer & Hattie, 2020) – read here
• How Much Does Education Improve Intelligence? A Meta-Analysis (Ritchie & Tucker-Drob, 2018) – read here
• A meta-analysis of factors related to recycling (Geiger et al., 2019) – read here
• Stress management interventions for police officers and recruits (Patterson, Chung & Swan, 2014) – read here

Other Types of Reviews

• Scoping Review: This type of review is used to map the key concepts underpinning a research area and the main sources and types of evidence available. It can be undertaken as stand-alone projects in their own right, or as a precursor to a systematic review.
• Rapid Review: This type of review accelerates the systematic review process in order to produce information in a timely manner. This is achieved by simplifying or omitting stages of the systematic review process.
• Integrative Review: This review method is more inclusive than others, allowing for the simultaneous inclusion of experimental and non-experimental research. The goal is to more comprehensively understand a particular phenomenon.
• Critical Review: This is similar to a narrative review but requires a robust understanding of both the subject and the existing literature. In a critical review, the reviewer not only summarizes the existing literature, but also evaluates its strengths and weaknesses. This is common in the social sciences and humanities .
• State-of-the-Art Review: This considers the current level of advancement in a field or topic and makes recommendations for future research directions. This type of review is common in technological and scientific fields but can be applied to any discipline.

How to Write a Narrative Review (Tips for Undergrad Students)

Most undergraduate students conducting a capstone research project will be writing narrative reviews. Below is a five-step process for conducting a simple review of the literature for your project.

• Search for Relevant Literature: Use scholarly databases related to your field of study, provided by your university library, along with appropriate search terms to identify key scholarly articles that have been published on your topic.
• Evaluate and Select Sources: Filter the source list by selecting studies that are directly relevant and of sufficient quality, considering factors like credibility , objectivity, accuracy, and validity.
• Analyze and Synthesize: Review each source and summarize the main arguments  in one paragraph (or more, for postgrad). Keep these summaries in a table.
• Identify Themes: With all studies summarized, group studies that share common themes, such as studies that have similar findings or methodologies.
• Write the Review: Write your review based upon the themes or subtopics you have identified. Give a thorough overview of each theme, integrating source data, and conclude with a summary of the current state of knowledge then suggestions for future research based upon your evaluation of what is lacking in the literature.

Literature reviews don’t have to be as scary as they seem. Yes, they are difficult and require a strong degree of comprehension of academic studies. But it can be feasibly done through following a structured approach to data collection and analysis. With my undergraduate research students (who tend to conduct small-scale qualitative studies ), I encourage them to conduct a narrative literature review whereby they can identify key themes in the literature. Within each theme, students can critique key studies and their strengths and limitations , in order to get a lay of the land and come to a point where they can identify ways to contribute new insights to the existing academic conversation on their topic.

Ankrah, S., & Omar, A. T. (2015). Universities–industry collaboration: A systematic review. Scandinavian Journal of Management, 31(3), 387-408.

Asghari, P., Rahmani, A. M., & Javadi, H. H. S. (2019). Internet of Things applications: A systematic review. Computer Networks , 148 , 241-261.

Dyrbye, L., & Shanafelt, T. (2016). A narrative review on burnout experienced by medical students and residents. Medical education , 50 (1), 132-149.

Geiger, J. L., Steg, L., Van Der Werff, E., & Ünal, A. B. (2019). A meta-analysis of factors related to recycling. Journal of environmental psychology , 64 , 78-97.

Martin, F., Sun, T., & Westine, C. D. (2020). A systematic review of research on online teaching and learning from 2009 to 2018. Computers & education , 159 , 104009.

Mavilidi, M. F., Ruiter, M., Schmidt, M., Okely, A. D., Loyens, S., Chandler, P., & Paas, F. (2018). A narrative review of school-based physical activity for enhancing cognition and learning: The importance of relevancy and integration. Frontiers in psychology , 2079.

Patterson, G. T., Chung, I. W., & Swan, P. W. (2014). Stress management interventions for police officers and recruits: A meta-analysis. Journal of experimental criminology , 10 , 487-513.

Reith, T. P. (2018). Burnout in United States healthcare professionals: a narrative review. Cureus , 10 (12).

Ritchie, S. J., & Tucker-Drob, E. M. (2018). How much does education improve intelligence? A meta-analysis. Psychological science , 29 (8), 1358-1369.

Roman, S., Sánchez-Siles, L. M., & Siegrist, M. (2017). The importance of food naturalness for consumers: Results of a systematic review. Trends in food science & technology , 67 , 44-57.

Sáiz-Vazquez, O., Puente-Martínez, A., Ubillos-Landa, S., Pacheco-Bonrostro, J., & Santabárbara, J. (2020). Cholesterol and Alzheimer’s disease risk: a meta-meta-analysis. Brain sciences, 10(6), 386.

Vermeir, P., Vandijck, D., Degroote, S., Peleman, R., Verhaeghe, R., Mortier, E., … & Vogelaers, D. (2015). Communication in healthcare: a narrative review of the literature and practical recommendations. International journal of clinical practice , 69 (11), 1257-1267.

Wisniewski, B., Zierer, K., & Hattie, J. (2020). The power of feedback revisited: A meta-analysis of educational feedback research. Frontiers in Psychology , 10 , 3087.

Yli-Huumo, J., Ko, D., Choi, S., Park, S., & Smolander, K. (2016). Where is current research on blockchain technology?—a systematic review. PloS one , 11 (10), e0163477.

Zestcott, C. A., Blair, I. V., & Stone, J. (2016). Examining the presence, consequences, and reduction of implicit bias in health care: a narrative review. Group Processes & Intergroup Relations , 19 (4), 528-542

• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 101 Hidden Talents Examples
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 15 Green Flags in a Relationship
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 15 Signs you're Burnt Out, Not Lazy
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 15 Toxic Things Parents Say to their Children

Leave a Comment Cancel Reply

Your email address will not be published. Required fields are marked *

• TemplateLab

Literature Review Templates

50 smart literature review templates (apa).

A literary review template is a type of written work that discusses published information about a specific subject matter. The length of the review doesn’t matter. It can be as simple as a summary of sources or can be as long as several pages. An outline for literature review can also evaluate these sources and advise to the readers regarding what’s relevant depending on certain conditions.

Table of Contents

• 1 Literature Review Templates
• 2 Why do you need a literature review template?
• 3 Literature Review Formats
• 4 Tips for creating a literature review template
• 5 Outlines For Literature Review
• 6 Compose the literature review
• 7 Strategies for composing your literature review template

Why do you need a literature review template?

A literary review template can serve as a guide about a specific topic. If you’re under time constraints to conduct more research, a literature review outline example can do you good as it provides you with an overview of what you intend to research on.

Even professionals of various fields rely on literary reviews to keep them updated in terms of what’s current in their fields. As for scholars, they can detect a writer’s credibility in a certain field by reading their literature review format. You can also use these works as a foundation for the investigation of a research paper.

Literature Review Formats

Tips for creating a literature review template

Literary review templates are surveys of scholarly sources on a specific subject matter. It gives a general summary of information relevant to a certain research problem or question. Here are the steps to follow when creating a literary review.

• Gather, assess, and choose the appropriate literature Before researching for literature for a review, you must have a topic that’s narrowly-defined. If you were to write a review for some research work or dissertation, you have to gather information related to the research problem or question. Having to understand the state of knowledge of your subject is the first step in creating your outline for literature review. Composing a literature review outline example for stand-alone research shouldn’t be that difficult. You only have to a good focus, then come up with a question that directs your search. This should be an answerable question without the need to generate or collect new data. Start the process by making a list of relevant keywords for the research topic in question. Based on the list, whenever you discover useful articles, check your list of references to find other relevant articles. During this process, you can identify any significant publications which didn’t show up when you performed a keyword search through recurring citations. It’s impossible for you to read all the available sources about a single topic. The best thing to do first is to read the abstract and determine if the articles are of any use. You have to do some evaluations on which of the sources are of value and relevant to the question. Also, make sure to only choose credible sources. Make it a point to read major theories and landmark studies in the field of your research. Logically, your scope of work depends upon the discipline and topic you have chosen. Make it a habit of writing down notes while you’re reading. Later on, you can incorporate these notes in your literature review format. Also, consider keeping track of the sources you have cited to avoid any consequential plagiarism cases. Making an annotated bibliography is a good suggestion. Include here a written paragraph for the summary and the analysis for each cited source. This can also be very helpful in reminding you about what you have read.
• Look for themes and connections When you start organizing a literature review format, you should identify the relationships between all of the sources that you have read. Based on what you have read and the notes you have taken, look for: Patterns and trends: Are there approaches which become less or more popular as time goes by? Themes: Identifying concepts or questions that repeat constantly across the different literature. Conflicts, contradictions, and debates: At what points do the sources disagree or agree? Pivotal publications: Identify any influential studies or theories which affected the direction of the field. Gaps: Try looking for answers for the following questions – What’s missing from the literature? Did you find any weaknesses that you need to address? The answers to these questions can help organize the structure of the literary review. If applicable, you can include how your research contributes to the existing knowledge.
• Plan the structure of your literature review template You can organize the whole body of your literary review through various approaches but at this point, you should already have an idea of the strategy you want to use even before writing your review. Depending on how long your review will be, you can use the following strategies: Chronological This is the simplest strategy where you map out the development of your topic over a period of time. If you use this approach, you should avoid merely listing or summarizing your sources chronologically. Make it a point to analyze the patterns, key events, and turning points that have influenced the direction of the field. If possible, give your own ideas about why and how certain developments came to happen. Thematic If you’ve discovered, in the course of your research, some recurring themes, you may organize your literary review into subsections which address the different parts of your topic. Methodology When you’re drawing your sources from various fields or disciplines which use different methods for research, you may end up with different conclusions and results. Perform analysis and try to compare these results that emerged from the different approaches. Theoretical In many cases, a literary review becomes the basis for theoretical frameworks. You may use this to talk about various definitions, theories, and models of important concepts. You may even argue about the significance of a theoretical approach or you can combine different theoretical ideas to come up with your own framework for research.

Outlines For Literature Review

Compose the literature review

The literary review isn’t any different in form from any other kind of academic texts as it also has the basic parts. What you included in each would depend upon your objective for writing the literature review:

• Introduction This part should clearly define the purpose and focus of the review. Dissertation: If you wrote the review as a part of a thesis or dissertation, you must reiterate the research question or central problem. Provide a short summary of the context as well. Stand-Alone: When writing this type of review, provide a short background regarding the topic along with its significance. Talk about the scope you plan to review along with your objective.
• Body If you have a lengthy review, it’s best to divide this part into sub-sections then come up with a subheading for each of them.
• Conclusion State in the conclusion, a summary of the key findings you have derived from the literature and emphasize their significance. Dissertation: Demonstrate how your research can address gaps and how it can contribute to gaining new knowledge. You can also discuss how you have used existing theories and methods to build a framework for your research. Stand-Alone: Discuss the overall effects of the literature or make suggestions for future research based on those gaps you have identified in your review.

Strategies for composing your literature review template

Most people write literature review templates in the sciences although occasionally, some are in the Humanities. In many experiments and laboratory reports, literature reviews constitute a section of the document. At times, some people write the review as the paper itself. Here are some strategies that may prove helpful when tasked to write a literature review:

• Find your focus Although they can differ, a literary review is like a term paper as you would organize both around ideas, not the sources themselves. This means that writing a review is not merely about listing your sources and going into their details one at a time. You also have to consider the themes and issues that connect your sources together.
• Convey your message Literary reviews may not have the usual thesis statement but you still need to inform your readers what to expect. Writing a simple statement is enough to let your readers know what your main organizing principle is.
• Organize the information There are instances when you may need additional sections in your review which are necessary for the study but don’t fit into the body’s organizational strategy. This depends on which sections you want to include. Only put in those that are necessary. To help you out, here are some sections that you may want to include in your review: Current Situation: This refers to the information that’s necessary to easily understand the focus or topic of the review. History: The chronological progression of the field, the literature, or an idea that’s necessary to understand the literature review. Include this only if the body isn’t already arranged chronologically. Methods or Standards: What criteria did you use to choose the sources in your review or the manner in which you want to present the information. Questions for Further Research: Are there questions about your field of research the review had sparked? Based on the review, what steps can you take to advance your research?

More Templates

Binder Cover Templates

Graph Paper Templates

Cover Page Templates

All About Me Templates

Essay Outline Templates

Table of Contents Templates

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• PLoS Comput Biol
• v.9(7); 2013 Jul

Ten Simple Rules for Writing a Literature Review

Marco pautasso.

1 Centre for Functional and Evolutionary Ecology (CEFE), CNRS, Montpellier, France

2 Centre for Biodiversity Synthesis and Analysis (CESAB), FRB, Aix-en-Provence, France

Literature reviews are in great demand in most scientific fields. Their need stems from the ever-increasing output of scientific publications [1] . For example, compared to 1991, in 2008 three, eight, and forty times more papers were indexed in Web of Science on malaria, obesity, and biodiversity, respectively [2] . Given such mountains of papers, scientists cannot be expected to examine in detail every single new paper relevant to their interests [3] . Thus, it is both advantageous and necessary to rely on regular summaries of the recent literature. Although recognition for scientists mainly comes from primary research, timely literature reviews can lead to new synthetic insights and are often widely read [4] . For such summaries to be useful, however, they need to be compiled in a professional way [5] .

When starting from scratch, reviewing the literature can require a titanic amount of work. That is why researchers who have spent their career working on a certain research issue are in a perfect position to review that literature. Some graduate schools are now offering courses in reviewing the literature, given that most research students start their project by producing an overview of what has already been done on their research issue [6] . However, it is likely that most scientists have not thought in detail about how to approach and carry out a literature review.

Reviewing the literature requires the ability to juggle multiple tasks, from finding and evaluating relevant material to synthesising information from various sources, from critical thinking to paraphrasing, evaluating, and citation skills [7] . In this contribution, I share ten simple rules I learned working on about 25 literature reviews as a PhD and postdoctoral student. Ideas and insights also come from discussions with coauthors and colleagues, as well as feedback from reviewers and editors.

Rule 1: Define a Topic and Audience

How to choose which topic to review? There are so many issues in contemporary science that you could spend a lifetime of attending conferences and reading the literature just pondering what to review. On the one hand, if you take several years to choose, several other people may have had the same idea in the meantime. On the other hand, only a well-considered topic is likely to lead to a brilliant literature review [8] . The topic must at least be:

• interesting to you (ideally, you should have come across a series of recent papers related to your line of work that call for a critical summary),
• an important aspect of the field (so that many readers will be interested in the review and there will be enough material to write it), and
• a well-defined issue (otherwise you could potentially include thousands of publications, which would make the review unhelpful).

Ideas for potential reviews may come from papers providing lists of key research questions to be answered [9] , but also from serendipitous moments during desultory reading and discussions. In addition to choosing your topic, you should also select a target audience. In many cases, the topic (e.g., web services in computational biology) will automatically define an audience (e.g., computational biologists), but that same topic may also be of interest to neighbouring fields (e.g., computer science, biology, etc.).

Rule 2: Search and Re-search the Literature

After having chosen your topic and audience, start by checking the literature and downloading relevant papers. Five pieces of advice here:

• keep track of the search items you use (so that your search can be replicated [10] ),
• keep a list of papers whose pdfs you cannot access immediately (so as to retrieve them later with alternative strategies),
• use a paper management system (e.g., Mendeley, Papers, Qiqqa, Sente),
• define early in the process some criteria for exclusion of irrelevant papers (these criteria can then be described in the review to help define its scope), and
• do not just look for research papers in the area you wish to review, but also seek previous reviews.

The chances are high that someone will already have published a literature review ( Figure 1 ), if not exactly on the issue you are planning to tackle, at least on a related topic. If there are already a few or several reviews of the literature on your issue, my advice is not to give up, but to carry on with your own literature review,

The bottom-right situation (many literature reviews but few research papers) is not just a theoretical situation; it applies, for example, to the study of the impacts of climate change on plant diseases, where there appear to be more literature reviews than research studies [33] .

• discussing in your review the approaches, limitations, and conclusions of past reviews,
• trying to find a new angle that has not been covered adequately in the previous reviews, and
• incorporating new material that has inevitably accumulated since their appearance.

When searching the literature for pertinent papers and reviews, the usual rules apply:

• be thorough,
• use different keywords and database sources (e.g., DBLP, Google Scholar, ISI Proceedings, JSTOR Search, Medline, Scopus, Web of Science), and
• look at who has cited past relevant papers and book chapters.

Rule 3: Take Notes While Reading

If you read the papers first, and only afterwards start writing the review, you will need a very good memory to remember who wrote what, and what your impressions and associations were while reading each single paper. My advice is, while reading, to start writing down interesting pieces of information, insights about how to organize the review, and thoughts on what to write. This way, by the time you have read the literature you selected, you will already have a rough draft of the review.

Of course, this draft will still need much rewriting, restructuring, and rethinking to obtain a text with a coherent argument [11] , but you will have avoided the danger posed by staring at a blank document. Be careful when taking notes to use quotation marks if you are provisionally copying verbatim from the literature. It is advisable then to reformulate such quotes with your own words in the final draft. It is important to be careful in noting the references already at this stage, so as to avoid misattributions. Using referencing software from the very beginning of your endeavour will save you time.

Rule 4: Choose the Type of Review You Wish to Write

After having taken notes while reading the literature, you will have a rough idea of the amount of material available for the review. This is probably a good time to decide whether to go for a mini- or a full review. Some journals are now favouring the publication of rather short reviews focusing on the last few years, with a limit on the number of words and citations. A mini-review is not necessarily a minor review: it may well attract more attention from busy readers, although it will inevitably simplify some issues and leave out some relevant material due to space limitations. A full review will have the advantage of more freedom to cover in detail the complexities of a particular scientific development, but may then be left in the pile of the very important papers “to be read” by readers with little time to spare for major monographs.

There is probably a continuum between mini- and full reviews. The same point applies to the dichotomy of descriptive vs. integrative reviews. While descriptive reviews focus on the methodology, findings, and interpretation of each reviewed study, integrative reviews attempt to find common ideas and concepts from the reviewed material [12] . A similar distinction exists between narrative and systematic reviews: while narrative reviews are qualitative, systematic reviews attempt to test a hypothesis based on the published evidence, which is gathered using a predefined protocol to reduce bias [13] , [14] . When systematic reviews analyse quantitative results in a quantitative way, they become meta-analyses. The choice between different review types will have to be made on a case-by-case basis, depending not just on the nature of the material found and the preferences of the target journal(s), but also on the time available to write the review and the number of coauthors [15] .

Rule 5: Keep the Review Focused, but Make It of Broad Interest

Whether your plan is to write a mini- or a full review, it is good advice to keep it focused 16 , 17 . Including material just for the sake of it can easily lead to reviews that are trying to do too many things at once. The need to keep a review focused can be problematic for interdisciplinary reviews, where the aim is to bridge the gap between fields [18] . If you are writing a review on, for example, how epidemiological approaches are used in modelling the spread of ideas, you may be inclined to include material from both parent fields, epidemiology and the study of cultural diffusion. This may be necessary to some extent, but in this case a focused review would only deal in detail with those studies at the interface between epidemiology and the spread of ideas.

While focus is an important feature of a successful review, this requirement has to be balanced with the need to make the review relevant to a broad audience. This square may be circled by discussing the wider implications of the reviewed topic for other disciplines.

Rule 6: Be Critical and Consistent

Reviewing the literature is not stamp collecting. A good review does not just summarize the literature, but discusses it critically, identifies methodological problems, and points out research gaps [19] . After having read a review of the literature, a reader should have a rough idea of:

• the major achievements in the reviewed field,
• the main areas of debate, and
• the outstanding research questions.

It is challenging to achieve a successful review on all these fronts. A solution can be to involve a set of complementary coauthors: some people are excellent at mapping what has been achieved, some others are very good at identifying dark clouds on the horizon, and some have instead a knack at predicting where solutions are going to come from. If your journal club has exactly this sort of team, then you should definitely write a review of the literature! In addition to critical thinking, a literature review needs consistency, for example in the choice of passive vs. active voice and present vs. past tense.

Rule 7: Find a Logical Structure

Like a well-baked cake, a good review has a number of telling features: it is worth the reader's time, timely, systematic, well written, focused, and critical. It also needs a good structure. With reviews, the usual subdivision of research papers into introduction, methods, results, and discussion does not work or is rarely used. However, a general introduction of the context and, toward the end, a recapitulation of the main points covered and take-home messages make sense also in the case of reviews. For systematic reviews, there is a trend towards including information about how the literature was searched (database, keywords, time limits) [20] .

How can you organize the flow of the main body of the review so that the reader will be drawn into and guided through it? It is generally helpful to draw a conceptual scheme of the review, e.g., with mind-mapping techniques. Such diagrams can help recognize a logical way to order and link the various sections of a review [21] . This is the case not just at the writing stage, but also for readers if the diagram is included in the review as a figure. A careful selection of diagrams and figures relevant to the reviewed topic can be very helpful to structure the text too [22] .

Rule 8: Make Use of Feedback

Reviews of the literature are normally peer-reviewed in the same way as research papers, and rightly so [23] . As a rule, incorporating feedback from reviewers greatly helps improve a review draft. Having read the review with a fresh mind, reviewers may spot inaccuracies, inconsistencies, and ambiguities that had not been noticed by the writers due to rereading the typescript too many times. It is however advisable to reread the draft one more time before submission, as a last-minute correction of typos, leaps, and muddled sentences may enable the reviewers to focus on providing advice on the content rather than the form.

Feedback is vital to writing a good review, and should be sought from a variety of colleagues, so as to obtain a diversity of views on the draft. This may lead in some cases to conflicting views on the merits of the paper, and on how to improve it, but such a situation is better than the absence of feedback. A diversity of feedback perspectives on a literature review can help identify where the consensus view stands in the landscape of the current scientific understanding of an issue [24] .

Rule 9: Include Your Own Relevant Research, but Be Objective

In many cases, reviewers of the literature will have published studies relevant to the review they are writing. This could create a conflict of interest: how can reviewers report objectively on their own work [25] ? Some scientists may be overly enthusiastic about what they have published, and thus risk giving too much importance to their own findings in the review. However, bias could also occur in the other direction: some scientists may be unduly dismissive of their own achievements, so that they will tend to downplay their contribution (if any) to a field when reviewing it.

In general, a review of the literature should neither be a public relations brochure nor an exercise in competitive self-denial. If a reviewer is up to the job of producing a well-organized and methodical review, which flows well and provides a service to the readership, then it should be possible to be objective in reviewing one's own relevant findings. In reviews written by multiple authors, this may be achieved by assigning the review of the results of a coauthor to different coauthors.

Rule 10: Be Up-to-Date, but Do Not Forget Older Studies

Given the progressive acceleration in the publication of scientific papers, today's reviews of the literature need awareness not just of the overall direction and achievements of a field of inquiry, but also of the latest studies, so as not to become out-of-date before they have been published. Ideally, a literature review should not identify as a major research gap an issue that has just been addressed in a series of papers in press (the same applies, of course, to older, overlooked studies (“sleeping beauties” [26] )). This implies that literature reviewers would do well to keep an eye on electronic lists of papers in press, given that it can take months before these appear in scientific databases. Some reviews declare that they have scanned the literature up to a certain point in time, but given that peer review can be a rather lengthy process, a full search for newly appeared literature at the revision stage may be worthwhile. Assessing the contribution of papers that have just appeared is particularly challenging, because there is little perspective with which to gauge their significance and impact on further research and society.

Inevitably, new papers on the reviewed topic (including independently written literature reviews) will appear from all quarters after the review has been published, so that there may soon be the need for an updated review. But this is the nature of science [27] – [32] . I wish everybody good luck with writing a review of the literature.

Acknowledgments

Many thanks to M. Barbosa, K. Dehnen-Schmutz, T. Döring, D. Fontaneto, M. Garbelotto, O. Holdenrieder, M. Jeger, D. Lonsdale, A. MacLeod, P. Mills, M. Moslonka-Lefebvre, G. Stancanelli, P. Weisberg, and X. Xu for insights and discussions, and to P. Bourne, T. Matoni, and D. Smith for helpful comments on a previous draft.

Funding Statement

This work was funded by the French Foundation for Research on Biodiversity (FRB) through its Centre for Synthesis and Analysis of Biodiversity data (CESAB), as part of the NETSEED research project. The funders had no role in the preparation of the manuscript.

• Open access
• Published: 24 August 2024

War and peace in public health education and training: a scoping review

• Lisa Wandschneider 1 , 2 ,
• Anna Nowak 1 , 2 ,
• Marta Miller 4 ,
• Anina Grün 1 , 2 ,
• Yudit Namer 2 , 5 ,
• Tomasz Bochenek 6 ,
• Lukasz Balwicki 7 ,
• Oliver Razum 1 , 2 &
• Colette Cunningham 3  

BMC Public Health volume  24 , Article number:  2303 ( 2024 ) Cite this article

137 Accesses

Metrics details

Armed conflict and war are public health disasters. Public health action has a crucial role in conflict-related emergencies and rehabilitation but also in war prevention and peace promotion. Translating this into public health training and competencies has just started to emerge, especially in Europe.

We conducted a Scoping Review to map and identify the role of public health education and training of public health workforce relating to the prevention of war and promoting peace, as reflected in the scientific literature. We searched in PubMed, CINAHL, PsycINFO, Embase, Web of Science Core Collections as well as the reference list of included material in English, German and Polish. Focusing initially on the European region, we later expanded the search outside of Europe.

We included 7 publications from opinion pieces to an empirical assessment of curricula and training. The educational programs were predominantly short-term and extra-curricular in postgraduate courses addressing both public health professionals in conflict-affected countries as well as countries not directly affected by war. Publications focused on public health action in times of war, without specifying the context and type of war or armed conflict. Competencies taught focused on emergency response and multi-disciplinary collaboration during emergencies, frequently drawing on experience and examples from natural disaster and disease outbreak management.

Conclusions

The scientific discourse on competences in public health education for times of war and for the promotion of peace, predominately focuses on immediate emergency response actions. The prevention of war and the promotion of peace are missing foci, that need to feature more prominently in public health training. Public Health Education and training should ensure that war prevention and peace promotion, as well as public health action in times of war, are included in their competencies for public health professionals.

Peer Review reports

Introduction

War has profound adverse effects on public health [ 1 ].

War and armed conflict have far-reaching consequences, affecting the lives of millions of people, resulting in the loss of human life, but also in the disruption of social infrastructure such as safe food and water supplies, housing, and access to health services, leading to increased mortality and morbidity. War-torn countries and populations experience an increase in Disability Adjusted Life Years (DALYs) lost [ 2 ]. In addition, mass displacement, which disproportionately affects women, children, the elderly and vulnerable groups, exposes people to precarious living conditions and arduous travel, affecting their health and well-being. War and armed conflict have profound long-term physical and psychological consequences for those involved. They destroy communities and the fabric of society is often irreversibly undermined. This affects public health as well as legalizing and promoting violence as a way of solving problems [ 2 ]. In this paper, war and armed conflict is defined as ‘hostile contention by means of armed forces, carried on between nations, states, or rulers, or between parties in the same nation or state; the employment of armed forces against a foreign power, or against an opposing party in the state [ 3 ]. The International Committee of Red Cross (ICRC) casebook differentiates between an international armed conflict which “occurs when one or more States have recourse to armed force against another State, regardless of the reasons or the intensity of this confrontation” and a non-international armed conflict in which one or more non-State armed groups are involved (the vast majority of conflicts since 1945) [ 4 ]. Other authors differentiate wars by causes or intentions, for example imperial wars [ 5 ] or wars of annexation, such as Russia’s invasion of Ukraine. The reason is that preventive measures may be quite different. Much of the literature such as Levy et al. [ 1 , 6 ] covers armed conflict but not wars of annexation [ 2 , 7 ]. Today’s wars are often hybrid, meaning they are being fought by military as well as by destabilizing, non-military means [ 8 ]. Debates on preventing wars of annexation need to cover the role of the military, and the way in which public health relates to it, but also political determinants. For reasons of readability, however, the term ‘war’ will be used consistently throughout the text to include armed conflict and wars of annexation, as well as hybrid elements of warfare.

Conceptualizing war in public health is a relatively recent development, while international relations and defense studies have a longstanding tradition of analyzing and theorizing war and peace (e.g. [ 5 , 9 , 10 ]). Public health approaches focusing primarily on emergency response and relief care [ 11 , 12 , 13 ]. However, there is a growing shift in public health to emphasize its preventive role in peacetime, with the aim of minimizing the health impact of war on affected populations [ 14 , 15 , 16 ]. At the same time, one of public health’s major aims – reducing and dismantling (health) inequities — is also a crucial determinant for preventing armed conflict [ 17 , 18 , 19 ] presenting another valuable angle for public health’s role in peacebuilding and preventing war. This shift is consistent with modern public health care, given the emergence of complex global public health crises such as climate change, biodiversity loss, migration, cybersecurity, inequities, and pandemics such as COVID-19 [ 20 ]. We are looking at syndemics, whereby a set of linked health problems interact synergistically and contribute to the excess burden of disease in a population [ 21 ]. The preventative role provides an opportunity to develop a comprehensive public health framework that can effectively address the needs during war, its prevention, and post-conflict periods. Hagopian and Jabbour (2022) [ 22 ] proposed such a framework, using the Primary, Secondary and Tertiary Prevention (PSTP) Framework to address global inequalities and injustices that may contribute to war. It includes primary prevention, which focuses on addressing the root causes and social determinants of war; secondary prevention, which aims to reduce harm and damage during conflict; and tertiary prevention, which involves rebuilding health systems in post-conflict settings [ 22 ]. Wars of annexation have more to do with (re-)building imperia, rather than with inequalities and injustices. Nonetheless, there is no agreed-upon theory on the cause(s) of war [ 10 ].

While the role of public health in emergencies is increasingly well explored in public health research and in interdisciplinary collaboration, the translation of this knowledge into public health training and competencies is only just beginning to emerge, particularly in Europe. The literature is increasingly recommending that public health education and training programs should include an understanding of the health impacts of war and conflict, including the epidemiology of war-related injuries and illnesses, environmental impacts of war, and mental health effects of war on soldiers and civilians [ 2 , 23 , 24 , 25 ]. Additionally, public health professionals should be trained to respond to the health impacts of war, including strategies for preventing and treating war-related injuries and illnesses, as well as promoting peace and non-violent conflict resolution [ 2 , 23 , 24 , 25 ]. ,.

In light of contemporary conflicts and geopolitical tensions, such as the war in Ukraine, as well as those in Syria, Yemen, Myanmar, Israel and Gaza, and other regions, training on effective and systematic public health practice to support affected populations is needed. The role of public health in both the prevention of war and the promotion of peace work is increasingly recognized and calls for its inclusion in public health education, and in the training of public health professionals [ 7 , 25 , 26 ]. Considering the present circumstances, there is a need for public health professionals to develop a skill set that enables them to address the challenges of war, prevention, as well as peacebuilding, so that they can confidently navigate an increasingly multi-disciplinary role and take an active place in the dialogue on the prevention of war and its consequences. We conducted a Scoping Review of the scientific literature in public health to map available evidence and discourse on war. We covered public health perspectives on war prevention, reaction to war, rehabilitation, and peace promotion within public health education and training. More specifically we aimed to answer the following research questions:

How is public health education addressing competencies related to war and peacebuilding? Are there existing examples of teaching and can we build on them for future training?

What are the gaps in public health education on war and how can we address them?

Are there existent frameworks which can be used or adapted to develop public health education programs focused on war, war prevention, and peace promotion?

The research team, representing expertise from schools of public health across Europe, developed the search strategy with the help of the Population, Comparison and Context (PCC) Framework [ 27 ]. We opted for a Scoping Review methodology, since it allows for ‘a preliminary assessment of potential size and scope of available research literature’ as well as ‘identify the nature and extent of research evidence’ [ 28 , 29 ]. It enabled us to map a diverse range of evidence e.g., implementation research on training programs as well as commentaries and opinion pieces, as indicators of an on-going discourse within the public health community. War and peacebuilding in public health education represent an under-researched area, therefore our Scoping Review allowed for the capture and extent, as well as the type of available evidence. Our methodology was developed using the PRISMA-ScR Checklist to comply with reporting and methodological standards (supplementary material 1) [ 30 ]. A review protocol was not published, but the authors used an internal methods protocol which has been updated throughout the process (Supplementary material 3).

Search strategy

We searched in academic databases PubMed, CINAHL, PsycINFO, Embase, Web of Science Core Collections (using keywords and MeSH terms) to explore the scientific discourse on war and peacebuilding in public health education. Exploring grey literature, such as the extent of material at the level of Schools of Public Health exceeded the scope of this review and requires additional data collection tools. As related fields like disaster management and preparedness in public health are well represented within the scientific discourse, limiting this first mapping to scientific databases only, allows for a direct comparison and therefore seemed reasonable.

In our review, we defined the population as the body of interest, i.e., education and teaching body. Our search terms were chosen to identify literature that focused on public health education and training on war, war prevention and peace promotion. This method allowed the authors to capture examples of a broad range of education programs and training, both from individual courses to curricula. The concept element represents the thematic focus on war and peacebuilding. We aimed to integrate different stages of war e.g., armed conflict, active war and peacebuilding. In addition, we linked these search terms with closely-related fields, such as disaster management and preparedness, especially as these are competencies that are frequently linked to the context of war [ 31 ]. These terms have been informed by conflict-related health research as well as conceptual models systematizing the impact of war and peace promotion on public health [ 11 , 12 , 13 , 15 , 16 ]. For the context, being the third and last element of the PCC Framework, we used public health as the discipline and added global health since this is a field where the topic of war is frequently documented and addressed.

An experienced librarian supported the development of the search strategy (Table  1 , search protocols for all databases can be found in supplementary material 2). We conducted the searches on 8th September 2022 and extracted the records into the reference manager Zotero. In addition to the search in scientific databases, we checked the reference lists of included sources for further records that could warrant inclusion.

Eligibility criteria and study selection

In our review, we only included sources that had a primary interest in war and peacebuilding in the public health education context. This included publications that identified curricula, workshops, competencies, skills-sets and capacity-building trainings. For the war and peacebuilding element, this required an explicit definition of war and/or peacebuilding as the field of action or interest. We included academic literature, ranging from peer-reviewed articles to commentaries and editorials to capture a broad spectrum of the scientific discourse. We did not exclude any publication based on study design or period of the study.

We excluded sources that mentioned war or peacebuilding but did not elaborate further on how the courses addressed the specific needs or circumstances. Also, any material that solely addressed terrorist attacks was considered ineligible despite representing a potential weapon or strategy of war. Terrorist attacks were considered ineligible because they pertained to one, timely limited event that usually did not destroy the infrastructure of a complete region or country. Other competencies and frameworks for public health professionals is therefore warranted. The scoping review languages of the potential sources was limited to English, German, Polish.

For the screening process, we conducted a pilot with a random sample of 5% of the total records. This ensured inter-rater reliability between the reviewers by detecting inconsistencies and allowed us to adapt the eligibility criteria accordingly. Then, the reviewers (LW, MM, CC) started with the title and abstract screening. For the full texts, the reviewers changed (LW, MM, AG, AN), so we again conducted a pilot of a random 5% sample of the records identified in the abstract and title screening. Disagreements on the eligibility of full texts were discussed with all reviewers (LW, MM, AG, AN, CC) and resolved through discussion.

Data charting and analysis

The data charting and extraction focused on the context of war and the education/teaching element of the material, rather than the study details. The data charting table includes basic study characteristics, information on the context of war and peacebuilding as well as the teaching intervention ( Table  2 ). The standardized form guided the data charting process and was also tested in another pilot round between the reviewers. LW, MM, AG and AN extracted the data and modified the charting form in an iterative process, which was then discussed with the core reviewer team LW, MM, AG, AN and CC.

The data charting form also guided the descriptive and narrative synthesis of the findings. To structure the narrative analysis, we categorized the findings by the phase of war differentiating between a) preparedness and prevention, b) ongoing conflict and emergency, as well as c) recovery and rehabilitation based on Hagopian and Jabbour’s framework [ 22 ].

The initial literature search resulted in a total of 4922 citations (Fig.  1 ). After removing duplicates, a total of 2913 articles were screened. In the subsequent stages of this process, a total of 272 full‐text reports were assessed for eligibility, of which 264 studies were excluded because the focus was not on war or peacebuilding, or not on public health education/training.

Flowchart on the process identifying studies. This Study Flow Chart details the flow of information throughout the distinct phases of the review: identification, screening and included studies for final review

A total of seven articles reported on existing education/training or recommendation for training fulfilling the inclusion criteria (Fig.  1 ) [ 20 , 32 , 33 , 34 , 35 , 36 , 37 ]. We classified the study design of the 7 articles into three groups: 4 were reports [ 32 , 33 , 34 , 36 ]; two were commentaries [ 20 , 37 ]; and one was an empirical qualitative study [ 35 ].

Study characteristics

The 7 articles were published between 1998 and 2019, with most of them produced in the Global North, including the US ( n  = 3) [ 20 , 32 , 35 ], Croatia and Bosnia-Herzegovina ( n  = 1) [ 33 ] and Sweden ( n  = 1) [ 34 ]. Only one was conducted in the Global South, the empirical study from South Sudan [ 37 ]. The included publications take different phases of war into account, some take on several within one study: 4 publications reported on early warning/prevention and preparedness [ 33 , 34 , 35 , 36 ]; 4 publications on conflict situations [ 20 , 33 , 35 , 37 ]; 3 on emergency [ 20 , 32 , 37 ]; and one study on recovery / rehabilitation [ 35 ]. Four publications related to specific armed conflicts and time periods; (Balkan (1994–2001) [ 32 ], Sweden (time period not specified) [ 34 ], South Sudan (2013) [ 37 ], and after the First World War (1920–1939) [ 36 ]. Three studies did not specify the geographical region or period of the study. The characteristics of each study are shown in Table  3 , depicting the type of publication, type of war, period, geographic setting, and phase of military operations. Table 3 provides an overview of the teaching interventions recommended or described by the included studies.

Prevention and Preparedness

Four ( n  = 4) publications reported or commented on early warning, prevention and preparedness for war or war-like situations on a professional level. McDonnell et al. recommend that applied epidemiologists should be prepared for war by acquiring knowledge on international law, human rights, and complex interventions by working on specific case studies during training to be able to conduct assessments of the conflict setting and to communicate health-related interventions effectively with stakeholders and the local population [ 35 ]. Joshi strengthens this approach to take into account the psychological burden of mental health professionals working with war-affected children. He recommends analyzing the situation properly, gaining knowledge about the region, culture, and people, but also reflecting on one’s own limitations and resources [ 33 ]. Both commentaries point to the importance of (interdisciplinary) collaboration and training [ 33 , 35 ].

Kulling & Holst and McGann have shown what a training module could look like. While Kulling & Holst refer to the Swedish context, McGann describes in a historical analysis what the training of public health nurses looked like after the First World War [ 34 , 36 ]. On a national level, according to Kulling & Holst different topics should be addressed in regional and local courses for health professionals, such as disaster medicine, management of the healthcare system in a disaster/crisis, command and control at an accident site, chemical accidents, decontamination methods, radiation accidents, microbiological preparedness / bioterrorism, psychiatric / psycho-social support and planning preparedness for chemical, biological, or nuclear/radiological (C B N R) events on a national level [ 34 ]. McGann describes that the training of public health nurses in the 1920s consisted of both a theoretical and a practical part [ 36 ]. In the theoretical part there are already overlaps with the modules recommended by Kulling & Holst [ 34 ]. McGann shows that lectures were given on public health nursing, hygiene, bacteriology, psychology, social conditions, and social administration [ 36 ]. The practical part consisted of work placements in nursing-related fields, such as child welfare centers, TB dispensaries or in a district nursing association [ 36 ].

Conflict and emergency

Burkle et al. and McDonnell et al. refer to the importance of combining medical skills and knowledge in emergency and crisis situations [ 20 , 35 ]. Both reports discuss advanced courses for the development of specific skills for work in humanitarian settings. According to the authors, knowledge of legal frameworks, communication skills, documentation of human rights violations, and the design and management of needs-based health services are necessary skills for public health professionals, with McDonnell et al. also emphasizing competence in qualitative and quantiative data [ 20 , 35 ]. Burkle et al. recommend developing an all-encompassing international program which should be continously evaluated and adapted according to emergency sitations [ 20 ]. Joshi adds the importance of compentency in interdisciplinary cooperation and training of local populations [ 33 ].

Evans et al. and Rathner & Katona describe specific training programs in more detail [ 32 , 37 ]. While Evans et al. describe a program for graduated professionals and mid-careers public health leaders [ 32 ], Ratner & Katona’s training program focuses not only on public health professionals but also on the general public [ 37 ]. Both programs consist of lectures and hands-on-activities or practical work. Evans et al. describe a graduate program based at the Center for Humanitarian Emergencies at Emory University in Atlanta, US. The program covers topics on emergency preparedness, logistics, mental health, needs assessment, nutrition, and risk communication through lectures and group discussions, followed by a field practica at in-country host institutions. The field practica cover areas such as emergency management or global health security [ 32 ]. Ratner & Katona’s program is set in South Sudan and involves first aid courses, providing participants with the skills and knowledge to care for themselves and others in times of medical emergencies. The training brings together people from different tribes or community groups and focuses on specific medical needs. Supported by the local community and local leaders, the training leads to intergroup communication, stigma reduction, and health-related collaboration between different population groups. It not only provided essential healthcare skills, but also served as a platform for peacebuilding and community-building [ 37 ].

Recovery, rehabilitation and peacebuilding

All authors understand their programs as preparation for war-like situations but only two focus specifically on peacebuilding [ 35 , 37 ]. Ratner & Katona’s teaching activity in South Sudan lead to peacebuilding, communication, and interaction between different groups in the local communities [ 37 ]. McDonald et al. emphasize conflict assessment for peacebuilding, using quantitative and qualitative methods and effective communication skills for policy changes and interdisciplinary and interinstitutional cooperation. According to the authors, knowledge about predictors of violent conflict is necessary [ 35 ]. None of the studies in our review pertain to recovery or rehabilitation-related competencies or training programs.

We identified 7 publications dealing with education and/or training for public health professionals that related to war and peacebuilding [ 20 , 32 , 33 , 34 , 35 , 36 , 37 ]. Most of the publications covered public health training from prevention and preparedness, conflict and emergency to recovery, rehabilitation and peacebuilding. Literature is scarce regarding public health education in the context of war, armed conflict and peace promotion. We found a broad range of different manuscripts including commentaries and opinion pieces on different types of war and armed conflict, but only 7 publications met the review’s inclusion criteria. We did not identify research studies comparing different teaching methods, training modules or evaluating programs. However, there is a growing awareness of the topic and not least since Russia’s invasion of Ukraine (e.g. [ 19 , 38 , 39 , 40 ].). Nevertheless, more in-depth research needs to be done in this area.

The included publications are heterogeneous in terms of population, time, and war phases. Most of the programs described in the publications are aimed at public health professionals. Three studies ( n  = 3) focused on specific professional groups (e.g. psychologists, surgeons, nurses) [ 20 , 33 , 36 ]. Only Evans et al. targeted undergraduate and postgraduate students [ 32 ]. Just one publication included the local population in their education program and, in contrast to the other publications the context of the education program was within an emergency situation whereby there was an acute need for action due to the war-like situation in South Sudan [ 37 ]. Two studies described the educational program in more detail: While Kulling & Holst presented a current program in Sweden [ 34 ], McGann took a historical perspective and described the education of public health nurses between the First and Second World War [ 36 ]. Both studies pointed to similar teaching content for preparedness of professionals. Two other studies addressed general principals such as knowledge on human rights and complex interventions or self-reflection [ 33 , 35 ]. None of these publications used a conceptual educational framework.

We note from our scoping review that teaching war in public health education programs or in the training of public health professionals is predominately short-term and extra-curricular in post-graduate courses. A better understanding is needed of the intersections between war and health and of the indispensable role public health practitioners, academics and advocates could play particularly given the increasing significance of war as a determinant affecting population health [ 31 ].

The immediate emergency response in times of war was the main area of action in and for public health education. Some of the studies drew from or were also closely entangled with emergency management and/or disaster management (without any specific reference to war or peacebuilding). This finding is not surprising, given the relevance of this much more advanced and established field of public health practice. Emergency response and management, including infectious disease outbreaks and/or disasters especially natural disasters, are widely included in international standards and recommendations. For example, US [ 41 ] for public health education [ 42 ]. These topics are also frequently and systematically implemented in dedicated public health degrees [ 43 , 44 , 45 , 46 ]. These competencies play a major role for public response in the context of war, e.g. for first response, multidisciplinary coordination and crisis situation [ 44 ]. However, we also found that emergency management and public health education on war were often entangled, which led to a high number of full-text screenings. Few of the studies and commentaries explicitly differentiated between natural disasters or outbreaks and war/armed conflicts; yet wars require additional skillsets and competencies. For example, conflicts frequently result in waves of trauma cases and public health hazards depending on conflict intensity. Also, infrastructure can be repeatedly destroyed or supplies to rebuild cannot reach communities in need – again depending on conflict activities. Accordingly, we encourage studies and commentaries on public health practice and analysis to more explicitly differentiate between natural disasters and armed conflicts [ 6 ].

Highlighting the different disciplines involved as well as the range of competencies required in different phases of war and peacebuilding, our review reiterates the importance of interdisciplinary collaboration for developing and implementing public health education on war and peacebuilding. Building on this evidence, the growing conceptual understanding [ 6 , 14 , 16 , 22 ] and existing content-analyses of war-related public health education [ 20 , 31 ] will help to systematically advance public health training as well as the scientific discourse on this topic to support evidence-based decision-making for curriculum adaptations, teaching methods as well as adaptation for peacebuilding and in times of war.

Peacebuilding and war prevention were less discussed in the publications included in this review compared to other phases of war. Whoerle et al. suggest that health education can serve as a potential platform for integrating peace education into school curricula. The integration of health and peace involves four key approaches that could be translated into competencies: adopting a socio-ecological perspective; employing complexity thinking and problem mapping, recognizing the continuum of resilience and trauma, and considering the community as a site for practical implementation, calling for interdisciplinary cooperation [ 16 ]. Barry S. Levy, one of the long-standing experts in this field, proposes that citizens should confront the powerful in their country [ 6 ]. This is important advice in democratic societies, and there are precedents of successful protest and civic disobedience such as Daniel Ellsberg’s activism against the Vietnam war, which gave a boost to the US anti-war movement [ 47 ]. We reiterate this call and encourage to focus in on the competencies in public health required to specifically include peace promotion and war prevention in addition to the emergency war response. This is particularly important in an ‘era of geopolitical uncertainty’ [ 48 ] where peace and war play a dominant role and should be reflected as determinants of health and accordingly systematically addressed in public health education [ 49 ]. However, we also realize that advice such as Levy’s is tailored more at civil wars and wars attempting regime change, rather than wars of annexation. Russia’s full-scale invasion of Ukraine in 2022, was the first such event in Europe since World War II. Internal civic action, as recommended by Levy, carries grave personal risk when applied against the Putin regime. Diplomacy has not been successful as Russia, in the eyes of many observers, violates international treaties and security assurances [ 50 ]. In consequence, neighboring states may have to rely on a sufficiently funded military to protect their populations. This conclusion may come counter-intuitive to public health proponents [ 51 ], while in defense theories and international law and relation studies it is widely discussed and elaborated on [ 52 , 53 ]. Again, drawing from interdisciplinary collaboration could ensure deeper analytical and theoretical understandings of the terminology or concepts used in war and peacebuilding and comparability across fields. Moreover, given the contentious nature of these issues, students and teaching institutions should learn how to discuss conflicts constructively and fairly, avoiding the escalations of recent campus discourses in the Israel-Gaza conflict.

Strengths and limitations

In this Scoping Review, we analyzed the scope and extent of scientific discourse on war and peace promotion in public health education. Using a scoping review methodology allowed for a systematized and comprehensive mapping, which has been the first of its kind on the topic, at least to our knowledge. We included studies solely with a primary interest in war and peacebuilding in the public health education context. This allowed for a nuanced analysis of education programs, initiatives, or similar, which is of particular value for the overall aim of advancing public health education.

We thereby contribute to the identification of gaps and addressing them in an important area of public health work. Specifically, we have identified gaps in the European literature arena, where the topic of war and peace promotion has been mostly neglected over the past decades. In addition, we have synthesized war and peace promotion in public health education, which are usually assessed in parallel.

We limited our review to scientific databases aiming to assess the scientific discourse on war and peacebuilding in public health education and training. As a result, the review has not captured any grey literature relating to the topic; it exceeded the scope of this review. Nonetheless, it would be an important next step to assess the status quo at institutions of public health education and training within the European region. This includes Schools of Public Health and Higher Institutes or Centers for Public Health Education, many of whom lead to a graduate degree in public health and which is accredited by a recognized body, or bodies approved for such purpose. This could include for example the Secretary of Education in each European Region country or other authorizing bodies. Of note, a study in this regard, has been undertaken for specific schools of public health in the United States of America [ 31 ]. Such a study in Europe would allow for a more detailed overview of resources and expertise to advance and support the development of initiatives integrate war more systematically into public health education programs.

Since we only included studies with a primary interest in war and peacebuilding in the public health education context, we did not consider the scientific discussion touching on this topic. This was reflected many of full texts screened that ended with just a few publications with findings that could be generalized. The variability in the studies in terms of the type of war described, e.g., armed conflict or war of annexation, may present a challenge in drawing generalizable conclusions.

This scoping review identified a lack of scientific discourse on the role of war and peace promotion in public health education. The few public health education and training programs identified primarily focus on the immediate emergency response in conflict-affected areas. These were often entangled with emergency preparedness in different contexts such as infectious disease outbreaks. Peace promotion and war prevention are missing foci. This suggests a need for a far greater emphasis of the topic in public health training as well as its inclusion in competency frameworks. In addition to training public health professionals for emergencies, training with an emphasis on war prevention and promoting peace should be developed and implemented.

Availability of data and materials

All data generated or analyzed during this study are included in this published article [and its supplementary information files].

Levy BS, Sidel VW. War and Public Health. Oxford: Oxford University Press; 2008.

Book   Google Scholar  

Kerridge BT, Khan MR, Sapkota A. Terrorism, civil war, one-sided violence and global burden of disease. Med Confl Surviv. 2012;28(3):199–218.

Article   PubMed   Google Scholar  

Oxford Learner's Dictionaries. war noun - Definition, pictures, pronunciation and usage notes. In: Oxford Advanced Learner’s Dictionary, Oxford; 2024. https://www.oxfordlearnersdictionaries.com/definition/english/war?q=war . Accessed 20 Aug 2024.

How does law protect in war? The "Online Casebook" [ https://casebook.icrc.org ]

English R: Modern War: A Very Short Introduction: Oxford University Press; 2013.

Levy BS: From Horror to Hope: Oxford University Press; 2022.

Wandschneider L, Namer Y, Davidovitch N, Nitzan D, Otok R, Leighton L, Signorelli C, Middleton J, Martin-Moreno JM, Chambaud L, et al. The role of Europe’s Schools of public health in times of war: ASPHER statement on the war against Ukraine. Public Health Rev. 2022;43:1604880.

Article   PubMed   PubMed Central   Google Scholar  

Khorram-Manesh A, Goniewicz K, Burkle FM. Social and healthcare impacts of the Russian-led hybrid war in Ukraine – a conflict with unique global consequences. Disaster Med Public Health Prep. 2023;17.

Galbreath DJ, Deni JR. Routledge Handbook Of Defence Studies. London; New York; London: Routledge Taylor & Francis Group Routledge; 2018.

Mann M. On Wars. New Haven: Yale University Press; 2023.

Abouzeid M, Elzalabany MK, Nuwayhid I, Jabbour S. Conflict-related health research in Syria, 2011–2019: a scoping review for The Lancet - AUB Commission on Syria. Confl Heal. 2021;15(1):92.

Article   Google Scholar  

Iqbal Z: War and Health: A Conceptual and Theoretical Framework. In: War and the Health of Nations. edn. Edited by Iqbal Z: Stanford University Press; 2010: 30–52.

De Jong JTVM. A public health framework to translate risk factors related to political violence and war into multi-level preventive interventions. Soc Sci Med. 2010;70(1):71–9.

Santa Barbara J, MacQueen G. Peace through health: key concepts. Lancet (London, England). 2004;364(9431):384–6.

Wiist WH, Barker K, Arya N, Rohde J, Donohoe M, White S, Lubens P, Gorman G, Hagopian A. The role of public health in the prevention of war: rationale and competencies. Am J Public Health. 2014;104(6):e34–47.

Woehrle LM. Connecting health and peacebuilding in theory and practice. Peace Rev. 2019;31(2):168–77.

Bahgat K, Gray B, Kendra D, Scott G, Solveig H, Håvard Mokleiv N, et al. Inequality and armed conflict: evidence and policy recommendations. In: Conflict Trends. PRIO Oslo; 2017. https://cdn.cloud.prio.org/files/654a94b6-62c7-4d29-8b02-1bfab0598158/Bahgat%20et%20al%20-%20Inequality%20and%20Armed%20Conflict%20Conflict%20Trends%201-2017.pdf?inline=true . Accessed 20 Aug 2024.

Baldwin C, Chapman C, Gray Z. Minority rights: the key to conflict prevention. In . Minority Rights Group International; 2007. https://minorityrights.org/resources/minority-rights-the-key-to-conflict-prevention/ . Accessed 20 Aug 2024.

Percival V, Thoms OT, Oppenheim B, Rowlands D, Chisadza C, Fewer S, Yamey G, Alexander AC, Allaham CL, Causevic S, et al. The lancet commission on peaceful societies through health equity and gender equality. Lancet (London, England). 2023;402(10413):1661–722.

Burkle FM. Challenges of Global Public Health Emergencies: Development of a Health-Crisis Management Framework. Tohoku J Exp Med. 2019;249(1):33–41.

Lancet T. Syndemics: health in context. Lancet (London, England). 2017;389(10072):881.

Hagopian A, Jabbour S. Global public health workers and academics must step up response to war. Am J Public Health. 2022;112(6):814.

AlKhaldi M, Kaloti R, Shella D, Al Basuoni A, Meghari H. Health system’s response to the COVID-19 pandemic in conflict settings: policy reflections from Palestine. Glob Public Health. 2020;15(8):1244–56.

Bdaiwi Y, Sabouni A, Patel P, Ekzayez A, Alchalati S, Abdrabbuh O, Abbara A, Glogowska M. Impact of armed conflict on health professionals’ education and training in Syria: a systematic review. BMJ Open. 2023;13(7):e064851.

Tulchinsky T, Jennings B, Viehbeck S. Integrating ethics in public health education: the process of developing case studies. Public Health Rev. 2015;36:4.

Namer Y, Wandschneider L, Middleton J, Davidovitch N, Razum O. How can schools of public health actively promote peace? Public Health Rev. 2021;42:1604459.

Peters MDJ, Marnie C, Tricco AC, Pollock D, Munn Z, Alexander L, McInerney P, Godfrey CM, Khalil H. Updated methodological guidance for the conduct of scoping reviews. JBI evidence synthesis. 2020;18(10):2119–26.

Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009;26(2):91–108.

Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018;18(1):143.

Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, Moher D, Peters MDJ, Horsley T, Weeks L, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med. 2018;169(7):467–73.

White SK, Lown B, Rohde JE. War or health? Assessing public health education and the potential for primary prevention. Public Health Rep. 2013;128(6):568–73.

Evans DP, Anderson M, Shahpar C, Del Rio C, Curran JW. Innovation in Graduate Education for Health Professionals in Humanitarian Emergencies. Prehosp Disaster Med. 2016;31(5):532–8.

Joshi PT. Guidelines for international trauma work. Int Rev Psychiatry. 1998;10(3):179–85.

Kulling PEJ, Holst JEA. Educational and training systems in Sweden for prehospital response to acts of terrorism. Prehosp Disaster Med. 2003;18(3):184–8.

McDonnell SM, Bolton P, Sunderland N, Bellows B, White M, Noji E. The role of the applied epidemiologist in armed conflict. Emerg Themes Epidemiol. 2004;1(1):4.

McGann S. Collaboration and conflict in international nursing, 1920–39. Nursing history review : official journal of the American Association for the History of Nursing. 2008;16:29–57.

Ratner KG, Katona LB. The peacebuilding potential of healthcare training programs. Confl Heal. 2016;10(1):29.

Obermeier A-M, Siri Aas R: Conflict Trends: A Global Overview, 1946–2022. In: PRIO Paper. Oslo; 2023.

Brake T-M, Razum O. Prevention of War: A Scoping Review on Primary Preventive Measures in Public Health. Public Health Rev. 2023;44:1606201.

Spiegel PB, Karadag O, Blanchet K, Undie C-C, Mateus A, Horton R. The CHH-Lancet Commission on Health, Conflict, and Forced Displacement: reimagining the humanitarian system. Lancet (London, England). 2024;403(10433):1215–7.

U.S. Department of Health and Human Services: Public Health emergency response guide for state, local, and tribal Public Health directors. Version 2.0. In.: Centers for Disease Control and Prevention; 2011.

Services; USDoHaH, Prevention; CfDCa: Public health emergency response guide for state, local, tribal, and territorial public health directors. Version 2.0. In.; 2018.

Gowing JR, Walker KN, Elmer SL, Cummings EA. Disaster Preparedness among Health Professionals and Support Staff: What is Effective? An Integrative Literature Review. Prehosp Disaster Med. 2017;32(3):321–8.

Khorram-Manesh A, Lupesco O, Friedl T, Arnim G, Kaptan K, Djalali AR, Foletti M, Ingrasia PL, Ashkenazi M, Arculeo C, et al. Education in Disaster Management: What Do We Offer and What Do We Need? Proposing a New Global Program. Disaster Med Public Health Prep. 2016;10(6):854–73.

Perpiñá-Galvañ J, Juliá-Sanchis R, Olmos-Castelló É, Mollá-Pérez S, Sanjuan-Quiles Á. European educational programmes in health emergency and disaster management: an integrative review. Int J Environ Res Public Health. 2021;18(21):11455.

Ripoll Gallardo A, Djalali A, Foletti M, Ragazzoni L, Della Corte F, Lupescu O, Arculeo C, Arnim G, Friedl T, Ashkenazi M, et al. Core Competencies in Disaster Management and Humanitarian Assistance: A Systematic Review. Disaster Med Public Health Prep. 2015;9(4):430–9.

Ellsberg D. Secrets: A memoir of Vietnam and The Pentagon Papers. New York: Penguin Books; 2003.

Google Scholar  

Labonté R, Martin G, Storeng KT. Editorial: Whither globalization and health in an era of geopolitical uncertainty? Glob Health. 2022;18(1):87.

Persaud A, Bhugra D, Valsraj K, Bhavsar V. Understanding geopolitical determinants of health. Bull World Health Organ. 2021;99(2):166–8.

Nations U: Memorandum on security assurances in connection with Ukraine’s accession to the Treaty on the Non-Proliferation of Nuclear Weapons. UNTS, 3007(52241), 167–182. In.; 1994.

Razum O, Wandschneider L. Public Health and War: Hope Among the Horrors? Am J Public Health. 2023;113(2):133–5.

Article   PubMed Central   Google Scholar  

Dönges HE, Hofmann SC. Defence As Security. In: Galbreath DJ, Deni JR, editors. Routledge handbook of defence studies. London; New York; London: Routledge Taylor & Francis Group; 2018. p. 29–39.

Chapter   Google Scholar  

Wither JK. Back to the future? Nordic total defence concepts. Def Stud. 2020;20(1):61–81.

Download references

Open Access funding enabled and organized by Projekt DEAL. None to declare.

Author information

Authors and affiliations.

Department of Epidemiology and International Public Health, School of Public Health, Bielefeld University, Bielefeld, Germany

Lisa Wandschneider, Anna Nowak, Anina Grün & Oliver Razum

Research Institute Social Cohesion, Bielefeld University, Bielefeld, Germany

Lisa Wandschneider, Anna Nowak, Anina Grün, Yudit Namer & Oliver Razum

School of Public Health, University College Cork, Cork, Ireland

Colette Cunningham

Medical University of Gdansk, Gdanks, Poland

Marta Miller

Department of Psychology, Health & Technology, Faculty of Behavioural, Management and Social Sciences, University of Twente, Enschede, Netherlands

Yudit Namer

Department of Nutrition and Drug Research, Institute of Public Health, Faculty of Health Sciences, Jagiellonian University Medical College, Krakow, Poland

Tomasz Bochenek

Department of Public Health and Social Medicine, Medical University of Gdansk, Gdansk, Poland

Lukasz Balwicki

You can also search for this author in PubMed   Google Scholar

Contributions

LW, YN, CC, TB, LB and OR conceptualized and designed the Scoping Review with important intellectual input from all co-authors. LW, MM, AG, AN and CC, conducted the screening. LW, MM, AG, AN extracted the data. LW, MM and AN analyzed the data. LW, MM, AN, AG and CC created the first draft of the manuscript. CC and LW edited the manuscript prior to submission to all authors, who then contributed input to the final manuscript. CC and LW edited the final draft manuscript. All authors contributed to the final manuscript and provided important intellectual input.

Corresponding author

Correspondence to Anna Nowak .

Ethics declarations

Ethics approval and consent to participate.

Not required.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., supplementary material 3., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Wandschneider, L., Nowak, A., Miller, M. et al. War and peace in public health education and training: a scoping review. BMC Public Health 24 , 2303 (2024). https://doi.org/10.1186/s12889-024-19788-w

Download citation

Received : 25 October 2023

Accepted : 13 August 2024

Published : 24 August 2024

DOI : https://doi.org/10.1186/s12889-024-19788-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Prevention of war
• Peace promotion
• Public health education
• Graduate education
• Workforce development

BMC Public Health

ISSN: 1471-2458

• General enquiries: [email protected]

• Open access
• Published: 21 August 2024

Evaluating the effectiveness of large language models in abstract screening: a comparative analysis

• Michael Li 1 ,
• Jianping Sun 2 &
• Xianming Tan   ORCID: orcid.org/0000-0002-5478-2269 1 , 3  

Systematic Reviews volume  13 , Article number:  219 ( 2024 ) Cite this article

139 Accesses

Metrics details

This study aimed to evaluate the performance of large language models (LLMs) in the task of abstract screening in systematic review and meta-analysis studies, exploring their effectiveness, efficiency, and potential integration into existing human expert-based workflows.

We developed automation scripts in Python to interact with the APIs of several LLM tools, including ChatGPT v4.0, ChatGPT v3.5, Google PaLM 2, and Meta Llama 2, and latest tools including ChatGPT v4.0 turbo, ChatGPT v3.5 turbo, Google Gemini 1.0 pro, Meta Llama 3, and Claude 3. This study focused on three databases of abstracts and used them as benchmarks to evaluate the performance of these LLM tools in terms of sensitivity, specificity, and overall accuracy. The results of the LLM tools were compared to human-curated inclusion decisions, gold standard for systematic review and meta-analysis studies.

Different LLM tools had varying abilities in abstract screening. Chat GPT v4.0 demonstrated remarkable performance, with balanced sensitivity and specificity, and overall accuracy consistently reaching or exceeding 90%, indicating a high potential for LLMs in abstract screening tasks. The study found that LLMs could provide reliable results with minimal human effort and thus serve as a cost-effective and efficient alternative to traditional abstract screening methods.

While LLM tools are not yet ready to completely replace human experts in abstract screening, they show great promise in revolutionizing the process. They can serve as autonomous AI reviewers, contribute to collaborative workflows with human experts, and integrate with hybrid approaches to develop custom tools for increased efficiency. As technology continues to advance, LLMs are poised to play an increasingly important role in abstract screening, reshaping the workflow of systematic review and meta-analysis studies.

Peer Review reports

Introduction

A systematic review is a rigorous form of research that collates and synthesizes all existing evidence on a specific research question [ 1 ]. It stands as a cornerstone not just in medical research but across diverse academic disciplines. Unlike traditional literature reviews, systematic reviews follow a comprehensive and standardized process, such as the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guideline [ 2 ], designed to minimize bias and ensure reproducibility. Therefore, these reviews are recognized as one of the zenith levels of evidence in evidence-based research [ 3 ] and play a pivotal role in shaping clinical guidelines, healthcare policies, and informing medical decisions [ 4 ].

Commencing with a well-articulated research question, a typical systematic review launches an exhaustive search strategy that subsequently sweeps through databases, such as PubMed and Embase, supplemented by additional sources such as clinical trial registries and pertinent article reference lists, with aims to capture all relevant studies and mitigate bias. Predetermined inclusion and exclusion criteria guide the subsequent screening and selection of studies, encompassing facts like study design, patient demographics, and intervention types. Reviewers, working independently, appraise each study’s eligibility, reconciling disagreements through discussions or third-party reviews. Data extraction and synthesis follow, either through meta-analysis or narrative synthesis, depending on the heterogeneity of the selected studies.

The practice of conducting systematic reviews has gained substantial popularity with considerable demand within the academic community. A notable reference [ 5 ] to this trend found that approximately 75 trials and potentially 11 systematic reviews are disseminated daily, based on data from around 2010. Moreover, a query for “systematic review” in the Google Scholar database yields approximately 17,000 entries that have been published within the year 2023 alone, which translates into about 51 systematic reviews per day, as observed on November 26, 2023. This expanding volume of literature underscores the critical role that systematic reviews play in the consolidation of research findings across various fields of study.

Despite their pivotal role, executing systematic reviews remains a formidable task due to the abstract screening, a key phase that can be overwhelmingly time-consuming due to its volume. For example, in Polanin et al. [ 6 ], the authors reported that the research staff screened 29,846 abstracts independently (14,923 unique citations were double screened) over the course of 189 days. In addition, the variability in reporting standards, use of jargon, and varied study designs can further complicate the abstract screening process [ 7 ]. Nevertheless, the repetitive nature of the task, combined with cognitive fatigue, can lead to human errors [ 8 , 9 ]. Recent advancements in machine learning (ML) and deep learning propose possible solutions to these challenges. However, traditional ML models, while promising, require domain-specific training, a time-consuming process that often demands manual labeling of datasets [ 10 ].

Advancements in natural language processing (NLP) and artificial intelligence (AI) are opening doors to address challenges in systematic reviews. Large language models (LLMs) like ChatGPT [ 11 ], PaLM [ 12 ], Llama [ 13 ], and Claude [ 14 ] are capturing the research community’s attention. Their collective potential, especially their capability to operate without exhaustive domain-specific training, makes them prime candidates for revolutionizing the systematic review process.

While each of the aforementioned AI tools brings its unique capabilities to the table, the fundamental question remains: How do they stack up, both individually and collectively, in the realm of abstract screening, against the human expert-based process? In pursuit of answers, this research seeks to thoroughly investigate the potential of ChatGPT, Google PaLM, Llama, and Claude in automating the crucial abstract screening phase integral to systematic reviews. Our goal is to rigorously compare the performance of these advanced AI-driven methods with existing machine learning (ML)-based approaches. In doing so, we aim to develop AI strategies that masterfully balance efficiency and accuracy with minimal human intervention, ultimately transforming systematic review practice across disciplines.

The use of NLP for abstract screening is established [ 10 , 15 ]. However, the application of LLMs specifically for this task is a nascent field [ 16 , 17 ]. This emerging area offers significant potential to improve efficiency and accuracy. Our study aims to fill this gap by providing a comprehensive analysis of LLM capabilities in abstract screening, laying the groundwork for future research and application. This is particularly relevant considering the rapid evolution of this technology, highlighting its potential to streamline systematic reviews now and in the future.

The remainder of this paper is structured to provide a comprehensive exploration of our topic. We begin with an in-depth examination of existing methods for abstract screening, including both manual and NLP-based approaches, laying the groundwork for understanding the current state of the field. We then introduce the use of large language model (LLM) tools for abstract screening, detailing our experimental design to meticulously evaluate their performance in this context. Subsequent sections present our empirical findings and results, shedding light on the capabilities and limitations of the AI tools in question. Finally, we engage in a thoughtful discussion, reflecting on the implications of our findings and considering the future trajectory of abstract screening in systematic reviews.

Existing approaches to abstract screening in systematic reviews

In the vast realm of systematic reviews, the critical task of abstract screening serves as a foundational step in curating the highest quality of evidence [ 2 ]. However, this process often presents significant challenges due to the involvement of sifting through large volumes of literature to identify those that align with predefined criteria. Over time, various methodologies, ranging from manual evaluations to sophisticated AI-driven techniques, have been proposed to address the complexities of this task. In this section, we first describe the existing approaches on their operational mechanisms and associated advantages and disadvantages.

Manual approach

Historically, the process of abstract screening was firmly rooted in manual evaluations. In this conventional approach, individual reviewers would scrutinize each abstract against predefined criteria [ 1 ]. The meticulous nature of this method required that multiple experts independently evaluate the literature to ensure both reliability and reduced biases [ 8 ]. While the depth of human expertise brought about nuanced understanding, the manual nature of this method made it both time-consuming and, at times, prone to human error [ 6 , 9 ].

NLP-based approach

As technology evolved, the field witnessed the incorporation of natural language processing (NLP) to automate abstract screening [ 10 ]. In this framework, abstract text undergoes preprocessing and vectorization. Supervised machine learning models, notably the support vector machine (SVM) and the random forest (RF), are then trained on this vectorized data to classify literature based on specific criteria [ 15 ]. The strength of this approach lies in its potential for efficiency. However, its efficacy and accuracy hinges heavily on the availability of a well-curated, labeled training set.

Zero-shot classification

A more recent and innovative approach is zero-shot classification, which was notably highlighted by Xian et al. [ 18 ]. Eschewing the need for an extensive labeled training dataset, zero-shot classification offers the allure of categorizing abstracts without prior domain-specific training. By calculating a probability score for each abstract, researchers obtain a dynamic measure of its alignment with predefined categories. While it does not require training for the machine learning model, the tradeoff is a loss of sensitivity and potential omissions of pertinent studies [ 19 ]. In this study, for a given set of abstracts, we first obtain embeddings (using OpenAI’s text embedding API, the babbage model) for abstracts and for a pre-specified description of an ideal study to include. We indeed use the inclusion/exclusion criteria (see “Examples of abstract screening by using LLMs,” “Automated workflow for streamlining abstract screening via ChatGPT and other tools,” and “ Study design ” sections) as the description of an ideal study to include. We then compute the cosine similarity score between each abstract (its embedding) and the pre-specified description (its embedding). We claim the top 10% (in terms of similarity) of abstracts as positive (i.e., a study should be included).

Hybrid approach

To reconcile the strengths and limitations of these models, a hybrid methodology has emerged [ 18 ]. It cleverly marries the rapid categorization capabilities of zero-shot classification with the precision of traditional machine learning models such as SVM and RF. Here, zero-shot classification provides an initial curation of all abstracts in the training set. Then, based on zero-shot classification’s curation, the abstracts with classification score higher than a pre-specified threshold are selected to undergo manual review to rectify zero-shot classification. The rectified classification then serves as a foundation for training traditional models, which are then employed for screening the broader dataset (e.g., in the testing set). The hybrid approach balances the speed of zero-shot and the precision of traditional ML, and potentially offers enhanced accuracy at reduced human efforts. However, this approach involves the use of multiple methodologies and still relies on well-curated, labeled training data (in this case, a subset of the whole training set). In this study, for a given set of abstracts, we rectify the “positive” abstracts (i.e., the top 10% of abstracts) identified by zero-shot, and then fit a classification model for rectified labels to the text embeddings of the corresponding abstracts using SVM. This learned classification model is then used to predict the class of other abstracts. We added the R code for the zero-shot and hybrid approaches to the GitHub repository https://github.com/mikeli380/LLMAbstractScreening .

Active learning approach

Active learning [ 20 , 21 , 22 ] is an innovative approach to machine learning that optimizes the training process by allowing the model to selectively query a human annotator for labels on the most informative data points. This method is particularly advantageous in scenarios where labeled data is scarce or expensive to obtain. Active learning models can start with minimal datasets, often requiring only one relevant and one irrelevant abstract, making them particularly suitable for tasks such as abstract screening, where the acquisition of large labeled datasets can be prohibitive.

The main advantage of active learning is its efficiency. By focusing on the most informative samples, it reduces the amount of data that needs to be labeled while still effectively training the model. This can significantly reduce the time and resources required for the annotation process. However, the effectiveness of active learning depends heavily on the initial selection of samples and the criteria used to determine the informativeness of subsequent data points. If not well calibrated, the model may request labels for data that do not significantly improve its performance, resulting in an inefficient use of resources. In addition, the iterative nature of querying and updating the model based on new labels can introduce complexity into the training process, potentially requiring more sophisticated infrastructure and oversight than traditional supervised learning methods.

While this study did not test active learning approaches for abstract screening, readers interested in exploring this methodology further are referred to [ 23 ] for detailed information on the application of active learning in abstract screening contexts.

• Large language models

Amidst the evolving methodologies described in the “ Existing approaches to abstract screening in systematic reviews ” section, modern AI tools based on large language models (LLMs), such as ChatGPT, PaLM, Llama, and Claude, are emerging as potential game-changers. Grounded in advanced language processing capabilities, these tools can be tailored to evaluate abstracts against nuanced criteria and offer detailed assessments and classifications. Their prowess signals transformative potential for abstract screening. In this section, we first present two examples to illustrate the potential of LLMs in the context of abstract screening, and then precede with our in-depth discussion on the study’s workflow and design that aims to critically investigate the performance of LLMs in this domain.

Examples of abstract screening by using LLMs

In this subsection, we demonstrate the process of LLM-based abstract screening using ChatGPT with two specified abstracts: (1) Millard et al. [ 24 ] and (2) Zawia et al. [ 25 ]. Using other LLM tools such as Llama, Google PaLM, or Claude for abstract screening is similar and will be skipped for brevity.

In constructing the prompts, we adopted a standardized approach to mimic a typical interaction between a senior researcher and a research assistant. Each prompt was designed to include three critical elements: a brief statement of the topic under review, the exact eligibility criteria as specified in the study guidelines, and the text of the abstract to be reviewed. This approach was designed to simulate a realistic and practical scenario that reflects common academic practices. In addition, to ensure consistency and objectivity in the responses, we required that the results of the LLM be in a binary “yes” or “no” format, directly indicating whether each abstract met the inclusion criteria. This format was chosen to eliminate any ambiguity in the model’s decisions and to facilitate straightforward analysis of its accuracy.

Abstract (1)

ChatGPT prompt: “I would like you to help me with conducting a systematic review on animal depression. I will provide the title and abstract for one journal article and would like you to screen the paper for inclusion. Here are the inclusion criteria: Does the study provide primary data of an animal model of depression or depressive-like phenotype with an appropriate control group? Here is the abstract:

Wistar-Kyoto (WKY) rats show endogenous depressive behavior that can be reversed by antidepressants. Given that WKYs exhibit decreased sensitivity to some antidepressants and treatment-resistant depressed patients often show hypothalamic-pituitary-thyroid (HPT) dysregulation, we examined the behavioral and HPT hormonal responses of WKYs to altered thyroid status. “Euthyroid” WKYs had elevated basal plasma TSH and T(3) levels as compared to Wistars. Hypothyroidism increased TSH levels more in WKYs than in Wistars and increased response latency in the open field test (OFT) of WKYs only. Administration of T(4) and T(3) suppressed plasma TSH equally in both strains. Wistars responded to increased T(3) levels with decreased response latency and increased activity in the OFT, but increased immobility in the forced swim test. In contrast, WKYs responded only to the high T(3) levels with decreased response latency in the OFT. These results suggest the existence of a decreased central nervous system sensitivity to thyroid hormones in WKYs that could be related to their depressive behavior.”

ChatGPT v4 response (accessed on 10/15/2023):

Abstract (2)

The induction of the activity of cerebral ornithine decarboxylase (EC 4.1.1.17) and mRNA by electrical stimulation exhibits regional differences. The effects of the enzyme inhibitor difluoromethylornithine on these regional variations was examined. Administration of this inhibitor resulted in pronounced depression of both basal and induced activity of ornithine decarboxylase in the hippocampus. Basal activity of the enzyme in the neocortex and the cerebellum appeared to be resistant to difluoromethylornithine but the induced enzyme activity was sensitive to the effects of this inhibitor. Susceptibility to difluoromethylornithine may be directly correlated with a slower turnover rate for ornithine decarboxylase. These results suggest that ornithine decarboxylase in the hippocampus may possess a longer half-life than its counterparts in other regions of the brain. Pretreatment with difluoromethylornithine had no effect on the induced ornithine decarboxylase mRNA in the neocortex. Thus, elevated activity of ornithine decarboxylase enzyme, due to electrical stimulation, appears to not have any effect on either the transcription or the decay rate of the induced ornithine decarboxylase mRNA. These findings support the concept of region-specific regulation of cerebral ornithine decarboxylase. ”

From the above two examples presented, it is evident that ChatGPT adeptly comprehends the posed queries and efficiently extracted the inclusion and exclusion criteria. It not only makes clear decisions regarding study inclusion but also articulates its rationale by cross-referencing each criterion.

It is worth noting that the same ChatGPT prompt might result in slightly different responses between different users due to the variability in its response generation, but the conclusion for the eligibility of an abstract is typically consistent. In addition, enhancing the specificity of the criteria within the prompts can further refine the screening process. To illustrate, consider the following enhanced prompts:

Enhanced ChatGPT prompt for abstract (1): “I seek assistance with a systematic review focused on animal depression. I’ll provide the abstract of a particular journal article and would appreciate an assessment for its inclusion based on the following criteria:

The study employs an animal model reflecting depression or a depressive-like phenotype.

It incorporates an appropriate control group.

Primary data is presented within the study.

and here are the exclusion criteria:

The study bypasses the use of an animal model.

The utilized animal model doesn’t characterize depression or a depressive-like phenotype.

Absence of a control group within the study.

The study is primarily a review, devoid of primary data.

Here is the abstract: …”.

With these revised prompts applied to the previously discussed abstracts, the conclusions remain consistent. Nonetheless, we contend that this enhanced structure not only provides a more rigorous framework but also aligns more closely with widely accepted practices in systematic review criteria definition.

Automated workflow for streamlining abstract screening via ChatGPT and other tools

For systematic reviews, particularly when dealing with voluminous data, efficient workflows are paramount. The ChatGPT API (application programming interface) offers a dynamic solution, enabling the automation of abstract screening on a large scale, which circumvents the labor-intensive process of manually inputting abstracts into a chatbot interface. In this subsection, we present an automated workflow for streamlining abstract screening via ChatGPT. Note, though this automated workflow uses ChatGPT as the platform, analogous workflows work for other AI platforms like PaLM, Llama, and Claude.

Automated workflow:

Data collection: The preliminary step entails accumulating a list of titles and abstracts. By utilizing carefully crafted keywords, we retrieve these from PubMed and other pertinent databases. This comprehensive approach ensures the potential inclusion of all relevant studies for a subsequent detailed screening. It is worth noting that while this list is expansive, most of these studies may not find their way into the final meta-analysis post-screening.

Automation through Python: We have devised a python script aimed at harnessing the capabilities of ChatGPT for evaluating the amassed abstracts.

This script interacts with the ChatGPT API (specifically, the GPT-4 version) and, when furnished with tailored prompts, extracts structured responses from ChatGPT.

Typically, the AI’s response commences with a succinct summary, delves into explanations aligned with each criterion, and concludes with a decisive judgment, as exemplified by the examples in the “ Examples of abstract screening by using LLMs ” section.

This automated process efficiently saves ChatGPT’s verdicts on each abstract for ensuing analyses. For instance, it extracts the final decisions regarding the inclusion or exclusion of studies and determines the stance on each pre-specified criterion for every abstract, as exemplified by the last example in the “ Examples of abstract screening by using LLMs ” section.

Additionally, to ascertain the efficiency and cost-effectiveness of this methodology, the script also monitors the time, token usage, and the financial implications of querying the OpenAI API.

In essence, we envision this procedure as delegating the meticulous task of poring over scientific summaries to an AI assistant. This virtual entity meticulously sifts through each summary, determining its alignment with stipulated criteria.

Tuning parameters in the ChatGPT API: The effectiveness of the ChatGPT API is not only dependent on the input data; it is also significantly influenced by adjustable parameters that can refine the responses. Parameters such as temperature, top k, and top p critically affect model performance by modulating the randomness and focus of the output. While fine-tuning these parameters can improve results, it requires significant technical expertise and resources. Defaults, which are rigorously tested by developers, strike a balance between output quality and ease of use, making LLMs more accessible to a wider range of users without the need for complex parameter optimization. While customization holds promise for custom applications, the default settings provide an efficient and practical solution that facilitates wider adoption of LLM technologies.

Given the complexity of fine-tuning these parameters to optimize performance specifically for abstract screening, our study primarily used the recommended default settings for these parameters as provided by the respective platforms (detailed in Table 1 ). This approach was chosen to maintain the feasibility of our experiments and to ensure that our findings are applicable to typical deployment scenarios.

The automated workflow described applies to other LLM tools or different versions of the same tools. Throughout our study, we have tested several popular LLM tools available to us. The specific LLM models and their versions used at the time of our first submission are detailed in Table  1 , along with the tuning parameters.

The field of large language models (LLMs) has evolved rapidly since we started this study in 2023. New models are frequently released, and existing versions are constantly updated. To account for these advances, we have expanded our analysis to include results from the latest versions of previously studied models, as well as a few entirely new ones. As a best practice, we set the temperature parameter to 0 for all latest models. Table 1 now includes these latest models and their versions.

A brief explanation of these parameters is as follows:

Temperature: The temperature controls the randomness of the outputs, with a range from 0 to 2. A temperature value greater than 1 is random and 0 is deterministic. The maximum temperature of 2 gives the most creative and variable outputs.

Max length: The max length is the maximum number of tokens for the model to generate as a response. A single word is generally 2–3 tokens.

Stop sequences: This parameter controls which tokens or phrases will stop the LLM from generating more text.

Top p: When generating text, the model samples from the top p percentage of most likely tokens to generate. The top p is the cumulative probability cutoff for the model’s selection of tokens to generate. Lower top p values mean sampling from a smaller, more top-weighted nucleus.

Top k: When generating text, the model samples from the top k most likely tokens. When deciding the next word to output, the model will select the most likely word when top k is lower.

Frequency penalty: This frequency penalty parameter controls how the LLM penalizes tokens that have already appeared in the input and output text. A frequency penalty of 0 implies that the frequency of the tokens does not impact the LLM and will generate them based on their probability.

Presence penalty: This parameter controls how the LLM penalizes tokens that have not been used recently. A presence penalty of 0 means that the LLM does not care about the recency of the tokens and will generate them based on their probability.

Study design

In our pursuit to assess ChatGPT’s proficiency in abstract screening, we selected certain benchmark databases that have existing performance data from other methodologies. This selection aids in a comparative analysis of performance.

In selecting the systematic reviews for our study, we used a systematic approach guided by specific criteria to ensure relevance and reliability. These studies were selected from the publicly available SYNERGY [ 23 ] dataset, which contains 26 systematic reviews from different disciplines. Key selection criteria included:

Clarity and conciseness of eligibility criteria: The selected studies had well-defined and explicit eligibility criteria. This clarity is essential for accurate replication of the study selection process, which is critical for assessing the performance of LLM tools in an analogous real-world application.

Completeness and cleanliness of data: We ensured that the selected reviews had complete datasets, with all necessary information on included and excluded studies clearly documented, minimizing the risk of ambiguities affecting our analysis.

In addition, to comply with the AMSTAR-2 [ 26 ] guidelines, in particular point 5, we reviewed the methodologies of these reviews to confirm the selection of studies was performed in duplicate and disagreements were resolved by consensus. While our analysis assumes that these systematic reviews adhere to high standards, we recognize the inherent limitations of using pre-existing datasets as a proxy for gold standards in the discussion section.

We picked the following 3 databases from the publicly available SYNERGY dataset [ 23 ]:

Bannach-Brown 2016 [ 27 ]—topic: use of animal models to study depressive behavior

Human-curated (gold standard) results: 1258 excluded abstracts and 230 included abstracts.

We randomly selected 100 excluded abstracts and 100 included abstracts for screening by LLM tools.

Meijboom 2021 [ 28 ]—topic: retransitioning of etanercept in patients with a rheumatic disease

Human-curated (gold standard) results: 599 excluded abstracts and all 32 included abstracts.

We randomly selected 100 excluded abstracts and 32 included abstracts for screening by LLM tools.

Menon 2022 [ 29 ]—topic: methodological rigor of systematic reviews in environmental health

Human-curated (gold standard) results: 896 excluded abstracts and 73 included abstracts.

We randomly selected 100 excluded abstracts and all 73 included abstracts for screening by LLM tools.

For each chosen database, abstracts were categorized as either “cases” (those included based on a gold standard) or “controls” (those excluded per the gold standard). From each category, we randomly selected 100 abstracts (we use all abstracts if there are less than 100 abstracts in that category in a database). These abstracts underwent evaluation by ChatGPT (v4.0) as per our established workflow. Subsequently, ChatGPT’s decisions were juxtaposed against the gold standard to determine sensitivity, specificity, and overall accuracy. The same abstracts were also processed using other LLM tools as listed in Table  1 to record their respective verdicts.

Statistical analysis

To quantify the efficacy of ChatGPT and other AI tools for each database, we calculated the following metrics: (1) sensitivity, (2) specificity, and (3) overall accuracy, where sensitivity is defined as the number of true positives divided by the sum of true positives and false negatives, specificity as the number of true negatives divided by the sum of true negatives and false positives, and accuracy as sum of true positives and true negatives divided by the total number of abstracts. For each metric, associated 95% confidence intervals were also determined. Although it is very common in the field to report F1 score, recall rate, and precision, we believe it is more appropriate to report sensitivity and specificity given this study design. In addition, F1 score, recall rate, and precision can be derived from sensitivity and specificity.

Furthermore, to explore the potential of a unified decision-making process, we combined the decisions from all AI tools using a voting mechanism. The majority decision across the tools was taken as the final verdict for each abstract. For this consolidated approach, we again computed sensitivity, specificity, overall accuracy, and the associated 95% CIs for each database. We also explore the use of latent class analysis (LCA), a model-based clustering approach, to derive consolidated decisions. More details on this LCA approach are provided in the “ Beyond majority voting ” section.

For a given database, 100 cases and 100 controls yield a two-sided 95% confidence interval with a half-width equal to 0.048 when the underline sensitivity (specificity) is approximately 95%.

All statistical analyses were conducted using the R statistical software (version 4.3.1). All tests were two-sided with an alpha level set at 0.05 unless otherwise mentioned.

To improve the transparency and reproducibility of studies using AI tools, we have included the TRIPOD + AI checklist [ 30 ] in our report. This checklist has been adapted to reflect the specifics of our research, which focuses on the evaluation of large language models for abstract screening rather than diagnostic or prognostic modeling. The completed checklist is presented in Table S 1 , to provide readers with a comprehensive overview of our study’s adherence to established reporting standards.

We present the results for each of the 3 databases. For each, we first present the prompts we used when we called LLM tools to screen an abstract, then present the performance data (accuracy, sensitivity, and specificity for each method or LLM tool), followed by a summary of the performance, and a comparison of the performance of the different methods to that based on ChatGPT v4.0.

Results on the Bannach-Brown 2016 database (see Table  2 )

The prompts we used for screening abstracts in this database are as follows:

Conduct a systematic review on animal depression. I provide the title and abstract for one journal article. Provide an overall assessment based on eligibility criteria with only one word answer yes or no with no explanation. Then, for each inclusion or exclusion criterion, answer with only one word, yes if it is included by the inclusion criterion or excluded by the exclusion criterion, and answer no if it does not meet the inclusion criterion or not excluded by the exclusion criterion. After answering all the criteria with yes or no, then provide an overall explanation.

Here is the eligibility criteria: Inclusion Criteria: 1. Any article providing primary data of an animal model of depression or depressive-like phenotype with an appropriate control group (specified above). 2. Animals of all ages, sexes and species, where depression-like phenotype intended to mimic the human condition have been induced. Including animal models where depressive-like phenotypes are induced in the presence of a comorbidity (e.g. obesity or cancer). 3. All studies that claim to model depression or depressive-like phenotypes in animals. Studies that induce depressive behavior or model depression and that also test a treatment or intervention (prior or subsequent to model induction), with no exclusion criteria based on dosage, timing or frequency. 4. Studies measuring behavioral, anatomical and structural, electrophysiological, histological and/or neurochemical outcomes and where genomic, proteomic or metabolomic outcomes are measured in addition to behavioral, anatomical, electrophysiological, histological or neurochemical outcomes. Exclusion Criteria: 1. Review article, editorials, case reports, letters or comments, conference or seminar abstracts, studies providing primary data but not appropriate control group. 2. Human studies and ex vivo, in vitro or in silico studies. Studies will be excluded if authors state an intention to induce or investigate only anxiety or anxious behavior. Studies will be excluded if there is no experimental intervention on the animals (e.g. purely observational studies). 3. Studies that investigate treatments or interventions, but no depressive behavior or model of depression is induced (e.g. toxicity and side-effect studies). 4. Where metabolic outcome measures are the primary outcome measure of a study. Where genomic, proteomic, metabolic or metabolomic outcomes are the sole outcome measures in a study, they will be excluded.

Here is the abstract:

Among all the LLM tools we tested, ChatGPT v4.0 stood out with the highest accuracy (0.945) and specificity (0.960), and satisfactory sensitivity (0.930). Combined decision using major voting seemed to improve sensitivity (0.970) considerably but did not improve specificity (0.870) much. Comparatively, the zero-shot method was less effective across these metrics, while the hybrid method attained superior accuracy (0.961) and specificity (0.982), albeit with reduced sensitivity (0.843). We acknowledge that our findings regarding the zero-shot and hybrid methods differ from those reported in [ 18 ] for the same set of abstracts. This discrepancy could be attributed to the use of distinct sets of embeddings in our analysis compared to the previous study.

For this database, the newer versions of ChatGPT (3.5-Turbo and 4-Turbo) did not improve performance over ChatGPT (v4.0). Gemini-1.0-pro (vs. PaLM 2) and Llama 3 (vs. Llama 2) improved over their older versions, but did not surpass the performance of ChatGPT (v4.0). Claude 3 performed well, but still did not surpass the performance of ChatGPT (v4.0).

Comparison between LLM tools. We compared the performance (sensitivity and specificity) between ChatGPTv4.0 and other LLM tools using the McNemar test and found that ChatGPTv4.0 performed significantly better ( p value = 0.002) than Google PaLM 2 in terms of sensitivity; ChatGPTv4.0 performed significantly better than ChatGPTv3.5 ( p value = 0.008) and better than Llama-2 ( p value < 0.001) in terms of specificity. Combining the decisions of different LLM tools using majority voting did not improve the overall accuracy compared to ChatGPTv4.0. Specifically, there was no statistically significant difference ( p value = 0.134) in sensitivity between the combined decision (majority voting) and ChatGPTv4.0, and the combined decision was significantly worse ( p value = 0.008) than ChatGPTv4.0 in terms of specificity.

Comparison between ChatGPT v4.0 and zero-shot and hybrid methods: We assessed the performance (sensitivity and specificity) of ChatGPT v4.0 against both the zero-shot and hybrid approaches using the McNemar test. Specifically, we aligned the screening results from 100 cases and 100 controls as per the ChatGPT v4.0 method and similarly for the zero-shot and hybrid methods, testing for inconsistencies between these approaches as previously done. Our analysis revealed that ChatGPT v4.0 significantly outperformed the zero-shot method in sensitivity ( p value < 0.001) but showed comparable effectiveness in specificity ( p value = 0.37). Additionally, ChatGPT v4.0 demonstrated marginally superior sensitivity compared to the hybrid method ( p value = 0.07), while its performance in specificity was similar ( p value = 1.00).

Results on the Meijboom 2021 database (see Table  3 )

Conduct a systematic review on transitioning patients from an originator to a corresponding biosimilar.

I provide the title and abstract for one journal article. Provide an overall assessment based on eligibility criteria with only one word answer yes or no, with no explanation. Then, for each inclusion or exclusion criterion, answer with only one word, yes if it is included by the inclusion criterion or excluded by the exclusion criterion, and answer no if it does not meet the inclusion criterion or not excluded by the exclusion criterion.

After answering all the criteria with yes or no, then provide an overall explanation.

Here is the eligibility criteria:

Articles were included if they met the following criteria:

Study involved transitioning from a TNFα inhibitor (including etanercept, infliximab, and adalimumab) originator to a biosimilar

The number of patients who retransitioned was reported or could be calculated

The article was an original research article published in a peer-reviewed journal

The article included baseline characteristics of the patients who transitioned

The article was written in English

The full-text version of the article could be obtained.

Transitioning was defined as patients in whom the biosimilar was introduced after the originator, without treatment with other drugs in between. Retransitioning was defined as restarting the originator directly after discontinuing a biosimilar, without treatment with other drugs in between. In summary, transitioning was defined as switching from the originator to a biosimilar; retransitioning was defined as switching from the originator to a biosimilar and back to the originator. Both transitioning and retransitioning involved changes with the same active biological substance.

Among all the LLM tools we tested, ChatGPT v4.0 stood out with the highest accuracy (0.840), but not with specificity (0.860) or satisfactory sensitivity (0.812). Compared to ChatGPTv4.0, combined decision using major voting did not improve overall accuracy (0.720), but improved sensitivity (1.000) at the sacrifice of specificity (0.630).

Comparison between LLM tools. We compared the performance (sensitivity and specificity) between ChatGPTv4.0 and other LLM tools using the McNemar test and found that ChatGPTv4.0 performed significantly better ( p value < 0.001) than Google PaLM 2, but significantly worse than ChatGPT3.5 ( p value = 0.001) and Llama 2 in terms of sensitivity; ChatGPTv4.0 performed significantly better than ChatGPTv3.5 ( p value < 0.001) and better than Llama 2 ( p value < 0.001), but worse than Google PaLM 2 ( p value = 0.002), in terms of specificity. Combining the decisions of different LLM tools using majority voting did not improve the overall accuracy compared to ChatGPTv4.0. Specifically, there was statistically significant difference ( p value = 0.008) in sensitivity between the combined decision (majority voting) and ChatGPTv4.0, and the combined decision was not significantly worse ( p value > 0.50) than ChatGPTv4.0 in terms of specificity.

For this database, the newer versions of ChatGPT (3.5-Turbo and 4-Turbo) did not improve performance over ChatGPT (v4.0), and Gemini-1.0-pro (vs. PaLM 2) did not improve performance either. However, Llama 3 (vs. Llama 2) improved over its older version and surpassed the performance of ChatGPT (v4.0). Claude 3 also slightly surpassed the performance of ChatGPT (v4.0).

Comparison between ChatGPT v4.0 and zero-shot and hybrid methods: We evaluated the performance of ChatGPT v4.0, focusing on sensitivity and specificity, in comparison with the zero-shot and hybrid approaches, employing the McNemar test as described above. In this analysis, we aligned the screening results from 32 cases and 100 controls for the tests. Our findings indicated that ChatGPT v4.0 significantly surpassed the zero-shot method in sensitivity ( p value = 0.0002) and exhibited marginally improved specificity ( p value = 0.099). Furthermore, ChatGPT v4.0 showed notably higher sensitivity than the hybrid method ( p value < 0.001), although its specificity was comparatively lower.

Results on the Menon 2022 database (see Table  4 )

“Conduct a systematic review on the methodological rigour of systematic reviews in environmental health.

I provide the title and abstract for one journal article.

Provide an overall assessment based on eligibility criteria with only one word answer yes or no, with no explanation.

Then, for each inclusion or exclusion criterion, answer with only one word, yes if it is included by the inclusion criterion or excluded by the exclusion criterion and answer no if it does not meet the inclusion criterion or not excluded by the exclusion criterion.

Here are the eligibility criteria:

To be eligible for inclusion in the SR sample, documents had to fulfill the following criteria:

Identify explicitly as a “systematic review” in their title

Assess the effect of a non-acute, non-communicable, environmental exposure on a health outcome. Environmental exposures can include air and water pollutants, radiation, noise, occupational hazards, lifestyle factors (like diet or physical activity) and lifestyle choices influenced by family and peers (like substance use), social and economic factors (like stress from work or living conditions).

Include studies in people or mammalian models

Be available in HTML format

Abstract X”

Among all the LLM tools we tested, ChatGPT v4.0 stood out with the highest accuracy (0.913) and specificity (0.932), but not with specificity (0.900). Compared to ChatGPTv4.0, combined decision using major voting did not improve overall accuracy (0.884) or sensitivity (0.808), but improved specificity (0.940).

Comparison between LLM tools. We compared the performance (sensitivity and specificity) between ChatGPTv4.0 and other LLM tools using the McNemar test and found that ChatGPTv4.0 performed significantly better than ChatGPT3.5 ( p value < 0.001), Google PaLM 2, and Llama 2 ( p value = 0.02) in terms of sensitivity; ChatGPTv4.0 performed worse than ChatGPTv3.5 and Google PaLM 2, in terms of specificity. Combining the decisions of different LLM tools using majority voting did not improve the overall accuracy compared to ChatGPTv4.0. Specifically, there was statistically significant difference ( p value = 0.008) in sensitivity between the combined decision (majority voting) and ChatGPTv4.0, and the combined decision was not significantly different ( p value = 0.134) than ChatGPTv4.0 in terms of specificity.

For this database, the newer versions of ChatGPT (3.5-Turbo and 4-Turbo) did not improve performance over ChatGPT (v4.0). However, both Gemini-1.0-pro (vs. PaLM 2) and Llama 3 (vs. Llama 2) improved over their older versions and surpassed the performance of ChatGPT (v4.0). Claude 3 also performed well but did not surpass the performance of ChatGPT (v4.0).

Comparison between ChatGPT v4.0 and zero-shot and hybrid methods: We aligned the screening results from 73 cases and 100 controls based on the ChatGPT v4.0 method, and similarly for the zero-shot and hybrid methods, to test for inconsistencies between these approaches, using the McNemar test as done in previous assessments. Our analysis showed that ChatGPT v4.0 significantly outperformed the zero-shot method in both sensitivity ( p value < 0.001) and specificity ( p value = 0.016). In comparison with the hybrid method, ChatGPT v4.0 also demonstrated superior sensitivity ( p value < 0.001) and better specificity ( p value = 0.04).

Monetary cost and time cost

To use the ChatGPT API or other LLM tools, the owners of these platforms charge a predetermined rate for access to the corresponding APIs. These fees are calculated in USD per thousand tokens, where tokens are the basic units used by these LLM platforms to quantify text length. In this context, a token can represent a word, a punctuation mark, or a character. The financial cost of screening 200 abstracts was approximately $6 for ChatGPT v4.0, $0.2 for ChatGPT v3.5, $10 for Llama 2 (using Replicate), while Google PaLM 2 offered its services for free to invited developers. Thus, the cumulative cost of evaluating 200 abstracts across all platforms was approximately $16.2. The cumulative cost of evaluating 200 abstracts across all latest models ($3 for GPT-4-Turbo, $0.05 for GPT-3.5-Turbo, free for Gemini-1.0-pro, $0.05 for Llama-3, $4 for Claude) was less, approximately $7.1. In terms of time efficiency, processing 200 abstracts with each of these LLM tools took approximately 10–20 min using a single thread. However, it is imperative to recognize that abstract screening lends itself well to parallelization. Consequently, one could significantly speed up the process by setting up multiple threads to simultaneously screen different subsets of abstracts, thereby reducing the overall time required for completion. This parallel approach not only increases efficiency, but also ensures that large amounts of data can be processed in a timely manner, making LLM tools even more attractive for large-scale abstract screening tasks. In summary, the monetary and time costs of using LLM tools for abstract screening are negligible compared to manual labeling.

Beyond majority voting

We have expanded our analysis to include a variety of approaches for synthesizing decisions across different LLM tools. Our methodology is inspired by the concept of combining multiple diagnostic tests in the absence of a gold standard, akin to situations where human expert consensus is not available. There are several publications discussing such scenarios [ 31 , 32 ], among which we proposed to use the latent class analysis (LCA) models.

Latent class analysis (LCA) is a statistical method used to identify subgroups within a population, which are not directly observed (hence “latent”) [ 33 , 34 , 35 ]. It is particularly useful when the research interest lies in categorizing individuals into mutually exclusive groups based on their responses to multiple observed variables. In the context of abstract screening, LCA can offer a sophisticated means of integrating decisions from different LLM tools without relying on a gold standard, typically provided by human expert consensus. This approach assumes that the unobserved subgroups (or “latent classes”) explain the dependence between the observed decisions made by each of the LLM tools.

Utilizing the LCA model, we treat the decisions from all LLM tools as dichotomous variables, corresponding to the adherence to each inclusion or exclusion criterion, as well as the overall decision. For instance, within the Bannach-Brown 2016 database (BB2016), there are eight criteria in total with four criteria each for inclusion and exclusion and one overall decision for inclusion, resulting in a total of nine binary items per LLM tool. In our analysis, we incorporated decisions from GPT v4.0, v3.5, and Llama 2. Decisions from Google PaLM 2 were excluded due to a high frequency (10% or more) of incomplete responses. Consequently, for the Bannach-Brown 2016 database, we worked with 27 binary items. For other databases such as Meijboom 2021 (Mj2021) and Menon 2022 (Me2022), the binary items totaled 21 and 15, respectively. It is important to note that LCA models were fitted to the binary data of each database independently.

The LCA model fitting process enables us to calculate the posterior probabilities of each abstract belonging to specific latent classes or subgroups. Abstracts are then categorized based on these probabilities, with assignment to the class for which an abstract has the highest posterior membership probability. The determination of the number of latent classes is a critical step in the LCA model fitting, which requires a priori specification. In our evaluation, we explored models with class numbers ranging from 2 to 6 and utilized the Bayesian information criterion (BIC) to identify the most “optimal” LCA model for our datasets.

Table 5 shows that after applying the Bayesian information criterion (BIC) to determine the most appropriate model for our data, we identified a 3-class model as the best fit for the binary decisions derived from the BB2016 database. Similarly, a 4-class model was optimal for the Mj2021 database, while a 3-class model was again best for the Me2022 database. The confusion matrices generated by the selected LCA models for each database provided a clear juxtaposition between the LLM-assigned classes and the actual labels of the abstracts (see Table  6 ).

The performance metrics derived from these models are noteworthy. For the BB2016 database, should we categorize abstracts within class with label 2 as “included” and assign the others in classes with label 1 or 3 to the “excluded” category, the LCA model achieved a sensitivity rate of 93% and a specificity rate of 96%, indicating a high degree of accuracy in classifying relevant and irrelevant abstracts. In the case of the Mj2021 database, if we view class 1 as “included” group and classes 2 and 3 as “excluded” group, the model achieved a perfect sensitivity rate of 100%, meaning that it correctly identified all relevant abstracts, although the specificity was lower at 79%, indicating a higher rate of false positives. Similarly, for the Me2022 database, considering class 1 to be the “excluded” category and classes 2 and 3 to be the “included” group, the model showed a sensitivity of 94.5% and a specificity of 83%, a good balance between identifying relevant abstracts and avoiding false inclusions.

These results highlight the robustness of the latent class analysis approach to the abstract screening task, providing an effective method for classifying abstracts when a gold standard is not available. The varying levels of sensitivity and specificity across databases underscore the need to tailor the LCA model to the specific characteristics of each dataset as well as further future research.

We have incorporated results from the latest LLM models to increase the robustness and relevance of our findings. However, to maintain consistency and comparability with our original analyses, we have not incorporated these new results into the previously established majority voting or latent class analysis (LCA) approaches. Instead, we have chosen to make all raw data, including results from these newer models, freely available in our GitHub repository. This approach allows interested researchers and practitioners to conduct further investigations or apply alternative methods of analysis. By providing access to this additional data, we aim to promote transparency and enable the broader community to engage with and potentially build upon our work.

This study began with a rigorous exploration of the capabilities of large language models (LLMs) in abstract screening. We used automation scripts developed in Python to interact with the APIs of several LLM tools, including Chat GPT v4.0, Google PaLM 2, and Meta Llama 2, as well as latest versions of these tools. Our central goal was to evaluate the efficiency and accuracy of these tools across three different databases of abstracts, leading us to a complex understanding of their potential in this context.

Large language models (LLMs), particularly ChatGPT, have garnered global attention since their inception. Employing LLMs for abstract screening in systematic reviews is an innovative concept [ 32 , 33 ] and remains underexplored. This study presents the first comprehensive evaluation of LLMs applied to systematic review processes. The findings are encouraging, suggesting that LLMs could revolutionize abstract screening. Specifically, ChatGPT v4.0 exhibited stellar performance across three test scenarios, achieving an accuracy of at least 85%. Furthermore, it attained sensitivity and specificity rates ranging from 80% to an impressive 95%. These exceptional outcomes highlight the substantial promise of LLMs in abstract screening, offering an efficient and capable alternative to the conventional, laborious approaches that typically necessitate extensive human annotation.

However, it is important to acknowledge that we are still in the early stages of integrating LLM tools into the abstract screening field, and they are not without their imperfections—for example, occasionally excluding many (20%) relevant studies, even with the best-performed LLM tools, ChatGPT v4.0. These tools are not a universal solution to all the challenges associated with abstract screening, and they are not ready to completely replace human expertise in this area. Instead, they should be embraced as invaluable assistants in the abstract screening process.

In discussing the limitations of our study, it is important to recognize the constraints associated with our dataset selection and model configuration. We used only three databases from the SYNERGY set, limiting the generalizability of our findings across disciplines and datasets. In addition, the reliance on human-curated labels as a gold standard, while necessary, introduces potential biases due to the retrospective nature of our analysis. These labels may contain errors, and the methodology used in the original reviews to resolve discrepancies may affect the validity of our conclusions.

The parameters chosen for our LLMs—temperature, top k, top p, and prompts—were set to defaults or based on natural conversations to balance output quality with ease of use. While this approach minimizes the need for technical expertise, it may not be optimal for all screening scenarios. In addition, the reliance of LLMs on abstracts alone, as opposed to full-text evaluations, presents a fundamental challenge; critical data influencing inclusion or exclusion may not be present in the abstracts, potentially compromising screening accuracy.

In addition, the rapid development of LLMs and their “black box” nature pose challenges to the transparency that is essential in scientific settings. The environmental impact of using these computationally intensive models is also significant [ 36 ], requiring sustainability considerations. Future research should focus on refining these tools to increase transparency and efficiency, and ensure their responsible development and use in systematic reviews.

Our research suggests that LLM tools are ready to take on a role in abstract screening and are poised to have an immediate and positive impact on the process. Their integration into abstract screening can manifest itself in a few ways. They can serve as autonomous AI reviewers, adding an extra layer of scrutiny and ensuring thoroughness. Our findings suggest that a collective decision, such as one derived from majority voting, can sometimes improve sensitivity, underscoring the potential of LLM tools as a reliable safeguard against oversight, ensuring both comprehensiveness and accuracy.

In addition, LLM tools can facilitate a synergistic partnership with human experts. They are adept at identifying “high-risk” abstracts where different LLM tools have reached different judgments, flagging them for further human evaluation, and promoting a careful and error-free screening process while minimizing human effort.

Another exciting prospect is the integration of LLM tools with hybrid approaches and active learning approach. In this scenario, LLM tools could autonomously annotate abstracts in the training set, minimizing the need for human labeling. Subsequently, these labeled abstracts could be used to train custom NLP models, paving the way for a streamlined and efficient abstract screening process, resulting in significant time and resource savings. Further research is needed to understand how the “uncertainty” in LLM-based decisions, when human-curated labels are not used, affects the performance of the hybrid approach. We also note from the test examples that the performance of hybrid approaches varies widely from example to example and depending on the text embedding tools used. Extensive research is needed to perfect the hybrid approaches.

Another future research direction is to explore how to fine-tune different versions of LLM tools and how to derive collective decisions. One idea is that by using different tuning parameters, one could propose different versions of LLM tools, and if they perform similarly in terms of accuracy but give different decisions (e.g., not highly correlated), we expect that better collective decisions would be obtained, as observed in many ensemble learning experiences [ 37 ]. However, this could be costly and require a lot of exploration.

More importantly, it would be particularly valuable to prospectively test these LLM tools, or integration of these tools with other approaches like zero-shot, active learning, in real-world meta-analysis and systematic review projects. This would provide deeper insights into their practical utility and effectiveness, and a more comprehensive understanding of their impact in live research environments.

In summary, while LLM tools may not be able to fully replace human experts in abstract screening, their ability to transform the screening process is both undeniable and imminent. With continued advances in technology and ongoing refinement, these tools will play a critical role in the future of abstract screening, ushering in a new era of efficiency and effectiveness.

Availability of data and materials

The datasets analyzed during the current study are available in the SYNERGY repository, https://github.com/asreview/synergy-dataset . We have created a GitHub repository https://github.com/mikeli380/LLMAbstractScreening to host our code (R/Python), abstracts, and intermediate results so that interested readers can replicate our findings.

Higgins JPT, et al. Cochrane handbook for systematic reviews of interventions. 2019.

Book   Google Scholar  

Moher D, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7). https://doi.org/10.1371/journal.pmed.1000097 .

Sackett DL, Rosenberg WMC, Gray JAM, Haynes RB,  Richardson WS. Evidence based medicine: what it is and what it isn’t. 1996. Clin Orthop Relat Res. 2007;455.  https://doi.org/10.1136/bmj.312.7023.71 .

Lavis JN, Posada FB, Haines PA, Osei E. Use of research to inform public policymaking. Lancet. 2004;364(9445). https://doi.org/10.1016/S0140-6736(04)17317-0 .

Bastian H, Glasziou P,  Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?. PLoS Med. 2010;7(9).   https://doi.org/10.1371/journal.pmed.1000326 .

Polanin JR, Pigott TD, Espelage DL, Grotpeter JK. Best practice guidelines for abstract screening large-evidence systematic reviews and meta-analyses. Res Synth Methods. 2019;10(3). https://doi.org/10.1002/jrsm.1354 .

Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med. 2008;358(3). https://doi.org/10.1056/nejmsa065779 .

Edwards P, Clarke M, DiGuiseppi C, Pratap S, Roberts I, Wentz R. Identification of randomized controlled trials in systematic reviews: accuracy and reliability of screening records. Stat Med. 2002;21(11). https://doi.org/10.1002/sim.1190 .

Wang Z, Nayfeh T, Tetzlaff J, O’Blenis P, Murad MH. Error rates of human reviewers during abstract screening in systematic reviews. PLoS ONE. 2020;15(1). https://doi.org/10.1371/journal.pone.0227742 .

O’Mara-Eves A, Thomas J, McNaught J, Miwa M, Ananiadou S. Using text mining for study identification in systematic reviews: a systematic review of current approaches. Syst Rev. 2015;4(1). https://doi.org/10.1186/2046-4053-4-5 .

OpenAI. Introducing ChatGPT Plus. OpenAI Blog; 2023. https://openai.com/blog/chatgpt-plus . Accessed 8 Nov 2023.

Google DeepMind. Introducing PaLM 2. Google Blog; 2023.  https://blog.google/technology/ai/google-palm-2-ai-large-language-model/ . Accessed 8 Nov 2023.

Meta AI Research. Meta and Microsoft introduce the next generation of Llama. Facebook AI Blog; 2023. https://ai.meta.com/blog/llama-2/ . Accessed 8 Nov 2023.

Claude.ai. Claude AI API reference. 2024. https://docs.anthropic.com/en/api/getting-started . Accessed 4 Jun 2024.

Wallace BC, Small K, Brodley CE, Lau J, Trikalinos TA. Deploying an interactive machine learning system in an evidence-based practice center: Abstrackr. 2012. https://doi.org/10.1145/2110363.2110464 .

Alshami A, Elsayed M, Ali E, Eltoukhy AEE, Zayed T. Harnessing the power of ChatGPT for automating systematic review process: methodology, case study, limitations, and future directions. Systems. 2023;11(7). https://doi.org/10.3390/systems11070351 .

Qureshi R, Shaughnessy D, Gill KAR, Robinson KA, Li T, Agai E. Are ChatGPT and large language models ‘the answer’ to bringing us closer to systematic review automation?  Syst Rev. 2023;12(1).   https://doi.org/10.1186/s13643-023-02243-z .

Xian Y, Lampert CH, Schiele B, Akata Z. Zero-shot learning-a comprehensive evaluation of the good, the bad and the ugly. IEEE Trans Pattern Anal Mach Intell. 2019;41(9). https://doi.org/10.1109/TPAMI.2018.2857768 .

Moreno-Garcia CF, Jayne C, Elyan E, Aceves-Martins M. A novel application of machine learning and zero-shot classification methods for automated abstract screening in systematic reviews. Decis Anal J. 2023;6.  https://doi.org/10.1016/j.dajour.2023.100162 .

Ferdinands G, et al. Performance of active learning models for screening prioritization in systematic reviews: a simulation study into the Average Time to Discover relevant records. Syst Rev. 2023;12(1). https://doi.org/10.1186/s13643-023-02257-7 .

Boetje J, van de Schoot R. The SAFE procedure: a practical stopping heuristic for active learning-based screening in systematic reviews and meta-analyses. Syst Rev. 2024;13(1). https://doi.org/10.1186/s13643-024-02502-7 .

Yang Y, Loog M. A benchmark and comparison of active learning for logistic regression. Pattern Recognit. 2018;83. https://doi.org/10.1016/j.patcog.2018.06.004 .

De Bruin RJ, Ma Y, Ferdinands G, Teijema J, Van de Schoot. SYNERGY - open machine learning dataset on study selection in systematic reviews. DataverseNL; 2023. https://dataverse.nl/dataset.xhtml?persistentId=doi:10.34894/HE6NAQ .

Millard SJ, Weston-Green K, Newell KA. The Wistar-Kyoto rat model of endogenous depression: a tool for exploring treatment resistance with an urgent need to focus on sex differences. Prog Neuropsychopharmacol Biol Psychiatry. 2020;101. https://doi.org/10.1016/j.pnpbp.2020.109908 .

Zawia NH, Mattia CJ, Bondy SC. Differential effects of difluoromethylornithine on basal and induced activity of cerebral ornithine decarboxylase and mRNA. Neuropharmacology. 1991;30(4). https://doi.org/10.1016/0028-3908(91)90058-J .

Shea BJ, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358. https://doi.org/10.1136/bmj.j4008 .

Bannach-Brown A, Liao J, Wegener G, Macleod M. Understanding in vivo modelling of depression in non-human animals: a systematic review protocol. Evid Based Preclin Med. 2016;3(2). https://doi.org/10.1002/ebm2.24 .

Meijboom RW, Gardarsdottir H, Becker ML, ten Wolde S, Egberts TCG, Giezen TJ. Incidence of and reasons and determinants associated with retransitioning from biosimilar etanercept to originator etanercept. BioDrugs. 2021;35(6). https://doi.org/10.1007/s40259-021-00501-x .

Menon JML, Struijs F, Whaley P. The methodological rigour of systematic reviews in environmental health. Crit Rev Toxicol. 2022;52(3). https://doi.org/10.1080/10408444.2022.2082917 .

Collins GS, et al. Protocol for development of a reporting guideline (TRIPOD-AI) and risk of bias tool (PROBAST-AI) for diagnostic and prognostic prediction model studies based on artificial intelligence. BMJ Open. 2021;11(7). https://doi.org/10.1136/bmjopen-2020-048008 .

Joseph L, Gyorkos TW, Coupal L. Bayesian estimation of disease prevalence and the parameters of diagnostic tests in the absence of a gold standard. Am J Epidemiol. 1995;141(3). https://doi.org/10.1093/oxfordjournals.aje.a117428 .

Hui SL, Walter SD. Estimating the error rates of diagnostic tests. Biometrics. 1980;36(1). https://doi.org/10.2307/2530508 .

Dendukuri N, Joseph L. Bayesian approaches to modeling the conditional dependence between multiple diagnostic tests. Biometrics. 2001;57(1). https://doi.org/10.1111/j.0006-341X.2001.00158.x .

Collins LM, Lanza ST. Latent class and latent transition analysis: with applications in the social, behavioral, and health sciences. 2010.

Google Scholar  

McCutcheon AL. Latent class analysis. Newbury Park: Sage; 1987.

Rillig MC, Ågerstrand M, Bi M, Gould KA, Sauerland U. Risks and benefits of large language models for the environment. Environm Sci Technol. 2023;57(9). https://doi.org/10.1021/acs.est.3c01106 .

Sagi O, Rokach L. Ensemble learning: a survey.  Wiley Interdiscip Rev Data Min Knowl Discov. 2018;8(4). https://doi.org/10.1002/widm.1249 .

Download references

Acknowledgements

The authors thank the reviewers and the Editor for their insightful comments and suggestions, which have greatly improved the quality of this work.

Not applicable.

Author information

Authors and affiliations.

Department of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, NC, 27599, USA

Michael Li & Xianming Tan

Department of Mathematics and Statistics, University of North Carolina at Greensboro, Greensboro, NC, 27402, USA

Jianping Sun

Department of Biostatistics and Lineberger Comprehensive Cancer Center, University of North Carolina at Chapel Hill, Chapel Hill, NC, 27599, USA

Xianming Tan

You can also search for this author in PubMed   Google Scholar

Contributions

XT conceived the paper. ML and XT devised the study. ML, JS, and XT analyzed the datasets. ML, JS, and XT wrote the manuscript. All authors edited and approved the final manuscript.

Corresponding author

Correspondence to Xianming Tan .

Ethics declarations

Ethics approval and consent to participate, competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1: table s1. tripod + ai checklist., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Li, M., Sun, J. & Tan, X. Evaluating the effectiveness of large language models in abstract screening: a comparative analysis. Syst Rev 13 , 219 (2024). https://doi.org/10.1186/s13643-024-02609-x

Download citation

Received : 20 December 2023

Accepted : 11 July 2024

Published : 21 August 2024

DOI : https://doi.org/10.1186/s13643-024-02609-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Abstract screening
• Automated text analysis
• Google PaLM

Systematic Reviews

ISSN: 2046-4053

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

• Open access
• Published: 21 August 2024

An analysis of the psychometric properties of the medication safety competence scale in Turkish

• Ayşe Aydinli 1 &
• Kamuran Cerit 2  

BMC Nursing volume  23 , Article number:  578 ( 2024 ) Cite this article

86 Accesses

Metrics details

Considering the key roles and responsibilities of nurses in ensuring medication safety, it is necessary to understand nurses’ competence in medication safety. Therefore, it was aimed to introduce a scale evaluating the medication safety competence of nurses into Turkish and to contribute to the literature by determining the medication safety competence levels of nurses.

A methodological and descriptive research design was utilised. The population consisted of nurses in Turkey, and the sample comprised 523 nurses who volunteered to participate.

The content validity index of the scale was 0.98, and the scale showed a good fit (χ 2 /df = 3.00, RMSEA = 0.062). The Cronbach’s alpha coefficient of the scale was 0.97, indicating high reliability. The mean score was 4.12, which was considered high. Participants who were 40 years old or above, married, and graduates of health vocational schools or postgraduate programs, along with those who had received medication safety training, had higher medication safety competence scores.

This study presents strong evidence that the Turkish version of the Medication Safety Competency Scale is valid and reliable when administered to nurses. The participants in this study had high levels of medication safety competence.

Peer Review reports

Introduction

A medication error is defined as a preventable event at any stage of drug therapy that results in incorrect drug use or harms the patient [ 1 ]. According to World Health Organization (WHO) reports, medication errors account for 20% of medical errors [ 2 ]. Medication errors decrease patients’ quality of life and result in high costs for healthcare institutions [ 3 ]. They can also lead to complications, permanent disability, and death [ 1 ]. Globally, concerns about medication errors are increasing, and various reports have emphasised the importance of reducing medication errors and improving patient safety [ 2 , 4 , 5 ].

Patient safety involves ensuring that patients are not harmed while receiving care, and medication safety is among the most important elements of patient safety [ 6 ]. Medication safety can be defined as ensuring that medications have the maximum therapeutic effect while minimising and preventing adverse reactions and accidental injuries during medication use [ 4 ]. Medication safety is a multidisciplinary and multi-stage process. Nurses constitute the majority of the healthcare team and are involved in many stages of the medication administration process; they are at the centre of medication administration and are involved in the most critical stage when any potential errors reach the patient [ 7 ].

Traditional nursing curricula consider the “right principles” as a basic standard for safe medication practices. However, nurses’ role in ensuring medication safety encompasses many other principles [ 3 ]. Limiting nurses’ responsibilities regarding medication safety to the right principles does not address all aspects of errors [ 8 ]. Medication safety requires nurses to use clinical judgment before, during, and after interventions. Nurses’ experience and knowledge are integral components of safe medication management in nursing practice [ 9 ]. Adverse effects caused by improper prescription, administration, or monitoring of medications can be decreased through good nursing practice [ 8 ].

Considering the high prevalence of medication errors and the key role of nurses in ensuring medication safety, the medication safety competence of nurses must be determined [ 4 ]. However, very few studies have assessed nurses’ medication safety competence [ 3 , 4 , 5 , 6 ]. Moreover, a Turkish scale to assess nurses’ the of and attitudes towards medication safety competence is needed.

Therefore, this study introduced a Turkish version of a scale evaluating the medication safety competence of nurses and administered it to nurses, contributing to the literature by determining the medication safety competence levels of nurses.

This study adapted the Medication Safety Competence Scale (MSCS) developed by Park and Seomun (2021) for use in a Turkish context and assessed its validity and reliability; subsequently, the scale was used to determine nurses’ the of medication safety competence [ 5 ]. Differences in the of medication safety competence between nurses with varying demographic characteristics were also investigated.

Study design and participants

This study was conducted methodologically and descriptively. This study was structured and reported according to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) checklist [ 10 ].

The population of the study comprised nurses working at various hospitals in Turkey. Convenience sampling method was used. The sample consisted of 523 nurses who were native Turkish speakers and voluntarily participated in the study. No one refused to participate in the study. There were no missing data except for 36 data related only to the worked clinic was conducted. However, these 36 data were not excluded; instead, analyses related to the clinic variable were conducted using 487 data (Tables  1 and 2 ).

The sample size for conducting CFA was determined to be at least 10 times the number of scale items [ 11 ]. This rule was met for 36 items with 523 cases. In descriptive analyses, the sufficiency of the sample size was determined by a post hoc power analysis. As a result of the post hoc power analysis conducted with G*Power 3.1.9.7, the power of the study was calculated as 85% with an effect size of 0.26 and a significance level of 0.05 [ 12 ].

Data collection

A demographic information form was used to determine the demographic characteristics of the nurses, and the MSCS was used to determine their the of medication safety competence.

The demographic information form consists of 10 questions. The form, which was prepared by the researchers in line with the literature, consists of questions with demographic characteristics of the nurses such as age, gender and worked clinics [ 3 , 4 , 5 , 6 ].

The MSCS was developed by Park and Seomun in 2021 [ 5 ]. The scale consists of 36 items divided into six subdimensions. The dimensions are patient-centred medication management (Items 1,4,5, 6,7,8, 13,24,26); multidisciplinary collaboration (Items 20,27,30,33); safety risk management (Items 2,15,16,21,25,28); management of effecting factors (Items 3,9,11,12,14,18); improvement of safety problems (Items 10,19,22,29,31,32,34,35); and responsibility in the nursing profession (Items 17,23,36). The scale is a five-point Likert scale. The total score ranges from 36 to 180. Scores between 36 and 75 represent poor medication safety competence, scores between 76 and 130 indicate moderate medication safety competence, and a score of 180 represents high medication safety competence [ 6 ].

Data were collected through an online survey between February 1 and March 31, 2023. Participants were reached via social media (WhatsApp, Instagram story, etc.) and invited to participate in the online survey prepared through Google Forms. An informed consent form was attached to the first part of the questionnaire and participation was voluntary. The response time of the questionnaires was 8–10 min.

This study was conducted in two stages: methodological and descriptive.

Methodological stage

This stage consisted of translation and psychometric testing. To adapt the scale for use in a Turkish context, permission was obtained from the researchers who developed the original scale. Back translation was used for language validity. The content validity of the Turkish version of the scale was tested. After receiving expert opinions, the scale was translated back into English (supplementary file- 1 ). After the translated version was sent to the researchers who developed the scale and approval was obtained, data collection was started with the Turkish form. Data were then collected for psychometric testing. The validity and reliability of the original scale were tested with data from the 523 participants.

Descriptive stage

In this stage, nurses’ the of medication safety competence were determined and analysed according to several demographic characteristics. Data from the 523 participants were used in this stage.

Data Analysis

Data were analysed with the Statistical Package for the Social Sciences (SPSS) v. 26.0, LISREL v. 8.80, and Microsoft Excel.

Descriptive statistics

Means and standard deviations were calculated for continuous data, and percentages were calculated for categorical data. The adequacy of the multivariate normal distribution of the data was assessed using Mardia’s skewness and kurtosis tests. Mann–Whitney U and Kruskal–Wallis tests were used for comparison between groups of nurses with different demographic characteristics.

Item analysis

To determine whether the scale had an ideal discrimination ability, the total score of the scale was ranked from high to low, and the difference between the first 27% and the last 27% was analysed. Furthermore, item-total score correlation coefficients were calculated.

Validity analysis

Eleven experts rated the items of the adapted scale from 1 to 4 for content validity (1: not appropriate; 2: partially appropriate, the item needs to be revised; 3: appropriate but minor changes are needed; 4 very appropriate). The item content validity index (I-CVI) and the scale content validity index (S-CVI) were calculated using the method proposed by Davis (1992) [ 13 ]. The I-CVI is the ratio of the number of experts who assign each item 3 or 4 points to the total number of experts. The S-CVI is the average I-CVI for all items.

Confirmatory factor analysis (CFA) was applied for the construct validity, and fit indices were evaluated. The values of chi-square (χ 2 )/degree of freedom (df), comparative fit index (CFI), root-mean-square error of approximation (RMSEA), non-normed fit index (NNFI), normed fit index (NFI), standardised root mean square residual (SRMR), root mean square residual (RMR), goodness of fit index (GFI), and adjusted goodness of fit index (AGFI) were examined. Also average variance extracted (AVE) and Construct reliability (CR) were examined for convergent validity.

Reliability analysis

Cronbach’s alpha (α) and split-half reliability were calculated to assess internal consistency. To assess test–retest reliability, intra-class correlation (ICC) was calculated by collecting data from 30 nurses at 2-week intervals. The data obtained for the test-retest were not included in the sample.

Ethical considerations

Ethical approval.

of the study was obtained from the Suleyman Demirel University Institutional Ethics Committee (decision number: 87432956-050.99-423263). Informed consent was obtained from the participating nurses in the first part of the online survey. The study was carried out in accordance with the principles of the Declaration of Helsinki.

Most participants (94.5%) were female, 61.2% were between the ages of 20–29 years, 53.7% were married, 72.7% had a Bachelor’s degree, and 54.9% had 1–5 years of professional experience. Furthermore, 79.2% of the participants reported that they had received training or courses on medication safety, 79.5% reported that medication administration principles were followed in the clinic where they worked, and 76.7% stated that medication administration was performed following the hospital’s medication administration rules and procedures (Table  1 ). The results of the methodological and descriptive stages of the study are provided in the following two sections.

Results of the methodological stage

The scale was translated into Turkish by four translators who were native Turkish speakers and fluent in English. The researchers then combined the four translations into a single form. In the second step, this form was translated back into English by an expert who was not one of the previous translators.

The expert review was conducted by nine nursing instructors and two nurses with master’s degrees. The I-CVI ranged from 0.80 to 1.00, and the S-CVI was 0.98. To assess the face validity of the Turkish form, a preliminary application was performed with 20 nurses. To ensure that they were comprehensible in Turkish, the expression “human factors” in the item “Understanding the role of human factors, such as fatigue, that affect medication safety” was changed to “personal factors”, and the expression “understanding the role” in “Understanding the role of environmental factors such as workflow, ergonomics, and resources, which affect medication safety” was changed to “understanding the effect”. With these adjustments, the scale form adapted to Turkish was finalised. Data from the pilot study were not included in the sample.

The results of CFA are shown in Table  3 . The fitness indices of the original scale (model 1) (χ 2 /df = 1921.97/579 = 3.32, RMSEA = 0.067, CFI = 0.98) were determined to be at an acceptable level (Fig.  1 ). However, modification indices were examined, and the original scale was modified sequentially as follows: item 28 and item 31, item 25 and item 31, item 10 and item 11, respectively. The modification of model 2 was achieved by freeing the error terms (permitting correlated errors) of the items without excluding any items (Fig.  2 ).

CFA results of Model I: standard loadings and error variances

CFA results of Model II: standard loadings and error variances

The fitness indices of the final scale (model 2) were as follows: χ 2 /df = 3.00, RMSEA = 0.062, CFI = 0.99, NFI = 0.98, GFI = 0.78, AGFI = 0.74. The standard loadings of the items in the final scale ranged from 0.46 to 0.83. The squared multiple correlations (SMC-R 2 ) ranged from 0.21 to 0.68 (Table  4 ). Furthermore, the fit indices attained in both models were acceptable. Nevertheless, model 2 exhibited superior χ2/df and RMSEA values, indicating that it outperformed model 1.

For convergent validity, the CRs ranged from 0.68 to 0.79 and were higher than the AVE values (0.40 to 0.60). The correlation between factors ranged from 0.687 to 0.868 (Table  5 ). Item analyses revealed that the item-total correlations were between 0.46 and 0.77 (Table  4 ). A statistically significant difference (t: −30.601, p  < 0.001) was observed between the mean scores of the groups with the lowest 27% of scores and the highest 27% of scores.

As shown in Table  5 , the Cronbach’s alpha coefficient of the scale was 0.97. The split-half reliability was 0.912, and the test–retest reliability (ICC) was 0.939.

Result of the descriptive stage

The participants’ total mean score of MSCS was 147.81 ± 21.29, indicating a average level of medication safety competence. The lowest score was obtained in the RNP dimension (11.88 ± 2.05 points) and the highest score was obtained in the PCMM dimension (38.21 ± 5.14 points) (Table  5 ). Table  2 presents the comparison of participants’ the of medication safety competence according to their demographic characteristics.

No differences were observed in the participants’ the of medication safety competence according to gender and the type of clinic they worked in. However, significant differences were found according to age, marital status, educational level, professional experience, and the type of hospital in which the nurses worked. Participants aged 40 years and older had higher the of medication safety competence than those aged 20–29 years; married participants had higher than single participants; and those with health vocational school and postgraduate degrees had higher than those with undergraduate degrees. In addition, nurses who received training or courses on medication safety, those who thought that the principles of medication administration were followed in the clinic where they worked, and those who thought that medication administration was performed following the hospital’s medication administration rules and procedures had higher the of medication safety competence ( p  < 0.05).

Discussion of the Methodological Stage

This study was conducted to determine the validity and reliability of a Turkish version of the MSCS. This study found that the Turkish version of the MSC scale meets the criteria of language validity, content validity, construct validity, and reliability. The validity and reliability of the scale have also been confirmed in Chinese and Persian [ 4 , 6 ].

In this study, the Turkish version of the scale was created using the back translation method for language validity. Then the content validity of the scale was evaluated according to Davis’s (1992) technique [ 13 ]. Because the CVI values of all items were above 0.80, no items were removed at this stage. CFA was performed to verify construct validity. Two models were analysed: the original scale (model 1) and the final scale (model 2). According to the factor loadings and modification indices of model 1 and model 2, the measurement validity of the Turkish version of the MSCS was confirmed. Thus, no items were removed from the scale at this stage.

The χ 2 /df of the final scale was 3.00; this value is considered acceptable, as it is less than 5 [ 14 ]. The RMSEA value of 0.062 is an important indicator of the acceptable fit of the final scale. The RMR (0.042) and SRMR (0.057) values showed perfect and acceptable fit, respectively [ 15 ]. The CFI was above 0.95, indicating perfect fit [ 16 ]. The NFI and NNFI were also above 0.95, indicating perfect fit [ 17 , 18 ]. In CFA, it is recommended that the factor loadings of the items factors should be above 0.50 [ 10 ]. In this study, the factor loadings of the items were between 0.46 and 0.83 (Table  4 ). Thus, the fit indices and item factor loadings confirmed the construct validity of the final scale. The AVEs of the factors were higher than 0.50 in all subdimensions except for RNP and MEF. Moreover, the CRs were between 0.68 and 0.79, higher than the recommended value of 0.70 and the AVE values (Table  5 ). These results confirmed the scale’s convergent validity.

Finally, item analyses showed that the items in the scale had good discrimination. Similarly, the Chinese version [ 4 ], the original scale [ 5 ] and the Persian version [ 6 ] also reported high factor loadings, acceptable fit indices and CRs above 0.70. In line with these results, it can be said that the scale provides an adequate level of validity.

Reliability

The reliability of the scale was evaluated by calculating split-half reliability, Cronbach’s alpha, CR and ICC. Cronbach’s alpha value should be at least 0.70 [ 11 , 19 ]. In the present study, Cronbach’s alpha was 0.97, whereas it was 0.94 in the Chinese version [ 4 ], 0.94 in the original scale [ 5 ], and 0.96 in the Persian version [ 6 ]. The CR values reported as more appropriate reliability measures for CFA-based studies [ 11 ] are higher than the proposed value of 0.70. These results show that the scale has good internal reliability in samples from different cultures. In this study, the split-half reliability of the scale was 0.912, and the test–retest reliability was 0.939. The split-half reliability of the Chinese version of the scale was 0.671, and the test–retest reliability was 0.703 [ 4 ]. For the Persian version, the test–retest reliability was also 0.90 [ 6 ]. These values indicate the stability of the various versions of the scale [ 20 ].

Discussion of the descriptive phase

To date, nurses’ medication safety competency has generally been examined within the framework of the right principles [ 21 , 22 ] or reporting medication errors [ 23 , 24 ]. However, medication safety is a concept that transcends the right principles [ 3 , 8 ], and previous studies measuring nurses’ medication safety competency have been insufficient. The medication safety competence scores of the nurses in this study were average. Mohebi et al. (2024) also reported the medication safety competence of nursing students at an average level. These results could indicate that the medication safety competence of nurses and students were adequate but needed further development [ 25 ]. These results may be explained by the fact that most of the nurses in the study received training or courses on medication safety. Moreover, in recent years, in Turkey and other countries, patient and medication safety issues have been important issues of health policies and hospitals in Turkey. Although the issue is important, a study conducted in Turkey found that almost half of the nurses reported that no institutional procedures were in place for medication safety in hospitals [ 26 ].

In this study nurses’ the of medication safety competence differ according to age, marital status, education level, professional experience, hospital type, and the types of training or courses on medication safety the nurses have received. More studies investigating nurses’ the of medication safety competence are warranted, along with studies assessing differences in the of medication safety competence between nurses with varying demographic characteristics. This study provides strong evidence for the reliability and validity of the scale in Turkish. It is also the first study to determine the medication safety competencies of nurses working in Turkey.

This study presents strong evidence that the Turkish version of the MSCS is valid and reliable among nurses. The medication safety competency levels of the nurses participating in this study were average. The assessment results of the scale provide a reference for nursing administrators to help them formulate educational plans improve the medication safety competence of nurses.

Limitations

One of the strengths of this study is its application of the scale to a large sample. Although the methodological results are important, the adapted scale is specific to nurses in Turkey. However, the results regarding nurses’ medication safety competence obtained in the study’s second stage provide a substantial contribution to the literature.

Data availability

Data Availability StatementThe data supporting this study’s findings are available on re-quest from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.

Mendes JR, Lopes MCBT, Vancini-Campanharo CR, Okuno MFP, Batista REA. Types and frequency of errors in the preparation and administration of drugs. Einstein (Sao Paulo). 2018;16(3):eAO4146. https://doi.org/10.1590/S1679-45082018AO4146 .

Article   PubMed   Google Scholar  

WHO. WHO launches global effort to halve medication-related errors in 5 years; 2017 [cited Mary 17, 2023]. http://www.who.int/mediacentre/news/releases/2017/medication-related-errors/en/

Choo J, Hutchinson A, Bucknall T. Nurses’ role in medication safety. J Nurs Manag. 2010;18(7):853–61. https://doi.org/10.1111/j.1365-2834.2010.01164.x .

Yang Z, Chen F, Lu Y, Zhang H. Psychometric evaluation of medication safety competence scale for clinical nurses. BMC Nurs. 2021;9(1):20. https://doi.org/10.1186/s12912-021-00679-z .

Article   Google Scholar  

Park J, Seomun G. Development and validation of the Medication Safety competence scale for nurses. West J Nurs Res. 2021;43(7):686–97. https://doi.org/10.1177/0193945920969929 .

Mohammadi F, Kouhpayeh SA, Bijani M, Farjam M, Faghihi A, Badiyepeymaiejahromi Z. Translation and psychometric assessment of a persian version of medication safety competence scale (MSCS) for clinical nurses. Sci Rep. 2023;8(1):13. https://doi.org/10.1038/s41598-023-29399-x .

Article   CAS   Google Scholar  

Adhikari R, Tocher J, Smith P, Corcoran J, MacArthur J. A multi-disciplinary approach to medication safety and the implication for nursing education and practice. Nurse Educ Today. 2014;34(2):185–90. https://doi.org/10.1016/j.nedt.2013.10.008 .

Vaismoradi M, Griffiths P, Turunen H, Jordan S. Transformational leadership in nursing and medication safety education: a discussion paper. J Nurs Manag. 2016;24(7):970–80. https://doi.org/10.1111/jonm.12387 .

Babaoğlu AB, Tekindal M, Büyükuysal MÇ, Tözün M, Elmalı F, Bayraktaroğlu T. Reporting of observational studies in epidemiology: Turkish adaptation of STROBE Criteria. Med J West Black Sea. 2021;5(1):86–93.

Google Scholar  

Rohde E, Domm E. Nurses’ clinical reasoning practices that support safe medication administration: an integrative review of the literature. J Clin Nurs. 2018;27(3–4):402–11. https://doi.org/10.1111/jocn.14077 .

Hair JF, Black WC, Babin B, Anderson RE, Tatham R. Multivariate Data Analysis: Cengage (8th Ed.). 2018.

Faul F, Erdfelder E, Lang AG, Buchner A. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods. 2007;39(2):175–91. https://doi.org/10.3758/BF03193146 .

Davis LL. Instrument review: getting the most from a panel of experts. Appl Nurs Res. 1992;5(4):194–7. https://doi.org/10.1016/s0897-1897(05)80008-4 .

Erkorkmaz U, Etikan I, Demir O, Ozdamar K, Sanisoglu SY. Confirmatory factor analysis and fit indices: review. Turkiye Klinikleri J Med Sci. 2013;33(1):210–23. https://doi.org/10.5336/medsci.2011-26747 .

Schermelleh-Engel K, Moosbrugger H, Müller H. Evaluating the fit of structural equation models: test of sig- nificance and descriptive goodness-of-fit measures. Methods Psychol Research-Online. 2003;8(2):23–74.

Karagöz Y. SPSS ve AMOS uygulamalı nicel-nitel-karma bilimsel araştırma yöntemleri ve yayın etiği. Sivas: Nobel Akademik Yayıncılık Eğitim Danışmanlık, 2017:24–551.

Schumacker RE, Lomax RG. A beginner’s guide to structural equation modelling. Mahwah, NJ: Lawrence Erlbaum Associates; 1996.

Tabachnick BG, Fidell LS. Experimental Designs Using ANOVA. Thomson/Brooks/Cole. 2007.

DeVellis RF. Scale Development: theory and applications. Sage Publications, 2017;31–5.

Koo TK, Li MY. A guideline of selecting and reporting ıntraclass correlation coefficients for reliability research. J Chiropr Med. 2017;16(4):346. https://doi.org/10.1016/j.jcm.2016.02.012 .

Martyn JA, Paliadelis P, Perry C. The safe administration of medication: nursing behaviours beyond the five-rights. Nurse Educ Pract 2019 May;37:109–14. https://doi.org/10.1016/j.nepr.2019.05.006

Hanson A, Haddad LM. Nursing rights of medication administration. StatPearls Publishing. 2021. https://www.ncbi.nlm.nih.gov/books/NBK560654/

Dirik HF, Samur M, Seren Intepeler S, Hewison A. Nurses’ identification and reporting of medication errors. J Clin Nurs. 2019;28(5–6):931–8. https://doi.org/10.1111/jocn.14716 .

Hung CC, Chu TP, Lee BO, Hsiao CC. Nurses’ attitude and intention of medication administration error reporting. J Clin Nurs. 2016;25(3–4):445–53. https://doi.org/10.1111/jocn.13071 .

Mohebi Z, Bijani M, Dehghan A. Investigating safe nursing care and medication safety competence in nursing students: a multicenter cross-sectional study in Iran. BMC Nurs. 2024;23(1):13. https://doi.org/10.1186/s12912-023-01684-0 . Published 2024 Jan 2.

Article   PubMed   PubMed Central   Google Scholar  

Güneş ÜY, Gürlek Ö, Sönmez M. Factors contributing to medication errors in Turkey: nurses’ perspectives. J Nurs Manag. 2014;22(3):295–303. https://doi.org/10.1111/jonm.12216 .

Download references

Acknowledgements

We thank the nurses who participated in this study and the experts who suggested content validity.

No funding has been received.

Author information

Authors and affiliations.

Department of Fundamental Nursing, Faculty of Health Sciences, Suleyman Demirel University, Isparta, Turkey

Ayşe Aydinli

Department of Nursing Management, Faculty of Health Sciences, Suleyman Demirel University, Isparta, Turkey

Kamuran Cerit

You can also search for this author in PubMed   Google Scholar

Contributions

AA and KC contributed to the study aim, research design, and overall structure of the manuscript. AA collected the data. KC conducted all the statistical analyses and drafted the manuscript. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Ayşe Aydinli .

Ethics declarations

Consent for publication.

Not applicable.

Competing interests

The authors declare no competing interests.

Declaration of interest

The authors declare that they have no known competing financial interests or personal relationships that could have influenced the work reported in this paper. No other potential conflicts of interest relevant to this study were reported.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Aydinli, A., Cerit, K. An analysis of the psychometric properties of the medication safety competence scale in Turkish. BMC Nurs 23 , 578 (2024). https://doi.org/10.1186/s12912-024-02240-0

Download citation

Received : 19 May 2024

Accepted : 07 August 2024

Published : 21 August 2024

DOI : https://doi.org/10.1186/s12912-024-02240-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Patient safety
• Clinical competence
• Reliability and validity

BMC Nursing

ISSN: 1472-6955

• General enquiries: [email protected]