aim for literature review

• Link to facebook
• Link to linkedin
• Link to twitter
• Link to youtube
• Writing Tips

What is the Purpose of a Literature Review?

4-minute read

• 23rd October 2023

If you’re writing a research paper or dissertation , then you’ll most likely need to include a comprehensive literature review . In this post, we’ll review the purpose of literature reviews, why they are so significant, and the specific elements to include in one. Literature reviews can:

1. Provide a foundation for current research.

2. Define key concepts and theories.

3. Demonstrate critical evaluation.

4. Show how research and methodologies have evolved.

5. Identify gaps in existing research.

6. Support your argument.

Keep reading to enter the exciting world of literature reviews!

What is a Literature Review?

A literature review is a critical summary and evaluation of the existing research (e.g., academic journal articles and books) on a specific topic. It is typically included as a separate section or chapter of a research paper or dissertation, serving as a contextual framework for a study. Literature reviews can vary in length depending on the subject and nature of the study, with most being about equal length to other sections or chapters included in the paper. Essentially, the literature review highlights previous studies in the context of your research and summarizes your insights in a structured, organized format. Next, let’s look at the overall purpose of a literature review.

Find this useful?

Subscribe to our newsletter and get writing tips from our editors straight to your inbox.

Literature reviews are considered an integral part of research across most academic subjects and fields. The primary purpose of a literature review in your study is to:

Provide a Foundation for Current Research

Since the literature review provides a comprehensive evaluation of the existing research, it serves as a solid foundation for your current study. It’s a way to contextualize your work and show how your research fits into the broader landscape of your specific area of study.  

Define Key Concepts and Theories

The literature review highlights the central theories and concepts that have arisen from previous research on your chosen topic. It gives your readers a more thorough understanding of the background of your study and why your research is particularly significant .

Demonstrate Critical Evaluation 

A comprehensive literature review shows your ability to critically analyze and evaluate a broad range of source material. And since you’re considering and acknowledging the contribution of key scholars alongside your own, it establishes your own credibility and knowledge.

Show How Research and Methodologies Have Evolved

Another purpose of literature reviews is to provide a historical perspective and demonstrate how research and methodologies have changed over time, especially as data collection methods and technology have advanced. And studying past methodologies allows you, as the researcher, to understand what did and did not work and apply that knowledge to your own research.  

Identify Gaps in Existing Research

Besides discussing current research and methodologies, the literature review should also address areas that are lacking in the existing literature. This helps further demonstrate the relevance of your own research by explaining why your study is necessary to fill the gaps.

Support Your Argument

A good literature review should provide evidence that supports your research questions and hypothesis. For example, your study may show that your research supports existing theories or builds on them in some way. Referencing previous related studies shows your work is grounded in established research and will ultimately be a contribution to the field.  

Literature Review Editing Services 

Ensure your literature review is polished and ready for submission by having it professionally proofread and edited by our expert team. Our literature review editing services will help your research stand out and make an impact. Not convinced yet? Send in your free sample today and see for yourself! 

Share this article:

Post A New Comment

Got content that needs a quick turnaround? Let us polish your work. Explore our editorial business services.

5-minute read

Free Email Newsletter Template (2024)

Promoting a brand means sharing valuable insights to connect more deeply with your audience, and...

6-minute read

How to Write a Nonprofit Grant Proposal

If you’re seeking funding to support your charitable endeavors as a nonprofit organization, you’ll need...

9-minute read

How to Use Infographics to Boost Your Presentation

Is your content getting noticed? Capturing and maintaining an audience’s attention is a challenge when...

8-minute read

Why Interactive PDFs Are Better for Engagement

Are you looking to enhance engagement and captivate your audience through your professional documents? Interactive...

7-minute read

Seven Key Strategies for Voice Search Optimization

Voice search optimization is rapidly shaping the digital landscape, requiring content professionals to adapt their...

Five Creative Ways to Showcase Your Digital Portfolio

Are you a creative freelancer looking to make a lasting impression on potential clients or...

Make sure your writing is the best it can be with our expert English proofreading and editing.

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

The objective of a literature review

Questions to Consider

B. In some fields or contexts, a literature review is referred to as the introduction or the background; why is this true, and does it matter?

The elements of a literature review • The first step in scholarly research is determining the “state of the art” on a topic. This is accomplished by gathering academic research and making sense of it. • The academic literature can be found in scholarly books and journals; the goal is to discover recurring themes, find the latest data, and identify any missing pieces. • The resulting literature review organizes the research in such a way that tells a story about the topic or issue.

The literature review tells a story in which one well-paraphrased summary from a relevant source contributes to and connects with the next in a logical manner, developing and fulfilling the message of the author. It includes analysis of the arguments from the literature, as well as revealing consistent and inconsistent findings. How do varying author insights differ from or conform to previous arguments?

Language in Action

A. How are the terms “critique” and “review” used in everyday life? How are they used in an academic context?

In terms of content, a literature review is intended to:

• Set up a theoretical framework for further research • Show a clear understanding of the key concepts/studies/models related to the topic • Demonstrate knowledge about the history of the research area and any related controversies • Clarify significant definitions and terminology • Develop a space in the existing work for new research

The literature consists of the published works that document a scholarly conversation or progression on a problem or topic in a field of study. Among these are documents that explain the background and show the loose ends in the established research on which a proposed project is based. Although a literature review focuses on primary, peer -reviewed resources, it may begin with background subject information generally found in secondary and tertiary sources such as books and encyclopedias. Following that essential overview, the seminal literature of the field is explored. As a result, while a literature review may consist of research articles tightly focused on a topic with secondary and tertiary sources used more sparingly, all three types of information (primary, secondary, tertiary) are critical.

The literature review, often referred to as the Background or Introduction to a research paper that presents methods, materials, results and discussion, exists in every field and serves many functions in research writing.

Adapted from Frederiksen, L., & Phelps, S. F. (2017). Literature Reviews for Education and Nursing Graduate Students. Open Textbook Library

Review and Reinforce

Two common approaches are simply outlined here. Which seems more common? Which more productive? Why? A. Forward exploration 1. Sources on a topic or problem are gathered. 2. Salient themes are discovered. 3. Research gaps are considered for future research. B. Backward exploration 1. Sources pertaining to an existing research project are gathered. 2. The justification of the research project’s methods or materials are explained and supported based on previously documented research.

Media Attributions

• 2589960988_3eeca91ba4_o © Untitled blue is licensed under a CC BY (Attribution) license

Sourcing, summarizing, and synthesizing:  Skills for effective research writing  Copyright © 2023 by Wendy L. McBride is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

What is a Literature Review? How to Write It (with Examples)

A literature review is a critical analysis and synthesis of existing research on a particular topic. It provides an overview of the current state of knowledge, identifies gaps, and highlights key findings in the literature. 1 The purpose of a literature review is to situate your own research within the context of existing scholarship, demonstrating your understanding of the topic and showing how your work contributes to the ongoing conversation in the field. Learning how to write a literature review is a critical tool for successful research. Your ability to summarize and synthesize prior research pertaining to a certain topic demonstrates your grasp on the topic of study, and assists in the learning process. 

Table of Contents

• What is the purpose of literature review? 
• a. Habitat Loss and Species Extinction: 
• b. Range Shifts and Phenological Changes: 
• c. Ocean Acidification and Coral Reefs: 
• d. Adaptive Strategies and Conservation Efforts: 

How to write a good literature review 

• Choose a Topic and Define the Research Question: 
• Decide on the Scope of Your Review: 
• Select Databases for Searches: 
• Conduct Searches and Keep Track: 
• Review the Literature: 
• Organize and Write Your Literature Review: 
• How to write a literature review faster with Paperpal? 
• Frequently asked questions 

What is a literature review?

A well-conducted literature review demonstrates the researcher’s familiarity with the existing literature, establishes the context for their own research, and contributes to scholarly conversations on the topic. One of the purposes of a literature review is also to help researchers avoid duplicating previous work and ensure that their research is informed by and builds upon the existing body of knowledge.

What is the purpose of literature review?

A literature review serves several important purposes within academic and research contexts. Here are some key objectives and functions of a literature review: 2  

1. Contextualizing the Research Problem: The literature review provides a background and context for the research problem under investigation. It helps to situate the study within the existing body of knowledge. 

2. Identifying Gaps in Knowledge: By identifying gaps, contradictions, or areas requiring further research, the researcher can shape the research question and justify the significance of the study. This is crucial for ensuring that the new research contributes something novel to the field. 

Find academic papers related to your research topic faster. Try Research on Paperpal  

3. Understanding Theoretical and Conceptual Frameworks: Literature reviews help researchers gain an understanding of the theoretical and conceptual frameworks used in previous studies. This aids in the development of a theoretical framework for the current research. 

4. Providing Methodological Insights: Another purpose of literature reviews is that it allows researchers to learn about the methodologies employed in previous studies. This can help in choosing appropriate research methods for the current study and avoiding pitfalls that others may have encountered. 

5. Establishing Credibility: A well-conducted literature review demonstrates the researcher’s familiarity with existing scholarship, establishing their credibility and expertise in the field. It also helps in building a solid foundation for the new research. 

6. Informing Hypotheses or Research Questions: The literature review guides the formulation of hypotheses or research questions by highlighting relevant findings and areas of uncertainty in existing literature. 

Literature review example

Let’s delve deeper with a literature review example: Let’s say your literature review is about the impact of climate change on biodiversity. You might format your literature review into sections such as the effects of climate change on habitat loss and species extinction, phenological changes, and marine biodiversity. Each section would then summarize and analyze relevant studies in those areas, highlighting key findings and identifying gaps in the research. The review would conclude by emphasizing the need for further research on specific aspects of the relationship between climate change and biodiversity. The following literature review template provides a glimpse into the recommended literature review structure and content, demonstrating how research findings are organized around specific themes within a broader topic. 

Literature Review on Climate Change Impacts on Biodiversity:

Climate change is a global phenomenon with far-reaching consequences, including significant impacts on biodiversity. This literature review synthesizes key findings from various studies: 

a. Habitat Loss and Species Extinction:

Climate change-induced alterations in temperature and precipitation patterns contribute to habitat loss, affecting numerous species (Thomas et al., 2004). The review discusses how these changes increase the risk of extinction, particularly for species with specific habitat requirements. 

b. Range Shifts and Phenological Changes:

Observations of range shifts and changes in the timing of biological events (phenology) are documented in response to changing climatic conditions (Parmesan & Yohe, 2003). These shifts affect ecosystems and may lead to mismatches between species and their resources. 

c. Ocean Acidification and Coral Reefs:

The review explores the impact of climate change on marine biodiversity, emphasizing ocean acidification’s threat to coral reefs (Hoegh-Guldberg et al., 2007). Changes in pH levels negatively affect coral calcification, disrupting the delicate balance of marine ecosystems. 

d. Adaptive Strategies and Conservation Efforts:

Recognizing the urgency of the situation, the literature review discusses various adaptive strategies adopted by species and conservation efforts aimed at mitigating the impacts of climate change on biodiversity (Hannah et al., 2007). It emphasizes the importance of interdisciplinary approaches for effective conservation planning. 

Strengthen your literature review with factual insights. Try Research on Paperpal for free!    

Writing a literature review involves summarizing and synthesizing existing research on a particular topic. A good literature review format should include the following elements. 

Introduction: The introduction sets the stage for your literature review, providing context and introducing the main focus of your review. 

• Opening Statement: Begin with a general statement about the broader topic and its significance in the field. 
• Scope and Purpose: Clearly define the scope of your literature review. Explain the specific research question or objective you aim to address. 
• Organizational Framework: Briefly outline the structure of your literature review, indicating how you will categorize and discuss the existing research. 
• Significance of the Study: Highlight why your literature review is important and how it contributes to the understanding of the chosen topic. 
• Thesis Statement: Conclude the introduction with a concise thesis statement that outlines the main argument or perspective you will develop in the body of the literature review. 

Body: The body of the literature review is where you provide a comprehensive analysis of existing literature, grouping studies based on themes, methodologies, or other relevant criteria. 

• Organize by Theme or Concept: Group studies that share common themes, concepts, or methodologies. Discuss each theme or concept in detail, summarizing key findings and identifying gaps or areas of disagreement. 
• Critical Analysis: Evaluate the strengths and weaknesses of each study. Discuss the methodologies used, the quality of evidence, and the overall contribution of each work to the understanding of the topic. 
• Synthesis of Findings: Synthesize the information from different studies to highlight trends, patterns, or areas of consensus in the literature. 
• Identification of Gaps: Discuss any gaps or limitations in the existing research and explain how your review contributes to filling these gaps. 
• Transition between Sections: Provide smooth transitions between different themes or concepts to maintain the flow of your literature review. 

Write and Cite as you go with Paperpal Research. Start now for free.   

Conclusion: The conclusion of your literature review should summarize the main findings, highlight the contributions of the review, and suggest avenues for future research. 

• Summary of Key Findings: Recap the main findings from the literature and restate how they contribute to your research question or objective. 
• Contributions to the Field: Discuss the overall contribution of your literature review to the existing knowledge in the field. 
• Implications and Applications: Explore the practical implications of the findings and suggest how they might impact future research or practice. 
• Recommendations for Future Research: Identify areas that require further investigation and propose potential directions for future research in the field. 
• Final Thoughts: Conclude with a final reflection on the importance of your literature review and its relevance to the broader academic community. 

Conducting a literature review

Conducting a literature review is an essential step in research that involves reviewing and analyzing existing literature on a specific topic. It’s important to know how to do a literature review effectively, so here are the steps to follow: 1  

Choose a Topic and Define the Research Question:

• Select a topic that is relevant to your field of study. 
• Clearly define your research question or objective. Determine what specific aspect of the topic do you want to explore? 

Decide on the Scope of Your Review:

• Determine the timeframe for your literature review. Are you focusing on recent developments, or do you want a historical overview? 
• Consider the geographical scope. Is your review global, or are you focusing on a specific region? 
• Define the inclusion and exclusion criteria. What types of sources will you include? Are there specific types of studies or publications you will exclude? 

Select Databases for Searches:

• Identify relevant databases for your field. Examples include PubMed, IEEE Xplore, Scopus, Web of Science, and Google Scholar. 
• Consider searching in library catalogs, institutional repositories, and specialized databases related to your topic. 

Conduct Searches and Keep Track:

• Develop a systematic search strategy using keywords, Boolean operators (AND, OR, NOT), and other search techniques. 
• Record and document your search strategy for transparency and replicability. 
• Keep track of the articles, including publication details, abstracts, and links. Use citation management tools like EndNote, Zotero, or Mendeley to organize your references. 

Review the Literature:

• Evaluate the relevance and quality of each source. Consider the methodology, sample size, and results of studies. 
• Organize the literature by themes or key concepts. Identify patterns, trends, and gaps in the existing research. 
• Summarize key findings and arguments from each source. Compare and contrast different perspectives. 
• Identify areas where there is a consensus in the literature and where there are conflicting opinions. 
• Provide critical analysis and synthesis of the literature. What are the strengths and weaknesses of existing research? 

Organize and Write Your Literature Review:

• Literature review outline should be based on themes, chronological order, or methodological approaches. 
• Write a clear and coherent narrative that synthesizes the information gathered. 
• Use proper citations for each source and ensure consistency in your citation style (APA, MLA, Chicago, etc.). 
• Conclude your literature review by summarizing key findings, identifying gaps, and suggesting areas for future research. 

Whether you’re exploring a new research field or finding new angles to develop an existing topic, sifting through hundreds of papers can take more time than you have to spare. But what if you could find science-backed insights with verified citations in seconds? That’s the power of Paperpal’s new Research feature!  

How to write a literature review faster with Paperpal?

Paperpal, an AI writing assistant, integrates powerful academic search capabilities within its writing platform. With the Research feature, you get 100% factual insights, with citations backed by 250M+ verified research articles, directly within your writing interface with the option to save relevant references in your Citation Library. By eliminating the need to switch tabs to find answers to all your research questions, Paperpal saves time and helps you stay focused on your writing.   

Here’s how to use the Research feature:  

• Ask a question: Get started with a new document on paperpal.com. Click on the “Research” feature and type your question in plain English. Paperpal will scour over 250 million research articles, including conference papers and preprints, to provide you with accurate insights and citations. 
• Review and Save: Paperpal summarizes the information, while citing sources and listing relevant reads. You can quickly scan the results to identify relevant references and save these directly to your built-in citations library for later access. 
• Cite with Confidence: Paperpal makes it easy to incorporate relevant citations and references into your writing, ensuring your arguments are well-supported by credible sources. This translates to a polished, well-researched literature review. 

The literature review sample and detailed advice on writing and conducting a review will help you produce a well-structured report. But remember that a good literature review is an ongoing process, and it may be necessary to revisit and update it as your research progresses. By combining effortless research with an easy citation process, Paperpal Research streamlines the literature review process and empowers you to write faster and with more confidence. Try Paperpal Research now and see for yourself.  

Frequently asked questions

A literature review is a critical and comprehensive analysis of existing literature (published and unpublished works) on a specific topic or research question and provides a synthesis of the current state of knowledge in a particular field. A well-conducted literature review is crucial for researchers to build upon existing knowledge, avoid duplication of efforts, and contribute to the advancement of their field. It also helps researchers situate their work within a broader context and facilitates the development of a sound theoretical and conceptual framework for their studies.

Literature review is a crucial component of research writing, providing a solid background for a research paper’s investigation. The aim is to keep professionals up to date by providing an understanding of ongoing developments within a specific field, including research methods, and experimental techniques used in that field, and present that knowledge in the form of a written report. Also, the depth and breadth of the literature review emphasizes the credibility of the scholar in his or her field.  

Before writing a literature review, it’s essential to undertake several preparatory steps to ensure that your review is well-researched, organized, and focused. This includes choosing a topic of general interest to you and doing exploratory research on that topic, writing an annotated bibliography, and noting major points, especially those that relate to the position you have taken on the topic. 

Literature reviews and academic research papers are essential components of scholarly work but serve different purposes within the academic realm. 3 A literature review aims to provide a foundation for understanding the current state of research on a particular topic, identify gaps or controversies, and lay the groundwork for future research. Therefore, it draws heavily from existing academic sources, including books, journal articles, and other scholarly publications. In contrast, an academic research paper aims to present new knowledge, contribute to the academic discourse, and advance the understanding of a specific research question. Therefore, it involves a mix of existing literature (in the introduction and literature review sections) and original data or findings obtained through research methods. 

Literature reviews are essential components of academic and research papers, and various strategies can be employed to conduct them effectively. If you want to know how to write a literature review for a research paper, here are four common approaches that are often used by researchers.  Chronological Review: This strategy involves organizing the literature based on the chronological order of publication. It helps to trace the development of a topic over time, showing how ideas, theories, and research have evolved.  Thematic Review: Thematic reviews focus on identifying and analyzing themes or topics that cut across different studies. Instead of organizing the literature chronologically, it is grouped by key themes or concepts, allowing for a comprehensive exploration of various aspects of the topic.  Methodological Review: This strategy involves organizing the literature based on the research methods employed in different studies. It helps to highlight the strengths and weaknesses of various methodologies and allows the reader to evaluate the reliability and validity of the research findings.  Theoretical Review: A theoretical review examines the literature based on the theoretical frameworks used in different studies. This approach helps to identify the key theories that have been applied to the topic and assess their contributions to the understanding of the subject.  It’s important to note that these strategies are not mutually exclusive, and a literature review may combine elements of more than one approach. The choice of strategy depends on the research question, the nature of the literature available, and the goals of the review. Additionally, other strategies, such as integrative reviews or systematic reviews, may be employed depending on the specific requirements of the research.

The literature review format can vary depending on the specific publication guidelines. However, there are some common elements and structures that are often followed. Here is a general guideline for the format of a literature review:  Introduction:   Provide an overview of the topic.  Define the scope and purpose of the literature review.  State the research question or objective.  Body:   Organize the literature by themes, concepts, or chronology.  Critically analyze and evaluate each source.  Discuss the strengths and weaknesses of the studies.  Highlight any methodological limitations or biases.  Identify patterns, connections, or contradictions in the existing research.  Conclusion:   Summarize the key points discussed in the literature review.  Highlight the research gap.  Address the research question or objective stated in the introduction.  Highlight the contributions of the review and suggest directions for future research.

Both annotated bibliographies and literature reviews involve the examination of scholarly sources. While annotated bibliographies focus on individual sources with brief annotations, literature reviews provide a more in-depth, integrated, and comprehensive analysis of existing literature on a specific topic. The key differences are as follows: 

References 

• Denney, A. S., & Tewksbury, R. (2013). How to write a literature review.  Journal of criminal justice education ,  24 (2), 218-234. 
• Pan, M. L. (2016).  Preparing literature reviews: Qualitative and quantitative approaches . Taylor & Francis. 
• Cantero, C. (2019). How to write a literature review.  San José State University Writing Center . 

Paperpal is an AI writing assistant that help academics write better, faster with real-time suggestions for in-depth language and grammar correction. Trained on millions of research manuscripts enhanced by professional academic editors, Paperpal delivers human precision at machine speed.  

Try it for free or upgrade to  Paperpal Prime , which unlocks unlimited access to premium features like academic translation, paraphrasing, contextual synonyms, consistency checks and more. It’s like always having a professional academic editor by your side! Go beyond limitations and experience the future of academic writing.  Get Paperpal Prime now at just US$19 a month!

Related Reads:

• Empirical Research: A Comprehensive Guide for Academics 
• How to Write a Scientific Paper in 10 Steps 
• How Long Should a Chapter Be?
• How to Use Paperpal to Generate Emails & Cover Letters?

6 Tips for Post-Doc Researchers to Take Their Career to the Next Level

Self-plagiarism in research: what it is and how to avoid it, you may also like, academic integrity vs academic dishonesty: types & examples, dissertation printing and binding | types & comparison , what is a dissertation preface definition and examples , the ai revolution: authors’ role in upholding academic..., the future of academia: how ai tools are..., how to write a research proposal: (with examples..., how to write your research paper in apa..., how to choose a dissertation topic, how to write a phd research proposal, how to write an academic paragraph (step-by-step guide).

• UConn Library
• Literature Review: The What, Why and How-to Guide
• Introduction

Literature Review: The What, Why and How-to Guide — Introduction

• Getting Started
• How to Pick a Topic
• Strategies to Find Sources
• Evaluating Sources & Lit. Reviews
• Tips for Writing Literature Reviews
• Writing Literature Review: Useful Sites
• Citation Resources
• Other Academic Writings

What are Literature Reviews?

So, what is a literature review? "A literature review is an account of what has been published on a topic by accredited scholars and researchers. In writing the literature review, your purpose is to convey to your reader what knowledge and ideas have been established on a topic, and what their strengths and weaknesses are. As a piece of writing, the literature review must be defined by a guiding concept (e.g., your research objective, the problem or issue you are discussing, or your argumentative thesis). It is not just a descriptive list of the material available, or a set of summaries." Taylor, D.  The literature review: A few tips on conducting it . University of Toronto Health Sciences Writing Centre.

Goals of Literature Reviews

What are the goals of creating a Literature Review?  A literature could be written to accomplish different aims:

• To develop a theory or evaluate an existing theory
• To summarize the historical or existing state of a research topic
• Identify a problem in a field of research 

Baumeister, R. F., & Leary, M. R. (1997). Writing narrative literature reviews .  Review of General Psychology , 1 (3), 311-320.

What kinds of sources require a Literature Review?

• A research paper assigned in a course
• A thesis or dissertation
• A grant proposal
• An article intended for publication in a journal

All these instances require you to collect what has been written about your research topic so that you can demonstrate how your own research sheds new light on the topic.

Types of Literature Reviews

What kinds of literature reviews are written?

Narrative review: The purpose of this type of review is to describe the current state of the research on a specific topic/research and to offer a critical analysis of the literature reviewed. Studies are grouped by research/theoretical categories, and themes and trends, strengths and weakness, and gaps are identified. The review ends with a conclusion section which summarizes the findings regarding the state of the research of the specific study, the gaps identify and if applicable, explains how the author's research will address gaps identify in the review and expand the knowledge on the topic reviewed.

• Example : Predictors and Outcomes of U.S. Quality Maternity Leave: A Review and Conceptual Framework:  10.1177/08948453211037398  

Systematic review : "The authors of a systematic review use a specific procedure to search the research literature, select the studies to include in their review, and critically evaluate the studies they find." (p. 139). Nelson, L. K. (2013). Research in Communication Sciences and Disorders . Plural Publishing.

• Example : The effect of leave policies on increasing fertility: a systematic review:  10.1057/s41599-022-01270-w

Meta-analysis : "Meta-analysis is a method of reviewing research findings in a quantitative fashion by transforming the data from individual studies into what is called an effect size and then pooling and analyzing this information. The basic goal in meta-analysis is to explain why different outcomes have occurred in different studies." (p. 197). Roberts, M. C., & Ilardi, S. S. (2003). Handbook of Research Methods in Clinical Psychology . Blackwell Publishing.

• Example : Employment Instability and Fertility in Europe: A Meta-Analysis:  10.1215/00703370-9164737

Meta-synthesis : "Qualitative meta-synthesis is a type of qualitative study that uses as data the findings from other qualitative studies linked by the same or related topic." (p.312). Zimmer, L. (2006). Qualitative meta-synthesis: A question of dialoguing with texts .  Journal of Advanced Nursing , 53 (3), 311-318.

• Example : Women’s perspectives on career successes and barriers: A qualitative meta-synthesis:  10.1177/05390184221113735

Literature Reviews in the Health Sciences

• UConn Health subject guide on systematic reviews Explanation of the different review types used in health sciences literature as well as tools to help you find the right review type
• << Previous: Getting Started
• Next: How to Pick a Topic >>
• Last Updated: Sep 21, 2022 2:16 PM
• URL: https://guides.lib.uconn.edu/literaturereview

What Is A Literature Review?

A plain-language explainer (with examples).

By:  Derek Jansen (MBA) & Kerryn Warren (PhD) | June 2020 (Updated May 2023)

If you’re faced with writing a dissertation or thesis, chances are you’ve encountered the term “literature review” . If you’re on this page, you’re probably not 100% what the literature review is all about. The good news is that you’ve come to the right place.

Literature Review 101

• What (exactly) is a literature review
• What’s the purpose of the literature review chapter
• How to find high-quality resources
• How to structure your literature review chapter
• Example of an actual literature review

What is a literature review?

The word “literature review” can refer to two related things that are part of the broader literature review process. The first is the task of  reviewing the literature  – i.e. sourcing and reading through the existing research relating to your research topic. The second is the  actual chapter  that you write up in your dissertation, thesis or research project. Let’s look at each of them:

Reviewing the literature

The first step of any literature review is to hunt down and  read through the existing research  that’s relevant to your research topic. To do this, you’ll use a combination of tools (we’ll discuss some of these later) to find journal articles, books, ebooks, research reports, dissertations, theses and any other credible sources of information that relate to your topic. You’ll then  summarise and catalogue these  for easy reference when you write up your literature review chapter. 

The literature review chapter

The second step of the literature review is to write the actual literature review chapter (this is usually the second chapter in a typical dissertation or thesis structure ). At the simplest level, the literature review chapter is an  overview of the key literature  that’s relevant to your research topic. This chapter should provide a smooth-flowing discussion of what research has already been done, what is known, what is unknown and what is contested in relation to your research topic. So, you can think of it as an  integrated review of the state of knowledge  around your research topic. 

What’s the purpose of a literature review?

The literature review chapter has a few important functions within your dissertation, thesis or research project. Let’s take a look at these:

Purpose #1 – Demonstrate your topic knowledge

The first function of the literature review chapter is, quite simply, to show the reader (or marker) that you  know what you’re talking about . In other words, a good literature review chapter demonstrates that you’ve read the relevant existing research and understand what’s going on – who’s said what, what’s agreed upon, disagreed upon and so on. This needs to be  more than just a summary  of who said what – it needs to integrate the existing research to  show how it all fits together  and what’s missing (which leads us to purpose #2, next). 

Purpose #2 – Reveal the research gap that you’ll fill

The second function of the literature review chapter is to  show what’s currently missing  from the existing research, to lay the foundation for your own research topic. In other words, your literature review chapter needs to show that there are currently “missing pieces” in terms of the bigger puzzle, and that  your study will fill one of those research gaps . By doing this, you are showing that your research topic is original and will help contribute to the body of knowledge. In other words, the literature review helps justify your research topic.  

Purpose #3 – Lay the foundation for your conceptual framework

The third function of the literature review is to form the  basis for a conceptual framework . Not every research topic will necessarily have a conceptual framework, but if your topic does require one, it needs to be rooted in your literature review. 

For example, let’s say your research aims to identify the drivers of a certain outcome – the factors which contribute to burnout in office workers. In this case, you’d likely develop a conceptual framework which details the potential factors (e.g. long hours, excessive stress, etc), as well as the outcome (burnout). Those factors would need to emerge from the literature review chapter – they can’t just come from your gut! 

So, in this case, the literature review chapter would uncover each of the potential factors (based on previous studies about burnout), which would then be modelled into a framework. 

Purpose #4 – To inform your methodology

The fourth function of the literature review is to  inform the choice of methodology  for your own research. As we’ve  discussed on the Grad Coach blog , your choice of methodology will be heavily influenced by your research aims, objectives and questions . Given that you’ll be reviewing studies covering a topic close to yours, it makes sense that you could learn a lot from their (well-considered) methodologies.

So, when you’re reviewing the literature, you’ll need to  pay close attention to the research design , methodology and methods used in similar studies, and use these to inform your methodology. Quite often, you’ll be able to  “borrow” from previous studies . This is especially true for quantitative studies , as you can use previously tried and tested measures and scales. 

How do I find articles for my literature review?

Finding quality journal articles is essential to crafting a rock-solid literature review. As you probably already know, not all research is created equally, and so you need to make sure that your literature review is  built on credible research . 

We could write an entire post on how to find quality literature (actually, we have ), but a good starting point is Google Scholar . Google Scholar is essentially the academic equivalent of Google, using Google’s powerful search capabilities to find relevant journal articles and reports. It certainly doesn’t cover every possible resource, but it’s a very useful way to get started on your literature review journey, as it will very quickly give you a good indication of what the  most popular pieces of research  are in your field.

One downside of Google Scholar is that it’s merely a search engine – that is, it lists the articles, but oftentimes  it doesn’t host the articles . So you’ll often hit a paywall when clicking through to journal websites. 

Thankfully, your university should provide you with access to their library, so you can find the article titles using Google Scholar and then search for them by name in your university’s online library. Your university may also provide you with access to  ResearchGate , which is another great source for existing research. 

Remember, the correct search keywords will be super important to get the right information from the start. So, pay close attention to the keywords used in the journal articles you read and use those keywords to search for more articles. If you can’t find a spoon in the kitchen, you haven’t looked in the right drawer. 

Need a helping hand?

How should I structure my literature review?

Unfortunately, there’s no generic universal answer for this one. The structure of your literature review will depend largely on your topic area and your research aims and objectives.

You could potentially structure your literature review chapter according to theme, group, variables , chronologically or per concepts in your field of research. We explain the main approaches to structuring your literature review here . You can also download a copy of our free literature review template to help you establish an initial structure.

In general, it’s also a good idea to start wide (i.e. the big-picture-level) and then narrow down, ending your literature review close to your research questions . However, there’s no universal one “right way” to structure your literature review. The most important thing is not to discuss your sources one after the other like a list – as we touched on earlier, your literature review needs to synthesise the research , not summarise it .

Ultimately, you need to craft your literature review so that it conveys the most important information effectively – it needs to tell a logical story in a digestible way. It’s no use starting off with highly technical terms and then only explaining what these terms mean later. Always assume your reader is not a subject matter expert and hold their hand through a journe y of the literature while keeping the functions of the literature review chapter (which we discussed earlier) front of mind.

Example of a literature review

In the video below, we walk you through a high-quality literature review from a dissertation that earned full distinction. This will give you a clearer view of what a strong literature review looks like in practice and hopefully provide some inspiration for your own. 

Wrapping Up

In this post, we’ve (hopefully) answered the question, “ what is a literature review? “. We’ve also considered the purpose and functions of the literature review, as well as how to find literature and how to structure the literature review chapter. If you’re keen to learn more, check out the literature review section of the Grad Coach blog , as well as our detailed video post covering how to write a literature review . 

Psst… there’s more!

This post is an extract from our bestselling short course, Literature Review Bootcamp . If you want to work smart, you don't want to miss this .

16 Comments

Thanks for this review. It narrates what’s not been taught as tutors are always in a early to finish their classes.

Thanks for the kind words, Becky. Good luck with your literature review 🙂

This website is amazing, it really helps break everything down. Thank you, I would have been lost without it.

This is review is amazing. I benefited from it a lot and hope others visiting this website will benefit too.

Timothy T. Chol [email protected]

Thank you very much for the guiding in literature review I learn and benefited a lot this make my journey smooth I’ll recommend this site to my friends

This was so useful. Thank you so much.

Hi, Concept was explained nicely by both of you. Thanks a lot for sharing it. It will surely help research scholars to start their Research Journey.

The review is really helpful to me especially during this period of covid-19 pandemic when most universities in my country only offer online classes. Great stuff

Great Brief Explanation, thanks

So helpful to me as a student

GradCoach is a fantastic site with brilliant and modern minds behind it.. I spent weeks decoding the substantial academic Jargon and grounding my initial steps on the research process, which could be shortened to a couple of days through the Gradcoach. Thanks again!

This is an amazing talk. I paved way for myself as a researcher. Thank you GradCoach!

Well-presented overview of the literature!

This was brilliant. So clear. Thank you

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

Reference management. Clean and simple.

How to write a literature review in 6 steps

What is a literature review?

How to write a literature review, 1. determine the purpose of your literature review, 2. do an extensive search, 3. evaluate and select literature, 4. analyze the literature, 5. plan the structure of your literature review, 6. write your literature review, other resources to help you write a successful literature review, frequently asked questions about writing a literature review, related articles.

A literature review is an assessment of the sources in a chosen topic of research.

A good literature review does not just summarize sources. It analyzes the state of the field on a given topic and creates a scholarly foundation for you to make your own intervention. It demonstrates to your readers how your research fits within a larger field of study.

In a thesis, a literature review is part of the introduction, but it can also be a separate section. In research papers, a literature review may have its own section or it may be integrated into the introduction, depending on the field.

➡️ Our guide on what is a literature review covers additional basics about literature reviews.

• Identify the main purpose of the literature review.
• Do extensive research.
• Evaluate and select relevant sources.
• Analyze the sources.
• Plan a structure.
• Write the review.

In this section, we review each step of the process of creating a literature review.

In the first step, make sure you know specifically what the assignment is and what form your literature review should take. Read your assignment carefully and seek clarification from your professor or instructor if needed. You should be able to answer the following questions:

• How many sources do I need to include?
• What types of sources should I review?
• Should I evaluate the sources?
• Should I summarize, synthesize or critique sources?
• Do I need to provide any definitions or background information?

In addition to that, be aware that the narrower your topic, the easier it will be to limit the number of sources you need to read in order to get a good overview of the topic.

Now you need to find out what has been written on the topic and search for literature related to your research topic. Make sure to select appropriate source material, which means using academic or scholarly sources , including books, reports, journal articles , government documents and web resources.

➡️ If you’re unsure about how to tell if a source is scholarly, take a look at our guide on how to identify a scholarly source .

Come up with a list of relevant keywords and then start your search with your institution's library catalog, and extend it to other useful databases and academic search engines like:

• Google Scholar
• Science.gov

➡️ Our guide on how to collect data for your thesis might be helpful at this stage of your research as well as the top list of academic search engines .

Once you find a useful article, check out the reference list. It should provide you with even more relevant sources. Also, keep a note of the:

• authors' names
• page numbers

Keeping track of the bibliographic information for each source will save you time when you’re ready to create citations. You could also use a reference manager like Paperpile to automatically save, manage, and cite your references.

Read the literature. You will most likely not be able to read absolutely everything that is out there on the topic. Therefore, read the abstract first to determine whether the rest of the source is worth your time. If the source is relevant for your topic:

• Read it critically.
• Look for the main arguments.
• Take notes as you read.
• Organize your notes using a table, mind map, or other technique.

Now you are ready to analyze the literature you have gathered. While your are working on your analysis, you should ask the following questions:

• What are the key terms, concepts and problems addressed by the author?
• How is this source relevant for my specific topic?
• How is the article structured? What are the major trends and findings?
• What are the conclusions of the study?
• How are the results presented? Is the source credible?
• When comparing different sources, how do they relate to each other? What are the similarities, what are the differences?
• Does the study help me understand the topic better?
• Are there any gaps in the research that need to be filled? How can I further my research as a result of the review?

Tip: Decide on the structure of your literature review before you start writing.

There are various ways to organize your literature review:

• Chronological method : Writing in the chronological method means you are presenting the materials according to when they were published. Follow this approach only if a clear path of research can be identified.
• Thematic review : A thematic review of literature is organized around a topic or issue, rather than the progression of time.
• Publication-based : You can order your sources by publication, if the way you present the order of your sources demonstrates a more important trend. This is the case when a progression revealed from study to study and the practices of researchers have changed and adapted due to the new revelations.
• Methodological approach : A methodological approach focuses on the methods used by the researcher. If you have used sources from different disciplines that use a variety of research methods, you might want to compare the results in light of the different methods and discuss how the topic has been approached from different sides.

Regardless of the structure you chose, a review should always include the following three sections:

• An introduction, which should give the reader an outline of why you are writing the review and explain the relevance of the topic.
• A body, which divides your literature review into different sections. Write in well-structured paragraphs, use transitions and topic sentences and critically analyze each source for how it contributes to the themes you are researching.
• A conclusion , which summarizes the key findings, the main agreements and disagreements in the literature, your overall perspective, and any gaps or areas for further research.

➡️ If your literature review is part of a longer paper, visit our guide on what is a research paper for additional tips.

➡️ UNC writing center: Literature reviews

➡️ How to write a literature review in 3 steps

➡️ How to write a literature review in 30 minutes or less

The goal of a literature review is to asses the state of the field on a given topic in preparation for making an intervention.

A literature review should have its own independent section. You should indicate clearly in the table of contents where it can be found, and address this section as “Literature Review.”

There is no set amount of words for a literature review; the length depends on the research. If you are working with a large amount of sources, then it will be long. If your paper does not depend entirely on references, then it will be short.

Most research papers include a literature review. By assessing the available sources in your field of research, you will be able to make a more confident argument about the topic.

Literature reviews are most commonly found in theses and dissertations. However, you find them in research papers as well.

Frequently asked questions

What is the purpose of a literature review.

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarize yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

Frequently asked questions: Academic writing

A rhetorical tautology is the repetition of an idea of concept using different words.

Rhetorical tautologies occur when additional words are used to convey a meaning that has already been expressed or implied. For example, the phrase “armed gunman” is a tautology because a “gunman” is by definition “armed.”

A logical tautology is a statement that is always true because it includes all logical possibilities.

Logical tautologies often take the form of “either/or” statements (e.g., “It will rain, or it will not rain”) or employ circular reasoning (e.g., “she is untrustworthy because she can’t be trusted”).

You may have seen both “appendices” or “appendixes” as pluralizations of “ appendix .” Either spelling can be used, but “appendices” is more common (including in APA Style ). Consistency is key here: make sure you use the same spelling throughout your paper.

The purpose of a lab report is to demonstrate your understanding of the scientific method with a hands-on lab experiment. Course instructors will often provide you with an experimental design and procedure. Your task is to write up how you actually performed the experiment and evaluate the outcome.

In contrast, a research paper requires you to independently develop an original argument. It involves more in-depth research and interpretation of sources and data.

A lab report is usually shorter than a research paper.

The sections of a lab report can vary between scientific fields and course requirements, but it usually contains the following:

• Title: expresses the topic of your study
• Abstract: summarizes your research aims, methods, results, and conclusions
• Introduction: establishes the context needed to understand the topic
• Method: describes the materials and procedures used in the experiment
• Results: reports all descriptive and inferential statistical analyses
• Discussion: interprets and evaluates results and identifies limitations
• Conclusion: sums up the main findings of your experiment
• References: list of all sources cited using a specific style (e.g. APA)
• Appendices: contains lengthy materials, procedures, tables or figures

A lab report conveys the aim, methods, results, and conclusions of a scientific experiment . Lab reports are commonly assigned in science, technology, engineering, and mathematics (STEM) fields.

The abstract is the very last thing you write. You should only write it after your research is complete, so that you can accurately summarize the entirety of your thesis , dissertation or research paper .

If you’ve gone over the word limit set for your assignment, shorten your sentences and cut repetition and redundancy during the editing process. If you use a lot of long quotes , consider shortening them to just the essentials.

If you need to remove a lot of words, you may have to cut certain passages. Remember that everything in the text should be there to support your argument; look for any information that’s not essential to your point and remove it.

To make this process easier and faster, you can use a paraphrasing tool . With this tool, you can rewrite your text to make it simpler and shorter. If that’s not enough, you can copy-paste your paraphrased text into the summarizer . This tool will distill your text to its core message.

Revising, proofreading, and editing are different stages of the writing process .

• Revising is making structural and logical changes to your text—reformulating arguments and reordering information.
• Editing refers to making more local changes to things like sentence structure and phrasing to make sure your meaning is conveyed clearly and concisely.
• Proofreading involves looking at the text closely, line by line, to spot any typos and issues with consistency and correct them.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

Avoid citing sources in your abstract . There are two reasons for this:

• The abstract should focus on your original research, not on the work of others.
• The abstract should be self-contained and fully understandable without reference to other sources.

There are some circumstances where you might need to mention other sources in an abstract: for example, if your research responds directly to another study or focuses on the work of a single theorist. In general, though, don’t include citations unless absolutely necessary.

An abstract is a concise summary of an academic text (such as a journal article or dissertation ). It serves two main purposes:

• To help potential readers determine the relevance of your paper for their own research.
• To communicate your key findings to those who don’t have time to read the whole paper.

Abstracts are often indexed along with keywords on academic databases, so they make your work more easily findable. Since the abstract is the first thing any reader sees, it’s important that it clearly and accurately summarizes the contents of your paper.

In a scientific paper, the methodology always comes after the introduction and before the results , discussion and conclusion . The same basic structure also applies to a thesis, dissertation , or research proposal .

Depending on the length and type of document, you might also include a literature review or theoretical framework before the methodology.

Whether you’re publishing a blog, submitting a research paper , or even just writing an important email, there are a few techniques you can use to make sure it’s error-free:

• Take a break : Set your work aside for at least a few hours so that you can look at it with fresh eyes.
• Proofread a printout : Staring at a screen for too long can cause fatigue – sit down with a pen and paper to check the final version.
• Use digital shortcuts : Take note of any recurring mistakes (for example, misspelling a particular word, switching between US and UK English , or inconsistently capitalizing a term), and use Find and Replace to fix it throughout the document.

If you want to be confident that an important text is error-free, it might be worth choosing a professional proofreading service instead.

Editing and proofreading are different steps in the process of revising a text.

Editing comes first, and can involve major changes to content, structure and language. The first stages of editing are often done by authors themselves, while a professional editor makes the final improvements to grammar and style (for example, by improving sentence structure and word choice ).

Proofreading is the final stage of checking a text before it is published or shared. It focuses on correcting minor errors and inconsistencies (for example, in punctuation and capitalization ). Proofreaders often also check for formatting issues, especially in print publishing.

The cost of proofreading depends on the type and length of text, the turnaround time, and the level of services required. Most proofreading companies charge per word or page, while freelancers sometimes charge an hourly rate.

For proofreading alone, which involves only basic corrections of typos and formatting mistakes, you might pay as little as $0.01 per word, but in many cases, your text will also require some level of editing , which costs slightly more.

It’s often possible to purchase combined proofreading and editing services and calculate the price in advance based on your requirements.

There are many different routes to becoming a professional proofreader or editor. The necessary qualifications depend on the field – to be an academic or scientific proofreader, for example, you will need at least a university degree in a relevant subject.

For most proofreading jobs, experience and demonstrated skills are more important than specific qualifications. Often your skills will be tested as part of the application process.

To learn practical proofreading skills, you can choose to take a course with a professional organization such as the Society for Editors and Proofreaders . Alternatively, you can apply to companies that offer specialized on-the-job training programmes, such as the Scribbr Academy .

Ask our team

Want to contact us directly? No problem.  We  are always here for you.

• Email [email protected]
• Start live chat
• Call +1 (510) 822-8066
• WhatsApp +31 20 261 6040

Our team helps students graduate by offering:

• A world-class citation generator
• Plagiarism Checker software powered by Turnitin
• Innovative Citation Checker software
• Professional proofreading services
• Over 300 helpful articles about academic writing, citing sources, plagiarism, and more

Scribbr specializes in editing study-related documents . We proofread:

• PhD dissertations
• Research proposals
• Personal statements
• Admission essays
• Motivation letters
• Reflection papers
• Journal articles
• Capstone projects

Scribbr’s Plagiarism Checker is powered by elements of Turnitin’s Similarity Checker , namely the plagiarism detection software and the Internet Archive and Premium Scholarly Publications content databases .

The add-on AI detector is powered by Scribbr’s proprietary software.

The Scribbr Citation Generator is developed using the open-source Citation Style Language (CSL) project and Frank Bennett’s citeproc-js . It’s the same technology used by dozens of other popular citation tools, including Mendeley and Zotero.

You can find all the citation styles and locales used in the Scribbr Citation Generator in our publicly accessible repository on Github .

Harvey Cushing/John Hay Whitney Medical Library

• Collections
• Research Help

YSN Doctoral Programs: Steps in Conducting a Literature Review

• Biomedical Databases
• Global (Public Health) Databases
• Soc. Sci., History, and Law Databases
• Grey Literature
• Trials Registers
• Data and Statistics
• Public Policy
• Google Tips
• Recommended Books
• Steps in Conducting a Literature Review

What is a literature review?

A literature review is an integrated analysis -- not just a summary-- of scholarly writings and other relevant evidence related directly to your research question.  That is, it represents a synthesis of the evidence that provides background information on your topic and shows a association between the evidence and your research question.

A literature review may be a stand alone work or the introduction to a larger research paper, depending on the assignment.  Rely heavily on the guidelines your instructor has given you.

Why is it important?

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Discovers relationships between research studies/ideas.
• Identifies major themes, concepts, and researchers on a topic.
• Identifies critical gaps and points of disagreement.
• Discusses further research questions that logically come out of the previous studies.

APA7 Style resources

APA Style Blog - for those harder to find answers

1. Choose a topic. Define your research question.

Your literature review should be guided by your central research question.  The literature represents background and research developments related to a specific research question, interpreted and analyzed by you in a synthesized way.

• Make sure your research question is not too broad or too narrow.  Is it manageable?
• Begin writing down terms that are related to your question. These will be useful for searches later.
• If you have the opportunity, discuss your topic with your professor and your class mates.

2. Decide on the scope of your review

How many studies do you need to look at? How comprehensive should it be? How many years should it cover? 

• This may depend on your assignment.  How many sources does the assignment require?

3. Select the databases you will use to conduct your searches.

Make a list of the databases you will search. 

Where to find databases:

• use the tabs on this guide
• Find other databases in the Nursing Information Resources web page
• More on the Medical Library web page
• ... and more on the Yale University Library web page

4. Conduct your searches to find the evidence. Keep track of your searches.

• Use the key words in your question, as well as synonyms for those words, as terms in your search. Use the database tutorials for help.
• Save the searches in the databases. This saves time when you want to redo, or modify, the searches. It is also helpful to use as a guide is the searches are not finding any useful results.
• Review the abstracts of research studies carefully. This will save you time.
• Use the bibliographies and references of research studies you find to locate others.
• Check with your professor, or a subject expert in the field, if you are missing any key works in the field.
• Ask your librarian for help at any time.
• Use a citation manager, such as EndNote as the repository for your citations. See the EndNote tutorials for help.

Review the literature

Some questions to help you analyze the research:

• What was the research question of the study you are reviewing? What were the authors trying to discover?
• Was the research funded by a source that could influence the findings?
• What were the research methodologies? Analyze its literature review, the samples and variables used, the results, and the conclusions.
• Does the research seem to be complete? Could it have been conducted more soundly? What further questions does it raise?
• If there are conflicting studies, why do you think that is?
• How are the authors viewed in the field? Has this study been cited? If so, how has it been analyzed?

Tips: 

• Review the abstracts carefully.  
• Keep careful notes so that you may track your thought processes during the research process.
• Create a matrix of the studies for easy analysis, and synthesis, across all of the studies.
• << Previous: Recommended Books
• Last Updated: Jun 20, 2024 9:08 AM
• URL: https://guides.library.yale.edu/YSNDoctoral

How to write a literature review introduction (+ examples)

The introduction to a literature review serves as your reader’s guide through your academic work and thought process. Explore the significance of literature review introductions in review papers, academic papers, essays, theses, and dissertations. We delve into the purpose and necessity of these introductions, explore the essential components of literature review introductions, and provide step-by-step guidance on how to craft your own, along with examples.

Why you need an introduction for a literature review

In academic writing , the introduction for a literature review is an indispensable component. Effective academic writing requires proper paragraph structuring to guide your reader through your argumentation. This includes providing an introduction to your literature review.

It is imperative to remember that you should never start sharing your findings abruptly. Even if there isn’t a dedicated introduction section .

When you need an introduction for a literature review

There are three main scenarios in which you need an introduction for a literature review:

What to include in a literature review introduction

It is crucial to customize the content and depth of your literature review introduction according to the specific format of your academic work.

Academic literature review paper

The introduction of an academic literature review paper, which does not rely on empirical data, often necessitates a more extensive introduction than the brief literature review introductions typically found in empirical papers. It should encompass:

Regular literature review section in an academic article or essay

In a standard 8000-word journal article, the literature review section typically spans between 750 and 1250 words. The first few sentences or the first paragraph within this section often serve as an introduction. It should encompass:

Introduction to a literature review chapter in thesis or dissertation

Some students choose to incorporate a brief introductory section at the beginning of each chapter, including the literature review chapter. Alternatively, others opt to seamlessly integrate the introduction into the initial sentences of the literature review itself. Both approaches are acceptable, provided that you incorporate the following elements:

Examples of literature review introductions

Example 1: an effective introduction for an academic literature review paper.

To begin, let’s delve into the introduction of an academic literature review paper. We will examine the paper “How does culture influence innovation? A systematic literature review”, which was published in 2018 in the journal Management Decision.

Example 2: An effective introduction to a literature review section in an academic paper

The second example represents a typical academic paper, encompassing not only a literature review section but also empirical data, a case study, and other elements. We will closely examine the introduction to the literature review section in the paper “The environmentalism of the subalterns: a case study of environmental activism in Eastern Kurdistan/Rojhelat”, which was published in 2021 in the journal Local Environment.

Thus, the author successfully introduces the literature review, from which point onward it dives into the main concept (‘subalternity’) of the research, and reviews the literature on socio-economic justice and environmental degradation.

Examples 3-5: Effective introductions to literature review chapters

Numerous universities offer online repositories where you can access theses and dissertations from previous years, serving as valuable sources of reference. Many of these repositories, however, may require you to log in through your university account. Nevertheless, a few open-access repositories are accessible to anyone, such as the one by the University of Manchester . It’s important to note though that copyright restrictions apply to these resources, just as they would with published papers.

Master’s thesis literature review introduction

Phd thesis literature review chapter introduction.

The second example is Deep Learning on Semi-Structured Data and its Applications to Video-Game AI, Woof, W. (Author). 31 Dec 2020, a PhD thesis completed at the University of Manchester . In Chapter 2, the author offers a comprehensive introduction to the topic in four paragraphs, with the final paragraph serving as an overview of the chapter’s structure:

PhD thesis literature review introduction

The last example is the doctoral thesis Metacognitive strategies and beliefs: Child correlates and early experiences Chan, K. Y. M. (Author). 31 Dec 2020 . The author clearly conducted a systematic literature review, commencing the review section with a discussion of the methodology and approach employed in locating and analyzing the selected records.

Steps to write your own literature review introduction

Master academia, get new content delivered directly to your inbox, the best answers to "what are your plans for the future", 10 tips for engaging your audience in academic writing, related articles, minor revisions: sample peer review comments and examples, sample emails to your thesis supervisor, co-authorship guidelines to successfully co-author a scientific paper, how to select a journal for publication as a phd student.

Why is it important to do a literature review in research?

The importance of scientific communication in the healthcare industry

The Importance and Role of Biostatistics in Clinical Research

 “A substantive, thorough, sophisticated literature review is a precondition for doing substantive, thorough, sophisticated research”. Boote and Baile 2005

Authors of manuscripts treat writing a literature review as a routine work or a mere formality. But a seasoned one knows the purpose and importance of a well-written literature review.  Since it is one of the basic needs for researches at any level, they have to be done vigilantly. Only then the reader will know that the basics of research have not been neglected.

The aim of any literature review is to summarize and synthesize the arguments and ideas of existing knowledge in a particular field without adding any new contributions.   Being built on existing knowledge they help the researcher to even turn the wheels of the topic of research.  It is possible only with profound knowledge of what is wrong in the existing findings in detail to overpower them.  For other researches, the literature review gives the direction to be headed for its success. 

The common perception of literature review and reality:

As per the common belief, literature reviews are only a summary of the sources related to the research. And many authors of scientific manuscripts believe that they are only surveys of what are the researches are done on the chosen topic.  But on the contrary, it uses published information from pertinent and relevant sources like

• Scholarly books
• Scientific papers
• Latest studies in the field
• Established school of thoughts
• Relevant articles from renowned scientific journals

and many more for a field of study or theory or a particular problem to do the following:

• Summarize into a brief account of all information
• Synthesize the information by restructuring and reorganizing
• Critical evaluation of a concept or a school of thought or ideas
• Familiarize the authors to the extent of knowledge in the particular field
• Encapsulate
• Compare & contrast

By doing the above on the relevant information, it provides the reader of the scientific manuscript with the following for a better understanding of it:

• It establishes the authors’  in-depth understanding and knowledge of their field subject
• It gives the background of the research
• Portrays the scientific manuscript plan of examining the research result
• Illuminates on how the knowledge has changed within the field
• Highlights what has already been done in a particular field
• Information of the generally accepted facts, emerging and current state of the topic of research
• Identifies the research gap that is still unexplored or under-researched fields
• Demonstrates how the research fits within a larger field of study
• Provides an overview of the sources explored during the research of a particular topic

Importance of literature review in research:

The importance of literature review in scientific manuscripts can be condensed into an analytical feature to enable the multifold reach of its significance.  It adds value to the legitimacy of the research in many ways:

• Provides the interpretation of existing literature in light of updated developments in the field to help in establishing the consistency in knowledge and relevancy of existing materials
• It helps in calculating the impact of the latest information in the field by mapping their progress of knowledge.
• It brings out the dialects of contradictions between various thoughts within the field to establish facts
• The research gaps scrutinized initially are further explored to establish the latest facts of theories to add value to the field
• Indicates the current research place in the schema of a particular field
• Provides information for relevancy and coherency to check the research
• Apart from elucidating the continuance of knowledge, it also points out areas that require further investigation and thus aid as a starting point of any future research
• Justifies the research and sets up the research question
• Sets up a theoretical framework comprising the concepts and theories of the research upon which its success can be judged
• Helps to adopt a more appropriate methodology for the research by examining the strengths and weaknesses of existing research in the same field
• Increases the significance of the results by comparing it with the existing literature
• Provides a point of reference by writing the findings in the scientific manuscript
• Helps to get the due credit from the audience for having done the fact-finding and fact-checking mission in the scientific manuscripts
• The more the reference of relevant sources of it could increase more of its trustworthiness with the readers
• Helps to prevent plagiarism by tailoring and uniquely tweaking the scientific manuscript not to repeat other’s original idea
• By preventing plagiarism , it saves the scientific manuscript from rejection and thus also saves a lot of time and money
• Helps to evaluate, condense and synthesize gist in the author’s own words to sharpen the research focus
• Helps to compare and contrast to  show the originality and uniqueness of the research than that of the existing other researches
• Rationalizes the need for conducting the particular research in a specified field
• Helps to collect data accurately for allowing any new methodology of research than the existing ones
• Enables the readers of the manuscript to answer the following questions of its readers for its better chances for publication
• What do the researchers know?
• What do they not know?
• Is the scientific manuscript reliable and trustworthy?
• What are the knowledge gaps of the researcher?

22. It helps the readers to identify the following for further reading of the scientific manuscript:

• What has been already established, discredited and accepted in the particular field of research
• Areas of controversy and conflicts among different schools of thought
• Unsolved problems and issues in the connected field of research
• The emerging trends and approaches
• How the research extends, builds upon and leaves behind from the previous research

A profound literature review with many relevant sources of reference will enhance the chances of the scientific manuscript publication in renowned and reputed scientific journals .

References:

http://www.math.montana.edu/jobo/phdprep/phd6.pdf

journal Publishing services  |  Scientific Editing Services  |  Medical Writing Services  |  scientific research writing service  |  Scientific communication services

Related Topics:

Meta Analysis

Scientific Research Paper Writing

Medical Research Paper Writing

Scientific Communication in healthcare

pubrica academy

Related posts.

Importance Of Proofreading For Scientific Writing Methods and Significance

Statistical analyses of case-control studies

Selecting material (e.g. excipient, active pharmaceutical ingredient, packaging material) for drug development

Comments are closed.

Writing the Literature Review

Barry Mauer and John Venecek

Note: Not all research papers contain literature reviews in their finished or published form. Check your assignment and guidelines to see if one is required. Even if a literature review is not required, you still need to read the available scholarly literature on your topic so you can join the scholarly conversation.

• The Literature Review

What is the Purpose of the Review?

What is the scope of the review, strategies for getting started, types of literature reviews, composition guidelines, how to locate reviews by discipline, key takeaways.

We also provide the following activities:

Types of Literature Reviews [Refresher]

Writing the literature review [refresher], the literature review [1].

Conducting a literary studies research project involves time and effort, with much of it going towards the development of a literature review . A literature review might fill several pages of your research paper and usually appears soon after an introduction but before you present your detailed argument. A literature review provides your audience with an overview of the available research about your area(s) of study, including the literary work, your theory, and methodology. The literature review demonstrates how these scholarly discussions have changed over time, and it allows you to position your research in relation to research that has come before yours. Your aim is to present the discussion up to this point. Depending on the nature of the assignment, you may also include your critical commentary on prior research, noting among this material the weaker and stronger arguments, breakthroughs and dead ends, blind spots and opportunities, the invention of key terms and methods, mistakes as well as misreadings, and so on.

Once you have gathered the research materials you need for your literature review, you have another task: conducting an analysis on the research to see where your original contribution fits into the scholarly conversation. As the saying goes, “we are standing on the shoulders of giants.” Your job is to show a portrait of these giants to your audience, and to show how your work relates to the portrait. On many scholarly topics, literature reviews already exist. You may refer to such existing reviews within your own, indicating any materials might have been overlooked, new developments that have arisen since the publication of the existing literature review, and new perspectives or insights you have about the materials.

Some beginning researchers try to tear down the work of other researchers in an effort to make their own work look good by comparison. It rarely works. First, it tends to make your audience skeptical of your claims. Second, it ignores the fact that even the mistakes, blind spots, and failures of other researchers contribute something to our knowledge. Albert Einstein didn’t disrespect Sir Isaac Newton by saying Newton’s theory of space was wrong and terrible and that Einstein’s own theory was great by comparison. He built upon Newton’s work, showing how it could be improved. If, however, a researcher willfully set out to deceive or distort or to tear down the work of other scholars without good reason, then their work does not deserve such deference.

Most literature reviews appear after the introduction. It presents your reader with relevant information about the scholarly discussion up to now. Later in your paper, you discuss your contribution. Before you begin work on your literature review, let’s discuss what we mean by “literature”; understand the purpose and scope of the review; establish criteria for selecting, organizing, and interpreting your findings; and discuss how to connect your findings to your research question.

Many students seek to “find sources that agree with my claim or idea.” That approach is too narrow, in our view. If we use such an approach, we may get the following results:

• Because we can find sources that agree with almost any claim, readers will wonder whether your claims are weak and the sources are cherry picked.
• While literary scholars sometimes cite authorities to support their claims, they don’t rely only on authority. They respect authority, but not too much. Your own claims need to rely more on evidence (from the literary text, historical and biographical information), and your critical and creative reasoning skills.
• Scholarship is a conversation; thus, the goal is less about finding agreement and more about joining the conversation with the aim of making a valuable contribution to the discussion.

The literature review provides your reader with an overview of the existing research about your topic or problem. It provides the context necessary for your reader to catch up with the scholarly conversation and then to appreciate the value of your contribution to it. The literature review sharpens the focus of your research and demonstrates your knowledge and understanding of the scholarly conversation around your topic, which, in turn, helps establish your credibility as a researcher.

Creating the literature review involves more than gathering citations. It is a qualitative process through which you will discover what is already known about your topic, and identify the key authorities, methods, and theoretical foundations, so you can begin to position your contributions within the scholarly conversation.

Defining the scope of your review will also help you establish criteria to determine the relevance of the sources you are finding. At this stage, you are not reading in-depth; instead, you are skimming through what has already been published and identifying the major concepts, theories, methodologies, and methods present within these published works. You should also be identifying connections, tensions, and contradictions within the already published works of your topic or problem. This involves building on the knowledge of others and understanding what methods, measures, and models we have inherited from previous researchers in our field.

Literature Reviews: Common Errors Made When Conducting a Literature Review [12 min 22 sec]

Video provided courtesy of the Center for Quality Research (CQR)

A literature review helps your reader understand the relationship of your research project to the work of other scholars. It covers the existing knowledge about a problem, and allows you to show the relevance/significance of your contribution to the discussion. Your reader may or may not have read scholarly literature about the theories, methodologies, and literary works you are discussing. But they want to know that you have read it and have thought about it. Your literature review provides not only a summary of the existing scholarship for readers; it also offers your perspective on it.

Begin your work on the literature review by synthesizing the various sources in your annotated bibliography .

For advice on Synthesizing Sources, consider the following from The Purdue Online Writing Lab: [2]

Note that  synthesizing is not the same as summarizing .

• A summary restates the information in one or more sources without providing new insight or reaching new conclusions.
• A synthesis draws on multiple sources to reach a broader conclusion.
• Don’t force a relationship between sources if there isn’t one. Not all of your sources have to complement one another.
• Do your best to highlight the relationships between sources in very clear ways.
• Don’t ignore any outliers in your research. It’s important to take note of every perspective (even those that disagree with your broader conclusions).

Not all humanities research projects contain literature reviews, but many do. Keep in mind that the type of literature review you choose (see list below) pertains to the secondary research – other scholarly sources – and not to the primary literary work. For instance, a literature review about Kate Chopin’s writing will be your thoughts about the scholarship on Chopin and not about Chopin’s text itself. You are summarizing what you see in the scholarly literature about Chopin’s writing. The literature review puts you in the position of authority not just on Chopin’s writing but on the scholarship about her writing. You are seeking to understand what scholars have said about her work. Scholars might belong to different schools of thought (psychoanalytic, feminist, Marxist, etc.). They might make different arguments about Chopin. They might use different methodological approaches. 

If your research involves two or more theories, such as psychology and genre studies, you may need to create multiple literature reviews, one for each theory or methodology. If the theories overlap with each other significantly (i.e., Marxism and Cultural Studies), you may combine them. Your literature review need not include everything about the subject area – you would need to write a book to cover a single theory – but only those concepts and methods that are most relevant to your research problem.

Factors to Consider When Developing Your Literature Review

• Determine the Scope : How broad or narrow should your literature review be? You may want to focus on recent scholarship only, or on a particular school of thought in the literature. Your scope is determined by your purpose; what is it you aim to achieve with your research?
• Establish Criteria : We discussed the importance of defining the purpose and scope of your review on the previous page, but it’s worth reviewing here as well. This step will help you establish important criteria and focus your searching. For example, how many sources will you need? What types of sources (primary, secondary, statistics, media)? Is currency important? Do you know who the prominent authors or theorists are in your subject area? Take some time to map out these or other important factors before you begin searching journals and databases.
• Consider Your Audience : Unlike a work cited page or an annotated bibliography, both of which are lists of sources, a literature review is essayistic and can be considered a precursor to your final paper. Therefore, it should be written in your own voice, and it should be geared toward a specific audience. Considering audience during this early stage will help focus your final paper as well.
• Find Models : We’ll discuss the different types of literature reviews and how to locate examples in the section below. However, even if you’re undecided about what type of review will work best for you, you may want to review some example literature reviews to get a sense of what they look like before you begin your own.

One piece of advice before starting: look for existing literature reviews on your area of scholarship. You can build on the work that other scholars have put into reviewing the scholarly literature. There’s no need to completely “reinvent the wheel” if some of the work is already done.

Scholars sometimes publish “stand-alone” literature reviews that are not part of a larger work; such literature reviews are valuable contributions to the field, as they summarize the state of knowledge for other scholars.

Maria J. Grant and Andrew Booth’s “A Typology of Reviews” identifies 14 distinct types of literature reviews. Further, the UCLA library created a chart to complement the article and for easy comparison of those 14 types of reviews. This section provides a brief summary of the most common literature reviews. For a more complete analysis, please see the full article and the chart .

To choose the most appropriate structure, put yourself in your reader’s shoes and think through their need for information. The literature review is about providing context for your contribution. How much context do people need? Keep it to the minimum necessary; compressing a lot of information into a small amount of text is a must.

These structures are not meant to be straightjackets but tools to help you organize your research. If you find that the tool is working, then keep using it. If not, switch tools or modify the one you are using. Keep in mind that the types of literature reviews are just different ways of organizing information. So, you can discuss literary trends without organizing your review of secondary literature by trend; your discussion can be organized by theory or theme, for examples. In our literature reviews, we are not recounting other scholars’ arguments at length but merely providing key concepts so we can summarize the discussion so far and position our own claims. You don’t have to adhere strictly to one structure or another. They are just organizing tools that help you manage your material (and help your reader make sense of it).

Types of Reviews

• Traditional or narrative reviews : This approach will generate a comprehensive, critical analysis of the published research on your topic. However, rather than merely compiling as many sources as possible, use this approach to establish a theoretical framework for your paper, establish trends, and identify gaps in the research. This process should bring your research question into clearer focus and help define a thesis that you will argue for in your paper. This is perhaps the most common and general type of literature review. The examples listed below are all designed to serve a more specific purpose.
• Argumentative : The purpose of an argumentative literature review is to select sources for the purpose of supporting or refuting a specific claim. While this type of review can help the author make a strong case for or against an issue, they can also be prone to claims of bias. Later in this textbook, we will read about the distinction between warranted and unwarranted bias . One is ok and the other is not.
• Chronological : A chronological review is used when the author wants to demonstrate the progression of how a theory, methodology, or issue has progressed over time. This method is most effective when there is a clear chronological path to the research about a specific historical event or trend as opposed to a more recursive theoretical concept.
• By trend : This is similar to the chronological approach except it focuses on clearly-defined trends rather than date ranges. This would be most appropriate if you want to illustrate changing perspectives or attitudes about a given issue when specific date ranges are less important than the ebb and flow of the trend.
• Thematic : In this type of literature review, the author will select specific themes that he or she feels are important to understanding a larger topic or concept. Then, the author will organize the sources around those themes, which are often based on relevance or importance. The value of this method is that the process of organizing the review by theme is similar to constructing an argument. This can help the author see how resources connect to each other and determine how as well as why specific sources support their thesis.
• Theoretical : The goal of this type of review is to examine how theory has shaped the research on a given topic. It establishes existing theoretical models, their connections, and how extensively they have been developed in the published research. For example, Jada applied critical race theory to her analysis of Sonny’s Blues , but she might also consider conducting a more comprehensive review of other theoretical frameworks such as feminism, Marxism, or postmodernism. Doing so could provide insight into alternate readings, and help her identify theoretical gaps such as unexplored or under-developed approaches to Baldwin’s work.
• Methodological : The approach focuses on the various methodologies used by researchers in a specific area rather than an analysis of their findings. In this case, you would create a framework of approaches to data collection related to your topic or research question. This is perhaps more common in education or the social and hard sciences where published research often includes a methods section, but it is sometimes appropriate for the digital humanities as well.
• Scoping : The aim of a scoping review is to provide a comprehensive overview or map of the published research or evidence related to a research question. This might be considered a prelude to a systematic review that would take the scoping review one step further toward answering a clearly defined research question. See below for more details.
• Systematic : The systematic review is most appropriate when you have a clearly-defined research question and have established criteria for the types of sources you need. In this way, the systematic review is less exploratory than other types of reviews. Rather, it is comprehensive, strategic, and focused on answering a specific research question. For this reason, the systematic review is more common in the health and social sciences, where comprehensiveness is more important. Literature reviews in the Humanities are not usually exhaustive but tend to show only the most representative or salient developments in the scholarship.
• Meta-analysis : Does your research deal with statistics or large amounts of data? If so, then a meta-analysis might be best for you rather than providing a critical review, the meta-analysis will summarize and synthesize the results of numerous studies that involve statistics or data to provide a more comprehensive picture than would be possible from just one study.

An argumentative literature review presents and takes sides in scholarly arguments about the literary work. It makes arguments about other scholars’ work. It does not necessarily involve a claim that the literary work is itself making an argument. Likewise, a chronological literature review presents the scholarly literature in chronological order.

You don’t need to keep strictly to one type. Scholars often combine features from various types of literature reviews. A sample review that combines the follow types –

• Argumentative
• Theoretical
• Methodological

– is the excellent work of Eiranen, Reetta, Mari Hatavara, Ville Kivimäki, Maria Mäkelä & Raisa Maria Toivo (2022) “ Narrative and Experience: Interdisciplinary Methodologies between History and Narratology , ” Scandinavian Journal of History , 47:1, 1-15

When writing your literature review, please follow these pointers:

• Conduct systematic searches
• Use Evidence
• Be Selective
• Use Quotes Sparingly
• Summarize & Synthesize
• Use Caution when Paraphrasing
• Use Your Own Voice

Advice from James Mason University’s “Literature Reviews: An Overview”

A note on synthesizing : Don’t make the common mistake of summarizing individual studies or articles one after the other. The goal is to synthesize — that is, to make observations about groups of studies. Synthesis often uses language like this:

• Much of the literature on [topic x ] focuses on [major themes].
• In recent years, researchers have begun investigating [facets a , b , and c ] of [topic x ].
• The studies in this review of [topic x ] confirm / suggest / call into question / support [idea / practice / finding / method / theory / guideline y ].
• In the reviewed studies [variable x ] was generally associated with higher / lower rates of [outcome y ].
• A limitation of some / most / all of these studies is [ y ].

Please see this sample annotated literature review  from James Mason University.

Structure of a literature review [2]

• Problematization: The 2 to 3 pages of problematization are a distinct, iterative, step. It may take doing such a statement a few times before moving forward to writing the actual paper.
• Search: Write down your keyword sets, your updated keyword sets, and databases. It is perfectly within a reviewer’s rights to ask for these details.
• Summary: Really getting to know major themes requires some annotation of articles. You want to identify core papers and themes and write about them. This helps you really learn the material. [ChatGPT or Wikipedia are no substitute for deep engagement with a paper.]
• Argument: Either outline or create a slide deck that help you express the arguments in your paper. Read them out loud. Have friends look at them. Present them. [Every literature review has an argument. If not, it’s a summary. A summary does not merit publication in a top outlet.]
• Unpacking: Once you’ve nailed the short pitch, unpack the full argument. [ a) Take time in each major section to map out a) the argument, b) the supporting evidence, and the takeaway. b) Take those major sections, reconcile them, make sure they don’t overlap, then move on to writing. c) Sketch out the paper’s sections, tables, figures, and appendices.]
• Writing: Writing is the easy part. You can always put words to the screen. [Revising and improving is hard. Make time to write every day. Improving requires feedback. Find a writing partner to give feedback. Create your tables and figures. Write to them. Make sure the words in the paper align to the visuals.]
• Communicate: When the paper is done, go back and create a paper presentation. [I do this for the papers that I’m most serious about. The act of storyboarding helps me sort out the small pieces of the story that don’t fit together. If I really want it to succeed, I present it. The act of presenting helps me get it right. My best papers sometimes take seven or eight presentations to get it right. Then I return to the paper and fine tune it. Only then, does it have a shot at a top outlet.]

Literature reviews can be published as part of a scholarly article, often after the introduction and sometimes with a header, but they can also be published as a standalone essay. To find examples of what reviews look like in your discipline, choose an appropriate subject database (such as MLA for literary criticism) and conduct a keyword search with the term “Literature Review” added in quotes:

Not only do these examples demonstrate how to structure different types of literature reviews, but some offer insights into trends and directions for future research. In the next section, we’ll take a closer look at some reading strategies to help guide you through this process.

Since scholars already have produced literature reviews on various scholarly conversations, you don’t always need to “reinvent the wheel” (start a literature review from nothing). You can find a published literature review and update it or amend it; scholars do that all the time. However, you must properly cite work you incorporate from others.

• What types of literature review will you be using for your paper? Why did you make this selection over others? If you haven’t made a selection yet, which types are you considering?
• What specific challenges do you face in following a literature review structure?
• If there are any elements of your assignment that need clarification, please list them.
• What was the most important lesson you learned from this page? What point was confusing or difficult to understand?
• In the “Back Matter” of this book, you will find a page titled “Rubrics.” On that page, we provide a rubric for Creating a Literature Review ↵
• Richard West, Brigham Young University, amended by Jason Thatcher, Temple University - https://www.linkedin.com/posts/jason-thatcher-0329764_academicwriting-topten2023-activity-7146507675021766656-BB0O ↵

Writing the Literature Review Copyright © 2021 by Barry Mauer and John Venecek is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

Free Al Office Suite with PDF Editor

Edit Word, Excel, and PPT for FREE.

Read, edit, and convert PDFs with the powerful PDF toolkit.

Microsoft-like interface, easy to use.

Windows • MacOS • Linux • iOS • Android

• Articles of Word

How to Write A Literature Review - Steps with Examples

This is something, as a student, I remember very well: writing literature reviews. They were always assigned, yet few of us knew how to write anything really impactful. For me, a good literature review is not the mere act of summarizing; rather, it is analysis, synthesis, and illumination all about discovered knowledge gaps. So let us break it down step-by-step and notice exactly how WPS Office can be used as your secret weapon in getting this one right.

What is a Literature Review & What are the Parts of it? 

It's not a book report for adults—a literature review is a critical examination of research that is already published, which plumbs deep into the scholarly conversation around your topic. Think of it this way: you are giving a guided tour through the general intellectual landscape, and you're not only pointing at landmarks but also explaining their importance, their relations to one another, and where the unknown areas can be.

The Blueprint: Introduction, Body, and Conclusion

Any well-constructed literature review will be built around the clear structure of an introduction, body, and conclusion.

Introduction: This is your opening act. Here you introduce your topic and lay out the central question or thesis your review will address. You might also give a sneak peek at the key themes or sources you'll be exploring, should you do a stand-alone review. This will also be a good place to explain how you picked and analyzed sources.

Body: This is the meat of your review. This is where you are going to put together the information from your sources in such a way that it makes sense. Again, do not just summarize, but also include your own ideas pointing out strengths as well as weaknesses of each document and relating the different studies. You will need to write clear paragraphs with effective transitions so that your reader can easily follow through the material.

Conclusion: Time to wind up: According to your literature review, there is a need to summarize the major findings and explain how they relate to your question. What are the big takeaways? What remains unanswered? Your conclusion should leave the reader with a great sense of evaluation about the present state of knowledge on a subject area and indication of where future research in this area might lead.

This framework will help you to structure a good literature review. Once more, this is only a rough expectation—remember, it is not etched in the stone. While the basic structure will usefully be applied as it is for most of the assignments or projects, sometimes maybe you will need to slightly adjust it according to the concrete needs of the assignment or project. The key is the following: Your review needs to be reader-friendly and organized, and it needs to communicate clearly the research findings.

How to organize the literature review [4 approaches]

Now that you have collected your sources and extracted their key insights, you are well on your way to developing a well-structured story. In many ways, this is akin to choosing the appropriate lens for a camera—the literature review snaps into focus. There are four common ways to approach literature review organization:

1. Chronological: This approach is almost like a timeline of ideas. You will trace the development of a topic in chronological order, so you will center on central milestones, swings in ideas, and influential debates.

2. Thematic: View this as thematically organizing your research. This will allow exploration of the subject under study in a more systematic way.

3. By Method: If you are dealing with research that utilizes a variety of methods, then this can be a revealing approach. You will draw out comparisons and contrasts between studies based on their methodology, where appropriate, pointing out the strengths and weaknesses of each approach.

4. Theoretical: This is commonly used within the humanities and social sciences, where theories are key. You will look at some of the several theoretical frameworks scholars have reached for to grasp your topic at hand, debating their strengths, limitations, and how they relate to each other.

The best approach for you will depend on what kind of research question you're asking and the body of literature involved. Don't be afraid to experiment and find the structure that works the best. You could also use a combination in your approach—like a primarily thematic approach with chronological elements there to help provide additional context for each theme.

How to Write a Literature Review Faster in 3 Steps

This type of strategic planning and effecting proper organization distinguishes an efficient literature review. The process of streamlining it is as follows:

Step 1: Gathering and Evaluating Relevant Sources

Research credible sources on academic databases like Google Scholar. Use specific keywords in order to find recent and influential publications that contribute to the topic at hand. Appraise every source according to your criteria of relevance and credibility.

Step 2: Identification of Themes and Literature Analysis

Skim through your selected sources in the search for emerging themes, debates, or gaps in the literature. Secondly, summarize key findings and methodologies for each source. Find the patterns or recurrent discussion which will help you categorize your review well and organize it.

Step 3: Outline and Structure Your Literature Review

Devise a clear structure for your literature review: introduce the topic and the thesis in the introduction, develop sources cohesively in the body, and summarize key findings in the conclusion. You could make use of organizational strategies such as chronological, thematic, methodological, or theoretical in representing your topic.

Use tools like WPS Office to plan your literature review and keep all of your sources well-organized. This will save you much time and guarantee that your literature review stays organized while you remain focused on your research objectives.

Remember: Do not simply list and summarize, but analyze and synthesize. Your literature review is not just a compilation of sources but one that critically relates the strengths and weaknesses of each piece of research, identifies the important debates in the area under consideration, and makes links between diverse pieces of research. WPS AI can help you to do this, through its identification of key terms, concepts, and relationships within the literature.

Bonus Tips: How to Perfect your Literature Review with WPS AI

Want WPS AI to be that magic weapon to help you make an extraordinary literature review? Here is how this intelligent assistant will supercharge your effort.

Annotation and Highlighting: WPS AI  permits direct annotation and highlighting of parts of interest within its software. This is quite useful to facilitate the marking of key findings, interesting quotes, or even areas in which authors have differed. By annotating through WPS AI, all critical points will be easy to refer to while you compose your review.

This WPS AI summarization tool will give you a condensed version of the long article or paper. It saves time by putting together exactly what the point or argument is from each source. On this, you will have a digest of several studies at your fingertips. This helps you easily compare and synthesize in your literature review.

Writing Assistance: Use WPS AI's writing tools to build your literature review section. These allow you to check the grammar, refine the sentence structure, work on the text length, and basically improve clarity. With these, you then ensure that it is well-written and easy for the readers to understand.

Build in these WPS AI features into your process of writing a literature review for refining workflow and bringing about a polished and insightful review that answers to academic standards.

FAQs about writing a literature review

Q1. what is the step before writing a literature review.

You must choose a topic, research existing literature, gather sources, determine themes, and make a defined scope of review before you begin writing your literature review.

Q2. Where should the literature review be placed within a dissertation?

Place the literature review after the introduction and before the methodology section of your dissertation.

Q3. Why do we need to write literature reviews?

Literature reviews would hence be a summary of earlier research on a topic, identification of gaps, building a context for fresh research, and devising credibility in an academic writing.

A literature review is one of the most critical steps of any research project. This aids in the placement of knowledge, pointing out the gaps, and placing one's research in a certain field. With accurate tools and strategies,or msg like WPS Office and WPS AI, the process can be streamlined in the production of quality literature reviews.

• 1. How to Write a Hook- Steps With Examples
• 2. How to Write a Call to Action - Steps with Examples
• 3. How to Write an Argumentative Essay- Steps with Examples
• 4. How to Write a Book Review [Tips with Examples]
• 5. How to Write a Proposal [ Steps & Examples]
• 6. ThinkFree Office Write Review: Features, Review and Download

15 years of office industry experience, tech lover and copywriter. Follow me for product reviews, comparisons, and recommendations for new apps and software.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Clinics (Sao Paulo)

Approaching literature review for academic purposes: The Literature Review Checklist

Debora f.b. leite.

I Departamento de Ginecologia e Obstetricia, Faculdade de Ciencias Medicas, Universidade Estadual de Campinas, Campinas, SP, BR

II Universidade Federal de Pernambuco, Pernambuco, PE, BR

III Hospital das Clinicas, Universidade Federal de Pernambuco, Pernambuco, PE, BR

Maria Auxiliadora Soares Padilha

Jose g. cecatti.

A sophisticated literature review (LR) can result in a robust dissertation/thesis by scrutinizing the main problem examined by the academic study; anticipating research hypotheses, methods and results; and maintaining the interest of the audience in how the dissertation/thesis will provide solutions for the current gaps in a particular field. Unfortunately, little guidance is available on elaborating LRs, and writing an LR chapter is not a linear process. An LR translates students’ abilities in information literacy, the language domain, and critical writing. Students in postgraduate programs should be systematically trained in these skills. Therefore, this paper discusses the purposes of LRs in dissertations and theses. Second, the paper considers five steps for developing a review: defining the main topic, searching the literature, analyzing the results, writing the review and reflecting on the writing. Ultimately, this study proposes a twelve-item LR checklist. By clearly stating the desired achievements, this checklist allows Masters and Ph.D. students to continuously assess their own progress in elaborating an LR. Institutions aiming to strengthen students’ necessary skills in critical academic writing should also use this tool.

INTRODUCTION

Writing the literature review (LR) is often viewed as a difficult task that can be a point of writer’s block and procrastination ( 1 ) in postgraduate life. Disagreements on the definitions or classifications of LRs ( 2 ) may confuse students about their purpose and scope, as well as how to perform an LR. Interestingly, at many universities, the LR is still an important element in any academic work, despite the more recent trend of producing scientific articles rather than classical theses.

The LR is not an isolated section of the thesis/dissertation or a copy of the background section of a research proposal. It identifies the state-of-the-art knowledge in a particular field, clarifies information that is already known, elucidates implications of the problem being analyzed, links theory and practice ( 3 - 5 ), highlights gaps in the current literature, and places the dissertation/thesis within the research agenda of that field. Additionally, by writing the LR, postgraduate students will comprehend the structure of the subject and elaborate on their cognitive connections ( 3 ) while analyzing and synthesizing data with increasing maturity.

At the same time, the LR transforms the student and hints at the contents of other chapters for the reader. First, the LR explains the research question; second, it supports the hypothesis, objectives, and methods of the research project; and finally, it facilitates a description of the student’s interpretation of the results and his/her conclusions. For scholars, the LR is an introductory chapter ( 6 ). If it is well written, it demonstrates the student’s understanding of and maturity in a particular topic. A sound and sophisticated LR can indicate a robust dissertation/thesis.

A consensus on the best method to elaborate a dissertation/thesis has not been achieved. The LR can be a distinct chapter or included in different sections; it can be part of the introduction chapter, part of each research topic, or part of each published paper ( 7 ). However, scholars view the LR as an integral part of the main body of an academic work because it is intrinsically connected to other sections ( Figure 1 ) and is frequently present. The structure of the LR depends on the conventions of a particular discipline, the rules of the department, and the student’s and supervisor’s areas of expertise, needs and interests.

Interestingly, many postgraduate students choose to submit their LR to peer-reviewed journals. As LRs are critical evaluations of current knowledge, they are indeed publishable material, even in the form of narrative or systematic reviews. However, systematic reviews have specific patterns 1 ( 8 ) that may not entirely fit with the questions posed in the dissertation/thesis. Additionally, the scope of a systematic review may be too narrow, and the strict criteria for study inclusion may omit important information from the dissertation/thesis. Therefore, this essay discusses the definition of an LR is and methods to develop an LR in the context of an academic dissertation/thesis. Finally, we suggest a checklist to evaluate an LR.

WHAT IS A LITERATURE REVIEW IN A THESIS?

Conducting research and writing a dissertation/thesis translates rational thinking and enthusiasm ( 9 ). While a strong body of literature that instructs students on research methodology, data analysis and writing scientific papers exists, little guidance on performing LRs is available. The LR is a unique opportunity to assess and contrast various arguments and theories, not just summarize them. The research results should not be discussed within the LR, but the postgraduate student tends to write a comprehensive LR while reflecting on his or her own findings ( 10 ).

Many people believe that writing an LR is a lonely and linear process. Supervisors or the institutions assume that the Ph.D. student has mastered the relevant techniques and vocabulary associated with his/her subject and conducts a self-reflection about previously published findings. Indeed, while elaborating the LR, the student should aggregate diverse skills, which mainly rely on his/her own commitment to mastering them. Thus, less supervision should be required ( 11 ). However, the parameters described above might not currently be the case for many students ( 11 , 12 ), and the lack of formal and systematic training on writing LRs is an important concern ( 11 ).

An institutional environment devoted to active learning will provide students the opportunity to continuously reflect on LRs, which will form a dialogue between the postgraduate student and the current literature in a particular field ( 13 ). Postgraduate students will be interpreting studies by other researchers, and, according to Hart (1998) ( 3 ), the outcomes of the LR in a dissertation/thesis include the following:

• To identify what research has been performed and what topics require further investigation in a particular field of knowledge;
• To determine the context of the problem;
• To recognize the main methodologies and techniques that have been used in the past;
• To place the current research project within the historical, methodological and theoretical context of a particular field;
• To identify significant aspects of the topic;
• To elucidate the implications of the topic;
• To offer an alternative perspective;
• To discern how the studied subject is structured;
• To improve the student’s subject vocabulary in a particular field; and
• To characterize the links between theory and practice.

A sound LR translates the postgraduate student’s expertise in academic and scientific writing: it expresses his/her level of comfort with synthesizing ideas ( 11 ). The LR reveals how well the postgraduate student has proceeded in three domains: an effective literature search, the language domain, and critical writing.

Effective literature search

All students should be trained in gathering appropriate data for specific purposes, and information literacy skills are a cornerstone. These skills are defined as “an individual’s ability to know when they need information, to identify information that can help them address the issue or problem at hand, and to locate, evaluate, and use that information effectively” ( 14 ). Librarian support is of vital importance in coaching the appropriate use of Boolean logic (AND, OR, NOT) and other tools for highly efficient literature searches (e.g., quotation marks and truncation), as is the appropriate management of electronic databases.

Language domain

Academic writing must be concise and precise: unnecessary words distract the reader from the essential content ( 15 ). In this context, reading about issues distant from the research topic ( 16 ) may increase students’ general vocabulary and familiarity with grammar. Ultimately, reading diverse materials facilitates and encourages the writing process itself.

Critical writing

Critical judgment includes critical reading, thinking and writing. It supposes a student’s analytical reflection about what he/she has read. The student should delineate the basic elements of the topic, characterize the most relevant claims, identify relationships, and finally contrast those relationships ( 17 ). Each scientific document highlights the perspective of the author, and students will become more confident in judging the supporting evidence and underlying premises of a study and constructing their own counterargument as they read more articles. A paucity of integration or contradictory perspectives indicates lower levels of cognitive complexity ( 12 ).

Thus, while elaborating an LR, the postgraduate student should achieve the highest category of Bloom’s cognitive skills: evaluation ( 12 ). The writer should not only summarize data and understand each topic but also be able to make judgments based on objective criteria, compare resources and findings, identify discrepancies due to methodology, and construct his/her own argument ( 12 ). As a result, the student will be sufficiently confident to show his/her own voice .

Writing a consistent LR is an intense and complex activity that reveals the training and long-lasting academic skills of a writer. It is not a lonely or linear process. However, students are unlikely to be prepared to write an LR if they have not mastered the aforementioned domains ( 10 ). An institutional environment that supports student learning is crucial.

Different institutions employ distinct methods to promote students’ learning processes. First, many universities propose modules to develop behind the scenes activities that enhance self-reflection about general skills (e.g., the skills we have mastered and the skills we need to develop further), behaviors that should be incorporated (e.g., self-criticism about one’s own thoughts), and each student’s role in the advancement of his/her field. Lectures or workshops about LRs themselves are useful because they describe the purposes of the LR and how it fits into the whole picture of a student’s work. These activities may explain what type of discussion an LR must involve, the importance of defining the correct scope, the reasons to include a particular resource, and the main role of critical reading.

Some pedagogic services that promote a continuous improvement in study and academic skills are equally important. Examples include workshops about time management, the accomplishment of personal objectives, active learning, and foreign languages for nonnative speakers. Additionally, opportunities to converse with other students promotes an awareness of others’ experiences and difficulties. Ultimately, the supervisor’s role in providing feedback and setting deadlines is crucial in developing students’ abilities and in strengthening students’ writing quality ( 12 ).

HOW SHOULD A LITERATURE REVIEW BE DEVELOPED?

A consensus on the appropriate method for elaborating an LR is not available, but four main steps are generally accepted: defining the main topic, searching the literature, analyzing the results, and writing ( 6 ). We suggest a fifth step: reflecting on the information that has been written in previous publications ( Figure 2 ).

First step: Defining the main topic

Planning an LR is directly linked to the research main question of the thesis and occurs in parallel to students’ training in the three domains discussed above. The planning stage helps organize ideas, delimit the scope of the LR ( 11 ), and avoid the wasting of time in the process. Planning includes the following steps:

• Reflecting on the scope of the LR: postgraduate students will have assumptions about what material must be addressed and what information is not essential to an LR ( 13 , 18 ). Cooper’s Taxonomy of Literature Reviews 2 systematizes the writing process through six characteristics and nonmutually exclusive categories. The focus refers to the reviewer’s most important points of interest, while the goals concern what students want to achieve with the LR. The perspective assumes answers to the student’s own view of the LR and how he/she presents a particular issue. The coverage defines how comprehensive the student is in presenting the literature, and the organization determines the sequence of arguments. The audience is defined as the group for whom the LR is written.
• Designating sections and subsections: Headings and subheadings should be specific, explanatory and have a coherent sequence throughout the text ( 4 ). They simulate an inverted pyramid, with an increasing level of reflection and depth of argument.
• Identifying keywords: The relevant keywords for each LR section should be listed to guide the literature search. This list should mirror what Hart (1998) ( 3 ) advocates as subject vocabulary . The keywords will also be useful when the student is writing the LR since they guide the reader through the text.
• Delineating the time interval and language of documents to be retrieved in the second step. The most recently published documents should be considered, but relevant texts published before a predefined cutoff year can be included if they are classic documents in that field. Extra care should be employed when translating documents.

Second step: Searching the literature

The ability to gather adequate information from the literature must be addressed in postgraduate programs. Librarian support is important, particularly for accessing difficult texts. This step comprises the following components:

• Searching the literature itself: This process consists of defining which databases (electronic or dissertation/thesis repositories), official documents, and books will be searched and then actively conducting the search. Information literacy skills have a central role in this stage. While searching electronic databases, controlled vocabulary (e.g., Medical Subject Headings, or MeSH, for the PubMed database) or specific standardized syntax rules may need to be applied.

In addition, two other approaches are suggested. First, a review of the reference list of each document might be useful for identifying relevant publications to be included and important opinions to be assessed. This step is also relevant for referencing the original studies and leading authors in that field. Moreover, students can directly contact the experts on a particular topic to consult with them regarding their experience or use them as a source of additional unpublished documents.

Before submitting a dissertation/thesis, the electronic search strategy should be repeated. This process will ensure that the most recently published papers will be considered in the LR.

• Selecting documents for inclusion: Generally, the most recent literature will be included in the form of published peer-reviewed papers. Assess books and unpublished material, such as conference abstracts, academic texts and government reports, are also important to assess since the gray literature also offers valuable information. However, since these materials are not peer-reviewed, we recommend that they are carefully added to the LR.

This task is an important exercise in time management. First, students should read the title and abstract to understand whether that document suits their purposes, addresses the research question, and helps develop the topic of interest. Then, they should scan the full text, determine how it is structured, group it with similar documents, and verify whether other arguments might be considered ( 5 ).

Third step: Analyzing the results

Critical reading and thinking skills are important in this step. This step consists of the following components:

• Reading documents: The student may read various texts in depth according to LR sections and subsections ( defining the main topic ), which is not a passive activity ( 1 ). Some questions should be asked to practice critical analysis skills, as listed below. Is the research question evident and articulated with previous knowledge? What are the authors’ research goals and theoretical orientations, and how do they interact? Are the authors’ claims related to other scholars’ research? Do the authors consider different perspectives? Was the research project designed and conducted properly? Are the results and discussion plausible, and are they consistent with the research objectives and methodology? What are the strengths and limitations of this work? How do the authors support their findings? How does this work contribute to the current research topic? ( 1 , 19 )
• Taking notes: Students who systematically take notes on each document are more readily able to establish similarities or differences with other documents and to highlight personal observations. This approach reinforces the student’s ideas about the next step and helps develop his/her own academic voice ( 1 , 13 ). Voice recognition software ( 16 ), mind maps ( 5 ), flowcharts, tables, spreadsheets, personal comments on the referenced texts, and note-taking apps are all available tools for managing these observations, and the student him/herself should use the tool that best improves his/her learning. Additionally, when a student is considering submitting an LR to a peer-reviewed journal, notes should be taken on the activities performed in all five steps to ensure that they are able to be replicated.

Fourth step: Writing

The recognition of when a student is able and ready to write after a sufficient period of reading and thinking is likely a difficult task. Some students can produce a review in a single long work session. However, as discussed above, writing is not a linear process, and students do not need to write LRs according to a specific sequence of sections. Writing an LR is a time-consuming task, and some scholars believe that a period of at least six months is sufficient ( 6 ). An LR, and academic writing in general, expresses the writer’s proper thoughts, conclusions about others’ work ( 6 , 10 , 13 , 16 ), and decisions about methods to progress in the chosen field of knowledge. Thus, each student is expected to present a different learning and writing trajectory.

In this step, writing methods should be considered; then, editing, citing and correct referencing should complete this stage, at least temporarily. Freewriting techniques may be a good starting point for brainstorming ideas and improving the understanding of the information that has been read ( 1 ). Students should consider the following parameters when creating an agenda for writing the LR: two-hour writing blocks (at minimum), with prespecified tasks that are possible to complete in one section; short (minutes) and long breaks (days or weeks) to allow sufficient time for mental rest and reflection; and short- and long-term goals to motivate the writing itself ( 20 ). With increasing experience, this scheme can vary widely, and it is not a straightforward rule. Importantly, each discipline has a different way of writing ( 1 ), and each department has its own preferred styles for citations and references.

Fifth step: Reflecting on the writing

In this step, the postgraduate student should ask him/herself the same questions as in the analyzing the results step, which can take more time than anticipated. Ambiguities, repeated ideas, and a lack of coherence may not be noted when the student is immersed in the writing task for long periods. The whole effort will likely be a work in progress, and continuous refinements in the written material will occur once the writing process has begun.

LITERATURE REVIEW CHECKLIST

In contrast to review papers, the LR of a dissertation/thesis should not be a standalone piece or work. Instead, it should present the student as a scholar and should maintain the interest of the audience in how that dissertation/thesis will provide solutions for the current gaps in a particular field.

A checklist for evaluating an LR is convenient for students’ continuous academic development and research transparency: it clearly states the desired achievements for the LR of a dissertation/thesis. Here, we present an LR checklist developed from an LR scoring rubric ( 11 ). For a critical analysis of an LR, we maintain the five categories but offer twelve criteria that are not scaled ( Figure 3 ). The criteria all have the same importance and are not mutually exclusive.

First category: Coverage

1. justified criteria exist for the inclusion and exclusion of literature in the review.

This criterion builds on the main topic and areas covered by the LR ( 18 ). While experts may be confident in retrieving and selecting literature, postgraduate students must convince their audience about the adequacy of their search strategy and their reasons for intentionally selecting what material to cover ( 11 ). References from different fields of knowledge provide distinct perspective, but narrowing the scope of coverage may be important in areas with a large body of existing knowledge.

Second category: Synthesis

2. a critical examination of the state of the field exists.

A critical examination is an assessment of distinct aspects in the field ( 1 ) along with a constructive argument. It is not a negative critique but an expression of the student’s understanding of how other scholars have added to the topic ( 1 ), and the student should analyze and contextualize contradictory statements. A writer’s personal bias (beliefs or political involvement) have been shown to influence the structure and writing of a document; therefore, the cultural and paradigmatic background guide how the theories are revised and presented ( 13 ). However, an honest judgment is important when considering different perspectives.

3. The topic or problem is clearly placed in the context of the broader scholarly literature

The broader scholarly literature should be related to the chosen main topic for the LR ( how to develop the literature review section). The LR can cover the literature from one or more disciplines, depending on its scope, but it should always offer a new perspective. In addition, students should be careful in citing and referencing previous publications. As a rule, original studies and primary references should generally be included. Systematic and narrative reviews present summarized data, and it may be important to cite them, particularly for issues that should be understood but do not require a detailed description. Similarly, quotations highlight the exact statement from another publication. However, excessive referencing may disclose lower levels of analysis and synthesis by the student.

4. The LR is critically placed in the historical context of the field

Situating the LR in its historical context shows the level of comfort of the student in addressing a particular topic. Instead of only presenting statements and theories in a temporal approach, which occasionally follows a linear timeline, the LR should authentically characterize the student’s academic work in the state-of-art techniques in their particular field of knowledge. Thus, the LR should reinforce why the dissertation/thesis represents original work in the chosen research field.

5. Ambiguities in definitions are considered and resolved

Distinct theories on the same topic may exist in different disciplines, and one discipline may consider multiple concepts to explain one topic. These misunderstandings should be addressed and contemplated. The LR should not synthesize all theories or concepts at the same time. Although this approach might demonstrate in-depth reading on a particular topic, it can reveal a student’s inability to comprehend and synthesize his/her research problem.

6. Important variables and phenomena relevant to the topic are articulated

The LR is a unique opportunity to articulate ideas and arguments and to purpose new relationships between them ( 10 , 11 ). More importantly, a sound LR will outline to the audience how these important variables and phenomena will be addressed in the current academic work. Indeed, the LR should build a bidirectional link with the remaining sections and ground the connections between all of the sections ( Figure 1 ).

7. A synthesized new perspective on the literature has been established

The LR is a ‘creative inquiry’ ( 13 ) in which the student elaborates his/her own discourse, builds on previous knowledge in the field, and describes his/her own perspective while interpreting others’ work ( 13 , 17 ). Thus, students should articulate the current knowledge, not accept the results at face value ( 11 , 13 , 17 ), and improve their own cognitive abilities ( 12 ).

Third category: Methodology

8. the main methodologies and research techniques that have been used in the field are identified and their advantages and disadvantages are discussed.

The LR is expected to distinguish the research that has been completed from investigations that remain to be performed, address the benefits and limitations of the main methods applied to date, and consider the strategies for addressing the expected limitations described above. While placing his/her research within the methodological context of a particular topic, the LR will justify the methodology of the study and substantiate the student’s interpretations.

9. Ideas and theories in the field are related to research methodologies

The audience expects the writer to analyze and synthesize methodological approaches in the field. The findings should be explained according to the strengths and limitations of previous research methods, and students must avoid interpretations that are not supported by the analyzed literature. This criterion translates to the student’s comprehension of the applicability and types of answers provided by different research methodologies, even those using a quantitative or qualitative research approach.

Fourth category: Significance

10. the scholarly significance of the research problem is rationalized.

The LR is an introductory section of a dissertation/thesis and will present the postgraduate student as a scholar in a particular field ( 11 ). Therefore, the LR should discuss how the research problem is currently addressed in the discipline being investigated or in different disciplines, depending on the scope of the LR. The LR explains the academic paradigms in the topic of interest ( 13 ) and methods to advance the field from these starting points. However, an excess number of personal citations—whether referencing the student’s research or studies by his/her research team—may reflect a narrow literature search and a lack of comprehensive synthesis of ideas and arguments.

11. The practical significance of the research problem is rationalized

The practical significance indicates a student’s comprehensive understanding of research terminology (e.g., risk versus associated factor), methodology (e.g., efficacy versus effectiveness) and plausible interpretations in the context of the field. Notably, the academic argument about a topic may not always reflect the debate in real life terms. For example, using a quantitative approach in epidemiology, statistically significant differences between groups do not explain all of the factors involved in a particular problem ( 21 ). Therefore, excessive faith in p -values may reflect lower levels of critical evaluation of the context and implications of a research problem by the student.

Fifth category: Rhetoric

12. the lr was written with a coherent, clear structure that supported the review.

This category strictly relates to the language domain: the text should be coherent and presented in a logical sequence, regardless of which organizational ( 18 ) approach is chosen. The beginning of each section/subsection should state what themes will be addressed, paragraphs should be carefully linked to each other ( 10 ), and the first sentence of each paragraph should generally summarize the content. Additionally, the student’s statements are clear, sound, and linked to other scholars’ works, and precise and concise language that follows standardized writing conventions (e.g., in terms of active/passive voice and verb tenses) is used. Attention to grammar, such as orthography and punctuation, indicates prudence and supports a robust dissertation/thesis. Ultimately, all of these strategies provide fluency and consistency for the text.

Although the scoring rubric was initially proposed for postgraduate programs in education research, we are convinced that this checklist is a valuable tool for all academic areas. It enables the monitoring of students’ learning curves and a concentrated effort on any criteria that are not yet achieved. For institutions, the checklist is a guide to support supervisors’ feedback, improve students’ writing skills, and highlight the learning goals of each program. These criteria do not form a linear sequence, but ideally, all twelve achievements should be perceived in the LR.

CONCLUSIONS

A single correct method to classify, evaluate and guide the elaboration of an LR has not been established. In this essay, we have suggested directions for planning, structuring and critically evaluating an LR. The planning of the scope of an LR and approaches to complete it is a valuable effort, and the five steps represent a rational starting point. An institutional environment devoted to active learning will support students in continuously reflecting on LRs, which will form a dialogue between the writer and the current literature in a particular field ( 13 ).

The completion of an LR is a challenging and necessary process for understanding one’s own field of expertise. Knowledge is always transitory, but our responsibility as scholars is to provide a critical contribution to our field, allowing others to think through our work. Good researchers are grounded in sophisticated LRs, which reveal a writer’s training and long-lasting academic skills. We recommend using the LR checklist as a tool for strengthening the skills necessary for critical academic writing.

AUTHOR CONTRIBUTIONS

Leite DFB has initially conceived the idea and has written the first draft of this review. Padilha MAS and Cecatti JG have supervised data interpretation and critically reviewed the manuscript. All authors have read the draft and agreed with this submission. Authors are responsible for all aspects of this academic piece.

ACKNOWLEDGMENTS

We are grateful to all of the professors of the ‘Getting Started with Graduate Research and Generic Skills’ module at University College Cork, Cork, Ireland, for suggesting and supporting this article. Funding: DFBL has granted scholarship from Brazilian Federal Agency for Support and Evaluation of Graduate Education (CAPES) to take part of her Ph.D. studies in Ireland (process number 88881.134512/2016-01). There is no participation from sponsors on authors’ decision to write or to submit this manuscript.

No potential conflict of interest was reported.

1 The questions posed in systematic reviews usually follow the ‘PICOS’ acronym: Population, Intervention, Comparison, Outcomes, Study design.

2 In 1988, Cooper proposed a taxonomy that aims to facilitate students’ and institutions’ understanding of literature reviews. Six characteristics with specific categories are briefly described: Focus: research outcomes, research methodologies, theories, or practices and applications; Goals: integration (generalization, conflict resolution, and linguistic bridge-building), criticism, or identification of central issues; Perspective: neutral representation or espousal of a position; Coverage: exhaustive, exhaustive with selective citations, representative, central or pivotal; Organization: historical, conceptual, or methodological; and Audience: specialized scholars, general scholars, practitioners or policymakers, or the general public.

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Advancements in pediatric audiological assessments using wideband acoustic immittance: a review.

1. Introduction

2. establishing normative data for pediatric wideband acoustic immittance, 2.1. comparative analysis of wideband absorbance at different ages, 2.2. comparative analysis of wba patterns in both children and adults, 2.3. comparative analysis of wideband absorbance between ethnicities.

Click here to enlarge figure

3. Characteristics of WAI in Ear Diseases

3.1. middle-ear diseases, 3.2. inner-ear diseases, 3.3. other applications, 3.4. application summary.

4. Further Clinical Implications with Case Studies

5. case 1: bilateral large vestibular aqueduct syndrome (lvas), 5.1. patient background and initial assessment, 5.2. wai findings, 5.3. clinical significance, 6. case 2: cholesteatoma, 6.1. medical history, 6.2. clinical examination, 6.3. wai findings, 6.4. surgical intervention, 6.5. clinical significance, 7. limitations, 8. future studies, 9. conclusions, author contributions, institutional review board statement, informed consent statement, data availability statement, conflicts of interest.

• Rosowski, J.J.; Wilber, L.A. Acoustic Immittance, Absorbance, and Reflectance in the Human Ear Canal. Semin. Hear. 2015 , 36 , 11–28. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Tanno, G.A.Y.; Santos, M.A.O.; Sanches, M.T.D.; Durante, A.S.; Almeida, K.; Gameiro, M.S.; Roque, N.; Sousa Neto, O.M. Analysis of wideband tympanometry in Meniere’s disease. Braz. J. Otorhinolaryngol. 2020 , 88 , 194–203. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Kei, J.; Zhao, F. Assessing Middle Ear Function in Infants ; Plural Publishing: San Diego, CA, USA, 2012. [ Google Scholar ]
• Aithal, S.; Aithal, V.; Kei, J.; Manuel, A. Effect of Negative Middle Ear Pressure and Compensated Pressure on Wideband Absorbance and Otoacoustic Emissions in Children. J. Speech Lang. Hear. Res. 2019 , 62 , 3516–3530. [ Google Scholar ] [ CrossRef ]
• Kaya, S.; Cicek Cinar, B.; Ozbal Batuk, M.; Ozgen, B.; Sennaroglu, G.; Genc, G.A.; Sennaroglu, L. Wideband tympanometry findings in inner ear malformations. Auris Nasus Larynx 2020 , 47 , 220–226. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Demir, E.; Afacan, N.N.; Celiker, M.; Celiker, F.B.; İnecikli, M.F.; Terzi, S.; Dursun, E. Can Wideband Tympanometry Be Used as a Screening Test for Superior Semicircular Canal Dehiscence? Clin. Exp. Otorhinolaryngol. 2019 , 12 , 249–254. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Mozaffari, M.; Nash, R.; Tucker, A.S. Anatomy and Development of the Mammalian External Auditory Canal: Implications for Understanding Canal Disease and Deformity. Front. Cell Dev. Biol. 2021 , 8 , 617354. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Campos Ude, P.; Hatzopoulos, S.; Śliwa, L.K.; Skarżyński, P.H.; Jędrzejczak, W.W.; Skarżyński, H.; Carvallo, R.M. Relationship Between Distortion Product—Otoacoustic Emissions (DPOAEs) and High-Frequency Acoustic Immittance Measures. Med. Sci. Monit. 2016 , 22 , 2028–2034. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Sanford, C.A.; Keefe, D.H.; Liu, Y.W.; Fitzpatrick, D.; McCreery, R.W.; Lewis, D.E.; Gorga, M.P. Sound-conduction effects on distortion-product otoacoustic emission screening outcomes in newborn infants: Test performance of wideband acoustic transfer functions and 1-kHz tympanometry. Ear Hear. 2009 , 30 , 635–652. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Hearing, J.C.o.I. Year 2007 position statement: Principles and guidelines for early hearing detection and intervention programs. Pediatrics 2007 , 120 , 898–921. [ Google Scholar ]
• Cain, S.E.; Gomes, T.; Leisner, D.; Lenzen, N.; Rall, E.; Schicke, E.; Uhler, K.M. Assessment of Hearing in Infants and Young Children: Clinical Guidance Document ; American Academy of Audiology: Reston, VA, USA, 2020; Available online: https://www.audiology.org/wp-content/uploads/2021/05/Clin-Guid-Doc_Assess_Hear_Infants_Children_1.23.20.pdf (accessed on 1 August 2024).
• Lin, D.; Lin, S.; Lin, Y.; Shu, B.; Hu, J.; Ye, S. Characteristics of Wideband Acoustic Immittance (WAI) Energy Absorption Rate in Infants and Young ChiIdren and AnaIysis of ReIated Factors. J. Audiol. Speech Pathol. 2022 , 30 , 523–527. [ Google Scholar ]
• Peng, S.; Li, G.; Yin, H.L.; Yao, X.Y. A preliminary study on wideband acoustic immittance of infants with normal bilateral middle ear function at the age of 1–5 months. J. Clin. Otorhinolaryngol. Head. Neck Surg. 2023 , 37 , 231–234. [ Google Scholar ]
• Stuppert, L.; Nospes, S.; Bohnert, A.; Läßig, A.K.; Limberger, A.; Rader, T. Clinical benefit of wideband-tympanometry: A pediatric audiology clinical study. Eur. Arch. Otorhinolaryngol. 2019 , 276 , 2433–2439. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Pan, J.; Yang, J. The clinical value of wideband tympanometry in the diagnosis of otitis media with effusion. J. Clin. Otorhinolaryngol. Head. Neck Surg. 2018 , 32 , 1309–1315. [ Google Scholar ]
• Beers, A.N.; Shahnaz, N.; Westerberg, B.D.; Kozak, F.K. Wideband reflectance in normal Caucasian and Chinese school-aged children and in children with otitis media with effusion. Ear Hear. 2010 , 31 , 221–233. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Shahnaz, N.; Aithal, S.; Bargen, G.A. Wideband Acoustic Immittance in Children. Semin. Hear. 2023 , 44 , 46–64. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Guo, Q.; Yao, H.; Sun, X.; Zhuang, R.; Zou, B. Study of the Wideband Absorbance in Chinese Pre-school Children with Normal Hearing. J. Audiol. Speech Pathol. 2021 , 29 , 373–377. [ Google Scholar ] [ CrossRef ]
• Durante, A.S.; Santos, M.; Roque, N.; Gameiro, M.S.; Almeida, K.; Sousa Neto, O.M. Wideband acoustic absorbance in children with Down syndrome. Braz. J. Otorhinolaryngol. 2019 , 85 , 193–198. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Sunil, L.K.; Perry, J.; Hudson, S. Racial Variations in Velopharyngeal and Craniometric Morphology in Children: An Imaging Study. J. Speech Lang. Hear. Res. 2016 , 59 , 27–38. [ Google Scholar ]
• Li, A.; Du, H.; Gao, J.; Xu, Y.; Zhao, N.; Gao, S.; Xing, Y.; Qian, X.; Gao, X.; Yang, Y. Characteristics of large vestibular aqueduct syndrome in wideband acoustic immittance. Front. Neurosci. 2023 , 17 , 1185033. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Lv, S. Age, Gender and Middle Ear Effusion Effects on Wideband Energy Reflectance. Master’s Thesis, Zhengzhou University, Zhengzhou, China, 2018. [ Google Scholar ]
• Sun, J.; Fu, Y.; Xu, B.; Bi, J.; Liu, J. Application of Wideband Tympanometry to Middle Ear Function Evaluation in Infants Aged 0–24 Months. Chin. J. Otol. 2021 , 19 , 258–264. [ Google Scholar ]
• Xing, Y.; Jiang, W.; Tong, Y.; Wu, Y.; Liu, W.; Qiao, Y. A preliminary study on the energy absorbance of wideband acoustic immittance in young adults with normal hearing. J. Otolaryngol. Ophthalmol. Shandong Univ. 2020 , 34 , 38–41. [ Google Scholar ] [ CrossRef ]
• Zhao, Z.; Lin, Y.; Ren, C.; Li, L.; Fan, X.; Guo, Q.; Zha, D. Characteristics of Wideband Acoustic Immittance in Patients with Secretory Otitis Media. J. Audiol. Speech Pathol. 2021 , 29 , 635–638. [ Google Scholar ]
• Karuppannan, A.; Barman, A. Wideband absorbance tympanometry: A novel method in identifying otosclerosis. Eur. Arch. Oto-Rhino-Laryngol. 2021 , 278 , 4305–4314. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Wang, X.; Su, W.; Luo, X. The diagnosticvalueof wideband acousticimmittancein otosclerosisand congenital auditoIy bone chain deformity. Chin. Arch. Otolaryngol.-Head Neck Surg. 2023 , 30 , 10–12. [ Google Scholar ]
• Keefe, D.H.; Sanford, C.A.; Ellison, J.C.; Fitzpatrick, D.F.; Gorga, M.P. Wideband aural acoustic absorbance predicts conductive hearing loss in children. Int. J. Audiol. 2012 , 51 , 880–891. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Pieterse, H.; Biagio-De Jager, L.; Hofmeyr, L.M.; Vinck, B. Wideband acoustic immittance in superior semicircular canal dehiscence. Auris Nasus Larynx 2022 , 49 , 921–927. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Zhang, L.; Wang, J.; Zhao, F.; Li, Y. Inner ear pressure evaluation using wideband tympanometry in children with Large Vestibular Aqueduct Syndrome (LVAS): A pilot study. Int. J. Pediatr. Otorhinolaryngol. 2020 , 128 , 109690. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Sato, E.; Nakashima, T.; Lilly, D.J.; Fausti, S.A.; Ueda, H.; Misawa, H.; Uchida, Y.; Furuhashi, A.; Asahi, K.; Naganawa, S. Tympanometric findings in patients with enlarged vestibular aqueducts. Laryngoscope 2002 , 112 , 1642–1646. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Zhang, X.; Tian, M.; Zhang, L. Correlation analysis of Wideband Acoustic Immittance with middle ear effusion viscosity and air bone conduction difference in children with otitis media. J. Med. Theory Pract. 2023 , 36 , 1932–1934. [ Google Scholar ]
• Callaham, S.; Newby, M.; Saoji, A.A.; Ramadan, J.; Carr, M.M. Assessment of pediatric middle ear effusions with wideband tympanometry. Otolaryngol. Head Neck Surg. 2021 , 165 , 465–469. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Jiang, W.; Huang, N.; Tian, C. Acoustic Absorbance Characteristics of Wideband Tympanometry in Children with Unilateral Cochlear Implant. J. Audiol. Speech Pathol. 2021 , 29 , 369–372. [ Google Scholar ]
• Kaf, W.A. Wideband energy reflectance findings in presence of normal tympanogram in children with Down’s syndrome. Int. J. Pediatr. Otorhinolaryngol. 2011 , 75 , 219–226. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Shahnaz, N.; Feeney Mp Fau—Schairer, K.S.; Schairer, K.S. Wideband acoustic immittance normative data: Ethnicity, gender, aging, and instrumentation. Ear Hear. 2013 , 34 , 27s–35s. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Merchant, G.R.; Al-Salim, S.; Tempero, R.M.; Fitzpatrick, D.; Neely, S.T. Improving the Differential Diagnosis of Otitis Media With Effusion Using Wideband Acoustic Immittance. Ear Hear 2021 , 42 , 1183–1194. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Sheng, H.; Zhou, Q.; Li, H.; Ren, Y.; Han, K.; Huang, M.; Chen, Y.; Li, Q.; Li, W.; Huang, Z. Characteristics and clinical significance of wideband tympanometry of otitis media with effusion in young children. Chin. J. Otorhinolaryngol. Head Neck Surg. 2021 , 56 , 579–585. [ Google Scholar ]
• Gao, M.; Yin, D.; Fan, M.; Dou, X. Eigenvalue analysis of wideband acoustic immittance in secretory otitis media in children of different ages. J. Clin. Otorhinolaryngol. Head Neck Surg. 2022 , 36 , 180–184. [ Google Scholar ]
• Miehe, J.; Mogensen, S.; Lyhne, N.; Skals, R.; Hougaard, D.D. Wideband tympanometry as a diagnostic tool for Meniere’s disease: A retrospective case-control study. Eur. Arch. Oto-Rhino-Laryngol. 2022 , 279 , 1831–1841. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Riggs, W.; Hiss, M.; Varadarajan, V.; Mattingly, J.; Adunka, O. Enlarged vestibular aqueduct: Intraoperative electrocochleography findings during cochlear implantation. Int. J. Pediatr. Otorhinolaryngol. 2020 , 134 , 110065. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Valvassori, G.E.; Clemis, J.D. The large vestibular aqueduct syndrome. Laryngoscope 1978 , 88 , 723–728. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Madden, C.; Halsted, M.; Benton, C.; Greinwald, J.; Choo, D. Enlarged vestibular aqueduct syndrome in the pediatric population. Otol. Neurotol. 2003 , 24 , 625–632. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Noguchi, Y.; Fukuda, S.; Fukushima, K.; Gyo, K.; Hara, A.; Nakashima, T.; Ogawa, K.; Okamoto, M.; Sato, H.; Usami, S.I.; et al. A nationwide study on enlargement of the vestibular aqueduct in Japan. Auris Nasus Larynx 2017 , 44 , 33–39. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Hura, N.; Stewart, M.; Walsh, J. Progression of hearing loss and cochlear implantation in large vestibular aqueduct syndrome. Int. J. Pediatr. Otorhinolaryngol. 2020 , 135 , 110133. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Jiang, W.; Li, X.; Mu, Y.; Zhang, H.; Konduru, N.; Qiao, Y.; Zhao, F.; Liu, W. Predictive accuracy of wideband absorbance in children with large vestibular aqueduct syndrome: A single-center retrospective study. Heliyon 2024 , 10 , E33776. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Ding, L.; Wang, C.; Shi, W. A Preliminary Study on Characteristics of Tympanic Membrane Absorbance in Patients with Large Vestibular Aqueduct Syndrome. Chin. J. Otol. 2021 , 19 , 16–20. [ Google Scholar ]
• Soares, J.C.; Urosas, J.G.; Calarga, K.S.; Pichelli, T.S.; Limongi, S.C.; Shahnaz, N.; Carvallo, R.M. Wideband reflectance in Down syndrome. Int. J. Pediatr. Otorhinolaryngol. 2016 , 87 , 164–171. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Merchant, G.R.; Schulz, K.M.; Patterson, J.N.; Fitzpatrick, D.; Janky, K.L. Effect of Cochlear Implantation on Vestibular Evoked Myogenic Potentials and Wideband Acoustic Immittance. Ear Hear 2020 , 41 , 1111–1124. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Wu, S.; Guan, R.; Wang, Y.; Zhang, B.; Li, P.; Shen, F.; Sun, J.; Sun, J. The Characteristics of Wideband Absorbance (WBA) in Infants with Cochlear Implant. J. Audiol. Speech Pathol. 2021 , 29 , 361–364. [ Google Scholar ]
• Merchant, G.R.; Neely, S.T. Conductive Hearing Loss Estimated from Wideband Acoustic Immittance Measurements in Ears With Otitis Media With Effusion. Ear Hear. 2023 , 44 , 721–731. [ Google Scholar ] [ CrossRef ]
• Liang, J.; Xiao, L.; Sun, X.Y.; Zou, B. Characteristics of the wideband absorbance of acoustic energy in children (3–7 years old) with otitis media with effusion. Int. J. Pediatr. Otorhinolaryngol. 2021 , 140 , 110496. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Aithal, V.; Aithal, S.; Kei, J.; Anderson, S.; Wright, D. Predictive Accuracy of Wideband Absorbance at Ambient and Tympanometric Peak Pressure Conditions in Identifying Children with Surgically Confirmed Otitis Media with Effusion. J. Am. Acad. Audiol. 2020 , 31 , 471–484. [ Google Scholar ] [ CrossRef ]
• Emara, A.A. Acoustically evoked, short latency negative response in children with sensorineural hearing loss. J. Laryngol. Otol. 2010 , 124 , 141–146. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Nong, D.X.; Ura M Fau-Kyuna, A.; Kyuna A Fau-Owa, T.; Owa T Fau-Noda, Y.; Noda, Y. Saccular origin of acoustically evoked short latency negative response. Otol. Neurotol. 2002 , 23 , 953–957. [ Google Scholar ] [ CrossRef ] [ PubMed ]

Share and Cite

Jiang, W.; Mu, Y.; Zhao, F.; Wang, P. Advancements in Pediatric Audiological Assessments Using Wideband Acoustic Immittance: A Review. Audiol. Res. 2024 , 14 , 684-700. https://doi.org/10.3390/audiolres14040058

Jiang W, Mu Y, Zhao F, Wang P. Advancements in Pediatric Audiological Assessments Using Wideband Acoustic Immittance: A Review. Audiology Research . 2024; 14(4):684-700. https://doi.org/10.3390/audiolres14040058

Jiang, Wen, Yi Mu, Fei Zhao, and Peng Wang. 2024. "Advancements in Pediatric Audiological Assessments Using Wideband Acoustic Immittance: A Review" Audiology Research 14, no. 4: 684-700. https://doi.org/10.3390/audiolres14040058

Article Metrics

Further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

A review of evaluation approaches for explainable AI with applications in cardiology

• Open access
• Published: 09 August 2024
• Volume 57 , article number  240 , ( 2024 )

Cite this article

You have full access to this open access article

• Ahmed M. Salih 1 , 2 , 3 ,
• Ilaria Boscolo Galazzo 4 ,
• Polyxeni Gkontra 5 ,
• Elisa Rauseo 1 ,
• Aaron Mark Lee 1 ,
• Karim Lekadir 5 , 6 ,
• Petia Radeva 7 ,
• Steffen E. Petersen 1 , 8 , 9 , 10 &
• Gloria Menegaz 4  

164 Accesses

1 Altmetric

Explore all metrics

Explainable artificial intelligence (XAI) elucidates the decision-making process of complex AI models and is important in building trust in model predictions. XAI explanations themselves require evaluation as to accuracy and reasonableness and in the context of use of the underlying AI model. This review details the evaluation of XAI in cardiac AI applications and has found that, of the studies examined, 37% evaluated XAI quality using literature results, 11% used clinicians as domain-experts, 11% used proxies or statistical analysis, with the remaining 43% not assessing the XAI used at all. We aim to inspire additional studies within healthcare, urging researchers not only to apply XAI methods but to systematically assess the resulting explanations, as a step towards developing trustworthy and safe models.

Similar content being viewed by others

Explainable Artificial Intelligence in Health Care: How XAI Improves User Trust in High-Risk Decisions

The role of explainability in AI-supported medical decision-making

When an Explanation is not Enough: An Overview of Evaluation Metrics of Explainable AI Systems in the Healthcare Domain

Explore related subjects.

• Artificial Intelligence
• Medical Imaging

Avoid common mistakes on your manuscript.

1 Introduction

Cardiovascular diseases are the leading global cause of death and represent a major healthcare burden (Vaduganathan et al. 2022 ). Advanced artificial intelligence (AI) models, especially those based on deep learning, have shown success in cardiac-related applications (Karatzia et al. 2022 ), taking advantage of the increasing availability of multi-source data including cardiac imaging techniques (e.g., cardiac magnetic resonance imaging [CMR], X-ray, ultrasound, echocardiograms), electrocardiogram (ECG) and electronic health records (EHR). However, models based on as convolutional neural networks (CNN) and reinforcement learning (e.g., Markov Decision Process and Q-learning) are generally considered black box, especially when it comes to more clinically-oriented applications, as the internal mechanisms and the rationale behind model outputs are not explicit (Linardatos et al. 2020 ). It is thus difficult for clinicians to trust model predictions which cannot be interpreted and lack transparency (Linardatos et al. 2020 ; Loh et al. 2022 ). Accordingly, eXplainable AI (XAI) has been proposed as a possible solution to make AI models more transparent and comprehensible (Mohseni et al. 2021 ), and thereby to enhance understanding, increase trust, and uncover potential risks associated with complex models (Szabo et al. 2022 ). In addition, XAI has a potential use in detecting biases in the underlying AI models, leading to improved generalizability and performance. XAI has experienced significant growth over the last few years with several methods being proposed to deal with the peculiarities of the different AI models and data, and providing either local or global explanations (Selvaraju et al. 2017 ; Chattopadhay et al. 2018 ; Lundberg and Lee 2017 ; Ribeiro et al. 2016 ; Plumb et al. 2018 ).

At the same time, the quickly growing and changing field of XAI has posed new challenges in the healthcare area, including the necessity of objective evaluations of the resulting explanations (Chaddad et al. 2023 ). While evaluation methods are often grouped according to different criteria in the literature, a common way of classifying XAI approaches is according to whether user involvement is required (human-centred) or not (computational-centred) (Doshi-Velez and Kim 2017 ). In particular, three main kinds of evaluations have been proposed:1) human-grounded, 2) application-grounded, 3) functionally-grounded evaluation (Doshi-Velez and Kim 2017 ). Human-grounded evaluation indicates that the XAI explanation is assessed by lay persons, for example by selecting the most reasonable option included in specific questionnaires listing the outcomes of multiple XAI methods. Such approaches might be useful only for simple tasks and can only provide a general sense as to the validity of the explanation. Application-grounded evaluation is still human-centred but, in this case, it refers to assessments done by the experts in the specific domain, for example cardiologists in cardiac-related applications. Finally, functionally-grounded evaluation indicates that the outcome of XAI is evaluated solely by some kind of proxies, statistical methods or formal definitions of interpretability with no human intervention (computer-centred) (Doshi-Velez and Kim 2017 ).

More recently, other evaluation approaches are emerging, although these are not included in the current taxonomy. The first one can be referred to as literature-grounded evaluation , where the outcome of the XAI is assessed based on comparisons with what is known in the literature and with previous findings. The second one, guideline-grounded evaluation , requires following specific guidelines to assess the outcome of XAI. It might involve both application- and functionality-grounded evaluations where the outcomes are evaluated by experts in the domain relying on some kind of proxies.

Starting from this scenario, this review contributes to the body of knowledge of XAI evaluation approaches, methods and metrics focusing on cardiac studies. We commence with an introduction to XAI and provide the taxonomy and the main approaches for evaluating XAI outputs. We then focus on summary statistics derived from a comprehensive literature review of XAI evaluation methods in the cardiac domain, subsequently delving into the practical applications of these XAI evaluation techniques within cardiac research. Lastly, we discuss open issues and future directions.

2 Rationale XAI

Arguably, XAI should narrow the gap between model accuracy and transparency by converting black box but accurate AI models into a more understandable form. XAI helps to elucidate how a model reached a specific decision, the extent to which model is certain and what are the regions of an image or group of features that dominated the model decision.

Explainability and interpretability are often used interchangeably, which might confuse the reader about what they represent. To clarify their meanings, Table  1 provides their definitions along with those of other common terms used in XAI and generally in AI field.

Figure  1 provides of an overview of the general workflow for an efficient XAI analysis pipeline, designed, in this case, for cardiac AI applications, although readily generalizable with respect to data acquisition methods, model architecture, application of XAI methods, evaluation of XAI outcomes and final decision. When acquiring data for cardiac assessments, the selection of data modalities depends on the aim of the task at hand, including the target disease, but also on other parameters such as cost, resource availability, and time constraints.

The main data types include imaging (e.g., CMR, echocardiography, ultrasound, nuclear perfusion scans) to evaluate the structure and function of the cardiac, ECG for the assessment of the cardiac electric activity, diagnostic measurements from laboratory exams such as blood tests, and other structured and unstructured patient information from EHR (e.g., demographics, risk factors, medical history, clinical notes among others). Notably, some diagnostic measurements, signal data, images and image-derived information may also be present in the patient’s EHR, which, in this context, encompasses all other pertinent patient-related data. Nonetheless, in this review, the term “EHR data” excludes imaging and signal.

According to the specific research or clinical questions, different modelling strategies using machine learning can be designed and developed. More precisely, regression models can be used to predict a continuous variable such as cardiac age, stroke volume or cardiac function parameters, while classification models can be employed to distinguish between two cases (e.g., control vs heart failure). In addition, segmentation models can be used to segment the anatomy of the cardiac and extract CMR metrics, and reconstruction models can help to improve the quality of cardiac images.

General illustration. MRI :magnetic resonance imaging, PDP partial dependence plot, ALE accumulated local effects, Grad-CAM gradient-weighted class activation mapping, LIME local interpretable model-agnostic explanations, SHAP shapley additive explanations, ROAR RemOve And Retrain, ERASER evaluating rationales and simple English reasoning. Created with BioRender.com

Once the optimal model is defined and its performance carefully evaluated (e.g., cross validation, independent test set), XAI methods can be applied to explain and interpret the model. The most appropriate XAI method can be chosen based on the model and data types. For example, SHAP (Shapley Additive Explanations, an XAI based on game theory) can be applied to both imaging and tabular data, while Gradient-weighted Class Activation Mapping (Grad-CAM) and DeepTaylor can be implemented on imaging and signal (e.g., ECG) data. Once a given XAI method has been applied, it is important to evaluate the explanation it provides (although this step is still rarely applied in the current literature and most of the cardiac studies do not focus on this additional analysis). The final step is to evaluate whether to trust, generalize and deploy the XAI after it has been appropriately evaluated.

2.1 Taxonomy of XAI

XAI approaches are typically categorized as either “ante-hoc” or “post-hoc” methods (Salih et al. 2023b ). Ante-hoc means that the explanation is intrinsic, and the model is self-explanatory (white-box model). On the contrary, post-hoc methods require the application of another model to explain the results of the AI model. Linear regression models are examples of ante-hoc XAI methods that are simple and directly interpretable. Indeed, the regression coefficients can indicate the importance of the different predictors and how they affect the models. On the other hand, CNN models belong to the post-hoc category as they require the application of other models for interpretation.

Another criterion that can be applied to classify a given XAI method is whether it is local or global. Local indicates that the resulting explanation can be provided for a specific data point or instance in the model. On the other hand, global provides general explanations for all instances in the model, for example the impact of a specific feature in the model for all instances. Grad-CAM (Selvaraju et al. 2017 ), DeepTaylor (Montavon et al. 2017 ), Layer-Wise Relevance Propagation (Bach et al. 2015 ; Wagner et al. 2024 ), LIME (Ribeiro et al. 2016 ) and guided backpropagation (Springenberg et al. 2014 ) are examples of XAI methods that provide local explanations, while partial dependence plots (PDP) (Greenwell et al. 2018 ), accumulated local effects plots (ALE) (Apley and Zhu 2020 ) and SHAP (Lundberg and Lee 2017 ) are examples of XAI models that provide global explanation (though SHAP can provide both kinds of explanation).

In addition, XAI can be categorized into model-specific or model-agnostic. Model-specific refers to any XAI model that was developed for a specific machine learning (ML) model. Conversely, model-agnostic includes all XAI methods that can be applied to any model, regardless its complexity or simplicity. XAI methods including SHAP and LIME can be considered as model-agnostic because they can be applied to any model.

Despite many XAI methods have been developed in the past five years, little attention has been given to the evaluation part and there is no standard measure or metric to assess their outcome yet (Silva et al. 2023 ). Moreover, XAI methods often assume that the end users in any domain have a certain level of expertise which qualifies them to understand and evaluate the quality and correctness of its outcome. However, such assumption cannot be met in several cases, making difficult a fair assessment of the XAI outcome by the end users (Bruijn et al. 2022 ). Another concern related to the current XAI methods is the lack of causality in the outcome. More precisely, current AI models primarily rely on identifying associations between the input and the output, which might not necessarily imply causation. Consequently, the explanations generated by XAI methods may not accurately reflect causal association (Molnar et al. 2022 ; Chou et al. 2022 ). In addition, current XAI methods based on input perturbations lack robustness against adversarial attacks and can be fooled to produce biased results (Slack et al. 2020 ).

All points mentioned above will be better illustrated and detailed in the following sections.

3 XAI evaluation methods

In this section, we introduce the main XAI evaluation methods, following the current taxonomy and further complementing it with other approaches that we retrieved from the current studies, and we believe being relevant. As introduced in Sect. 1 , evaluation methods can be categorized as follows: human-grounded (lay person), application-grounded (expert in the domain), functionality-grounded (proxy), literature-grounded and guideline-grounded. The main examples for each category will be discussed, although for more details on each metric and method we refer the interested readers to specific reviews on this topic as it is out of the scope of the current review (Mohseni et al. 2021 ; Kumarakulasinghe et al. 2020 ; Linardatos et al. 2020 ; Lopes et al. 2022 ).

3.1 Human and application-grounded evaluations

The approaches belonging to these categories require the participation of humans in the evaluation, either lay persons (human-grounded) or domain experts (application-grounded). Here, the main challenge is that the evaluations done by humans, especially when involving lay persons, are partially subjective, as they depend on the level of expertise, main domain knowledge and individual judgment. Indeed, the same explanation can be satisfying for one user but totally incomprehensible for another and there might be a lack of consensus between participants. However, the involvement of experts in the field might partially mitigate this intrinsic limitation, thus making application-grounded evaluations more suitable especially in the healthcare domain.

In this case, qualitative measures informing on the clinical relevance, plausibility and complexity of a given XAI explanation are usually provided by the experts. The following criteria represent some of the proposed notions to qualitatively evaluate the XAI outcomes.

Completeness: It can be defined as whether the explanation is complete to the end users or not. Completeness involves full details related to the boundary of the used data, the model, the XAI method, limitations, evaluation metrics and how to interpret the results (Cui et al. 2019 ).

Simplicity: It is related to the cases where the task is well-known and related to daily-life issues where it is easy to distinguish and decide if the explanation is good or bad (Montavon et al. 2018 ).

Evaluating Rationales And Simple English Reasoning (ERASER): It is a benchmark to evaluate models applied to natural language processing applications. They proposed several metrics to evaluate the explanation considering human rationales as ground truth (DeYoung et al. 2019 ).

Plausibility: It is one of the most precious metrics to evaluate any XAI method. It measures if the explanation provided by the machine is inline with the expert explanation and expectation. In other words, it assesses whether a human is convinced by the explanation or not (Jin et al. 2023b ).

Simulatability: It indicates that the model behavior can be predicted when it is applied to new data (Hase and Bansal 2020 ). This is a significant metric as it means that the end users understand how the simulatable models work. It is divided into two tasks, the first one refers to the user ability to predict the explanation for a given input, while the second one is the ability of the users to predict the changes in the explanation when a given perturbation is applied to the input data.

Complexity: It indicates the degree of complexity of the explanation when debugging the XAI method. In simple words, it is the needed time to understand the explanation (Cui et al. 2019 ). In addition, this measure refers to the amount of information held in the XAI outcomes (Gilpin et al. 2018 ) and is a measure of conciseness, meaning that an explanation should consist of a few strong features (Chalasani et al. 2020 ), making the interpretation of the XAI outcomes easier and more robust.

Clinical relevance: It means that the explanation should be in agreement with the physicians’s opinions and support their clinical decision and reasoning (Di Martino and Delmastro 2022 ). Some proposed frameworks tried to further quantify clinical relevance by calculating additional measures such as the percentage of explanations that are accepted by physicians or the percentage of overlap between XAI and physicians explanation (Kumarakulasinghe et al. 2020 ).

Another possibility is to combine the evaluations by the experts with statistical analyses (proper of functionally-grounded evaluations) to identify whether there is an agreement between what was depicted by a given XAI method as most relevant (e.g., specific feature or imaging region) and the opinion by the expert. In this way, an objective quantification of the level of concordance can be derived and used as additional metric to evaluate the XAI outcomes.

Importantly, some limitations have to be acknowledged when relying on application-grounded evaluation. Indeed, such an approach is expensive as each study in a specific domain needs its own experts for the assessment, is time consuming, and thus might be less appropriate in critical clinical settings where immediate XAI evaluations are needed (e.g, intensive care units), and might require the involvement of more expert users when the task is particularly demanding. Moreover, for some measures such as complexity and completeness, the partial subjectiveness might still exist despite the involvement of experts, as end users with different level of expertise might lead to different opinions on these metrics.

3.2 Proxy-grounded evaluation

Functionality-grounded (or proxy-grounded) approaches represent methods that use quantitative proxies, metrics, axioms, and statistics to assess the quality of the XAI outcomes. In addition, they might use some formal definitions of explainability or interpretability to evaluate the results. Such methods are promising because they do not require human intervention or experts in the domain, and they can be applied to assess the value and robustness of novel XAI methods (Doshi-Velez and Kim 2017 ). However, some limitations must be acknowledged also in this case. Firstly, it is hard to determine which is the most suitable proxy to evaluate a given XAI method. Then, this approach does not consider clinical relevance and plausibility, as it does not involve experts. In addition, such methods might be biased by part of the data or by the adopted XAI model, making the evaluations less reliable.

In what follows, we will discuss some of the most common proxies that have been proposed so far for evaluating XAI outcomes.

Sensitivity: It indicates that if two identical models have different outputs and same input but differ in one feature, then the attribution of that feature should not be zero (Hooker et al. 2019 ; Sundararajan et al. 2017 ). In addition, if a feature does not contribute to the model output, then zero attribution should be given to that feature;

Selectivity or RemOve And Retrain (ROAR): It was proposed to measure the accuracy of attribution estimates in deep neural networks. It evaluates the changes in accuracy a given model experiences when the top features identified by XAI are removed. If a sharp reduction occurs, it is likely that the removed inputs are highly informative and that the XAI importance estimates are correct. If not, this means that the removed features hold only marginal information and thus the XAI outcomes can be considered of poor quality (Hooker et al. 2019 ; Montavon et al. 2018 );

Continuity: It means that the explanation of two instances should be nearly equivalent if their data points are also nearly equivalent (Montavon et al. 2018 ). In other words, it is the variation in the explanation in the input domain;

Correctness: It means that the explanation should correctly explain and identify the main components of the model that mostly drive the outcome (Kuppa and Le-Khac 2020 ). However, such assumption is hard to define due to the lack of ground truth. In Yalcin et al. ( 2021 ), authors defined correctness by building datasets with known explanation and then correlated the explanation with the model accuracy;

Consistency: It refers to what degree or extent the explanation will be different when different models are applied to the same data (Leventi-Peetz and Weber 2022 ). In addition, it measures how the explanation will be changed if the input data are altered or transformed compared to the explanation of the original input data (Kuppa and Le-Khac 2020 ).

Normalized movement rate (NMR): It was proposed as a measure to assess whether the XAI models are robust against the collinearity among the used predictors in the model (Salih et al. 2022 ). NMR is calculated by checking and quantifying how the predictors change their indexes in the list of the most informative predictors (from a given XAI method) when the top one is removed iteratively. The smaller the NMR value, the more robust the model against collinearity or the predictors are independent which consequently provide more reliable explanation. On the other hand, the closer the NMR value is to 1, the weaker the model against the collinearity and the explanation is not realistic.

Computation time: It is another criterion to be considered in the evaluation of the XAI outcomes. It is vital that the required time for generating an explanation is as short as possible, especially in some cases where time is very critical (Kakogeorgiou and Karantzalos 2021 ). Explainability methods requiring long computation times might be difficult to integrate in complex pipelines when real-time performance is required. However, the trade-off between computational time and accuracy/reliability of the explanations should be always considered, especially in the healthcare domain.

3.3 Literature-grounded evaluation

Besides human-centered and computer-centered approaches, XAI outcomes are often evaluated by the different researchers and users using previous literature findings as benchmarks ( literature-grounded evaluation). This category of evaluation methods is somehow close to the expert-grounded evaluation as it considers the findings from the experts in the domain. However, this approach has some drawbacks, especially in terms of subjectiveness. Indeed, the users might tend to be more selective while searching in the literature, ending up in choosing the findings that are more in line with their XAI outcomes and partially ignoring the mismatched ones. This might limit the generalizability of the XAI outcomes and might provide only a partial evaluation. While the importance of referring to the state-of-the-art to aid in evaluating a given explanation is undeniable and should be increasingly pursued in all XAI research studies in the healthcare domain, we believe that literature findings should only be used as additional confirmation to prove the reliability and plausibility of the results, and that they should be complemented with other measures and comparisons. Moreover, any different data, model or XAI method should be acknowledged, as these can have a significant impact on the outcomes and subsequent evaluation.

3.4 Guideline-grounded evaluation

Recently, another approach has been proposed to assess the quality of the XAI outcome by relying on guidelines combining both proxy and expert-grounded methods (Chen et al. 2022 ; Jin et al. 2023a ). Guideline-grounded evaluation usually assesses the outcome of XAI through a pipeline where the input is given by the XAI outcome and there is a specific evaluation criterion in every step. Seven guidelines’ steps of assessment were proposed by Jin and colleagues (Jin et al. 2023a ) to examine any XAI method and its explanations in clinical settings. Such clinical guidelines are mixing both proxy and expert methods including clinical relevance, computational efficiency, informative plausibility, truthfulness, and understandability (Jin et al. 2023b ). Another set of guidelines for medical image analysis applications were proposed by Chen et al. ( 2022 ), emerging as result of their systematic review paper on 68 studies. The proposed guideline (INTRPRT) has several parts including incorporation (IN), interpretability (IN), target (T), reporting (R), prior (PR), and task (T). The proposed INTRPRT guideline suggests a human-centered design to develop transparent AI in healthcare. More in detail, incorporation indicates including an adequate number of end-users (clinicians) to collaborate with the designers during the construction and assessment of the model. Interpretability refers to the technical aspects of the model to make the model transparent. Target determines the final users of the transparent AI algorithms. Reporting indicates summarizing all approaches and aspects used to evaluate the transparency of the model. Prior in particular points to previous findings, sources or information related to the target users. This will help the designers to understand the end-users better while designing a transparent model. Finally, task refers to the aim of the model, whether it is for segmentation, classification of prediction.

While being promising, such an approach still poses several challenges, given by the complexity in defining general and appropriate guidelines. The different applications in the medical domain might require more faceted and human-centered approaches that should increasingly involve the target end users to build together more transparent models and verify that the assumptions are valid.

4 Literature review in numbers

In the current work, we investigated the evaluation methods applied to XAI outcomes in cardiac studies within the existing literature. Following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, we conducted a search across four academic databases, namely Web of Science, Scopus, PubMed, and IEEE Xplore. The aim was to collect all published papers that applied XAI methods in any context related to cardiac applications. It should be noted that the search was limited to papers published in English language, without year restrictions. The search query consisted of four parts: (1) “cardiac” or “heart”, (2) terms related to cardiac imaging acquisition methods or cardiac data, 3) terms related to artificial intelligence methods and a wide range of XAI methods (Table S1 ). The search encompassed both paper titles and abstracts. The most recent search was conducted on 20/08/2023.

Figure  2 outlines the workflow that was followed to select the studies to include in the review according to the PRISMA guidelines. Following the initial literature search across the four repositories and subsequent removal of duplicate papers, 501 unique papers were collected. Thereafter, these papers were manually assessed to ensure alignment with the aim of this study, excluding those that did not use XAI methods or cardiac data. These steps resulted in a final sample of 213 papers to be included in the review.

Notably, most of the reviewed studies applied classification models (170) to predict a condition versus a control, and they were primarily focused on certain cardiac conditions like arrhythmia’s and ischemic heart disease (IHD). Twenty-three studies used regression models to predict a continuous variable, mainly targeting cardiac age and CMR metrics including left ventricular geometry and left ventricular ejection fraction. Few papers involved segmentation, clustering or image re-construction models in their studies.

Workflow adhering to PRISMA guidelines, detailing the exclusion and inclusion criterion used in the search process, along with the final number of papers considered in the review

Fig.  3 provides an overview of the data modalities used in the studies included in the review and for each category of evaluation approaches. ECG and EHR were the most frequently used data, followed by CMR and echocardiogram. EHR data includes (in our review) cognitive tests, lab tests metrics and any data not considered in imaging, ECG or sound data. ECG and EHR-related patient health information acquired through questionnaires are more readily accessible compared to imaging data, and particularly, CMR, which can be expensive and time consuming. Nonetheless, CMR remains the gold standard for assessing the cardiac structure and function due to its ability to provide unique, in-depth information not attainable by other means. The availability of large biomedical repositories, such as the UK Biobank (Petersen et al. 2015 ), might result in an increase in the number of studies using CMR data in the coming years. It is worth mentioning that many studies (13) employed multiple data modalities, such as ECG and EHR, ECG and CMR, simultaneously. This explains why the total count of papers for each data type exceeds the total number of studies included in the review.

Data modalities used in cardiac studies. A All cardiac studies, B cardiac studies applied proxy-grounded evaluation approaches, C cardiac studies applied expert-grounded evaluation approach, D cardiac studies applied literature-grounded evaluation approach, E cardiac studies did not apply any kind of evaluation to XAI outcomes. ECG electrocardiography, EHR electronic health records, CMR cardiac magnetic resonance imaging, CT computed tomography, EI electrocardiographic imaging, PET positron emission tomography, MPI myocardial perfusion imaging, MCTP myocardial computed tomography perfusion, HI histology images, SI scintigraphy images

Figure  4 provides the number of papers according to XAI method employed. It shows that the majority of the papers applied SHAP, followed by Grad-CAM and LIME. This can be attributed to the versatility of SHAP and LIME, which can be applied to both imaging and tabular data, as those found in EHR data. On the contrary, Grad-CAM can be applied to imaging and signal (e.g., ECG) data. This is somehow expected because these methods have attracted significant attention across various domains, including cardiac research. In addition, their ease of implementation, facilitated by publicly available packages and in multiple programming languages, has further contributed to their popularity. It should be noted that the figure shows the most frequently used XAI methods in cardiac studies, rather than an exhaustive list. In addition, there exist many studies that applied more than one XAI method in their analysis. For more details on the used XAI methods in cardiac studies, please refer to Table S2 .

Distribution of the number of cardiac studies employing different XAI methods. Grad-CAM Gradient-weighted Class Activation Mapping, LIME Local Interpretable Model-agnostic Explanations, SHAP Shapley Additive Explanations

Figure  5 shows the distribution of studies depending on their primary area of focus. Articles predominantly concentrating on specific diseases were organized according to their principal disease domains. Specifically, the studies included in the “Cardiac Arrhythmia” group explored various forms of bradyarrhythmia and tachyarrhythmia, as well as related-treatments such as ablation. The “Cardiomyopathies” group encompassed studies focusing on non-ischemic cardiomyopathies. The “Heart Failure”, “Valvular Heart Disease”, and “Congenital Heart Disease” groups comprised works specifically centered around those respective conditions. Additionally, the “Other Cardiac Conditions” category covered a wide range of topics, including stroke, peripheral artery disease, pregnancy, pulmonary hypertension, and other cardiac conditions. Some articles, rather than focusing on specific disease domains, primarily addressed tasks such as image segmentation, detection of cardiac abnormalities, and imaging or ECG-based phenotyping. These articles were collectively categorized under the label “Others”.

The figure shows that cardiac arrhythmia (41 studies) stands out as the most frequently studied cardiac condition. This is probably due to the fact that cardiac arrhythmia can be effectively studied by means of ECG data, which is readily obtainable, and the most common data modality used in the reviewed studies. Heart failure is the second most examined condition, encompassing 30 works. This is primarily attributed to the feasibility of investigating heart failure using non-imaging EHR data which ranks as the second most prevalent data type used in the reviewed studies.

The distribution of the diseases targeted in cardiac studies

For the remainder of this review, we will group the papers based on the category of evaluation approach applied to their XAI outcomes. In total, we have identified four distinct evaluation approach categories for the cardiac domain: (i) expert-grounded, (ii) proxy-grounded, (iii) literature-grounded, and (iv) none. Papers that relied on cardiologists or clinicians to assess the outcome of XAI were classified as part of the expert-grounded category. Studies using any proxy, statistical method, or other quantitative metrics to evaluate the XAI outcome fell into the proxy-grounded evaluation category. Literature-grounded evaluation included the works where findings from previous publications were used to assess the outcome of XAI. Typically, these works cite previous publications to support their findings. The last group included those works that did not apply any kind of evaluation to the XAI outcome.

Figure  6 shows the distribution of papers employing different evaluation methods to XAI outcomes. The figure highlights that most papers did not apply any evaluation method, followed by those that applied literature-grounded evaluation. In addition, it shows that expert-grounded methods were less frequently employed than other methods. Notably, 8 studies used two different evaluation methods simultaneously, and they are represented in both categories within the figure.

Distribution of the number of papers across four categories of XAI evaluation approaches: (i) literature-grounded, (ii) expert-grounded, (iii) proxy-grounded, (iv) none

In addition, we have also assessed whether the findings derived from the XAI outcome were in line with the results of the evaluation method. For instance, if an XAI model identified a specific region in CMR as the most informative region for distinguishing between control and heart failure, and this aligned with the expert’s opinion or the applied proxy, it was considered as a match between the XAI outcome and the evaluation approach outcome. Similarly, a mismatch would be recorded if the outcome of the XAI and the evaluation approach did not concur. Cases, where only part of the explanation aligned with expected or established knowledge, are labeled as partial matches.

In this context, Fig.  7 illustrates that the results most evaluation approaches aligned with the outcomes of XAI. This alignment is particularly evident in the literature-grounded approach as this is the most used evaluation approach. Remarkably, only one study (Aufiero et al. 2022 ) deviated from this pattern, as its XAI outcomes contradicted prior findings.

Matching the outcome of the evaluation with the outcome of XAI

5 A review of XAI evaluations in cardiology

The following four sections discuss the papers that applied an evaluation method to assess the effectiveness of the used XAI algorithm. Moreover, we provide statistics and tables with information regarding the utilised data types and XAI methods, grouped by the evaluation approach employed.

5.1 Expert-grounded evaluation in cardiac applications

Twenty-three papers relied on expert-grounded evaluations to assess the outcomes of their XAI methods, either alone or in combination with proxies and literature-grounded approaches (Table  2 ). The experts were represented by cardiologists, physicians or clinicians with different years of experience.

Differently from proxy-grounded evaluations, a greater variety of XAI methods could be found in these 23 reviewed papers, including “if-then” rule, SHAP, Grad-CAM and Saliency maps. The outcomes of the XAI were mostly inline with what was expected by the experts. In particular, the outcomes of eleven works were fully inline with what was expected, while the remaining twelve were partially inline.

In Pičulin et al. ( 2022 ), Zhang et al. ( 2021 ), Sangroya et al. ( 2022 ) and Vazquez et al. ( 2021 ), SHAP was used as XAI method alongside with others such as integrated gradients (Zhang et al. 2021 ) and domain concepts (Sangroya et al. 2022 ). “If-then” rule was used as XAI method to explain the classification models applied to detect heart failure (Li et al. 2020 ; Kukar et al. 2011 ; Kwon et al. 2018 ).

Table  2 shows that the majority of the studies applied expert-grounded evaluation did not report the number of experts involved in the evaluation, their medical specialty nor the years of experience of the experts. Four studies (Decoodt et al. 2023 ; Sager et al. 2021 ; Jones et al. 2020 ; Li et al. 2020 ) included one expert in the evaluation without reporting the years of experience apart from one (Decoodt et al. 2023 ). One study (Pičulin et al. 2022 ) involved a decent number of experts and experience. They proposed a model to predict the clinical statues 10-years ahead for those experienced hypertrophic cardiomyopathy. They applied SHAP to explain the model and its outcome was evaluated by 13 medical experts with 16 years (SD 8) of experience. Manual segmentation of two cardiologists with more than 10 years of experiences were used to assess the outcome of a class activation map applied to a deep learning model to estimate left ventricle volume (Pérez-Pelegrí et al. 2021 ). The “If-then” rule was implemented as XAI method in Kukar et al. ( 2011 ) for a model diagnosing patients with coronary artery disease automatically. The proposed method evaluates myocardial scintigraphy imaging and extracts parameters to then be combined in another model for classification matter. To assess the XAI method, four expert physicians assessed the cardiac images and provided the level of coronary artery congestion by attributing values to the different myocardial regions. The model yielded attributes that closely mirrored the assessments offered by the expert physicians and the “If-then” rule.

More details on each study using expert-grounded evaluations are presented in Table S2 .

5.2 Proxy-grounded evaluation in cardiac applications

Twenty-seven papers applied proxy-grounded methods, either alone or in conjunction with other approaches, to evaluate the outcomes of XAI methods. The evaluation results of the majority aligned with the outcome of the XAI methods, either fully or partially. No contradiction between the evaluation outcome and the XAI outcome. One paper (Prifti et al. 2021 ) did not comment or compare the results of the evaluation method and the XAI outcome. Table  3 summarizes the papers that applied proxy-grounded evaluation methods to assess the XAI outcomes. It indicates that ECG and EHR were the predominant data types used, while Grad-CAM and SHAP were the most frequently used XAI methods.

More precisely, out of the 27, six works (Wall et al. 2022 ; Zhang et al. 2021 ; Singh and Sharma 2022 ; Le et al. 2023 ; Karri et al. 2021 ; Vazquez et al. 2021 ) used SHAP as the XAI method across different domains, including stroke, arrhythmia, atrial fibrillation and hospital mortality. The results of the XAI evaluation were either fully or partially inline with the outcome of SHAP. The evaluation metrics included permutation importance, accuracy reduction, sanity check and checking the value of the logistic regression coefficients to assess whether a feature is informative or not.

Selectivity or RemOve And Retrain (ROAR) method was applied to two works (Pham et al. 2023 ; Dakshit et al. 2022 ) to evaluate if the model identified the correct features that drive model outcome. Another two works (Wall et al. 2022 ; Prifti et al. 2021 ) followed the same criterion but instead of removing the top features, they were permuted. Statistical methods and models were used to evaluate the outcome of XAI methods. Permutation importance served as a proxy for evaluating the list of informative predictors produced by SHAP to estimate cardiac age using ECG features (Wall et al. 2022 ). The proxy results confirmed that the identified features by SHAP have a significant impact on the model outcome. Another assessment of SHAP involved using the coefficient values of logistic regression as a proxy (Karri et al. 2021 ). In this study, multiple models were developed to classify patients with postoperative atrial fibrillation. For the best performing model, SHAP was applied to obtain the most important features in the model’s decision. Moreover, the authors compared the list of the features provided by SHAP with the coefficient values produced by logistic regression. They found that there is a partial match between the coefficient value of the features and their index of order in the list provided by SHAP.

Grad-CAM was employed to explain a multilabel classification model distinguishing between healthy control and eight cardiac diseases using ECG (Ganeshkumar et al. 2021 ). To assess whether the model learnt relevant features, they calculated the correlation between the activation map provided by Grad-CAM for each disease and their respective variations in the ECG. The results of the correlation confirmed that the model decision was indeed driven by the right features in ECG. In another study, SHAP, LIME and Grad-CAM were used to explain a model distinguishing between individuals with arrhythmia from control using ECG (Singh and Sharma 2022 ). To evaluate the outcome of the XAI methods, they used rectified linear unit and gaussian filter to smooth the generated feature maps from each XAI method. Subsequently, they segmented the ECG into windows and fed it to each XAI method to generate saliency plots for each class. Finally, heatmaps were generated based on the values of the feature importance. This approach confirmed whether the model effectively searched in the ECG segments during classification. More details on each study that used proxy-grounded approach is presented in table Table S2 .

5.3 Literature-grounded evaluation in cardiac applications

Most of the papers (seventy-nine) included in this review used a literature-grounded approach to evaluate the performance of XAI(Table  4 ). SHAP and Grad-CAM were the most common XAI methods applied in these studies providing literature-grounded evaluations (Fig.  8 ).

The number of the XAI methods used in cardiac applications. Grad-CAM Gradient-weighted Class Activation Mapping, LIME Local Interpretable Model-agnostic Explanations, SHAP Shapley Additive Explanations

Going into more details of some of these studies, Aufiero et al. ( 2022 ) identified new ECG features using a DL model combined with Grad-CAM in congenital long QT syndrome patients. Their approach identified the QRS complex as the most relevant feature that dominated the classifier decision, a novel finding that had never previously been reported in this condition. Another study (Gandin et al. 2023 ) used EHR to devise a deep learning model for predicting the risk of developing heart failure in diabetic patients. To understand the model outcome and the role of the included features, the authors adopted (Gandin et al. 2023 ) partial dependence plot (PDP) (Greenwell et al. 2018 ), which identified as highly relevant features such as diuretics, diabetes duration, arterial hypertension and Charlson comorbidity index. As acknowledged by the authors themselves, these features are well-known and have been previously reported in heart failure patients.

An ML model was developed to distinguish individuals with heart amyloidosis from hypertrophic cardiomyopathy using EHR and echocardiography data (Wu et al. 2023 ). They implemented information gain of XGBoost to identify the most important features in the model. Previous findings support significant predictors to disseminate between the two conditions. More details of each study used literature-grounded approach is represented in Table S2 .

5.4 No evaluation method

Ninety-two papers included XAI in their framework but did not apply any kind of evaluation to assess the XAI performance and corresponding outcomes. Table  5 summarizes the used data and the XAI methods. ECG data were the most common ones, followed by EHR and CMR (Fig.  3 ).

In terms of XAI methods, SHAP (25) and Grad-CAM (20) were the prevalent XAI choices for these studies, similarly to what found in the other papers employing XAI in combination with some kinds of evaluation (Fig.  9 ).

6 Discussion

In this section we detail key observations from our review of XAI research in cardiac study algorithms. We list challenges that XAI developers or users might face and we provide recommendations for the development of XAI, where possible.

6.1 Notes on the cardiac studies

A range of data modalities, model architectures, cardiac conditions, XAI and evaluation approaches were present in the studies included in this review and are summarised below.

Data modalities: Most studies in this review used either ECG or EHR data. ECG data may be acquired rapidly, easily, and cheaply compared to imaging data such are CMR. However, ECGs report the electrical function of the cardiac, while CMR imaging provides structural and functional information. The modality/modalities for the data leading to an optimal model result (as determined by accuracy for example) will vary on a case-by-case basis depending on the modelling objective.

Model architectures: Most of the algorithms that were used were binary classification models. A small number of studies used regression models to estimate a continues variable. We note that regression models can also be used to discriminate between two conditions through (1) comparing against a normal reference range for a specific phenotype (e.g. left ventricular ejection fraction, left ventricular end-systolic volume), or (2) predicting a continues variable (e.g cardiac age, left ventricular mass) for a cohort free of cardiac diseases with validation on a cohort with the cardiac condition under examination.

Cardiac conditions: Arrhythmia and heart failure were the most examined conditions, which may be driven the availability of ECG data. Although coronary (ischemic) heart disease is the most common cardiac disease worldwide (British Heart Foundation 2023 ), it was [the least] investigated compared to other cardiac conditions. This is because we used broader terms such as “cardiac” and “heart” to encompass a wide array of studies within the field, rather than focusing narrowly on specific conditions like coronary artery disease (CAD). This approach might have probably limited the number of papers specifically focused on CAD. Incorporating more specific keywords could have increased the CAD-related publications, but that would also necessitate including a variety of terms for other cardiac conditions, which was beyond the scope of our paper. Most of the studies investigated heart failure used EHR as the input data, as mentioned above. Other cardiac conditions may be less investigated due to reduced incidence and/or reduced availability of the specific data modalities necessary.

XAI model: Most of the XAI reviewed here used the SHAP and Grad-CAM methods followed by LIME. These methods have contributed significantly to the body knowledge of XAI, but they are imperfect and have their own drawbacks including concerning against adversarial attacks and localize multiple occurrences within an image (Slack et al. 2020 ; Chattopadhay et al. 2018 ). The results produced by these three methods are easy to understand and interpret, which may have enhanced the uptake of the methods, as could the ready availability of software code and packages.

Evaluation approach: 43% of the papers did not use any kind of evaluation approach to assess the performance of XAI. In addition, 37% used literature-grounded approach followed by 11% using proxy-grounded approach and 11% using expert-grounded evaluation. The literature-grounded approach was the most used one due to the ease of carrying out reviews using different repositories including IEEE Xplore, Web of Science and PubMed. The expert-grounded approach is the least used because it specialist reviewer time is costly and time-consuming to carry out on all XAI outputs. The proxy-grounded approach is still under development which may explain why only 11% papers evaluated XAI performance using this approach. The majority of studies did not evaluate the model results which may happen when developing a new XAI model or examining a rare condition where literature and expert-grounded approaches might not be available.

Expert-grounded evaluation: The authors of the papers using the expert-grounded approach to assess the XAI outcomes included physicians (Halme et al. 2022 ), clinicians (Jin et al. 2021 ) and internists (Hur et al. 2020 ), categories of professionals likely experienced in the relevant cardiac diseases. Notably, however, only three papers (Pérez-Pelegrí et al. 2021 ; Pičulin et al. 2022 ; Decoodt et al. 2023 ) out of the 23 in total mentioned the number of years of relevant experience when evaluating XAI performance. One study used 13 experts to assess XAI (Pičulin et al. 2022 ) and four used one expert (Decoodt et al. 2023 ; Sager et al. 2021 ; Jones et al. 2020 ; Li et al. 2020 ), while the majority did not specify the number of experts employed.

XAI evaluation outcome: Enormous number of the papers that applied any kind of evaluation approach got a match between the XAI outcome and the outcome of the used evaluation approach, especially with literature-grounded approach as it is the most used one. The reason behind that could be the examined cardiac conditions are very complex (e.g. heart failure) and there is more than one factor affecting the condition significantly and simultaneously. Accordingly, even if the outcomes of two XAI methods vary for the same condition, yet they still carry informative predictors for that condition and match with the previous findings or with expert opinions.

6.2 Model performance vs model explainability

Ideally, model performance and explainability would be defining features of a good model. Here we consider the relationship between these two characteristics.

Inaccurate perception: A common perception is that the models with high performance are less explainable while more explainable models are those with a lower performance. However, there are many approaches to explainability each with different applicability and utility and this perception requires qualification. The defined aim of the explainability is to produce a framework for the end-user to understand how the results are produced using granular features, as opposed to the complex internal workings of the model architecture. The utility of a given explainability output for a specific end-user is subjective. The results of a recent empirical study (Herm et al. 2023 ) showed that the trade-off curve between model performance and model interpretability is not gradual.

Explanation form: Explanations may comprise: lists of informative predictors; highlighted informative regions within an image; uncertainty quantifiers; “what-if” rules; and the probability of an instance belong to a specific class. Some explainability metrics may be more significant than others in a given domain. For instance, uncertainty quantifiers might be more significant than a list of informative predictors in a model using few numbers of predictors. Not all explainability metrics will be suitable for a given model, even if it is of high performance.

Trade-off between model performance and model explainability: In some cases simple but adequate models with more detailed explanations might be preferable to comparably performing but complex models with reduced level of explanation. One factor in the decision of which model to use might be the domain.

Explanation perceived by end-user: Model metrics such as accuracy, F1 or mean absolute error are objective qualifiers of a model. However, as XAI methods are means to explain the model for end-users, such explanations are subjective as it is left to each end-user to assess utility.

Simple tasks: Classification or regression using simple tabular data can be performed using either simple or complex models, with either typically having similar performance and, in some cases the, former outperform the latter (Herm et al. 2023 ). Accordingly, it is recommended that simple models should be applied in such cases when they are more explainable.

6.3 A reasonable implementation of XAI

It is difficult to determine which of the reviewed papers applied a more reliable and understandable XAI method to end users cardiologists. This is because understanding the outcome of XAI is rather subjective which might differ from a cardiologist to another. Moreover, applying a specific kind of XAI method or evaluation approach is subject to the available data and resources to evaluate the outcomes. However, in Zhang et al. ( 2021 ) we believe the authors implemented and evaluated XAI in a robust and reasonable way. First of all, they applied three XAI methods that are Guided saliency, DeepSHAP, and Integrated Gradients. It is recommended that different XAI methods should be implemented to compare and contrast the XAI outcomes from different methods because each method has its own limitations. Secondly, they applied two approaches to evaluate the outcome of XAI that are: expert and proxy-based evaluation. Indeed, it is vital to include the experts in the evaluation of XAI outcome in this stage. On the other hand, including proxy-based approach would assure to evaluate the outcome of XAI objectively. They compared the annotations of the three XAI methods with experts’ annotations using two metrics named Congruence and Annotation Classification. Finally, they performed correlation between the explainability metrics and the model performance including accuracy and specificity to explore whether the used explainability metrics are consistence with the model performance. As XAI still in the development stage and not mature yet, we believe that what the authors did resulted in a more reliable and trusted XAI outcome to the end users.

6.4 Challenges and solutions

The performance of a machine learning model depends on several aspects including sample size, normative features, redundant features, noise, feature collinearity, model architecture, optimisation method, training and validate approaches, and other factors. We list recommendations that might help improving model performance and allow XAI to be evaluated fairly below.

Sample size: Both simple and complex machine learning models may perform better with larger datasets and variety of data which may be difficult to obtain in the healthcare domain. In addition, unbalanced data happens frequently in healthcare data which might negatively impact the model performance or generalizability. In these cases data augmentation and transfer learning might help to increase sample size, balance the data and train the model on sufficient number of samples.

Use different models: XAI methods are model-dependent which means their utility depends on the performance of the model being explained. Model performance will depend on the underlying data distribution. In addition, some models might be more or less affected by sample size and the number of features than others. There is not always a standard way to apply a specific model architecture to specific data. This can be examined through exploring variety of models covering simple and complex models: the architecture that achieves a better performance then can be used with XAI to explain how the model works, respecting the premise that, performance being comparable, simpler models are preferable.

Apply several XAI: XAI methods are not perfect and vary in bias toward specific data, impact of collinearity among predictors, image resolution and lack of causality. Some methods may be more suitable in some domains than others or work with better with specific classes of models. Furthermore, ultimately, it is the end user who determines which XAI method is more meaningful to them.

Evaluation approach and the domain: It is hard to decide which evaluation approach to choose when evaluating the performance of a given XAI method. This choice may be domain dependent, for example proxy-grounded approaches might be preferable when testing a new product or service where misleading explanations might not be expensive (or harmful). In our opinion, including cardiologists (expert-grounded) alone or alongside with other metrics (proxy-grounded) in cardiac studies to explain the model is still of crucial importance: XAI evaluation is immature and under active development (Salih et al. 2023b ).

Blind evaluation: We believe that including experts in the evaluation of XAI models is valuable. However, the evaluation process should itself be well designed and blinded so that the experts (e.g. cardiologists) provide their explanations and expectations before knowing the outcome of XAI to reduce this source of bias. In addition, in some of the reviewed papers, the number of the physicians whose evaluated XAI was low (Pérez-Pelegrí et al. 2021 ) which questions the reproducibility of their evaluation. While it might be difficult to include many experienced cardiologists as expert evaluators, if an expert-grounded approach is considered, there should be adequate number of experts with qualified experience to assess XAI to ensure it is reliable and reproducible.

Collinearity: Many factors including high blood pressure, smoking, alcohol, physical activity, obesity, and diabetes increase the risk of stroke and other cardiac disease. These factors may be related, for example: physical activity and obesity; smoking and alcohol use; and high blood pressure and diabetes. These factors have different clinical interpretations and are often used together in machine learning models when studying cardiac disease. However, XAI methods might be affected by collinearity among predictors and provide unrealistic or biased explanation (Salih et al. 2024 ). Different attempts and solutions have been proposed to deal with the collinearity including (Salih et al. 2022 , 2024 ; Aas et al. 2021 ) which should be considered if feature selection or dimensionality reduction method is not employed.

Use literature as confirmation: Literature-grounded evaluation is a straightforward and immediate way to assess the performance of XAI models. The availability of large biomedical repositories, including the UK Biobank (Petersen et al. 2015 ) which contains both ECG and CMR data in around 100,000 participants, has increased the volume of cardiac studies published. However, comparing published results requires that the impact of dataset, sample size and model differences are considered as these all affecting the performance of XAI. In addition, if by one side the agreement with previous literature enforces the plausibility of the results, by the other it should not be considered as a must, because this would lead to discard new yet unpublished findings.

7 Conclusion

The rapid success in data processing, availability of large biomedical and healthcare datasets and repositories, and variety of XAI models led to an increase the adoption of interpretable models applied to cardiac studies. However, XAI evaluation is not mature yet and still in the development process and might take more time to be adopted in clinical decision-making. In this work we reviewed XAI evaluation approaches applied to cardiac studies. XAI evaluation is an essential step in XAI modelling specially in healthcare sectors. Including a reasonable number of experienced cardiologists to assess the performance of XAI is indispensable even if other approaches of evaluation are adopted. Including experts in the evaluation of XAI provides several key benefits that are: I) making the model more trustful, II) assisting to improve XAI performance and making it more transparent and III) avoiding biased decision derived by the model. Although XAI evaluation is still to be improved and tested on different datasets, machine learning models and XAI methods, their contributions hold high value and push the process toward more mature approaches and metrics.

Data availability

No datasets were generated or analysed during the current study.

Aas K, Jullum M, Løland A (2021) Explaining individual predictions when features are dependent: more accurate approximations to Shapley values. Artif Intell 298:103502

Article   MathSciNet   Google Scholar  

Abdullah TA, Zahid MSBM, Tang TB, Ali W, Nasser M (2022) Explainable deep learning model for cardiac arrhythmia classification. In: 2022 International conference on future trends in smart communities (ICFTSC). IEEE, pp 87–92

Abdullah TA, Zahid MSM, Ali W, Hassan SU (2023) B-LIME: an improvement of lime for interpretable deep learning classification of cardiac arrhythmia from ECG signals. Processes 11(2):595

Article   Google Scholar  

Abraham VM, Booth G, Geiger P, Balazs GC, Goldman A (2022) Machine-learning models predict 30-day mortality, cardiovascular complications, and respiratory complications after aseptic revision total joint arthroplasty. Clin Orthop Relat Res 480(11):2137–2145

Agrawal A, Chauhan A, Shetty MK, Gupta MD, Gupta A et al (2022) ECG-iCOVIDNet: interpretable ai model to identify changes in the ecg signals of post-covid subjects. Comput Biol Med 146:105540

Alabed S, Uthoff J, Zhou S, Garg P, Dwivedi K, Alandejani F, Gosling R, Schobs L, Brook M, Shahin Y et al (2022) Machine learning cardiac-MRI features predict mortality in newly diagnosed pulmonary arterial hypertension. Eur Heart J Digit Health 3(2):265–275

Alkhodari M, Widatalla N, Wahbah M, Al Sakaji R, Funamoto K, Krishnan A, Kimura Y, Khandoker AH (2022) Deep learning identifies cardiac coupling between mother and fetus during gestation. Front Cardiovasc Med 9:926965

Anand A, Kadian T, Shetty MK, Gupta A (2022) Explainable ai decision model for ECG data of cardiac disorders. Biomed Signal Process Control 75:103584

Angelaki E, Marketou ME, Barmparis GD, Patrianakos A, Vardas PE, Parthenakis F, Tsironis GP (2021) Detection of abnormal left ventricular geometry in patients without cardiovascular disease through machine learning: an ECG-based approach. J Clin Hypertens 23(5):935–945

Apama C, Rohini P, Pandiyarasan V (2022) Interpretation of ResNet-50 model for MI related cardiac events using explainable grad-cam approach. In: Current directions in biomedical engineering, vol 8. De Gruyter, Berlin, pp 723–726

Apley DW (2020) Zhu, J: Visualizing the effects of predictor variables in black box supervised learning models. J R Stat Soc Ser B Stat Methodol 82(4):1059–1086

Arrieta AB, Díaz-Rodríguez N, Del Ser J, Bennetot A, Tabik S, Barbado A, García S, Gil-López S, Molina D, Benjamins R et al (2020) Explainable artificial intelligence (XAI): concepts, taxonomies, opportunities and challenges toward responsible AI. Inf Fusion 58:82–115

Attia ZI, Lerman G, Friedman PA (2021) Deep neural networks learn by using human-selected electrocardiogram features and novel features. Eur Heart J Digit Health 2(3):446–455

Aufiero S, Bleijendaal H, Robyns T, Vandenberk B, Krijger C, Bezzina C, Zwinderman AH, Wilde AA, Pinto YM (2022) A deep learning approach identifies new ECG features in congenital long QT syndrome. BMC Med 20(1):1–12

Bach S, Binder A, Montavon G, Klauschen F, Müller K-R, Samek W (2015) On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE 10(7):0130140

Bacoyannis T, Ly B, Cedilnik N, Cochet H, Sermesant M (2021) Deep learning formulation of electrocardiographic imaging integrating image and signal information with data-driven regularization. EP Europace 23(Supplement-1):55–62

Bahani K, Moujabbir M, Ramdani M (2021) An accurate fuzzy rule-based classification systems for heart disease diagnosis. Sci Afr 14:01019

Google Scholar  

Bahrami N, Retson T, Blansit K, Wang K, Hsiao A (2019) Automated selection of myocardial inversion time with a convolutional neural network: spatial temporal ensemble myocardium inversion network (STEMI-Net). Magn Reson Med 81(5):3283–3291

Beetz M, Banerjee A, Grau V (2022) Multi-domain variational autoencoders for combined modeling of MRI-based biventricular anatomy and ECG-based cardiac electrophysiology. Front Physiol 13:886723

Beetz M, Corral Acero J, Banerjee A, Eitel I, Zacur E, Lange T, Stiermaier T, Evertz R, Backhaus SJ, Thiele H et al (2022) Interpretable cardiac anatomy modeling using variational mesh autoencoders. Front Cardiovasc Med 9:983868

Bhardwaj A, Singh S, Joshi D (2023) Explainable deep convolutional neural network for valvular heart diseases classification using pcg signals. IEEE Trans Instrum Meas. https://doi.org/10.1109/TIM.2023.3274174

Biffi C, Oktay O, Tarroni G, Bai W, De Marvao A, Doumou G, Rajchl M, Bedair R, Prasad S, Cook S et al (2018) Learning interpretable anatomical features through deep generative models: application to cardiac remodeling. In: Medical image computing and computer assisted intervention—MICCAI 2018: 21st international conference, Granada, Spain, 16–20 September 2018, Proceedings, Part II 11. Springer, Cham, pp 464–471

Bodini M, Rivolta MW, Sassi R (2021) Opening the black box: interpretability of machine learning algorithms in electrocardiography. Philos Trans R Soc A 379(2212):20200253

Brisimi TS, Xu T, Wang T, Dai W, Adams WG, Paschalidis IC (2018) Predicting chronic disease hospitalizations from electronic health records: an interpretable classification approach. Proc IEEE 106(4):690–707

British Heart Foundation B (2023) Global Heart & Circulatory Diseases Factsheet. https://www.bhf.org.uk/

Bruijn H, Warnier M, Janssen M (2022) The perils and pitfalls of explainable AI: strategies for explaining algorithmic decision-making. Gov Inf Q 39(2):101666

Cao Y, Liu W, Zhang S, Xu L, Zhu B, Cui H, Geng N, Han H, Greenwald SE (2022) Detection and localization of myocardial infarction based on multi-scale ResNet and attention mechanism. Front Physiol 13:24

Cetin I, Stephens M, Camara O, Ballester MAG (2023) Attri-VAE: attribute-based interpretable representations of medical images with variational autoencoders. Comput Med Imaging Graph 104:102158

Chaddad A, Peng J, Xu J, Bouridane A (2023) Survey of explainable AI techniques in healthcare. Sensors 23(2):634

Chalasani P, Chen J, Chowdhury AR, Wu X, Jha S (2020) Concise explanations of neural networks using adversarial training. In: International conference on machine learning. PMLR, pp 1383–1391

Chattopadhay A, Sarkar A, Howlader P, Balasubramanian VN (2018) Grad-CAM++: generalized gradient-based visual explanations for deep convolutional networks. In: 2018 IEEE Winter conference on applications of computer vision (WACV). IEEE, pp 839–847

Chen P, Dong W, Wang J, Lu X, Kaymak U, Huang Z (2020) Interpretable clinical prediction via attention-based neural network. BMC Med Inform Decis Mak 20(3):1–9

Chen H, Gomez C, Huang C-M, Unberath M (2022) Explainable medical imaging AI needs human-centered design: guidelines and evidence from a systematic review. npj Digit Med 5(1):156

Chen S, Hu W, Yang Y, Cai J, Luo Y, Gong L, Li Y, Si A, Zhang Y, Liu S et al (2023a) Predicting six-month re-admission risk in heart failure patients using multiple machine learning methods: a study based on the Chinese heart failure population database. J Clin Med 12(3):870

Chen L, Fu G, Jiang C (2023b) Deep learning-derived 12-lead electrocardiogram-based genotype prediction for hypertrophic cardiomyopathy: a pilot study. Ann Med 55(1):2235564

Chen C, Zhao HY, Zheng SH, Ramachandra RA, He X, Zhang YH, Sudarshan VK (2023b) Interpretable hybrid model for an automated patient-wise categorization of hypertensive and normotensive electrocardiogram signals. Comput Methods Programs Biomed Update 3:100097

Cho J, Lee B, Kwon J-M, Lee Y, Park H, Oh B-H, Jeon K-H, Park J, Kim K-H (2021) Artificial intelligence algorithm for screening heart failure with reduced ejection fraction using electrocardiography. ASAIO J 67(3):314–321

Choi BS, Yoo SK, Moon J, Chung SY, Oh J, Baek S, Kim Y, Chang JS, Kim H, Kim JS (2023) Acute coronary event (ACE) prediction following breast radiotherapy by features extracted from 3D, CT, dose, and cardiac structures. Med Phys 50(10):6409–6420

Chou Y-L, Moreira C, Bruza P, Ouyang C, Jorge J (2022) Counterfactuals and causability in explainable artificial intelligence: theory, algorithms, and applications. Inf Fusion 81:59–83

Clough JR, Oksuz I, Puyol-Antón E, Ruijsink B, King AP, Schnabel JA (2019) Global and local interpretability for cardiac MRI classification. In: International conference on medical image computing and computer-assisted intervention. Springer, Cham, pp 656–664

Commandeur F, Slomka PJ, Goeller M, Chen X, Cadet S, Razipour A, McElhinney P, Gransar H, Cantu S, Miller RJ et al (2020) Machine learning to predict the long-term risk of myocardial infarction and cardiac death based on clinical risk, coronary calcium, and epicardial adipose tissue: a prospective study. Cardiovasc Res 116(14):2216–2225

Cui X, Lee JM, Hsieh J (2019) An integrative 3C evaluation framework for explainable artificial intelligence. In: Twenty-fifth Americas conference on information systems, Cancun

Dakshit S, Maweu BM, Dakshit S, Prabhakaran B (2022) Core-set selection using metrics-based explanations (CSUME) for multiclass ECG. In: 2022 IEEE 10th international conference on healthcare informatics (ICHI). IEEE, pp 217–225

Decoodt P, Liang TJ, Bopardikar S, Santhanam H, Eyembe A, Garcia-Zapirain B, Sierra-Sosa D (2023) Hybrid classical-quantum transfer learning for cardiomegaly detection in chest X-rays. J Imaging 9(7):128

DeYoung J, Jain S, Rajani NF, Lehman E, Xiong C, Socher R, Wallace BC (2019) Eraser: a benchmark to evaluate rationalized NLP models. arXiv preprint. arXiv:1911.03429

Di Martino F, Delmastro F (2022) Explainable ai for clinical and remote health applications: a survey on tabular and time series data. Artif Intell Rev 56:5261–5315

Diaz Ochoa JG, Maier L, Csiszar O (2023) Bayesian logical neural networks for human-centered applications in medicine. Front Bioinform 3:1082941

Ding Z, Chen G, Zhang L, Baheti B, Wu R, Liao W, Liu X, Hou J, Mao Z, Guo Y et al (2023) Residential greenness and cardiac conduction abnormalities: epidemiological evidence and an explainable machine learning modeling study. Chemosphere 339:139671

Doborjeh M, Doborjeh Z, Merkin A, Krishnamurthi R, Enayatollahi R, Feigin V, Kasabov N (2022) Personalised spiking neural network models of clinical and environmental factors to predict stroke. Cogn Comput 14:2187–2202

Dong T, Sinha S, Zhai B, Fudulu DP, Chan J, Narayan P, Judge A, Caputo M, Dimagli A, Benedetto U et al (2023) Cardiac surgery risk prediction using ensemble machine learning to incorporate legacy risk scores: a benchmarking study. Digit Health 9:20552076231187604

Doshi-Velez F, Kim B (2017) Towards a rigorous science of interpretable machine learning. arXiv preprint. arXiv:1702.08608

Duffy G, Jain I, He B, Ouyang D (2021) Interpretable deep learning prediction of 3D assessment of cardiac function. In: Pacific symposium on biocomputing 2022. World Scientific, Singapore, pp 231–241

Duval A, Nogueira D, Dissler N, Maskani Filali M, Delestro Matos F, Chansel-Debordeaux L, Ferrer-Buitrago M, Ferrer E, Antequera V, Ruiz-Jorro M et al (2023) A hybrid artificial intelligence model leverages multi-centric clinical data to improve fetal heart rate pregnancy prediction across time-lapse systems. Hum Reprod 38(4):596–608

Fan Y, Dong J, Wu Y, Shen M, Zhu S, He X, Jiang S, Shao J, Song C (2022) Development of machine learning models for mortality risk prediction after cardiac surgery. Cardiovasc Diagn Ther Diagn Ther 12(1):12

Fang R, Lu C-C, Chuang C-T, Chang W-H (2022) A visually interpretable detection method combines 3-D ECG with a multi-VGG neural network for myocardial infarction identification. Comput Methods Programs Biomed 219:106762

Fang H, Shi C, Chen C-H (2020) BioExpDNN: bioinformatic explainable deep neural network. In: 2020 IEEE International conference on bioinformatics and biomedicine (BIBM). IEEE, pp 2461–2467

Forte JC, Yeshmagambetova G, Grinten ML, Scheeren TW, Nijsten MW, Mariani MA, Henning RH, Epema AH (2022) Comparison of machine learning models including preoperative, intraoperative, and postoperative data and mortality after cardiac surgery. JAMA Netw Open 5(10):2237970–2237970

Gandin I, Saccani S, Coser A, Scagnetto A, Cappelletto C, Candido R, Barbati G, Di Lenarda A (2023) Deep-learning-based prognostic modeling for incident heart failure in patients with diabetes using electronic health records: a retrospective cohort study. PLoS ONE 18(2):0281878

Ganeshkumar M, Ravi V, Sowmya V, Gopalakrishnan E, Soman K (2021) Explainable deep learning-based approach for multilabel classification of electrocardiogram. IEEE Trans Eng Manag. https://doi.org/10.1109/TEM.2021.3104751

Gao S, Zhou H, Gao Y, Zhuang X (2023) BayeSeg: Bayesian modeling for medical image segmentation with interpretable generalizability. arXiv preprint. arXiv:2303.01710

Gee AH, Garcia-Olano D, Ghosh J, Paydarfar D (2019) Explaining deep classification of time-series data with learned prototypes. In: CEUR workshop proceedings, vol 2429. NIH Public Access, p 15

Ghorbani A, Ouyang D, Abid A, He B, Chen JH, Harrington RA, Liang DH, Ashley EA, Zou JY (2020) Deep learning interpretation of echocardiograms. NPJ Digit Med 3(1):10

Gilpin LH, Bau D, Yuan BZ, Bajwa A, Specter M, Kagal L (2018) Explaining explanations: an overview of interpretability of machine learning. In: 2018 IEEE 5th International conference on data science and advanced analytics (DSAA). IEEE, pp 80–89

Gkontra P, Quaglio G, Garmendia AT, Lekadir K (2023) Challenges of machine learning and AI (what is next?), responsible and ethical AI. In: Clinical applications of artificial intelligence in real-world data. Springer, Cham, p 263

González S, Hsieh W-T, Burba D, Chen TP-C, Wang C-L, Wu VC-C, Chang S-H (2022) Interpretable estimation of the risk of heart failure hospitalization from a 30-second electrocardiogram. In: 2022 E-Health and bioengineering conference (EHB). IEEE, pp 1–4

Goswami PP, Deshpande T, Rotake DR, Singh SG (2023) Near perfect classification of cardiac biomarker troponin-i in human serum assisted by SnS2-CNT composite, explainable ML, and operating-voltage-selection-algorithm. Biosens Bioelectron 220:114915

Greenwell BM, Boehmke BC, McCarthy AJ (2018) A simple and effective model-based variable importance measure. arXiv preprint. arXiv:1805.04755

Guleria P, Naga Srinivasu P, Ahmed S, Almusallam N, Alarfaj FK (2022) XAI framework for cardiovascular disease prediction using classification techniques. Electronics 11(24):4086

Guo F, Ng M, Goubran M, Petersen SE, Piechnik SK, Neubauer S, Wright G (2020) Improving cardiac MRI convolutional neural network segmentation on small training datasets and dataset shift: a continuous kernel cut approach. Med Image Anal 61:101636

Halme H-L, Ihalainen T, Suomalainen O, Loimaala A, Mätzke S, Uusitalo V, Sipilä O, Hippeläinen E (2022) Convolutional neural networks for detection of transthyretin amyloidosis in 2D scintigraphy images. EJNMMI Res 12(1):1–11

Hamatani Y, Nishi H, Iguchi M, Esato M, Tsuji H, Wada H, Hasegawa K, Ogawa H, Abe M, Fukuda S et al (2022) Machine learning risk prediction for incident heart failure in patients with atrial fibrillation. JACC Asia 2(6):706–716

Haque A, Stubbs D, Hubig NC, Spinale FG, Richardson WJ (2022) Interpretable machine learning predicts cardiac resynchronization therapy responses from personalized biochemical and biomechanical features. BMC Med Inform Decis Mak 22(1):282

Hase P, Bansal M (2020) Evaluating explainable AI: which algorithmic explanations help users predict model behavior? arXiv preprint. arXiv:2005.01831

Herm L-V, Heinrich K, Wanner J, Janiesch C (2023) Stop ordering machine learning algorithms by their explainability! A user-centered investigation of performance and explainability. Int J Inf Manage 69:102538

Ho ES, Ding Z (2022) Electrocardiogram analysis of post-stroke elderly people using one-dimensional convolutional neural network model with gradient-weighted class activation mapping. Artif Intell Med 130:102342

Hong S, Xiao C, Ma T, Li H, Sun J (2019) MINA: multilevel knowledge-guided attention for modeling electrocardiography signals. arXiv preprint. arXiv:1905.11333

Hong L, Xu H, Ge C, Tao H, Shen X, Song X, Guan D, Zhang C (2022) Prediction of low cardiac output syndrome in patients following cardiac surgery using machine learning. Front Med 9:973147

Hooker S, Erhan D, Kindermans P-J, Kim B (2019) A benchmark for interpretability methods in deep neural networks. In: Advances in neural information processing systems 32 (NeurIPS 2019)

Hu L-H, Betancur J, Sharir T, Einstein AJ, Bokhari S, Fish MB, Ruddy TD, Kaufmann PA, Sinusas AJ, Miller EJ et al (2020) Machine learning predicts per-vessel early coronary revascularization after fast myocardial perfusion spect: results from multicentre refine spect registry. Eur Heart J Cardiovasc Imaging 21(5):549–559

Hu Y, Feng T, Wang M, Liu C, Tang H (2023) Detection of paroxysmal atrial fibrillation from dynamic ECG recordings based on a deep learning model. J Pers Med 13(5):820

Huang Z, Gan Y, Lye T, Liu Y, Zhang H, Laine A, Angelini E, Hendon C (2023) Cardiac adipose tissue segmentation via image-level annotations. IEEE J Biomed Health Inf. https://doi.org/10.1109/JBHI.2023.3263838

Hur C, Wi J, Kim Y (2020) Facilitating the development of deep learning models with visual analytics for electronic health records. Int J Environ Res Public Health 17(22):8303

Huynh J, Masoudi S, Noorbakhsh A, Mahmoodi A, Kligerman S, Yen A, Jacobs K, Hahn L, Hasenstab K, Pazzani M et al (2022) Deep learning radiographic assessment of pulmonary edema: optimizing clinical performance, training with serum biomarkers. IEEE Access 10:48577–48588

Janik A, Dodd J, Ifrim G, Sankaran K, Curran K (2021) Interpretability of a deep learning model in the application of cardiac MRI segmentation with an ACDC challenge dataset. In: Medical imaging 2021: image processing, vol 11596. SPIE, pp 861–872

Jekova I, Christov I, Krasteva V (2022) Atrioventricular synchronization for detection of atrial fibrillation and flutter in one to twelve ECG leads using a dense neural network classifier. Sensors 22(16):6071

Jiang M, Qiu Y, Zhang W, Zhang J, Wang Z, Ke W, Wu Y, Wang Z (2022) Visualization deep learning model for automatic arrhythmias classification. Physiol Meas 43(8):085003

Jiao Y, Yuan J, Sodimu OM, Qiang Y, Ding Y (2022) Deep neural network-aided histopathological analysis of myocardial injury. Front Cardiovasc Med 8:724183

Jin Y, Liu J, Liu Y, Qin C, Li Z, Xiao D, Zhao L, Liu C (2021) A novel interpretable method based on dual-level attentional deep neural network for actual multilabel arrhythmia detection. IEEE Trans Instrum Meas 71:1–11

Jin W, Li X, Fatehi M, Hamarneh G (2023a) Guidelines and evaluation of clinical explainable AI in medical image analysis. Med Image Anal 84:102684

Jin W, Li X, Hamarneh G (2023b) Rethinking ai explainability and plausibility. arXiv preprint. arXiv:2303.17707

Johnson E, Mohan S, Gaudio A, Smailagic A, Faloutsos C, Campilho A (2022) Heartspot: privatized and explainable data compression for cardiomegaly detection. In: 2022 IEEE-EMBS international conference on biomedical and health informatics (BHI). IEEE, pp 01–04

Jones Y, Deligianni F, Dalton J (2020) Improving ECG classification interpretability using saliency maps. In: 2020 IEEE 20th international conference on bioinformatics and bioengineering (BIBE). IEEE, pp 675–682

Kakogeorgiou I, Karantzalos K (2021) Evaluating explainable artificial intelligence methods for multi-label deep learning classification tasks in remote sensing. Int J Appl Earth Obs Geoinf 103:102520

Kan C, Ye Z, Zhou H, Cheruku SR (2023) DG-ECG: multi-stream deep graph learning for the recognition of disease-altered patterns in electrocardiogram. Biomed Signal Process Control 80:104388

Karatzia L, Aung N, Aksentijevic D (2022) Artificial intelligence in cardiology: hope for the future and power for the present. Front Cardiovasc Med 9:945726

Karoui A, Bendahmane M, Zemzemi N (2021) Cardiac activation maps reconstruction: a comparative study between data-driven and physics-based methods. Front Physiol 12:686136

Karri R, Kawai A, Thong YJ, Ramson DM, Perry LA, Segal R, Smith JA, Penny-Dimri JC (2021) Machine learning outperforms existing clinical scoring tools in the prediction of postoperative atrial fibrillation during intensive care unit admission after cardiac surgery. Heart Lung Circ 30(12):1929–1937

Kawakami M, Karashima S, Morita K, Tada H, Okada H, Aono D, Kometani M, Nomura A, Demura M, Furukawa K et al (2022) Explainable machine learning for atrial fibrillation in the general population using a generalized additive model – a cross-sectional study. Circ Rep 4(2):73–82

Khurshid S, Friedman S, Reeder C, Di Achille P, Diamant N, Singh P, Harrington LX, Wang X, Al-Alusi MA, Sarma G et al (2022) ECG-based deep learning and clinical risk factors to predict atrial fibrillation. Circulation 145(2):122–133

Killian MO, Tian S, Xing A, Hughes D, Gupta D, Wang X, He Z ( (2023) Prediction of outcomes after heart transplantation in pediatric patients using national registry data: evaluation of machine learning approaches. JMIR Cardio 7:45352

Kofler A, Pali M-C, Schaeffter T, Kolbitsch C (2023) Deep supervised dictionary learning by algorithm unrolling-application to fast 2D dynamic MR image reconstruction. Med Phys 50(5):2939–2960

Kogan E, Didden E-M, Lee E, Nnewihe A, Stamatiadis D, Mataraso S, Quinn D, Rosenberg D, Chehoud C, Bridges C (2023) A machine learning approach to identifying patients with pulmonary hypertension using real-world electronic health records. Int J Cardiol 374:95–99

Kor C-T, Li Y-R, Lin P-R, Lin S-H, Wang B-Y, Lin C-H (2022) Explainable machine learning model for predicting first-time acute exacerbation in patients with chronic obstructive pulmonary disease. J Pers Med 12(2):228

Kucukseymen S, Arafati A, Al-Otaibi T, El-Rewaidy H, Fahmy AS, Ngo LH, Nezafat R (2022) Noncontrast cardiac magnetic resonance imaging predictors of heart failure hospitalization in heart failure with preserved ejection fraction. J Magn Reson Imaging 55(6):1812–1825

Kukar M, Kononenko I, Grošelj C (2011) Modern parameterization and explanation techniques in diagnostic decision support system: a case study in diagnostics of coronary artery disease. Artif Intell Med 52(2):77–90

Kumarakulasinghe NB, Blomberg T, Liu J, Leao AS, Papapetrou P (2020) Evaluating local interpretable model-agnostic explanations on clinical machine learning classification models. In: 2020 IEEE 33rd International symposium on computer-based medical systems (CBMS). IEEE, pp 7–12

Kuppa A, Le-Khac N-A (2020) Black box attacks on explainable artificial intelligence (XAI) methods in cyber security. In: 2020 international joint conference on neural networks (IJCNN). IEEE, pp 1–8

Kwon BC, Choi M-J, Kim JT, Choi E, Kim YB, Kwon S, Sun J, Choo J (2018) RetainVis: visual analytics with interpretable and interactive recurrent neural networks on electronic medical records. IEEE Trans Vis Comput Graph 25(1):299–309

Kwon J, Kim K-H, Medina-Inojosa J, Jeon K-H, Park J, Oh B-H (2020) Artificial intelligence for early prediction of pulmonary hypertension using electrocardiography. J Heart Lung Transplant 39(8):805–814

Kwon J, Kim K-H, Eisen HJ, Cho Y, Jeon K-H, Lee SY, Park J, Oh B-H (2021) Artificial intelligence assessment for early detection of heart failure with preserved ejection fraction based on electrocardiographic features. Eur Heart J Digit Health 2(1):106–116

Lagopoulos A, Hristu-Varsakelis D (2022) Measuring the left ventricular ejection fraction using geometric features. In: 2022 IEEE 35th International symposium on computer-based medical systems (CBMS). IEEE, pp 1–6

Le KH, Pham HH, Nguyen TB, Nguyen TA, Thanh TN, Do CD (2023) LightX3ECG: a lightweight and explainable deep learning system for 3-lead electrocardiogram classification. Biomed Signal Process Control 85:104963

Lee H, Shin M (2021) Learning explainable time-morphology patterns for automatic arrhythmia classification from short single-lead ECGS. Sensors 21(13):4331

Lee SH, Geng H, Arnold J, Caruana R, Fan Y, Rosen MA, Apte AP, Deasy JO, Bradley JD, Xiao Y (2023) Interpretable machine learning for choosing radiation dose-volume constraints on cardio-pulmonary substructures associated with overall survival in NRG oncology RTOG 0617. Int J Radiat Oncol Biol Phys 117(5):1270–1286

Leur RR, Taha K, Bos MN, Heijden JF, Gupta D, Cramer MJ, Hassink RJ, Harst P, Doevendans PA, Asselbergs FW et al (2021) Discovering and visualizing disease-specific electrocardiogram features using deep learning: proof-of-concept in phospholamban gene mutation carriers. Circ Arrhythm Electrophysiol 14(2):009056

Leventi-Peetz A-M, Weber K (2022) Rashomon effect and consistency in explainable artificial intelligence (XAI). In: Proceedings of the future technologies conference (FTC) 2022, vol 1. Springer, Cham, pp 796–808

Li R, Yin C, Yang S, Qian B, Zhang P (2020) Marrying medical domain knowledge with deep learning on electronic health records: a deep visual analytics approach. J Med Internet Res 22(9):20645

Li J, Liu S, Hu Y, Zhu L, Mao Y, Liu J (2022) Predicting mortality in intensive care unit patients with heart failure using an interpretable machine learning model: retrospective cohort study. J Med Internet Res 24(8):38082

Liang Y, Guo C (2023) Heart failure disease prediction and stratification with temporal electronic health records data using patient representation. Biocybern Biomed Eng 43(1):124–141

Lin Y-C, Lee Y-C, Tsai W-C, Beh W-K, Wu A-YA (2020) Explainable deep neural network for identifying cardiac abnormalities using class activation map. In: 2020 Computing in cardiology. IEEE, pp 1–4

Linardatos P, Papastefanopoulos V, Kotsiantis S (2020) Explainable AI: a review of machine learning interpretability methods. Entropy 23(1):18

Lindow T, Palencia-Lamela I, Schlegel TT, Ugander M (2022) Heart age estimated using explainable advanced electrocardiography. Sci Rep 12(1):9840

Lisboa PJ, Jayabalan M, Ortega-Martorell S, Olier I Medved D Nilsson J (2022) Enhanced survival prediction using explainable artificial intelligence in heart transplantation. Sci Rep 12(1):19525

Liu Z, Cao Q, Jin Q, Lin J, Lv G, Chen K (2023) Accurate detection of arrhythmias on raw electrocardiogram images: an aggregation attention multi-label model for diagnostic assistance. Med Eng Phys 114:103964

Liu J, Yuan G, Yang C, Song H, Luo L (2023) An interpretable CNN for the segmentation of the left ventricle in cardiac MRI by real-time visualization. CMES Comput Model Eng Sci 135(2):1571–1587

Lo Iacono F, Maragna R, Pontone G, Corino VD (2023) A robust radiomic-based machine learning approach to detect cardiac amyloidosis using cardiac computed tomography. Front Radiol 3:1193046

Loh HW, Ooi CP, Seoni S, Barua PD, Molinari F, Acharya UR (2022) Application of explainable artificial intelligence for healthcare: a systematic review of the last decade (2011–2022). Comput Methods Programs Biomed 226:107161

Loncaric F, Castellote P-MM, Sanchez-Martinez S, Fabijanovic D, Nunno L, Mimbrero M, Sanchis L, Doltra A, Montserrat S, Cikes M et al (2021) Automated pattern recognition in whole-cardiac cycle echocardiographic data: capturing functional phenotypes with machine learning. J Am Soc Echocardiogr 34(11):1170–1183

Lopes RR, Bleijendaal H, Ramos LA, Verstraelen TE, Amin AS, Wilde AA, Pinto YM, Mol BA, Marquering HA (2021) Improving electrocardiogram-based detection of rare genetic heart disease using transfer learning: an application to phospholamban p.Arg14del mutation carriers. Comput Biol Med 131:104262

Lopes P, Silva E, Braga C, Oliveira T, Rosado L (2022) XAI systems evaluation: a review of human and computer-centred methods. Appl Sci 12(19):9423

Lu S, Chen R, Wei W, Belovsky M, Lu X (2021) Understanding heart failure patients ehr clinical features via shap interpretation of tree-based machine learning model predictions. In: AMIA annual symposium proceedings, vol 2021. American Medical Informatics Association, Bethesda, p 813

Lundberg SM, Lee S-I (2017) A unified approach to interpreting model predictions. In: 31st Conference on neural information processing systems (NIPS 2017), Long Beach

Ly B, Finsterbach S, Nuñez-Garcia M, Jaïs P, Garreau D, Cochet H, Sermesant M (2022) Interpretable prediction of post-infarct ventricular arrhythmia using graph convolutional network. In: International workshop on statistical atlases and computational models of the heart. Springer, Cham, pp 157–167

Ma F, Wang Y, Xiao H, Yuan Y, Chitta R, Zhou J, Gao J (2019) Incorporating medical code descriptions for diagnosis prediction in healthcare. BMC Med Inform Decis Mak 19(6):1–13

Mahajan A, Esper S, Oo TH, McKibben J, Garver M, Artman J, Klahre C, Ryan J, Sadhasivam S, Holder-Murray J et al (2023) Development and validation of a machine learning model to identify patients before surgery at high risk for postoperative adverse events. JAMA Netw Open 6(7):2322285–2322285

Maiorana E, Romano C, Schena E, Massaroni C (2023) BIOWISH: biometric recognition using wearable inertial sensors detecting heart activity. IEEE Trans Depend Secure Comput. https://doi.org/10.1109/TDSC.2023.3268360

Makimoto H, Shiraga T, Kohlmann B, Magnisali CE, Gerguri S, Motoyama N, Clasen L, Bejinariu A, Klein K, Makimoto A et al (2022) Efficient screening for severe aortic valve stenosis using understandable artificial intelligence: a prospective diagnostic accuracy study. Eur Heart J Digit Health 3(2):141–152

Markov N, Ushenin K, Bozhko Y (2023) A convolutional recurrent model for the identification of patients with atrial fibrillation based on heart rate variability data during sinus rhythm. In: 2023 IEEE Ural-Siberian conference on biomedical engineering, radioelectronics and information technology (USBEREIT). IEEE, pp 072–075

Melzi P, Tolosana R, Cecconi A, Sanz-Garcia A, Ortega GJ, Jimenez-Borreguero LJ, Vera-Rodriguez R (2021) Analyzing artificial intelligence systems for the prediction of atrial fibrillation from sinus-rhythm ECGs including demographics and feature visualization. Sci Rep 11(1):22786

Meng J, Xing R (2022) Inside the “black box’’: embedding clinical knowledge in data-driven machine learning for heart disease diagnosis. Cardiovasc Digit Health J 3(6):276–288

Michel P, Ngo N, Pons J-F, Delliaux S, Giorgi R (2021) A filter approach for feature selection in classification: application to automatic atrial fibrillation detection in electrocardiogram recordings. BMC Med Inform Decis Mak 21(4):1–17

Miran SM, Nelson SJ, Zeng-Treitler Q (2021) A model-agnostic approach for understanding heart failure risk factors. BMC Res Notes 14(1):184

Miranda E, Adiarto S, Bhatti FM, Zakiyyah AY, Aryuni M, Bernando C (2023) Understanding arteriosclerotic heart disease patients using electronic health records: a machine learning and shapley additive explanations approach. Healthc Inf Res 29(3):228–238

Mohseni S, Zarei N, Ragan ED (2021) A multidisciplinary survey and framework for design and evaluation of explainable AI systems. ACM Trans Interact Intell Syst (TiiS) 11(3–4):1–45

Mokhtari M, Tsang T, Abolmaesumi P, Liao R (2022) EchoGNN: explainable ejection fraction estimation with graph neural networks. In: International conference on medical image computing and computer-assisted intervention. Springer, Berlin, pp 360–369

Molnar C, König G, Herbinger J, Freiesleben T, Dandl S, Scholbeck CA, Casalicchio G, Grosse-Wentrup M, Bischl B (2022) General pitfalls of model-agnostic interpretation methods for machine learning models. In: xxAI-beyond explainable AI: international workshop, held in conjunction with ICML 2020, 18 July 2020, Vienna, Austria, revised and extended papers. Springer, Cham, pp 39–68

Montavon G, Lapuschkin S, Binder A, Samek W, Müller K-R (2017) Explaining nonlinear classification decisions with deep Taylor decomposition. Pattern Recogn 65:211–222

Montavon G, Samek W, Müller K-R (2018) Methods for interpreting and understanding deep neural networks. Digit Signal Process 73:1–15

Moreno-Sanchez PA (2020) Development of an explainable prediction model of heart failure survival by using ensemble trees. In: 2020 IEEE international conference on big data (Big Data). IEEE, pp 4902–4910

Mousavi S, Afghah F, Acharya UR (2020) HAN-ECG: an interpretable atrial fibrillation detection model using hierarchical attention networks. Comput Biol Med 127:104057

Nankani D, Baruah RD (2021) Ventricular arrhythmia classification and interpretation using residual neural network with guided backpropagation. In: TENCON 2021-2021 IEEE Region 10 conference (TENCON). IEEE, pp 574–579

Nankani D, Baruah RD (2022) Atrial fibrillation classification and prediction explanation using transformer neural network. In: 2022 International joint conference on neural networks (IJCNN). IEEE, pp 01–08

Nguyen MB, Dragulescu A, Chaturvedi R, Fan C-PS, Villemain O, Friedberg MK, Mertens LL (2022) Understanding complex interactions in pediatric diastolic function assessment. J Am Soc Echocardiogr 35(8):868–877

Niu J, Lu Y, Xu R, Fang F, Hong S, Huang L, Xue Y, Fei J, Zhang X, Zhou B et al (2023) The prognostic value of intraoperative HRV during anesthesia in patients presenting for non-cardiac surgery. BMC Anesthesiol 23(1):1–10

Novelli C, Taddeo M, Floridi L (2023) Accountability in artificial intelligence: what it is and how it works. AI Soc. https://doi.org/10.1007/s00146-023-01635-y

Nurmaini S, Partan RU, Bernolian N, Sapitri AI, Tutuko B, Rachmatullah MN, Darmawahyuni A, Firdaus F, Mose JC (2022) Deep learning for improving the effectiveness of routine prenatal screening for major congenital heart diseases. J Clin Med 11(21):6454

Ogbomo-Harmitt S, Muffoletto M, Zeidan A, Qureshi A, King AP, Aslanidi O (2023) Exploring interpretability in deep learning prediction of successful ablation therapy for atrial fibrillation. Front Physiol 14:1054401

Oliveira M, Seringa J, Pinto FJ, Henriques R, Magalhães T (2023) Machine learning prediction of mortality in acute myocardial infarction. BMC Med Inform Decis Mak 23(1):1–16

Ovalle-Magallanes E, Avina-Cervantes JG, Cruz-Aceves I, Ruiz-Pinales J (2020) Transfer learning for stenosis detection in X-ray coronary angiography. Mathematics 8(9):1510

Painchaud N, Duchateau N, Bernard O, Jodoin P-M (2022) Echocardiography segmentation with enforced temporal consistency. IEEE Trans Med Imaging 41(10):2867–2878

Panicacci S, Donati M, Fanucci L, Bellini I, Profili F, Francesconi P (2019) Exploring machine learning algorithms to identify heart failure patients: the Tuscany region case study. In: 2019 IEEE 32nd International symposium on computer-based medical systems (CBMS). IEEE, pp 417–422

Patel J, Ladani A, Sambamoorthi N, LeMasters T, Dwibedi N, Sambamoorthi U (2021) Predictors of co-occurring cardiovascular and gastrointestinal disorders among elderly with osteoarthritis. Osteoarthritis Cartilage Open 3(2):100148

Peng S, Huang J, Liu X, Deng J, Sun C, Tang J, Chen H, Cao W, Wang W, Duan X et al (2022) Interpretable machine learning for 28-day all-cause in-hospital mortality prediction in critically ill patients with heart failure combined with hypertension: a retrospective cohort study based on medical information Mart for intensive care database-IV and EICU databases. Front Cardiovasc Med 9:994359

Pérez-Pelegrí M, Monmeneu JV, López-Lereu MP, Pérez-Pelegrí L, Maceira AM, Bodí V, Moratal D (2021) Automatic left ventricle volume calculation with explainability through a deep learning weak-supervision methodology. Comput Methods Programs Biomed 208:106275

Petersen SE, Matthews PM, Francis JM, Robson MD, Zemrak F, Boubertakh R, Young AA, Hudson S, Weale P, Garratt S et al (2015) UK biobank’s cardiovascular magnetic resonance protocol. J Cardiovasc Magn Reson 18(1):1–7

Pham T-H, Yin C, Mehta L, Zhang X, Zhang P (2023) A fair and interpretable network for clinical risk prediction: a regularized multi-view multi-task learning approach. Knowl Inf Syst 65(4):1487–1521

Pičulin M, Smole T, Žunkovič B, Kokalj E, Robnik-Šikonja M, Kukar M, Fotiadis DI, Pezoulas VC, Tachos NS, Barlocco F et al (2022) Disease progression of hypertrophic cardiomyopathy: modeling using machine learning. JMIR Med Inform 10(2):30483

Pieszko K, Shanbhag AD, Singh A, Hauser MT, Miller RJ, Liang JX, Motwani M, Kwieciński J, Sharir T, Einstein AJ et al (2023) Time and event-specific deep learning for personalized risk assessment after cardiac perfusion imaging. NPJ Digit Med 6(1):78

Plumb G, Molitor D, Talwalkar AS (2018) Model agnostic supervised local explanations. In: Advances in neural information processing systems 31 (NeurIPS 2018)

Portella JJ, Andonian BJ, Brown DE, Mansur J, Wales D, West VL, Kraus WE, Hammond WE (2022) Using machine learning to identify organ system specific limitations to exercise via cardiopulmonary exercise testing. IEEE J Biomed Health Inform 26(8):4228–4237

Prifti E, Fall A, Davogustto G, Pulini A, Denjoy I, Funck-Brentano C, Khan Y, Durand-Salmon A, Badilini F, Wells QS et al (2021) Deep learning analysis of electrocardiogram for risk prediction of drug-induced arrhythmias and diagnosis of long QT syndrome. Eur Heart J 42(38):3948–3961

Qu Y, Deng X, Lin S, Han F, Chang HH, Ou Y, Nie Z, Mai J, Wang X, Gao X et al (2022) Using innovative machine learning methods to screen and identify predictors of congenital heart diseases. Front Cardiovasc Med 8:797002

Ragnarsdottir H, Manduchi L, Michel H, Laumer F, Wellmann S, Ozkan E, Vogt JE (2022) Interpretable prediction of pulmonary hypertension in newborns using echocardiograms. In: DAGM German conference on pattern recognition. Springer, Cham, pp 529–542

Rao S, Li Y, Ramakrishnan R, Hassaine A, Canoy D, Cleland J, Lukasiewicz T, Salimi-Khorshidi G, Rahimi K (2022) An explainable transformer-based deep learning model for the prediction of incident heart failure. IEEE J Biomed Health Inform 26(7):3362–3372

Rashed-Al-Mahfuz M, Moni MA, Lio’ P, Islam SMS, Berkovsky S, Khushi M, Quinn JM (2021) Deep convolutional neural networks based ECG beats classification to diagnose cardiovascular conditions. Biomed Eng Lett 11:147–162

Rauf A, Ullah A, Rathi U, Ashfaq Z, Ullah H, Ashraf A, Kumar J, Faraz M, Akhtar W, Mehmoodi A et al (2023) Predicting stroke and mortality in mitral stenosis with atrial flutter: a machine learning approach. Ann Noninvasive Electrocardiol 28(5):e13078

Ribeiro MT, Singh S, Guestrin C (2016) “Why should i trust you?” Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp 1135–1144

Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention—MICCAI 2015: 18th international conference, Munich, Germany, 5–9 October 2015, Proceedings, Part III 18. Springer, Cham, pp 234–241

Roseiro M, Henriques J, Paredes S, Rocha T, Sousa J (2023) An interpretable machine learning approach to estimate the influence of inflammation biomarkers on cardiovascular risk assessment. Comput Methods Programs Biomed 230:107347

Rouhi R, Clausel M, Oster J, Lauer F (2021) An interpretable hand-crafted feature-based model for atrial fibrillation detection. Front Physiol 12:657304

Rueda C, Rodríguez-Collado A, Fernández I, Canedo C, Ugarte MD, Larriba Y (2022) A unique cardiac electrocardiographic 3D model. Toward interpretable AI diagnosis. iScience 25(12):105617

Sager S, Bernhardt F, Kehrle F, Merkert M, Potschka A, Meder B, Katus H, Scholz E (2021) Expert-enhanced machine learning for cardiac arrhythmia classification. PLoS ONE 16(12):0261571

Saito Y, Omae Y, Fukamachi D, Nagashima K, Mizobuchi S, Kakimoto Y, Toyotani J, Okumura Y (2022) Quantitative estimation of pulmonary artery wedge pressure from chest radiographs by a regression convolutional neural network. Heart Vessels 37(8):1387–1394

Sakai A, Komatsu M, Komatsu R, Matsuoka R, Yasutomi S, Dozen A, Shozu K, Arakaki T, Machino H, Asada K et al (2022) Medical professional enhancement using explainable artificial intelligence in fetal cardiac ultrasound screening. Biomedicines 10(3):551

Salih A, Galazzo IB, Cruciani F, Brusini L, Radeva P (2022) Investigating explainable artificial intelligence for mri-based classification of dementia: a new stability criterion for explainable methods. In: 2022 IEEE International Conference on Image Processing (ICIP), pp. 4003–4007. IEEE

Salih AM, Pujadas ER, Campello VM, McCracken C, Harvey NC, Neubauer S, Lekadir K, Nichols TE, Petersen SE, Raisi-Estabragh Z (2023a) Image-based biological heart age estimation reveals differential aging patterns across cardiac chambers. J Magn Reson Imaging 58(6):1797–1812

Salih A, Boscolo Galazzo I, Gkontra P, Lee AM, Lekadir K, Raisi-Estabragh Z, Petersen SE (2023b) Explainable artificial intelligence and cardiac imaging: toward more interpretable models. Circ Cardiovasc Imaging 16(4):e014519

Salih AM, Galazzo IB, Raisi-Estabragh Z, Petersen SE, Menegaz G, Radeva P (2024) Characterizing the contribution of dependent features in XAI methods. IEEE J Biomed Health Inform. https://doi.org/10.1109/JBHI.2024.3395289

Sammani A, Leur RR, Henkens MT, Meine M, Loh P, Hassink RJ, Oberski DL, Heymans SR, Doevendans PA, Asselbergs FW et al (2022) Life-threatening ventricular arrhythmia prediction in patients with dilated cardiomyopathy using explainable electrocardiogram-based deep neural networks. Europace 24(10):1645–1654

Sang Y, Beetz M, Grau V (2022) Generation of 12-lead electrocardiogram with subject-specific, image-derived characteristics using a conditional variational autoencoder. In: 2022 IEEE 19th International symposium on biomedical imaging (ISBI). IEEE, pp 1–5

Sangha V, Nargesi AA, Dhingra LS, Khunte A, Mortazavi BJ, Ribeiro AH, Banina E, Adeola O, Garg N, Brandt CA et al (2022) Detection of left ventricular systolic dysfunction from electrocardiographic images. Circulation. https://doi.org/10.1161/CIRCULATIONAHA.122.06264

Sangroya A, Jain S, Vig L, Anantaram C, Ukil A, Khandelwal S (2022) Generating conceptual explanations for DL based ECG classification model. In: The International FLAIRS conference proceedings, vol 35

Sanjeevi G, Gopalakrishnan U, Pathinarupothi RK, Madathil T (2023) Automatic diagnostic tool for detection of regional wall motion abnormality from echocardiogram. J Med Syst 47(1):13

Sannino G, De Pietro G, De Falco I (2021) Automatic extraction of interpretable knowledge to predict the survival of patients with heart failure. In: 2021 IEEE/ACM conference on connected health: applications, systems and engineering technologies (CHASE). IEEE, pp 166–173

Sawano S, Kodera S, Katsushika S, Nakamoto M, Ninomiya K, Shinohara H, Higashikuni Y, Nakanishi K, Nakao T, Seki T et al (2022) Deep learning model to detect significant aortic regurgitation using electrocardiography. J Cardiol 79(3):334–341

Sbrollini A, Leoni C, Jongh MC, Morettini M, Burattini L, Swenne CA (2022) Feature contributions to ECG-based heart-failure detection: deep learning vs. statistical analysis. In: 2022 Computing in cardiology (CinC), vol 498. IEEE, pp 1–4

Schrutka L, Anner P, Agibetov A, Seirer B, Dusik F, Rettl R, Duca F, Dalos D, Dachs T-M, Binder C et al (2022) Machine learning-derived electrocardiographic algorithm for the detection of cardiac amyloidosis. Heart 108(14):1137–1147

Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-CAM: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE international conference on computer vision, pp 618–626

Shah B, Kunal S, Bansal A, Jain J, Poundrik S, Shetty MK, Batra V, Chaturvedi V, Yusuf J, Mukhopadhyay S et al (2022) Heart rate variability as a marker of cardiovascular dysautonomia in post-covid-19 syndrome using artificial intelligence. Indian Pacing Electrophysiol J 22(2):70–76

Sharma Y, Coronato N, Brown DE (2022) Encoding cardiopulmonary exercise testing time series as images for classification using convolutional neural network. In: 2022 44th annual international conference of the IEEE Engineering in Medicine & Biology Society (EMBC). IEEE, pp 1611–1614

Shi Z, Zeng G, Zhang L, Zhuang X, Li L, Yang G, Zheng G (2018) Bayesian voxdrn: A probabilistic deep voxelwise dilated residual network for whole heart segmentation from 3d mr images. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2018: 21st International Conference, Granada, Spain, September 16-20, 2018, Proceedings, Part IV 11, pp. 569–577. Springer

Shi H, Yang D, Tang K, Hu C, Li L, Zhang L, Gong T, Cui Y (2022) Explainable machine learning model for predicting the occurrence of postoperative malnutrition in children with congenital heart disease. Clin Nutr 41(1):202–210

Shin SJ, Park J, Lee S-H, Yang K, Park RW (2021) Predictability of mortality in patients with myocardial injury after noncardiac surgery based on perioperative factors via machine learning: retrospective study. JMIR Med Inform 9(10):32771

Silva A, Schrum M, Hedlund-Botti E, Gopalan N, Gombolay M (2023) Explainable artificial intelligence: evaluating the objective and subjective impacts of XAI on human–agent interaction. Int J Hum Comput Interact 39(7):1390–1404

Singh P, Sharma A (2022) Interpretation and classification of arrhythmia using deep convolutional network. IEEE Trans Instrum Meas 71:1–12

Singh A, Kwiecinski J, Miller RJ, Otaki Y, Kavanagh PB, Van Kriekinge SD, Parekh T, Gransar H, Pieszko K, Killekar A et al (2022) Deep learning for explainable estimation of mortality risk from myocardial positron emission tomography images. Circ Cardiovasc Imaging 15(9):014526

Singh A, Miller RJ, Otaki Y, Kavanagh P, Hauser MT, Tzolos E, Kwiecinski J, Van Kriekinge S, Wei C-C, Sharir T et al (2023) Direct risk assessment from myocardial perfusion imaging using explainable deep learning. Cardiovasc Imaging 16(2):209–220

Slack D, Hilgard S, Jia E, Singh S, Lakkaraju H (2020) Fooling lime and shap: Adversarial attacks on post hoc explanation methods. In: Proceedings of the AAAI/ACM conference on AI, ethics, and society, pp 180–186

Smole T, Žunkovič B, Pičulin M, Kokalj E, Robnik-Šikonja M, Kukar M, Fotiadis DI, Pezoulas VC, Tachos NS, Barlocco F et al (2021) A machine learning-based risk stratification model for ventricular tachycardia and heart failure in hypertrophic cardiomyopathy. Comput Biol Med 135:104648

Soares E, Angelov P, Gu X (2020) Autonomous learning multiple-model zero-order classifier for heart sound classification. Appl Soft Comput 94:106449

Soto JT, Weston Hughes J, Sanchez PA, Perez M, Ouyang D, Ashley EA (2022) Multimodal deep learning enhances diagnostic precision in left ventricular hypertrophy. Eur Heart J Digit Health 3(3):380–389

Springenberg JT, Dosovitskiy A, Brox T, Riedmiller M (2014) Striving for simplicity: the all convolutional net. arXiv preprint. arXiv:1412.6806

Stabellini N, Dmukauskas M, Bittencourt MS, Cullen J, Barda AJ, Moore JX, Dent S, Abdel-Qadir H, Kawatkar AA, Pandey A et al (2023) Social determinants of health and racial disparities in cardiac events in breast cancer. J Natl Compr Canc Netw 21(7):705–714

Sun J, Darbehani F, Zaidi M, Wang B (2020) SAUNet: shape attentive U-Net for interpretable medical image segmentation. In: Medical image computing and computer assisted intervention—MICCAI 2020: 23rd international conference, Lima, Peru, 4–8 October 2020, Proceedings, Part IV 23. Springer, pp 797–806

Sun R, Wang X, Jiang H, Yan Y, Dong Y, Yan W, Luo X, Miu H, Qi L, Huang Z (2022) Prediction of 30-day mortality in heart failure patients with hypoxic hepatitis: Development and external validation of an interpretable machine learning model. Front Cardiovasc Med 9:1035675

Sundararajan M, Taly A, Yan Q (2017) Axiomatic attribution for deep networks. In: International conference on machine learning. PMLR, pp 3319–3328

Szabo L, Raisi-Estabrag Z, Salih A, McCracken C, Pujadas ER, Gkontra P, Kiss M, Maurovich-Horvath P, Vago H, Merkely B et al (2022) Clinician’s guide to trustworthy and responsible artificial intelligence in cardiovascular imaging. Front Cardiovasc Med 9:1016032

Tamarappoo BK, Lin A, Commandeur F, McElhinney PA, Cadet S, Goeller M, Razipour A, Chen X, Gransar H, Cantu S et al (2021) Machine learning integration of circulating and imaging biomarkers for explainable patient-specific prediction of cardiac events: a prospective study. Atherosclerosis 318:76–82

Tang Q, Cen X, Pan C (2022) Explainable and efficient deep early warning system for cardiac arrest prediction from electronic health records. Math Biosci Eng 19(10):9825–9841

Tong Q, Li C, Si W, Liao X, Tong Y, Yuan Z, Heng PA (2019) RIANet: recurrent interleaved attention network for cardiac mri segmentation. Comput Biol Med 109:290–302

Tsuji T, Hirata Y, Kusunose K, Sata M, Kumagai S, Shiraishi K, Kotoku J (2023) Classification of chest X-ray images by incorporation of medical domain knowledge into operation branch networks. BMC Med Imaging 23(1):1–18

Ukil A, Marin L, Jara AJ (2023) Priv-Aug-Shap-ECGResNet: privacy preserving shapley-value attributed augmented resnet for practical single-lead electrocardiogram classification. In: ICASSP 2023-2023 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 1–5

Vaduganathan M, Mensah GA, Turco JV, Fuster V, Roth GA (2022) The global burden of cardiovascular diseases and risk: a compass for future health. American College of Cardiology Foundation, Washington

Vafaeezadeh M, Behnam H, Hosseinsabet A, Gifani P (2022) Automatic morphological classification of mitral valve diseases in echocardiographic images based on explainable deep learning methods. Int J Comput Assist Radiol Surg 17(2):413–425

Valvano G, Leo A, Tsaftaris SA (2022) Regularizing disentangled representations with anatomical temporal consistency. In: Biomedical image synthesis and simulation. Elsevier, Amsterdam, pp 325–346

Vaulet T, Al-Memar M, Fourie H, Bobdiwala S, Saso S, Pipi M, Stalder C, Bennett P, Timmerman D, Bourne T et al (2022) Gradient boosted trees with individual explanations: an alternative to logistic regression for viability prediction in the first trimester of pregnancy. Comput Methods Programs Biomed 213:106520

Vazquez B, Fuentes-Pineda G, Garcia F, Borrayo G, Prohias J (2021) Risk markers by sex for in-hospital mortality in patients with acute coronary syndrome: a machine learning approach. Inform Med Unlocked 27:100791

Vijayarangan S, Murugesan B, Vignesh R, Preejith S, Joseph J, Sivaprakasam M (2020) Interpreting deep neural networks for single-lead ecg arrhythmia classification. In: 2020 42nd annual international conference of the IEEE engineering in medicine & biology society (EMBC). IEEE, pp 300–303

Wagner P, Mehari T, Haverkamp W, Strodthoff N (2024) Explaining deep learning for ECG analysis: building blocks for auditing and knowledge discovery. Comput Biol Med 176:108525

Wall HE, Hassing G-J, Doll R-J, Westen GJ, Cohen AF, Selder JL, Kemme M, Burggraaf J, Gal P (2022) Cardiac age detected by machine learning applied to the surface ecg of healthy subjects: creation of a benchmark. J Electrocardiol 72:49–55

Wang Q et al (2021a) Machine learning-based risk prediction of malignant arrhythmia in hospitalized patients with heart failure. ESC Heart Fail 8(6):5363–5371

Wang S, Li J, Sun L, Cai J, Wang S, Zeng L, Sun S (2021b) Application of machine learning to predict the occurrence of arrhythmia after acute myocardial infarction. BMC Med Inform Decis Mak 21:1–14

Wang J, Liu X, Wang F, Zheng L, Gao F, Zhang H, Zhang X, Xie W, Wang B (2021c) Automated interpretation of congenital heart disease from multi-view echocardiograms. Med Image Anal 69:101942

Wang Y, Chen W, Tang T, Xie W, Jiang Y, Zhang H, Zhou X, Yuan K (2022a) Cardiac segmentation method based on domain knowledge. Ultrason Imaging 44(2–3):105–117

Wang X, Qi M, Dong C, Zhang H, Yang Y, Zhao H (2022b) Accurately identifying coronary atherosclerotic heart disease through merged beats of electrocardiogram. In: 2022 IEEE international conference on bioinformatics and biomedicine (BIBM). IEEE, pp 1249–1254

Wang J, Xie W, Cheng M, Wu Q, Wang F, Li P, Fan B, Zhang X, Wang B, Liu X (2022c) Assessment of transcatheter or surgical closure of atrial septal defect using interpretable deep keypoint stadiometry. Research 2022:9790653

Wang K, Yan LZ, Li WZ, Jiang C, Wang NN, Zheng Q, Dong NG, Shi JW (2022d) Comparison of four machine learning techniques for prediction of intensive care unit length of stay in heart transplantation patients. Front Cardiovasc Med 9:863642

Wesołowski S, Lemmon G, Hernandez EJ, Henrie A, Miller TA, Weyhrauch D, Puchalski MD, Bray BE, Shah RU, Deshmukh VG et al (2022) An explainable artificial intelligence approach for predicting cardiovascular outcomes using electronic health records. PLoS Digit Health 1(1):0000004

Wickramasinghe NL, Athif M (2022) Multi-label classification of reduced-lead ecgs using an interpretable deep convolutional neural network. Physiol Meas 43(6):064002

Wong XY, Ang YK, Li K, Chin YH, Lam SSW, Tan KBK, Chua MCH, Ong MEH, Liu N, Pourghaderi AR et al (2022) Development and validation of the sarica score to predict survival after return of spontaneous circulation in out of hospital cardiac arrest using an interpretable machine learning framework. Resuscitation 170:126–133

Wongvibulsin S, Wu KC, Zeger SL et al (2020) Improving clinical translation of machine learning approaches through clinician-tailored visual displays of black box algorithms: development and validation. JMIR Med Inform 8(6):15791

Wouters PC, Leur RR, Vessies MB, Stipdonk AM, Ghossein MA, Hassink RJ, Doevendans PA, Harst P, Maass AH, Prinzen FW et al (2023) Electrocardiogram-based deep learning improves outcome prediction following cardiac resynchronization therapy. Eur Heart J 44(8):680–692

Wu Z-W, Zheng J-L, Kuang L, Yan H (2023) Machine learning algorithms to automate differentiating cardiac amyloidosis from hypertrophic cardiomyopathy. Int J Cardiovasc Imaging 39(2):339–348

Wu C, Zhang H, Chen J, Gao Z, Zhang P, Muhammad K, Del Ser J (2022) Vessel-Gan: angiographic reconstructions from myocardial ct perfusion with explainable generative adversarial networks. Futur Gener Comput Syst 130:128–139

Wu Z, Li Y, Xu Z, Liu H, Liu K, Qiu P, Chen T, Lu X (2023) Prediction of preoperative in-hospital mortality rate in patients with acute aortic dissection by machine learning: a two-centre, retrospective cohort study. BMJ Open 13(4):066782

Xiao R, Ding C, Hu X, Clifford GD, Wright DW, Shah AJ, Al-Zaiti S, Zègre-Hemsey JK (2023) Integrating multimodal information in machine learning for classifying acute myocardial infarction. Physiol Meas 44(4):044002

Xing J, Ghadimi S, Abdi M, Bilchick KC, Epstein FH, Zhang M (2021) Deep networks to automatically detect late-activating regions of the heart. In: 2021 IEEE 18th international symposium on biomedical imaging (ISBI). IEEE, pp 1902–1906

Yalcin O, Fan X, Liu S (2021) Evaluating the correctness of explainable ai algorithms for classification. arXiv preprint. arXiv:2105.09740

Yang H, Shan C, Kolen AF, With PH (2022) Weakly-supervised learning for catheter segmentation in 3d frustum ultrasound. Comput Med Imaging Graph 96:102037

Ye X, Huang Y, Lu Q (2021) Explainable prediction of cardiac arrhythmia using machine learning. In: 2021 14th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), pp. 1–5. IEEE

Yin C, Zhao R, Qian B, Lv X, Zhang P (2019) Domain knowledge guided deep learning with electronic health records. In: 2019 IEEE International conference on data mining (ICDM). IEEE, pp 738–747

Yoo J, Jun TJ, Kim Y-H (2021) xECGNet: fine-tuning attention map within convolutional neural network to improve detection and explainability of concurrent cardiac arrhythmias. Comput Methods Programs Biomed 208:106281

Yue Y, Zhu X (2023) Automated coronary artery disease detection using deep learning on ECG datasets. In: Proceedings of the 2023 3rd International conference on bioinformatics and intelligent computing, pp 242–245

Zeng X, Hu Y, Shu L, Li J, Duan H, Shu Q, Li H (2021) Explainable machine-learning predictions for complications after pediatric congenital heart surgery. Sci Rep 11(1):17244

Zeng Z, Wang Q, Yu Y, Zhang Y, Chen Q, Lou W, Wang Y, Yan L, Cheng Z, Xu L et al (2022) Assessing electrocardiogram changes after ischemic stroke with artificial intelligence. PLoS ONE 17(12):0279706

Zeng Z, Tian X, Li L, Diao Y, Zhang T (2024) An interpretable machine learning model to predict off-pump coronary artery bypass grafting-associated acute kidney injury. Adv Clin Exp Med 33(5):473–481

Zhang X, Qian B, Li X, Wei J, Zheng Y, Song L, Zheng Q (2019a) An interpretable fast model for predicting the risk of heart failure. In: Proceedings of the 2019 SIAM international conference on data mining. SIAM, Philadelphia, pp 576–584

Zhang X, Qian B, Li Y, Yin C, Wang X, Zheng Q (2019b) KnowRisk: an interpretable knowledge-guided model for disease risk prediction. In: 2019 IEEE International conference on data mining (ICDM). IEEE, pp 1492–1497

Zhang O, Ding C, Pereira T, Xiao R, Gadhoumi K, Meisel K, Lee RJ, Chen Y, Hu X (2021a) Explainability metrics of deep convolutional networks for photoplethysmography quality assessment. IEEE Access 9:29736–29745

Zhang D, Yang S, Yuan X, Zhang P (2021b) Interpretable deep learning for automatic diagnosis of 12-lead electrocardiogram. iScience 24(4):102373

Zhang P, Ma C, Song F, Sun Y, Feng Y, He Y, Zhang T, Zhang G (2023) D2AFNet: a dual-domain attention cascade network for accurate and interpretable atrial fibrillation detection. Biomed Signal Process Control 82:104615

Zhao S, Diao X, Xia Y, Huo Y, Cui M, Wang Y, Yuan J, Zhao W (2023) Automated ICD coding for coronary heart diseases by a deep learning method. Heliyon 9(3):e14037

Zhou D, Qiu H, Wang L, Shen M (2023) Risk prediction of heart failure in patients with ischemic heart disease using network analytics and stacking ensemble learning. BMC Med Inform Decis Mak 23(1):99

Download references

AMS is supported by a British Heart Foundation project grant (PG/21/10619), Barts Charity (G-002523) and by Leicester City Football Club (LCFC) Programme. ER is supported by the mini-Centre for Doctoral Training (CDT) award through the Faculty of Science and Engineering, Queen Mary University of London, United Kingdom. This work acknowledges the support of the National Institute for Health and Care Research Barts Biomedical Research Centre (NIHR203330); a delivery partnership of Barts Health NHS Trust, Queen Mary University of London, St George’s University Hospitals NHS Foundation Trust and St George’s University of London. SEP received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 825903 (euCanSHare project). P.G. and K.L are partly funded by the Horizon Europe projects and innovation programme under grant agreement no. 101057849 (DataTools4Heart project) and grant agreement no 101080430 (AI4HF project).

Author information

Authors and affiliations.

William Harvey Research Institute, NIHR Barts Biomedical Research Centre, Queen Mary University of London, Charterhouse Square, London, EC1M 6BQ, UK

Ahmed M. Salih, Elisa Rauseo, Aaron Mark Lee & Steffen E. Petersen

Department of Population Health Sciences, University of Leicester, University Rd, Leicester, LE1 7RH, UK

Ahmed M. Salih

Department of Computer Science, University of Zakho, Duhok road, Zakho, Kurdistan, Iraq

Department of Engineering for Innovative Medicine, University of Verona, S. Francesco, 22, 37129, Verona, Italy

Ilaria Boscolo Galazzo & Gloria Menegaz

Artificial Intelligence in Medicine Lab (BCN-AIM), Departament de Matemàtiques i Informàtica, Universitat de Barcelona, Gran Via de les Corts Catalanes, 585, 08007, Barcelona, Spain

Polyxeni Gkontra & Karim Lekadir

Institució Catalana de Recerca i Estudis Avançats (ICREA), Passeig Lluís Companys 23, Barcelona, Spain

Karim Lekadir

Departament de Matemàtiques i Informàtica, Universitat de Barcelona, Gran Via de les Corts Catalanes, 585, 08007, Barcelona, Spain

Petia Radeva

Barts Heart Centre, St Bartholomew’s Hospital, Barts Health NHS Trust, West Smithfield, London, UK

Steffen E. Petersen

Health Data Research, London, UK

Alan Turing Institute, London, UK

You can also search for this author in PubMed   Google Scholar

Contributions

AMS: conceptualization, methodology, investigation, writing-original draft and visualization. IBG, PG and AML: writing, review and editing. ER, KL, PR, SEP and GM: commented and contributed to all the following versions. All authors read and approved the final manuscript

Corresponding author

Correspondence to Ahmed M. Salih .

Ethics declarations

Conflict of interest.

The authors declare that they have no Conflict of interest.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 14 KB)

Supplementary file3 (xlsx 47 kb), rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Salih, A.M., Galazzo, I.B., Gkontra, P. et al. A review of evaluation approaches for explainable AI with applications in cardiology. Artif Intell Rev 57 , 240 (2024). https://doi.org/10.1007/s10462-024-10852-w

Download citation

Accepted : 03 July 2024

Published : 09 August 2024

DOI : https://doi.org/10.1007/s10462-024-10852-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Find a journal
• Publish with us
• Track your research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

Nudging Toward Sustainable Food Consumption at University Canteens: A Systematic Review and Meta-Analysis

Affiliations.

• 1 Department of Food Science, University of Copenhagen, Frederiksberg, Denmark.
• 2 Department of Food, Nutrition, and Culinary Science, Umeå University, Umeå, Sweden.
• 3 Department of Food Science, University of Copenhagen, Frederiksberg, Denmark. Electronic address: [email protected].
• PMID: 37930295
• DOI: 10.1016/j.jneb.2023.09.006

Introduction: This systematic literature review and meta-analysis investigated the effectiveness of the nudging approach toward sustainable food consumption in the university canteen context.

Methods: The systematic literature search was carried out in 5 databases, Web of Science, PubMed, Scopus, ProQuest, and the Royal Library, identifying 14 eligible studies and selecting 9 articles containing adequate information for meta-analysis. The nudging strategies were classified using the typology of interventions in the proximal physical microenvironments framework that resulted in 5 different intervention types: availability, position, size, presentation, and information that belonged to either intervention class-altering properties or placement.

Results: The study identified presentation, availability, and information as the most promising nudge intervention for achieving sustainable food consumption at the university canteen or similar settings. Nudging by altering the properties had a small effect size (d = 0.16), and nudging by altering placement showed a medium effect size (d = 0.21).

Discussion: Nudging interventions implemented after understanding consumers' current behavior showed positive effectiveness toward sustainable food consumption rather than implementing random nudges.

Conclusions and implications: It is important that future studies aim to achieve sustainable food consumption by understanding canteen user food preferences and food choice motives before designing a nudging strategy.

Keywords: meta-analysis; nudging; sustainable food consumption; systematic literature review; university canteen.

Copyright © 2023 The Authors. Published by Elsevier Inc. All rights reserved.

PubMed Disclaimer

Similar articles

• A systematic review and meta-analysis of the effectiveness of nudging to increase fruit and vegetable choice. Broers VJV, De Breucker C, Van den Broucke S, Luminet O. Broers VJV, et al. Eur J Public Health. 2017 Oct 1;27(5):912-920. doi: 10.1093/eurpub/ckx085. Eur J Public Health. 2017. PMID: 28655176 Review.
• Privacy nudges for disclosure of personal information: A systematic literature review and meta-analysis. Ioannou A, Tussyadiah I, Miller G, Li S, Weick M. Ioannou A, et al. PLoS One. 2021 Aug 27;16(8):e0256822. doi: 10.1371/journal.pone.0256822. eCollection 2021. PLoS One. 2021. PMID: 34449821 Free PMC article.
• Nudging to get our food choices on a sustainable track. Vandenbroele J, Vermeir I, Geuens M, Slabbinck H, Van Kerckhove A. Vandenbroele J, et al. Proc Nutr Soc. 2020 Feb;79(1):133-146. doi: 10.1017/S0029665119000971. Epub 2019 Jun 28. Proc Nutr Soc. 2020. PMID: 31250781 Review.
• Customer support for nudge strategies to promote fruit and vegetable intake in a university food service. Yi S, Kanetkar V, Brauer P. Yi S, et al. BMC Public Health. 2022 Apr 10;22(1):706. doi: 10.1186/s12889-022-13054-7. BMC Public Health. 2022. PMID: 35399080 Free PMC article.
• Determinants of Food Choice and Perceptions of Supermarket-Based Nudging Interventions among Adults with Low Socioeconomic Position: The SUPREME NUDGE Project. Harbers MC, Middel CNH, Stuber JM, Beulens JWJ, Rutters F, van der Schouw YT. Harbers MC, et al. Int J Environ Res Public Health. 2021 Jun 7;18(11):6175. doi: 10.3390/ijerph18116175. Int J Environ Res Public Health. 2021. PMID: 34200437 Free PMC article.

Publication types

• Search in MeSH

Related information

Linkout - more resources, full text sources.

• Elsevier Science
• Ovid Technologies, Inc.

• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

• Open access
• Published: 06 August 2024

Salivary 8-hydroxy-2′-deoxyguanosine levels in patients with oral cancer: a systematic review and meta-analysis

• Mario Alberto Alarcón-Sánchez   ORCID: orcid.org/0000-0001-6727-7969 1 ,
• Lilibeth-Stephania Escoto-Vasquez   ORCID: orcid.org/0000-0003-1608-8732 2 &
• Artak Heboyan   ORCID: orcid.org/0000-0001-8329-3205 3 , 4 , 5  

BMC Cancer volume  24 , Article number:  960 ( 2024 ) Cite this article

198 Accesses

Metrics details

DNA is an important target for oxidative attack and its modification may increase the risk of mutagenesis. The aim of this study was to evaluate and compare salivary levels of the oxidative stress biomarker 8-hydroxy-2′-deoxyguanosine (8-OHdG) in patients with oral cancer (OC) compared to the control group by a comprehensive search of the available literature.

The present systematic review and meta-analysis followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines and was registered in Open Science Framework (OSF): https://doi.org/10.17605/OSF.IO/X3YMR. Four electronic databases were used to identify studies for this systematic review: PubMed, Scopus, ScienceDirect, and Web of Science from January 15, 2005, to April 15, 2021. The Joanna Briggs Institute (JBI) tool was used to assess article quality.

Of the 166 articles identified, 130 articles were excluded on the basis of title and abstract screening (duplicates, reviews, etc.). Thirty-six articles were evaluated at full text and 7 articles met the inclusion criteria. Of these, only 5 studies had compatible data for quantitative analysis. An increase in salivary 8-OHdG levels was found in patients with OC compared to healthy subjects, but without statistical significance. 8-OHdG: SMD = 2,72 (95%CI= -0.25–5.70); * p  = 0.07.

Conclusions

This systematic review and meta-analysis suggests a clear trend of increased 8-OHdG levels in saliva of OC patients compared to the control group. However, further studies are required to clarify and understand the altered levels of this oxidative stress marker.

Peer Review reports

Oral cancer (OC) is a malignant neoplasm that in ≈ 90% of cases, histologically corresponds to oral squamous cell carcinoma (OSCC) [ 1 ], which can arise de novo or from oral potentially malignant disorders (OPMD) such as oral erythroplasia, oral submucosal fibrosis and/or oral leukoplakia [ 2 ] OSCC represents the 16th most common cancer worldwide, with more than 377,000 new cases per year [ 3 ]. The 5-year survival rate is 50% and decreases to 30% as the stage of the disease advances [ 4 ], making it a major social, economic and public health problem [ 5 ]. Tobacco and alcohol are the main etiological factors contributing to its development [ 6 ]. OSCC most frequently affects men in the 5th to 6th decade of life [ 7 ]. Clinically, the lesions present as asymptomatic, nonhealing ulcers of variable size and indurated borders [ 8 ]. These lesions are usually found on the tongue, floor of the mouth, buccal mucosa, alveolar ridges, retromolar trigone and hard palate [ 9 ]. The diagnosis is made by a thorough, visual and clinical examination of the oral cavity and confirmed by histopathological study of surgical biopsy as part of the gold standard [ 10 ].

Currently, clinicians and researchers in the field have made several efforts to find molecules indicative of the onset and progression or transformation of OPMD to OSCC [ 11 ]. In this regard, saliva is a biofluid that, unlike others, is more accessible, cost-effective, simple to collect, and noninvasive [ 12 ]. Moreover, it reflects the oxidative status of subjects with this type of lesions [ 13 ].

Oxidative stress is an important process in the pathobiology of OSCC [ 14 ]. High concentrations of reactive oxygen species (ROS) unbalance the antioxidant protection mechanisms provided by enzymes such as glutathione peroxidase, reduced glutathione, superoxide dismutase and malondialdehyde [ 15 ].

The production of oxygen free radicals such as hydroxyl radicals (HO•) can cause significant damage to DNA strands. Interactions occurring between these molecules, and in particular on the nitrogenous base guanine form C8-hydroguanine (8-OHGua) and by electron abstraction mechanisms, 8-hydroxy-2’-deoxyguanosine (8-OHdG) is formed [ 16 ].

Some research has been published reporting differences in salivary levels of 8-OHdG and its possible association as a marker of DNA damage in patients with OC compared to the healthy population [ 17 , 18 , 19 , 20 , 21 , 22 , 23 ], however, to date, no systematic review summarizing those findings has been published.

Therefore, this study aims to evaluate and compare the salivary levels of the oxidative stress biomarker 8-OHdG in OC patients compared to the healthy control group by a comprehensive search of the available literature.

Protocol and register

The Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) guidelines were used to construct the protocol for the present systematic review and meta-analysis [ 24 ]. The Open Science Framework (OSF: https://doi.org/10.17605/OSF.IO/X3YMR ) platform was used to register the study.

PICOD and research question

Population: Patients with OC.

Intervention: Quantification of the analyte 8-OHdG in saliva of OC patients and systemically healthy patients.

Comparators: Systemically healthy subjects.

Outcomes: Observed changes in salivary 8-OHdG levels in OC patients and systemically healthy subjects.

Design: Case-control studies.

The research question was: Are there significant differences between salivary 8-OHdG levels in patients with OC compared to the control group?

Elegibility criteria

Inclusion criteria.

Case-control studies.

Studies approved by the institutional ethics committee.

Studies that will confirm the diagnosis of OSCC by biopsy.

Stimulated/unstimulated saliva samples.

Techniques such as Enzyme-Linked ImmunoSorbent Assay (ELISA) and DNA damage assay.

Investigations that will show numerical values (mean ± standard deviation) of 8-OHdG levels.

Exclusion criteria

Investigations in cell lines or animal models.

Case reports and case series.

Book chapters and encyclopedias.

Systematic, narrative, scoping, bibliometric reviews and meta-analyses.

Healthy controls with comorbidities and other systemic disorders.

Research published in a language other than English.

Research published before 2005.

Electronic and manual literature search

An electronic search was carried out in four databases: PubMed, Scopus, ScienceDirect and Web of Science from January 15, 2005, to April 15, 2021. For PubMed, the following search strategy was employed: (((“8-Hydroxy-2’-Deoxyguanosine”[Mesh]) AND “Saliva”[Mesh]) AND) AND “Mouth Neoplasms”[Majr]. For the rest, the keywords “8-hydroxy-2′-deoxyguanosine”, “8-OHdG”, “Saliva”, “Biomarkers” and “Oral Cancer” were used, along with the use of Boolean operators “OR” and “AND”. A manual search was also carried out in the following Journals: “ Medicina Oral Patologia Oral y Cirugia Bucal ”, “ Oral Diseases ” “ Oral Surgery , Oral Medicine , Oral Pathology ” and “ Journal of Oral Pathology & Medicine ”.

Screening process

M.A.A.S and A.H screened study titles and abstracts independently. Duplicates and research unrelated to the topic of interest were then discarded. Finally, a full-text analysis of potentially eligible articles was performed by applying the inclusion and exclusion criteria. Disagreements were resolved by consensus.

Data extraction

M.A.A.S and L.S.E.V performed the data collection procedure independently, in predefined tables with Word software (Microsoft):

Name of first author and year of publication.

Country of origin.

Number of cases and controls.

Habits; alcoholism and smoking.

Techniques for detection of the analyte of interest.

Mean value ± standard deviation of salivary 8-OHdG levels.

Quantitative variables were represented with mean ± standard deviation, while qualitative data with absolute and relative frequency n (%).

Quality assessment

M.A.A.S and A.H assessed the quality of the included studies independently. The Joanna Briggs Institute (JBI) tool [ 25 ] was used. The items taken into account were aspects related to comparability, exposure, confounders, time, and statistical analysis of cases and controls. In the study, quality was assessment on a scale ranging from 0 to 100%. Studies scoring between of 0–49% was categorized as low quality, those scoring between 50% and 69% moderate quality, and studies scoring above 70% were categorized as high quality. Any discrepancies were resolved through group discussion.

Statistical analysis

Quantitative analysis on 8-OHdG levels assessed in ng/ml, between the OC patients group vs. control group, was performed using STATA 15 V software (StataCorp, College Station, TX, USA). The standardized mean difference (SMD) method with a 95% confidence interval (CI) was used. A random effects model was used due to the presence of heterogeneity (> 50%=moderate), which was estimated using the Cochrane Q test and quantified with the (I 2 ) statistic. A p * value  ≤  0.05 was considered statistically significant. A forest plot was constructed to visualize estimates with 95% CI. Funnel plot and Egger linear regression were used to assess publication bias.

Study selection

Initially 165 articles were found in the four electronic databases, including PubMed ( n  = 4 papers), Scopus ( n  = 4 papers), ScienceDirect ( n  = 156 papers) and Web of Science ( n  = 1 paper). In the manual search, one more article was found, giving a total of 166 articles. In the identification phase, duplicates were eliminated ( n  = 10). Next, based on title and abstract, 156 remaining studies were reviewed. Applying the eligibility criteria, 120 more records were excluded (reviews n  = 66; encyclopedias n  = 4; book chapters n  = 32; others n  = 18), giving a total of 36 potentially relevant records. After analyzing the full text of the remaining articles, 29 articles were excluded because they were not related to the topic of interest. Therefore, a total of 7 articles were included for qualitative analysis and of those, 5 articles were analyzed quantitatively in the present review. Details of the study selection are shown in Fig.  1 .

PRISMA flow diagram of the study selection process. PRISMA Preferred Reporting Items for Systematic and Meta-Analyses

Clinical and demographic features of studies included

A total of 7 articles with a case-control design were reviewed in this study [ 17 , 18 , 19 , 20 , 21 , 22 , 23 ]. The total number of individuals studied in the included investigations was 664 of which 351 represented the case group (patients with OC) and 313 represented the control group (healthy subjects). The ages of the patients ranged from 30 to 82 years, with a mean age ± (SD) of 58 ± 7.04 years, of which 44% were male, 34% were female and the rest (22%) did not specify gender [ 22 ]. 41% of patients were smokers [ 17 , 18 , 19 , 22 , 23 ], while 24% were alcohol drinkers [ 17 , 18 , 19 , 22 ]. Most of the articles were published after 2012 (6:86%) [ 17 , 18 , 19 , 20 , 21 , 22 ]. The oldest study was from 2006 [ 23 ], and the most recent from 2021 [ 17 ]. The seven studies were published in six different countries [ 17 , 18 , 19 , 20 , 21 , 22 , 23 ]. Two (29%) studies were conducted in India [ 18 , 22 ], the rest (14.2%) were conducted in Korea [ 17 ], Poland [ 19 ], Belgium [ 20 ], Iran [ 21 ], and Israel [ 23 ]. The names of the journals where the articles were published are also described (Table  1 ).

The enzyme-linked immunosorbent assay (ELISA) emerged as the predominant immunodetection method for quantifying salivary 8-OHdG levels in OC patients (86%) [ 17 , 18 , 19 , 20 , 21 , 23 ], followed by the DNA damage quantification kit (BioVision, USA) (14%) [ 22 ]. In addition, qualitative analysis revealed that there is an increase in salivary 8-OHdG levels in patients with OC compared to the control group [ 18 , 20 , 21 , 22 , 23 ] (Table  2 ).

JBI assessment for case-controls studies

Based on the score obtained, 57% of the studies [ 17 , 18 , 19 , 20 ] showed high quality, while 43% showed moderate quality [ 21 , 22 , 23 ] (Table  3 ).

Meta-analysis: comparison of salivary 8-OHdG levels in oral cancer patients and control group

As shown in Fig.  2 , five articles [ 17 , 18 , 19 , 20 , 22 ] compared the difference between salivary 8-OHdG levels in OC patients ( n  = 300) and healthy controls ( n  = 258). An increase in salivary 8-OHdG levels was found compared to the healthy population (SMD = 2.72 (95%CI= -0.25–5.70); p  = 0.07), but without statistical significance. Study heterogeneity was moderate (I 2  = 64.2%, * p  = 0.025), therefore, a random-effects model was used to pool the results. The funnel plot samples the asymmetry and possibility of publication bias. Egger’s test (t = 2.10, p  = 0.127) showed no evidence of bias (Fig.  2 panel A and B).

Forest plot comparing the 8-OHdG levels in saliva of A ) control group vs. exposure group. B ) Funnel plot to check the publication bias

OC is a multifactorial disease that arises through a complex sequence of events marked by diverse genetic and epigenetic modifications [ 26 , 27 ]. One key underlying mechanism in its development and progression is oxidative stress, its role has been characterized by increased pro-oxidant activity, with a consequent decrease in antioxidant activity [ 28 ]. Thus, due to high metabolic activity and loss of mitochondrial function, tumor cells generate more ROS than normal cells, which increases susceptibility to free radicals and indeed oxidative stress [ 29 ]. Despite its significance in the pathogenesis of oral cancer, the clinical assessment of oxidative stress in this context has been limited.

To date, the expression of the oxidative stress marker 8-OHdG, which is formed from the oxidation of damaged DNA guanine, has only been assessed in plasma samples OSCC tissue biopsies and saliva (Fig.  3 ). In plasma 8-OHdG levels have been found to be lower in subjects with OPMD compared to healthy controls [ 30 ]. On the other hand, it has been reported that 80% of samples of OSSC tissue (24/30) showed strong immunostaining intensity, preferentially in the cytoplasm (70%) and nucleus (30%) of neoplastic cells. In addition, tumors exceeding 4 cm in size more frequently expressed 8-OHdG in the cytoplasm. Thus, greater oxidative damage occurs when both subcellular localization structures express 8-OHdG [ 31 ]. In saliva 8-OHdG detection has emerged as a promising tool for assessing oxidative burden and its relationship with oral cancer risk and progression. Saliva is a biofluid that is composed of a rich source of biomolecules (proteins, carbohydrates, lipids and DNA) [ 32 ]. Its collection is relatively simple, inexpensive, reproducible and does not require much time with the patient [ 33 ]. Therefore, it can be used in large-scale studies to search for biomarkers capable of discriminating between healthy and diseased subjects [ 34 ]. At present, scientists have conducted many studies and more than 100 potential biomarkers have been reported until 2024 [ 35 ]. The use of validated salivary biomarkers with high sensitivity and specificity can be used as a valuable tool for both screening and early detection of OSCC, which will increase the quality of medical care [ 36 , 37 ].

Expression of 8-hydroxy-2′-deoxyguanosine in tissue, saliva and plasma biopsies. Created with BioRender: Accessed June 2nd, 2024

In this systematic review, 7 case-control articles were included for the analysis of 8-OHdG in saliva of 351 OC patients and 313 healthy volunteers. The meta-analysis revealed that there is an increase in salivary 8-OHdG levels in the exposure group compared to the control group, but without statistical significance ( p  = > 0.05). Of the previously evaluated studies, two of them [ 17 , 19 ] found no significant association of DNA damage marker 8-OHdG and OC. While five of them [ 18 , 20 , 21 , 22 , 23 ], showed that salivary 8-OHdG levels were increased in patients with OC compared to the control group, reflecting the redox imbalance in these patients.

Nandakumar et al., 2020 [ 18 ] found that mean salivary 8-OHdG values increased progressively from healthy controls to subjects with oral submucous fibrosis and patients with OSCC. This gradient increase in 8-OHdG reflects the increased DNA damage and oxidative stress environment under these conditions. In this study also, smoking was positively correlated with salivary 8-OHdG levels in patients with OSCC. Whereas, betel chewing habit was positively correlated with salivary 8-OHdG levels in patients with oral submucous fibrosis, thus these habits along with alcohol consumption act synergistically and further aggravate the disease process [ 36 , 38 , 39 ]. Kaur et al., 2016 [ 20 ] demonstrated in their study that patients with PML and OC showed significantly higher levels of 8-OHdG and malondialdehyde, and lower levels of vitamin E and C compared to the healthy population. These findings are also consistent with that reported with Hosseini et al., 2012 [ 21 ] whereby, patients with PML such as oral leukoplakia, lichen planus and oral submucous fibrosis and OSCC are more susceptible to an imbalance of antioxidant-oxidative stress status [ 21 ]. Likewise, Kumar et al., 2012 [ 22 ] reported in their study an alteration of this system and observed a substantial increase in the levels of ROS, reactive nitrogen species and 8-OHdG in saliva cell DNA, along with a decrease in the levels of total antioxidant capacity and glutathione. These results are similar to those reported by Bahar et al., 2006 [ 23 ] who showed that oxidative and nitrative stress altered salivary composition in OC patients. Nitrates and nitrites increased substantially, while antioxidant enzymes were reduced, thus explaining the oxidative DNA and protein damage, possibly due to the promotion of OC.

Limitations

The present review had some limitations. On the one hand, the inclusion of a limited number of case-control studies, with a small sample size, as well as a moderate heterogeneity of the results obtained, which could be explained by the use of different clinical staging systems to classify patients with OSCC, two different methodologies to quantify salivary 8-OHdG levels, age, gender, ethnicity and geographic location, as well as other confounding factors such as smoking and alcohol intake could be altering the values of this marker. Future studies should consider these aspects to analyze the effect of oxidative stress on OC.

The present systematic review with subsequent meta-analysis revealed that the concentration of 8-OHdG in saliva of OC patients was 2,72ng/mL higher than that of healthy individuals, but without statistical significance p  = 0.07. This trend toward greater decay reflects an increase in oxygen free radical activity during carcinogenesis in OC. However, further studies are required to clarify and confirm these results.

Data availability

The data supporting this study’s findings are available from the corresponding author upon reasonable request.

Abbreviations

OHdG-8-hidroxi-2’-desoxiguanosina

Oral squamous cell carcinoma

Potentially malignant lesions

OHGua-C8-hidroguanina

Reactive oxygen species

Bugshan A, Farooq I. Oral squamous cell carcinoma: metastasis, potentially associated malignant disorders, etiology and recent advancements in diagnosis. F1000Res. 2020;9:229. https://doi.org/10.12688/f1000research.22941.1 .

Article   PubMed   PubMed Central   CAS   Google Scholar  

Bastías D, Maturana A, Marín C, Martínez R, Niklander SE. Salivary biomarkers for oral Cancer detection: an exploratory systematic review. Int J Mol Sci. 2024;25:2634. https://doi.org/10.3390/ijms25052634 .

Muller S, Tilakaratne WM. Update from the 5th Edition of the World Health Organization Classification of Head and Neck tumors: Tumours of the oral cavity and Mobile Tongue. Head Neck Pathol. 2022;16:54–62. https://doi.org/10.1007/s12105-021-01402-9 .

Article   PubMed   PubMed Central   Google Scholar  

Almangush A, Mäkitie AA, Triantafyllou A, de Bree R, Strojan P, Rinaldo A, et al. Staging and grading of oral squamous cell carcinoma: an update. Oral Oncol. 2020;107:104799. https://doi.org/10.1016/j.oraloncology.2020.104799 .

Article   PubMed   Google Scholar  

Warnakulasuriya S, Kerr AR. Oral Cancer screening: past, Present, and Future. J Dent Res. 2021;100:1313–20. https://doi.org/10.1177/00220345211014795 .

Article   PubMed   CAS   Google Scholar  

Chamoli A, Gosavi AS, Shirwadkar UP, Wangdale KV, Behera SK, Kurrey NK et al. Overview of oral cavity squamous cell carcinoma: Risk factors, mechanisms, and diagnostics. Oral Oncol. 2021;121:105451. https://doi.org/10.1016/j.oraloncology.2021.105451 .

Rivera C. Essentials of oral cancer. Int J Clin Exp Pathol. 2015;8:11884–94.

PubMed   PubMed Central   CAS   Google Scholar  

Bagan J, Sarrion G, Jimenez Y. Oral cancer: clinical features. Oral Oncol. 2010;46:414–7. https://doi.org/10.1016/j.oraloncology.2010.03.009 .

Warnakulasuriya S. Clinical features and presentation of oral potentially malignant disorders. Oral Surg Oral Med Oral Pathol Oral Radiol. 2018;125:582–90. https://doi.org/10.1016/j.oooo.2018.03.011 .

Lousada-Fernandez F, Rapado-Gonzalez O, Lopez-Cedrun JL, Lopez-Lopez R, Muinelo-Romay L, Suarez-Cunqueiro MM. Liquid biopsy in oral Cancer. Int J Mol Sci. 2018;19:1704. https://doi.org/10.3390/ijms19061704 .

Barros O, D’Agostino VG, Lara Santos L, Vitorino R, Ferreira R. Shaping the future of oral cancer diagnosis: advances in salivary proteomics. Expert Rev Proteom. 2024;2:149–68. https://doi.org/10.1080/14789450.2024.2343585 .

Article   CAS   Google Scholar  

Khurshid Z, Zafar MS, Khan RS, Najeeb S, Slowey PD, Rehman IU. Role of salivary biomarkers in oral Cancer detection. Adv Clin Chem. 2018;86:23–70. https://doi.org/10.1016/bs.acc.2018.05.002 .

Zhou Q, Ye F, Zhou Y. Oxidative stress-related biomarkers in oral squamous cell carcinoma patients: a systematic review and meta-analysis. Biomark Med. 2023;17:337–47. https://doi.org/10.2217/bmm-2022-0846 .

Katakwar P, Metgud R, Naik S, Mittal R. Oxidative stress marker in oral cancer: a review. J Cancer Res Ther. 2016;12:438–46. https://doi.org/10.4103/0973-1482.151935 .

Ferraguti G, Terracina S, Petrella C, Greco A, Minni A, Lucarelli M, et al. Alcohol and Head and Neck Cancer: updates on the role of oxidative stress, genetic, epigenetics, oral microbiota, antioxidants, and Alkylating agents. Antioxid (Basel). 2022;11:145. https://doi.org/10.3390/antiox11010145 .

Martin KR, Barrett JC. Reactive oxygen species as double-edged swords in cellular processes: low-dose cell signaling versus high-dose toxicity. Hum Exp Toxicol. 2002;21:71–5. https://doi.org/10.1191/0960327102ht213oa .

Shin YJ, Vu H, Lee JH, Kim HD. Diagnostic and prognostic ability of salivary MMP-9 for oral squamous cell carcinoma: a pre-/post-surgery case and matched control study. PLoS ONE. 2021;16:e0248167. https://doi.org/10.1371/journal.pone.0248167 .

Nandakumar A, Nataraj P, James A, Krishnan R. Estimation of salivary 8-Hydroxydeoxyguanosine (8-OHdG) as a potential biomarker in assessing progression towards malignancy: a case-control study. Asian Pac J Cancer Prev. 2020;21:2325–9. https://doi.org/10.31557/APJCP.2020.21.8.2325 .

Babiuch K, Bednarczyk A, Gawlik K, Pawlica-Gosiewska D, Kęsek B, Darczuk D, et al. Evaluation of enzymatic and non-enzymatic antioxidant status and biomarkers of oxidative stress in saliva of patients with oral squamous cell carcinoma and oral leukoplakia: a pilot study. Acta Odontol Scand. 2019;77:408–18. https://doi.org/10.1080/00016357.2019.1578409 .

Kaur J, Politis C, Jacobs R. Salivary 8-hydroxy-2-deoxyguanosine, malondialdehyde, vitamin C, and vitamin E in oral pre-cancer and cancer: diagnostic value and free radical mechanism of action. Clin Oral Investig. 2016;20:315–9. https://doi.org/10.1007/s00784-015-1506-4 .

Agha-Hosseini F, Mirzaii-Dizgah I, Farmanbar N, Abdollahi M. Oxidative stress status and DNA damage in saliva of human subjects with oral lichen planus and oral squamous cell carcinoma. J Oral Pathol Med. 2012;41:736–40. https://doi.org/10.1111/j.1600-0714.2012.01172.x .

Kumar A, Pant MC, Singh HS, Khandelwal S. Determinants of oxidative stress and DNA damage (8-OhdG) in squamous cell carcinoma of head and neck. Indian J Cancer. 2012;49:309–15. https://doi.org/10.4103/0019-509X.104499 .

Bahar G, Feinmesser R, Shpitzer T, Popovtzer A, Nagler RM. Salivary analysis in oral cancer patients: DNA and protein oxidation, reactive nitrogen species, and antioxidant profile. Cancer. 2007;109:54–9. https://doi.org/10.1002/cncr.22386 .

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;29:372. https://doi.org/10.1136/bmj.n71 .

Article   Google Scholar  

Moola S, Munn Z, Tufanaru C, Aromataris E, Sears K, Sfetcu R et al. Chapter 7: Systematic reviews of etiology and risk. In: Aromataris E, Munn Z, editors. JBI Manual for Evidence Synthesis. JBI, 2020. https://synthesismanual.jbi.global

Georgaki M, Theofilou I, Pettas E, Stoufi E, Younis RH, Kolokotronis A, et al. Understanding the complex pathogenesis of oral cancer: a comprehensive review. Oral Surg Oral Med Oral Pathol Oral Radiol. 2021;5:566–79. https://doi.org/10.1016/j.oooo.2021.04.004 .

Bashir A, Khan ZA, Maqsood A, Prabhu N, Saleem MM, Alzarea BK, et al. The evaluation of clinical signs and symptoms of malignant tumors involving the Maxillary Sinus: recommendation of an examination sieve and risk alarm score. Healthc (Basel). 2023;11:194. https://doi.org/10.3390/healthcare11020194 .

Klaunig JE. Oxidative stress and Cancer. Curr Pharm Des. 2018;24:4771–8. https://doi.org/10.2174/1381612825666190215121712 .

Lin Y, Jiang M, Chen W, Zhao T, Wei Y. Cancer and ER stress: mutual crosstalk between autophagy, oxidative stress and inflammatory response. Biomed Pharmacother. 2019;118:109249. https://doi.org/10.1016/j.biopha.2019.109249 .

Senghore T, Li YF, Sung FC, Tsai MH, Hua CH, Liu CS, et al. Biomarkers of oxidative stress Associated with the risk of potentially malignant oral disorders. Anticancer Res. 2018;38:5211–6. https://doi.org/10.21873/anticanres.12844 .

Prieto-Correa JR, Bologna-Molina R, González-González R, Molina-Frechero N, Soto-Ávila JJ, Isiordia-Espinoza M, et al. DNA oxidative damage in oral cancer: 8-hydroxy-2´-deoxyguanosine immunoexpression assessment. Med Oral Patol Oral Cir Bucal. 2023;28:e530–8. https://doi.org/10.4317/medoral.25924 .

Zhou Y, Liu Z. Saliva biomarkers in oral disease. Clin Chim Acta. 2023;548:117503. https://doi.org/10.1016/j.cca.2023.117503 .

Min H, Zhu S, Safi L, Alkourdi M, Nguyen BH, Upadhyay A, et al. Salivary Diagnostics in Pediatrics and the Status of Saliva-based biosensors. Biosens (Basel). 2023;13:206. https://doi.org/10.3390/bios13020206 .

Britze TE, Jakobsen KK, Grønhøj C, von Buchwald C. A systematic review on the role of biomarkers in liquid biopsies and saliva samples in the monitoring of salivary gland cancer. Acta Otolaryngol. 2023;143:709–13. https://doi.org/10.1080/00016489.2023.2238757 .

Khijmatgar S, Yong J, Rübsamen N, Lorusso F, Rai P, Cenzato N, et al. Salivary biomarkers for early detection of oral squamous cell carcinoma (OSCC) and head/neck squamous cell carcinoma (HNSCC): a systematic review and network meta-analysis. Jpn Dent Sci Rev. 2024;60:32–9. https://doi.org/10.1016/j.jdsr.2023.10.003 .

Krishna Prasad RB, Sharma A, Babu HM. An insight into salivary markers in oral cancer. Dent Res J (Isfahan). 2013;10:287–95.

PubMed   Google Scholar  

Nagler RM. Saliva as a tool for oral cancer diagnosis and prognosis. Oral Oncol. 2009;45:1006–10. https://doi.org/10.1016/j.oraloncology.2009.07.005 .

Kumar M, Nanavati R, Modi TG, Dobariya C. Oral cancer: etiology and risk factors: a review. J Cancer Res Ther. 2016;12:458–63. https://doi.org/10.4103/0973-1482.186696 .

Shah SU, Nigar S, Yousofi R, Maqsood A, Altamash S, Lal A, et al. Comparison of triamcinolone with pentoxifylline and vitamin E efficacy in the treatment of stage 2 and 3 oral submucous fibrosis: a randomized clinical trial. SAGE Open Med. 2023;11:20503121231200757. https://doi.org/10.1177/20503121231200757 .

Download references

Acknowledgements

No external funding was received.

Author information

Authors and affiliations.

Biomedical Science, Faculty of Chemical-Biological Sciences, Autonomous University of Guerrero, Chilpancingo de los Bravo, Guerrero, 39090, Mexico

Mario Alberto Alarcón-Sánchez

Department of Oral Medicine and Pathology, Postgraduate Division, Dental School, National Autonomous University of Mexico, Mexico City, 04510, Mexico

Lilibeth-Stephania Escoto-Vasquez

Department of Research Analytics, Saveetha Institute of Medical and Technical Sciences, Saveetha Dental College and Hospitals, Saveetha University, Chennai, 600 077, India

Artak Heboyan

Department of Prosthodontics, Faculty of Stomatology, Yerevan State Medical University after Mkhitar Heratsi, Str. Koryun 2, Yerevan, 0025, Armenia

Department of Prosthodontics, School of Dentistry, Tehran University of Medical Sciences, North Karegar St, Tehran, Iran

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization, M.A.A.-S.; methodology, M.A.A.-S.; software, M.A.A.-S.; validation, M.A.A.-S, and M.N.-V.; formal analysis, M.A.A.-S, A.H. and M.N.-V.; investigation, M.A.A.-S.; resources, M.A.A.-S.; data curation, M.A.A.-S.; writing—original draft preparation, M.A.A.-S.; writing—review and editing, M.A.A.-S, L.S.E.-V and A.H.; visualization, M.A.A.-S, L.S.E.-V and A.H.; supervision, M.A.A.-S, and A.H.; project administration, M.A.A.-S and A.H. All authors have read and agreed to the published version of the manuscript.

Corresponding authors

Correspondence to Mario Alberto Alarcón-Sánchez or Artak Heboyan .

Ethics declarations

Ethical approval.

Not applicable.

Informed consent

Consent for publication, competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Alarcón-Sánchez, M.A., Escoto-Vasquez, LS. & Heboyan, A. Salivary 8-hydroxy-2′-deoxyguanosine levels in patients with oral cancer: a systematic review and meta-analysis. BMC Cancer 24 , 960 (2024). https://doi.org/10.1186/s12885-024-12746-0

Download citation

Received : 08 June 2024

Accepted : 01 August 2024

Published : 06 August 2024

DOI : https://doi.org/10.1186/s12885-024-12746-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• 8-hydroxy-2′-deoxyguanosine
• Squamous cell carcinoma of head and neck

ISSN: 1471-2407

• Submission enquiries: [email protected]
• General enquiries: [email protected]