a scoping literature review

• Open access
• Published: 19 November 2018

Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach

• Zachary Munn   ORCID: orcid.org/0000-0002-7091-5842 1 ,
• Micah D. J. Peters 1 ,
• Cindy Stern 1 ,
• Catalin Tufanaru 1 ,
• Alexa McArthur 1 &
• Edoardo Aromataris 1  

BMC Medical Research Methodology volume  18 , Article number:  143 ( 2018 ) Cite this article

907k Accesses

4979 Citations

864 Altmetric

Metrics details

Scoping reviews are a relatively new approach to evidence synthesis and currently there exists little guidance regarding the decision to choose between a systematic review or scoping review approach when synthesising evidence. The purpose of this article is to clearly describe the differences in indications between scoping reviews and systematic reviews and to provide guidance for when a scoping review is (and is not) appropriate.

Researchers may conduct scoping reviews instead of systematic reviews where the purpose of the review is to identify knowledge gaps, scope a body of literature, clarify concepts or to investigate research conduct. While useful in their own right, scoping reviews may also be helpful precursors to systematic reviews and can be used to confirm the relevance of inclusion criteria and potential questions.

Conclusions

Scoping reviews are a useful tool in the ever increasing arsenal of evidence synthesis approaches. Although conducted for different purposes compared to systematic reviews, scoping reviews still require rigorous and transparent methods in their conduct to ensure that the results are trustworthy. Our hope is that with clear guidance available regarding whether to conduct a scoping review or a systematic review, there will be less scoping reviews being performed for inappropriate indications better served by a systematic review, and vice-versa.

Peer Review reports

Systematic reviews in healthcare began to appear in publication in the 1970s and 1980s [ 1 , 2 ]. With the emergence of groups such as Cochrane and the Joanna Briggs Institute (JBI) in the 1990s [ 3 ], reviews have exploded in popularity both in terms of the number conducted [ 1 ], and their uptake to inform policy and practice. Today, systematic reviews are conducted for a wide range of purposes across diverse fields of inquiry, different evidence types and for different questions [ 4 ]. More recently, the field of evidence synthesis has seen the emergence of scoping reviews, which are similar to systematic reviews in that they follow a structured process, however they are performed for different reasons and have some key methodological differences [ 5 , 6 , 7 , 8 ]. Scoping reviews are now seen as a valid approach in those circumstances where systematic reviews are unable to meet the necessary objectives or requirements of knowledge users. There now exists clear guidance regarding the definition of scoping reviews, how to conduct scoping reviews and the steps involved in the scoping review process [ 6 , 8 ]. However, the guidance regarding the key indications or reasons why reviewers may choose to follow a scoping review approach is not as straightforward, with scoping reviews often conducted for purposes that do not align with the original indications as proposed by Arksey and O’Malley [ 5 , 6 , 7 , 8 , 9 , 10 ]. As editors and peer reviewers for various journals we have noticed that there is inconsistency and confusion regarding the indications for scoping reviews and a lack of clarity for authors regarding when a scoping review should be performed as opposed to a systematic review. The purpose of this article is to provide practical guidance for reviewers on when to perform a systematic review or a scoping review, supported with some key examples.

Indications for systematic reviews

Systematic reviews can be broadly defined as a type of research synthesis that are conducted by review groups with specialized skills, who set out to identify and retrieve international evidence that is relevant to a particular question or questions and to appraise and synthesize the results of this search to inform practice, policy and in some cases, further research [ 11 , 12 , 13 ]. According to the Cochrane handbook, a systematic review ‘uses explicit, systematic methods that are selected with a view to minimizing bias, thus providing more reliable findings from which conclusions can be drawn and decisions made.’ [ 14 ] Systematic reviews follow a structured and pre-defined process that requires rigorous methods to ensure that the results are both reliable and meaningful to end users. These reviews may be considered the pillar of evidence-based healthcare [ 15 ] and are widely used to inform the development of trustworthy clinical guidelines [ 11 , 16 , 17 ].

A systematic review may be undertaken to confirm or refute whether or not current practice is based on relevant evidence, to establish the quality of that evidence, and to address any uncertainty or variation in practice that may be occurring. Such variations in practice may be due to conflicting evidence and undertaking a systematic review should (hopefully) resolve such conflicts. Conducting a systematic review may also identify gaps, deficiencies, and trends in the current evidence and can help underpin and inform future research in the area. Systematic reviews can be used to produce statements to guide clinical decision-making, the delivery of care, as well as policy development [ 12 ]. Broadly, indications for systematic reviews are as follows [ 4 ]:

Uncover the international evidence

Confirm current practice/ address any variation/ identify new practices

Identify and inform areas for future research

Identify and investigate conflicting results

Produce statements to guide decision-making

Despite the utility of systematic reviews to address the above indications, there are cases where systematic reviews are unable to meet the necessary objectives or requirements of knowledge users or where a methodologically robust and structured preliminary searching and scoping activity may be useful to inform the conduct of the systematic reviews. As such, scoping reviews (which are also sometimes called scoping exercises/scoping studies) [ 8 ] have emerged as a valid approach with rather different indications to those for systematic reviews. It is important to note here that other approaches to evidence synthesis have also emerged, including realist reviews, mixed methods reviews, concept analyses and others [ 4 , 18 , 19 , 20 ]. This article focuses specifically on the choice between a systematic review or scoping review approach.

Indications for scoping reviews

True to their name, scoping reviews are an ideal tool to determine the scope or coverage of a body of literature on a given topic and give clear indication of the volume of literature and studies available as well as an overview (broad or detailed) of its focus. Scoping reviews are useful for examining emerging evidence when it is still unclear what other, more specific questions can be posed and valuably addressed by a more precise systematic review [ 21 ]. They can report on the types of evidence that address and inform practice in the field and the way the research has been conducted.

The general purpose for conducting scoping reviews is to identify and map the available evidence [ 5 , 22 ]. Arskey and O’Malley, authors of the seminal paper describing a framework for scoping reviews, provided four specific reasons why a scoping review may be conducted [ 5 , 6 , 7 , 22 ]. Soon after, Levac, Colquhoun and O’Brien further clarified and extended this original framework [ 7 ]. These authors acknowledged that at the time, there was no universally recognized definition of scoping reviews nor a commonly acknowledged purpose or indication for conducting them. In 2015, a methodological working group of the JBI produced formal guidance for conducting scoping reviews [ 6 ]. However, we have not previously addressed and expanded upon the indications for scoping reviews. Below, we build upon previously described indications and suggest the following purposes for conducting a scoping review:

To identify the types of available evidence in a given field

To clarify key concepts/ definitions in the literature

To examine how research is conducted on a certain topic or field

To identify key characteristics or factors related to a concept

As a precursor to a systematic review.

To identify and analyse knowledge gaps

Deciding between a systematic review and a scoping review approach

Authors deciding between the systematic review or scoping review approach should carefully consider the indications discussed above for each synthesis type and determine exactly what question they are asking and what purpose they are trying to achieve with their review. We propose that the most important consideration is whether or not the authors wish to use the results of their review to answer a clinically meaningful question or provide evidence to inform practice. If the authors have a question addressing the feasibility, appropriateness, meaningfulness or effectiveness of a certain treatment or practice, then a systematic review is likely the most valid approach [ 11 , 23 ]. However, authors do not always wish to ask such single or precise questions, and may be more interested in the identification of certain characteristics/concepts in papers or studies, and in the mapping, reporting or discussion of these characteristics/concepts. In these cases, a scoping review is the better choice.

As scoping reviews do not aim to produce a critically appraised and synthesised result/answer to a particular question, and rather aim to provide an overview or map of the evidence. Due to this, an assessment of methodological limitations or risk of bias of the evidence included within a scoping review is generally not performed (unless there is a specific requirement due to the nature of the scoping review aim) [ 6 ]. Given this assessment of bias is not conducted, the implications for practice (from a clinical or policy making point of view) that arise from a scoping review are quite different compared to those of a systematic review. In some cases, there may be no need or impetus to make implications for practice and if there is a need to do so, these implications may be significantly limited in terms of providing concrete guidance from a clinical or policy making point of view. Conversely, when we compare this to systematic reviews, the provision of implications for practice is a key feature of systematic reviews and is recommended in reporting guidelines for systematic reviews [ 13 ].

Exemplars for different scoping review indications

In the following section, we elaborate on each of the indications listed for scoping reviews and provide a number of examples for authors considering a scoping review approach.

Scoping reviews that seek to identify the types of evidence in a given field share similarities with evidence mapping activities as explained by Bragge and colleagues in a paper on conducting scoping research in broad topic areas [ 24 ]. Chambers and colleagues [ 25 ] conducted a scoping review in order to identify current knowledge translation resources (and any evaluations of them) that use, adapt and present findings from systematic reviews to suit the needs of policy makers. Following a comprehensive search across a range of databases, organizational websites and conference abstract repositories based upon predetermined inclusion criteria, the authors identified 20 knowledge translation resources which they classified into three different types (overviews, summaries and policy briefs) as well as seven published and unpublished evaluations. The authors concluded that evidence synthesists produce a range of resources to assist policy makers to transfer and utilize the findings of systematic reviews and that focussed summaries are the most common. Similarly, a scoping review was conducted by Challen and colleagues [ 26 ] in order to determine the types of available evidence identifying the source and quality of publications and grey literature for emergency planning. A comprehensive set of databases and websites were investigated and 1603 relevant sources of evidence were identified mainly addressing emergency planning and response with fewer sources concerned with hazard analysis, mitigation and capability assessment. Based on the results of the review, the authors concluded that while there is a large body of evidence in the field, issues with its generalizability and validity are as yet largely unknown and that the exact type and form of evidence that would be valuable to knowledge users in the field is not yet understood.

To clarify key concepts/definitions in the literature

Scoping reviews are often performed to examine and clarify definitions that are used in the literature. A scoping review by Schaink and colleagues 27 was performed to investigate how the notion of “patient complexity” had been defined, classified, and understood in the existing literature. A systematic search of healthcare databases was conducted. Articles were assessed to determine whether they met the inclusion criteria and the findings of included articles were grouped into five health dimensions. An overview of how complexity has been described was presented, including the varying definitions and interpretations of the term. The results of the scoping review enabled the authors to then develop a complexity framework or model to assist in defining and understanding patient complexity [ 27 ].

Hines et al. [ 28 ] provide a further example where a scoping review has been conducted to define a concept, in this case the condition bronchopulmonary dysplasia. The authors revealed significant variation in how the condition was defined across the literature, prompting the authors to call for a ‘comprehensive and evidence-based definition’. [ 28 ]

To examine how research is conducted on a certain topic

Scoping reviews can be useful tools to investigate the design and conduct of research on a particular topic. A scoping review by Callary and colleagues 29 investigated the methodological design of studies assessing wear of a certain type of hip replacement (highly crosslinked polyethylene acetabular components) [ 29 ]. The aim of the scoping review was to survey the literature to determine how data pertinent to the measurement of hip replacement wear had been reported in primary studies and whether the methods were similar enough to allow for comparison across studies. The scoping review revealed that the methods to assess wear (radiostereometric analysis) varied significantly with many different approaches being employed amongst the investigators. The results of the scoping review led to the authors recommending enhanced standardization in measurements and methods for future research in this field [ 29 ].

There are other examples of scoping reviews investigating research methodology, with perhaps the most pertinent examples being two recent scoping reviews of scoping review methods [ 9 , 10 ]. Both of these scoping reviews investigated how scoping reviews had been reported and conducted, with both advocating for a need for clear guidance to improve standardization of methods [ 9 , 10 ]. Similarly, a scoping review investigating methodology was conducted by Tricco and colleagues 30 on rapid review methods that have been evaluated, compared, used or described in the literature. A variety of rapid review approaches were identified with many instances of poor reporting identified. The authors called for prospective studies to compare results presented by rapid reviews versus systematic reviews.

Scoping reviews can be conducted to identify and examine characteristics or factors related to a particular concept. Harfield and colleagues (2015) conducted a scoping review to identify the characteristics of indigenous primary healthcare service delivery models [ 30 , 31 , 32 ]. A systematic search was conducted, followed by screening and study selection. Once relevant studies had been identified, a process of data extraction commenced to extract characteristics referred to in the included papers. Over 1000 findings were eventually grouped into eight key factors (accessible health services, community participation, culturally appropriate and skilled workforce, culture, continuous quality improvement, flexible approaches to care, holistic health care, self-determination and empowerment). The results of this scoping review have been able to inform a best practice model for indigenous primary healthcare services.

Scoping reviews conducted as precursors to systematic reviews may enable authors to identify the nature of a broad field of evidence so that ensuing reviews can be assured of locating adequate numbers of relevant studies for inclusion. They also enable the relevant outcomes and target group or population for example for a particular intervention to be identified. This can have particular practical benefits for review teams undertaking reviews on less familiar topics and can assist the team to avoid undertaking an “empty” review [ 33 ]. Scoping reviews of this kind may help reviewers to develop and confirm their a priori inclusion criteria and ensure that the questions to be posed by their subsequent systematic review are able to be answered by available, relevant evidence. In this way, systematic reviews are able to be underpinned by a preliminary and evidence-based scoping stage.

A scoping review commissioned by the United Kingdom Department for International Development was undertaken to determine the scope and nature of literature on people’s experiences of microfinance. The results of this scoping review were used to inform the development of targeted systematic review questions that focussed upon areas of particular interest [ 34 ].

In their recent scoping review on the conduct and reporting of scoping reviews, Tricco and colleagues 10 reveal only 12% of scoping reviews contained recommendations for the development of ensuing systematic reviews, suggesting that the majority of scoping review authors do not conduct scoping reviews as a precursor to future systematic reviews.

To identify and analyze gaps in the knowledge base

Scoping reviews are rarely solely conducted to simply identify and analyze gaps present in a given knowledge base, as examination and presentation of what hasn’t been investigated or reported generally requires exhaustive examination of all of what is available. In any case, because scoping reviews tend to be a useful approach for reviewing evidence rapidly in emerging fields or topics, identification and analysis of knowledge gaps is a common and valuable indication for conducting a scoping review. A scoping review was recently conducted to review current research and identify knowledge gaps on the topic of “occupational balance”, or the balance of work, rest, sleep, and play [ 35 ]. Following a systematic search across a range of relevant databases, included studies were selected and in line with predetermined inclusion criteria, were described and mapped to provide both an overall picture of the current state of the evidence in the field and to identify and highlight knowledge gaps in the area. The results of the scoping review allowed the authors to illustrate several research ‘gaps’, including the absence of studies conducted outside of western societies, the lack of knowledge around peoples’ levels of occupational balance, as well as a dearth of evidence regarding how occupational balance may be enhanced. As with other scoping reviews focussed upon identifying and analyzing knowledge gaps, results such as these allow for the identification of future research initiatives.

Scoping reviews are now seen as a valid review approach for certain indications. A key difference between scoping reviews and systematic reviews is that in terms of a review question, a scoping review will have a broader “scope” than traditional systematic reviews with correspondingly more expansive inclusion criteria. In addition, scoping reviews differ from systematic reviews in their overriding purpose. We have previously recommended the use of the PCC mnemonic (Population, Concept and Context) to guide question development [ 36 ]. The importance of clearly defining the key questions and objectives of a scoping review has been discussed previously by one of the authors, as a lack of clarity can result in difficulties encountered later on in the review process [ 36 ].

Considering their differences from systematic reviews, scoping reviews should still not be confused with traditional literature reviews. Traditional literature reviews have been used as a means to summarise various publications or research on a particular topic for many years. In these traditional reviews, authors examine research reports in addition to conceptual or theoretical literature that focuses on the history, importance, and collective thinking around a topic, issue or concept. These types of reviews can be considered subjective, due to their substantial reliance on the author’s pre-exiting knowledge and experience and as they do not normally present an unbiased, exhaustive and systematic summary of a topic [ 12 ]. Regardless of some of these limitations, traditional literature reviews may still have some use in terms of providing an overview of a topic or issue. Scoping reviews provide a useful alternative to literature reviews when clarification around a concept or theory is required. If traditional literature reviews are contrasted with scoping reviews, the latter [ 6 ]:

Are informed by an a priori protocol

Are systematic and often include exhaustive searching for information

Aim to be transparent and reproducible

Include steps to reduce error and increase reliability (such as the inclusion of multiple reviewers)

Ensure data is extracted and presented in a structured way

Another approach to evidence synthesis that has emerged recently is the production of evidence maps [ 37 ]. The purpose of these evidence maps is similar to scoping reviews to identify and analyse gaps in the knowledge base [ 37 , 38 ]. In fact, most evidence mapping articles cite seminal scoping review guidance for their methods [ 38 ]. The two approaches therefore have many similarities, with perhaps the most prominent difference being the production of a visual database or schematic (i.e. map) which assists the user in interpreting where evidence exists and where there are gaps [ 38 ]. As Miake-Lye states, at this stage ‘it is difficult to determine where one method ends and the other begins.’ [ 38 ] Both approaches may be valid when the indication is for determining the extent of evidence on a particular topic, particularly when highlighting gaps in the research.

A further popular method to define and scope concepts, particularly in nursing, is through the conduct of a concept analysis [ 39 , 40 , 41 , 42 ]. Formal concept analysis is ‘a process whereby concepts are logically and systematically investigated to form clear and rigorously constructed conceptual definitions,’ [ 42 ] which is similar to scoping reviews where the indication is to clarify concepts in the literature. There is limited methodological guidance on how to conduct a concept analysis and recently they have been critiqued for having no impact on practice [ 39 ]. In our opinion, scoping reviews (where the purpose is to systematically investigate a concept in the literature) offer a methodologically rigorous alternative to concept analysis with their results perhaps being more useful to inform practice.

Comparing and contrasting the characteristics of traditional literature reviews, scoping reviews and systematic reviews may help clarify the true essence of these different types of reviews (see Table 1 ).

Rapid reviews are another emerging type of evidence synthesis and a substantial amount of literature have addressed these types of reviews [ 43 , 44 , 45 , 46 , 47 ]. There are various definitions for rapid reviews, and for simplification purposes, we define these review types as ‘systematic reviews with shortcuts.’ In this paper, we have not discussed the choice between a rapid or systematic review approach as we are of the opinion that perhaps the major consideration for conducting a rapid review (as compared to a systematic or scoping review) is not the purpose/question itself, but the feasibility of conducting a full review given financial/resource limitations and time pressures. As such, a rapid review could potentially be conducted for any of the indications listed above for the scoping or systematic review, whilst shortening or skipping entirely some steps in the standard systematic or scoping review process.

There is some overlap across the six listed purposes for conducting a scoping review described in this paper. For example, it is logical to presume that if a review group were aiming to identify the types of available evidence in a field they would also be interested in identifying and analysing gaps in the knowledge base. Other combinations of purposes for scoping reviews would also make sense for certain questions/aims. However, we have chosen to list them as discrete reasons in this paper in an effort to provide some much needed clarity on the appropriate purposes for conducting scoping reviews. As such, scoping review authors should not interpret our list of indications as a discrete list where only one purpose can be identified.

It is important to mention some potential abuses of scoping reviews. Reviewers may conduct a scoping review as an alternative to a systematic review in order to avoid the critical appraisal stage of the review and expedite the process, thinking that a scoping review may be easier than a systematic review to conduct. Other reviewers may conduct a scoping review in order to ‘map’ the literature when there is no obvious need for ‘mapping’ in this particular subject area. Others may conduct a scoping review with very broad questions as an alternative to investing the time and effort required to craft the necessary specific questions required for undertaking a systematic review. In these cases, scoping reviews are not appropriate and authors should refer to our guidance regarding whether they should be conducting a systematic review instead.

This article provides some clarification on when to conduct a scoping review as compared to a systematic review and clear guidance on the purposes for conducting a scoping review. We hope that this paper will provide a useful addition to this evolving methodology and encourage others to review, modify and build upon these indications as the approach matures. Further work in scoping review methods is required, with perhaps the most important advancement being the recent development of an extension to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) for scoping reviews [ 48 ] and the development of software and training programs to support these reviews [ 49 , 50 ]. As the methodology advances, guidance for scoping reviews (such as that included in the Joanna Briggs Institute Reviewer’s Manual) will require revision, refining and updating.

Scoping reviews are a useful tool in the ever increasing arsenal of evidence synthesis approaches. Researchers may preference the conduct of a scoping review over a systematic review where the purpose of the review is to identify knowledge gaps, scope a body of literature, clarify concepts, investigate research conduct, or to inform a systematic review. Although conducted for different purposes compared to systematic reviews, scoping reviews still require rigorous and transparent methods in their conduct to ensure that the results are trustworthy. Our hope is that with clear guidance available regarding whether to conduct a scoping review or a systematic review, there will be less scoping reviews being performed for inappropriate indications better served by a systematic review, and vice-versa.

Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7(9):e1000326.

Article   Google Scholar  

Chalmers I, Hedges LV, Cooper H. A brief history of research synthesis. Eval Health Prof. 2002;25(1):12–37.

Jordan Z, Munn Z, Aromataris E, Lockwood C. Now that we're here, where are we? The JBI approach to evidence-based healthcare 20 years on. Int J Evid Based Healthc. 2015;13(3):117–20.

Munn Z, Stern C, Aromataris E, Lockwood C, Jordan Z. What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences. BMC Med Res Methodol. 2018;18(1):5.

Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.

Peters MD, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. Int J Evid Based Healthc. 2015;13(3):141–6.

Levac D, Colquhoun H, O'Brien KK. Scoping studies: advancing the methodology. Implement Sci. 2010;5(1):1.

Colquhoun HL, Levac D, O'Brien KK, et al. Scoping reviews: time for clarity in definition, methods, and reporting. J Clin Epidemiol. 2014;67(12):1291–4.

Pham MT, Rajić A, Greig JD, Sargeant JM, Papadopoulos A, McEwen SA. A scoping review of scoping reviews: advancing the approach and enhancing the consistency. Res Synth Methods. 2014;5(4):371–85.

Tricco AC, Lillie E, Zarin W, et al. A scoping review on the conduct and reporting of scoping reviews. BMC Med Res Methodol. 2016;16:15.

Pearson A. Balancing the evidence: incorporating the synthesis of qualitative data into systematic reviews. JBI Reports. 2004;2:45–64.

Aromataris E, Pearson A. The systematic review: an overview. AJN The American Journal of Nursing. 2014;114(3):53–8.

Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ (Clinical research ed). 2009;339:b2700.

Higgins J, Green S, eds. Cochrane handbook for systematic reviews of interventions. Version 5.1.0 [updated March 2011]. ed: The Cochrane Collaboration 2011.

Munn Z, Porritt K, Lockwood C, Aromataris E, Pearson A. Establishing confidence in the output of qualitative research synthesis: the ConQual approach. BMC Med Res Methodol. 2014;14:108.

Pearson A, Jordan Z, Munn Z. Translational science and evidence-based healthcare: a clarification and reconceptualization of how knowledge is generated and used in healthcare. Nursing research and practice. 2012;2012:792519.

Steinberg E, Greenfield S, Mancher M, Wolman DM, Graham R. Clinical practice guidelines we can trust. Institute of Medicine. Washington, DC: National Academies Press; 2011.

Gough D, Thomas J, Oliver S. Clarifying differences between review designs and methods. Systematic Reviews. 2012;1:28.

Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Inf Libr J. 2009;26(2):91–108.

Tricco AC, Tetzlaff J, Moher D. The art and science of knowledge synthesis. J Clin Epidemiol. 2011;64(1):11–20.

Armstrong R, Hall BJ, Doyle J, Waters E. ‘Scoping the scope’ of a cochrane review. J Public Health. 2011;33(1):147–50.

Anderson S, Allen P, Peckham S, Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. 2008;6(1):1.

Pearson A, Wiechula R, Court A, Lockwood C. The JBI model of evidence-based healthcare. International Journal of Evidence-Based Healthcare. 2005;3(8):207–15.

PubMed   Google Scholar  

Bragge P, Clavisi O, Turner T, Tavender E, Collie A, Gruen RL. The global evidence mapping initiative: scoping research in broad topic areas. BMC Med Res Methodol. 2011;11:92.

Chambers D, Wilson PM, Thompson CA, Hanbury A, Farley K, Light K. Maximizing the impact of systematic reviews in health care decision making: a systematic scoping review of knowledge-translation resources. Milbank Q. 2011;89(1):131–56.

Challen K, Lee AC, Booth A, Gardois P, Woods HB, Goodacre SW. Where is the evidence for emergency planning: a scoping review. BMC Public Health. 2012;12:542.

Schaink AK, Kuluski K, Lyons RF, et al. A scoping review and thematic classification of patient complexity: offering a unifying framework. Journal of comorbidity. 2012;2(1):1–9.

Hines D, Modi N, Lee SK, Isayama T, Sjörs G, Gagliardi L, Lehtonen L, Vento M, Kusuda S, Bassler D, Mori R. Scoping review shows wide variation in the definitions of bronchopulmonary dysplasia in preterm infants and calls for a consensus. Acta Paediatr. 2017;106(3):366–74.

Callary SA, Solomon LB, Holubowycz OT, Campbell DG, Munn Z, Howie DW. Wear of highly crosslinked polyethylene acetabular components. Acta Orthop. 2015;86(2):159–68.

Davy C, Harfield S, McArthur A, Munn Z, Brown A. Access to primary health care services for indigenous peoples: a framework synthesis. Int J Equity Health. 2016;15(1):163.

Harfield S, Davy C, Kite E, et al. Characteristics of indigenous primary health care models of service delivery: a scoping review protocol. JBI Database System Rev Implement Rep. 2015;13(11):43–51.

Harfield SG, Davy C, McArthur A, Munn Z, Brown A, Brown N. Characteristics of indigenous primary health care service delivery models: a systematic scoping review. Glob Health. 2018;14(1):12.

Peters MDJ LC, Munn Z, Moola S, Mishra RK (2015) , Protocol. Adelaide: the Joanna Briggs Institute UoA. What are people’s views and experiences of delivering and participating in microfinance interventions? A systematic review of qualitative evidence from South Asia.

Peters MDJ LC, Munn Z, Moola S, Mishra RK People’s views and experiences of participating in microfinance interventions: A systematic review of qualitative evidence. London: EPPI-Centre: social science research unit, UCL Institute of education, University College London; 2016.

Wagman P, Håkansson C, Jonsson H. Occupational balance: a scoping review of current research and identified knowledge gaps. J Occup Sci. 2015;22(2):160–9.

Peters MD. In no uncertain terms: the importance of a defined objective in scoping reviews. JBI Database System Rev Implement Rep. 2016;14(2):1–4.

Hetrick SE, Parker AG, Callahan P, Purcell R. Evidence mapping: illustrating an emerging methodology to improve evidence-based practice in youth mental health. J Eval Clin Pract. 2010;16(6):1025–30.

Miake-Lye IM, Hempel S, Shanman R, Shekelle PG. What is an evidence map? A systematic review of published evidence maps and their definitions, methods, and products. Systematic reviews. 2016;5(1):1.

Draper P. A critique of concept analysis. J Adv Nurs. 2014;70(6):1207–8.

Gibson CH. A concept analysis of empowerment. J Adv Nurs. 1991;16(3):354–61.

Article   CAS   Google Scholar  

Meeberg GA. Quality of life: a concept analysis. J Adv Nurs. 1993;18(1):32–8.

Ream E, Richardson A. Fatigue: a concept analysis. Int J Nurs Stud. 1996;33(5):519–29.

Tricco AC, Antony J, Zarin W, et al. A scoping review of rapid review methods. BMC Med. 2015;13:224.

Ganann R, Ciliska D, Thomas H. Expediting systematic reviews: methods and implications of rapid reviews. Implement Sci. 2010;5:56.

Harker J, Kleijnen J. What is a rapid review? A methodological exploration of rapid reviews in health technology assessments. Int J Evid Based Healthc. 2012;10(4):397–410.

Khangura S, Konnyu K, Cushman R, Grimshaw J, Moher D. Evidence summaries: the evolution of a rapid review approach. Syst Rev. 2012;1:10.

Munn Z, Lockwood C, Moola S. The development and use of evidence summaries for point of care information systems: a streamlined rapid review approach. Worldviews Evid-Based Nurs. 2015;12(3):131–8.

Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

Munn Z, Aromataris E, Tufanaru C, Stern C, Porritt K, Farrow J, Lockwood C, Stephenson M, Moola S, Lizarondo L, McArthur A. The development of software to support multiple systematic review types: the Joanna Briggs institute system for the unified management, assessment and review of information (JBI SUMARI). Int J Evid Based Healthc. 2018. (in press)

Stern C, Munn Z, Porritt K, et al. An international educational training course for conducting systematic reviews in health care: the Joanna Briggs Institute's comprehensive systematic review training program. Worldviews Evid-Based Nurs. 2018;15(5):401–8.

Download references

Acknowledgements

No funding was provided for this paper.

Availability of data and materials

Not applicable.

Author information

Authors and affiliations.

The Joanna Briggs Institute, The University of Adelaide, 55 King William Road, North Adelaide, 5005, South Australia

Zachary Munn, Micah D. J. Peters, Cindy Stern, Catalin Tufanaru, Alexa McArthur & Edoardo Aromataris

You can also search for this author in PubMed   Google Scholar

Contributions

ZM: Led the development of this paper and conceptualised the idea for a paper on indications for scoping reviews. Provided final approval for submission. MP: Contributed conceptually to the paper and wrote sections of the paper. Provided final approval for submission. CS: Contributed conceptually to the paper and wrote sections of the paper. Provided final approval for submission. CT: Contributed conceptually to the paper and wrote sections of the paper. Provided final approval for submission. AM: Contributed conceptually to the paper and reviewed and provided feedback on all drafts. Provided final approval for submission. EA: Contributed conceptually to the paper and reviewed and provided feedback on all drafts. Provided approval and encouragement for the work to proceed. Provided final approval for submission.

Corresponding author

Correspondence to Zachary Munn .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

All the authors are members of the Joanna Briggs Institute, an evidence-based healthcare research institute which provides formal guidance regarding evidence synthesis, transfer and implementation. Zachary Munn is a member of the editorial board of this journal. The authors have no other competing interests to declare.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Munn, Z., Peters, M.D.J., Stern, C. et al. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol 18 , 143 (2018). https://doi.org/10.1186/s12874-018-0611-x

Download citation

Received : 21 February 2018

Accepted : 06 November 2018

Published : 19 November 2018

DOI : https://doi.org/10.1186/s12874-018-0611-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Systematic review
• Scoping review
• Evidence-based healthcare

BMC Medical Research Methodology

ISSN: 1471-2288

• General enquiries: [email protected]

Scoping Review Guide

• Is a Scoping Review Right for Me?
• Scoping Review vs Systematic Review
• Standards & Guidelines for a Scoping Review
• An Example Scoping Review
• Step 1 - Define the Review Objectives and Question(s)
• Step 2 - Create the Search Strategy
• Step 3 - Conduct the Searches
• Step 4 - Select Studies to be Included
• Step 5 - Perform Data Extraction and Interpret Findings
• Step 6 - Present Findings
• Help with Conducting a Scoping Review

Laurel Scheinfeld

What is a Scoping Review?

Scoping reviews are a "preliminary assessment of potential size and scope of available research literature.  Scoping reviews aim to identify the nature and extent of research evidence (usually including ongoing research)." ( Grant and Booth 2009). 

Scoping Reviews are best: When a body of literature has not yet been comprehensively reviewed, or exhibits a large, complex, or heterogeneous nature not amenable to a more precise systematic review. They are used to map existing literature in terms of nature, features, and volume. Scoping reviews clarify working definitions and conceptual boundaries of a topic or field and identify gaps in existing literature/research. (Peters M, Godfrey C, Khalil H, et al) Scoping reviews may be used as the precursor to a systematic review. (Munn et al., 2018)

• Next: Is a Scoping Review Right for Me? >>
• Last Updated: Jun 26, 2024 9:48 AM
• URL: https://guides.library.stonybrook.edu/ScR
• Request a Class
• Hours & Locations
• Ask a Librarian
• Special Collections
• Library Faculty & Staff

Library Administration: 631.632.7100

• Stony Brook Home
• Campus Maps
• Web Accessibility Information
• Accessibility Barrier Report Form

Comments or Suggestions? | Library Webmaster

Except where otherwise noted, this work by SBU Libraries is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License .

Systematic Reviews: Scoping Reviews

• Systematic Review Defined
• Guides, Standards and Resources
• Data Extraction
• Risk of Bias
• WCM's Data Retention Policy & Systematic Reviews
• Scoping Reviews
• Narrative Reviews
• Covidence This link opens in a new window
• Collaborations 2022-present
• Collaborations 2012-2021

What is a Scoping Review?

Our Systematic Review Service can also assist you and your team with Scoping Reviews. 

A scoping review is a relatively new approach to evidence synthesis and differs from systematic reviews in its purpose and aims. The purpose of a scoping review is to provide an overview of the available research evidence without producing a summary answer to a guide clinical decision-making.

Scoping reviews are a form of knowledge synthesis, which incorporate a range of study designs to comprehensively summarize and synthesize evidence with the aim of informing practice, programs, and policy and providing direction to future research priorities.

The general purpose for conducting scoping reviews is to identify and map the available evidence.

Adapted from: Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005 Feb;8(1):19–32.

Sucharew H, Macaluso, M. Methods for Research Evidence Synthesis: The Scoping Review Approach. J. Hosp. Med 2019;7;416-418.

How is this different from a Systematic Review?

• Scoping reviews share a number of the same processes as systematic reviews, as they both use rigorous and transparent methods to comprehensively identify and analyze all the relevant literature pertaining to a research question.
• The key differences between the two review methods can be attributed to their differing purposes and aims. The purpose of a scoping review is to map the body of literature on a topic area . The purpose of a systematic review is to synthesize the best available research on a specific intervention . 
• Scoping reviews identify key characteristics or factors related to a concept. They do not produce statements to guide decision-making. 
• A scoping review seeks to present an overview of a potentially large and diverse body of literature pertaining to a broad topic . A systematic review attempts to collate empirical evidence from a relatively smaller number of studies pertaining to a focused research question. 
• Scoping reviews aim to provide a descriptive overview of the reviewed material without critically appraising individual studies or synthesizing evidence from different studies (no risk of bias or meta-analysis/statistical pooling is performed). In contrast, systematic reviews aim to provide a synthesis of evidence from studies assessed for risk of bias .

Adapted from: Pham MT, Rajić A, Greig JD, Sargeant JM, Papadopoulos A, McEwen SA. A scoping review of scoping reviews: advancing the approach and enhancing the consistency. Res Synth Methods. 2014 Dec;5(4):371–85.  

The PCC Question Development Framework

Because the aim of a scoping review differs from that of a systematic review, question development may not fit into the PICO (Patient/Intervention/Comparison/Outcome) framework . Therefore, PCC (Population/Concept/Context) may be a more useful framework.

Per JBI's Scoping Review Manual : "The 'PCC' mnemonic is recommended as a guide to construct a clear and meaningful title for a scoping review. The PCC mnemonic stands for the Population, Concept, and Context. There is no need for explicit outcomes, interventions or phenomena of interest to be stated for a scoping review; however elements of each of these may be implicit in the concept under examination."

When do I perform a Systematic Review? When do I perform a Scoping Review?

• When you have a specific clinical question that fits into the PICO framework or a hypothesis you are looking to test, you'll want to perform a systematic review. 
• If you are looking for a broad overview on a topic, with no hypothesis or specific clinical question, you'll want to perform a scoping review.  

Adapted from: Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018 Nov 19;18(1):143.

How are both Systematic and Scoping Reviews different from Traditional Literature Reviews?

Systematic and scoping reviews aim to be comprehensive, transparent, reproducible, and unbiased – this is not typically the case with a traditional literature review. With clear and explicit methodology, the reader knows exactly how the authors of a study came to their conclusions, rather than relying on expert opinion or subjective selection that is usually found in a literature review. 

Additional Resources

• Peters MDJ, Marnie C, Tricco AC, Pollock D, Munn Z, Alexander L, McInerney P, Godfrey CM, Khalil H. Updated methodological guidance for the conduct of scoping reviews . JBI Evid Synth. 2020 Oct;18(10):2119-2126.
• Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): checklist and explanation . Ann Intern Med. 2018 Oct 2;169(7):467–73.
• Peters MDJ, Godfrey C, McInerney P, Munn Z, Tricco AC, Khalil, H. Chapter 11: Scoping Reviews (2020 version) . In: Aromataris E, Munn Z (Editors). JBI Manual for Evidence Synthesis, JBI, 2020.
• Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach . BMC Med Res Methodol. 2018 Nov 19;18(1):143.
• Arksey H, O’Malley L. Scoping studies: towards a methodological framework . Int J Soc Res Methodol. 2005 Feb;8(1):19–32.
• Pham MT, Rajić A, Greig JD, Sargeant JM, Papadopoulos A, McEwen SA. A scoping review of scoping reviews: advancing the approach and enhancing the consistency . Res Synth Methods. 2014 Dec;5(4):371–85.
• Colquhoun HL, Levac D, O’Brien KK, Straus S, Tricco AC, Perrier L, et al. Scoping reviews: time for clarity in definition, methods, and reporting . J Clin Epidemiol. 2014 Dec;67(12):1291–4.
• Peters MDJ, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews . Int J Evid Based Healthc. 2015 Sep;13(3):141–6.
• << Previous: WCM's Data Retention Policy & Systematic Reviews
• Next: Narrative Reviews >>
• Last Updated: Sep 12, 2024 8:58 AM
• URL: https://med.cornell.libguides.com/systematicreviews

• Methodology
• Open access
• Published: 15 March 2023

Mapping reviews, scoping reviews, and evidence and gap maps (EGMs): the same but different— the “Big Picture” review family

• Fiona Campbell   ORCID: orcid.org/0000-0002-4141-8863 1 ,
• Andrea C. Tricco 2 ,
• Zachary Munn 3 ,
• Danielle Pollock 3 ,
• Ashrita Saran 4 ,
• Anthea Sutton 5 ,
• Howard White 6 &
• Hanan Khalil 7  

Systematic Reviews volume  12 , Article number:  45 ( 2023 ) Cite this article

26k Accesses

51 Citations

185 Altmetric

Metrics details

A Correction to this article was published on 01 April 2023

This article has been updated

Scoping reviews, mapping reviews, and evidence and gap maps are evidence synthesis methodologies that address broad research questions, aiming to describe a bigger picture rather than address a specific question about intervention effectiveness. They are being increasingly used to support a range of purposes including guiding research priorities and decision making. There is however a confusing array of terminology used to describe these different approaches. In this commentary, we aim to describe where there are differences in terminology and where this equates to differences in meaning. We demonstrate the different theoretical routes that underpin these differences. We suggest ways in which the approaches of scoping and mapping reviews may differ in order to guide consistency in reporting and method. We propose that mapping and scoping reviews and evidence and gap maps have similarities that unite them as a group but also have unique differences. Understanding these similarities and differences is important for informing the development of methods used to undertake and report these types of evidence synthesis.

Peer Review reports

Introduction

Evidence synthesis(defined broadly as the rigorous collation, evaluation and analysis of literature, studies, and reports) is increasingly viewed as critical to inform decision making in policy and practice. Over the past three decades, as various methods of evidence synthesis have emerged and evolved, the systems and labels used to categorize different review types have proliferated. A recent catalog of evidence synthesis approaches and terms identified 48 distinct review types [ 1 ]. Moher et al. (2015) [ 2 ], describes them as a “family” of evidence synthesis products that have arisen in response to policymakers and other stakeholders needs for diverse forms of information. This growth reflects the increased value placed on evidence synthesis to inform decision making, and we now see evidence synthesis used to address a broader range of research questions beyond effectiveness, along with tailored approaches (in terms of methods and products) to evidence synthesis as appropriate for different research needs, purposes, situations, and audiences [ 3 ].

Examples of approaches that are increasingly seen in the published literature are scoping reviews, mapping reviews, and evidence and gap maps (EGMs). Scoping reviews, mapping reviews, and EGMs are relatively new approaches that rarely appeared before 2009 [ 4 , 5 ]. Scoping reviews, evidence maps, and evidence and gap maps have been grouped together as “Big Picture” approaches due to their shared purpose and approaches. These Big Picture reviews can be contrasted with systematic reviews (addressing interventions, diagnostic test accuracy, prognosis, etc.) as they have a broader scope as compared to the (normally) narrower scope of classic systematic reviews. There have been consistent yearly increases in the publication of scoping, mapping, and evidence and gap maps [ 6 ]. Despite this, there remains confusion as to their application, meaning, and whether differences exist between them. This commentary aims to clarify these approaches, identify any differences between them, and provide recommendations for reviewers.

Terminology matters

This growing and evolving family of evidence synthesis types presents some challenges [ 7 ].

Firstly, there is the challenge of choosing the correct approach, particularly when terms are used inconsistently in the literature. The selection of an appropriate review approach will ensure the correct methods are employed using the appropriate standards for both its conduct and reporting. Indexing and wider dissemination can be challenging for researchers when there is ambiguity in terms [ 8 , 9 ].

Scoping reviews and mapping reviews—how are they used in the literature

Scoping reviews, mapping reviews, and evidence maps are terms that are not used consistently in the literature, with different terms used to describe similar approaches and review objectives. The same term is also used to describe different approaches and review objectives. Within the published literature, the terms scoping reviews and mapping reviews appear to be used in three different ways. Firstly, the terms “mapping” and “scoping” reviews are used interchangeably, referring to the same type of review methodology [ 5 , 6 , 10 ]. This approach is also one that is used in the PRISMA Extension for Scoping Reviews (PRISMA-ScR) [ 11 ], providing guidance to inform reporting standards [ 12 ]. This may therefore have been influential in increasing the use of the term scoping review over the use of the term mapping review. Examination of published reviews does not reveal differences in method between these approaches (Campbell et al., 2022 publication in press).

Secondly, we see the terms used as complementary to the other. Some definitions tend to use the terms in a way which suggest that mapping is a specific approach to scoping—or vice versa. For example, “scoping reviews can usefully map the evidence in a number of ways” [ 13 ] and “scoping reviews are a way of mapping the key concepts” [ 14 ]. Lukersmith et al. (2016) [ 15 ] and Fernadez-Sotos et al. (2019) [ 16 ] suggest that the term map is a descriptive term used to describe one of the purposes of the scoping review. A mapping review may also scope the literature. It has also been suggested that when the term mapping is included in the description of the method that the review will incorporate a geographical mapping exercise or charting of the data in a tabular or any other visual format that can plot or portray the data.

Finally, we see scoping and mapping used to describe different types of evidence synthesis, and a distinction is made between mapping and scoping reviews [ 1 , 17 ]. These authors suggest that scoping reviews are “preliminary assessment of potential size and scope of available research literature which aims to identify nature and extent of research evidence (usually including ongoing research)”. It also is a term that has emerged within the systematic review field to describe the preliminary work undertaken with information specialists in planning the review, by getting a sense of the size of the literature, to identify key terms and theories and potentially clinical experts [ 18 ]. Within these definitions, mapping reviews are distinguished from a scoping review because the subsequent outcome may involve either further review work or primary research and this outcome is not known beforehand. For the purpose of this paper, we will refer to these as a scoping exercise instead of a formal scoping review methodology. Scoping exercises within this definition would not usually be regarded as a final output in their own right, primarily because of limitations in their rigor mean that they hold the potential for bias.

Gough et al. (2012) [ 19 ] suggest that the term scoping review often describes a more rapid, and so usually non-systematic, approach to describing the nature of the literature on a topic area, sometimes as part of planning for a systematic review compared with a standard systematic review. It is also important to note that there are published rapid scoping reviews where streamlined methods are used, but transparency and rigor are maintained to produce quicker results for decision-making purposes. Examples of these types of rapid scoping reviews include rapid responses to policy questions during the COVID-19 pandemic [ 20 , 21 ].

An alternative view of the difference comes from Bragge et al. (2011) [ 22 ] who suggests that a scoping review is distinguished from mapping by the inclusion of research results in the description of relevant evidence, whereas maps simply describe what is there without collating and summarizing the results of the studies.

So, even where the types of products are seen as different, there is not a consistent approach in this difference. Nevertheless, understanding why they are considered different is important in considering what is lost, in terms of an apt descriptor, if the terms are amalgamated and used interchangeably.

Historical origins

One reason that the terms scoping and mapping have emerged to describe two similar methodological approaches addressing broad types of research questions lies in the academic traditions from which they derive and the epistemological foundations upon which these are built. Scoping reviews and scoping review methodological guidance [ 12 ] tends to cite the framework defined by Arksey and O’Malley (2005) [ 23 ] and later enhancements by Levac et al. (2010) [ 24 ]. These approaches have their roots in sociological sciences. In contrast, the term evidence mapping was first used by Katz et al. (2003) [ 25 ] and has roots in the natural sciences. This was the term adopted by the EPPI Center in an early publication of a mapping review and is the term used by the Center for Environmental Evidence for the environmental sciences. The approach to evidence mapping accompanied by a visual evidence and gap map has been developed by several agencies (see Saran and White, 2018) [ 26 ], most notably by the International Initiative for Impact Evaluation (3ie) [ 27 ] in the field of international development and subsequently adopted and adapted to a wider a range of sectors through the Campbell Collaboration. These include, for example, transport [ 28 ], youth violence, disability (Saran et al. [ 29 ]), employment (Campbell et al. [ 30 ]), and health and elder abuse [ 31 ] (Table 1 ).

Suggested approaches for distinguishing between mapping reviews and mapping reviews with EGMs and scoping reviews

The emergence of two terms (scoping and mapping) to describe approaches that have much in common in terms of their objectives and methods suggests that the terms used will be shaped more by the academic background of the researcher than by inherent differences in the approaches.

Currently, as we have shown, there are many instances where “mapping and scoping” are used interchangeably. We argue, in this paper, that while there is considerable overlap between these approaches, there is value in creating a distinction between scoping reviews, mapping reviews, and evidence gap maps. They also could be considered complementary, and a review may have elements of both “mapping” and “scoping.” Each approach, within this family of “broad approach and exploratory reviews” however has a shared objective which is to overview a wider research/topic area, rather than to address a tightly focused question. The methods thereafter diverge in part to address the nature of the research question, the research objectives, the topic area, the depth required for the data extraction, and the expertise of the review team.

We propose that a useful distinction is to see mapping, scoping, and EGMs sitting within the same family of types addressing broad questions but sitting on a spectrum in some of their underpinning epistemologies, concepts, and hence objectives (Fig. 1 ).

This is illustrated in the figure below:

The Big Picture review family (commonalities and differences in approaches)

Scoping review

These review types have been variously defined and described in the literature as described above. To address the confusion in this field, a recent formal definition of scoping reviews has been proposed, describing scoping reviews as follows:

It is a type of evidence synthesis that aims to systematically identify and map the breadth of evidence available on a particular topic, field, concept, or issue, often irrespective of source (i.e., primary research, reviews, non-empirical evidence) within or across particular contexts. Scoping reviews can clarify key concepts/definitions in the literature and identify key characteristics or factors related to a concept, including those related to methodological research [ 32 ].

They can be more exploratory than mapping reviews and EGMs, not requiring an a priori set of codes in order to describe data and may draw upon a range of sources of information (i.e., primary research, reviews, non-empirical evidence) within or across particular contexts. The approach can be more iterative, inductive, or deductive [ 32 ]. The nature of the “cataloging” and coding may be in response to what is found within the literature or using pre-defined categorization codes. Scoping reviews can also be used to identify concepts and clarify terms in the literature. In contrast to a mapping review where the process of coding is predefined. Within a scoping review, the data extracted may be textual and descriptive, allowing for example an analysis of concepts and categories using simple content analysis. It may include both predefined coding and also exploration of themes (for example, Kelly-Blake et al. 2018 [ 33 ]). In contrast, along a continuum, mapping reviews will address broader questions, use predefined coding, and adopt less in-depth data extraction.

Mapping review

Mapping reviews are also a transparent, rigorous, and systematic approach to identifying, describing, and cataloging evidence and evidence gaps in a broader topic area. They are to collate, describe, and catalog the available evidence relating to the question of interest [ 18 ]. They aim to answer the question “what do we know about a topic,” or “what and where research exists on a particular area.” A mapping review typically extracts only descriptive information about the studies and applies predefined codes (high level data). In this sense, they may be informed by an “aggregative” logic. A mapping review may or may not be accompanied by an EGM but provides visual summaries in the form of tables and graphs within the text [ 36 ]. These types of reviews may well have broader focus than a scoping review, with more limited data extracted from the included papers.

Evidence and gap maps

Evidence and gap maps are described as “a systematic presentation of all relevant evidence of a specified kind for a particular sector, sub-sector, or geography”. Evidence and gap maps (EGMs) are a systematic evidence synthesis product which displays the available evidence relevant to a specific research question. EGMs consist of primary dimensions or framework (rows and columns) and secondary dimensions or filters, enabling exploration of the map using a particular focus (e.g., looking at particular populations or study designs). It creates a visual, web-based, and interactive output [ 34 ].

This type of evidence synthesis generally uses a deductive approach with a pre-specified framework to classify the data and identify gaps in the literature. However, if no suitable framework is available, then the research team can develop their own by drawing on the range of resources, such as strategy documents, policy document, and funder reports. This is one of the major differences between mapping with an EGM review and scoping reviews (for the latter, an inductive or deductive approach may be used to identify relevant data elements so the framework for classification of the data and identification of gaps does not need to be pre-specified). Evidence gap maps may accompany a mapping review as a visual representation of the included studies or can stand independently from an accompanying mapping review.

All three of these approaches are characterized by seeking to address a broader topic area rather than a specific intervention or exposure. They are an appropriate tool if the research question is one in which multiple dimensions need to be considered, for example, multiple interventions, outcomes, or types of evidence. They do not aim to synthesize data but rather describe, categorize, and catalog findings. They aim to do so by applying defined methods to ensure transparency and rigor in the process of identifying, screening, data extraction, and interpreting findings. By addressing a broad topic area these approaches support the following purposes [ 3 ]:

Knowledge generation to support broad research questions and objectives such as the following:

What types of evidence are available in a given field?

How are concepts or definitions used within the literature?

How and where research is conducted on a certain topic?

The type of broad research question will inform the choice of approach. Scoping reviews are more likely to address open questions and the concepts may be emergent such “how is a key term used within the literature,” in contrast a mapping review may address more closed questions such as “how often the key term is used within the literature and within which population groups.” An evidence gap map will similarly address a closed question, for example, “is the term used in the following types of population group: children, adolescents, older people, and people with chronic conditions.”

Scoping reviews can provide an approach that allows exploration and clarification of key concepts and definitions within the literature, as well as how research is undertaken. As this approach does not require predefined categories, it allows for more descriptive data extraction. Often the question will be narrower than in a mapping review, allowing a greater depth of exploration of the included studies.

These approaches enable a better understanding is gained of phenomena by seeing it within a wider context. Olson et al. 2021 [ 37 ] uses the allegory of the blind monks who examine the elephant, where close inspection of one part of the whole means that meaning is lost. A complete picture is needed to really understand what the elephant is. It is clear, when seeking to operationalize what is meant by a “broad” topic area that perspective matters. For a cell biologist, the cell nucleus might be a broad topic, which a single country might be too narrow a perspective for the geographer. Understanding this unique feature of “Big Picture” reviews is perhaps easier when seen in contrast to the approach used in a systematic review examining the effectiveness of a single intervention. A Big Picture review question will look at multiple interventions or exposures and multiple outcomes or effects, seeking not to synthesize but to describe (Table 2 ).

To provide a foundation for guiding future research priorities and decisions by identifying available evidence and gaps in research

Mapping reviews and EGMs incorporate a framework that is generated during development of the protocol—it is this framework which guides the development of the data extraction tool or coding tool. This framework becomes the “map” against which existing evidence is plotted.

Identifying research gaps is often a stated part of all types of research; indeed, implications for “research and practice” are an expected part of all health and social care-related research. Identifying research gaps is often a primary purpose of scoping, mapping, and mapping reviews with EGMs more than other types of review design. In particular, mapping reviews with or without evidence gap maps address this purpose with a transparency and rigor that is unique.

Evidence and gap maps aim to enable evidence to be located, both by showing what is there but also in demonstrating knowledge gaps. In order to identify knowledge gaps, an EGM begins by developing the framework against which the evidence is plotted. The development of the framework adheres to the following principles. Firstly, it may be constructed using an existing, widely accepted international typology for either interventions, exposures, or outcomes. Secondly, if no suitable framework is available then the research team may draw on a range of resources including consultation with stakeholders and relevant published theories to ensure the comprehensiveness of the framework. Without such a structure, the gaps are not identified in a systematic way, but rather inferred and chosen by the review authors (no doubt well informed) but nevertheless influenced by their own perspectives and bias. This may be particularly apparent where a review is undertaken to pave the way for further primary research by the same team. Review teams could be strongly invested in identifying their own planned research as the “research gap.”

Evidence gap maps are a systematic approach to identifying the evidence and in particular—its gaps. No other review methodology has developed a systematic approach to identifying gaps in the evidence with this level of rigor and transparency. A limitation of the approach is that it only charts what is known and does not allow a more exploratory approach that may be employed in a scoping review.

Mapping and mapping reviews with EGMs aim to describe the state of evidence for a question or topic. The review questions may therefore be open framed and broad. However, the question can be close framed and narrow. Key elements of the question can be formulated by a framework such as PO (population, outcome). For an EGM, the objectives are formalized in the framework which defines the scope of the map [ 34 ].

To inform policy decisions, where an overview of an area may be more helpful than specific questions about specific types of interventions

Mapping (with or without an EGM) and scoping reviews often have pertinence for policy makers as they are able to cover the breadth of science often needed for policy-based questions; however, it needs to be remembered that the mapping approaches do not synthesize the findings and not include quality or risk of bias appraisal. These factors may limit their value to support some types of policy decisions. However, a mapping review with an accompanying EGM can take users to the research papers and facilitate the ready location of relevant evidence. An EGM can take users to the research papers and facilitate the ready location of relevant evidence. One example has been the use of a country evaluation map used by the Office of the Prime Minister of Uganda to identify studies to inform policy work [ 38 ]. Similarly, scoping reviews can inform policy and further research through identifying the available literature pertaining to a particular topic, along with clarifying key concepts and definitions.

As a stepping stone to building the evidence architecture

Evidence mapping and EGMs may be used as a first step towards the generation of evidence-based decision-making products, such as guidance, checklists, and online decision-making tools [ 39 ]. Maps will identify the (i) existing reviews which are suitable to use a basis for guidance, etc., (ii) where there are clusters of primary studies but no review so reviews may be commissioned in priority areas to inform guidance, etc., and (iii) important policy areas in which evidence is missing. To serve this purpose, the map should be regularly updated (maintained).

While the literature is inconsistent in its definitions of these types of reviews, and different reviews use different terminology to describe methods that appear very similar, many of these differences reflect the different research traditions and adoption of terms within organizations undertaking these types of syntheses. We argue that there is value in having these distinct terms to describe the different approaches within this family of broad review types. Scoping reviews allow a more inductive, in-depth approach with, including fewer included studies and a greater level of data extraction compared with mapping reviews. Mapping reviews and evidence gap maps address more closed questions, with pre-specified items defined and code-able when contrasted with scoping reviews. Evidence gap maps offer a visual, interactive output for users to locate evidence. The predefined framework offers a rigor to locating gaps in the existing literature and displaying these differences which is unique to these approaches.

This proposed new “Big Picture” review family within evidence synthesis contributes to the wide array of possible approaches to synthesizing literature. This multitude of choice presents challenges in selecting the correct evidence synthesis methodology. One tool that has been developed to assist in the appropriate selection of a method is the “right review” tool ( https://whatreviewisrightforyou.knowledgetranslation.net/ ). The tool enables researchers to answer a series of simple questions regarding the type of research questions they are undertaking for their review and selects an appropriate type of review based on their answers to the questions. The tool currently includes 41 different types of evidence synthesis methods [ 40 ].

A recent development has been changes made to the SR Toobox ( http://systematicreviewtools.com/index.php ) to include searching for tools to support different review types, as well as for different stages of the review. The Big Picture review family is increasingly well supported by methodological guidance and automation tools to support the process of undertaking high quality systematic reviews.

The existing guidance for the conduct and reporting of scoping reviews also applies to mapping reviews (JBI). Further development is needed in the methods of preparing a coding framework, particularly when the mapping review will also include the development of an interactive EGM. Current models of good practice exist; however, current guidance and reporting standards are limited.

This commentary details and describes some of the broad approaches within the evidence synthesis toolkit, specifically scoping reviews, mapping reviews, and EGMs. We have identified similarities and differences, based on our expert experience, between these reviews. We propose grouping them as a family of evidence synthesis to address broad research question and objectives. In so doing, we advocate that adherence to the principles of rigor and transparency that give users of evidence synthesis confidence in the reliability of the results of the review.

Change history

01 april 2023.

A Correction to this paper has been published: https://doi.org/10.1186/s13643-023-02224-2

Sutton A, Clowes M, Preston L, et al. Meeting the review family: exploring review types and associated information retrieval requirements. Health Info Libr J. 2019;36(3):202–22.

Article   PubMed   Google Scholar  

Moher D, Stewart L, Shekelle P. All in the family: systematic reviews, rapid reviews, scoping reviews, realist reviews, and more. Syst Rev. 2015;4(1):1–2.

Article   PubMed   PubMed Central   Google Scholar  

Munn Z, Stern C, Aromataris E, et al. What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences. BMC Med Res Methodol. 2018;18(1):5. https://doi.org/10.1186/s12874-017-0468-4 .

Khalil H, Peters M, Godfrey CM, et al. An evidence-based approach to scoping reviews. Worldviews on Evidence-Based Nursing. 2016;13(2):118–23.

Tricco AC, Antony J, Soobiah C, et al. Knowledge synthesis methods for integrating qualitative and quantitative data: a scoping review reveals poor operationalization of the methodological steps. J Clin Epidemiol. 2016;73:29–35.

Colquhoun HL, Levac D, O’Brien KK, et al. Scoping reviews: time for clarity in definition, methods, and reporting. J Clin Epidemiol. 2014;67(12):1291–4.

Khalil H, Tamara L, Rada G, et al. Challenges of evidence synthesis during the 2020 COVID pandemic: a scoping review. J Clin Epidemiol. 2022;142:10–8.

Khalil H, Peters MD, Tricco AC, et al. Conducting high quality scoping reviews-challenges and solutions. J Clin Epidemiol. 2021;130:156–60.

Littell JH. Conceptual and practical classification of research reviews and other evidence synthesis products. Campbell Syst Rev. 2018;14(1):1–21.

Article   Google Scholar  

Peters MD, Godfrey C, McInerney P, et al. Chapter 11: scoping reviews (2020 version). JBI manual for evidence synthesis, JBI 2020;2020

Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

Peters MD, Marnie C, Tricco AC, et al. Updated methodological guidance for the conduct of scoping reviews. JBI evidence synthesis. 2020;18(10):2119–26.

Anderson S, Allen P, Peckham S, et al. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health research policy and systems. 2008;6(1):1–12.

Peters MD, Godfrey CM, Khalil H, et al. Guidance for conducting systematic scoping reviews. JBI Evidence Implementation. 2015;13(3):141–6.

Google Scholar  

Lukersmith MS, Millington M, Salvador-Carulla L. What is case management? A scoping and mapping review. International journal of integrated care 2016;16(4)

Fernandez-Sotos P, Torio I, Fernandez-Caballero A, et al. Social cognition remediation interventions: a systematic mapping review. PLoS ONE. 2019;14(6): e0218720.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009;26(2):91–108.

James KL, Randall NP, Haddaway NR. A methodology for systematic mapping in environmental sciences. Environmental evidence. 2016;5(1):1–13.

Gough D, Thomas J, Oliver S. Clarifying differences between review designs and methods. Syst Rev. 2012;1(1):1–9.

Embrett M, Sim SM, Caldwell HA, et al. Barriers to and strategies to address COVID-19 testing hesitancy: a rapid scoping review. BMC Public Health. 2022;22(1):1–10.

Foster CR, Campbell F, Blank L, et al. A scoping review of the experience of implementing population testing for SARS-CoV-2. Public Health. 2021;198:22–9.

Article   CAS   PubMed   Google Scholar  

Bragge P, Clavisi O, Turner T, et al. The Global Evidence Mapping Initiative: scoping research in broad topic areas. BMC Med Res Methodol. 2011;11(1):92. https://doi.org/10.1186/1471-2288-11-92 .

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.

Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing the methodology. Implement Sci. 2010;5(1):1–9.

Katz DL, Williams A-l, Girard C, et al. The evidence base for complementary and alternative medicine: methods of evidence mapping with application to CAM. Alternative therapies in health and medicine 2003;9(4):22–37.

Saran A, White H. Evidence and gap maps: a comparison of different approaches. Campbell Syst Rev. 2018;14(1):1–38.

Snilstveit B, Vojtkova M, Bhavsar A, et al. Evidence gap maps-a tool for promoting evidence-informed policy and prioritizing future research. World bank policy research working paper 2013(6725)

Malhotra SK, White H, Dela Cruz NAO, et al. Studies of the effectiveness of transport sector interventions in low-and middle-income countries: an evidence and gap map. Campbell Syst Rev. 2021;17(4): e1203.

PubMed   PubMed Central   Google Scholar  

Saran A, White H, Kuper H. Evidence and gap map of studies assessing the effectiveness of interventions for people with disabilities in low‐and middle‐income countries. Campbell Systematic Reviews. 2020;16(1):e1070.

Campbell F, Chambers D, Llewellyen J. Wong R. Employment and Health: An Evidence and Gap Map. Campbell Syt Revs (in press). 2023.

Mikton C, Beaulieu M, Yon Y, et al. PROTOCOL: Global elder abuse: a megaa me of systematic reviews on prevalence, consequences, risk and protective factors and interventions. Campbell Syst Rev. 2022;18(2): e1227.

Munn Z, Pollock D, Khalil H, et al. What are scoping reviews? Providing a formal definition of scoping reviews as a type of evidence synthesis. JBI Evid Synth. 2022;20(4):950–2. https://doi.org/10.11124/jbies-21-00483[publishedOnlineFirst:2022/03/08] .

Kelly-Blake K, Garrison NA, Fletcher FE, et al. Rationales for expanding minority physician representation in the workforce: a scoping review. Med Educ. 2018;52(9):925–35.

White H, Albers B, Gaarder M, et al. Guidance for producing a Campbell evidence and gap map. Campbell Syst Rev. 2020;16(4): e1125.

Gough D, Oliver S, Thomas J. An introduction to systematic reviews: Sage 2017.

Miake-Lye IM, Hempel S, Shanman R, et al. What is an evidence map? A systematic review of published evidence maps and their definitions, methods, and products. Syst Rev. 2016;5(1):28. https://doi.org/10.1186/s13643-016-0204-x .

Olson AW, Stratton TP, Isetts BJ, et al. Seeing the elephant: a systematic scoping review and comparison of patient-centeredness conceptualizations from three seminal perspectives. J Multidiscip Healthc. 2021;14:973.

White H, Lubanga T, Rathinam F, et al. Development evaluations in Uganda 2000–2018. 2021

White H. The strategic use of evidence and gap maps to build the evidence architecture. In: Working CM, ed. London and Oxford: Centre of Excellence for Development Impact and Learning (CEDIL), 2021.

Amog K, Courvoisier M, Mak M, et al. The web-based “right review” tool asks reviewers simple questions to suggest methods from 41 knowledge synthesis methods. J Clin Epidemiol. 2022;147:42–51.

Davis R, Campbell R, Hildon Z, et al. Theories of behaviour and behaviour change across the social and behavioural sciences: a scoping review. Health Psychol Rev. 2015;9(3):323–44.

Pollock A, Campbell P, Struthers C, et al. Stakeholder involvement in systematic reviews: a scoping review. Syst Rev. 2018;7(1):1–26.

Baxter DG, Hilbrecht M, Wheaton CT. A mapping review of research on gambling harm in three regulatory environments. Harm Reduct J. 2019;16(1):1–19.

Download references

Acknowledgements

Andrea Tricco is funded by the Tier 2 Canada Research Chair in Knowledge Synthesis.

Zachary Munn is funded by an NHMRC Investigator grant APP1195676.

Author information

Authors and affiliations.

Population Health Sciences Institute, Newcastle University, Newcastle, UK

Fiona Campbell

Knowledge Translation Program of the Li Ka Shing Knowledge Institute, St. Michael’s Hospital, University of Toronto in the Dalla Lana School of Public Health & Institute of Health Policy, Management, and Evaluation, Toronto, Canada

Andrea C. Tricco

JBI, Faculty of Health and Medical Sciences, University of Adelaide, Adelaide, Australia

Zachary Munn & Danielle Pollock

International Development Coordinating Group, Campbell Collaboration, Oslo, Norway

Ashrita Saran

ScHARR, University of Sheffield, Sheffield, United Kingdom

Anthea Sutton

Evaluation and Evidence Synthesis, Global Development Network, New Delhi, India

Howard White

School of Psychology and Public Health, Department of Public Health, La Trobe University, Melbourne, Australia

Hanan Khalil

You can also search for this author in PubMed   Google Scholar

Contributions

FC and HW conceived the idea for the paper, and FC initiated the initial draft. HK, AT, HW, ZM, AS, and AS contributed to the comments and edits of subsequent drafts of the paper. HK and FC are co-chairs in the NAVIGATOR method group and took a lead in the preparation of this work. The final manuscript has been read, edited, and agreed by all the contributing authors.

Corresponding author

Correspondence to Fiona Campbell .

Ethics declarations

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The original online version of this article was revised: The authors identified an error in the author name of Zachary Munn and Danielle Pollock and affiliation of Fiona Campbell. The incorrect author names are: Zacchary Munn, Dannielle Pollock corrected to Zachary Munn, Danielle Pollock and affiliation 2 has been removed from author Fiona Campbell.

Useful resources: https://guides.temple.edu/c.php?g=78618&p=4156607 and https://wiki.joannabriggs.org/display/MANUAL/11.2+Development+of+a+scoping+review+protocol .

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Campbell, F., Tricco, A.C., Munn, Z. et al. Mapping reviews, scoping reviews, and evidence and gap maps (EGMs): the same but different— the “Big Picture” review family. Syst Rev 12 , 45 (2023). https://doi.org/10.1186/s13643-023-02178-5

Download citation

Received : 03 August 2022

Accepted : 24 January 2023

Published : 15 March 2023

DOI : https://doi.org/10.1186/s13643-023-02178-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Systematic Reviews

ISSN: 2046-4053

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

• University Libraries
• Research Guides
• Reviewing Research: Literature Reviews, Scoping Reviews, Systematic Reviews
• Differentiating the Three Review Types

Reviewing Research: Literature Reviews, Scoping Reviews, Systematic Reviews: Differentiating the Three Review Types

• Framework, Protocol, and Writing Steps
• Working with Keywords/Subject Headings
• Citing Research

The Differences in the Review Types

Grant, M.J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies. H ealth Information & Libraries Journal , 26: 91-108. https://doi.org/10.1111/j.1471-1842.2009.00848.x   The objective of this study is to provide descriptive insight into the most common types of reviews, with illustrative examples from health and health information domains.

• What Type of Review is Right for you (Cornell University)

Literature Reviews

Literature Review: it is a product and a process.

As a product , it is a carefully written examination, interpretation, evaluation, and synthesis of the published literature related to your topic. It focuses on what is known about your topic and what methodologies, models, theories, and concepts have been applied to it by others.

The process is what is involved in conducting a review of the literature.

• It is ongoing
• It is iterative (repetitive)
• It involves searching for and finding relevant literature.
• It includes keeping track of your references and preparing and formatting them for the bibliography of your thesis

• Literature Reviews (University of North Carolina at Chapel Hill) This handout will explain what literature reviews are and offer insights into the form and construction of literature reviews in the humanities, social sciences, and sciences.

Scoping Reviews

Scoping reviews are a " preliminary assessment of potential size and scope of available research literature . Aims to identify nature and extent of research evidence (usually including ongoing research)." Grant and Booth (2009).

Scoping reviews are not mapping reviews: Scoping reviews are more topic based and mapping reviews are more question based.

• examining emerging evidence when specific questions are unclear - clarify definitions and conceptual boundaries
• identify and map the available evidence
• a scoping review is done prior to a systematic review
• to summarize and disseminate research findings in the research literature
• identify gaps with the intention of resolution by future publications

• Scoping review timeframe and limitations (Touro College of Pharmacy

Systematic Reviews

Many evidence-based disciplines use ‘systematic reviews," this type of review is a specific methodology that aims to comprehensively identify all relevant studies on a specific topic, and to select appropriate studies based on explicit criteria . ( https://cebma.org/faq/what-is-a-systematic-review/ )

• clearly defined search criteria
• an explicit reproducible methodology
• a systematic search of the literature with the defined criteria met
• assesses validity of the findings - no risk of bias
• a comprehensive report on the findings, apparent transparency in the results

• Better evidence for a better world Browsable collection of systematic reviews
• Systematic Reviews in the Health Sciences by Molly Maloney Last Updated Jul 26, 2024 894 views this year
• Next: Framework, Protocol, and Writing Steps >>

Systematic Reviews & Other Review Types

• What is a Systematic Review?

What is a Scoping Review?

• What is a literature review?
• What is a Rapid Review?
• What is a Mixed Methods Review?
• What is a Network Meta-Analysis?
• What is an Overview of Reviews?
• What is a Meta-Syntheses?
• What is an Integrative Review?
• What is a Diagnostic Test Accuracy Review?
• What is a Living Systematic Review?

Scoping reviews are a "preliminary assessment of potential size and scope of available research literature.  Aims to identify nature and extent of research evidence (usually including ongoing research)."   Grant and Booth (2009).

Requires fewer data sources and doesn't require assessing individual studies for risk of bias. 

Often a scoping review is confused with a mapping review.  They are  not systematic reviews , but the methodology is closely related. 

Scoping Reviews are best:

"When a body of literature has not yet been comprehensively reviewed, or exhibits a large, complex, or heterogeneous nature not amenable to a more precise systematic review."

To map existing literature in terms of nature, features, volume

To clarify working definitions and conceptual boundaries of a topic or field

To identify gaps in existing literature/research

(Pete rs M, Godfrey C,  Khalil  H, et al)

How a Scoping Review Differs from a Systematic Review

Timeframe:  12+ months, (same amount of time as a systematic review or longer).

Question:  Answers broader questions beyond those related to the effectiveness of treatments or interventions.  A priori review protocol is recommended.

Sources and searches: Is still as comprehensive as a systematic review but much broader.  May involve multiple structured searches rather than a single structured search.  This will produce more results than a systematic review.  Must include a modified PRISMA flow diagram.

Selection:  Based on inclusion/exclusion criteria, due to the iterative nature of a scoping review some changes may be necessary.  May require more time spent screening articles due to the larger volume of results from broader questions.

Appraisal:  Not applicable for scoping reviews. 

Synthesis: The extraction of data for a scoping review may include a charting table or form.  Results may include a logical diagram or table or any descriptive form that aligns with the scope and objectives of the review.  May incorporate a numerical summary and qualitative thematic analysis.

Source: MDJ Peters et al. (2015), Levac et al. (2010)

Scoping Review Resources

• PRISMA SCR-Scoping Reviews Statement and Checklist
• Scoping studies: advancing the methodology
• Scoping Studies: Towards a Methodological Framework
• A scoping review of scoping reviews: advancing the approach and enhancing the consistency
• An Evidence-Based Approach to Scoping Reviews
• Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach
• CURRENT BEST PRACTICES FOR THE CONDUCT OF SCOPING REVIEWS
• Scoping reviews: establishing the role of the librarian
• Methodology for JBI Scoping Reviews

Limitations of a Scoping Review

• Is not easier than a systematic review.
• Is not faster than a systematic review, may take longer.
• More citations to screen
• Different screening criteria/process than a systematic review
• Often leads to a broader, less defined search.
• Requires multiple structured searches instead of one.
• Increased emphasis for hand searching the literature.
• May require larger teams because of larger volume of literature.
• Inconsistency in the conduct of scoping reviews.

Other names for a Scoping Review

Scoping Study, Systematic Scoping Review, Scoping Report, Scope of the Evidence, Rapid Scoping Review, Structured Literature Review, Scoping Project, Scoping Meta Review

Temple Attribution

Adapted with permission from Temple University Libraries. https://guides.temple.edu/systematicreviews

• Last Updated: Mar 13, 2024 4:27 PM
• URL: https://touromed.libguides.com/review_types

Grand Valley State University

• University Libraries

Search the Library for Articles, Books, and More Find It!

More research tools:

• Subject Guides
• Find, Borrow, Request
• Teaching & Scholarship
• Visit, Study, Learn
• Systematic and Scoping Reviews

Systematic and Scoping Reviews: Home

• Types of Reviews
• Review Guidelines
• 0. Plan your Review
• 1. Define the Question
• 2. Check for Recent Reviews and Protocols
• 3. Write and register your protocol
• Developing your Search Terms
• Database Search Tips and Filters
• Grey Literature
• Record and Report your Search Strategy
• 5. Organize & Screen Search Results
• 6. Appraise the Studies
• 7. Extract Data
• 8. Analyze / Synthesize Data
• 9. Write the Review
• Rapid Reviews
• Scoping Reviews
• Librarian Support

What is Evidence Synthesis?

'Evidence synthesis' is a collective term for types of literature research that bring together all relevant information on a well-formulated research question using a consistent, reproducible methodology. Most forms of evidence synthesis have one or more sets of guidelines for conducting a high-quality review. Systematic reviews and scoping reviews are two of the more common types.

Evidence syntheses should be conducted in an unbiased, reproducible way to provide evidence for practice and policy-making, as well as to identify gaps in the research. Some types include a meta-analysis, a more quantitative process of synthesizing and visualizing data retrieved from various studies.

Although systematic reviews are one of the most well-known review types, there are a variety of different types of reviews that vary in terms of scope, comprehensiveness, time constraints, and types of studies included. For more information about different review types, visit the Types of Reviews section.

About this guide

This guide presents practical tools and advice for conducting Systematic and Scoping Reviews and other evidence syntheses and comprehensive literature search projects:

• Review types
• Outline of the SR process
• Formulating an effective search strategy
• Selecting & searching databases
• Managing search results
• Reporting search methods
• How GVSU librarians can help

This guide does NOT replace the understanding of research design and methodology you will gain from reading sources such as the Cochrane Collaboration Handbook or the JBI Manual for Evidence Synthesis . Researchers new to systematic reviews, scoping reviews, and other forms of comprehensive evidence synthesis are strongly encouraged to read a guide appropriate to their review type and research question. If you're not sure which would be the most useful, your librarian can make recommendations.

A free, asynchronous training course on conducting systematic reviews and meta-analyses is available from Johns Hopkins University through Coursera at https://www.coursera.org/learn/systematic-review

Guide Credit

This guide was created with permission from the Evidence Synthesis and Systematic Review guides of the University of Washington , University of Michigan , and Cornell University .

• Next: Types of Reviews >>
• Last Updated: Sep 11, 2024 8:22 AM
• URL: https://libguides.gvsu.edu/SR

Login to LibApps

Email us! [email protected]

Text Us! (616) 818-0219

Mary Idema Pew Library (616) 331-3500

Steelcase Library (616) 331-6799

Frey Foundation Learning Commons (616) 331-5933

Social Media

Committed to Equality

Federal Depository Library Program

• GVSU is an AA/EO Institution
• Privacy Policy
• Disclosures
• Copyright © 2018 GVSU

• Open access
• Published: 11 September 2024

Too stressed to think? A scoping review of the literature for healthcare educators utilising high acuity clinical scenarios

• Jason Betson   ORCID: orcid.org/0000-0002-9883-8586 1 , 2 ,
• Erich C. Fein   ORCID: orcid.org/0000-0003-4119-0130 4 ,
• David Long   ORCID: orcid.org/0000-0002-6278-7377 2 &
• Peter Horrocks   ORCID: orcid.org/0000-0002-0220-175X 3  

BMC Medical Education volume  24 , Article number:  990 ( 2024 ) Cite this article

1 Altmetric

Metrics details

The practise of paramedicine can be highly stressful particularly where urgent lifesaving decisions need to be made. Traditionally, educators have adopted the approach of placing students in simulated stressful situations as a way of learning to cope with these challenges. It is unclear from the literature whether traditional stress inoculation enhances or hinders learning. This scoping review aims to identify and examine both the peer-reviewed and grey literature reporting physiological stress responses to high-acuity scenarios in paramedicine and cognate healthcare disciplines.

Adhering strictly to JBI Evidence Synthesis Manual for conducting a scoping review, medical subject headings and areas, keywords and all other possible index terms were searched across EBSCOhost (Medline, CINAHL and APA PsycInfo), Scopus and, PubMed. English language articles both published (peer-reviewed academic papers, reports and conference proceedings) and unpublished (grey literature, Google Scholar reports) were included, and publications citing retrieved articles were also checked.

Searches performed across five electronic databases identified 52 articles where abstracts indicated potential inclusion. From this, 22 articles which reported physiological or psychophysiological responses to stressful scenario-based education were included.

This review identified that an acceptable level of stress during simulation can be beneficial, however a point can be exceeded where stress becomes a hinderance to learning resulting in underperformance. By identifying strategies to moderate the impact of acute stress, educators of paramedic and other healthcare students can utilise high-acuity clinical scenarios to their andragogical armamentarium which has the potential to improve real-world clinical outcomes.

Peer Review reports

Introduction

In many high-income countries, paramedic education has progressively moved from a post-employment vocational training model of the latter part of the 20th century to now sit firmly within the pre-employment tertiary education sector [ 1 ]. This evolution of education enables paramedics to provide high-level emergency care as new-to-practice clinicians in high pressure, time-critical environments [ 2 ]. To do this, education providers often utilise high-fidelity simulations to apply clinical or other skills in realistic environments. These simulations, are often comprised of high-acuity scenarios which are designed to depict a high severity of illness or injury [ 3 ] requiring rapid medical interventions, which can invoke increased physiological and cognitive stress. It may be the case that if these simulations are too stressful, clinical learnings from them may be lost due to the high stress load the participant is exposed to as shown by Takahashi, et al. [ 4 ] who identified higher cortisol levels post stress exposure in university students, which correlated with an increased level of memory impairment and poorer performance.

Links between physiological stress and knowledge application have also been reported in the paramedicine sector. LeBlanc, et al. [ 5 ] demonstrated that clinicians made more drug calculation errors following exposure to stressful events, whilst senior paramedics exhibited clinical and documentational vulnerabilities during high-acuity scenarios [ 6 ]. In the emerging field of undergraduate paramedicine education research, few studies have explored high-acuity scenario-based education and any associated physiological and cognitive stress. This is in contrast to Harvey, et al. [ 7 ], LeBlanc, et al. [ 6 ] and, more recently Hase, et al. [ 8 ] who have recommended that training in high-acuity areas of medicine should include challenge-promoting interventions specifically relevant to stress mitigation.

In the expanding cohort of university-trained paramedicine students, research on empathy [ 9 ], prevention of mental health and psychological disorders [ 10 ], workplace violence [ 11 ], physical characteristics [ 12 ] and pre-employment fitness testing [ 13 ] have been published. However, linkages between time critical high-acuity scenario-based education with resultant physiological stress and its potential impacts on cognitive decision-making has not been studied. A recent systematic review explored the physiological responses to acute stress in workers of several occupations, mostly within the human service industry [ 14 ]. Whilst this paper draws appropriate conclusions about acute physiological changes leading to performance decrement, possible implications for frontline healthcare workers were limited by a small number of healthcare-based studies included within the review. A gap also exists between self-awareness of one’s own physiological stress and how this may impact clinical judgement. Therefore, the purpose of this review was to better understand the physiological and cognitive stress responses observed in the participants undertaking high-acuity clinical scenarios. By appreciating the existence of contributory factors and how they influence stress, educators of paramedics and other healthcare workers can determine which elements of physiologically and mentally stressful scenario-based education can be considered in the design of their own programs.

Study design

Full systematic literature reviews (SLRs) are generally considered to be the foundation for evidence-based practice, particularly in healthcare [ 15 ]. This form of evidence synthesis relies on an extensive base of published literature and is frequently used to validate or refute current practice [ 16 ]. However, within the scope of the present study, little extant research reports on physiological changes triggered by high stress learning situations or the consequences this effect has on clinical performance. Given the inter-relationship between acute stress and the degradation of cognitive decision-making ability [ 7 , 17 ], further research is warranted to characterise this physiological response in undergraduate paramedicine students. In this paper, we employed a scoping review method to explore the extent of published and unpublished literature from cognate heath disciplines to identify key characteristics or factors related to our topic of interest.

Our final protocol was registered on the 21st March 2023, and is publicly available on the Open Science Framework platform ( https://osf.io/dxchy/ ).

Identifying the research question

This scoping review aims to identify and map the scope of current published literature related to physiological stress responses to high-acuity scenarios and, importantly, identify and analyse the knowledge gaps [ 18 ]. To achieve the aim, the following search strategy was employed:

Participants: higher education students or students in non-university training programs studying towards a recognised healthcare qualification.

Concept: any study that incorporates clinical scenarios / simulations where physiological (cardiovascular or endocrine) /or psychophysiological data is recorded.

Context: any undergraduate or postgraduate higher education setting or equivalent non-university training facility for the participants mentioned above.

Search strategy and eligibility criteria

The latest version [ 19 ] of Joanna Briggs Institute (JBI) comprehensive guide for authors conducting a scoping reviews [ 20 ] has been followed step-by-step within this review. A search period restriction from 2000 onwards was applied due to the rapid expansion of wearable technology including augmented and virtual reality. To ensure the review examined the acute physiological stress response, it was necessary to focus on articles that assessed markers of stress in real-time as participants were exposed to a stress-inducing task. An initial search was conducted across three prominent databases (Medline, PubMed and Scopus) to determine key terms as a guide to developing a thorough search strategy. From this and with the assistance of a senior research librarian, the secondary search expanded all identified keywords and incorporated medical subject headings (MeSH), major subject areas, and all other possible index terms as noted in the Appendix 1 . The protocol incorporated both published (peer-reviewed academic papers, reports and conference proceedings) and unpublished (incorporating theses and dissertations, research and technical reports) evidence but did exclude non-English language articles. Sources were gathered using EBSCOhost (including Medline, CINAHL and APA PsycInfo) Scopus, and PubMed. Google Scholar was also searched as there is a small body of evidence that suggests this search engine produces highly comprehensive results [ 21 , 22 ] whilst also searching ‘grey literature’ (published informally, non-commercially or remains unpublished), a format neglected by other databases. Selection of papers for inclusion in the study were then undertaken independently by two members of the research team (DL and EF). Finally, any other articles that cited the retrieved articles were also checked using citation alert with the ISI Web of Knowledge (Appendix 1 ).

Extracting and charting the data

Data were extracted from the included studies by two reviewers (JB and PH) utilising the JBI template of evidence details, characteristics and results extraction instrument [ 19 ]. Initial piloting of the data extraction resulted in some additional data being sought from each publication to allow quality appraisal to occur. This refined data extraction gathered details about study year, study country, study aim, study setting, study design, interventions, and comparators. Additionally, the data included sample size, methods, results, and author recommendations. A third member of the review team (DL) performed an accuracy check.

Quality assessment

Methodological validity and risk-of-bias appraisal, undertaken concurrently with data charting, was performed via the Mixed Methods Appraisal Tool (MMAT) version 2018 critical appraisal instrument designed by Hong, et al. [ 23 ]. For the purpose of this scoping review, an overall score was calculated from mean values of each section to determine methodological quality of each reviewed study (Appendix 4 ). The authors agreed that no cut-off scores would be applicable as the use of the MMAT was not for inclusion or exclusion purpose, but rather to describe the quality of the of publications reported in this review.

Synthesis of results

The first author performed narrative synthesis of identified themes and discussed these with the review team for validation. Descriptive results are subsequently reported which align with the intended scope and objectives of this review.

Ethics statement

Ethical approval was not required for this scoping review.

The search strategy yielded 1427 results, of which 52 remained after title and abstract proofing and duplicate removal (Level 1). Consensus was not reached on seven papers with resolution sought from a third member of the review team (PH) (Appendix 2 ). Of the 52 studies, 30 were excluded for reasons outlined in Appendix 3 . In addition, reference lists of three excluded review articles were checked, although nil additional suitable articles were identified. Unpublished (grey) literature was also assessed with no additional studies deemed suitable for inclusion. Figure  1 illustrates a Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) flow diagram [ 24 ] of the process and Table  1 lists the 22 studies deemed eligible for inclusion.

PRISMA flow diagram findings

From the twenty-two studies identified that met the inclusion criteria, the majority originated from Europe and North America. Only one study [ 25 ] involved paramedics or paramedicine students. Twelve studies involved medical or surgical trainees [ 7 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 ], five studied nursing or nurse anaesthetist students [ 37 , 38 , 39 , 40 , 41 ], three studies involved physiotherapy students [ 42 , 43 , 44 ], and one study examined psychology students [ 45 ].

No studies were identified that warranted exclusion based on major methodological flaws on any significant risk of bias. However, study design flaws and lower levels of evidence were common. Of most concern were poorly described methodologies and under-powered sample sizes incapable of producing statistically significant results (see Appendix 4 for tabulated quality assessment results). Examining the methodology used, sixteen studies adopted a mixed methods approach and six utilised a quantitative method. Randomised controlled trials were reported in seven papers and a battery of different metrics were recorded across the studies. Heart rate variability and salivary cortisol levels were the most frequently reported objective data, whilst the state trait anxiety inventory was the most common subjective measure (see Table  2 ).

In general terms, the twenty-two included studies had similar aims centring around determining how successful high stress simulation could be at replicating clinical experience. Most studies involved both male and female participants with samples sizes ranging from n  = 8 to n  = 166, with a mean of n  = 53 and a median of n  = 33. Multiple studies assessed and compared stress levels of participants in different situations, and then used these results to determine if stress had affected clinical performance. Other studies used similar data to improve education or training with the aim of ultimately increasing student confidence and performance. The key outcomes from the included studies are summarised in Table  3 .

This review identified studies exploring physiological responses of participants undertaking high stress scenario-based education or training. While acknowledging much of the evidence was of low methodological quality [ 46 ] and therefore limits generalisability, the results still provide some useful insights that may be used to inform educators of future paramedics and other healthcare workers.

One of the key findings from this review was the identification of pre-performance or anticipatory anxiety exhibited across multiple studies [ 37 , 38 , 45 ]. This is an area where simulation may not replicate clinical work. Students aware of an upcoming scenario well in advance have ample time to prepare and mount a physiological stress response. This could be controlled if students were given little notice, however this was not commonly reported in the studies. Healthcare educators utilising scenario-based education may choose to restrict prior notification as a means of assessing any changes in the stress response amongst their students. In high-acuity clinical work, paramedics usually have little time to prepare, which may reduce the anticipatory stress response. Potentially this may be seen as positive, as stress has been demonstrated to lead to poorer performance is some paramedic research [ 5 , 6 ]. However, the evident stress of attending high-acuity cases must also be considered and its impact on performance. In the context of anticipatory anxiety predicting future performance, little research has examined its immediate effect on motor task performance.

The reviewed publications also provide contradictory support for simulation as a tool to replicate the psychophysiological stress of high-acuity clinical work. Baker, et al. [ 37 ], in a study with trainees in the highly specialised field of anaesthetics, found simulation was able to replicate the physical and procedural forms of clinical work, however it was unable to replicate the intrinsic level of stress the trainees exhibited when working with a real patient in an operating theatre. These results are potentially influenced by small participant numbers ( n  = 8) and may also be applicable to highly specialised and highly technical fields such as anaesthesia. For paramedicine, contemporary literature [ 47 ] highlights simulation allowing for the training of skills that are rarely needed or rarely practiced in the field and supports recommendation made by O’Meara, et al. [ 2 ]. For educators of paramedics and other healthcare workers, simulating high-acuity situations is a crucial way to expose students to potential clinical scenarios they may face early in their career. In designing programs of study, careful use of stress-inducing high-acuity simulation can be a beneficial but can also lead to continued underperformance if the stress is chronically too high.

Barbadoro, et al. [ 26 ] and Judd, et al. [ 43 ] found simulation provided a higher level of stress in their participants when compared to equitable clinical work, whilst Demaria, et al. [ 28 ] found that high stress situations can be beneficial for learning. This benefit of high stress learning was also supported by the work of Keitel, et al. [ 32 ], who found increased levels of the key stress hormone cortisol correlated with improved memory retention and medical performance amongst medical trainees. An increased stress response was also reported when supervisors or assessors were present within the simulation [ 30 , 31 ] and, unsurprisingly, vital signs as a measure of physiological stress, increased when the simulation itself was exertive [ 25 , 27 , 36 ]. The stress placed on students involved in high-acuity simulation must be further studied to allow educators to determine what level of anxiety may enhance learning without impeding performance.

Performance ability or academic standing was also found to correlate with stress. McKay, et al. [ 38 ] found low performers increased stress and performed poorly, whereas high performers also increased stress but performed superbly in a cohort of student nurses. Paramedicine courses may show similar trends, with students likely to self-assess their academic abilities and stress tolerance. Educators could potentially use real-time learning analytics to offer tailored support and guidance based on live biometric data, proactively aiding students. This would be resource intensive for academics with large student numbers; but in smaller cohorts, the individual feedback around acceptable stress to achieve simulated clinical success may enhance the education program.

Lacking from the literature is a detailed discussion of a variety of variables related to student stress responses from the level of acuity of a scenario. These confounding variables, such as pre-established coping styles and perceived stress intensity within participants, need to be quantified to accurately gauge the success of any interventions aimed at alleviating the stress response, and in examining what levels of anxiety may enhance learning without impeding performance.

Limitations

Whilst the systematic approach to this scoping review explored multiple electronic bibliographic repositories, there is potential some contemporary conference proceedings, dissertations and theses, along with grey literature not readily available in electronic databases or Google Scholar, may have been missed. Non-English literature may have added value to this review and we attempted to seek translated papers where possible, but we accept that some results may have been missed through this process.

Directions and recommendations for future research

This scoping review identified inconsistencies and varying methodologies for the assessment of participant stress response in scenario-based education. Recommendations should be developed to identify gold standard quantification of psychophysiological stress responses during high stress scenarios. This would then allow meta-analysis or other systematic synthesis of data to be undertaken to accurately determine any inter-relationship between acute stress and the degradation of cognitive decision-making for healthcare education programs. In addition, variables related to student stress responses from the level of acuity of a scenario should be investigated. For example, individual differences in participants such as pre-established coping styles and strategies, perceived stress intensity, perceived control of stress or coping skill, as well as context specific stressors such as the outcomes associated with scenario performance (e.g., high stakes versus low stakes outcomes) may all be important variables for future research.

The studies identified in this scoping review have shown high-acuity simulation can induce stress comparable with paramedicine clinical practice. For educators, understanding the factors or elements which contribute to an acceptable level of stress can allow participants the opportunity to fail and learn from their errors during simulation. This further provides opportunities to improve student outcomes in paramedicine and other healthcare education by facilitating high-acuity clinical scenarios that challenge students without inducing stress levels that hinder performance. As educational and wearable technology further evolves, utilisation of real-time biofeedback through passive measurement devices also hold promise as an intervention to reduce the negative effects of acute physiological stress during training scenarios.

Data availability

Data supporting Fig.  1 ; Tables  1 , 2 and 3 and available within the Supplementary Information (Appendices).

Hou XY, Rego J, Service M. Review article: paramedic education opportunities and challenges in Australia. Emerg Med Australasia. 2013;25(2):114–9.

Article   Google Scholar  

O’Meara P, Furness S, Gleeson R. Educating paramedics for the future: a holistic approach. J Health Human Serv Adm. 2017;40(2):219–53.

Google Scholar  

Chrimes N. The Vortex: a universal ‘high-acuity implementation tool’ for emergency airway management. Br J Anaesth. 2016;117(Suppl 1):i20–7.

Takahashi T, Ikeda K, Ishikawa M, Tsukasaki T, Nakama D, Tanida S, et al. Social stress-induced cortisol elevation acutely impairs social memory in humans. Neurosci Lett. 2004;363(2):125–30.

LeBlanc VR, MacDonald RD, McArthur B, King K, Lepine T. Paramedic performance in calculating drug dosages following stressful scenarios in a human patient simulator. Prehospital Emerg Care. 2005;9(4):439–44.

LeBlanc VR, Regehr C, Tavares W, Scott AK, Macdonald R, King K. The impact of stress on paramedic performance during simulated critical events. Prehosp Disaster Med. 2012;27(4):369–74.

Harvey A, Nathens AB, Bandiera G, Leblanc VR. Threat and challenge: cognitive appraisal and stress responses in simulated trauma resuscitations. Med Educ. 2010;44(6):587–94.

Hase A, O’Brien J, Moore LJ, Freeman P. The relationship between challenge and threat states and performance: a systematic review. Sport Exerc Perform Psychol. 2019;8(2):123–44.

Williams B, Boyle M, Earl T. Measurement of empathy levels in undergraduate paramedic students. Prehosp Disaster Med. 2013;28(2):145–9.

Wild J, El-Salahi S, Tyson G, Lorenz H, Pariante CM, Danese A, et al. Preventing PTSD, depression and associated health problems in student paramedics: protocol for PREVENT-PTSD, a randomised controlled trial of supported online cognitive training for resilience versus alternative online training and standard practice. BMJ Open. 2018;8(12):e022292.

Boyle M, McKenna L. Paramedic student exposure to workplace violence during clinical placements - a cross-sectional study. Nurse Educ Pract. 2017;22:93–7.

Davies S, Naidoo N, Parr B. Physical performance characteristics of South African male and female Emergency Care students (ECS). Ergon SA: J Ergon Soc South Afr. 2008;20(2):3–14.

Thornton KE, Sayers MG. Unfit for duty? Evaluation of 4 years of paramedic preemployment fitness screening test results. Prehospital Emerg Care. 2014;18(2):201–6.

Frazier SE, Parker SH. Measurement of physiological responses to acute stress in multiple occupations: a systematic review and implications for front line healthcare providers. Transl Behav Med. 2019;9(1):158–66.

Munn Z, Porritt K, Lockwood C, Aromataris E, Pearson A. Establishing confidence in the output of qualitative research synthesis: the ConQual approach. BMC Med Res Methodol. 2014;14(1):108.

Pearson A, Jordan Z, Munn Z. Translational science and evidence-based healthcare: a clarification and reconceptualisation of how knowledge is generated and used in healthcare. Nurs Res Pract. 2012;2012:792519.

Domes G, Heinrichs M, Rimmele U, Reichwald U, Hautzinger M. Acute stress impairs recognition for positive words–association with stress-induced cortisol secretion. Stress. 2004;7(3):173–81.

Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018;18(1):143.

Peters MDJ, Marnie C, Tricco AC, Pollock D, Munn Z, Alexander L, et al. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synthesis. 2020;18(10):2119–26.

Peters MD, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. Int J Evid Based Healthc. 2015;13(3):141–6.

Bajpai A, Davuluri S, Haridas H, Kasliwal G, Ks HD. S, In search of the right literature search engine(s). Nat Precedings. 2011.

Gehanno JF, Rollin L, Darmoni S. Is the coverage of Google Scholar enough to be used alone for systematic reviews. BMC Med Inf Decis Mak. 2013;13(1):7.

Hong QN, Fàbregues S, Bartlett G, Boardman F, Cargo M, Dagenais P, et al. The mixed methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers. Educ Inform. 2018;34(4):285–91.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. PLoS Med. 2021;18(3):e1003583.

MacQuarrie AS, Hunter JR, Sheridan S, Hlushak A, Sutton C, Wickham J. Paramedic Student Clinical Performance during High-Fidelity Simulation after a physically demanding Occupational Task: a pilot randomized crossover trial. Simul Healthcare: J Soc Simul Healthc. 2022;17(4):234–41.

Barbadoro P, Brunzini A, Dolcini J, Formenti L, Luciani A, Messi D, et al. Stress responses in high-fidelity simulation and standard simulation training among medical students. BMC Med Educ. 2023;23(1):116.

Bialka S, Copik M, Ubych A, Marciniak R, Smereka J, Szarpak L, et al. Effect of high-fidelity simulation on alpha-amylase activity and concentrations of secretory immunoglobulin class A, cortisol, and testosterone among medical students. Endocrine. 2021;73(2):431–8.

Demaria S Jr., Bryson EO, Mooney TJ, Silverstein JH, Reich DL, Bodian C, et al. Adding emotional stressors to training in simulated cardiopulmonary arrest enhances participant performance. Med Educ. 2010;44(10):1006–15.

DeMaria S, Silverman ER, Lapidus KA, Williams CH, Spivack J, Levine A, et al. The impact of simulated patient death on medical students’ stress response and learning of ACLS. Med Teach. 2016;38(7):730–7.

Feeley AA, Feeley IH, McManus R, Lunn JV, Sheehan E, Merghani K. Evaluating the impact of Supervision on Surgical trainees stress response during simulated Surgical procedures; a crossover randomized Trial. J Surg Educ. 2022;79(6):1379–86.

Flinn JT, Miller A, Pyatka N, Brewer J, Schneider T, Cao CG. The effect of stress on learning in surgical skill acquisition. Med Teach. 2016;38(9):897–903.

Keitel A, Ringleb M, Schwartges I, Weik U, Picker O, Stockhorst U, et al. Endocrine and psychological stress responses in a simulated emergency situation. Psychoneuroendocrinology. 2011;36(1):98–108.

Los K, Chmielewski J, Cebula G, Bielecki T, Torres K, Luczynski W. Relationship between mindfulness, stress, and performance in medical students in pediatric emergency simulations. GMS J Med Educ. 2021;38(4):Doc78.

Martin-Rodriguez F, Castro Villamor MA, Lopez-Izquierdo R, Portillo Rubiales RM, Ortega GJ, Sanz-Garcia A. Can anxiety in undergraduate students in a high-fidelity clinical simulation be predicted? A randomized, sham-controlled, blinded trial. Nurse Educ Today. 2021;98:104774.

Rieber N, Betz L, Enck P, Muth E, Nikendei C, Schrauth M, et al. Effects of medical training scenarios on heart rate variability and motivation in students and simulated patients. Med Educ. 2009;43(6):553–6.

Tramèr L, Becker C, Hochstrasser S, Marsch S, Hunziker S. Association of electrocardiogram alterations of rescuers and performance during a simulated cardiac arrest: a prospective simulation study. PLoS ONE. 2018;13(6):e0198661.

Baker BG, Bhalla A, Doleman B, Yarnold E, Simons S, Lund JN, et al. Simulation fails to replicate stress in trainees performing a technical procedure in the clinical environment. Med Teach. 2017;39(1):53–7.

McKay KA, Buen JE, Bohan KJ, Maye JP. Determining the relationship of acute stress, anxiety, and salivary alpha-amylase level with performance of student nurse anesthetists during human-based anesthesia simulator training. AANA J. 2010;78(4):301–9.

Nakayama N, Arakawa N, Ejiri H, Matsuda R, Makino T. Heart rate variability can clarify students’ level of stress during nursing simulation. PLoS ONE. 2018;13(4).

Park HJ, Choi D, Park HA, Lee CA. Nurse evaluation of stress levels during CPR training with heart rate variability using smartwatches according to their personality: a prospective, observational study. PLoS ONE. 2022;17(6).

Stecz P, Makara-Studzińska M, Białka S, Misiołek H. Stress responses in high-fidelity simulation among anesthesiology students. Sci Rep. 2021;11(1):17073.

Beltrán-Velasco AI, Ruisoto-Palomera P, Bellido-Esteban A, García-Mateos M, Clemente-Suárez VJ. Analysis of Psychophysiological Stress Response in higher education students undergoing clinical practice evaluation. J Med Syst. 2019;43(3):68.

Judd BK, Alison JA, Waters D, Gordon CJ. Comparison of psychophysiological stress in Physiotherapy Students Undertaking Simulation and Hospital-based Clinical Education. Simul Healthcare: J Soc Simul Healthc. 2016;11(4):271–7.

Palekar TJ, Mokashi MG, Anwer S, Kakrani AL, Khandare SD, Alghadir AH. Effect of galvanic skin resistance-aided Biofeedback Training in reducing the pulse rate, respiratory rate, and blood pressure due to perceived stress in Physiotherapy students. / Fizyoterapi Öğrencilerinde Algılanan Strese Bağlı Nabız Hızı, Solunum Hızı ve Kan Basıncının Azaltılmasında Galvanik Deri Direnci Destekli Bio-geri Bildirim Eğitiminin Etkisi. Turkish J Phys Med Rehabilitation / Turkiye Fiziksel Tip ve Rehabilitasyon Dergisi. 2015;61(2):116–9.

Beltrán-Velasco AI, Bellido-Esteban A, Ruisoto-Palomera P, Clemente-Suárez VJ. Use of Portable Digital devices to analyze autonomic stress response in psychology objective structured clinical examination. J Med Syst. 2018;42(2):35.

Burns PB, Rohrich RJ, Chung KC. The levels of evidence and their role in evidence-based Medicine. Plast Reconstr Surg. 2011;128(1):305–10.

Diamond A, Bilton N. The current state on the Use of Simulation in Paramedic Education. Australasian J Paramedicine. 2021;18:1–5.

Download references

Acknowledgements

The authors wish to acknowledge and thank senior research librarian Meena Gupta for assistance with developing and refining key search terms within the research protocol.

No funding sources or any specific materials need to be disclosed.

Author information

Authors and affiliations.

Faculty of Health, Australian Catholic University, Building 403, Daniel Mannix Building, 8 – 14 Brunswick St, Fitzroy, VIC, 3065, Australia

Jason Betson

School of Health and Medical Sciences, Centre for Health Research, University of Southern Queensland, Ipswich, Australia

Jason Betson & David Long

School of Clinical Sciences, Queensland University of Technology, Brisbane, Australia

Peter Horrocks

School of Psychology and Wellbeing, Centre for Health Research, University of Southern Queensland, Toowoomba, Australia

Erich C. Fein

You can also search for this author in PubMed   Google Scholar

Contributions

JB, DL, EF and PH conceived the study. JB and DL equally designed the study approach. DL, EF and PH undertook the review. JB and PH undertook data extraction and quality control. JB interpreted the data. JB, DL and EF drafted the manuscript and circulated to authors for contribution. All authors edited drafts and approved the current manuscript for publication. JB as the corresponding author is responsible for the overall content.

Corresponding author

Correspondence to Jason Betson .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Betson, J., Fein, E.C., Long, D. et al. Too stressed to think? A scoping review of the literature for healthcare educators utilising high acuity clinical scenarios. BMC Med Educ 24 , 990 (2024). https://doi.org/10.1186/s12909-024-05949-3

Download citation

Received : 05 February 2024

Accepted : 23 August 2024

Published : 11 September 2024

DOI : https://doi.org/10.1186/s12909-024-05949-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Physiological stress
• Decision-making
• Paramedicine

BMC Medical Education

ISSN: 1472-6920

• General enquiries: [email protected]

• Open access
• Published: 06 September 2024

A scoping review of health literacy in rare disorders: key issues and research directions

• Una Stenberg   ORCID: orcid.org/0000-0002-6007-8630 1 , 2 ,
• Lydia Westfal 1 ,
• Andreas Dybesland Rosenberger 3 ,
• Kristin Ørstavik 4 ,
• Maria Flink 5 ,
• Heidi Holmen 6 ,
• Silje Systad 7 ,
• Karl Fredrik Westermann 2 &
• Gry Velvin 8  

Orphanet Journal of Rare Diseases volume  19 , Article number:  328 ( 2024 ) Cite this article

Metrics details

The ability to find, understand, appraise and utilise health information is crucial among individuals living with rare disorders. The aim of this study was to give a comprehensive overview of the literature on health literacy in adult persons with rare disorders.

We applied a scoping review methodology and performed a systematic search in 2021 in bibliographic databases. Searches were conducted in Medline (Ovid), Embase (Ovid), PsycInfo (Ovid), CINAHL (ebsco), and ERIC (Ovid). References were sorted and evaluated for inclusion using EndNote and Covidence. This review was guided by the question “What are the characteristics of research on health literacy in rare disorders?”

The database searches yielded 75 eligible reports. A total of 6223 individuals with rare disorders were represented alongside 1707 caregivers. The reports in this review have included study participants representing a total of 80 different rare disorders with unique ORPHA and ICD-10 codes. The results revealed that persons with rare disorders often exhibit gaps in health literacy through a lack of knowledge and access to information related to self-management, their own diagnosis and health, as well as daily coping and social rights. In addition, the importance of aid and information from healthcare personnel and the significance of getting social support from others in the same situation were accentuated.

This review emphasizes the importance of reinforcing health literacy among persons with rare disorders through peer support and education. This is the first review to give a comprehensive and state-of-the-art overview of literature investigating health literacy among persons with rare disorders and offers a basis for further research.

Introduction

In Europe, a disorder is considered rare when it affects less than 1:2000 individuals [ 1 ]. According to current calculations, more than 7000 different rare disorders have been identified. However, it is plausible that the actual number may be as high as 10,000 [ 2 ]. Although each rare disorder affects a limited quantity of individuals, it is estimated that the combined prevalence of all rare diseases is 3,5–5,9% [ 3 ]. Accordingly, up to 36 million people residing in the European Union are living with a rare disease [ 4 ]. Out of the total rare disorders, 72% have a genetic aetiology, and 70% have childhood onset [ 3 ]. Whilst there is a large clinical diversity between the rare disorders, they tend to have some aspects in common; they are known for being chronic, complicated, mostly degenerating, and often disabling [ 5 ].

Persons with rare disorders face some unique challenges in accessing information on their diagnosis, which may lead to issues in making beneficial health choices regarding treatment and care [ 6 ]. A key issue with rare disorders is the lack of research in the field [ 7 , 8 ]. Insufficient evidence and knowledge on rare diseases in general pose challenges both for professionals and people with these diseases [ 8 ]. Due to healthcare professionals’ limited understanding of their rare disorder in general, as well as a lack of information provided, persons with rare disorders often need to search for health-related information themselves [ 6 ]. A systematic review published in 2017 aimed to provide an overview of adults` shared experience of living with a rare disorder, found that in 12 out of 21 reports, persons with rare disorders reported progressively becoming “experts” on their own diagnosis [ 9 ]. In some cases, those living with rare disorders possess more information about the condition than the healthcare professionals they encounter [ 10 ].

Healthcare systems are increasingly challenging to navigate [ 11 ]. Simultaneously, the healthcare services share prospective aims of prioritising digitization, enabling more home-based care, promoting shared decision-making, and ensuring equitable access to services [ 5 , 12 , 13 ]. Managing one’s health while dealing with a rare disorder and the responsibility of seeking information can be especially demanding due to the challenging standards set by the healthcare system [ 6 ].

Increased participation and responsibility for one’s own health impose a demand on the individual to have adequate health literacy. Health literacy pertains to individuals’ ability to manage the complex health requirements of today’s society and make informed decisions regarding health [ 14 ]. This includes understanding the factors that affect one’s health, addressing health challenges, and making appropriate health choices. There is a lack of consensus on the definition of health literacy, and multiple interpretations have been made [ 14 ]. A review by Sørensen et al. [ 15 ] identified as many as 17 different definitions of health literacy and created a working definition of health literacy by considering the contents of each interpretation. The inclusive definition according to Sørensen et al. is stated as follows:

“Health literacy is linked to literacy and entails people’s knowledge , motivation and competencies to access , understand , appraise , and apply health information in order to make judgments and take decisions in everyday life concerning healthcare , disease prevention and health promotion to maintain or improve quality of life during the life course.” (ref p. 3).

Along with the comprehensive definition, Sørensen et al. developed an integrated model of health literacy [ 15 ]. The model has been widely used to understand the complex interaction between individual skills and abilities related to health literacy, social and environmental factors, and health outcomes. The core elements of the model are four cognitive competencies; to access, understand, appraise, and apply health-related information. These four competencies allow a person to manoeuvre three identified domains on the health spectrum: healthcare, disease prevention, and health promotion. The model suggests that an individual’s ability to access and use health information is determined by their own skills, motivation, and knowledge as well as the social and environmental context they reside within. These conditions, accordingly, affect individuals’ ability to address their health and ultimately impact their health outcomes.

Sorensen’s model emphasises that components such as empowerment, health outcomes, and health behaviour are interlaced and connected to an individual’s health literacy. Enhancing the level of health literacy allows individuals to become more empowered and take charge of their health, participate in health-promoting behaviours, and gradually attain improved health outcomes [ 15 ]. Thus, participation and empowerment can give persons with rare disorders enhanced control over their own health and treatment, and increased involvement in decision-making processes that concern their health. This may lead to better health outcomes and elevated health-related quality of life, which remain crucial as persons with rare disorders report lower quality of life compared to those with more common chronic conditions [ 16 ]. They can feel stigmatised and marginalised in the healthcare system, and it can be challenging to find psychosocial support. Examining how to increase health literacy and empowerment for persons with rare disorders can therefore be an important and relevant direction for further research. Health literacy of individuals with rare disorders is an emerging field of research, and the literature is based on a wide range of study methodologies [ 7 , 8 ]. Hence, this scoping review aims to give a comprehensive overview of empirical reports (from primary research studies) investigating health literacy among persons with rare disorders as reported in the international literature, by identifying characteristics of definitions, study populations, methods and interventions.

Study design and research questions

The scoping review process described by Arksey and O`Malley [ 17 ] aims to: “(…) map rapidly the key concepts underpinning a research area and the main sources and types of evidence available and can be undertaken as a stand-alone project in their own right , especially where an area is complex or has not been reviewed comprehensively before. ” A scoping review methodology is also suitable for examining the extent, range, variety, and characteristics of evidence on a topic, but also to identify research gaps. This scoping review was conducted according to the five-stage framework by Arksey and O`Malley [ 17 ], enhanced by Levac [ 18 ] and Daudt [ 19 ] and reported according to the PRISMA Extension for Scoping Reviews [ 20 ] (shown in Additional file 1 ). A protocol for this review is available on request.

The aim of this review was to identify the characteristics of research on health literacy in rare disorders. The specific research questions were:

What are the characteristics of study populations?

When and where have reports on health literacy been carried out?

What are the characteristics of research questions used to investigate health literacy?

What are the characteristics of methods used to investigate health literacy?

What are the characteristics of assessment tools used to measure health literacy?

What are the characteristics of interventions that have been described in the reports?

How is health literacy defined or described in the reports?

How is access to health information and support for individuals with rare disorders described the reports?

Overarching participatory approach

The study group in this scoping review included one co-researcher, one with experiential knowledge trained in research methods, several experienced healthcare professionals in the field of rare disorders, working in clinical practice (specialized health care), and experienced researchers in health literacy and scoping review methodology. All members have been involved in all stages of the review process.

Eligibility criteria

This scoping review included primary research reports that investigated health literacy in adults with rare disorders. Reports were included if they had investigated the individual`s capacities, skills and motivation to make judgements and decisions in everyday life concerning healthcare, disease prevention and health promotion in persons with a rare disorder. While being 18 years of age or older was set as a search criterion, reports that included both adults and persons below 18 were not excluded. Empirical reports in English and Scandinavian languages published in peer-reviewed journals were included. All study designs were included. Dissertations, reports published in abstract form only, editorials, commentaries and duplicates were excluded.

Systematic searches

In the first stage, research questions were developed by the study group in a highly iterative process. We agreed to apply a broad variety of synonyms, conducting many and extensive pilot searches and simultaneously enhancing the search strategy, and clarify the criteria for inclusion and exclusion of reports. A senior academic librarian, in close collaboration with the first author, developed a systematic literature search using MeSH-terms and free search terms combining a comprehensive set of synonyms and terms for health literacy and rare disorders. Both the librarian and the researchers in the study group had experience with previous literature searches in the field of rare disorders. The literature searches complied with the PICO principles and applied a combination of “OR” within groups and “AND” between groups. Searches were conducted in Medline (Ovid), Embase (Ovid), PsycInfo (Ovid), CINAHL (ebsco), and ERIC (Ovid) for publications between 2010 and 2021. No other sources for literature were searched for this review. The complete search strategy is displayed in Additional file 2 .

Selection of publications

All titles and abstracts were reviewed by the first author (US) and one of the co-authors independently using the systematic review software Covidence (Veritas Health Innovation). Disagreements and conflicts were resolved through discussion with a third review author.

Data extraction

All data from the included reports were extracted according to study characteristics, participant characteristics included ORPHA and ICD-codes, description of interventions, methods, assessment tools, definitions and understanding of health literacy was collected using data extraction forms and reported separately for each study in evidence summaries (Supplementary Material 4 – 9 : Tables 2–6). A full reference list of included reports is presented in Additional file 3 . Extracted data is presented in a descriptive manner using text, tables and figures. All members of the study group participated in the data extraction. We did not attempt to contact the authors in this review process.

The search of the online databases resulted in 5999 reports when duplicates were removed. From these, 5794 were excluded because they did not fulfil the inclusion criteria. A total of 177 reports were downloaded in full text and read by two authors. Of these, 102 reports were excluded, leaving 75 to undergo analysis in this review (Fig. 1). All the included reports were in English language.

Prisma flow diagram

Characteristics of study populations

A total of 6223 persons with a rare disorder and 1707 caregivers were represented in the 75 included reports. About 70% of the reports were based on data from samples with less than 100 participants. About 15% of the reports were based on samples with more than 200 participants. Of the included participants in the reports, about 60% were female. Of the reports that reported the mean age of the participants, approximately 75% of the participants were between 30 and 50 years of age. About 15% of the reports had participants with a mean age over 55 years, and eleven reports had participants with a mean age under 25 years.

The reports in this review have included study participants representing a total of 80 different rare disorders with unique ORPHA and ICD-10 codes. A detailed description of diagnoses is given in Table 1 (Additional file 4 ). Five of the reports included participants across rare disorders but did not specify what type of disorders. Most of the rare disorders had been investigated in one or two reports, but a few disorders were investigated in several reports: different types of Hemophilia were investigated in 24/75 reports, Cystic Fibrosis in 14/75 reports, Huntington’s disease in 7/75 reports, Scleroderma in 4/75 reports and Myotonic dystrophy type 1, Neurofibromatosis type 1 and Spina bifida in 3/75 reports.

Where and when have reports on health literacy been carried out?

Of the 75 included reports, 21 were conducted in the USA, 11 in Canada and eight in the UK (see Table 2 for details in Additional file 9 ). The included reports were published between 2010 and 2021, 54/75 after 2016.

Characteristics of research questions

The research questions most frequently investigated among the included reports were related to assessments of experienced knowledge and different health- and/or psychosocial outcomes (31/75). The second most investigated research questions (27/75) were about persons with rare disorders’ views, experiences and understanding of their own condition, care, health information, management, transition process or peer support (see Table 6 for details in Additional file 8 ). In addition, 16 reports were conducted to evaluate an intervention aimed to improve or strengthen participants` knowledge, health literacy or coping, and therefore included in this review.

Characteristics of methodological design

Of the included reports, 28/75 applied a quantitative cross-sectional design to explore characteristics of patient groups in terms of knowledge and disease-related variables. Among the cross-sectional reports, both digital and paper-based surveys were used, and some gathered data through medical charts or personal interviews. In addition, fourteen reports applied an experimental design investigating either the feasibility or effects of specific interventions, mainly to increase knowledge or health literacy. Among the qualitative designs (28/75), individual interviews were frequently applied, less so focus groups. Most of the qualitative reports aimed to explore experiences and gain insight into the views of persons who are living with a rare disorder, for example, needs of information and support, barriers to care and communication with health care providers. To present the qualitative results, a thematic analysis approach was most frequently applied. A minority of reports (5/75) reported a mixed or multi-method approach, combining interviews and surveys (see Table 3 for more details, Additional file 5 ).

Characteristics of assessment tools

Five of the assessment tools measured health literacy specifically. However, 23 standardized assessment tools aimed to assess important aspects relevant to health literacy, such as self-management skills, coping and medication adherence. Table 4 provides an overview of the standardised assessment tools used to measure health outcomes (Additional file 6 ). Quality of life was the outcome assessed most frequently (10/75) and was most commonly assessed with SF36 (4/75). Seven reports examined anxiety levels, while six estimated depression. Hospital Anxiety and Depression Scale (HAD) was the most commonly utilized tool to assess anxiety and depression (3/75). Correspondingly, 27 study-specific assessment tools sought to achieve outcomes closely related to health literacy, including health information-seeking patterns, medication information sources and knowledge, attitude and behaviour towards their condition. For a more detailed review of study-specific assessment tools, see Table 5 (Additional file 7 ).

Characteristics of interventions

A total of 16/75 of the reports included interventions. Each intervention originated from a distinct study and had diverse characteristics in terms of study design, objectives, intended recipients, implementation settings, and delivery personnel, including healthcare professionals and peers. Additional information regarding this is provided in Table 6 (Additional file 8 ). The interventions encompassed both face-to-face approaches, such as individual sessions [ 21 , 22 , 23 , 24 , 25 , 26 , 27 ] and group-based patient education [ 22 , 26 , 28 , 29 , 30 , 31 , 32 , 33 , 34 ], and written information/online training [ 29 , 31 , 35 , 36 , 37 ]. The interventions took place in a variety of settings, including hospitals, clinics, and online platforms. The common thread between the interventions is that they all share the objective of enhancing patient outcomes and experiences through education, support, and empowerment. For example, they aim to improve knowledge, health literacy, and self-treatment skills, as well as to promote treatment adherence and reduce interruptions in care. In 9/16 interventions, the primary aim was to improve knowledge or understanding of the patient’s particular health condition or treatment. These nine interventions applied various components such as audiovisual materials, individualised training courses, or booklets. Out of those nine interventions, six demonstrated a significant ( p  < 0.05) improvement in knowledge of the targeted health condition or treatment [ 21 , 25 , 28 , 32 , 36 , 37 ].

Out of all interventions, 5/16 aimed predominantly at reducing psychiatric symptoms, such as depression, anxiety, and somatic symptom severity. Several interventions displayed positive effects on mental health, including improvements in emotional health, coping strategies, and quality of life [ 22 , 26 , 28 , 29 , 32 , 33 , 35 ]. Examples of such interventions included group counselling and group mindfulness training. The interventions were evaluated using methods such as self-report questionnaires, physiological measures, and clinical assessments. The outcomes measured included improvements in physical health, mental health, quality of life, and social support.

Description of health literacy

Only 6/75 reports described in the introduction how they defined health literacy [ 21 , 38 , 39 , 40 , 41 , 42 ]. Five of these reports were based on the understanding and definition of health literacy as the cognitive and social skills that determine the motivation and ability of individuals to gain access to, understand and use information in ways that promote and maintain good health (WHO). One of the reports defined health literacy as “the patients’ skills on reading, listening, analysing decisions making and applying these skills to the situation related to health monitoring and coordination for strategy plan in term of health promotion” [ 42 ].

Access to health information and support

Most of the reports included in this review investigated knowledge or understanding of one’s own health and diagnosis, and access to health information. Persons with rare disorders commonly lack information about:

Own diagnosis and health [ 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 ].

Self-management and daily coping [ 6 , 10 , 54 , 59 , 60 , 61 , 62 , 63 ].

Medication, treatment options and research-based recommendations [ 6 , 10 , 28 , 51 , 64 , 65 , 66 , 67 ].

Peer and professional support [ 53 , 54 , 55 ].

Clinical trials and research [ 53 , 54 , 55 ].

Sexual knowledge [ 68 , 69 , 70 , 71 ].

Behaviour and attitude [ 28 , 72 , 73 ].

Social rights [ 28 , 60 ].

Pregnancy and childbirth [ 51 , 60 ].

Ageing [ 71 ].

Navigation and coordination [ 23 ].

The most important sources of health information summarized among the included reports were physicians, the internet, patient organizations and spouse/partner [ 74 , 75 , 76 ]. Transitions in life can be challenging and generate new needs for information and care. Three of the reports investigated the transition process from paediatric to adult services [ 23 , 77 , 78 ]. Persons with rare disorders and their family caregivers call for health information on various aspects of the disease burden including medical research and treatment, coping strategies, management, symptoms and general knowledge about the disease [ 57 , 63 ].

Only a few reports investigated how persons with rare disorders are navigating in healthcare and their experiences of healthcare services. These reports found that many persons with rare disorders feel let down by the system- and lack trust in the standards of health care [ 54 , 79 , 80 , 81 ]. Several reports described the frustration among persons with rare disorders because of a lack of knowledge about diagnosis and medication by healthcare professionals [ 54 , 56 , 59 , 62 , 73 , 81 , 82 ] and concerns about poor communication and information provision [ 83 ].

Some of the reports described the experiences of persons with rare disorders concerning limited access to peer- and professional support, like specialized care, treatment plans and access to peer groups [ 34 , 53 , 61 , 62 , 84 , 85 , 86 ]. Persons with rare disorders missed the engagement in health care to assist in their management of the disease [ 85 ], and one report claimed that hospital visits could be reduced with more information [ 52 ].

Several reports have investigated peer support [ 6 , 22 , 28 , 44 , 50 , 81 , 82 , 87 , 88 , 89 ]. Persons with rare disorders who connected and interacted with fellow individuals with rare disorders reported great improvements in overall health, disease severity, motivation to take care of health, emotional well-being and satisfaction with their primary treating physician [ 66 , 69 ].

This scoping review identified 75 reports presenting data on rare disorders and aspects of health literacy, thereby providing valuable insight into the characteristics of research in the field of health literacy in individuals with rare disorders. A total of 6223 individuals with rare disorders and 1707 caregivers were included, and 80 different rare disorders were represented. Most of the studies were published after 2016, and were conducted in the USA, Canada and UK. The most frequently investigated research questions were related to different health- and psychosocial outcomes, understanding of own condition, health information and support, or concerning evaluation of an intervention. The reports used a variety of research methodologies, including qualitative, quantitative, and mixed methods approaches. Cross-sectional designs were frequently employed to depict patient characteristics, knowledge and health-related variables, and qualitative designs were commonly used to capture the perspectives of persons living with rare disorders. In total 23 standardized assessment tools and 27 study-specific assessment used in the reports. Only five assessment tools measured health literacy specifically. Some of the reports also assessed interventions to improve elements such as knowledge, health literacy and coping strategies. These interventions encompassed both face-to-face approaches, such as individual sessions and group-based patient education.

Only six reports had described how they defined health literacy. Five of these reports were based on the understanding and WHO-definition of health literacy as the cognitive and social skills that determine the motivation and ability of individuals to gain access to, understand and use information in ways that promote and maintain good health (WHO). Concerning access to health information and support, the results revealed that individuals with rare disorders often exhibit gaps in knowledge and access to information related to self-management, their own diagnosis and health, as well as daily coping and social rights. In addition, the importance of aid and information from healthcare personnel and the significance of getting social support from others in the same situation were accentuated.

A recurring issue identified among the reports was that individuals with rare disorders consistently encounter challenges in accessing information on their own health and diagnosis, self-managing and coping [ 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 ]. This observation has been established in previous research and can sometimes be ascribed to a lack of knowledge among healthcare personnel [ 6 , 9 , 10 ]. The understanding and appraisal of health information could pose difficulties since the information available on rare conditions often is complex and contains medical terminology that is challenging to comprehend. This particularly applies to those with cognitive impairments, which pertains to 44% of the rare disease population [ 90 ]. More than 7000 rare disorders are identified, and only 80 of these disorders are represented in this review. More than 50% of the included reports have included study participants with Haemophilia, Cystic Fibrosis and Huntington’s disease, which means that a range of different rare disorders have not been included in health literacy research. A majority of the interventions in this study focused on increasing knowledge and understanding of one’s own health and treatment. Acquiring the skills to apply health knowledge to everyday life efficiently can profoundly impact health outcomes and is especially important when it comes to self-management, such as adherence to medication and treatment [ 15 , 91 , 92 , 93 , 94 , 95 ].

Another possible challenge related to access to information and support is the often-large geographical distances between persons with rare disorders. This may result in difficulties when it comes to meeting or participating in peer-support groups in person [ 96 ]. Peer interactions appear particularly important in this population [ 6 , 53 , 54 , 55 ], and several of the perceived benefits of the interventions in this review were associated with the recognition, acceptance and companionship encountered within peer-support groups [ 16 , 28 , 32 , 35 ]. Interestingly, none of the included reports explored the potential benefits of online peer support, which has been found to be an effective supplement to in-person meetings in people with other disorders.

While there is reason to believe that health literacy has a significant impact on health outcomes, only one of the included reports investigated this possible correlation, finding that individuals who possessed adequate health literacy displayed more favourable health-related outcomes [ 38 ]. In that report, the authors observed that individuals who possessed adequate health literacy displayed more favourable health-related outcomes. None of the included reports explored health literacy across various types of rare disorders. One prominent finding across the reviewed reports is the shortage of accessible health information specifically targeted towards individuals with rare conditions. There is a need to investigate if there are structural or social barriers that limit access to information and support for the population. Furthermore, it would be valuable to examine the underlying factors that impact health literacy in persons with rare disorders, including the association between health literacy and socio-demographic variables, health status, self-efficacy and health-related quality of life. Another potential research topic could be to evaluate the success of interventions aiming at improving health literacy in persons with rare disorders and their caregivers.

To the best of our knowledge, only four previous reports have explicitly aimed to examine the levels of health literacy in persons with rare disorders [ 38 , 39 , 40 , 42 ]. Furthermore, the data does not provide enough information to say anything about relatives’ health literacy. Enhancing health literacy is known to be an enabler for improved empowerment and participation, which is associated with positive health outcomes [ 13 , 94 ]. Empowerment is especially important in the field of rare disorders, due to the unique challenges of low prevalence, limited knowledge and expertise, and compromised quality of life [ 94 ].

To achieve a better understanding of health literacy in rare disorders, we could benefit from the incorporation of different perspectives, including those of persons with rare disorders, their family members and healthcare providers. We need future research on how different dimensions of health literacy, and interventions aiming to strengthen health literacy, influence health outcomes according to health care, disease prevention and health promotion. We need to achieve a deeper understanding of how the personal determinants of health literacy, such as individual skills and motivation, interact with situational determinants, such as social and environmental factors, to shape health outcomes. To properly address the executive challenges faced by persons with rare disorders we need a greater understanding of health literacy in rare disorders [ 28 ]. The integrated model of health literacy [ 15 ] can serve as a tool to point us in the right direction when designing future research projects.

The strength of our work lies in providing a comprehensive overview of the reported findings from research on health literacy in rare disorders. We conducted an up-to-date systematic search in five databases without restrictions. Despite using an array of synonyms in database searches to maximise the identification of relevant reports, the search terms used are not exhaustive. Hence, some reports may not have been detected. To reduce the risk of selection bias, two authors independently assessed the abstracts and reports in full text according to the a priori eligibility criteria. Further, in line with the scoping review framework, we have not evaluated the methodological quality or risk of bias among the included reports. This may be seen as a limitation; however, the purpose of scoping reviews is to give an overview of the available research literature, characterise a research area and pinpoint gaps in knowledge that should be addressed in future systematic reviews.

This review has important implications for practice. Healthcare does not offer curative treatment options for most rare disorders, and several reports suggest the development of consensus recommendations for care. To optimise health and secure continuity of care several reports included in this review recommend formalisation of the transition process through the courses of illness and life. Moreover, the results reveal that some of the key challenges for persons with rare disorders are related to important aspects of health literacy, such as accessing, understanding, and applying health information. Our findings indicate a need for strengthened health literacy in the rare disease population, that could be accomplished by developing health communication strategies tailored to the needs and preferences of persons with rare conditions. Healthcare personnel can play a significant role in enhancing health literacy, which is an additional implication for practice. Health care personnel can achieve this by offering clear and understandable health-related information and encouraging an active dialogue between patients and professionals. Another way for healthcare personnel to assist persons with rare disorders is by offering them the support needed to accept, cope, and effectively manage their condition [ 97 ].

This scoping review consists of 75 reports presenting data on rare disorders and aspects of health literacy, thereby providing valuable insight into the characteristics of research in the field of health literacy in individuals with rare disorders. In total, 6223 individuals with rare disorders and 1707 caregivers were included, and 80 different rare disorders were represented. Most of the studies were published after 2016, and were conducted in the USA, Canada and UK.

The findings of this scoping review demonstrate that persons with rare disorders experience considerable gaps in knowledge and information, particularly in relation to their own diagnosis and health, treatment options, self-management and coping strategies. Moreover, the lack of diagnosis-specific knowledge and limited information provided by healthcare professionals are identified as a common concern among persons with rare disorders. Access to, and understanding, health information is key aspects of health literacy. Therefore, our results imply a need for increased awareness regarding the state of health literacy among individuals with rare disorders. The points of view expressed in this review offer valuable perspectives that can help health personnel in outlining the communicative strategy when caring for individuals with rare disorders.

This review provides a solid understanding block for future research into the emerging field of health literacy in rare disorders, by examining the challenges that persons with rare conditions encounter. Moreover, the findings enable us to develop a better understanding of the care and support persons with a rare disorder and their family members require.

These results pave the way for future research that looks to improve the healthcare experience of those with rare disorders and their caretakers and shed light on the importance of empowering the rare disease population through peer support, participation, education and increased health literacy. Future reports in this field are necessary to develop strategies and interventions that improve health literacy and enhance health outcomes and the quality of life for individuals with rare disorders.

Data availability

All data generated or analysed during this review are included in this published article (and its additional files).

EURORDIS. What is a rare disease? [Internet]. EURORDIS. 2022 [cited 2023 Jun 2]. https://www.eurordis.org/information-support/what-is-a-rare-disease/

Haendel M, Vasilevsky N, Unni D, Bologa C, Harris N, Rehm H, et al. How many rare diseases are there? Nat Rev Drug Discov. 2020;19:77–8.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Nguengang Wakap S, Lambert DM, Olry A, Rodwell C, Gueydan C, Lanneau V, et al. Estimating cumulative point prevalence of rare diseases: analysis of the Orphanet database. Eur J Hum Genet. 2020;28:165–73.

Article   PubMed   Google Scholar  

Rare diseases [Internet]. European Commission. [cited 2023 Jun 6]. https://health.ec.europa.eu/non-communicable-diseases/expert-group-public-health/rare-diseases_en

The Ministry of Health and Care Services. Nasjonal strategi for sjeldne diagnoser [Internet]. 2021 Aug. https://www.regjeringen.no/no/dokumenter/nasjonal-strategi-for-sjeldne-diagnoser/id2867121/

Stanarević Katavić S. Health information behaviour of rare disease patients: seeking, finding and sharing health information. Health Info Libr J. 2019;36:341–56.

Zhu Q, Nguyễn Ð-T, Sheils T, Alyea G, Sid E, Xu Y, et al. Scientific evidence based rare disease research discovery with research funding data in knowledge graph. Orphanet J Rare Dis. 2021;16:483.

Article   PubMed   PubMed Central   Google Scholar  

Stoller JK. The challenge of Rare diseases. Chest. 2018;153:1309–14.

von der Lippe C, Diesen PS, Feragen KB. Living with a rare disorder: a systematic review of the qualitative literature. Mol Genet Genomic Med. 2017;5:758–73.

Kesselheim AS, McGraw S, Thompson L, O’Keefe K, Gagne JJ. Development and use of new therapeutics for rare diseases: views from patients, caregivers, and advocates. Patient. 2015;8:75–84.

Kickbusch I, Pelikan J, Apfel F, Tsouros AD. Health literacy. The solid facts [Internet]. WHO Regional Office for Europe, Copenhagen; 2013. https://apps.who.int/iris/handle/10665/326432

Sør-Øst H. Regional utviklingsplan 2040 [Internet]. 2022 [cited 2023 May 25]. https://helse-sorost.no/regional-utviklingsplan-2040

Omsorgsdepartementet H-. og. Strategi for å øke helsekompetansen i befolkningen 2019–2023. 2019 [cited 2023 May 25]; https://www.regjeringen.no/no/dokumenter/strategi-for-a-oke-helsekompetansen-i-befolkningen-2019-2023/id2644707/

Kickbusch I, Maag D. Health Literacy [Internet]. International encyclopedia of public health. Elsevier; 2008. pp. 204–11. https://doi.org/10.1016/b978-012373960-5.00584-0

Sørensen K, Van den Broucke S, Fullam J, Doyle G, Pelikan J, Slonska Z et al. Health literacy and public health: a systematic review and integration of definitions and models. BMC Public Health [Internet]. 2012;12. https://doi.org/10.1186/1471-2458-12-80

Bogart KR, Irvin VL. Health-related quality of life among adults with diverse rare disorders. Orphanet J Rare Dis. 2017;12:177.

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8:19–32.

Article   Google Scholar  

Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing the methodology. Implement Sci. 2010;5:69.

Daudt HML, van Mossel C, Scott SJ. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. BMC Med Res Methodol. 2013;13:48.

Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for scoping reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med. 2018;169:467–73.

Bhatt N, Boggio L, Simpson ML. Using an educational intervention to assess and improve disease-specific knowledge and health literacy and numeracy in adolescents and young adults with haemophilia A and B. Haemophilia. 2021;27:229–36.

Article   CAS   PubMed   Google Scholar  

Bogart KR, Frandrup E, Locke T, Thompson H, Weber N, Yates J et al. Rare place where I feel normal: Perceptions of a social support conference among parents of and people with Moebius syndrome. Res Dev Disabil. 2017;64:143–51.

Chaudhry SR, Keaton M, Nasr SZ. Evaluation of a cystic fibrosis transition program from pediatric to adult care. Pediatr Pulmonol. 2013;48:658–65.

Dicianno BE, Lovelace J, Peele P, Fassinger C, Houck P, Bursic A, et al. Effectiveness of a Wellness Program for individuals with Spina Bifida and spinal cord Injury within an Integrated Delivery System. Arch Phys Med Rehabil. 2016;97:1969–78.

le Doré S, Grinda N, Ferré E, Roussel-Robert V, Frotscher B, Chamouni P, et al. The hemarthrosis-simulating knee model: a useful tool for individualized education in patients with hemophilia (GEFACET study). J Blood Med. 2021;12:133–8.

Ringqvist K, Borg K, Möller MC. Tolerability and psychological effects of a multimodal day-care rehabilitation program for persons with Huntington’s disease. J Rehabil Med. 2021;53:jrm00143.

van Balen EC, Krawczyk M, Gue D, Jackson S, Gouw SC, van der Bom JG, et al. Patient-centred care in haemophilia: patient perspectives on visualization and participation in decision-making. Haemophilia. 2019;25:938–45.

Chaleat-Valayer E, Amélie Z, Marie-Hélène B, Perretant I, Sandrine T. Therapeutic education program for patients with hypermobile Ehlers-Danlos syndrome: feasibility and satisfaction of the participants. Education thérapeutique Du patient -. Therapeutic Patient Educ. 2019;11:10202.

Hoefnagels JW, Fischer K, Bos RAT, Driessens MHE, Meijer SLA, Schutgens REG, et al. A feasibility study on two tailored interventions to improve adherence in adults with haemophilia. Pilot Feasibility Stud. 2020;6:189.

O’Mahar K, Holmbeck GN, Jandasek B, Zukerman J. A camp-based intervention targeting independence among individuals with spina bifida. J Pediatr Psychol. 2010;35:848–56.

Raphaelis S, Mayer H, Ott S, Hornung R, Senn B. Effects of Written Information and Counseling on illness-related uncertainty in Women with Vulvar Neoplasia. Oncol Nurs Forum. 2018;45:748–60.

PubMed   Google Scholar  

Rovira-Moreno E, Abuli A, Codina-Sola M, Valenzuela I, Serra-Juhe C, Cuscó I, et al. Beyond the disease itself: a cross-cutting educational initiative for patients and families with rare diseases. J Genet Couns. 2021;30:693–700.

Stubberud J, Langenbahn D, Levine B, Stanghelle J, Schanke A-K. Emotional health and coping in spina bifida after goal management training: a randomized controlled trial. Rehabil Psychol. 2015;60:1–16.

Delisle VC, Gumuchian ST, Pelaez S, Malcarne VL, El-Baalbaki G, Körner A, et al. Reasons for non-participation in scleroderma support groups. Clin Exp Rheumatol. 2016;34(Suppl 100):56–62.

Depping MK, Uhlenbusch N, Härter M, Schramm C, Löwe B. Efficacy of a brief, peer-delivered self-management intervention for patients with Rare Chronic diseases: a Randomized Clinical Trial. JAMA Psychiatry. 2021;78:607–15.

Mulders G, de Wee EM, Vahedi Nikbakht-Van, de Sande MCVM, Kruip MJHA, Elfrink EJ, Leebeek FWG. E-learning improves knowledge and practical skills in haemophilia patients on home treatment: a randomized controlled trial. Haemophilia. 2012;18:693–8.

Smolich L, Charen K, Sherman SL. Health knowledge of women with a fragile X premutation: improving understanding with targeted educational material. J Genet Couns. 2020;29:983–91.

Jackson AD, Kirwan L, Gibney S, Jeleniewska P, Fletcher G, Doyle G. Associations between health literacy and patient outcomes in adolescents and young adults with cystic fibrosis. Eur J Public Health. 2020;30:112–8.

Merker VL, McDannold S, Riklin E, Talaei-Khoei M, Sheridan MR, Jordan JT, et al. Health literacy assessment in adults with neurofibromatosis: electronic and short-form measurement using FCCHL and Health LiTT. J Neurooncol. 2018;136:335–42.

Riklin E, Talaei-Khoei M, Merker VL, Sheridan MR, Jordan JT, Plotkin SR, et al. First report of factors associated with satisfaction in patients with neurofibromatosis. Am J Med Genet A. 2017;173:671–7.

LaDonna KA, Ghavanini AA, Venance SL. Truths and misinformation: a qualitative exploration of myotonic dystrophy. Can J Neurol Sci. 2015;42:187–94.

Parvizi MM, Lankarani KB, Handjani F, Ghahramani S, Parvizi Z, Rousta S. Health literacy in patients with epidermolysis bullosa in Iran. J Educ Health Promot. 2017;6:105.

Laberge L, Prévost C, Perron M, Mathieu J, Auclair J, Gaudreault M, et al. Clinical and genetic knowledge and attitudes of patients with myotonic dystrophy type 1. Public Health Genomics. 2010;13:424–30.

Rosnau K, Hashmi SS, Northrup H, Slopis J, Noblin S, Ashfaq M. Knowledge and self-esteem of individuals with neurofibromatosis type 1 (NF1). J Genet Couns. 2017;26:620–7.

Ioannou L, Massie J, Collins V, McClaren B, Delatycki MB. Population-based genetic screening for cystic fibrosis: attitudes and outcomes. Public Health Genomics. 2010;13:449–56.

Lewis KL, John B, Condren M, Carter SM. Evaluation of medication-related self-care skills in patients with cystic fibrosis. J Pediatr Pharmacol Ther. 2016;21:502–11.

PubMed   PubMed Central   Google Scholar  

Lindvall K, Colstrup L, Loogna K, Wollter I, Grönhaug S. Knowledge of disease and adherence in adult patients with haemophilia. Haemophilia. 2010;16:592–6.

Lonabaugh KP, O’Neal KS, McIntosh H, Condren M. Cystic fibrosis-related education: are we meeting patient and caregiver expectations? Patient Educ Couns. 2018;101:1865–70.

Mälstam E, Bensing S, Asaba E. Everyday managing and living with autoimmune Addison’s disease: exploring experiences using photovoice methods. Scand J Occup Ther. 2018;25:358–70.

Naik H, Shenbagam S, Go AM, Balwani M. Psychosocial issues in erythropoietic protoporphyria - the perspective of parents, children, and young adults: a qualitative study. Mol Genet Metab. 2019;128:314–9.

Takeuchi T, Muraoka K, Yamada M, Nishio Y, Hozumi I. Living with idiopathic basal ganglia calcification 3: a qualitative study describing the lives and illness of people diagnosed with a rare neurological disease. Springerplus. 2016;5:1713.

De la Corte-Rodriguez H, Rodriguez-Merchan EC, Alvarez-Roman T, Martin-Salces M, Garcia-Barcenilla S, Jimenez-Yuste V. Health education and empowerment in adult patients with haemophilia. Expert Rev Hematol. 2019;12:989–95.

Braisch U, Martinez-Horta S, MacDonald M, Orth M. Important but not enough - information about HD related topics and peer and professional support for young adults from HD families. J Huntingtons Dis. 2016;5:379–87.

Bryson B, Bogart K, Atwood M, Fraser K, Locke T, Pugh K, et al. Navigating the unknown: a content analysis of the unique challenges faced by adults with rare diseases. J Health Psychol. 2021;26:623–35.

Coathup V, Teare HJA, Minari J, Yoshizawa G, Kaye J, Takahashi MP, et al. Using digital technologies to engage with medical research: views of myotonic dystrophy patients in Japan. BMC Med Ethics. 2016;17:51.

Etchegary H. Healthcare experiences of families affected by Huntington disease: need for improved care. Chronic Illn. 2011;7:225–38.

Mohan R, Radhakrishnan N, Varadarajan M, Anand S. Assessing the current knowledge, attitude and behaviour of adolescents and young adults living with haemophilia. Haemophilia. 2021;27:e180–6.

Garrino L, Picco E, Finiguerra I, Rossi D, Simone P, Roccatello D. Living with and treating rare diseases: experiences of patients and professional health care providers. Qual Health Res. 2015;25:636–51.

Mooney J, Poland F, Spalding N, Scott DGI. In One Ear and Out the Other–Its a Lot to Take in’: A Qualitative Study Exploring the Informational Needs of Patients with ANCA-Associated Vasculitis. Musculoskeletal [Internet]. 2013; https://onlinelibrary.wiley.com/doi/abs/ https://doi.org/10.1002/msc.1030

David V, Feldman D, Danner-Boucher I, Rhun AL, Guyomarch B, Ravilly S, et al. Identifying the educational needs of lung transplant recipients with cystic fibrosis. Prog Transpl. 2015;25:18–25.

Gumuchian ST, Peláez S, Delisle VC, Carrier M-E, Jewett LR, El-Baalbaki G, et al. Understanding coping strategies among people living with scleroderma: a focus group study. Disabil Rehabil. 2018;40:3012–21.

Arya S, Wilton P, Page D, Boma-Fischer L, Floros G, Winikoff R, et al. They don’t really take my bleeds seriously: barriers to care for women with inherited bleeding disorders. J Thromb Haemost. 2021;19:1506–14.

Kurtz NS, Cote C, Heatwole C, Gagnon C, Youssof S. Patient-reported disease burden in oculopharyngeal muscular dystrophy. Muscle Nerve. 2019;60:724–31.

Pakhale S, Baron J, Armstrong M, Tasca G, Gaudet E, Aaron SD, et al. Lost in translation? How adults living with cystic fibrosis understand treatment recommendations from their healthcare providers, and the impact on adherence to therapy. Patient Educ Couns. 2016;99:1319–24.

Shepherd LM, Tahrani AA, Inman C, Arlt W, Carrick-Sen DM. Exploration of knowledge and understanding in patients with primary adrenal insufficiency: a mixed methods study. BMC Endocr Disord. 2017;17:47.

Arran N, Craufurd D, Simpson J. Illness perceptions, coping styles and psychological distress in adults with Huntington’s disease. Psychol Health Med. 2014;19:169–79.

Dellon EP, Helms SW, Hailey CE, Shay R, Carney SD, Schmidt HJ, et al. Exploring knowledge and perceptions of palliative care to inform integration of palliative care education into cystic fibrosis care. Pediatr Pulmonol. 2018;53:1218–24.

Shoshan L, Ben-Zvi D, Meyer S, Katz-Leurer M. Sexuality in relation to independence in daily functions among young people with spina bifida living in Israel. Rehabil Nurs. 2012;37:11–7. quiz 17–8.

Walsh MB, Charen K, Shubeck L, McConkie-Rosell A, Ali N, Bellcross C, et al. Men with an FMR1 premutation and their health education needs. J Genet Couns. 2021;30:1156–67.

Sylvain C, Lamothe L, Berthiaume Y, Rabasa-Lhoret R. How patients’ representations of cystic fibrosis-related diabetes inform their health behaviours. Psychol Health. 2016;31:1129–44.

Arnold E, Lane S, Webert KE, Chan A, Walker I, Tufts J, et al. What should men living with haemophilia need to know? The perspectives of Canadian men with haemophilia. Haemophilia. 2014;20:219–25.

Torres-Ortuño A, Cuesta-Barriuso R, Nieto-Munuera J, Galindo-Piñana P, López-Pina J-A. The behaviour and perception of illness: modulating variables of adherence in patients with haemophilia. Vox Sang [Internet]. 2018; https://doi.org/10.1111/vox.12669

Keyte R, Egan H, Nash EF, Regan A, Jackson C, Mantzios M. An exploration into experiences and attitudes regarding risky health behaviours in an adult cystic fibrosis population. Psychol Health Med. 2020;25:1013–9.

Carpenter DM, DeVellis RF, Hogan SL, Fisher EB, DeVellis BM, Jordan JM. Use and perceived credibility of medication information sources for patients with a rare illness: differences by gender. J Health Commun. 2011;16:629–42.

Dwyer AA, Quinton R, Morin D, Pitteloud N. Identifying the unmet health needs of patients with congenital hypogonadotropic hypogonadism using a web-based needs assessment: implications for online interventions and peer-to-peer support. Orphanet J Rare Dis. 2014;9:83.

Litzkendorf S, Frank M, Babac A, Rosenfeldt D, Schauer F, Hartz T, et al. Use and importance of different information sources among patients with rare diseases and their relatives over time: a qualitative study. BMC Public Health. 2020;20:860.

Molster C, Urwin D, Di Pietro L, Fookes M, Petrie D, van der Laan S, et al. Survey of healthcare experiences of Australian adults living with rare diseases. Orphanet J Rare Dis. 2016;11:30.

Lindsay S, Fellin M, Cruickshank H, McPherson A, Maxwell J. Youth and parents’ experiences of a new inter-agency transition model for spina bifida compared to youth who did not take part in the model. Disabil Health J. 2016;9:705–12.

Skirton H, Williams JK, Jackson Barnette J, Paulsen JS. Huntington disease: families’ experiences of healthcare services. J Adv Nurs. 2010;66:500–10.

Domaradzki J. Family caregivers’ experiences with healthcare services–a case of Huntington’s disease. Psychiatr Pol. 2016;50:375–91.

Katavic SS, Tanackovic SF, Badurina B. Illness perception and information behaviour of patients with rare chronic diseases. Inflamm Res [Internet]. 2016 [cited 2023 Jun 14];21. https://doi.org/10.1111/hir.12261

Socha Hernandez AV, Deeks LS, Shield AJ. Understanding medication safety and Charcot-Marie-tooth disease: a patient perspective. Int J Clin Pharm. 2020;42:1507–14.

Akanuwe JNA, Laparidou D, Curtis F, Jackson J, Hodgson TL, Siriwardena AN. Exploring the experiences of having Guillain-Barré syndrome: a qualitative interview study. Health Expect. 2020;23:1338–49.

Foley G, Timonen V, Hardiman O. Understanding psycho-social processes underpinning engagement with services in motor neurone disease: a qualitative study. Palliat Med. 2014;28:318–25.

Borghi L, Moreschi C, Toscano A, Comber P, Vegni E, The PKU. ME study: a qualitative exploration, through co-creative sessions, of attitudes and experience of the disease among adults with phenylketonuria in Italy. Mol Genet Metabolism Rep. 2020;23:100585.

CAS   Google Scholar  

Kazmerski TM, Gmelin T, Slocum B, Borrero S, Miller E. Attitudes and decision making related to pregnancy among Young women with cystic fibrosis. Matern Child Health J. 2017;21:818–24.

Flewelling KD, Sellers DE, Sawicki GS, Robinson WM, Dill EJ. Social support is associated with fewer reported symptoms and decreased treatment burden in adults with cystic fibrosis. J Cyst Fibros. 2019;18:572–6.

Kirk S, Milnes L. An exploration of how young people and parents use online support in the context of living with cystic fibrosis. Health Expect. 2016;19:309–21.

Salvatore V, Gilstrap A, Williams KR, Thorat S, Stevenson M, Gwosdow AR, et al. Evaluating the impact of peer support and connection on the quality of life of patients with familial chylomicronemia syndrome. Expert Opin Orphan Drugs. 2018;6:497–505.

Article   CAS   Google Scholar  

López-Bastida J, Oliva-Moreno J, Linertová R, Serrano-Aguilar P. Social/economic costs and health-related quality of life in patients with rare diseases in Europe. Eur J Health Economics: HEPAC: Health Econ Prev care. 2016;17(Suppl 1):1–5.

Sørensen K, Van den Broucke S, Pelikan JM, Fullam J, Doyle G, Slonska Z et al. Measuring health literacy in populations: illuminating the design and development process of the European Health Literacy Survey Questionnaire (HLS-EU-Q). BMC Public Health [Internet]. 2013;13. https://doi.org/10.1186/1471-2458-13-948

Berkman ND, Sheridan SL, Donahue KE, Halpern DJ, Crotty K. Low health literacy and health outcomes: an updated systematic review. Ann Intern Med. 2011;155:97–107.

Geboers B, Reijneveld SA, Jansen CJM, de Winter AF. Health Literacy Is Associated With Health Behaviors and Social factors among older adults: results from the LifeLines Cohort Study. J Health Commun. 2016;21:45–53.

Mackey LM, Doody C, Werner EL, Fullen B. Self-management skills in Chronic Disease Management: what role does health literacy have? Med Decis Mak. 2016;36:741–59.

Paasche-Orlow MK, Wolf MS. The causal pathways linking health literacy to health outcomes. Am J Health Behav. 2007;31:19–26.

de Vrueh R, de Baekelandt Erf JMH. Background paper 6.19. Rare diseases. Geneva: World Health Organization; 2013.

Google Scholar  

Uhlenbusch N, Löwe B, Härter M, Schramm C, Weiler-Normann C, Depping MK. Depression and anxiety in patients with different rare chronic diseases: a cross-sectional study. PLoS ONE. 2019;14:e0211343.

Download references

Acknowledgements

The authors want to thank leaders and colleagues. A special thank goes to librarian Hilde Iren Flaatten who has conducted systematic searches after literature and Anne Siri Albrigtsen for important contributions to this review. Several of the authors are members of Euro-NMD.

This work received funding from Norwegian National Advisory Unit on Rare Disorders. In addition the work was performed as part of regular activities at Frambu Resource Centre for Rare Disorders and Norwegian National Advisory Unit on Learning and Mastery in Health, Oslo University Hospital.

Author information

Authors and affiliations.

Frambu Resource Center for Rare Disorders, Sandbakkveien 18, Siggerud, 1404, Norway

Una Stenberg & Lydia Westfal

Norwegian National Advisory Unit on Learning and Mastery in Health, Oslo University Hospital, Postboks 4959 Nydalen, Oslo, 0424, Norway

Una Stenberg & Karl Fredrik Westermann

National Neuromuscular Centre Norway, University Hospital of North-Norway, Hansine Hansens vei 37, Tromsø, 9019, Norway

Andreas Dybesland Rosenberger

Section for Rare Neuromuscular Disorders and Unit for Congenital and Hereditary Neuromuscular Disorders (EMAN), Department of Neurology, Oslo University Hospital, Postboks 4950 Nydalen, Oslo, 0424, Norway

Kristin Ørstavik

Department of Neurobiology, Care Sciences and Society, Karolinska Institutet, Solnavägen 1, Solna, 171 77, Sweden

Maria Flink

Oslo Metropolitan University, Postbox 4, St. Olavs place, Oslo, N-0130, Norway

Heidi Holmen

National Centre for Rare Epilepsy-Related Disorders, Department of Rare Disorders, Division of Paediatric and Adolescent Medicine, Oslo University Hospital, Postboks, 4950 Nydalen, 0424, Oslo, Norway

Silje Systad

TRS Resource Centre for Rare Diseases, Sunnaas Rehabilitation Hospital, Bjørnemyrveien 11, 1453, Bjørnemyr, Norway

You can also search for this author in PubMed   Google Scholar

Contributions

All the authors (US, GV, SS, KØ, KFW, HH, ADR, MF) except LW have contributed in the conception, design and analysis of data and interpretation of data. US has led all the phases of the review, and LW has contributed in the analysis and interpretation of data, and in writing the manuscript. All authors have read and approved the final manuscript.

Corresponding author

Correspondence to Una Stenberg .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, supplementary material 3, supplementary material 4, supplementary material 5, supplementary material 6, supplementary material 7, supplementary material 8, supplementary material 9, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Stenberg, U., Westfal, L., Dybesland Rosenberger, A. et al. A scoping review of health literacy in rare disorders: key issues and research directions. Orphanet J Rare Dis 19 , 328 (2024). https://doi.org/10.1186/s13023-024-03332-5

Download citation

Received : 25 October 2023

Accepted : 21 August 2024

Published : 06 September 2024

DOI : https://doi.org/10.1186/s13023-024-03332-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Health literacy
• Rare disorders
• Rare disease
• Rare genetic disorder
• Rare developmental defect
• Integrative model of health literacy
• Scoping review

Orphanet Journal of Rare Diseases

ISSN: 1750-1172

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

Understanding scoping reviews: Definition, purpose, and process

Affiliation.

• 1 School of Nursing, Loyola University New Orleans, New Orleans, Louisiana.
• PMID: 27245885
• DOI: 10.1002/2327-6924.12380

Background/purpose: Scoping review design represents a methodology that allows assessment of emerging evidence, as well as a first step in research development. Despite its increasing use, to date no article reflecting use of scoping review methodology has been submitted for review at JAANP. The purpose of this article is to provide detailed information on scoping reviews, including definition, related processes, and uses, and discuss the relationship to nurse practitioner (NP) practice, policy, education, and research. The longer-term goal is that NPs will understand the related techniques, consider the methodology as a viable one for NP scholarship, and bring related reports to the forefront of NP publications.

Methods: This manuscript represents a brief report. Processes to develop the brief include detailed search and review of scoping review literature in CINAHL and PubMed. Both methodologic reports and reviews were included. Definitions and uses of scoping reviews were reviewed.

Conclusion: The definition and process of scoping review are evolving. Although there is controversy regarding the methodology, there is increasing visibility of scoping review methodology in the published literature since the year 2000, with over 500 published reviews currently available.

Implications for practice: A well-executed scoping review has potential to inform NP practice, policy, education, and research.

Keywords: Scoping review; concept mapping; evidence base; research methods; review.

©2016 American Association of Nurse Practitioners.

PubMed Disclaimer

Similar articles

• A scoping review on the conduct and reporting of scoping reviews. Tricco AC, Lillie E, Zarin W, O'Brien K, Colquhoun H, Kastner M, Levac D, Ng C, Sharpe JP, Wilson K, Kenny M, Warren R, Wilson C, Stelfox HT, Straus SE. Tricco AC, et al. BMC Med Res Methodol. 2016 Feb 9;16:15. doi: 10.1186/s12874-016-0116-4. BMC Med Res Methodol. 2016. PMID: 26857112 Free PMC article.
• An Evidence-Based Approach to Scoping Reviews. Khalil H, Peters M, Godfrey CM, McInerney P, Soares CB, Parker D. Khalil H, et al. Worldviews Evid Based Nurs. 2016 Apr;13(2):118-23. doi: 10.1111/wvn.12144. Epub 2016 Jan 28. Worldviews Evid Based Nurs. 2016. PMID: 26821833
• Scoping reviews in occupational therapy: the what, why, and how to. McKinstry C, Brown T, Gustafsson L. McKinstry C, et al. Aust Occup Ther J. 2014 Apr;61(2):58-66. doi: 10.1111/1440-1630.12080. Epub 2013 Oct 11. Aust Occup Ther J. 2014. PMID: 24125011
• A scoping review of scoping reviews: advancing the approach and enhancing the consistency. Pham MT, Rajić A, Greig JD, Sargeant JM, Papadopoulos A, McEwen SA. Pham MT, et al. Res Synth Methods. 2014 Dec;5(4):371-85. doi: 10.1002/jrsm.1123. Epub 2014 Jul 24. Res Synth Methods. 2014. PMID: 26052958 Free PMC article. Review.
• Study protocol for a scoping review on rehabilitation scoping reviews. Colquhoun HL, Jesus TS, O'Brien KK, Tricco AC, Chui A, Zarin W, Lillie E, Hitzig SL, Straus S. Colquhoun HL, et al. Clin Rehabil. 2017 Sep;31(9):1249-1256. doi: 10.1177/0269215516688514. Epub 2017 Jan 25. Clin Rehabil. 2017. PMID: 28118743 Review.
• Adherence to Cardiac Rehabilitation in Patients with Acute Myocardial Infarction After PCI: A Scoping Review. Li P, Zhang W, Wu B. Li P, et al. J Multidiscip Healthc. 2024 Aug 28;17:4165-4176. doi: 10.2147/JMDH.S483512. eCollection 2024. J Multidiscip Healthc. 2024. PMID: 39220330 Free PMC article. Review.
• Psychosocial interventions for improving the physical health of young people and adults with attention deficit hyperactivity disorder: a scoping review. Ward JH, McBride A, Price A, Delgado TN. Ward JH, et al. BMC Psychiatry. 2024 Aug 20;24(1):569. doi: 10.1186/s12888-024-06009-2. BMC Psychiatry. 2024. PMID: 39164688 Free PMC article. Review.
• Potential of Massage Therapy for Improved Growth and Development Among Infants Under 9 Months: A Systematic Scoping Review of Intervention Type, Technique, and Outcome. Rakhmawati W, Mediani HS, Dhamayanti M, Maryati I, Setiawan AS, Hendrawati S, Maryam NNA, Hasanah A, Mulyana AM, Mariani D, Rochimat I. Rakhmawati W, et al. J Multidiscip Healthc. 2024 Aug 13;17:3931-3943. doi: 10.2147/JMDH.S475731. eCollection 2024. J Multidiscip Healthc. 2024. PMID: 39155976 Free PMC article. Review.
• Knowledge about research and facilitation of co-creation with children. Protocol for the article "scoping review of research about co-creation with children". Samland B, Larsen T, Pedersen L. Samland B, et al. PLoS One. 2024 Aug 9;19(8):e0307766. doi: 10.1371/journal.pone.0307766. eCollection 2024. PLoS One. 2024. PMID: 39121109 Free PMC article. Review.
• Collaboration, coordination and communication as facilitators of transitions for patients with advanced cancer: a scoping review linked to the Pal-Cycles project. Hooley RL, Payne S, Begovic D, Correa-Morales JE, Harding A, Hasselaar J, Preston N. Hooley RL, et al. BMC Palliat Care. 2024 Aug 7;23(1):204. doi: 10.1186/s12904-024-01510-7. BMC Palliat Care. 2024. PMID: 39112983 Free PMC article. Review.
• Search in MeSH

LinkOut - more resources

Full text sources.

• Ovid Technologies, Inc.
• Wolters Kluwer

Other Literature Sources

• scite Smart Citations

Miscellaneous

• NCI CPTAC Assay Portal

• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

• Open access
• Published: 12 September 2024

Quality indicators for occupational therapy: a scoping review

• Thomas Ballmer 1 ,
• Sara Frey 1 ,
• Andrea Petrig 2 &
• Brigitte E. Gantschnig 1 , 3  

BMC Health Services Research volume  24 , Article number:  1054 ( 2024 ) Cite this article

Metrics details

Occupational therapists are increasingly asked to demonstrate the effectiveness, appropriateness, and efficiency of their interventions to funding bodies. However, the extent to which this is practiced and the health policy context within which such a practice is situated differs internationally. The aim of this scoping review was to establish which quality indicators are used internationally for this purpose.

We conducted a scoping review, limiting our search to Europe and the English-speaking world. To search for suitable literature, we used specialized databases from medicine, health sciences, and related fields, including CINAHL Complete and MEDLINE, as well as free internet search via Google. Furthermore, we contacted national occupational therapy associations from several countries asking for access to documents found within this search that were only accessible to association members.

The screening process identified 32 studies and documents from six national contexts. We identified and described process-level indicators, functional outcome indicators, one outcome indicator based on individual goal attainment (the Goal Attainment Scale, or GAS), and PRO-Ergo, a patient-reported experience measure (PREM). There was little information on the use of quality indicators to demonstrate the effectiveness, appropriateness, and efficiency of occupational therapy services to funding bodies in Europe and the English-speaking world that was openly available. The identified process indicators were in most cases not specific to occupational therapy interventions. Functional outcome indicators were highly specific to certain client groups or health conditions and partially appropriate for use in occupational therapy. The GAS was found to be a highly customizable measure which allowed an evaluation on body structure and function levels as well as activity and participation levels. PRO-Ergo was focused on the clients’ subjective view and their experience with occupational therapy interventions.

Conclusions

All identified quality indicators have advantages and disadvantages. Process-level indicators specific to occupational therapy could be a chance to foster the use of best practice methods. GAS and PRO-Ergo seem to be the most versatile assessment, allowing an evaluation on the level of activity and participation. Functional outcome indicators that cover a broad area of client problems may be useful additional quality indicators for some areas of practice.

Peer Review reports

Against a background of rising health care costs and increasing demands for health care services, the need to improve efficiency, appropriateness, and effectiveness of health care service provision is becoming more pressing [ 1 ]. In these efforts, quality indicators play a key role. Quality indicators are measures that allow for the quantification of the quality of different aspects of the structures, processes, and outcomes of health care provision [ 2 , 3 , 4 ]. Structural indicators operate on a health system or organizational level. They commonly refer to the use and/or accessibility of resources, for instance the rate of patients per doctor or access to specialised health technologies [ 4 ]. Process-level indicators represent how and what kind of services have been provided to patients, for instance the “Proportion of patients assessed by a doctor within 24 hours of referral” or the “Proportion of patients treated according to clinical guidelines” [ 4 ]. Outcome indicators refer to the results of these services, for instance the change in functional status for patients with knee impairments [ 5 ].

These indicators can be reported in the form of rates or proportions (e.g., percentage of patients 65 years old or older discharged home within 4 weeks following hospital treatment for hip fracture) or of means or averages (e.g., patients’ mean improvement on a test of mobility from admission to discharge) [ 4 , 6 ]. The interpretation of these measures allows stakeholders within health systems to identify areas of service provision that excel or lack in terms of efficiency, appropriateness, and effectiveness. This information can serve as a basis for health systems-related decision-making [ 6 ], be it the direct allocation of resources or the creation of incentives for improvement, sometimes by tying reimbursement for various actors (e.g., hospitals, doctors/ therapists in private practice) to their performance on quality indicators [ 2 ]. Thus, the way that these quality indicators are designed has far reaching consequences for health policy, but also the day-to-day reality of health care professionals, as well as the health and well-being of their patients [ 2 ].

Occupational therapists are no exception when it comes to this increasing demand to demonstrate the effectiveness, appropriateness, and efficiency of health care services. Switzerland is one of the countries where this demand is currently being codified into law more explicitly. The Swiss Federal Act on Health Insurance (HIA; Art. 58 et seq.) and the Ordinance on Compulsory Health Care (OAMal; Art. 77, para 1), revised in 2021, obliges service providers and health insurers to enter into nationwide contractual agreements on quality development. These quality contracts regulate, among other things, the measurement of quality, and therefore require the definition of suitable quality indicators.

However, the question what quality indicators are ‘suitable’ is not always easily answered. Quality indicators are often highly specific to a certain health care setting or patient group and cannot be easily adapted to other areas or groups [ 2 ], which in turn creates the need for a large amount of different indicators. Furthermore, it has been criticized that not all quality indicators are appropriate to represent the contributions of different health professions to health outcomes on different levels specific to those professions (e.g., body function vs. activity/ participation levels) [ 7 ]. It stands to reason that quality indicators used to demonstrate quality of service provision to funding bodies need to be able to reflect plausible results of the intervention that is being funded.

This scoping review was commissioned by the Swiss National Association of Occupational Therapy (EVS/ASE) to serve as a basis for the definition of quality indicators in contractual quality agreements with the two large health insurance associations in Switzerland, focusing on process-level and outcome indicators. For the aforementioned reasons, the results of this review are, however, of interests not only to occupational therapists and their partners in the health care system, but to health professionals in general.

The aim of this scoping review was to establish an overview of quality indicators that are being used internationally (focusing on Europe and the English-speaking world) to demonstrate the effectiveness, appropriateness, and/or efficiency of occupational therapy interventions to funding bodies and, if applicable, whether experience exists regarding the suitability of these quality indicators. In the following, we will describe the framework and method we used in this process.

World Federation of Occupational Therapists quality indicators framework

In compiling an overview of currently known quality indicators for occupational therapy, we used the World Federation of Occupational Therapists’ (WFOT) Quality Indicators Framework matrix as a guide for the identification of indicators [ 3 ]. The framework was created in response to the increasing demand on occupational therapists to demonstrate the effectiveness, appropriateness, and/or efficiency of their interventions, and to foster the development and use of quality indicators appropriate to occupational therapy practice. The framework conceptualizes different types of quality indicators for occupational therapy using a matrix whose vertical axis consists of quality dimensions, while the horizontal axis represents quality perspectives (see Table 1 ). In the Structure column, the availability of the appropriate number of competent occupational therapists in the right place at the right time is addressed, as well as the question of “whether other types of physical, financial, technical, and social resources necessary to provide quality occupational therapy services are continuously available in an economic, socially and environmentally sustainable manner” [ 3 ]. As the column name implies, these questions are situated on a structural level and regard questions of health care policy and workforce planning. In the Process column, “the ability of intended users to access occupational therapy” [ 3 ] also seems situated more in that era, while the categories optimal use of resources as well as success in attaining occupational therapy goals in the Outcome column directly refer to the effectiveness and efficiency of occupational therapy interventions. Lastly, Satisfaction throughout service delivery addresses the client perspective, while Incidents resulting in harm addresses patient safety and critical incidents.

For the purpose of this review, we focused on the columns Process and Outcome and the rows Efficiency , Effectiveness , and Person-Centeredness. The column Structure , as laid out above, is situated more on a health and educational policy level, while the row Safety was outside the purview of our study because relevant indicators had already been defined by the contractual partners (see above).

Scoping review

To answer the question which quality indicators are used internationally to demonstrate the effectiveness, appropriateness, and efficiency of occupational therapy interventions to funding bodies, we conducted a literature review. Since we assumed the available literature on the topic to not be primarily comprised of scientific studies, but to also include other documents of diverse provenance (e.g., strategy documents, magazine articles), we chose the scoping review method. Scoping reviews allow for greater flexibility in terms of the types of texts included compared to other types of literature reviews [ 8 ].

Data collection

We conducted this scoping review in July 2023 based on the Joanna Briggs Institute manual on evidence synthesis [ 9 ]. After formulating the research question, we defined initial relevant keywords (see Table 2 ), inclusion and exclusion categories. To search for suitable literature, we used specialized databases from medicine, health sciences, and related fields, including CINAHL Complete and MEDLINE, as well as free internet search via Google. Furthermore, in collaboration with the Swiss Association of Occupational Therapy (EVS/ASE), we contacted several national occupational therapy associations, as well as other international contacts of the EVS/ASE, to ask for access to documents found within this search that were only accessible to association members.

Data analysis

We screened the documents we found using the online platform Covidence [ 10 ]. The selection process was carried out by the first and second author. In a first step, we screened titles and abstracts of documents and included or excluded the documents based on defined criteria (see Table 3 ). To resolve conflicts between the two reviewers regarding inclusion or exclusion, we consulted another team member, the last author. In the following step, we applied the same procedure for the included full texts. This time, the two reviewers discussed any disagreements and decided if the document would be included or excluded. Then, we extracted data relevant to the research question from the documents and synthesized the data.

The screening process identified 32 studies and documents from six national contexts. In Fig. 1 , the screening process is visualized. In Table 4 , the number of documents and studies per country and a short description of the relevant quality indicators (if any) identified is provided. Although we used due diligence in our search process, it is highly likely that some relevant documents were neither openly available on the internet nor the subject of articles in specialized databases.

Screening process studies and documents

United States of America

Most documents identified in our review were situated in the US-American context and were concerned with the provision of quality indicators to demonstrate effectiveness, appropriateness, and/or efficiency of health care services to Medicare. Medicare is a public health insurance program that provides health care coverage for Americans 65 years old or older and certain younger people with disabilities [ 11 ]. Medicare consists of three parts that cover hospital insurance (part A), medical insurance (part B), and prescription drug coverage (part C). In the available documents, mainly two reimbursement systems were described: the Merit-Based Payment System (MIPS) and the Outcome and Assessment Information Set (OASIS), which will be the focus of our analysis. Other similar systems, namely the Minimum Data Set and the CARE-tool used in hospital settings, were also mentioned [ 12 , 13 , 14 , 15 ]. Private insurers have their own criteria for reimbursement, but they are laid out less transparently than in the case of Medicare [ 16 , 17 ].

Merit-based incentive payment system (MIPS)

The Merit-Based Incentive Payment System (MIPS) is the main reimbursement system currently in use for reimbursing health care services provided to patients covered under Medicare. MIPS is concerned with the reimbursement of part B services. It is part of the Quality Payment Program (QPP) that is based on the Medicare Access and CHIP Reauthorization Act (MACRA) of 2015. The aim of establishing QPP was to base reimbursement more on quality of care [ 18 ]. Eligible health care professions include physicians, nurse practitioners, occupational therapists, physical therapists, and several other professions.

Under MIPS, clinicians yearly report data in four areas: quality , improvement activities , promoting interoperability , and cost . Cost is calculated automatically based on claims submitted to Medicare. In promoting interoperability , clinicians report on a set of measures and objectives connected to digitization (e.g., use of electronic health records, e-prescribing). In the area improvement activities , clinicians have to attest to between 2 and 4 predefined activities that improve access to care, enhance client engagement, and/or improve processes. Finally, in the area quality , clinicians are asked to provide at least six quality measures, one of which must be an outcome measure or another high priority measure. These measures must be provided for a minimum of 70% of patients over the respective 12-month period [ 19 ]. Centers for Medicare & Medicaid Services (CMS), the federal agency responsible for Medicare and Medicaid, has defined measures suitable for physical and occupational therapists (see Table 5 ). However, it has been proposed that quality indicators that are less generic and more reflective of the contribution of occupational therapists be developed and included [ 20 ].

Each MIPS area is scored individually. For instance, the quality area is scored based on the completeness of the required data and their quality in relationship to benchmarks. These benchmarks are based on performance data from a baseline period (usually two years prior to the reporting year). The area scores are weighted and transformed into a total MIPS score between 0 and 100 points. Clinicians that score below 75 points will suffer a negative payment adjustment through Medicare of up to -9%. Clinicians that score 75 points and above will receive a positive payment adjustment. The factor depends on statutory budget neutrality requirements (i.e., how much money is available under an existing budget). If a clinician scores 89% or above, they will receive an additional payment adjustment for exceptional performance, again depending on budget neutrality requirements [ 19 ]. As of 2020, occupational therapists only needed to report in the MIPS areas quality and performance improvement. These areas were, therefore, reweighted so that quality accounted for 85% and performance improvement for 15% of the total MIPS score [ 21 , 22 ].

The MIPS requires a lot of reporting and has been described as at times “tedious” [ 18 ]. In 2021, 3.31% of clinicians suffered a negative payment adjustment, while 86.12% achieved a positive payment adjustment, with 77.86% achieving an additional adjustment for exceptional performance. As the number of eligible clinicians changed drastically between 2020 and 2021 due to changes in eligibility rules, it is difficult to compare 2021 data with earlier years [ 23 ]. The Covid-19-pandemic is another factor that makes it harder to draw conclusions on the performance of the new system. Furthermore, MIPS has been criticized by the American Occupational Therapy Association for being physician-centred and not sufficiently reflective of the services of non-physician health professionals [ 7 ].

Section GG self-care (activities of daily living) and mobility items

The Section GG Self-Care (Activities of Daily Living) and Mobility Items form [ 24 ] is a form used over different settings (skilled nursing facilities, home health care, inpatient rehabilitation) to evaluate self-care skills and activities of daily living (see Table 6 ). While it is not an explicit occupational therapy assessment - the Centers for Medicare and Medicaid Services solely state that it should be coded by qualified clinicians [ 25 ] – it is often used by occupational therapists, and occupational therapists have been urged to demonstrate their contribution to the interprofessional team by claiming this task [ 20 , 21 ].

Switzerland

In Switzerland, there has been a contractual quality agreement between the Swiss Association of Occupational Therapy (EVS/ASE) and the associations of private insurances dating back to 2011 [ 26 ]. Since 2019, this agreement has been expanded to include not only general health insurance, but also accident, disability, and military insurance cases. The effectiveness of occupational therapy services is evaluated using the Goal Attainment Scale (GAS) as a quality indicator. GAS [ 27 ] is a standardized, valid, and reliable assessment that expresses the degree of achievement of individually set goals in a numerical value from − 2 (worse outcome than expected) to + 2 (much better outcome than expected) (see Fig. 2 ).

Example of a completed Goal Attainment Scale. Note: adapted from EVS/ASE [ 28 ]

All self-employed occupational therapists as well as occupational therapy organizations and their employees are obliged to record five cases with the GAS and document them on an online platform each year. 10 cases per language region are randomly selected and checked for content quality. Reasons for non-participation or incomplete participation must be declared on the online platform. Unjustified non-participation can be sanctioned [ 29 ]. The implementation of this procedure was accompanied by a research project evaluating the quality and content of goal setting by Swiss occupational therapists [ 30 ].

In 2020, 2159 occupational therapists were registered on the online platform, documenting 8106 clients [ 29 ]. The number of registered therapists has continuously risen from 2016, when there were 1265, to 2020. Reported outcomes in terms of goal attainment have remained stable in this time frame. Goals seem to have been set in a realistic, measurable manner. These results have been deemed as positive by representatives of all contractual parties [ 29 ].

The Netherlands

Pro-ergo questionnaire (patient reported experience measure).

While information on the use of quality indicators for reimbursement by Dutch occupational therapist was not to be found online using our search terms, representatives of the Dutch professional association that were contacted directly by the EVS/ASE reported that they are not asked to provide quality indicators to funding bodies (EVS/ASE, personal communication, October 2023). However, it was also reported to us through personal contacts (J. Leenders, personal communication, September 2023) that some Dutch insurance companies do require occupational therapists to provide the PRO-Ergo questionnaire [ 31 ], a patient reported experience measure (PREM) that includes a number of statements on activities, self-management, social environment, and satisfaction with occupational therapy services that are rated on a scale of 0 to 10 (see Table 7 ).

Unfortunately, we could not identify any descriptions of experiences with this measure.

Apart from this, occupational therapists’ role in reimbursement for therapeutic aids is described in the literature, including standardised measures for funding bodies. However, this does not concern the reimbursement of occupational therapy services themselves [ 32 ].

Other countries

Apart from USA, Switzerland, and the Netherlands, little information was available on quality indicators used by occupational therapists to demonstrate the effectiveness, appropriateness, and/or efficiency of their services to funding bodies. For the UK and some Canadian provinces, we found the use of more general quality indicators that are non-specific to occupational therapy and mainly process-level, e.g., “The percentage of patients with hypertension aged 16 to 74 years in whom there is an annual assessment of physical activity, using GPPAQ, in the preceding 15 months” and similar [ 33 , 34 ]. In Germany, occupational therapists are not providing quality indicators to funding bodies in the outpatient sector, while the indicators used in the inpatient sector are limited to the amount and/ or duration of occupational therapy sessions (Deutscher Verband Ergotherapie, personal communication, June 2023). However, the development of quality indicators is a stated goal of the German professional association of occupational therapists (DVE), as stated in a current position paper [ 35 , 36 ], and has been for several years [ 37 ]. Representatives of the Swedish professional association that were contacted directly by the EVS/ASE reported that they are not asked to provide quality indicators to funding bodies (EVS/ASE, personal communication, October 2023).

The aim of this scoping review was to establish which quality indicators are used nationally and internationally (focusing on Europe and English-speaking countries abroad) to demonstrate the effectiveness, appropriateness, and/or efficiency of occupational therapy interventions to funding bodies and, if applicable, whether experience exists regarding the suitability of these quality indicators. There was little information that was openly available, which could mean that quality indicators are either not in widespread use, that information on their use is not accessible, or both. While we can, therefore, not claim to give a complete overview on all quality indicators used for occupational therapy services in Europe and English-speaking countries abroad, the reporting systems that we have identified in our opinion do show a certain breadth of the possible use of quality indicators in this field. In essence, we identified two reporting systems that utilise process-level indicators, two that utilise outcome-level indicators, and one the utilises both types of indicators. Table 8 visualizes how these systems can be organized within the WFOT Quality Indicator Framework.

Process-level indicators

The process-level indicators identified as part of the National Institute for Health and Care Excellence (NICE) menu of indicators [ 33 ] and the Quality Improvement Plan for Ontario Health Teams (QIPOH) indicator list [ 34 ] mainly concern the percentage of patients over a certain reporting period for whom a certain intervention, procedure, test, or similar has been carried out. The process-level indicators on the 2023 MIPS Quality Measures List [ 5 ] for physical therapy and occupational therapy serve the same purpose, but on the individual client level (i.e., has a certain intervention, procedure, test or similar been carried out for this client). This allows the responsible agencies or funding bodies to assess the degree to which the reporting professionals or institutions are adhering to best practice guidelines or similar, either in general (NICE, QIPOH) or on an individual level (MIPS).

However, the NICE menu of indicators and the QIPOH indicator list include few, if any, quality indicators that are appropriate to reflect occupational therapy interventions. Some of the process-level indicators in the MIPS Quality Measures List are more appropriate to reflect the specific contributions of occupational therapy to health outcomes (e.g., related to fall risk management or functional status assessment). However, they are also very specific to certain client groups and/ or health conditions (e.g., “Rheumatoid Arthritis (RA): Functional Status Assessment”).

Outcome indicators

Unlike process-level indicators, outcome indicators necessarily operate on an individual level. The outcome indicators on the 2023 MIPS Quality Measures List [ 5 ] for physical- and/or occupational therapy are very well suited to record the outcomes of occupational therapy interventions, but also very specific to certain client groups and/or health conditions (e.g., “Functional Status Change for Patients with Elbow, Wrist or Hand Impairments”). The Section GG Self-Care (Activities of Daily Living) and Mobility Items form, on the other hand, is not specific to certain client groups and/ or health conditions, but appropriate for all clients that have problems in performing activities of daily living (ADL) – which is a key domain for occupational therapy [ 38 ]. In content and scoring, Section GG is very similar to ADL assessment forms like the Barthel index [ 39 ] that, while not necessarily specific to occupational therapy, are also commonly used by occupational therapists.

The GAS [ 27 ] works on a different level than the outcome indicators included in the MIPS and the Section GG form as it does not measure objective functional change, but goal attainment . As demonstrated in Fig. 2 , this does not only make it possible to assess functional change on the level of activity and participation (e.g., “Client has achieved sufficient fist grip to be able to hold narrow tool handles for easy tasks”) beyond ADLs, but also to assess facets of change relevant to clients that are not considered by strictly functional assessments, like psychosocial aspects (e.g., “Client is comfortable going to the pub even without a bandage and uses the affected hand when greeting (handshake)”). It is also not specific to a certain client group or a specific health condition but can be used as a generic assessment across all domains of occupational therapy (and, potentially, other professions).

While the GAS relies on specific, measurable goals [ 27 ], PRO-Ergo is, as a PREM, by definition a subjective assessment. It assesses not only the subjective outcome of an intervention, but also the client’s satisfaction with that intervention. Where GAS is a client- or patient-centred assessment in the collaborative setting of goals, PRO-Ergo is fully focused on the client’s subjective experience of the intervention and its outcome.

To synthesize these findings, all indicators we have identified in the literature have advantages and disadvantages. Process-level indicators allow the responsible agencies or funding bodies to assess the degree to which the reporting professionals or institutions are adhering to best practice. However, using these with occupational therapists would necessitate the creation of specific process-level indicators for this purpose, based on guidelines for best practice.

Specific functional outcomes like the ones included in the MIPS Quality Indicator List [ 5 ] are an appropriate way to demonstrate functional change as an outcome of an occupational therapy intervention for persons with acute and/or chronic health conditions. They are, however, often highly specific to certain client groups and health conditions. To utilize this kind of functional outcomes for quality indication for occupational therapy across the board, there would need to be a large pool of items to draw from to cover the breadth of occupational therapy practice. In contrast, Section GG [ 25 ], an assessment used for the evaluation of ADL skills, is more general. It or a similar assessment could possibly be used across all fields that require an assessment of ADL.

The GAS [ 27 ] and PRO-Ergo [ 31 ] have the advantage that they are usable across all fields of practice and client groups, provided that certain clients are not able to participate in the assessment process (e.g., persons with severe dementia), necessitating the involvement of proxies (e.g., significant others). Both, make it possible to assess outcomes on the activity and participation level. Between the two assessments, GAS is the more labour-intensive, as individual collaborative goals are defined in collaboration with the clients, which reflects the professions client-centred approach.

Given that all different kinds of quality indicators have their advantages and disadvantages, the implementation of a specific kind of quality indicator for a specific health profession will likely be shaped by current health policy priorities on one side and the interest of practitioners, represented by professional associations and/ or unions, on the other side. In Switzerland, these interests and priorities dovetail for occupational therapy when it comes to using quality indicators to maintain or increase standards of care [ 40 , 41 ]. Because another urgent goal of Swiss health policy is containing health care spending [ 40 ], it is important that representatives of health care professionals’ interests (i.e. professional associations, unions) take an active role in shaping the implementation of quality indicators. The use of quality indicators can also be an opportunity for smaller health care professions to sharpen their profile and demonstrate their value to funding bodies and policymakers. However, to achieve the latter, quality indicators have to be able to represent the specific contribution of this profession to clients’ health and well-being (e.g., in the case of occupational therapy, gains in independence, autonomy and/ or participation).

On the other hand, complex quality indicator reporting systems have been criticized for placing a high administrative burden on practitioners (Khullar et al. 2021). Beyond that, health care policy makers may try to increase efficiency by tying health care provider reimbursement to their performance on quality indicators (which is already the case in some areas of the MIPS). This can however have the unwanted effect of providers prioritizing care that is easily measurable instead of what is best for the individual client (Wagenschieber & Blunck, 2024). A focus on more flexible and client-centred outcome indicators could possibly help mitigate these risks.

While we have been focussing on occupational therapy, most of the quality indicator systems we have looked at (e.g., MIPS, NICE, QIPOH, GAS) in this study are used by multiple health care professions. Challenges and opportunities are similar, and we therefore believe that these results are transferable to other professions.

Limitations

While we have identified little information on quality indicators used to demonstrate effectiveness, appropriateness, and efficiency of occupational therapy services to funding bodies, we cannot say for certain if this is due to their limited use or to the limited accessibility of relevant information. We were also not able within this study to examine quality indicators being used beyond Europe and the English-speaking world. Also, we were not able to elicit much information on stakeholders’ experiences with these indicators and their usefulness. This is mainly due to the apparent dearth of studies that explore these experiences.

Further research

While results of this study have added additional dimensions to quality indicator frameworks like the WFOT QUEST [ 3 ], further research into stakeholders’ (e.g., occupational therapists, insurance companies, patients) experiences with these different quality indicators, as well as the latters’ relationship to economic (e.g., cost per patient) and other indicators (e.g., hospital admissions, return to work rates) could solidify our understanding of the positive or negative effects of quality indicators on the practice of occupational therapy and, potentially, other health professions. Also, an in-depth review on the use of structural indicators to evaluate efficiency, appropriateness, and effectiveness of health care service provision and the health care policies they are embedded in would be needed to put evidence on process- and outcome-indicator use into a larger context.

Among the quality indicators we identified for this report, all have their advantages and disadvantages. The establishment of process-level indicators specific to occupational therapy could be a chance to foster the use of best practice methods, based on available evidence or existing guidelines (see, e.g., [ 42 ]).

In terms of outcome indicators, GAS and PRO-Ergo seem to be the most versatile assessments, while also taking into account the Federal Council’s call for patient-centredness in quality assurance [ 43 ]. Also, they allow an evaluation on the level of activity and participation (e.g., in work and employment), not solely on the level of body functions and structures (e.g., musculoskeletal functions). As the goals in the GAS are formulated individually, these can include the activity and participation levels as well as body function and structure levels.

While functional outcome indicators (e.g., change in range of motion) present easily understandable data, they are often highly specific to certain client groups and/or health conditions. The definition of functional outcome indicators for every possible client group or health condition may be a disproportionate effort. However, functional outcome indicators that cover a broad area of client problems, like Section GG [ 25 ] or a similar assessment of ADL, may be useful additional quality indicators for some areas of practice.

There was little information on the use of quality indicators to demonstrate the effectiveness, appropriateness, and efficiency of occupational therapy services to funding bodies in Europe and English-speaking countries abroad that was openly available. This could mean that the use of such quality indicators is either not that widespread, that information on their use is not very accessible, or both. Furthering research in these areas, including patients’ perspectives, and fostering the accessibility of such documents is therefore highly recommended.

The results of this study have practical implications for health care policymakers, professional associations or unions representing health care professionals as well as other stakeholders in the health care field (e.g., insurances). Especially if the goal of quality indicators is maintaining or increasing standards of care, quality indicators should be chosen in order to accurately reflect the specific contributions of different health care professions.

Availability of data and materials

No datasets were generated or analysed during the current study.

Abbreviations

Activities of daily living

Centers for Medicare & Medicaid Services

German professional association of occupational therapists

Swiss National Association of Occupational Therapy

Goal Attainment Scale

Federal Act on Health Insurance (Switzerland)

Medicare Access and CHIP Reauthorization Act

Merit-Based Payment System

National Institute for Health and Care Excellence

Ordinance on Compulsory Health Care (Switzerland)

Outcome and Assessment Information Set

Patient reported experience measure

Quality Improvement Plan for Ontario Health Teams

Quality Payment Program

World Federation of Occupational Therapists

European Commission. Tools and methodologies to Assess the Efficiency of Health Care Services in Europe. European Union; 2019.

Google Scholar  

Kötter T, Blozik E, Scherer M. Methods for the guideline-based development of quality indicators–a systematic review. Implement Sci. 2012;7(1):21.

Article   PubMed   PubMed Central   Google Scholar  

World Federation of Occupational Therapists. QUEST Quality Evaluation Strategy Tool: An essential guide for using quality indicators in occupational therapy [Internet]. 2023. https://wfot.org/checkout/23421/30025 .

Mainz J. Defining and classifying clinical indicators for quality improvement. Int J Qual Health Care. 2003;15(6):523–30.

Article   PubMed   Google Scholar  

Centers for Medicare & Medicaid Services. 2023 MIPS Quality Measures List [Internet]. 2022. https://qpp-cm-prod-content.s3.amazonaws.com/uploads/2217/2023%20MIPS%20Quality%20Measures%20List.xlsx .

Arah OA, Klazinga NS, Delnoij DM, Asbroek AT, Custers T. Conceptual frameworks for health systems performance: a quest for effectiveness, quality, and improvement. Int J Qual Health Care. 2003;15(5):377–98.

Article   CAS   PubMed   Google Scholar  

Parsons H, American Occupational Therapy Association. Response to congressional request for information (RFI) on the current state of the Medicare Access and CHIP Reauthorization Act (MACRA) [Internet]. 2022 [cited 2024 Jan 10]. https://www.aota.org/-/media/corporate/files/advocacy/aota-response-macra-rfi.pdf .

Peterson J, Pearce PF, Ferguson LA, Langford CA. Understanding scoping reviews: definition, purpose, and process. J Am Assoc Nurse Pract. 2017;29(1):12–6.

Peters MD, Godfrey C, McInerney P, Munn Z, Tricco AC, Khalil H. Scoping reviews. Joanna Briggs Inst Rev Man. 2017;2015:1–24.

Veritas Health Innovation. Covidence systematic review software [Internet]. Melbourne, Australia. 2023. Available from: www.covidence.org .

U.S. Centers for Medicare and Medicaid Services. What’s Medicare? [Internet]. 2023 [cited 2023 Aug 15]. https://www.medicare.gov/what-medicare-covers/your-medicare-coverage-choices/whats-medicare .

Centers for Medicare & Medicaid Services. Outcome Assessment Information Set Version E (OASIS-E) [Internet]. 2022. https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/HomeHealthQualityInits/HHQIOASISUserManual .

Mor V. A comprehensive clinical assessment tool to inform policy and practice: applications of the minimum data set. Med Care. 2004;42(4 Suppl):III50–9.

PubMed   Google Scholar  

Pardasaney PK, Deutsch A, Iriondo-Perez J, Ingber MJ, McMullen T. Measuring Inpatient Rehabilitation Facility Quality of Care: Discharge Self-Care Functional Status Quality measure. Arch Phys Med Rehabil. 2018;99(6):1035–41.

Trundy-Whitten L. New Patient Driven Payment Model from CMS [Internet]. 2018 [cited 2023 Jun 13]. https://www.berrydunn.com/news-detail/new-patient-driven-payment-model-from-cmswhat-to-expect-and-what-to-do .

AETNA. Occupational Therapy - Medical Clinical Policy Bulletins [Internet]. 2023 [cited 2023 Jun 13]. https://www.aetna.com/cpb/medical/data/200_299/0250.html .

OPTUM. Physical Therapy and Occupational Therapy Initial Evaluation and Reevaluation Reimbursement Policy [Internet]. 2020 [cited 2023 Jun 13]. https://www.myoptumhealthphysicalhealth.com/Documents/Reimbursement%20Policies/0044_PT_OTReimbursementPolicy.pdf .

Valenzano MFOXR. 2019 [cited 2023 Jun 13]. Is MIPS Good or Bad for PTs, OTs & SLPs? Your Medicare Update. https://www.foxrehab.org/mips-documentation-patient-care/ .

Centers for Medicare & Medicaid Services. Merit-based Incentive Payment System (MIPS) - Traditional MIPS Scoring Guide for the 2022 Performance Year [Internet]. Department of Health and Human Services; 2022 [cited 2023 Aug 15]. https://qpp-cm-prod-content.s3.amazonaws.com/uploads/1970/2022%20Traditional%20MIPS%20Scoring%20Guide.pdf .

Leland NE, Crum K, Phipps S, Roberts P, Gage B. Advancing the Value and Quality of Occupational Therapy in Health Service Delivery. Am J Occup Ther. 2015;69(1):p69010900101–7.

Article   Google Scholar  

American Occupational Therapy Association. Quality Payment Program (QPP) Merit-based Incentive Payment System (MIPS) Changes for 2020 | AOTA [Internet]. 2019 [cited 2023 Jun 13]. https://www.aota.org/advocacy/advocacy-news/2019/quality-payment-program-merit-based-incentive-payment-system-changes-2020 .

Byars G. Expressing the Unique Value of Occupational Therapy – Utah Occupational Therapy Association [Internet]. 2019 [cited 2023 Jun 13]. https://utahotassociation.org/expressing-the-unique-value-of-occupational-therapy/ .

Centers for Medicare & Medicaid Services. 2021 Quality Payment Program Experience Report [Internet]. Department of Health and Human Services; 2022 [cited 2023 Aug 15]. https://qpp-cm-prod-content.s3.amazonaws.com/uploads/2433/2021%20QPP%20Experience%20Report.pdf .

American Occupational Therapy Association. Section GG Self-Care (Activities of Daily Living) and Mobility Items [Internet]. 2022 [cited 2023 Jun 15]. https://www.aota.org/practice/domain-and-process/-/media/e85d779d45e14bdf8f4285abe75e1047.ashx .

Centers for Medicare & Medicaid Services. Coding Section GG Self-Care & Mobility Activities Included on the Post-Acute Care Item Sets: Key Questions to Consider When Coding [Internet]. 2023. https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/HomeHealthQualityInits/Downloads/GG-Self-Care-and-Mobility-Activities-Decision-Tree.pdf .

Santésuisse. Messprogramm in der Ergotherapie unterstützt kontinuierliche Qualitätsverbesserung [Internet]. 2021 [cited 2023 Jun 8]. https://www.santesuisse.ch/politik-medien/themen/qualitaet#content-tab-5 .

Kiresuk TJ, Smith AE, Cardillo JE. Goal attainment scaling: Applications, theory, and measurement. 1994.

ErgotherapeutInnen-Verband Schweiz. Qualität in der Ergotherapie [Internet]. 2022 [cited 2023 Sep 20]. https://www.ergotherapie.ch/verband/qualitaetsmanagement/ .

Petrig A, Ruth W. Siegenthaler-Tschanz, Karin, Roth, Felix, Deppeler, Sandro, Züge, Isabelle. Bericht der Paritätischen Kommission Qualität EVS/SRK/santésuisse 2020 [Internet]. 2021 [cited 2023 Sep 20]. https://www.ergotherapie.ch/download.php?cat=6JLvL2DYoAPKasSNvUf3Dw%3D%3D&id=42 .

Page J, Roos K, Bänziger A, Margot-Cattin I, Agustoni S, Rossini E, et al. Formulating goals in occupational therapy: state of the art in Switzerland. Scand J Occup Ther. 2015;22(6):403–15.

Ergotherapie Nederland. Pro-Ergo Vragenlijst - Ergotherapie [Internet]. 2017 [cited 2023 Sep 22]. https://info.ergotherapie.nl/file/download/default/1CF999424EEE153BDFDD1492DF344DFE/170705%20PRO-Ergo%20VERSIE%20juli%202017.pdf .

Hubert M. [The practice of needs assessment for the supply with technical aids in the Netherlands]. Prax Berdarfsermittl Zur Hilfsmittelversorgung Den Niederlanden. 2003;42(1):52–9.

CAS   Google Scholar  

National Institute for Health and Care Excellence. Indicator development programme: The NICE menu of indicators [Internet]. 2023 [cited 2023 Sep 20]. https://www.nice.org.uk/standards-and-indicators/indicators .

Ontario Health. Collaborative Quality Improvement Plan [Internet]. 2022 [cited 2023 Sep 20]. https://e-qip.ca/wp-content/uploads/2022/01/cQIP_OHT_Indicators_Technical_Specifications_EN.pdf .

Deutscher Verband Ergotherapie, Perspektiven DVE. / Ergotherapie 2029 [Internet]. 2022 [cited 2023 Jun 8]. https://dve.info/resources/pdf/dve/ziele/4632-perspektiven-2029/file .

Deutscher Verband Ergotherapie. Schwerpunkte des DVE 2023 [Internet]. 2022 [cited 2023 Jun 8]. https://dve.info/resources/pdf/dve/ziele/4892-dve-schwerpunkt-2023/file .

Deutscher Verband Ergotherapie. Geschäftsbericht 2013. DVE Aktuell. 2014;2014:01.

American Occupational Therapy Association. Occupational Therapy Practice Framework: domain and process—Fourth Edition. Am J Occup Ther. 2020;74(Supplement2):p74124100101–87.

Mahoney FI, Barthel DW. Functional evaluation: the Barthel Index: a simple index of independence useful in scoring improvement in the rehabilitation of the chronically ill. Md State Med J [Internet]. 1965 [cited 2024 Jan 9]; https://psycnet.apa.org/record/2012-30334-001 .

Federal Council. Health Policy Strategy of the Federal Council 2020–2030 [Die gesundheitspolitische Strategie des Bundesrates 2020–2030] [Internet]. Federal Office of Public Health. 2019. https://www.bag.admin.ch/dam/bag/de/dokumente/nat-gesundheitsstrategien/gesundheit-2030/strategie-gesundheit2030.pdf.download.pdf/strategie-gesundheit-2030.pdf .

Ergotherapie-Verband Schweiz. (2016). Leitbild des ErgotherapeutInnen-Verbandes Schweiz EVS. Ergotherapie-Verband Schweiz. https://www.ergotherapie.ch/download.php?cat=6JLvL2DYoAPKasSNvUf3Dw%3D%3D&id=8 .

Weise A. Ergotherapeutische Leitlinien Ergotherapie. 2016;2016(9):7.

Federal Council. Aims of the Federal Council for Quality Development for the years 2022–2024 [Ziele des Bundesrates zur Qualitätsentwicklung für die Jahre 2022–2024] [Internet]. Federal Office of Public Health. 2022. file:///C:/Users/balh/Downloads/BAG_Vierjahresziele_DE.pdf

Download references

Acknowledgements

We would like to thank Barbara Iseli Sczepanski, co-managing director of the EVS/ASE, for her cooperation throughout this project.

This study was commissioned and fully funded by the Swiss National Association of Occupational Therapy (EVS/ASE) to serve as a basis for the definition of quality indicators in contractual quality agreements with the two large health insurance associations in Switzerland. Specifically, the study was part of a funding request to the Federal Quality Commission (FQC) for the development of a new online platform that will allow Swiss occupational therapists to provide quality indicators to insurances. Open access funding provided by ZHAW Zurich University of Applied Sciences.

Open access funding provided by ZHAW Zurich University of Applied Sciences

Author information

Authors and affiliations.

Institute of Occupational Therapy, School of Health Sciences, ZHAW Zurich University of Applied Sciences, Winterthur, Switzerland

Thomas Ballmer, Sara Frey & Brigitte E. Gantschnig

Swiss National Association of Occupational Therapy (EVS/ASE), Bern, Switzerland

Andrea Petrig

Department of Rheumatology and Immunology, University Hospital (Inselspital) and University of Bern, Bern, Switzerland

Brigitte E. Gantschnig

You can also search for this author in PubMed   Google Scholar

Contributions

TB was responsible for writing the majority of the manuscript. SF contributed significantly to the Background and Methods section. SF was responsible for the data collection (literature search). The data analysis was conducted by TB and SF in tandem, with support from BG. The results, discussion, and conclusion sections were conceptualized by TB and SF and put into writing by TB. AP was involved in the planning of the study and gave feedback regarding background, method and results throughout the process. All authors gave feedback to all sections and read and approved the final manuscript. After the peer reviewers’ feedback, TB prepared the initial revision of the paper. AP, SF and BG gave feedback and added to this revision, which TB subsequently finalized.

Corresponding author

Correspondence to Thomas Ballmer .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Ballmer, T., Frey, S., Petrig, A. et al. Quality indicators for occupational therapy: a scoping review. BMC Health Serv Res 24 , 1054 (2024). https://doi.org/10.1186/s12913-024-11548-1

Download citation

Received : 25 March 2024

Accepted : 05 September 2024

Published : 12 September 2024

DOI : https://doi.org/10.1186/s12913-024-11548-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Occupational therapy
• Reimbursement
• Quality improvement
• Quality indicators

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Wiley Open Access Collection

A scoping review of scoping reviews: advancing the approach and enhancing the consistency

a Department of Population Medicine, Ontario Veterinary College, University of Guelph, Guelph, Ontario, N1G 2W1, Canada

b Division of Public Health Risk Sciences, Laboratory for Foodborne Zoonoses, Public Health Agency of Canada, 160 Research Lane, Suite 206, Guelph, Ontario, N1G 5B2, Canada

Andrijana Rajić

c Food Safety and Quality Unit, Food and Agriculture Organization of the United Nations, Viale delle Terme di Caracalla, 00153, Rome, Italy

Judy D Greig

Jan m sargeant.

d Centre for Public Health and Zoonoses, Ontario Veterinary College, University of Guelph, Guelph, Ontario, N1G 2W1, Canada

Andrew Papadopoulos

Scott a mcewen, associated data.

The scoping review has become an increasingly popular approach for synthesizing research evidence. It is a relatively new approach for which a universal study definition or definitive procedure has not been established. The purpose of this scoping review was to provide an overview of scoping reviews in the literature.

A scoping review was conducted using the Arksey and O'Malley framework. A search was conducted in four bibliographic databases and the gray literature to identify scoping review studies. Review selection and characterization were performed by two independent reviewers using pretested forms.

The search identified 344 scoping reviews published from 1999 to October 2012. The reviews varied in terms of purpose, methodology, and detail of reporting. Nearly three-quarter of reviews (74.1%) addressed a health topic. Study completion times varied from 2 weeks to 20 months, and 51% utilized a published methodological framework. Quality assessment of included studies was infrequently performed (22.38%).

Conclusions

Scoping reviews are a relatively new but increasingly common approach for mapping broad topics. Because of variability in their conduct, there is a need for their methodological standardization to ensure the utility and strength of evidence. © 2014 The Authors. Research Synthesis Methods published by John Wiley & Sons, Ltd.

1. Background

The scoping review has become an increasingly popular approach for synthesizing research evidence (Davis et al. , 2009 ; Levac et al. , 2010 ; Daudt et al. , 2013 ). It aims to map the existing literature in a field of interest in terms of the volume, nature, and characteristics of the primary research (Arksey and O'Malley, 2005 ). A scoping review of a body of literature can be of particular use when the topic has not yet been extensively reviewed or is of a complex or heterogeneous nature (Mays et al. , 2001 ). They are commonly undertaken to examine the extent, range, and nature of research activity in a topic area; determine the value and potential scope and cost of undertaking a full systematic review; summarize and disseminate research findings; and identify research gaps in the existing literature (Arksey and O'Malley, 2005 ; Levac et al. , 2010 ). As it provides a rigorous and transparent method for mapping areas of research, a scoping review can be used as a standalone project or as a preliminary step to a systematic review (Arksey and O'Malley, 2005 ).

Scoping reviews share a number of the same processes as systematic reviews as they both use rigorous and transparent methods to comprehensively identify and analyze all the relevant literature pertaining to a research question (DiCenso et al. , 2010 ). The key differences between the two review methods can be attributed to their differing purposes and aims. First, the purpose of a scoping review is to map the body of literature on a topic area (Arksey and O'Malley, 2005 ), whereas the purpose of a systematic review is to sum up the best available research on a specific question (Campbell Collaboration, 2013 ). Subsequently, a scoping review seeks to present an overview of a potentially large and diverse body of literature pertaining to a broad topic, whereas a systematic review attempts to collate empirical evidence from a relatively smaller number of studies pertaining to a focused research question (Arksey and O'Malley, 2005 ; Higgins and Green, 2011 ). Second, scoping reviews generally include a greater range of study designs and methodologies than systematic reviews addressing the effectiveness of interventions, which often focus on randomized controlled trials (Arksey and O'Malley, 2005 ). Third, scoping reviews aim to provide a descriptive overview of the reviewed material without critically appraising individual studies or synthesizing evidence from different studies (Arksey and O'Malley, 2005 ; Brien et al. , 2010 ). In contrast, systematic reviews aim to provide a synthesis of evidence from studies assessed for risk of bias (Higgins and Green, 2011 ).

Scoping reviews are a relatively new approach for which there is not yet a universal study definition or definitive procedure (Arksey and O'Malley, 2005 ; Anderson et al. , 2008 ; Davis et al. , 2009 ; Levac et al. , 2010 ; Daudt et al. , 2013 ). In 2005, Arksey and O'Malley published the first methodological framework for conducting scoping reviews with the aims of clarifying when and how one might be undertaken. They proposed an iterative six-stage process: (1) identifying the research question, (2) identifying relevant studies, (3) study selection, (4) charting the data, (5) collating, summarizing and reporting the results, and (6) an optional consultation exercise (Arksey and O'Malley, 2005 ). Arksey and O'Malley intended for their framework to stimulate discussion about the value of scoping reviews and provide a starting point toward a methodological framework. Since its publication, a few researchers have proposed enhancements to the Arksey and O'Malley framework based on their own experiences with it (Brien et al. , 2010 ; Levac et al. , 2010 ; Daudt et al. , 2013 ) or a review of a selection of scoping reviews (Anderson et al. , 2008 ; Davis et al. , 2009 ).

In recent years, scoping reviews have become an increasingly adopted approach and have been published across a broad range of disciplines and fields of study (Anderson et al. , 2008 ). To date, little has been published of the extent, nature, and use of completed scoping reviews. One study that explored the nature of scoping reviews within the nursing literature found that the included reviews ( N = 24) varied widely in terms of intent, procedure, and methodological rigor (Davis et al. , 2009 ). Another study that examined 24 scoping reviews commissioned by a health research program found that the nature and type of the reports were wide ranging and reported that the value of scoping reviews is ‘increasingly limited by a lack of definition and clarity of purpose’ (Anderson et al. , 2008 ). Given that these studies examined only a small number of scoping reviews from select fields, it is not known to what extent scoping reviews have been undertaken in other fields of research and whether these findings are representative of all scoping reviews as a whole. A review of scoping reviews across the literature can provide a better understanding of how the approach has been used and some of the limitations and challenges encountered by scoping review authors. This information would provide a basis for the development and adoption of a universal definition and methodological framework.

The purpose of this paper is to provide an overview of existing scoping reviews in the literature. The four specific objectives of this scoping review were to (1) conduct a systematic search of the published and gray literature for scoping review papers, (2) map out the characteristics and range of methodologies used in the identified scoping reviews, (3) examine reported challenges and limitations of the scoping review approach, and (4) propose recommendations for advancing the approach and enhancing the consistency with which they are undertaken and reported.

This scoping review began with the establishment of a research team consisting of individuals with expertise in epidemiology and research synthesis (Levac et al. , 2010 ). The team advised on the broad research question to be addressed and the overall study protocol, including identification of search terms and selection of databases to search.

The methodology for this scoping review was based on the framework outlined by Arksey and O'Malley ( 2005 ) and ensuing recommendations made by Levac et al . ( 2010 ). The review included the following five key phases: (1) identifying the research question, (2) identifying relevant studies, (3) study selection, (4) charting the data, and (5) collating, summarizing, and reporting the results. The optional ‘consultation exercise’ of the framework was not conducted. A detailed review protocol can be obtained from the primary author upon request.

2.1. Research question

This review was guided by the question, ‘What are the characteristics and range of methodologies used in scoping reviews in the literature?’ For the purposes of this study, a scoping review is defined as a type of research synthesis that aims to ‘map the literature on a particular topic or research area and provide an opportunity to identify key concepts; gaps in the research; and types and sources of evidence to inform practice, policymaking, and research’ (Daudt et al. , 2013 ).

2.2. Data sources and search strategy

The initial search was implemented on June 17, 2011, in four electronic databases: MEDLINE/PubMed (biomedical sciences, 1946–present), SciVerse Scopus (multidisciplinary; 1823–present), CINAHL/EBSCO (nursing and allied health; 1981–present) and Current Contents Connect/ISI Web of Knowledge (multidisciplinary current awareness; 1998–present). The databases were selected to be comprehensive and to cover a broad range of disciplines. No limits on date, language, subject or type were placed on the database search. The search query consisted of terms considered by the authors to describe the scoping review and its methodology: scoping review, scoping study, scoping project, literature mapping, scoping exercise, scoping report, evidence mapping, systematic mapping, and rapid review. The search query was tailored to the specific requirements of each database (see Additional file 1).

Applying the same search string that was used for the search in SciVerse Scopus (Elsevier), a web search was conducted in SciVerse Hub (Elsevier) to identify gray literature. The a priori decision was made to screen only the first 100 hits (as sorted by relevance by Scopus Hub) after considering the time required to screen each hit and because it was believed that further screening was unlikely to yield many more relevant articles (Stevinson and Lawlor, 2004 ). The following websites were also searched manually: the Health Services Delivery Research Programme of the National Institute for Health Research ( http://www.netscc.ac.uk/hsdr/ ), the National Co-ordinating Centre for NHS Service Delivery and Organisation ( http://php.york.ac.uk/inst/spru/pubs/main.php ), NHS Evidence by the National Institute for Health and Clinical Excellence ( http://evidence.nhs.uk/ ), the University of York Social Policy Research Unit ( http://php.york.ac.uk/inst/spru/pubs/main.php ), the United Kingdom's Department of Health ( http://www.dh.gov.uk/en/index.htm ), and Google ( http://www.google.com ).

The reference lists of 10 randomly selected relevant articles (Hazel, 2005 ; Vissandjee et al. , 2007 ; Gagliardi et al. , 2009 ; Meredith et al. , 2009 ; Bassi et al. , 2010 ; Ravenek et al. , 2010 ; Sawka et al. , 2010 ; Churchill et al. , 2011 ; Kushki et al. , 2011 ; Spilsbury et al. , 2011 ) and eight review articles on scoping reviews (Arksey and O'Malley, 2005 ; Anderson et al. , 2008 ; Davis et al. , 2009 ; Grant and Booth, 2009 ; Hetrick et al. , 2010 ; Levac et al. , 2010 ; Rumrill et al. , 2010 ; Armstrong et al. , 2011 ) were manually searched to identify any further scoping reviews not yet captured. A ‘snowball’ technique was also adopted in which citations within articles were searched if they appeared relevant to the review (Hepplestone et al. , 2011 ; Jaskiewicz and Tulenko, 2012 ).

A follow-up search of the four bibliographic databases and gray literature sources was conducted on October 1, 2012 to identify any additional scoping reviews published after the initial search [see Additional file 1]. A search of Google with no date restrictions was also conducted at this time; only the first 100 hits (as sorted by relevance by Google) were screened.

2.3. Citation management

All citations were imported into the web-based bibliographic manager RefWorks 2.0 (RefWorks-COS, Bethesda, MD), and duplicate citations were removed manually with further duplicates removed when found later in the process. Citations were then imported into the web-based systematic review software DistillerSR (Evidence Partners Incorporated, Ottawa, ON) for subsequent title and abstract relevance screening and data characterization of full articles.

2.4. Eligibility criteria

A two-stage screening process was used to assess the relevance of studies identified in the search. Studies were eligible for inclusion if they broadly described the use of a scoping review methodology to identify and characterize the existing literature or evidence base on a broad topic. Because of limited resources for translation, articles published in languages other than English, French, or Spanish were excluded. Papers that described the scoping review process without conducting one and reviews of scoping reviews were excluded from the analysis, but their reference list was reviewed to identify additional scoping reviews. When the same data were reported in more than one publication (e.g., in a journal article and electronic report), only the article reporting the most complete data set was used.

2.5. Title and abstract relevance screening

For the first level of screening, only the title and abstract of citations were reviewed to preclude waste of resources in procuring articles that did not meet the minimum inclusion criteria. A title and abstract relevance screening form was developed by the authors and reviewed by the research team (see Additional file 2). The form was pretested by three reviewers (M. P., J. G., I. Y.) using 20 citations to evaluate reviewer agreement. The overall kappa of the pretest was 0.948, where a kappa of greater than 0.8 is considered to represent a high level of agreement (Dohoo et al. , 2012 ). As there were no significant disagreements among reviewers and the reviewers had no revisions to recommend, no changes were made to the form. The title and abstract of each citation were independently screened by two reviewers. Reviewers were not masked to author or journal name. Titles for which an abstract was not available were included for subsequent review of the full article in the data characterization phase. Reviewers met throughout the screening process to resolve conflicts and discuss any uncertainties related to study selection (Levac et al. , 2010 ). The overall kappa was 0.90.

2.6. Data characterization

All citations deemed relevant after title and abstract screening were procured for subsequent review of the full-text article. For articles that could not be obtained through institutional holdings available to the authors, attempts were made to contact the source author or journal for assistance in procuring the article. A form was developed by the authors to confirm relevance and to extract study characteristics such as publication year, publication type, study sector, terminology, use of a published framework, quality assessment of individual studies, types of data sources included, number of reviewers, and reported challenges and limitations (see Additional file 3). This form was reviewed by the research team and pretested by all reviewers (M. P., A. R., J. G., I. Y., K. G.) before implementation, resulting in minor modifications to the form. The characteristics of each full-text article were extracted by two independent reviewers (M. P. and J. G./K. G.). Studies excluded at this phase if they were found to not meet the eligibility criteria. Upon independently reviewing a batch of 20 to 30 articles, the reviewers met to resolve any conflicts and to help ensure consistency between reviewers and with the research question and purpose (Levac et al. , 2010 ).

2.7. Data summary and synthesis

The data were compiled in a single spreadsheet and imported into Microsoft Excel 2010 (Microsoft Corporation, Redmond, WA) for validation and coding. Fields allowing string values were examined for implausible values. The data were then exported into STATA version 12 (StataCorp, College Station, TX) for analyses. Descriptive statistics were calculated to summarize the data. Frequencies and percentages were utilized to describe nominal data.

3.1. Search and selection of scoping reviews

The original search conducted in June 2011 yielded 2528 potentially relevant citations. After deduplication and relevance screening, 238 citations met the eligibility criteria based on title and abstract and the corresponding full-text articles were procured for review. Four articles could not be procured and were thus not included in the review (Levy and Sanghvi, 1986 ; Bhavaraju, 1987 ; Centre for Reviews and Dissemination, 2004 ; Connell et al. , 2006 ). After data characterization of the full-text articles, 182 scoping reviews remained and were included in the analysis. The updated search in October 2012 produced 758 potentially relevant citations and resulted in another 162 scoping reviews being included. In total, 344 scoping reviews were included in the study. The flow of articles through identification to final inclusion is represented in Figure ​ Figure1 1 .

PRISMA flowchart of study selection process.

Many citations were excluded upon screening at the title and abstract level as several terms used in the search algorithm also corresponded to other study designs. For example, the term ‘scoping study’ was also used to describe studies that assessed the chemical composition of samples (e.g., Behrens et al. , 1998 ; Banks and Banks, 2001 ; Forrest et al. , 2011 ) and preliminary mining studies (Butcher, 2002 ; Bhargava et al. , 2008 ). ‘Scoping exercise’ also described studies that scoped an issue using questionnaires, focus groups, and/or interviews (e.g., Malloch and Burgess, 2007 ; Willis et al. , 2011 ; Norwood and Skinner, 2012 ). ‘Rapid review’ was also used to describe the partial rescreening of negative cervical smears as a method of internal quality assurance (e.g., Faraker and Boxer, 1996 ; Frist, 1997 ; Shield and Cox, 1998 ). ‘Systematic mapping’ was also used in studies pertaining to topographic mapping (e.g., Noda and Fujikado, 1987 ; Gunnell, 1997 ; Liu et al. , 2011 ) and mapping of biomolecular structures (e.g., Camargo et al. , 1976 ; Descarries et al. , 1982 ; Betz et al. , 2006 ).

3.2. General characteristics of included scoping reviews

The general characteristics of scoping reviews included in this study are reported in Table ​ Table1. 1 . All included reviews were published between 1999 and October 2012, with 68.9% (237/344) published after 2009. Most reviews did not report the length of time taken to conduct the review; for the 12.8% (44/344) that did, the mean length was approximately 5.2 months with a range of 2 weeks to 20 months. Journal articles (64.8%; 223/344) and government or research station reports (27.6%; 95/344) comprised the majority of documents included in the review. The number of journal articles was slightly underrepresented as 10 were excluded as duplicates because the same scoping review was also reported in greater detail in a report. The included reports ranged greatly in length, from four pages (Healthcare Improvement Scotland, 2012 ) to over 300 pages (Wallace et al. , 2006 ).

General characteristics of included scoping reviews ( n = 344)

The included scoping reviews varied widely in terms of the terminology used to describe the methodology. ‘Scoping review’ was the term most often used, reported in 61.6% (212/344) of included studies. An explicit definition or description of what study authors meant by ‘scoping review’ was reported in 63.1% (217/344) of articles. Most definitions centered around scoping reviews as a type of literature that identifies and characterizes, or maps, the available research on a broad topic. However, there was some divergence in how study authors characterized the rigor of the scoping review methodology. The terms ‘systematic’, ‘rigorous’, ‘replicable’, and ‘transparent’ were frequently used to describe the methodology, and several authors described scoping reviews to be comparable in rigor to systematic reviews (Gagliardi et al. , 2009 ; Liu et al. , 2010 ; Ravenek et al. , 2010 ; Feehan et al. , 2011 ; Heller et al. , 2011 ). In contrast, some studies described the methodology as less rigorous or systematic than a systematic review (Cameron et al. , 2008 ; Levac et al. , 2009 ; Campbell et al. , 2011 ). Brien et al. ( 2010 ) commented that scoping reviews were ‘often misinterpreted to be a less rigorous systematic review, when in actual fact they are a different entity’.

Some reviews were conducted as stand-alone projects while others were undertaken as parts of larger research projects. Study authors reported that a main purpose or objective for the majority of articles (97.4%; 335/344) was to identify, characterize, and summarize research evidence on a topic, including identification of research gaps. Only 6.4% (22/344) of included articles conducted the scoping review methodology to identify questions for a systematic review. As response options were not mutually exclusive, some reviews reported multiple purposes and/or objectives. A commissioning source was reported in 31.4% (108/344) of reviews; some reported that they were specifically commissioned to advise a funding body as to what further research should be undertaken in an area (e.g., Arksey et al. , 2002 ; Carr-Hill et al. , 2003 ; Fotaki et al. , 2005 ; Baxter et al. , 2008 ; Williams et al. , 2008 ; Trivedi et al. , 2009 ; Crilly et al. , 2010 ; Brearley et al. , 2011 ).

The majority of the included scoping reviews addressed a health topic, making up 74.1% (255/344) of reviews. The use of scoping reviews in software engineering—or ‘systematic mapping’ as termed in the sector—has increased in recent years with 92.7% (38/41) published after 2010. The topics examined in the included scoping reviews ranged greatly, spanning from data on multiplayer online role-playing games (Meredith et al. , 2009 ), to factors that influence antibiotic prophylaxis administration (Gagliardi et al. , 2009 ). The topics investigated were generally broad in nature, such as ‘what is known about the diagnosis, treatment and management of obesity in older adults’ (Decaria et al. , 2012 ). Some reviews that were conducted under short time frames (e.g., 1 month) addressed more specific questions such as ‘what is the published evidence of an association between hospital volume and operative mortality for surgical repair (open and endovascular) of unruptured and ruptured abdominal aortic aneurysms?’ (Healthcare Improvement Scotland, 2011 ).

3.3. Methodological characteristics of included scoping reviews

The methodological characteristics of included scoping reviews are reported in Table ​ Table2. 2 . Approximately half of the reviews (50.6%; 174/344) reported using one or more methodological frameworks for carrying out the scoping review. Framework use varied greatly between reviews from different sectors, such as in 85.4% (35/41) of reviews from the software engineering sector and in 44.0% (89/202) of health sector reviews. Overall, the Arksey and O'Malley ( 2005 ) framework was the most frequently used, reported in 62.6% (109/174) of studies that reported using a framework. Among reviews from the software engineering sector that reported using a framework, frameworks by Kitchenham and Charters ( 2007 ) (40.0%; 14/35) and Petersen et al . ( 2008 ) (51.4%; 18/35) were most commonly employed. The use of a framework increased over time, from 31.6% (6/19) of reviews published from 2000 to 2004, to 42.5% (37/87) of reviews from 2005 to 2009, and to 55.3% (131/237) of reviews published from 2010 onward.

Methodological characteristics of included reviews ( n = 344)

Following the search, 79.7% (174/344) of reviews used defined inclusion and exclusion criteria to screen out studies that were not relevant to the review question(s). Among these, only six reviews explicitly reported that criteria were redefined or amended on a post hoc basis during the review process (While et al. , 2005 ; Marsella, 2009 ; Crooks et al. , 2010 ; Johnston et al. , 2010 ; Snyder et al. , 2011 ; Victoor et al. , 2012 ). The selection criteria in a few reviews were unclear due to ambiguous wording such as ‘real paper’ (Saraiva et al. , 2012 ), ‘scientific papers’ (Victoor et al. , 2012 ), and ‘culling low-interest articles’ (Catts et al. , 2010 ). Compared with the study selection process, fewer details were generally reported about the data characterization (or charting) of individual studies. Nearly a quarter of reviews (23.8%; 82/344) did not report any detail as to how the included studies were characterized, and it was unclear in 33.4% (115/344) as to how many reviewers were involved.

The majority of included reviews (77.7%, 267/344) did not assess the methodological quality of individual studies. A number of these studies reported that quality assessment was not conducted as it is not a priority in scoping reviews or part of the scoping review methodology. Two studies reported the use of publication in a peer-reviewed publication as a proxy for good quality (Baxter et al. , 2008 ; Pita et al. , 2011 ) and another reported using studies included in existing reviews or meta-analyses to ‘overcome’ the lack of quality assessment (MacDougall, 2011 ). Of the 22.4% (77/344) of articles that reported a critical appraisal step, the rigor with which it was conducted ranged from the reviewer's subjective assessment using a scale of high, medium, or low (Roland et al. , 2006 ), to the use of published tools such as the Jadad scale (Jadad et al. , 1996 ) for randomized control trials (Deshpande et al. , 2009 ; Borkhoff et al. , 2011 ).

The level of detail reported about the search strategy varied considerably across the reviews. Table ​ Table3 3 displays information about the search strategy reported in the included reviews by time. Overall, the detail of reporting for the search increased numerically over time. For example, 78.06% of reviews published after 2009 reported complete strings or a complete list of search terms, compared with 57.89% of reviews published between 2000 and 2004 and 67.82% of reviews published between 2005 and 2009.

Search strategy details reported in included reviews, by year

Table ​ Table4 4 summarizes how some of the results of the included reviews were reported and ‘charted’. A flow diagram was used to display the flow of articles from the initial search to final selection in 35.8% of reviews (123/344). Characteristics of included studies were often displayed in tables (82.9%; 285/344), ranging from basic tables that described the key characteristics of each included study, to cross-tabulation heat maps that used color-coding to highlight cell values. Study characteristics were also mapped graphically in 28.8% (99/344) of reviews, often in the form of histograms, scatterplots, or pie charts. Reviews from the software engineering sector frequently used bubble charts to map the data (Figure ​ (Figure2 2 is an example of a bubble chart). In summarizing the reviewed literature, 77.6% (267/344) of reviews noted gaps where little or no research had been conducted, and 77.9% (268/344) recommended topics or questions for future research.

Reporting of results the included scoping reviews

Bubble plot of scoping reviews published by year and sector. The size of a bubble is proportional to the number of scoping reviews published in the year and sector corresponding to the bubble coordinates.

Stakeholder consultation is an optional sixth-step in the Arksey and O'Malley ( 2005 ) framework and was reported in 39.8% (137/344) of reviews. This optional step was reported in 34.9% (38/109) of reviews that used the Arksey and O'Malley framework, compared with 42.13% (99/235) of reviews that did not. Stakeholders were most often consulted at the search phase to assist with keyword selection for the search strategy or help identify potential studies to include in the review (74.5%; 102/137). Stakeholders were less frequently involved in the interpretation of research findings (30.7%; 42/137) and in the provision of comments at the report writing stage (24.1%; 33/137). Ongoing interaction with stakeholders throughout the review process was reported in 25.9% (89/344) of all reviews. Comparing between sectors, the proportion of reviews that reported consulting with stakeholders was highest in the social sciences sector (71.4%; 10/14) and lowest in the software engineering sector (2.4%; 1/41).

3.4. Reported challenges and limitations

Limitations in the study approach were reported in 71.2% (245/344) of reviews. The most frequent limitation reported in the reviews was the possibility that the review may have missed some relevant studies (32.0%; 110/344). This limitation was frequently attributed to database selection (i.e., searching other databases may have identified additional relevant studies), exclusion of the gray literature from the search, time constraints, or the exclusion of studies published in a language other than English. In comparison with systematic reviews, one review noted that it was ‘unrealistic to retrieve and screen all the relevant literature’ in a scoping review due to its broader focus (Gentles et al. , 2010 ), and a few noted that all relevant studies may not have been identified as scoping reviews are not intended to be as exhaustive or comprehensive (Cameron et al. , 2008 ; Levac et al. , 2009 ; Boydell et al. , 2012 ).

The balance between breadth and depth of analysis was a challenge reported in some reviews. Brien et al. ( 2010 ) and Cronin de Chavez et al . ( 2005 ) reported that it was not feasible to conduct a comprehensive synthesis of the literature given the large volume of articles identified in their reviews. Depth of analysis was also reported to be limited by the time available to conduct the review (Freeman et al. , 2000 ; Gulliford et al. , 2001 ; Templeton et al. , 2006 ; Cahill et al. , 2008 ; Bostock et al. , 2009 ; Brodie et al. , 2009 ).

The lack of critical appraisal of included studies was reported as a study limitation in 16.0% (55/344) of reviews. One review commented that this was the primary limitation of scoping reviews (Feehan et al. , 2011 ), and others noted that without this step, scoping reviews cannot identify gaps in the literature related to low quality of research (Hand and Letts, 2009 ; Brien et al. , 2010 ). Additionally, two reviews reported that their results could not be used to make recommendations for policy or practice because they did not assess the quality of included studies (Bostrom et al. , 2011 ; Churchill et al. , 2011 ). Conversely, Njelesani et al . ( 2011 ) noted that ‘by not addressing the issues of quality appraisal, this study dealt with a greater range of study designs and methodologies than would have been included in a systematic review’, and McColl et al. ( 2009 ) commented that ‘the emphasis of a scoping study is on comprehensive coverage, rather than on a particular standard of evidence’.

4. Discussion

In this paper, we provided an overview of scoping reviews identified in the gray and published literature. Our search for scoping reviews in the published and gray literature aimed to be comprehensive while also balancing practicality and available resources. It was not within the remit of this scoping review to assess the methodological quality of individual scoping reviews included in the analysis. Based on the characteristics, range of methodologies and reported challenges in the included scoping reviews, we have proposed some recommendations for advancing the scoping review approach and enhancing the consistency with which they are undertaken and reported.

4.1. Overview of included scoping reviews

Our results corroborate that scoping reviews are a relatively new approach that has gained momentum as a distinct research activity in recent years. The identified reviews varied in terms of terminology, purpose, methodological rigor, and level of detail of reporting; therefore, there appears to be a lack of clarity or agreement around the appropriate methodology for scoping reviews. In a scoping review that reviewed 24 scoping reviews from the nursing literature, Davis et al. ( 2009 ) also reported that the included scoping reviews varied widely in terms of intent, procedural, and methodological rigor. Given that scoping reviews are a relatively new methodology for which there is not yet a universal study definition, definitive procedure or reporting guidelines, the variability with which scoping reviews have been conducted and reported to date is not surprising. However, efforts have been made by scoping review authors such as Arksey and O'Malley ( 2005 ); Anderson et al. ( 2008 ); Davis et al. ( 2009 ); Brien et al. ( 2010 ); Levac et al. ( 2010 ) and Daudt et al. ( 2013 ) to guide other researchers in undertaking and reporting scoping reviews, as well as clarifying, enhancing, and standardizing the methodology. Their efforts seem to be having some impact given the increase in the number of scoping reviews disseminated in the published and gray literature, the growth in the use of a methodological framework, and the greater amount of detail and consistency with which scoping review processes have been reported.

4.2. Recommendations

Levac et al. ( 2010 ) remarked that discrepancies in nomenclature between ‘scoping reviews’, ‘scoping studies’, ‘scoping literature reviews’, ‘scoping exercises’, and so on lead to confusion, and consequently used the term ‘scoping study’ for consistency with the Arksey and O'Malley framework. We agree that there is a need for consistency in terminology; however, we argue that the term ‘scoping review’ should be adopted in favor of ‘scoping study’ or the other terms that have been used to describe the method. Our review has found that ‘scoping review’ is the most commonly used term in the literature to denote the methodology and that a number of the other terms (i.e., scoping study, scoping exercise, and systematic mapping) have been used to describe a variety of primary research study designs. Furthermore, we find that the word ‘review’ more explicitly indicates that the term is referring to a type of literature review, compared with ‘study’ or ‘exercise’.

As scoping reviews share many of the same processes with the more commonly known systematic review, many of the included reviews compared and contrasted the two methods. We concur with Brien et al. ( 2010 ) that scoping reviews are often misinterpreted as a less rigorous version of a systematic review, when in fact they are a ‘different entity’ with a different set of purposes and objectives. We contend that researchers adopting a systematic review approach but with concessions in rigor to shorten the timescale, refer to the process as a ‘rapid review’. Scoping reviews are one method among many available to reviewing the literature (Arksey and O'Malley, 2005 ), and researchers need to consider their research question or study purpose when deciding which review approach is most appropriate. Additionally, given that some of the included reviews took over 1 year to complete, we agree that it would be wrong to necessarily assume that scoping reviews represent a quick alternative to a systematic review (Arksey and O'Malley, 2005 ).

There is an ongoing deliberation in the literature regarding the need for quality assessment of included studies in the scoping review process. While Arksey and O'Malley stated that ‘quality assessment does not form part of the scoping (review) remit’, they also acknowledged this to be a limitation of the method. This may explain why quality assessment was infrequently performed in the included reviews and why it was reported as a study limitation among a number of these reviews. In their follow-up recommendations to the Arksey and O'Malley framework, Levac et al. ( 2010 ) did not take a position on the matter but recommended that the debate on the need for quality assessment continue. However, a recent paper by Daudt et al. ( 2013 ) asserts that it is a necessary component of scoping reviews and should be performed using validated tools. We argue that scoping reviews should include all relevant literature regardless of methodological quality, given that their intent is to present an overview of the existing literature in a field of interest without synthesizing evidence from different studies (Arksey and O'Malley, 2005 ). In doing so, scoping reviews can provide a more complete overview of all the research activity related to a topic. However, we also recognize that some form of quality assessment of all included studies would enable the identification of gaps in the evidence base—and not just where research is lacking—and a better determination of the feasibility of a systematic review. The debate on the need for quality assessment should consider the challenges in assessing quality among the wide range of study designs and large volume of literature that can be included in scoping reviews (Levac et al. , 2010 ).

The lack of consistency among the included reviews was not surprising given the lack of a universal definition or purpose for scoping reviews (Anderson et al. , 2008 ; Davis et al. , 2009 ; Levac et al. , 2010 ; Daudt et al. , 2013 ). The most commonly cited definition scoping reviews may be the one set forth by Mays et al . ( 2001 ) and used by Arksey and O'Malley: ‘scoping studies aim to map rapidly the key concepts underpinning a research area and the main sources and types of evidence available and can be undertaken as standalone projects in their own right, especially where an area is complex or has not been reviewed extensively before’. However, we believe that a recently proposed definition by Daudt et al . ( 2013 ) is more straightforward and fitting of the method: ‘scoping studies aim to map the literature on a particular topic or research area and provide an opportunity to identify key concepts; gaps in the research; and types and sources of evidence to inform practice, policymaking, and research’. While we would replace the term ‘scoping studies’ with ‘scoping reviews’, we endorse the Daudt et al . definition because it clearly articulates that scoping reviews are a type of literature review and removes the emphasis away from being ‘rapid’ process.

It has been suggested that the optimal scoping review is ‘one that demonstrates procedural and methodological rigor in its application’ (Davis et al. , 2009 ). We found that some scoping reviews were not reported in sufficient detail to be able to demonstrate ‘rigor in its application’. When there is a lack of clarity or transparency relating to methodology, it is difficult to distinguish poor reporting from poor design. We agree that it is crucial for scoping review authors to clearly report the processes and procedures undertaken—as well as any limitations of the approach—to ensure that readers have sufficient information to determine the value of findings and recommendations (Arksey and O'Malley, 2005 ; Davis et al. , 2009 ). The development of reporting guidelines for scoping reviews would help to ensure the quality and transparency of those undertaken in the future (Brien et al. , 2010 ). Given that reporting guidelines do not currently exist for scoping reviews (Brien et al. , 2010 ), researchers conducting scoping reviews may want to consider using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses ( http://prisma-statement.org/ ) as a guide, where applicable.

4.3. Strengths and limitations of this scoping review

This scoping review used rigorous and transparent methods throughout the entire process. It was guided by a protocol reviewed by a research team with expertise in knowledge synthesis and scoping reviews. To ensure a broad search of the literature, the search strategy included four electronic bibliographic databases, the reference list of eighteen different articles, two internet search engines, the websites of relevant organizations, and the snowball technique. The relevance screening and data characterization forms were pretested by all reviewers and revised as needed prior to implementation. Each citation and article was reviewed by two independent reviewers who met in regular intervals to resolve conflicts. Our use of a bibliographic manager (RefWorks) in combination with systematic review software (DistillerSR) ensured that all citations and articles were properly accounted for during the process. Furthermore, an updated search was performed in October 2012 to enhance the timeliness of this review.

This review may not have identified all scoping reviews in the published and gray literature despite attempts to be as comprehensive as possible. Our search algorithm included nine different terms previously used to describe the scoping process; however, other terms may also exist. Although our search included two multidisciplinary databases (i.e., Scopus, Current Contents) and Google, the overall search strategy may have been biased toward health and sciences. Searching other bibliographic databases may have yielded additional published scoping reviews. While our review included any article published in English, French or Spanish, our search was conducted using only English terms. We may have missed some scoping reviews in the gray literature as only the first 100 hits from each Web search were screened for inclusion. Furthermore, we did not contact any researchers or experts for additional scoping reviews we may have missed.

Other reviewers may have included a slightly different set of reviews than those included in this present review. We adopted Arksey and O'Malley's definition for scoping reviews at the outset of the study and found that their simple definition was generally useful in guiding study selection. However, we encountered some challenges during study selection with reviews that also reported processes or definitions more typically associated with narrative, rapid or systematic reviews. We found that some reviews blurred the line between narrative and scoping reviews, between scoping and rapid reviews, and between scoping and systematic reviews. Our challenges echoed the questions: ‘where does one end and the other start?’ (Arksey and O'Malley, 2005 ) and ‘who decides whether a particular piece of work is a scoping (review) or not?’ (Anderson et al. , 2008 ). For this review, the pair of reviewers used their judgment to determine whether each review as a whole sufficiently met our study definition of a scoping review. On another note, characterization and interpretation of the included reviews were also subject to reviewer bias.

5. Conclusions

This scoping review of scoping reviews characterized and described the nature of scoping reviews in the published and gray literature. Scoping reviews are a relatively new approach to reviewing the literature, which has increased in popularity in recent years. As the purpose, methodological process, terminology, and reporting of scoping reviews have been highly variable, there is a need for their methodological standardization to maximize the utility and relevance of their findings. We agree that the establishment of a common definition and purpose for scoping reviews is an important step toward enhancing the consistency with which they are conducted (Levac et al. , 2010 ); this would provide a common platform from which debates regarding the methodology can ensue, and the basis for future methodological frameworks and reporting guidelines. We hope that the results of our study can contribute to the ongoing collective work of a number of researchers to further clarifying and enhancing the scoping review methodology.

Acknowledgments

Funding for this project was provided by the OMAFRA-University of Guelph Knowledge Translation and Transfer Program (#299514).

We would like to thank our project collaborators Lisa Waddell, Dr. Barbara Wilhelm, and Dr. Ian Young for their expertise and guidance throughout the project. We would also like to acknowledge Dr. Ian Young and Kathleen Gropp for their assistance in screening articles for the review.

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

A. R. and S. M. conceived of the study. M. P., A. R., and S. M. participated in the design of the study. M. P., J. G., and A. R. undertook the literature review process. All authors drafted the manuscript. All authors read and approved the final manuscript.

Supporting Information

Additional supporting information may be found in the online version of this article at the publisher's web site.

Supporting info item

• Anderson S, Allen P, Peckham S, Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health research policy and systems. 2008; 6 :7. DOI: 10.1186/1478-4505-6-7 . [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Arksey H, O'Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology: Theory and Practice. 2005; 8 (1):19–32. DOI: 10.1080/1364557032000119616 . [ Google Scholar ]
• Arksey H, O'Malley L, Baldwin S, Harris J. Services to Support Carers of People with Mental Health Problems: Overview Report for the National Co-ordinating Centre for NHS Service Delivery and Organisation R & D (NCCSDO) Southampton: National Co-ordinating Centre for NHS Service Delivery and Organisation; 2002. [ Google Scholar ]
• Armstrong R, Hall BJ, Doyle J, Waters E. ‘Scoping the scope’ of a Cochrane review. Journal of Public Health. 2011; 33 (1):147–150. [ PubMed ] [ Google Scholar ]
• Banks SB, Banks D. Abandoned mines drainage: impact assessment and mitigation of discharges from coal mines in the UK. Engineering Geology. 2001; 60 (1-4):31–37. [ Google Scholar ]
• Bassi J, Lau F, Bardal S. Use of information technology in medication reconciliation: a scoping review. Annals of Pharmacotherapy. 2010; 44 (5):885–897. DOI: 10.1345/aph.1M699 . [ PubMed ] [ Google Scholar ]
• Baxter K, Glendinning C, Clarke S. Making informed choices in social care: the importance of accessible information. Health and Social Care in the Community. 2008; 16 (2):197–207. DOI: 10.1111/j.1365-2524.2007.00742.x . [ PubMed ] [ Google Scholar ]
• Behrens EA, Sylvester P, Clearfield A. Assessment of a sodium nonatitanate and pharmacosiderate-type ion exchangers for strontium and cesium removal item DOE waste simulants. Environmental Science and Technology. 1998; 32 (1):101–107. DOI: 10.1021/es9704794 . [ Google Scholar ]
• Betz SF, Reinhart GJ, Lio FM, Chen C, Struthers RS. Overlapping, nonidentical binding sites of different classes of nonpeptide antagonists for the human gonadotropin-releasing hormone receptor. Journal of Medicinal Chemistry. 2006; 49 (2):637–647. DOI: 10.1021/jm0506928 . [ PubMed ] [ Google Scholar ]
• Bhargava R, Jewkes C, Domanti A. Proceedings of the 13th Australian Tunnelling Conference 2008: Melbourne, Victoria. Carlton: The Australasian Institute of Mining and Metallurgy. AusIMM; 2008. Microtunnelling solution for Bulimba creek trunk sewer; pp. 109–114. [ Google Scholar ]
• Bhavaraju MP. Composite-system Reliability Evaluation: Phase 1, Scoping Study: Final Report. Newark, NJ: Public Service Electric and Gas Co; 1987. [ Google Scholar ]
• Borkhoff CM, Wieland ML, Myasoedova E, Ahmad Z, Welch V, Hawker GA, Li LC, Buchbinder R, Ueffing E, Beaton D, Cardiel MH, Gabriel SE, Guillemin F, Adebajo AO, Bombardier C, Hajjaj-Hassouni N, Tugwell P. Reaching those most in need: a scoping review of interventions to improve health care quality for disadvantaged populations with osteoarthritis. Arthritis Care & Research. 2011; 63 (1):39–52. [ PubMed ] [ Google Scholar ]
• Bostock L, Brodie I, Clapton J, Fish S, Morris M, Kearney P, Rutter D. Increasing the Number of Care Leavers in ‘settled, Safe accommodation’: Scoping Review 3. London: Centre for Excellence and Outcomes in Children and Young People's Services; 2009. [ Google Scholar ]
• Bostrom AM, Slaughter SE, Chojecki D, Estabrooks CA. What Do We know about knowledge translation in the care of older adults? A scoping review. Journal of the American Medical Directors Association. 2011; 13 :210–219. DOI: 10.1016/j.jamda.2010.12.004 . [ PubMed ] [ Google Scholar ]
• Boydell KM, Gladstone BM, Volpe T, Allemang B, Stasiulis E. The production and dissemination of knowledge: a scoping review of arts-based health research. Forum: Qualitative Social Research. 2012; 13 (1) Art. 32. [ Google Scholar ]
• Brearley SG, Stamataki Z, Addington-Hall J, Foster C, Hodges L, Jarrett N, Richardson A, Scott I, Sharpe M, Stark D, Siller C, Ziegler L, Amir Z. The physical and practical problems experienced by cancer survivors: a rapid review and synthesis of the literature. European Journal of Oncology Nursing. 2011; 15 (3):204–212. DOI: 10.1016/j.ejon.2011.02.005 . [ PubMed ] [ Google Scholar ]
• Brien SE, Lorenzetti DL, Lewis S, Kennedy J, Ghali WA. Overview of a formal scoping review on health system report cards. Implementation Science. 2010; 5 (1):2. DOI: 10.1186/1748-5908-5-2 . [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Brodie I, Bostock L, Clapton J, Fish S, Fisher M, Morris M, Kearney P, Rutter D. Improving Educational Outcomes for Looked-after Children and Young People: Scoping Review 1. London: Centre for Excellence and Outcomes in Children and Young People's Services; 2009. [ Google Scholar ]
• Butcher RJ. A scoping study method for determining the viability of block caving a hard rock orebody. CIM Bulletin. 2002; 95 (1058):70–75. [ Google Scholar ]
• Cahill J, Barkham M, Hardy G, Gilbody S, Richards D, Bower P, Audin K, Connell J. A review and critical appraisal of measures of therapist–patient interactions in mental health settings. Health Technology Assessment. 2008; 12 (24):1–86. [ PubMed ] [ Google Scholar ]
• Camargo LA, Saad WA, Netto CR, Gentil CG, Antunes-Rodrigues J, Covian MR. Effects of catecholamines injected into the septal area of the rat brain on natriuresis, kaliuresis and diuresis. Canadian Journal of Physiology and Pharmacology. 1976; 54 (3):219–228. [ PubMed ] [ Google Scholar ]
• Cameron JI, Tsoi C, Marsella A. Optimizing stroke systems of care by enhancing transitions across care environments. Stroke. 2008; 39 (9):2637–2643. DOI: 10.1161/STROKEAHA.107.501064 . [ PubMed ] [ Google Scholar ]
• Campbell Collaboration. What is a Systematic Review? 2013. Available at: http://www.campbellcollaboration.org/what_is_a_systematic_review/ [Accessed: 2013, 04/14] [ Google Scholar ]
• Campbell C, Parent M, Plangger K, Fulgoni GM. Instant innovation: from zero to full speed in fifteen years how online offerings have reshaped marketing research. Journal of Advertising Research. 2011; 51 (1):72–86. DOI: 10.2501/JAR-51-1-072-086 . [ Google Scholar ]
• Carr-Hill R, Currie L, Dixon P. Skill Mix in Secondary Care: A Scoping Exercise. Southampton: National Co-ordinating Centre for NHS Service Delivery and Organisation R & D; 2003. [ Google Scholar ]
• Catts SV, O'Toole BI, Carr VJ, Lewin T, Neil A, Harris MG, Frost ADJ, Crissman BR, Eadie K, Evans RW. Appraising evidence for intervention effectiveness in early psychosis: conceptual framework and review of evaluation approaches. Australian and New Zealand Journal of Psychiatry. 2010; 44 (3):195–219. DOI: 10.3109/00048670903487167 . [ PubMed ] [ Google Scholar ]
• Centre for Reviews and Dissemination. A rapid scoping review of the feasibility of a COX-2 inhibition strategy for patients with transitional cell bladder cancer. A report to the funders. York: University of York; 2004. [ Google Scholar ]
• Churchill P, Otal D, Pemberton J, Ali A, Flageole H, Walton JM. Sclerotherapy for lymphatic malformations in children: a scoping review. Journal of Pediatric Surgery. 2011; 46 (5):912–922. DOI: 10.1016/j.jpedsurg.2011.02.027 . [ PubMed ] [ Google Scholar ]
• Connell J, Barkham M, Cahill J, Gilbody S, Madill A. A Systematic Scoping Review of the Research in Higher and Further Education. Lutterworth: British Association for Counselling & Psychotherapy; 2006. ISBN: 1905114176. [ Google Scholar ]
• Crilly T, Jashapara A, Ferlie E. Research Utilisation & Knowledge Mobilisation: A Scoping Review of the Literature. Southampton: National Institute for Health Research Service Delivery and Organisation Programme; 2010. [ Google Scholar ]
• Cronin De Chavez A, Backett-Milburn K, Parry O, Platt S. Understanding and researching wellbeing: its usage in different disciplines and potential for health research and health promotion. Health Education Journal. 2005; 64 (1):70–87. DOI: 10.1177/001789690506400108 . [ Google Scholar ]
• Crooks VA, Kingsbury P, Snyder J, Johnston R. What is known about the patient's experience of medical tourism? A scoping review. BMC Health Services Research. 2010; 10 :266. DOI: 10.1186/1472-6963-10-266 . [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Daudt HM, van Mossel C, Scott SJ. Enhancing the scoping study methodology: a large, inter-professional team's experience with Arksey and O'Malley's framework. BMC Medical Research Methodology. 2013; 13 :48. DOI: 10.1186/1471-2288-13-48 . [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Davis K, Drey N, Gould D. What are scoping studies? A review of the nursing literature. International Journal of Nursing Studies. 2009; 46 (10):1386–1400. DOI: 10.1016/j.ijnurstu.2009.02.010 . [ PubMed ] [ Google Scholar ]
• Decaria JE, Sharp C, Petrella RJ. Scoping review report: obesity in older adults. International Journal of Obesity (2005) 2012; 36 (9):1141–1150. DOI: 10.1038/ijo.2012.29 . [ PubMed ] [ Google Scholar ]
• Descarries L, Watkins K, Garcia S, Beaudet A. The serotonin neurons in nucleus raphe dorsalis of adult rat: a light and electron microscope radioautographic study. Journal of Comparative Neurology. 1982; 207 (3):239–254. [ PubMed ] [ Google Scholar ]
• Deshpande A, Khoja S, Lorca J, McKibbon A, Rizo C, Husereau D, Jadad AR. Asynchronous telehealth: a scoping review of analytic studies. Open Medicine. 2009; 3 (2):39–61. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• DiCenso A, Martin-Misener R, Bryant-Lukosius D, Bourgeault I, Kilpatrick K, Donald F, Kaasalainen S, Harbman P, Carter N, Kioke S, Abelson J, McKinlay RJ, Pasic D, Wasyluk B, Vohra J, Charbonneau-Smith R. Advanced practice nursing in Canada: overview of a decision support synthesis. Nursing Leadership (Toronto, Ont.) 2010; 23 :15–34. [ PubMed ] [ Google Scholar ]
• Dohoo IR, Martin W, Stryhn H. Methods in Epidemiologic Research. Charlottetown, Prince Edward Island: VER Inc; 2012. [ Google Scholar ]
• Faraker CA, Boxer ME. Rapid review (partial rescreening) of cervical cytology. Four years experience and quality assurance implications. Journal of Clinical Pathology. 1996; 49 (7):587–591. DOI: 10.1136/jcp.49.7.587 . [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Feehan LM, Beck CA, Harris SR, MacIntyre DL, Li LC. Exercise prescription after fragility fracture in older adults: a scoping review. Osteoporosis International. 2011; 22 (5):1289–1322. DOI: 10.1007/s00198-010-1408-x . [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Forrest F, Lorenz K, Thompson T, Keenliside J, Kendall J, Charest J. A scoping study of livestock antimicrobials in agricultural streams of Alberta. Canadian Water Resources Journal. 2011; 36 (1):1–16. DOI: 10.4296/cwrj3601001 . [ Google Scholar ]
• Fotaki M, Boyd A, Smith E, McDonald R, Roland M, Sheaff R, Edwards A, Elwyn G. Patient Choice and the Organisation and Delivery of Health Services: Scoping Review. Southampton: National Co-ordinating Centre for NHS Service Delivery and Organisation; 2005. [ Google Scholar ]
• Freeman G, Shepperd S, Robinson I, Ehrich K, Richards S. Continuity of Care: Report of a Scoping Exercise. Southampton: National Co-ordinating Centre for NHS Service Delivery and Organisation R & D; 2000. [ Google Scholar ]
• Frist S. Rapid review of cervical cytology. Journal of Clinical Pathology. 1997; 50 (1):87. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Gagliardi AR, Fenech D, Eskicioglu C, Nathens AB, McLeod R. Factors influencing antibiotic prophylaxis for surgical site infection prevention in general surgery: a review of the literature. Canadian Journal of Surgery. 2009; 52 (6):481–489. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Gentles SJ, Lokker C, McKibbon KA. Health information technology to facilitate communication involving health care providers, caregivers, and pediatric patients: a scoping review. Journal of Medical Internet Research. 2010; 12 (2):e22. DOI: 10.2196/jmir.1390 . [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information and Libraries Journal. 2009; 26 (2):91–108. DOI: 10.1111/j.1471-1842.2009.00848.x . [ PubMed ] [ Google Scholar ]
• Gulliford M, Morgan M, Hughes D, Beech R, Figeroa-Munoz J, Gibson B, Hudson M, Arumugam C, Connell P, Mohiddin A, Sedgwick J. Access to Health Care: Report of a Scoping Exercise. Southampton: National Co-ordinating Centre for NHS Service Delivery and Organisation R & D; 2001. [ Google Scholar ]
• Gunnell Y. Topography, palaeosurfaces and denudation over the Karnataka Uplands, southern India. Geological Society Special Publication. 1997; 120 (120):249–267. [ Google Scholar ]
• Hand C, Letts L. Occupational Therapy Research and Practice involving Adults with Chronic Diseases: A Scoping Review and Internet Scan. Ottawa: Canadian Association of Occupational Therapists; 2009. [ Google Scholar ]
• Hazel N. Holidays for children and families in need: an exploration of the research and policy context for social tourism in the UK. Children and Society. 2005; 19 (3):225–236. DOI: 10.1002/chi.838 . [ Google Scholar ]
• Healthcare Improvement Scotland. What is the Published Evidence of an Association Between Hospital Volume and Operative Mortality for Surgical Repair (Open and Endovascular) of Unruptured and Ruptured Abdominal Aortic Aneurysms? Edinburgh: Healthcare Improvement Scotland; 2011. [ Google Scholar ]
• Healthcare Improvement Scotland. In Patients With Severe Medically Refractory Gastroparesis (Such as Those Requiring Nutritional Support), how Effective and Cost Effective is Gastric Electrical Stimulation (EnterraTM Device) in Reducing Symptoms, Reducing Requirement for Nutritional Support or Hospitalisation and Improving Quality of Life, When Compared With Medical or Alternative Surgical Management? Edingburgh: Healthcare Improvement Scotland; 2012. [ Google Scholar ]
• Heller T, McCubbin JA, Drum C, Peterson J. Physical activity and nutrition health promotion interventions: what is working for people with intellectual disabilities? Intellectual and Developmental Disabilities. 2011; 49 (1):26–36. DOI: 10.1352/1934-9556-49.1.26 . [ PubMed ] [ Google Scholar ]
• Hepplestone S, Holden G, Irwin B, Parkin HJ, Thorpe L. Using technology to encourage student engagement with feedback: a literature review. ALT-J: Research in Learning Technology. 2011; 19 (2):117–127. DOI: 10.1080/21567069.2011.586677 . [ Google Scholar ]
• Hetrick SE, Parker AG, Callahan P, Purcell R. Evidence mapping: illustrating an emerging methodology to improve evidence-based practice in youth mental health. Journal of Evaluation in Clinical Practice. 2010; 16 (6):1025–1030. DOI: 10.1111/j.1365-2753.2008.01112.x . [ PubMed ] [ Google Scholar ]
• Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions. 2011. [updated March 2011]. The Cochrane Collaboration. Available at: http://www.cochrane-handbook.org . [ Google Scholar ]
• Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, McQuay HJ. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Controlled Clinical Trials. 1996; 17 (1):1–12. [ PubMed ] [ Google Scholar ]
• Jaskiewicz W, Tulenko K. Increasing community health worker productivity and effectiveness: a review of the influence of the work environment. Human Resources for Health. 2012; 10 (38) DOI: 10.1186/1478-4491-10-38 . [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Johnston R, Crooks VA, Snyder J, Kingsbury P. What is known about the effects of medical tourism in destination and departure countries? A scoping review. International Journal for Equity in Health. 2010; 9 :24. DOI: 10.1186/1475-9276-9-24 . [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Kitchenham B, Charters S. Guidelines for Performing Systematic Literature Reviews in Software Engineering. Technical Report EBSE 2007-001. United Kingdom: Keele University and Durham University Joint Report; 2007. [ Google Scholar ]
• Kushki A, Chau T, Anagnostou E. Handwriting difficulties in children with autism spectrum disorders: a scoping review. Journal of Autism and Developmental Disorders. 2011; 41 :1–11. DOI: 10.1007/s10803-011-1206-0 . [ PubMed ] [ Google Scholar ]
• Levac D, Wishart L, Missiuna C, Wright V. The application of motor learning strategies within functionally based interventions for children with neuromotor conditions. Pediatric Physical Therapy. 2009; 21 (4):345–355. DOI: 10.1097/PEP.0b013e3181beb09d . [ PubMed ] [ Google Scholar ]
• Levac D, Colquhoun H, O'Brien KK. Scoping studies: advancing the methodology. Implementation Science. 2010; 5 (1):69. DOI: 10.1186/1748-5908-5-69 . [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Levy RD, Sanghvi AP. 1986. Value-based utility planning: scoping study. Final Report. Electric Power Research Institute (Report) EPRI EM .
• Liu P, Parker AG, Hetrick SE, Callahan P, de Silva S, Purcell R. An evidence map of interventions across premorbid, ultra-high risk and first episode phases of psychosis. Schizophrenia Research. 2010; 123 (1):37–44. DOI: 10.1016/j.schres.2010.05.004 . [ PubMed ] [ Google Scholar ]
• Liu S, Wang H, Fang S, Huang G, Tian L, Zhou D, Anonymous . Proceedings of the Society for Exploration Geophysicists International Exposition and 81st Annual Meeting: San Antonio, Texas, USA. Red Hook, NY: Curran Associates, Inc; 2011. 3D traveltime computation from rugged topography in VTI/TTI media; pp. 320–324. DOI: 10.1190/1.3627862 . [ Google Scholar ]
• MacDougall A. Is periodontal disease related to adverse pregnancy outcomes? A scoping review. Canadian Journal of Dental Hygiene. 2011; 45 (1):53–60. [ Google Scholar ]
• Malloch M, Burgess C. A Scoping Study of Services for Young Runaways: Final Report. Stirling, Scotland: Scottish Coalition for Young Runaways; 2007. [ Google Scholar ]
• Marsella A. Exploring the literature surrounding the transition into palliative care: a scoping review. International Journal of Palliative Nursing. 2009; 15 (4):186–189. [ PubMed ] [ Google Scholar ]
• Mays N, Roberts E, Popay J. Synthesizing research evidence. In: Fulop N, Allen P, Clarke A, Black N, editors. Studying the Organisation and Delivery of Health Services: Research methods. London: Routledge; 2001. pp. 188–219. [ Google Scholar ]
• McColl MA, Shortt S, Godwin M, Smith K, Rowe K, O'Brien P, Donnelly C. Models for integrating rehabilitation and primary care: a scoping study. Archives of Physical Medicine and Rehabilitation. 2009; 90 (9):1523–1531. DOI: 10.1016/j.apmr.2009.03.017 . [ PubMed ] [ Google Scholar ]
• Meredith A, Hussain Z, Griffiths MD. Online gaming: a scoping study of massively multi-player online role playing games. Electronic Commerce Research. 2009; 9 (1-2):3–26. DOI: 10.1007/s10660-009-9029-1 . [ Google Scholar ]
• Njelesani J, Couto S, Cameron D. Disability and rehabilitation in Tanzania: a review of the literature. Disability and Rehabilitation. 2011; 33 :2196–2207. DOI: 10.3109/09638288.2011.563817 . [ PubMed ] [ Google Scholar ]
• Noda H, Fujikado T. Topography of the oculomotor area of the cerebellar vermis in macaques as determined by microstimulation. Journal of Neurophysiology. 1987; 58 (2):359–378. [ PubMed ] [ Google Scholar ]
• Norwood J, Skinner B. Implementing RFID in a hospital library: a scoping study. Health Information & Libraries Journal. 2012; 29 (2):162–165. DOI: 10.1111/j.1471-1842.2012.00987.x . [ PubMed ] [ Google Scholar ]
• Petersen K, Feldt R, Mujtaba S, Mattsson M. Systematic mapping studies in software engineering. In: Visaggio G, Baldassarre MT, Linkman S, Turner M, editors. Proceedings of the 12th International Conference on Evaluation and Assessment in Software Engineering (EASE 2008): Bari, Italy. Swindon: BCS eWIC; 2008. pp. 1–10. [ Google Scholar ]
• Pita C, Pierce GJ, Theodossiou I, Macpherson K. An overview of commercial fishers' attitudes towards marine protected areas. Hydrobiologia. 2011; 670 :289–306. DOI: 10.1007/s10750-011-0665-9 . [ Google Scholar ]
• Ravenek MJ, Bryson-Campbell MM, Shaw L, Hughes ID. Perspectives on prevention, assessment, and rehabilitation of low back pain in WORK. Work. 2010; 35 (3):269–282. DOI: 10.3233/WOR-2010-0990 . [ PubMed ] [ Google Scholar ]
• Roland M, McDonald R, Sibbald B. Outpatient Services and Primary Care: A Scoping Review of Research into Strategies for Improving Outpatient Effectiveness and Efficiency. Southampton: National Co-ordinating Centre for the NIHR SDO; 2006. [ Google Scholar ]
• Rumrill PD, Fitzgerald SM, Merchant WR. Using scoping literature reviews as a means of understanding and interpreting existing literature. Work. 2010; 35 (3):399–404. DOI: 10.3233/WOR-2010-0998 . [ PubMed ] [ Google Scholar ]
• Saraiva J, Barreiros E, Almeida A, Lima F, Alencar A, Lima G, Soares S, Castor F. Aspect-oriented software maintenance metrics: a systematic mapping study. In: Baldassarre MT, Genero M, Mendes E, Piattini M, editors. Proceedings of the 16th International Conference on Evaluation & Assessment in Software Engineering: Ciudad Real, Spain. United Kingdom: The Institution of Engineering and Technology; 2012. pp. 253–262. DOI: 10.1049/ic.2012.0033 . [ Google Scholar ]
• Sawka AM, Ismaila N, Cranney A, Thabane L, Kastner M, Gafni A, Woodhouse LJ, Crilly R, Cheung AM, Adachi JD, Josse RG, Papaioannou A. A scoping review of strategies for the prevention of hip fracture in elderly nursing home residents. PLoS ONE. 2010; 5 (3):e9515. DOI: 10.1371/journal.pone.0009515 . [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Shield PW, Cox NC. The sensitivity of rapid (partial) review of cervical smears. Cytopathology. 1998; 9 (2):84–92. DOI: 10.1046/j.1365-2303.1998.00138.x . [ PubMed ] [ Google Scholar ]
• Snyder J, Crooks VA, Johnston R, Kingsbury P. What do we know about Canadian involvement in medical tourism?: a scoping review. Open Medicine. 2011; 5 (3):e139–48. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Spilsbury K, Hewitt C, Stirk L, Bowman C. The relationship between nurse staffing and quality of care in nursing homes: a systematic review. International Journal of Nursing Studies. 2011; 48 (6):732–750. DOI: 10.1016/j.ijnurstu.2011.02.014 . [ PubMed ] [ Google Scholar ]
• Stevinson C, Lawlor DA. Searching multiple databases for systematic reviews: added value or diminishing returns? Complementary Therapies in Medicine. 2004; 12 (4):228–232. DOI: 10.1016/j.ctim.2004.09.003 . [ PubMed ] [ Google Scholar ]
• Templeton L, Zohhadi S, Galvani S, Velleman R. “Looking Beyond Risk” Parental Substance Misuse: Scoping Study. Edinburgh: Scottish Executive; 2006. [ Google Scholar ]
• Trivedi D, Brooks F, Bunn F, Graham M. Early fatherhood: a mapping of the evidence base relating to pregnancy prevention and parenting support. Health Education Research. 2009; 24 (6):999–1028. DOI: 10.1093/her/cyp025 . [ PubMed ] [ Google Scholar ]
• Victoor A, Delnoij DM, Friele RD, Rademakers JJ. Determinants of patient choice of healthcare providers: a scoping review. BMC Health Services Research. 2012; 12 (1):272. DOI: 10.1186/1472-6963-12-272 . [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Vissandjee B, Hyman I, Spitzer DL, Apale A, Kamrun N. Integration, clarification, substantiation: sex, gender, ethnicity and migration as social determinants of women's health. Journal of International Women's Studies. 2007; 8 (4):32–48. [ Google Scholar ]
• Wallace LM, Koutantji M, Spurgeon P, Vincent C, Benn J, Earll L. Reporting Systems: A Scoping Study of Methods of Providing Feedback Within an Organization—Report to the Department of Health Patient Safety Research Programme. United Kingdom: Department of Health Patient Safety Research Programme; 2006. [ Google Scholar ]
• While A, Forbes A, Ullman R, Murgatroyd B. The Contribution of Nurses, Midwives and Health Visitors to Child Health and Child Health Services: A Scoping Review. Southampton: National Co-ordinating Centre for NHS Service Delivery and Organisation; 2005. [ Google Scholar ]
• Williams B, Powell A, Hoskins G, Neville R. Exploring and explaining low participation in physical activity among children and young people with asthma: a review. BMC Family Practice. 2008; 9 :40. DOI: 10.1186/1471-2296-9-40 . [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Willis P, Ward N, Fish J. Searching for LGBT carers: mapping a research agenda in social work and social care. British Journal of Social Work. 2011; 41 (7):1304–1320. DOI: 10.1093/bjsw/bcr114 . [ Google Scholar ]