example of quantitative research in environmental science

Research Topics & Ideas: Environment

F inding and choosing a strong research topic is the critical first step when it comes to crafting a high-quality dissertation, thesis or research project. Here, we’ll explore a variety research ideas and topic thought-starters related to various environmental science disciplines, including ecology, oceanography, hydrology, geology, soil science, environmental chemistry, environmental economics, and environmental ethics.

NB – This is just the start…

The topic ideation and evaluation process has multiple steps . In this post, we’ll kickstart the process by sharing some research topic ideas within the environmental sciences. This is the starting point though. To develop a well-defined research topic, you’ll need to identify a clear and convincing research gap , along with a well-justified plan of action to fill that gap.

If you’re new to the oftentimes perplexing world of research, or if this is your first time undertaking a formal academic research project, be sure to check out our free dissertation mini-course. Also be sure to also sign up for our free webinar that explores how to develop a high-quality research topic from scratch.

Overview: Environmental Topics

• Ecology /ecological science
• Atmospheric science
• Oceanography
• Soil science
• Environmental chemistry
• Environmental economics
• Environmental ethics
• Examples  of dissertations and theses

Topics & Ideas: Ecological Science

• The impact of land-use change on species diversity and ecosystem functioning in agricultural landscapes
• The role of disturbances such as fire and drought in shaping arid ecosystems
• The impact of climate change on the distribution of migratory marine species
• Investigating the role of mutualistic plant-insect relationships in maintaining ecosystem stability
• The effects of invasive plant species on ecosystem structure and function
• The impact of habitat fragmentation caused by road construction on species diversity and population dynamics in the tropics
• The role of ecosystem services in urban areas and their economic value to a developing nation
• The effectiveness of different grassland restoration techniques in degraded ecosystems
• The impact of land-use change through agriculture and urbanisation on soil microbial communities in a temperate environment
• The role of microbial diversity in ecosystem health and nutrient cycling in an African savannah

Topics & Ideas: Atmospheric Science

• The impact of climate change on atmospheric circulation patterns above tropical rainforests
• The role of atmospheric aerosols in cloud formation and precipitation above cities with high pollution levels
• The impact of agricultural land-use change on global atmospheric composition
• Investigating the role of atmospheric convection in severe weather events in the tropics
• The impact of urbanisation on regional and global atmospheric ozone levels
• The impact of sea surface temperature on atmospheric circulation and tropical cyclones
• The impact of solar flares on the Earth’s atmospheric composition
• The impact of climate change on atmospheric turbulence and air transportation safety
• The impact of stratospheric ozone depletion on atmospheric circulation and climate change
• The role of atmospheric rivers in global water supply and sea-ice formation

Topics & Ideas: Oceanography

• The impact of ocean acidification on kelp forests and biogeochemical cycles
• The role of ocean currents in distributing heat and regulating desert rain
• The impact of carbon monoxide pollution on ocean chemistry and biogeochemical cycles
• Investigating the role of ocean mixing in regulating coastal climates
• The impact of sea level rise on the resource availability of low-income coastal communities
• The impact of ocean warming on the distribution and migration patterns of marine mammals
• The impact of ocean deoxygenation on biogeochemical cycles in the arctic
• The role of ocean-atmosphere interactions in regulating rainfall in arid regions
• The impact of ocean eddies on global ocean circulation and plankton distribution
• The role of ocean-ice interactions in regulating the Earth’s climate and sea level

Tops & Ideas: Hydrology

• The impact of agricultural land-use change on water resources and hydrologic cycles in temperate regions
• The impact of agricultural groundwater availability on irrigation practices in the global south
• The impact of rising sea-surface temperatures on global precipitation patterns and water availability
• Investigating the role of wetlands in regulating water resources for riparian forests
• The impact of tropical ranches on river and stream ecosystems and water quality
• The impact of urbanisation on regional and local hydrologic cycles and water resources for agriculture
• The role of snow cover and mountain hydrology in regulating regional agricultural water resources
• The impact of drought on food security in arid and semi-arid regions
• The role of groundwater recharge in sustaining water resources in arid and semi-arid environments
• The impact of sea level rise on coastal hydrology and the quality of water resources

Topics & Ideas: Geology

• The impact of tectonic activity on the East African rift valley
• The role of mineral deposits in shaping ancient human societies
• The impact of sea-level rise on coastal geomorphology and shoreline evolution
• Investigating the role of erosion in shaping the landscape and impacting desertification
• The impact of mining on soil stability and landslide potential
• The impact of volcanic activity on incoming solar radiation and climate
• The role of geothermal energy in decarbonising the energy mix of megacities
• The impact of Earth’s magnetic field on geological processes and solar wind
• The impact of plate tectonics on the evolution of mammals
• The role of the distribution of mineral resources in shaping human societies and economies, with emphasis on sustainability

Topics & Ideas: Soil Science

• The impact of dam building on soil quality and fertility
• The role of soil organic matter in regulating nutrient cycles in agricultural land
• The impact of climate change on soil erosion and soil organic carbon storage in peatlands
• Investigating the role of above-below-ground interactions in nutrient cycling and soil health
• The impact of deforestation on soil degradation and soil fertility
• The role of soil texture and structure in regulating water and nutrient availability in boreal forests
• The impact of sustainable land management practices on soil health and soil organic matter
• The impact of wetland modification on soil structure and function
• The role of soil-atmosphere exchange and carbon sequestration in regulating regional and global climate
• The impact of salinization on soil health and crop productivity in coastal communities

Topics & Ideas: Environmental Chemistry

• The impact of cobalt mining on water quality and the fate of contaminants in the environment
• The role of atmospheric chemistry in shaping air quality and climate change
• The impact of soil chemistry on nutrient availability and plant growth in wheat monoculture
• Investigating the fate and transport of heavy metal contaminants in the environment
• The impact of climate change on biochemical cycling in tropical rainforests
• The impact of various types of land-use change on biochemical cycling
• The role of soil microbes in mediating contaminant degradation in the environment
• The impact of chemical and oil spills on freshwater and soil chemistry
• The role of atmospheric nitrogen deposition in shaping water and soil chemistry
• The impact of over-irrigation on the cycling and fate of persistent organic pollutants in the environment

Topics & Ideas: Environmental Economics

• The impact of climate change on the economies of developing nations
• The role of market-based mechanisms in promoting sustainable use of forest resources
• The impact of environmental regulations on economic growth and competitiveness
• Investigating the economic benefits and costs of ecosystem services for African countries
• The impact of renewable energy policies on regional and global energy markets
• The role of water markets in promoting sustainable water use in southern Africa
• The impact of land-use change in rural areas on regional and global economies
• The impact of environmental disasters on local and national economies
• The role of green technologies and innovation in shaping the zero-carbon transition and the knock-on effects for local economies
• The impact of environmental and natural resource policies on income distribution and poverty of rural communities

Need a helping hand?

Topics & Ideas: Environmental Ethics

• The ethical foundations of environmentalism and the environmental movement regarding renewable energy
• The role of values and ethics in shaping environmental policy and decision-making in the mining industry
• The impact of cultural and religious beliefs on environmental attitudes and behaviours in first world countries
• Investigating the ethics of biodiversity conservation and the protection of endangered species in palm oil plantations
• The ethical implications of sea-level rise for future generations and vulnerable coastal populations
• The role of ethical considerations in shaping sustainable use of natural forest resources
• The impact of environmental justice on marginalized communities and environmental policies in Asia
• The ethical implications of environmental risks and decision-making under uncertainty
• The role of ethics in shaping the transition to a low-carbon, sustainable future for the construction industry
• The impact of environmental values on consumer behaviour and the marketplace: a case study of the ‘bring your own shopping bag’ policy

Examples: Real Dissertation & Thesis Topics

While the ideas we’ve presented above are a decent starting point for finding a research topic, they are fairly generic and non-specific. So, it helps to look at actual dissertations and theses to see how this all comes together.

Below, we’ve included a selection of research projects from various environmental science-related degree programs to help refine your thinking. These are actual dissertations and theses, written as part of Master’s and PhD-level programs, so they can provide some useful insight as to what a research topic looks like in practice.

• The physiology of microorganisms in enhanced biological phosphorous removal (Saunders, 2014)
• The influence of the coastal front on heavy rainfall events along the east coast (Henson, 2019)
• Forage production and diversification for climate-smart tropical and temperate silvopastures (Dibala, 2019)
• Advancing spectral induced polarization for near surface geophysical characterization (Wang, 2021)
• Assessment of Chromophoric Dissolved Organic Matter and Thamnocephalus platyurus as Tools to Monitor Cyanobacterial Bloom Development and Toxicity (Hipsher, 2019)
• Evaluating the Removal of Microcystin Variants with Powdered Activated Carbon (Juang, 2020)
• The effect of hydrological restoration on nutrient concentrations, macroinvertebrate communities, and amphibian populations in Lake Erie coastal wetlands (Berg, 2019)
• Utilizing hydrologic soil grouping to estimate corn nitrogen rate recommendations (Bean, 2019)
• Fungal Function in House Dust and Dust from the International Space Station (Bope, 2021)
• Assessing Vulnerability and the Potential for Ecosystem-based Adaptation (EbA) in Sudan’s Blue Nile Basin (Mohamed, 2022)
• A Microbial Water Quality Analysis of the Recreational Zones in the Los Angeles River of Elysian Valley, CA (Nguyen, 2019)
• Dry Season Water Quality Study on Three Recreational Sites in the San Gabriel Mountains (Vallejo, 2019)
• Wastewater Treatment Plan for Unix Packaging Adjustment of the Potential Hydrogen (PH) Evaluation of Enzymatic Activity After the Addition of Cycle Disgestase Enzyme (Miessi, 2020)
• Laying the Genetic Foundation for the Conservation of Longhorn Fairy Shrimp (Kyle, 2021).

Looking at these titles, you can probably pick up that the research topics here are quite specific and narrowly-focused , compared to the generic ones presented earlier. To create a top-notch research topic, you will need to be precise and target a specific context with specific variables of interest . In other words, you’ll need to identify a clear, well-justified research gap.

Find The Perfect Research Topic

How To Choose A Research Topic: 5 Key Criteria

How To Choose A Research Topic Step-By-Step Tutorial With Examples + Free Topic...

Research Topics & Ideas: Automation & Robotics

Research Topics & Ideas: Robotics 50 Topic Ideas To Kickstart Your Research...

Research Topics & Ideas: Sociology

Research Topics & Ideas: Sociology 50 Topic Ideas To Kickstart Your Research...

Research Topics & Ideas: Public Health & Epidemiology

Research Topics & Ideas: Public Health 50 Topic Ideas To Kickstart Your Research...

Research Topics & Ideas: Neuroscience

Research Topics & Ideas: Neuroscience 50 Topic Ideas To Kickstart Your Research...

📄 FREE TEMPLATES

Research Topic Ideation

Proposal Writing

Literature Review

Methodology & Analysis

Academic Writing

Referencing & Citing

Apps, Tools & Tricks

The Grad Coach Podcast

12 Comments

research topics on climate change and environment

Researched PhD topics on environmental chemistry involving dust and water

I wish to learn things in a more advanced but simple way and with the hopes that I am in the right place.

Thank so much for the research topics. It really helped

the guides were really helpful

Research topics on environmental geology

Thanks for the research topics….I need a research topic on Geography

hi I need research questions ideas

Implications of climate variability on wildlife conservation on the west coast of Cameroon

I want the research on environmental planning and management

I want a topic on environmental sustainability

It good coaching

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

Submit Comment

• Print Friendly

• Search Menu
• Sign in through your institution
• Advance articles
• Editor's Choice
• Special Collections
• Author Guidelines
• Submission Site
• Open Access
• Reasons to submit
• About BioScience
• Journals Career Network
• Editorial Board
• Advertising and Corporate Services
• Self-Archiving Policy
• Potentially Offensive Content
• Terms and Conditions
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

Quantitative environmental science.

• Article contents
• Figures & tables
• Supplementary Data

Scott L Collins, Quantitative Environmental Science, BioScience , Volume 71, Issue 12, December 2021, Page 1199, https://doi.org/10.1093/biosci/biab131

• Permissions Icon Permissions

There is little argument that today's ecologists and conservation biologists are becoming more and more quantitative. A few years ago, I was part of a working group led by Stephanie Hampton that was held at the National Center for Ecological Analysis and Synthesis. The group focused on the training that is needed today for data-intensive environmental research. In a paper from that workshop that was published in BioScience ( https://doi.org/10.1093/biosci/bix025 ), we noted that quantitative skills to perform data-intensive research were generally lacking among most environmental scientists. We argued that, like writing skills, basic math skills should be taught across the curriculum. In addition, to be competitive in an increasingly quantitative world, students and professionals needed to acquire some degree of understanding of data management and processing, analysis, coding, and visualization along with communication skills for presentation and collaboration.

It is somewhat ironic that I was involved in developing these recommendations. I have a confession to make. This will come as no surprise to my colleagues and collaborators (and my graduate students), but my limited quantitative data processing skills have completely eroded over the decades since graduate school, when we ran SAS code on a mainframe computer. And yet, I completely agree that developing quantitative skills needs to be an essential component of undergraduate and graduate training. For example, this year, our National Science Foundation–sponsored Research Experience for Undergraduates (REU) program in dryland ecology held a 2-day Data Carpentries workshop for the REU students (and me). These very well structured and organized workshops provide an excellent entrée into data management and coding in R, the most popular data processing language for ecologists. The students then expanded their coding skills for data analysis and visualization during the rest of the summer session while I proceeded to forget everything I learned.

In this issue of BioScience , Nathan Emery and colleagues ( https://academic.oup.com/bioscience/article-lookup/doi/10.1093/biosci/biab107 ) up the game considerably. Here, the authors primarily focus on data science, per se, including the skills noted by Hampton and colleagues, as well as “being able to scale analyses for high-performance computing, write scripts, and use command line interfaces, version control, and high-performance computing clusters.” That is, environmental scientists could be engaged in training the next generation of data scientists. These authors maintain that teaching such “quantitative literacy” requires competent instructors but that most environmental scientists do not have sufficient data skills to incorporate data science into their courses. Herein lies the problem: A large gap exists between computational needs and the skill set of most environmental scientists. Worse yet, many training opportunities are targeted primarily toward early career scientists reducing the likelihood that more senior scientists will gain little more than a rudimentary understanding of these tool skills.

All in all, the needs are obvious, the intentions are well meaning, but the solutions are complicated and challenging. Time during the semester is limited, and with more and more demand to teach writing, math, and data science across the curriculum, instructors will have to gain new skills and willingly adjust content to meet the needs of students entering an increasingly quantitative world.

Email alerts

Citing articles via.

• Recommend to your Library

Affiliations

• Online ISSN 1525-3244
• Copyright © 2024 American Institute of Biological Sciences
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

• Current Students
• Faculty / Staff
• Paying for College
• Alumni Services
• Maine Transfer Guarantee
• Program Finder
• Affordable, Flexible, Accessible
• Distance Education
• All Online Courses & Degrees
• Bachelor’s Degrees Online
• Master’s Degrees Online
• Start Dates
• Admissions, Costs & Aid
• Partnerships
• Faculty and Contacts
• Academic and Career Support
• Student Testimonials
• Distance Education Advantage
• In-Person Education
• Sustainable Ventures
• Careers & Outcomes
• Strategic Plan
• Project Evolution
• Project Stratus
• About Unity
• Office of the President
• Our Evolution
• Sustainable Achievements & Initiatives
• Reinventing College
• Extended Reality (XR)
• Commencement
• Give to Unity Environmental University
• Institutional Communications
• Unity Environmental University News
• Unity Store

Frustrated with the 2024-2025 FAFSA ® Challenges? Unity Distance Education Is Here to Help and Support: Navigating the Issues and Finding Solutions

Frustrated with the 2024-2025 FAFSA ® Challenges?

The Importance of Quantitative Methods in Environmental Science and Sustainability Measurement

March 23, 2017

Home  /  News  /  The Importance of Quantitative Methods in Environmental Science and Sustainability Measurement

Environmental science is a continuously evolving academic field that seeks to help us gain a progressively better understanding of our natural world and develop effective solutions for important sustainability issues. Challenging the status quo to address environmental problems requires solid evidence to persuade decision makers of the necessity of change. This makes quantitative literacy essential for sustainability professionals to interpret scientific data and implement management procedures.

With our world facing increasingly complex environmental issues, quantitative techniques reduce the numerous uncertainties by providing a reliable representation of reality, enabling us to proceed toward potential solutions with greater confidence. A wide range of statistical tools and approaches are now available for sustainability scientists to measure environmental indicators and inform responsible policy-making.

How Quantitative Methods Provide Context for Environmental Science and Sustainability

Environmental science brings a transdisciplinary systems approach to analyzing sustainability concerns. As the intrinsic concept of sustainability can be interpreted according to diverse values and definitions, quantitative methods based on rigorous scientific research are crucial for establishing an evidence-based consensus on pertinent issues that provide a foundation for meaningful policy implementation.

Statistical evidence is often necessary to defend conservation conclusions

Descriptive and inferential statistical evidence provides a strong foundation for defending conclusions to various audiences. Applying an appropriate range of data sources and quantitative models can produce logical inferences to estimate the probability of future results while quantifying the extent of uncertainty, limits, and future research needs. Given the urgency of environmental issues like climate change and the prevalence of skeptics, effectively summarizing and communicating irrefutable results of complex statistical analyses can make the difference in developing successful courses of action.

How an M.S. in Sustainability Integrates Quantitative Methods

In an M.S. in Sustainability , such as natural resource management, students acquire the fundamental quantitative literacy to correctly evaluate and interpret ecological literature. They learn how to design effective studies, integrate quantitative models, and apply advanced statistical approaches.

For example, Bayesian methods are used to enable scientists to systematically factor in various forms of prior evidence while observing how conclusions change with the new information. This allows a quicker reaction to emerging conditions. Bayesian statistical inference has successfully been applied in conservation biology, addressing many of the problems inherent in standard hypothesis testing while including important factors causing uncertainty. It provides an alternate framework for decision-making that permits more options and better conclusions.

Statistical Models Mitigate Environmental Science and Sustainability Uncertainty

The principles of statistics and probability, multivariate analysis, and spatial analysis methods provide a common ground for scientists, engineers, and other environmental professionals to communicate with each other. Despite the sophistication of the latest mathematical models, the enormous complexity of interactions between environmental systems introduces some level of uncertainty into all predictions.

The quantitative methods acquired in a Sustainability Master’s online combine information from various sources to create more informed predictions, while importantly providing the scientific reasoning to accurately describe what is known and what is not. This quantification of uncertainty makes it impossible to dismiss climate and conservation models, therefore providing a clearer impetus for change.

Interested in pursuing a rewarding career in environmental science and sustainability ?

Contact Unity College to learn about our online Master’s Professional Science programs.

Start Your Journey

Looking for Answers

Get More Info

© Unity Environmental University 2024. “America’s Environmental University.™”

Privacy Overview

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Wiley Open Access Collection

Quantitative approaches in climate change ecology

Christopher j brown.

* School of Biological Sciences, The University of Queensland, St Lucia, QLD 4072, Australia

† Climate Adaptation Flagship, CSIRO Marine and Atmospheric Research, Ecosciences Precinct, Brisbane, QLD 4001, Australia

David S Schoeman

‡ Environmental Science Research Institute, School of Environmental Sciences, University of Ulster, Coleraine, BT52 1SA, UK

§ Department of Zoology, Nelson Mandela Metropolitan University, PO Box 77000, Port Elizabeth, 6031, South Africa

William J Sydeman

¶ Farallon Institute for Advanced Ecosystem Research, PO Box 750756, Petaluma, CA 94952, USA

Keith Brander

** National Institute of Aquatic Resources, Technical University of Denmark, Charlottenlund Castle, DK-2920, Charlottenlund, Denmark

Lauren B Buckley

†† Department of Biology, University of North Carolina, Chapel Hill, NC 27566, USA

Michael Burrows

‡‡ Scottish Association for Marine Science, Scottish Marine Institute, Oban, Argyll, PA 37 1QA, UK

Carlos M Duarte

§§ Department of Global Change Research, IMEDEA (UIB-CSIC), Instituto Mediterráneo de Estudios Avanzados, 07190, Esporles, MallorcaSpain

¶¶ The UWA Ocean Institute, University of Western Australia, 35 Stirling Highway, Crawley, WA 6009, Australia

Pippa J Moore

*** Centre for Marine Ecosystems Research, Edith Cowan University, Perth, WA 6027, Australia

††† Institute of Biological, Environmental and Rural Sciences, Aberystwyth University, Aberystwyth, SY23 3DA, UK

John M Pandolfi

‡‡‡ Australian Research Council Centre of Excellence for Coral Reef Studies, School of Biological Sciences, The University of Queensland, St. Lucia, QLD 4072, Australia

Elvira Poloczanska

William venables.

§§§ CSIRO Mathematics, Informatics and Statistics, Ecosciences Precinct, Brisbane, QLD 4001, Australia

Anthony J Richardson

¶¶¶ Centre for Applications in Natural Resource Mathematics (CARM), School of Mathematics and Physics, University of Queensland, St Lucia, QLD 4072, Australia

Contemporary impacts of anthropogenic climate change on ecosystems are increasingly being recognized. Documenting the extent of these impacts requires quantitative tools for analyses of ecological observations to distinguish climate impacts in noisy data and to understand interactions between climate variability and other drivers of change. To assist the development of reliable statistical approaches, we review the marine climate change literature and provide suggestions for quantitative approaches in climate change ecology. We compiled 267 peer-reviewed articles that examined relationships between climate change and marine ecological variables. Of the articles with time series data ( n = 186), 75% used statistics to test for a dependency of ecological variables on climate variables. We identified several common weaknesses in statistical approaches, including marginalizing other important non-climate drivers of change, ignoring temporal and spatial autocorrelation, averaging across spatial patterns and not reporting key metrics. We provide a list of issues that need to be addressed to make inferences more defensible, including the consideration of (i) data limitations and the comparability of data sets; (ii) alternative mechanisms for change; (iii) appropriate response variables; (iv) a suitable model for the process under study; (v) temporal autocorrelation; (vi) spatial autocorrelation and patterns; and (vii) the reporting of rates of change. While the focus of our review was marine studies, these suggestions are equally applicable to terrestrial studies. Consideration of these suggestions will help advance global knowledge of climate impacts and understanding of the processes driving ecological change.

Introduction

Although our knowledge of the impacts of anthropogenic climate change on biological systems is informed by the intersection of scientific theory, modelling, experiment and observation, it is only through observation that we can track the response of the biosphere to climate change. Understanding the extent of climate change impacts on ecosystems and their interactions with other anthropogenic stressors is a key requirement for informing policy debates on climate change and devising adaptive management responses ( Harley et al ., 2006 ; Edwards et al ., 2010 ). Our knowledge of observed biological impacts of climate change is biased towards terrestrial systems ( Richardson & Poloczanska, 2008 ); the analysis of observed climate impacts by the Intergovernmental Panel on Climate Change (2007) (their Figure 1.9) also indicates geographical imbalance in data availability.

Identifying the mechanisms driving change is especially challenging with marine biological data, because of short-term abiotic and biotic influences superimposed upon natural decadal climate cycles in the ocean-atmosphere system that can mask or accentuate climate change impacts ( Hare & Mantua, 2000 ; Beaugrand et al ., 2008 ; Möllmann et al ., 2008 ). Anthropogenic drivers other than climate change, including eutrophication ( Allen et al ., 1998 ), fishing ( Hsieh et al ., 2008 ; Genner et al ., 2010 ), pollution ( Perry et al ., 2005 ) and species introductions ( Loebl et al ., 2006 ) also interact with and complicate apparent ecological responses to climate change. Spatial variability in anthropogenic impacts and climate change ( Halpern et al ., 2008 ) mean that predictions from one region do not necessarily transfer to other regions. Furthermore, the availability of long time series suitable for generating baselines and for reliably testing hypotheses regarding climate impacts has been limited by funding and logistic issues ( Duarte et al ., 1992 ; Southward et al ., 2005 ; Edwards et al ., 2010 ). Despite these challenges, a long history of research has examined the influence of climate and other drivers on marine fisheries and ecosystem dynamics ( ICES 1948 , Colebrook, 1986 ; Ohman & Venrick, 2003 ; Southward et al ., 2005 ). Climate change ecology has emerged from this research (e.g. Hawkins et al ., 2003 ; Litzow & Ciannelli, 2007 ) and seeks to determine the extent of anthropogenic climate change impacts on ecosystems.

Appropriate statistical analyses are critical to ensure a sound basis for inferences made in climate change ecology. Many ecologists are trained in classical approaches more suited to testing effects in controlled experimental designs than in long-term observational data ( Hobbs & Hilborn, 2006 ). Observational data are collected in space and time, so replicates may show strong dependences or autocorrelation effects and explanatory variables are often confounded ( Legendre et al ., 2002 ). Approaches that do not account for these issues may increase the risk of incorrect inferences and reduce power to detect relationships between climate variables and biological responses. Inference strength will also depend on the summary statistic chosen to represent biological responses, such as a species’ range edges or centre. Climate change ecology requires a greater awareness of statistical issues and the appropriate tools for obtaining reliable inferences from limited data sources.

Here, we provide suggestions for making defensible inferences in climate change ecology. We reviewed the literature on observed responses of biota to climate change to assess and describe current statistical practices in marine climate change ecology. On the basis of our assessment, we identify areas where the application of appropriate statistical approaches could be strengthened, including testing other potentially important drivers of change and their interactions with climate, consideration of temporal auto-correlation in time series, consideration of spatial heterogeneity and reporting of rates of change. We then provide suggestions for reliable statistical approaches that consider limitations of available data and highlight individual studies where statistical analyses were particularly innovative and reliable. We emphasize the strengths of individual studies to underscore lessons for the broader research community. While our focus is marine, our suggestions for statistical approaches are equally relevant for climate change research on land. Application of defensible statistical approaches will provide a more rigorous foundation for climate change ecology, improve predictive power and speed delivery of science to policy-makers and managers.

Assessment of current statistical approaches in climate change ecology

We searched the peer-reviewed literature on climate change ecology for articles examining climate change impacts on the basis of observational studies. Our literature search was comprehensive and multi-faceted: extensive searches using Web of Science© and Google Scholar; citation searches; assessing every article published in key journals (Global Change Biology, Marine Ecology Progress Series, Progress in Oceanography, Global Ecology and Biogeography), analysis of reference lists in comprehensive reviews; assessment of studies from existing databases ( Rosenzweig et al ., 2008 ; Tasker, 2008 ; Wassmann et al ., 2011 ) and our knowledge of various marine habitats. Studies were retained for analysis if the authors assessed the impacts of climate change on marine taxa, if there were data over multiple years after 1960 (when signals of anthropogenic climate change first became apparent), and if the primary climate variable investigated (e.g. temperature, sea ice) showed a change that the authors considered consistent with the physical impacts of anthropogenic climate change. We thus included studies with biological responses that were consistent or inconsistent with climate change. Only studies with observational data were considered for the review; therefore, studies with only experimental or modelling results were excluded. This process resulted in 267 studies published from 1991 to 2010, 186 of which used regularly sampled time series data. Time series generally started during or after the onset of anthropogenic warming in the 1960s (82% of time series studies in our review); however, several started before the 1960s (e.g. Ohman & Venrick, 2003 ; Reid et al ., 2003 , Southward et al ., 2005 ). Data from palaeo-ecological studies dated as far back as 1700s ( Field et al ., 2006 ). For the time series studies, we recorded the type of statistical analysis used to relate climate and ecological variables, whether non-climatic factors were considered in analysis, and the methods used to deal with auto-correlation.

The review showed an accelerating number of studies with time series data published in climate change ecology through time ( Fig. 1a ), consistent with the overall increase in climate change impacts studies published through time ( Hoegh-Guldberg & Bruno, 2010 ). The proportion of studies using statistics to test relationships between climate and ecological variables has increased, doubling since before 2000 ( Fig. 1b ). The percentage of time series studies that accounts for or considers temporal auto-correlation remained around 65% ( Fig. 1c ). Both spatial analysis and modelling that accommodates non-climatic factors in addition to climate variables show increases over time, although rates of use remain low ( Fig. 1d,e ). Studies that report metrics on rates of change (e.g. km shifted per decade), useful for comparative studies and climate impacts syntheses, have also increased, although currently, only 41% of time series studies report these metrics ( Fig. 1f ). Together, the trends in use of statistics and reporting suggest that climate change ecologists have gradually been increasing their use of more reliable statistical methods, but overall, there is room to improve adoption and application of these methods.

Attributes through time of marine studies in climate change ecology. (a) Number of studies. The remainder show percentage of studies (b) using statistical tests, (c) accounting for temporal autocorrelation, (d) using spatial analysis, (e) accommodating non-climatic factors and (f) reporting of metrics (distribution and phenology studies only).

To assess how statistical analyses might be currently perceived in the climate change ecology literature and whether those using more reliable statistics might be more highly cited, we recorded the number of citations each paper from the database received (on 12th February 2011) and tested whether citations were related to the statistical characteristics of the analysis. We used several binary predictors to reflect characteristics and included: whether temporal autocorrelation was accounted for; whether spatial analysis was conducted; whether metrics on rates of change were reported; whether multiple predictors were considered. Publication year was included as a covariate (using a cubic spline) to account for the growth of citations over time. We used a generalized linear model with negative binomial errors ( Venables & Ripley, 2002 ) to model the effect of statistical characteristics on citation rate.

Generally, it might be expected that more reliable statistical approaches and reporting of metrics would improve a study's usefulness in the literature and hence the citation rate. Indeed, studies that use spatial methods may be cited slightly more often ( Fig. 2c ). Furthermore, studies that reported metrics on rates of change may also have slightly higher citation rates, suggesting that these studies are used more often in the literature because of the ease of comparison ( Fig. 2e ). Relative to the effect of years in print, the improvement in citations was slight and studies that accounted for temporal autocorrelation or modelled multiple factors were not cited more often ( Fig. 2b,d ).

Effect size plots from a negative binomial GLM that analysed the number of citations for marine climate impact studies using time series data. Shown are effect sizes plots with 95% confidence intervals for (a) cubic spline for publication year (3 degrees of freedom) (b) whether temporal autocorrelation was considered; (c) whether spatial analysis was conducted; (d) whether alternative non-climatic factors were considered in analysis; and (e) whether metrics were reported. For each plot parameters for other categorical variables were fixed to ‘No’ and 1 year in print.

The results of our review and citation analysis may indicate both inadequate awareness of appropriate statistical techniques for analysis of observational data and a lack of suitable data to support more sophisticated analyses. That studies employing more reliable statistical approaches were not more highly cited indicates a need for greater scrutiny of statistical approaches in marine climate change ecology. Data limitations are also important, and greater funding of marine ecological time series would allow a more comprehensive analysis of climate change impacts ( Duarte et al ., 1992 ; Southward et al ., 2005 ; Edwards et al ., 2010 ). Nevertheless, there are studies in the marine climate change ecology literature and from other research areas that illustrate a range of effective statistical approaches for maximizing the utility of available data. In the following sections, we use these studies as examples of how to make the most of available data, address statistical issues and as a basis for suggesting reliable methods for statistical analysis in climate change ecology.

Data requirements for assessing climate change impacts

Strongest inferences on impacts of climate change require observational data that cover long time spans and large spatial scales ( Parmesan et al ., 2011 ). However, funding constraints on the extent of data collection have limited the length of time series and their spatial extent (e.g. Southward et al ., 2005 ; Edwards et al ., 2010 ). Some examples of long time series that have persisted through funding cycles are the Continuous Plankton Recorder survey in the North Atlantic and North Pacific ( Colebrook, 1986 ; Reid et al ., 2003 ); the California Cooperative Oceanic Fisheries Investigations in the Californian Current ( Ohman & Venrick, 2003 ); and fish, zooplankton and rocky shore surveys conducted from Plymouth, UK ( Southward et al ., 1995 , 2005 ).

Longer and higher frequency time series data provide greater opportunities to investigate the effects of climate and anthropogenic impacts on ecosystems. In the English Channel, long-term cycles of rocky shore and pelagic fish communities coincide with cycles of cold and warm periods, providing strong evidence that modern shifts to warmer-water communities are a consequence of warming in the region ( Hawkins et al ., 2003 ; Southward et al ., 2005 ). Likewise, longer time series are required to provide baselines for assessing the impacts of anthropogenic climate change. Data from the English Channel demonstrate that while communities have cycled naturally over long periods, recent changes have exceeded those observed in the last warm period, in the 1950s, and are probably a result of anthropogenic climate change ( Mieszkowska et al ., 2007 ). Distinguishing the effects of multiple drivers also requires data that allow contrasts between strengths of each driver, because if the drivers co-vary strongly, it will be difficult to determine their individual effects. In this case, longer time series or data collected over a larger spatial scale potentially provide greater opportunities for sampling contrasts.

Comparing historical and contemporary data sets

Baselines for assessing climate impacts for data-poor regions or taxa can be obtained by conducting surveys in sites where historical data are available and comparisons can be made between present and historical data. While most studies in our database were based on regularly collected samples, samples collected at irregular intervals or those comparing two distinct periods in time were also common ( Fig. 3d ).

Number of studies that (a) use different methods for relating climate and biological time series; (b) use different methods for adjusting for temporal auto-correlation; (c) group data, use spatial analysis or do not consider spatial autocorrelation, and (d) use different data types. The number of relevant studies included for each figure is indicated. Studies that applied multiple different types of methods were counted once for each method. GLM, generalized linear model; GAM, generalized additive model; PCA, principal components analysis.

Data collection designs that pre-date the advent of modern statistical approaches pose challenges to comparisons with contemporary data sets ( Tingley & Beissinger, 2009 ). Differences in survey methods between past and present programs may confound biological responses to climate change. Similarly, a major problem for range-shift studies is determining the difference between true absences of species at a site and false absences that result from missed detection or historical records restricted to few species ( Tingley & Beissinger, 2009 ).

Nevertheless, historical data are valuable and should not be discarded because they pose challenges to analysis. Indeed, appropriate statistical approaches can assist with the integration of old and contemporary data. Often, careful consideration of changes in data collection methodology can identify biases that can then be factored out in analysis, for instance, by comparing changes in relative rather than absolute abundances of species ( Fodrie et al ., 2010 ). Tingley & Beissinger (2009) review approaches for comparisons of historical and contemporary data in range shift studies. In particular, methods for estimating detection probability of a species are useful for distinguishing false and true absences to provide more accurate mapping of range shifts. The lack of temporal continuity in comparisons with historical data also limits the ability to analyse the relationship between climate variables and species distribution. Using historical and contemporary data on seaweed distribution, Lima et al . (2007) apply a randomization procedure to explore whether range differences between the two time-periods are significantly greater than would be expected on the basis of distances between modern sub-populations. This approach allows Lima et al . (2007) to make stronger inferences about observed changes in range size.

Caution is required in the interpretation of differences between two points in time because patterns of variability in the intervening years are not captured. For instance, in the North-East Atlantic, comparisons between the 1960s and 2005 exaggerate warming because of unusually cold years in the 1960s ( Hawkins et al ., 2003 ; Southward et al ., 2005 ). Although two-point comparisons have been applied to a broad range of taxa in the literature, the most reliable comparisons will come from taxa with low inter-annual variability relative to the magnitude of change between the two time periods. The relative magnitude of inter-annual variability can sometimes be estimated by comparison to species with similar ecology or directly from data if multiple years are available at analysis start or end points (e.g. Sagarin et al ., 1999 had multiple years of data, from 1931–1933 and 1993–1996). A further disadvantage of point comparisons is the low power for discriminating among multiple drivers of change because most drivers will have changed between historical and present studies.

Nevertheless, point comparison analysis can at least partially overcome the disadvantages of low temporal resolution by including data on many species. For example, Fodrie et al . (2010) repeated historical surveys and compared abundances of fish in seagrass meadows between the 1970s and the present day. The community analysis revealed that cold-water species were less likely and warm-water species more likely to be observed in the present day, a result consistent with mechanisms of a climate change impact. Furthermore, a t -test comparing the pooled abundance of warm-water species between the historical period and the present day confirmed that warm-water species had increased in relative abundance. A final t -test showed a significant warming in regional temperature. It is important to note that the historical and recent period studied by Fodrie et al . (2010) were sufficiently separated in time (1970s vs. 2000s) to allow for a clear warming signal.

Retrospective data in climate impact studies

Given the relative paucity of long biological and ecological time series, retrospective methods for obtaining data to test for impacts of climate change provide a rich and relatively untapped resource. In particular, fast sedimentation rates in many areas of the ocean preserve micro-organisms over centuries to millennia and these sedimentary records can be examined in relation to recent climate changes. We found 13 retrospective studies in the literature review of climate change ecology and these included studies of fish otoliths ( Thresher et al ., 2007 ), calcifying plankton from sediment cores ( Field et al ., 2006 ) and coral cores ( De'ath et al ., 2009 ). Retrospective studies have great potential importance for assessing shifts in patterns of biological variability before and after the onset of warming, because they date to before detection of global warming signals in the 1960s.

Field et al . (2006) used sediment cores from the Californian Current region to examine long-term changes in the planktonic foraminifera community. Foraminfera preserve well in sedimentary records because of their calcium carbonate shell. The time series dated back to before global industrialization and demonstrated a shift from a cold-water community to a warm-water community around the 1970s that was unprecedented in the past 200 years. Furthermore, the shift in community structure showed a strong correlation with reconstructions of sea surface temperature.

A major shortcoming of many retrospective studies is the limited number of samples or sediment cores that can be obtained. So while temporal coverage may be high, spatial or sample-based replication may be low. The Field et al . (2006) study was based on just a few sediment cores, due to the difficulty of obtaining deep-sea cores. This limits the ability to examine temporal patterns in climate impacts over broad spatial scales using retrospective analyses.

Addressing statistical issues

A major challenge in statistical analysis is simultaneously minimizing risks of attributing causality to simple associative relationships and of missing relationships that are the result of real ecological processes. Properly formulated statistical tests of the relationship between the ecological variable of interest and a variable indicative of climate change help minimize these risks. Of the time series studies we reviewed, 47 (25%) did not use statistical tests to relate ecological trends to climate variables.

Aside from using properly formulated statistical tests, these errors can be minimized by formulating plausible mechanisms for the form, direction and magnitude of biological change. An understanding of mechanisms helps to build confidence that statistically weak but mechanistically plausible relationships are sound (for instance, when data are limited) and, similarly, helps to exclude statistically significant but spurious relationships. For example, inferential strength from observational studies can be improved by coupling the study with appropriate experimental studies (three studies in our review, Chevaldonné & Lejeusne, 2003 ; Iglesias-Rodriguez et al ., 2008 ; Halloran et al ., 2008 ). Chevaldonné & Lejeusne (2003) showed long-term declines in cold-water mysid abundances in Mediterranean caves attributable to warming. They were able to grow these mysids in the laboratory to demonstrate that contemporary warming was beyond their preferred temperature range. This approach is potentially a powerful way to investigate the mechanisms driving climate responses in organisms amenable to experimentation (e.g. intertidal invertebrates, macro-algae and corals). Hewitt et al . (2007) provide a comprehensive review on strategies for integrating small-scale, manipulative studies with large-scale correlative studies.

Accommodating multiple factors in analyses

When investigating ecosystem change, a host of anthropogenic impacts (including climate) and natural dynamics are confounded, complicating interpretation and potentially leading to spurious conclusions when important drivers are not included in analysis. Statistical analyses in our review were predominately univariate (correlation or simple linear regression, Fig. 3a ), which do not allow consideration of multiple factors and their interactions. Only 24 time series studies (13%) in the literature reviewed explicitly considered factors other than climatic variables in statistical analysis (e.g. Hsieh et al ., 2008 ; Poloczanska et al ., 2008 ). At the simplest and coarsest level, the often-strong trends in the primary climate variables considered (temperature, sea ice) can be correlated with increases in anthropogenic threats of eutrophication, fishing and pollution, as increases in both CO 2 emissions and human threats are a consequence of increases in human population and activity ( Halpern et al ., 2008 ). The lack of inclusion of alternative factors also implies that key interactions between drivers, which could be important for predicting and managing ecosystem responses to climate change, are not being addressed.

Of the studies that consider multiple factors in analysis, generalized linear modelling (including multiple regression), a method common in the broader ecological literature, was the most popular ( Fig. 3a , e.g. Dulvy et al ., 2008 ). Generalized additive models were also used by seven studies. There is already an extensive literature discussing application of these methods to modelling multiple factors and their interactions (see Table 1 for more details) and therefore we describe two examples below where innovative approaches were used to understand the influence of multiple explanatory variables.

Summary of statistical approaches described in the text, with references, and appropriate routines in the free statistical package R. See http://cran.r-project.org/ to obtain R, its packages and user guides ( R Development Core Team, 2010 )

NA, not available.

Along with climate change, fishing pressure is arguably the most widespread human impact on marine ecosystems ( Halpern et al ., 2008 ). Unfortunately, data on exploitation rates often do not exist or are difficult to obtain (but see Dulvy et al ., 2008 ; Genner et al ., 2010 ). Hsieh et al . (2008) used a novel approach to overcome the lack of data on temporal dynamics of exploitation rate. They analysed changes in the distribution of larval fish under ocean warming. To account for exploitation rates, they conducted a comparative analysis of the effects of climate on the spatial distribution of exploited and unexploited fish species. By comparing impacts of climate on species with similar life-history traits, they were able to partly control for effects introduced by differences among species, and focus on impacts of exploitation and climate on fish distribution. Importantly, their analysis demonstrated a synergism between climate and fishing impacts, with exploited species being more sensitive to climate-driven range shifts than unexploited species. As more studies incorporate climate change and other human threats into their statistical models, we should develop a greater understanding of how we can manage our marine systems to minimize the effect of climate change.

Considering multiple factors may also help test competing hypotheses regarding the structure of underlying relationships between a species, climate and its ecosystem. Analysis of multiple hypotheses is also important for assessing uncertainty in the outcomes of climate change impacts. Hobbs & Hilborn (2006) provide a useful guide on how multiple model formulations can be tested against observed data. One approach for multi-model inference is to develop structured models using path analysis and then to compare their ability to predict observations ( Table 1 ). For example, Poloczanska et al . (2008) investigated the recruitment of two barnacle species in relation to warming temperatures by constructing a hierarchy of models of increasing complexity. Different models considered the response of each species to warming individually and including interactions between species such as resource and interference competition. They found that climate change may be impacting directly one species, which was, in turn, impacting its competitor via interference competition. In this case, testing the ability of different models to predict observations provided a more reliable assessment of the climate change signal by identifying both the direct and indirect mechanisms of the climate change impact.

Identifying spurious relationships and accounting for auto-correlation in biological data

Temporal and spatial autocorrelation arise from non-independence of observations and are a common feature of time series and geographical studies ( Legendre et al ., 2002 ). Autocorrelation can be caused by factors exogenous to the variables of interest, such as unknown environmental effects on population size, and factors endogenous to the variables of interest, such as the effect of intra-specific competition species on population size. Temporal autocorrelation is commonly strong in marine ecological data. For instance, the same individuals will be counted in multiple years in population counts of longer lived species and data from heavily fished species are often strongly autocorrelated due to effects of economic development of fishing fleets and management regimes. Autocorrelation can occur over multiple time-scales in a dataset, including seasonal patterns at short time-scales and long-term trends due to gradual changes in observation methods or evolutionary change in the species studied. Similarly, spatial autocorrelation can occur at a range of scales. For instance, small-scale spatial autocorrelation may be observed in species that aggregate to breed or where individuals of a species disperse to avoid competition, and large-scale autocorrelation may be present if important environmental gradients are unspecified in models.

A basic assumption of most inferential statistical tests – that residuals are independently and identically distributed – will be violated if residuals are autocorrelated. Thus, autocorrelation that is unaccounted for can result in misleading inferences. In autocorrelated data, each measurement does not contribute a full degree of freedom to the analysis, so degrees of freedom in statistical tests are over-estimated, and this inflates the Type-I error rate (falsely rejecting true null hypotheses). For instance, Worm & Myers (2003) estimated effective degrees of freedom from fisheries data, and found that degrees of freedom may be inflated by up to six times in cod–shrimp correlations if autocorrelation is not considered. In many cases, exogenous autocorrelation may be removed if appropriate covariates are included in the model. Alternatively, it is necessary either to explicitly model the autocorrelation structure, or to adjust degrees of freedom in statistical tests (i.e. estimate the effective sample size, given the autocorrelation) on the basis of the autocorrelation structure (see Table 1 for how methods on detecting autocorrelation).

In the review of the climate change ecology literature, 68 studies (49%) analysing biological changes over time considered temporal autocorrelation ( Fig. 3b ). Further, 19 studies (21%) with data at multiple locations made explicit use of spatial methods that either accounted for spatial autocorrelation or modelled covariates spatially ( Fig. 3c ). Most studies grouped spatial data, thus not only avoiding issues with spatial autocorrelation, but also potentially removing important ecological patterns from analysis. In the following section, we discuss examples from climate change ecology that deal with temporal autocorrelation, spatial autocorrelation and spatial patterns in statistical analyses (for details of methods see Table 1 ). Many methods are common to both types of autocorrelation and therefore we provide references for further details.

Accounting for temporal autocorrelation and spurious relationships

The simplest approach to deal with temporal autocorrelation is to remove autocorrelation by differencing the climate and biological data series (subtract each data point from next data point in the time series, Table 1 ) over the autocorrelation time-scales prior to statistical analysis ( Pyper & Peterman, 1998 ). De-trending (subtracting the long-term trend from each data point, Table 1 ) may also be desirable to remove shared long-term trends because time series commonly trend without a causal link. However, removing trends can reduce the power to detect real relationships ( Pyper & Peterman, 1998 ) and, in some cases, differencing or detrending can increase the autocorrelation in a dataset. For instance, if measurements in a time series are independent, detrending the time series will create a dependency among data points. Historically, such data transformations were used to obtain datasets that met the assumptions of the statistical tests available. The advent of modern model-based approaches that accommodate autocorrelation processes provides the opportunity to avoid the shortcomings of data transformations.

When the climate–biological relationship is expected to operate over longer time-scales, the data can be smoothed using a filter before conducting statistical tests such as regression. Smoothing reduces the influence of short-term variability that is not of primary interest. For instance, Litzow & Ciannelli (2007) use a smoother to examine the inter-annual relationships among abundances of predators, prey and physical conditions, and Dulvy et al . (2008) use a smoothing filter on their environmental data to capture the integrated influence of the environment on species’ distribution over several years.

A method of accounting for autocorrelation in correlation tests that has gained particular favour in studies of climate impacts on plankton and fish communities (24 studies in all, e.g. Richardson & Schoeman, 2004 ; Litzow & Ciannelli, 2007 ; Möllmann et al ., 2008 ; Nye et al ., 2009 ; Beaugrand & Kirby, 2010 ), is to explicitly adjust the degrees of freedom downwards relative to the amount of temporal autocorrelation in the time series, before calculating significance levels ( Table 1 ). Pyper & Peterman (1998) used simulated data to test error rates for different methods of adjusting the degrees of freedom on a significance test of correlation coefficients. Their simulations indicated that methods for adjusting degrees of freedom reduce the risk of falsely attributing significance to a relationship without the loss of power that de-trending the data may cause. Thus, despite the greater technical knowledge required, these approaches are generally preferable to de-trending the data before testing a correlation. Dale & Fortin (2009) describe two straightforward methods for undertaking such analyses in a spatial context.

Potentially, the most powerful procedure for accounting for auto-correlation is to use an auto-regressive model ( Table 1 ). An auto-regression can be advantageous over correlation approaches with adjusted degrees of freedom because regression allows estimation of the rate of change of the biological variable and for multiple covariates to be considered simultaneously. Estimates of the autocorrelation structure may also suggest mechanisms for its cause. Researchers should carefully consider the mechanisms behind the proposed term, rather than choosing an auto-regressive model based on goodness of fit alone, because adding an auto-regressive term to a model can reduce the power to detect a change. For example, Brodeur et al . (2008) consider the effect of sea ice extent and temperature on the biomass of jellyfish in the Bering Sea. Autocorrelation in jellyfish biomass from 1 year to the next was expected because the biomass of jellyfish in 1 year should depend upon the biomass of animals reproducing in the previous year. Brodeur et al . (2008) used generalized additive modelling to build multiple models that regress jellyfish biomass against climate variables, whilst accounting for autocorrelation by using 1-year lagged jellyfish biomass as a factor in the model. They then compared the ability of the models to predict data using a generalized cross-validation approach ( Wood, 2006 ). As expected, jellyfish biomass in 1 year was strongly positively associated with biomass in the preceding year. Sea ice and temperature were also correlated with jellyfish biomass after accounting for the autocorrelation effect. Their analysis thus revealed potential interactions between climate and jellyfish growth, without concerns that significance would be spuriously inflated by temporal autocorrelation.

A major source of new methods for time series analysis has been economics. The concept of cointegration was developed by econometricians to allow inferences on causality of long-term relationships without the loss of power associated with differencing time series to obtain stationarity ( Engle & Granger, 1987 ). Two time series are said to be cointegrated in the first order if the residuals from a linear combination of the time series are stationary (the mean does not change through time). Tests for cointegration distinguish between time series with independent stochastic trends and those that share a long-term relationship ( Table 1 ). For instance, consider two time series for temperature and fish recruitment. If both time series have an increasing trend, we might difference the time series and correlate the resulting series to test for a relationship. However, if temperature really does drive long-term trends in recruitment, then differencing the time series will reduce the power to detect a real causal effect. Alternatively, we could test for cointegration of the time series. Cointegration of the time series would imply a causal driver of the shared long-term trend between the time series, whereas if the time series are not cointegrated, then we have not properly accounted for a causal relationship.

Cointegration has also been extended to multivariate and higher order analysis of time series with multiple orders of integration ( Kirchgässner & Wolters, 2007 ). Cointegration proved extremely useful in the analysis of economic time series, with Engle and Granger awarded the 2003 Nobel Memorial Prize in Economic Science for their contribution to time series analysis. It is thus surprising that this approach has been almost entirely ignored in ecological time series analysis. Interested readers should refer to Kirchgässner & Wolters (2007) for an introduction to cointegration methods accessible to ecologists.

Accommodating spatial patterns and autocorrelation in biological data

One approach to account for spatial patterns in data is to perform a meta-analysis of study regions ( Worm & Myers, 2003 ). Richardson & Schoeman (2004) analysed the correlation between phytoplankton abundance and sea surface temperature in a 45-year time series for multiple areas of the North-East Atlantic. They found no relationship in most areas when significance tests were adjusted for temporal autocorrelation. However, the study covered a gradient of mean annual temperature ranging from about 6 to 20 °C. Thus, Richardson & Schoeman (2004) used meta-analysis to inspect the correlation between mean annual temperature in each region and the temporal abundance–temperature correlation. The meta-analysis showed a significant negative correlation, implying that temperature rise positively impacted phytoplankton abundance in cold regions, negatively impacted abundance in warm regions and had little effect in intermediate regions. Such a result was consistent with the proposed mechanism for climate impacts, with phytoplankton growth being limited by low temperature in cold regions and thermal stratification in warm regions. Thus, the analysis of spatial patterns in this study revealed ecologically important signals, which would have remained hidden if the data were aggregated.

An alternative approach to modelling spatial patterns is to include spatial covariates in multiple regression or generalized additive models ( Table 1 ). De'ath et al . (2009) analysed data on coral growth and calcification using coral cores from 69 reefs across the Great Barrier Reef, Australia. By measuring growth rings in coral cores, calcification rates as far back as 1572 could be estimated. De'ath et al . (2009) used a generalized additive model to determine whether there were long-term trends in calcification and if calcification changes could be related to temperature changes. Their study area covered a significant spatial temperature gradient, so they divided temperature into spatial and temporal covariates. Thus, they were able to distinguish between the spatial effect of higher calcification rates in warmer regions and the temporal effect of more variable calcification rates during warmer years.

Spatial patterns in data cannot always be removed by including additional covariates in analysis. Where this spatial autocorrelation occurs, it should be considered in statistical tests ( Table 1 ). Richardson & Schoeman (2004) took a simple approach and reduced spatial autocorrelation in their meta-analysis by using only spatially discontinuous sites. The downsides of this approach are that data are excluded from analysis and that it cannot account for spatial autocorrelation occurring across larger areas. As with temporal autocorrelation, spatial autocorrelation can also be estimated and accounted for in tests. Mueter & Litzow (2008) compared changes in the distribution of abundance of fish species between two time periods. They fitted models of spatial autocorrelation to their data ( Pinheiro & Bates, 2000 ) and found a weak spatial autocorrelation that might inflate standard errors in statistical tests by 10%. Thus, to reduce the risk of detecting spuriously significant distribution change, they added an additional 10% to the standard errors before testing.

Ideally, spatial autocorrelation would be included explicitly as a process in a spatio-temporal model. Examples include accounting for spatial structure in error terms or response variables by adjusting the variance–covariance matrices in regressions or conducting geographically weighted regressions ( Kissling & Carl, 2008 ). We found no examples in the literature we reviewed probably because such models are technically challenging to develop. Data requirements can also be intensive, with a need for data across numerous locations. For the technically inclined, Diggle & Ribeiro (2007) provide a starting point for geostatistical analysis.

Often, biological data are collected at discrete locations, where samples from the same location are expected to be more similar than samples from different locations, although the likely causes of sample dependencies are unknown. De'ath et al .'s (2009) data were replicated at discrete locations, with multiple calcification measurements from each core and multiple cores at each reef. If replicates from the same location are treated as independent samples, they might spuriously inflate the degrees of freedom in statistical tests. Alternatively, pooling samples would considerably reduce the sample size and the power to detect causal relationships ( Venables & Ripley, 2002 ; Table 1 ). De'ath et al . (2009) accounted for the nested structure in the data by including random effects for cores and reefs in their generalized additive model. Calcification measurements from the same core were treated as random deviates from an overall core mean value and similarly for reefs. Accounting for the nested structure allowed reliable inferences on the temporal and spatial effects of temperature while preserving the power of the analysis. Random effects analyses are also useful when data are too limited and spatially unresolved to properly estimate spatial autocorrelation in a geo-statistical analysis.

Modelling changes in variability, cycles and periods

Most cases discussed so far have focussed on the effect of climate change on trends in ecological response variables. Climate impacts may also be detected through the examination of changes in the variability of ecological responses, including changes in the magnitude, frequency and period of ecological responses. Beaugrand et al . (2008) examined variability in metrics for cod recruitment and plankton community structure, size and diversity in the North Atlantic. Spatial analysis of these metrics revealed increased variability coinciding near the mean annual 10 °C isotherm, potentially indicating an ecological threshold separating different community types. Examination of the temporal variance in the community metrics demonstrated increased community variance in an area as the water warmed and the 10 °C isotherm moved polewards through an area. This increase in variance may indicate a shift in community composition to one that represents a more southerly biogeographical province.

Large-scale climate cycles may also drive periodic biological patterns. Sophisticated approaches have been developed by physical scientists that allow time series to be decomposed into their component cycles. These methods may be particularly useful for the analysis of highly temporally resolved long-term marine ecological data, and allow the separation of long-term trends from decadal cycles in the ocean. One flexible approach is wavelet analysis ( Torrence & Compo, 1998 ), which decomposes a time series into time and frequency domains, thus allowing examination of the dynamics of dominant cycles in the data ( Table 1 ). Jenouvrier et al . (2005) applied wavelet analysis to time series of seabird abundance, breeding success and environmental variables thought to affect seabird foraging success. They showed that in the early 1980s, there was a shift in the periodicity of both the seabird time series and the environmental time series, coincident with large-scale ocean warming. Thus, Jenouvrier et al . (2005) were able to detect changes in population variability potentially driven by climate warming that might not have been detected by examining trends in abundance or breeding success.

Metrics of phenology and distribution

The interpretation of climate impacts may often be assisted by deriving metrics of biological responses from raw observations that are readily associated with climate change. Overall, climate change is expected to lead to a polewards migration of species’ biogeographical ranges and an advance in the timing of phenological events (e.g. reproduction, migration). Derived metrics have proved useful for meta-analyses in climate change ecology. In particular, reports of rates of change in distribution (e.g. km decade −1 or km °C −1 ) or phenology (days decade −1 , days °C −1 ) are easily incorporated into global meta-analyses and syntheses, including those by the IPCC (2007) . Despite the benefits, reporting of these metrics is still not widespread in marine climate change ecology (18 out of 55 phenology and distribution studies with regularly sampled data reported metrics of change).

There is a range of analogous response metrics for phenology or distribution, which have similar statistical strengths and weaknesses. In studies of phenology, metrics include timing of an event on the basis of a single individual (e.g. arrival of the first individual), the mean or median timing of the event, the timing of the last event (e.g. departure of the last individual), or the duration of the event. Similarly, analyses of distribution shifts may use the range edges, range centre or range size as an indicator of range shift. The statistic used to represent the range or date changes should be carefully considered.

There are a suite of indicators that are reliant upon single individuals or single sites, such as the first individual to breed, or the northernmost sighted individual. These are statistically weak indicators of phenological change and distribution shifts because they are dependent upon only a single individual or site and ignore the majority of the population. More reliable metrics of changes in phenology and distribution are based on data on populations, such as recoding of the distribution of individual breeding dates in a population, abundance across the range or presence at different sites. In these cases, quantiles can be used to indicate the beginning of an event or the edge of a range. For instance, Juanes et al . (2004) analysed the dates of arrival for salmon to breeding streams using the cumulative dates of arrival of 25%, 50% and 75% of all fish, and Greve et al . (2005) analysed the start and end of the season using 15% and 85% of the annual cumulative abundance thresholds for plankton.

Commonly, the spread of abundance across a species’ distribution has been assumed to be normal, on the basis of early macro-ecological theory ( Brown, 1984 ). In this circumstance, mean spatial location (e.g. mean latitude of occurrence) would be an appropriate metric for the distribution centre. In reality, distributions of abundance may often be non-normal, in which case, the most appropriate metric for representing a distribution centre will depend upon the spatial arrangement of site presences and abundance ( Sagarin et al ., 2006 ). For instance, Hsieh et al . (2009) analysed changes in the mean and median distributions of larval fish, and found that changes in the median were more reliable than those of the mean, due to the influence of extreme values on the mean.

Bimodal data can cause problems for standard statistical tests and may occur commonly in phenological data. For instance, plankton blooms may occur in both spring and autumn in temperate regions ( Edwards & Richardson, 2004 ), and many intertidal species have multiple spawning events ( Moore et al ., 2011 ). To deal with bimodality, Edwards & Richardson (2004) split the seasonal peaks into spring and autumn categories and analysed both as separate responses. Moore et al . (2011) used the 25th percentile to indicate the timing of spawning, thus avoiding biases in the mean spawning time caused by the bimodality of the data, but placing emphasis on first spawning peak.

While statistics based on the range centre statistics are popular for summarizing distribution data, it is important to consider which aspect of a range is most biologically relevant and provides the greatest ability to distinguish the effects of climate change. For instance, understanding the dynamics of the equatorward edge of a species’ range may be important for conservation of genetic diversity with global warming ( Hampe & Petit, 2005 ). If the data were summarized using a centroid metric, this distinction may not be made. Multiple leading range edges, such as those for intertidal species on complex coastlines, may also provide greater opportunities for inferring the effects of climate change, because multiple observations of range shifts can be made for the same species within a reasonably small area.

A further consideration is that the study region usually does not cover the entire range of a species, particularly for cosmopolitan marine species (e.g. Perry et al ., 2005 ; Hsieh et al ., 2009 ; Nye et al ., 2009 ). In this instance, the measured distribution centre does not provide a reliable estimate of the actual distribution centre. Most studies have addressed this issue by classifying species as being in the northern, southern or central parts of their ranges. Thus, the change in the mean observed distribution can be interpreted in terms of the biogeographical affinity of the species.

A final consideration for distribution shifts is whether to analyse purely the latitudinal component of a range shift, or the total distance of the range shift, which may be greater if the shift has a longitudinal component. In the oceans, temperature gradients are not strictly north–south, so species should not be expected to simply shift to higher latitudes in response to warming. For instance, the northern North Sea cools southwards, and species in this region may be moving towards the equator with ocean warming ( Perry et al ., 2005 ; Philippart et al ., 2011 ). Thus, it may often be more meaningful to analyse the total distribution shift and report its direction in relation to prevailing temperature gradients and direction of warming in the region. Furthermore, some range shifts may be more evident as changes in the organism's depth distribution ( Dulvy et al ., 2008 ). While few datasets resolve depth (only four studies in the literature review analysed changes in depth), the potential for depth changes to hide horizontal distributional shifts should be considered, at least when formulating expectations.

Community-wide studies

A major strength of Fodrie et al . (2010) , as well as other examples above ( Jenouvrier et al ., 2005 ; Field et al ., 2006 ; Hsieh et al ., 2008 ; Genner et al ., 2010 ), comes from the analysis of data from multiple species. In fact, 197 (69%) of the studies in our review reported data from more than one species. On ecological grounds, different species are expected to respond to climate change in different ways. Such differences could be expected between cold-water and warm-waters species or exploited and unexploited species. Analysis of community data thus gives researchers greater opportunities to test for changes that are consistent or inconsistent with climate change, relative to other sources of variability that may confound analyses based on single species.

Analyses of climate change impacts on communities can proceed with a combination of single-species analyses or with aggregated descriptors of community structure, such as diversity or multivariate statistics. In such studies, species-level impacts should also be reported, because they facilitate inclusion of results into syntheses. Without the reporting of species-level change, impacts of climate on some taxa may be underestimated by syntheses, or non-significant changes missed. For instance, the study of changes in distribution of 36 zooplankton species by Beaugrand et al . (2002) was included as only six assemblages in Parmesan & Yohe's (2003) meta-analysis because species-level changes were not reported in the original paper. This tendency towards reporting only assemblage-level changes may lead to a bias in reporting fewer but more consistent impacts for plankton communities compared with higher trophic levels, which are often analysed on a single-species basis. An additional consideration with community data is that phylogenetic similarity between species may result in similar responses to climate change. Controlling for phylogeny in studies of climate impacts is emerging as a powerful approach for understanding how species’ traits determine climate change responses ( Davis et al ., 2010 ).

Limitations on publication space in peer-reviewed journals may preclude inclusion of species-level impacts in the main body of a paper. Furthermore, competition to publish in the journals with the greatest impact also biases the published literature towards reporting positive results ( Møller & Jennions, 2001 ); in the case of climate change ecology, this may mean over-representation of biological changes that are consistent with anthropogenic climate change. Both these biases are a serious problem for synthesis and for progressing the assessment of climate impacts on marine ecosystems. To overcome these issues, we recommend that the data or meta-data on species-level changes be provided in a repository, either as online supplementary material in the journal or an institutional repository (e.g. Table 2 ). This will assist interpretation of climate impacts and encourage re-analysis from different viewpoints.

Information on some online data repositories

Conclusions

We suggest that the issues discussed in this review should be considered when planning and conducting analyses in climate change ecology, and also when interpreting the reliability of published results from other studies. A summary of our suggestions is included below and are ordered roughly according to the sequence that they might be most useful. These suggestions are equally applicable to marine and terrestrial studies.

• Consider how spatial and temporal resolution of data will influence the strength of inferences about drivers of change. For example, long time series with frequent observations, over large regions and over multiple climate cycles provide an ideal basis for interpreting recent anthropogenic climate change. Longer term palaeo-ecological data can also provide valuable baselines for assessing climate impacts.
• Formulate alternative hypotheses for causal relationships between the ecological and climate variables. In some cases, observational studies can be coupled with experimental studies that shed light on the mechanisms driving change. In formulating alternative hypotheses, consider important drivers of ecological change, such as climate variability, ecosystem dynamics, other anthropogenic drivers of change (e.g. eutrophication, overfishing) and interactive effects. Where possible, data should be obtained on these drivers.
• Identify response variables. Many different response variables may be derived from some datasets. The most statistically reliable response variables will generally have the largest sample size (e.g. using quantiles of distribution limits rather than the northerly most sighting of a single individual) and will be formulated to address the proposed hypotheses (e.g. north–south distributional changes may be irrelevant in regions with east–west currents). Non-conventional response variables may also reveal new patterns, such as considering changes in ecological variability rather than changes in the mean.
• Formulate the identified processes as a statistical model or a series of models. Ideally, the models will include all drivers of change identified in step 2. Where possible, model-based approaches should be used rather than data transformations. Where temporal data cannot be obtained on key drivers, indirect approaches can be useful, such as comparisons among species. Furthermore, application of analytical methods beyond those traditionally used by ecologists (i.e. correlation and linear regression) will shed new light on the understanding of climate impacts. Promising methods rarely used in ecology include tests of cointegration, wavelets for the analysis of ecological cycles and spatio-temporal models.
• Temporal autocorrelation should be considered in analysis if using time series data. Temporal autocorrelation patterns can often be reduced using filters, detrending or differencing. A more powerful approach for two variables can be to adjust the degrees of freedom in significance tests or to use a test of cointegration. If multiple predictors may influence the response, autoregressive models may be used and also allow estimation of rates of change.
• Spatial autocorrelation and patterns should be considered if using spatial data. Spatial patterns can be ignored in analysis by grouping or averaging the data to a single value in space; however, this approach reduces the information content of the data. In some cases, meta-analysis, generalized additive models, mixed-effects models and geostatistics can be used to assist understanding the processes driving spatial patterns. Where spatial non-independence of data points cannot be accounted for by using covariates, it can be modelled explicitly. For spatially continuous data, models of spatial autocorrelation or spatial covariates can be used to account for non-independence of data points. Mixed-effects models can be used for data collected at discrete sites.
• Metrics summarizing the rate-of-change for all species studied should be reported. Species-level metrics assist the uptake of the results of a study by other researchers and help in building global understanding of marine climate impacts. Registering data with an online database is encouraged ( Table 2 ).

Consideration of these suggestions should help climate change ecologists apply appropriate statistical approaches to their data and afford them some confidence in the robustness of their results. We hope that this work will also encourage the re-analysis of archived datasets using appropriate approaches. A solid statistical basis for climate change ecology will help advance policy debates on climate change, improve predictions of impacts and aid the development of strategies for adaptive management.

Acknowledgments

We thank the National Center for Ecological Analysis and Synthesis for supporting the working group ‘Towards Understanding the Marine Biological Impacts of Climate Change’. B. Clarke, N. Dulvy, S. Hawkins, C. Parmesan, F. Schwing and an anonymous reviewer provided useful comments and D. Hogan and S.-A. Thompson assisted with database compilation.

• Allen JR, Slinn DJ, Shammon TM, Hartnoll RG, Hawkins SJ. Evidence for eutrophication of the Irish Sea over four decades. Limnology and Oceanography. 1998; 43 :1970–1974. [ Google Scholar ]
• Beaugrand G, Kirby RR. Climate, plankton and cod. Global Change Biology. 2010; 16 :1268–1280. [ Google Scholar ]
• Beaugrand G, Reid PC, Ibañez F, Lindley AJ, Edwards M. Reorganization of North Atlantic marine copepod biodiversity and climate. Science. 2002; 296 :1692–1694. [ PubMed ] [ Google Scholar ]
• Beaugrand G, Edwards M, Brander K, Luczak C, Ibañez F. Causes and projections of abrupt climate-driven ecosystem shifts in the North Atlantic. Ecology Letters. 2008; 11 :1157–1168. [ PubMed ] [ Google Scholar ]
• Brodeur RD, Decker MB, Ciannelli L, et al. Rise and fall of jellyfish in the eastern Bering Sea in relation to climate regime shifts. Progress in Oceanography. 2008; 77 :103–111. [ Google Scholar ]
• Brown JH. On the relationship between abundance and distribution of species. The American Naturalist. 1984; 124 :255–279. [ Google Scholar ]
• Chevaldonné P, Lejeusne C. Regional warming-induced species shift in north-west Mediterranean marine caves. Ecology Letters. 2003; 6 :371–379. [ Google Scholar ]
• Colebrook JM. Environmental influences on long-term variability in marine plankton. Hydrobiologia. 1986; 142 :309–325. [ Google Scholar ]
• Crawley MJ. Statistics: An Introduction Using R. Chichester: Wiley; 2005. [ Google Scholar ]
• Dale MRT, Fortin MJ. Spatial autocorrelation and statistical tests: some solutions. Journal of Agricultural, Biological, and Environmental Statistics. 2009; 14 :188–206. [ Google Scholar ]
• Davis CC, Willis CG, Primack RB, Miller-Rushing AJ. The importance of phylogeny to the study of phenological response to global climate change. Philosophical Transactions of the Royal Society: Series B. 2010; 365 :3201–3213. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• De'ath G, Lough JM, Fabricius KE. Declining coral calcification on the Great Barrier Reef. Science. 2009; 323 :116–119. [ PubMed ] [ Google Scholar ]
• Diggle PJ, Ribeiro PJ. Model-Based Geostatistics. New York: Springer; 2007. [ Google Scholar ]
• Duarte CM, Cebrian J, Marba N. Uncertainty of detecting sea change. Nature. 1992; 356 :190. [ Google Scholar ]
• Dulvy NK, Rogers SI, Jennings S, Stelzenmüller V, Dye SR, Skjoldal HR. Climate change and deepening of the North Sea fish assemblage: a biotic indicator of warming seas. Journal of Applied Ecology. 2008; 45 :1029–1039. [ Google Scholar ]
• Edwards M, Richardson AJ. Impact of climate change on marine pelagic phenology and trophic mismatch. Nature. 2004; 430 :881–884. [ PubMed ] [ Google Scholar ]
• Edwards M, Beaugrand G, Hays GC, Koslow JA, Richardson AJ. Multi-decadal oceanic ecological datasets and their application in marine policy and management. Trends in Ecology and Evolution. 2010; 25 :602–610. [ PubMed ] [ Google Scholar ]
• Engle RF, Granger CWJ. Co-integration and error correction: representation, estimation and testing. Econometrica. 1987; 55 :251–276. [ Google Scholar ]
• Field DB, Baumgartner TR, Charles CD, Ferreira-Bartrina V, Ohman MD. Planktonic foraminifera of the Californian current reflect 20 th -century warming. Science. 2006; 311 :63–66. [ PubMed ] [ Google Scholar ]
• Fodrie FJ, Heck KL, Powers SP, Graham WM, Robinson KL. Climate-related, decadal-scale assemblage changes of seagrass-associated fishes in the northern Gulf of Mexico. Global Change Biology. 2010; 16 :48–59. [ Google Scholar ]
• Forcada J, Trathan PN, Reid K, Murphy EJ, Croxall JP. Contrasting population changes in sympatric penguin species in association with climate warming. Global Change Biology. 2006; 12 :411–423. [ Google Scholar ]
• Genner MJ, Sims DW, Southward AJ, et al. Body size-dependent responses of a marine fish assemblage to climate change and fishing over a century-long scale. Global Change Biology. 2010; 16 :517–527. [ Google Scholar ]
• Grace JB. Structural Equation Modeling and Natural Systems. Cambridge: Cambridge University Press; 2006. [ Google Scholar ]
• Greve W, Prinage S, Zidowitz H, Nast J, Reiners F. On the phenology of North Sea ichthyoplankton. ICES Journal of Marine Science. 2005; 62 :1216–1223. [ Google Scholar ]
• Halloran PR, Hall IR, Colmenero-Hidalgo E, Rickaby REM. Evidence for a multi-species coccolith volume change over the past two centuries: understanding a potential ocean acidification response. Biogeosciences. 2008; 5 :1651–1655. [ Google Scholar ]
• Halpern BS, Walbridge S, Selkoe KA, et al. A global map of human impact on marine ecosystems. Science. 2008; 319 :948–952. [ PubMed ] [ Google Scholar ]
• Hampe A, Petit RJ. Conserving biodiversity under climate change: the rear edge matters. Ecology Letters. 2005; 8 :461–467. [ PubMed ] [ Google Scholar ]
• Hare SR, Mantua NJ. Empirical evidence for North Pacific regime shifts in 1977 and 1989. Progress in Oceanography. 2000; 47 :103–145. [ Google Scholar ]
• Harley CDG, Hughes AR, Hultgren KM, et al. The impacts of climate change in coastal marine systems. Ecology Letters. 2006; 9 :228–241. [ PubMed ] [ Google Scholar ]
• Hawkins SJ, Southward AJ, Genner MJ. Detection of environmental change in a marine ecosystem – evidence from the western English Channel. The Science of the Total Environment. 2003; 310 :245–256. [ PubMed ] [ Google Scholar ]
• Hermant M, Lobry J, Bonhommeau S, Poulard SC, Le Pape O. Impact of warming on abundance and occurrence of flatfish populations in the Bay of Biscay (France) Journal of Sea Research. 2010; 64 :45–53. [ Google Scholar ]
• Hewitt JE, Thrush SF, Dayton PK, Bonsdorff E. The effect of spatial and temporal heterogeneity on the design and analysis of empirical studies of scale-dependent systems. The American Naturalist. 2007; 169 :398–408. [ PubMed ] [ Google Scholar ]
• Hobbs NT, Hilborn R. Alternatives to statistical hypothesis testing in ecology: a guide to self teaching. Ecological Applications. 2006; 16 :5–19. [ PubMed ] [ Google Scholar ]
• Hoegh-Guldberg O, Bruno JF. The impact of climate change on the world's marine ecosystems. Science. 2010; 328 :1523–1528. [ PubMed ] [ Google Scholar ]
• Hsieh CH, Reiss CS, Hewitt RP, Sugihara G. Spatial analysis shows that fishing enhances the climatic sensitivity of marine fishes. Canadian Journal of Fisheries and Aquatic Sciences. 2008; 65 :947–961. [ Google Scholar ]
• Hsieh CH, Kim HJ, Watson W, Di Lorenzo E, Sugihara G. Climate-driven changes in abundance and distribution of larvae of oceanic fishes in the southern Californian region. Global Change Biology. 2009; 15 :2137–2152. [ Google Scholar ]
• ICES. Climatic Changes in the Arctic in relation to Plants and Animals. Contributions to Special Scientific Meetings. 1948; 125 :5–52. Rapports et Procés-Verbaux des Réunions du Conseil International pour l'Exploration de la Mer. [ Google Scholar ]
• Iglesias-Rodriguez MD, Halloran PR, Rickaby REM. Phytoplankton calcification in a high-CO2 world. Science. 2008; 320 :336–340. [ PubMed ] [ Google Scholar ]
• International pour l'Exploration de la Mer. Climatic changes in the Arctic in relation to plants and animals. [In] Contributions to Special Scientific Meetings 1948. Rapports et Procès-Verbaux des Réunions du Conseil. 1948; 125 :5–52. [ Google Scholar ]
• IPCC (Intergovernmental Panel on Climate Change) Climate Change 2007: Impacts, Adaptation and Vulnerability. Contribution of Working Group II to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change. Geneva, Switzerland: IPCC; 2007. Available at: http://www.ipcc.ch (accessed 12 February 2011) [ Google Scholar ]
• Jenouvrier S, Weimerskirch H, Barbraud C, Park YH, Cazelles B. Evidence of a shift in the cyclicity of Antarctic seabird dynamics linked to climate. Proceedings of the Royal Society B: Biological Sciences. 2005; 272 :887–895. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Juanes F, Gephard S, Beland KF. Long-term changes in migration timing of adult Atlantic salmon ( Salmo salar ) at the southern edge of the species distribution. Canadian Journal of Fisheries and Aquatic Sciences. 2004; 61 :2392–2400. [ Google Scholar ]
• Kirby RR, Beaugrand G. Trophic amplification of climate warming. Proceedings of the Royal Society B: Biological Sciences. 2009; 276 :4095–4103. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Kirchgässner G, Wolters J. Introduction to modern time series analysis. Berlin: Springer; 2007. [ Google Scholar ]
• Kissling WD, Carl G. Spatial autocorrelation and the selection of simultaneous autoregressive models. Global Ecology and Biogeography. 2008; 17 :59–71. [ Google Scholar ]
• Legendre P, Legendre L. Numerical Ecology. New York: Elsevier; 1998. [ Google Scholar ]
• Legendre P, Dale MRT, Fortin MJ, Gurevitch J, Hohn M, Myers D. The consequences of spatial structure for the design and analysis of ecological field surveys. Ecography. 2002; 25 :601–615. [ Google Scholar ]
• Lima FP, Ribeiro PA, Queiroz N, Hawkins SJ, Santos AM. Do distributional shifts of northern and southern species of algae match the warming pattern? Global Change Biology. 2007; 13 :2592–2604. [ Google Scholar ]
• Litzow MA, Ciannelli L. Oscillating trophic control induces community reorganization in a marine ecosystem. Ecology Letters. 2007; 10 :1124–1134. [ PubMed ] [ Google Scholar ]
• Loebl M, van Beuksekom JEE, Reise K. Is the spread of the neophyte Spartina anglica enhanced by increasing temperatures? Aquatic Ecology. 2006; 40 :315–324. [ Google Scholar ]
• Mieszkowska N, Hawkins SJ, Burrows MT, Kendall MA. Long-term changes in the geographic distribution and population structures of Osilinus lineatus (Gastropoda: Trochidae) in Britian and Ireland. Journal of the Marine Biological Association of the United Kingdom. 2007; 87 :537–545. [ Google Scholar ]
• Møller AP, Jennions MD. Testing and adjusting for publication bias. Trends in Ecology and Evolution. 2001; 16 :580–586. [ Google Scholar ]
• Möllmann C, Müller-Karulis B, Kornilovs G, St John MA. Effect of climate and overfishing on zooplankton dynamics and ecosystem structure: regime shifts, trophic cascades, and feedback loops in a simple ecosystem. ICES Journal of Marine Science. 2008; 65 :302–310. [ Google Scholar ]
• Moore AJ, Thompson RC, Hawkins SJ. Phenological changes in intertidal con-specific gastropods in response to climate warming. Global Change Biology. 2011; 17 :709–719. [ Google Scholar ]
• Mueter FJ, Litzow MA. Sea ice retreat alters the biogeography of the Bering Sea continental shelf. Ecological Applications. 2008; 18 :309–320. [ PubMed ] [ Google Scholar ]
• Nye JA, Link JS, Hare JA, Overholtz WJ. Changing spatial distribution of fish stocks in relation to climate and population size on the Northeast United States continental shelf. Marine Ecology Progress Series. 2009; 393 :111–129. [ Google Scholar ]
• Ohman MD, Venrick EL. CalCOFI in a changing ocean. Oceanography. 2003; 16 :76–85. [ Google Scholar ]
• Parmesan C, Yohe G. A globally coherent fingerprint of climate change impacts across natural systems. Nature. 2003; 421 :37–42. [ PubMed ] [ Google Scholar ]
• Parmesan C, Duarte C, Poloczanska E, Richardson AJ, Singer MC. Overstretching attribution. Nature Climate Change. 2011; 1 :2–4. [ Google Scholar ]
• Perry AL, Low PJ, Ellis JR, Reynolds JD. Climate change and distribution shifts in marine fishes. Science. 2005; 308 :1912–1915. [ PubMed ] [ Google Scholar ]
• Philippart CJM, Anadón R, Danovaro R, et al. Impacts of climate change on European marine ecosystems: observations, expectations and indicators. Journal of Experimental Marine Biology and Ecology. 2011; 400 :52–69. [ Google Scholar ]
• Pinheiro JC, Bates DM. Mixed-Effects Models in S and S-Plus. New York: Springer; 2000. [ Google Scholar ]
• Poloczanska ES, Hawkins SJ, Southward AJ, Burrows MT. Modeling the response of populations of competing species to climate change. Ecology. 2008; 89 :3138–3149. [ PubMed ] [ Google Scholar ]
• Pyper BJ, Peterman RM. Comparison of methods to account for autocorrelation in correlation analyses of fish data. Canadian Journal of Fisheries and Aquatic Sciences. 1998; 55 :2127–2140. [ Google Scholar ]
• R Development Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing. Vienna, Austria: R Development Core Team; 2010. Available at: http://www.R-project.org (accessed 12 February 2011) [ Google Scholar ]
• Reid PC, Colebrook JM, Matthews JBL, Aiken J Continuous Plankton Recorder Team. The Continuous Plankton Recorder: concepts and history, from Plankton Indicator to undulating recorders. Progress in Oceanography. 2003; 58 :117–173. [ Google Scholar ]
• Richardson AJ, Poloczanska ES. Ocean Science: under-resourced, under threat. Science. 2008; 320 :1294–1295. [ PubMed ] [ Google Scholar ]
• Richardson AJ, Schoeman DS. Climate impact on plankton ecosystems in the Northeast Atlantic. Science. 2004; 305 :1609–1612. [ PubMed ] [ Google Scholar ]
• Rosenzweig C, Karoly D, Vicarelli M. Attributing physical and biological impacts to anthropogenic climate change. Nature. 2008; 453 :353–358. [ PubMed ] [ Google Scholar ]
• Sagarin RD, Barry JP, Gilman SE, Baxter CH. Climate-related changes in an intertidal community over short and long time-scales. Ecological Monographs. 1999; 69 :465–490. [ Google Scholar ]
• Sagarin RD, Gaines SD, Gaylord B. Moving beyond assumptions to understand abundance distributions across the range of species. Trends in Ecology and Evolution. 2006; 21 :524–530. [ PubMed ] [ Google Scholar ]
• Southward AJ, Hawkins SJ, Burrows MT. Seventy years’ observations of changes in distribution and abundance of zooplankton and intertidal organisms in the Western English Channel in relation to rising sea temperature. Journal of Thermal Biology. 1995; 20 :127–155. [ Google Scholar ]
• Southward AJ, Langmead O, Hardman-Mountford NJ, et al. Long-term oceanographic and ecological research in the western English Channel. Advances in Marine Biology. 2005; 47 :1–105. [ PubMed ] [ Google Scholar ]
• Tasker ML. The Effect of Climate Change on the Distribution and Abundance of Marine Species in the OSPAR Maritime Area. Copenhagen, Denmark: ICES Cooperative Research; 2008. Report No. 293. [ Google Scholar ]
• Thresher RE, Koslow JA, Morison AK, Smith DC. Depth-mediated reversal of the effects of climate change on long-term growth rates of exploited marine fish. Proceedings of the National Academy of Sciences of Sciences of the United States of America. 2007; 104 :7461–7465. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Tingley MW, Beissinger SR. Detecting range shifts from historical species occurrences: new perspectives on old data. Trends in Ecology and Evolution. 2009; 24 :625–633. [ PubMed ] [ Google Scholar ]
• Torrence C, Compo GP. A practical guide to wavelets analysis. Bulletin of the American Meteorological Society. 1998; 79 :61–78. [ Google Scholar ]
• Venables WN, Ripley BD. Modern Applied Statistics with S. New York: Springer; 2002. [ Google Scholar ]
• Votier SC, Hatchwell BJ, Mears M, Birkhead TR. Changes in the timing of egg-laying of a colonial seabird in relation to population size and environmental conditions. Marine Ecology Progress Series. 2009; 393 :225–233. [ Google Scholar ]
• Wassmann P, Duarte CM, Agusti S, Sejr MK. Footprints of climate change in the Arctic marine ecosystem. Global Change Biology. 2011; 17 :1235–1249. [ Google Scholar ]
• Wiafe G, Yaqub HB, Mensah MA, Frid CLJ. Impact of climate change on long-term zooplankton biomass in the upwelling of the Gulf of Guinea. ICES Journal of Marine Science. 2008; 65 :318–324. [ Google Scholar ]
• Wood SN. Generalized Additive Models. An Introduction with R. Boca Raton, FL: Chapman & Hall; 2006. [ Google Scholar ]
• Worm B, Myers RA. Meta-analysis of cod–shrimp interactions reveals top–down control in oceanic food webs. Ecology. 2003; 84 :162–173. [ Google Scholar ]
• Zuur AF, Ieno EN, Walker NJ, Saveliev AA, Smith GM. Mixed Effects Models and Extensions in Ecology with R. New York: Springer; 2009. [ Google Scholar ]

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Review Article
• Published: 03 November 2022

Improving quantitative synthesis to achieve generality in ecology

• Rebecca Spake   ORCID: orcid.org/0000-0003-4671-2225 1 ,
• Rose E. O’Dea   ORCID: orcid.org/0000-0001-8177-5075 2 ,
• Shinichi Nakagawa   ORCID: orcid.org/0000-0002-7765-5182 3 ,
• C. Patrick Doncaster   ORCID: orcid.org/0000-0001-9406-0693 4 ,
• Masahiro Ryo   ORCID: orcid.org/0000-0002-5271-3446 5 , 6 ,
• Corey T. Callaghan   ORCID: orcid.org/0000-0003-0415-2709 7 &
• James M. Bullock   ORCID: orcid.org/0000-0003-0529-4020 8  

Nature Ecology & Evolution volume  6 ,  pages 1818–1828 ( 2022 ) Cite this article

4332 Accesses

17 Citations

63 Altmetric

Metrics details

• Biodiversity
• Conservation biology
• Ecological modelling

Synthesis of primary ecological data is often assumed to achieve a notion of ‘generality’, through the quantification of overall effect sizes and consistency among studies, and has become a dominant research approach in ecology. Unfortunately, ecologists rarely define either the generality of their findings, their estimand (the target of estimation) or the population of interest. Given that generality is fundamental to science, and the urgent need for scientific understanding to curb global scale ecological breakdown, loose usage of the term ‘generality’ is problematic. In other disciplines, generality is defined as comprising both generalizability—extending an inference about an estimand from the sample to the population—and transferability—the validity of estimand predictions in a different sampling unit or population. We review current practice in ecological synthesis and demonstrate that, when researchers fail to define the assumptions underpinning generalizations and transfers of effect sizes, generality often misses its target. We provide guidance for communicating nuanced inferences and maximizing the impact of syntheses both within and beyond academia. We propose pathways to generality applicable to ecological syntheses, including the development of quantitative and qualitative criteria with which to license the transfer of estimands from both primary and synthetic studies.

This is a preview of subscription content, access via your institution

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 12 digital issues and online access to articles

111,21 € per year

only 9,27 € per issue

Buy this article

• Purchase on SpringerLink
• Instant access to full article PDF

Prices may be subject to local taxes which are calculated during checkout

Similar content being viewed by others

Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences

Empirical evidence of widespread exaggeration bias and selective reporting in ecology

A checklist for maximizing reproducibility of ecological niche models

Houlahan, J. E., McKinney, S. T., Anderson, T. M. & McGill, B. J. The priority of prediction in ecological understanding. Oikos 126 , 1–7 (2017).

Article   Google Scholar  

Lawton, J. H. Are there general laws in ecology? Oikos 84 , 177–192 (1999).

Elliott-Graves, A. Generality and causal interdependence in ecology. Philos. Sci. 85 , 1102–1114 (2018).

Fox, J. W. The many roads to generality in ecology. Philos. Top. 9 , 83–104 (2019).

McGill, B. J. et al. Species abundance distributions: moving beyond single prediction theories to integration within an ecological framework. Ecol. Lett. 10 , 995–1015 (2007).

Article   PubMed   Google Scholar  

MacArthur, R. H. & Wilson, E. O. An equilibrium theory of insular zoogeography. Evolution 17 , 373–387 (1963).

Gurevitch, J., Fox, G. A., Wardle, G. M., Inderjit & Taub, D. Emergent insights from the synthesis of conceptual frameworks for biological invasions. Ecol. Lett. 14 , 407–418 (2011).

Article   CAS   PubMed   Google Scholar  

Borer, E. T. et al. Finding generality in ecology: a model for globally distributed experiments. Methods Ecol. Evol. 5 , 65–73 (2014).

Gurevitch, J., Koricheva, J., Nakagawa, S. & Stewart, G. Meta-analysis and the science of research synthesis. Nature 555 , 175–182 (2018).

Anderson, S. C. et al. Trends in ecology and conservation over eight decades. Front. Ecol. Environ. 19 , 274–282 (2021).

Kneale, D., Thomas, J., O’Mara-Eves, A. & Wiggins, R. How can additional secondary data analysis of observational data enhance the generalisability of meta-analytic evidence for local public health decision making? Res. Synth. Methods 10 , 44–56 (2019).

Aguinis, H., Pierce, C. A., Bosco, F. A., Dalton, D. R. & Dalton, C. M. Debunking myths and urban legends about meta-analysis. Organ. Res. Methods 14 , 306–331 (2011).

Polit, D. F. & Beck, C. T. Generalization in quantitative and qualitative research: myths and strategies. Int. J. Nurs. Stud. 47 , 1451–1458 (2010).

Cardinale, B. J., Gonzalez, A., Allington, G. R. H. & Loreau, M. Is local biodiversity declining or not? A summary of the debate over analysis of species richness time trends. Biol. Conserv. 219 , 175–183 (2018).

Lundberg, I., Johnson, R. & Stewart, B. M. What is your estimand? Defining the target quantity connects statistical evidence to theory. Am. Sociol. Rev. 86 , 532–565 (2021).

Lawrance, R. et al. What is an estimand & how does it relate to quantifying the effect of treatment on patient-reported quality of life outcomes in clinical trials? J. Patient-Rep. Outcomes 4 , 68 (2020).

Article   PubMed   PubMed Central   Google Scholar  

Findley, M. G., Kikuta, K. & Denly, M. External validity. Annu. Rev. Polit. Sci. 24 , 365–393 (2021).

Pearl, J. & Bareinboim, E. External validity: from do-calculus to transportability across populations. Stat. Sci. 29 , 579–595 (2014).

Westreich, D., Edwards, J. K., Lesko, C. R., Cole, S. R. & Stuart, E. A. Target validity and the hierarchy of study designs. Am. J. Epidemiol. 188 , 438–443 (2019).

Carpenter, C. J. Meta-analyzing apples and oranges: how to make applesauce instead of fruit salad. Hum. Commun. Res. 46 , 322–333 (2020).

Rohrer, J. M. & Arslan, R. C. Precise answers to vague questions: issues with interactions. Adv. Methods Pract. Psychol. Sci. 4 , 1–19 (2021).

Google Scholar  

Breslow, N. E. & Clayton, D. G. Approximate inference in generalized linear mixed models. J. Am. Stat. Assoc. 88 , 9–25 (1993).

Koricheva, J. & Gurevitch, J. Uses and misuses of meta-analysis in plant ecology. J. Ecol. 102 , 828–844 (2014).

Gonzalez, A. et al. Estimating local biodiversity change: a critique of papers claiming no net loss of local diversity. Ecology 97 , 1949–1960 (2016).

Konno, K. et al. Ignoring non-English-language studies may bias ecological meta-analyses. Ecol. Evol. 10 , 6373–6384 (2020).

Nakagawa, S. et al. Methods for testing publication bias in ecological and evolutionary meta-analyses. Methods Ecol. Evol. 13 , 4–21 (2022).

Rosenthal, R. The file drawer problem and tolerance for null results. Psychol. Bull. 86 , 638–641 (1979).

Leung, B. et al. Clustered versus catastrophic global vertebrate declines. Nature 588 , 267–271 (2020).

Rothman, K. J., Gallacher, J. E. J. & Hatch, E. E. Why representativeness should be avoided. Int. J. Epidemiol. 42 , 1012–1014 (2013).

Spake, R. et al. Implications of scale dependence for cross-study syntheses of biodiversity differences. Ecol. Lett. 24 , 374–390 (2021).

Spake, R. & Doncaster, C. P. Use of meta-analysis in forest biodiversity research: key challenges and considerations. For. Ecol. Manag. 400 , 429–437 (2017).

Christie, A. P. et al. Simple study designs in ecology produce inaccurate estimates of biodiversity responses. J. Appl. Ecol. 56 , 2742–2754 (2019).

Nakagawa, S., Noble, D. W. A., Senior, A. M. & Lagisz, M. Meta-evaluation of meta-analysis: ten appraisal questions for biologists. BMC Biol. 15 , 18 (2017).

Higgins, J. P. T. & Thompson, S. G. Quantifying heterogeneity in a meta-analysis. Stat. Med. 21 , 1539–1558 (2002).

Schielzeth, H. & Nakagawa, S. Conditional repeatability and the variance explained by reaction norm variation in random slope models. Methods Ecol. Evol. 13 , 1214–1223 (2022).

Nakagawa, S. et al. The orchard plot: cultivating a forest plot for use in ecology, evolution, and beyond. Res. Synth. Methods 12 , 4–12 (2021).

Lorah, J. Effect size measures for multilevel models: definition, interpretation, and TIMSS example. Large-Scale Assess. Educ. 6 , 8 (2018).

O’Connor, M. I. et al. A general biodiversity–function relationship is mediated by trophic level. Oikos 126 , 18–31 (2017).

Ojha, M., Naidu, D. G. T. & Bagchi, S. Meta-analysis of induced anti-herbivore defence traits in plants from 647 manipulative experiments with natural and simulated herbivory. J. Ecol . 110 , 799–816 (2022).

Dodds, K. C. et al. Material type influences the abundance but not richness of colonising organisms on marine structures. J. Environ. Manag. 307 , 114549 (2022).

Dornelas, M. et al. Assemblage time series reveal biodiversity change but not systematic loss. Science 344 , 296–299 (2014).

Senior, A. M. et al. Heterogeneity in ecological and evolutionary meta- analyses: its magnitude and implications. Ecology 97 , 3293–3299 (2016).

Blowes, S. A. et al. The geography of biodiversity change in marine and terrestrial assemblages. Science 366 , 339–345 (2019).

Nakagawa, S. & Cuthill, I. C. Effect size, confidence interval and statistical significance: a practical guide for biologists. Biol. Rev. 82 , 591–605 (2007).

Glass, G. V. Primary, secondary, and meta-analysis of research. Educ. Res. 5 , 3–8 (1976).

Glass, G. V. Meta‐analysis at 25: a personal history. Education in Two Worlds https://ed2worlds.blogspot.com/2022/07/meta-analysis-at-25-personal-history.html (2000).

Cooper, H. M. Organizing knowledge syntheses: a taxonomy of literature reviews. Knowl. Soc. 1 , 104–126 (1988).

Soranno, P. A. et al. Cross-scale interactions: quantifying multi-scaled cause-effect relationships in macrosystems. Front. Ecol. Environ. 12 , 65–73 (2014).

Gerstner, K. et al. Will your paper be used in a meta-analysis? Make the reach of your research broader and longer lasting. Methods Ecol. Evol. 8 , 777–784 (2017).

Hortal, J. et al. Seven shortfalls that beset large-scale knowledge of biodiversity. Annu. Rev. Ecol. Evol. Syst. 46 , 523–549 (2015).

Simons, D. J., Shoda, Y. & Lindsay, D. S. Constraints on Generality (CoG): a proposed addition to all empirical papers. Perspect. Psychol. Sci. 12 , 1123–1128 (2017).

Yarkoni, T. The generalizability crisis. Behav. Brain Sci . https://doi.org/10.1017/S0140525X20001685 (2020).

Lopez, P. M., Subramanian, S. V. & Schooling, C. M. Effect measure modification conceptualized using selection diagrams as mediation by mechanisms of varying population-level relevance. J. Clin. Epidemiol. 113 , 123–128 (2019).

Campbell, D. T. in Advances in QuasiExperimental Design and Analysis (ed. Trochim, W.) 67–77 (Jossey-Bass, 1986).

Spake, R. et al. Meta‐analysis of management effects on biodiversity in plantation and secondary forests of Japan. Conserv. Sci. Pract. 1 , e14 (2019).

Forest Ecosystem Diversity Basic Survey (in Japanese) (Forestry Agency of Japan, 2019); https://www.rinya.maff.go.jp/j/keikaku/tayouseichousa/index.html

Ito, S., Ishigamia, S., Mizoue, N. & Buckley, G. P. Maintaining plant species composition and diversity of understory vegetation under strip-clearcutting forestry in conifer plantations in Kyushu, southern Japan. For. Ecol. Manag. 231 , 234–241 (2006).

Utsugi, E. et al. Hardwood recruitment into conifer plantations in Japan: effects of thinning and distance from neighboring hardwood forests. For. Ecol. Manag. 237 , 15–28 (2006).

Kominami, Y. et al. Classification of bird-dispersed plants by fruiting phenology, fruit size, and growth form in a primary lucidophyllous forest: an analysis, with implications for the conservation of fruit–bird interactions. Ornthological Sci. 2 , 3–23 (2003).

Tsujino, R. & Matsui, K. Forest regeneration inhibition in a mixed broadleaf-conifer forest under sika deer pressure. J. For. Res. 27 , 230–235 (2021).

Spake, R., Soga, M., Catford, J. A. & Eigenbrod, F. Applying the stress-gradient hypothesis to curb the spread of invasive bamboo. J. Appl. Ecol. 58 , 1993–2003 (2021).

Mize, T. D. Best practices for estimating, interpreting, and presenting nonlinear interaction effects. Sociol. Sci. 6 , 81–117 (2019).

Karaca-Mandic, P., Norton, E. C. & Dowd, B. Interaction terms in nonlinear models. Health Serv. Res. 47 , 255–274 (2012).

Spake, R. et al. Forest damage by deer depends on cross-scale interactions between climate, deer density and landscape structure. J. Appl. Ecol. 57 , 1376–1390 (2020).

McCabe, C. J., Kim, D. S. & King, K. M. Improving present practices in the visual display of interactions. Adv. Methods Pract. Psychol. Sci. 1 , 147–165 (2018).

Shackelford, G. E. et al. Dynamic meta-analysis: a method of using global evidence for local decision making. BMC Biol. 19 , 33 (2021).

Christie, A. P. et al. Innovation and forward‐thinking are needed to improve traditional synthesis methods: a response to Pescott and Stewart. J. Appl. Ecol. 59 , 1191–1197 (2022).

Haddaway, N. R. et al. EviAtlas: a tool for visualising evidence synthesis databases. Environ. Evid . 8 , 22 (2019).

Delory, B. M., Li, M., Topp, C. N. & Lobet, G. archiDART v3.0: a new data analysis pipeline allowing the topological analysis of plant root systems. F1000Research 7 , 22 (2018).

Perkel, J. M. The future of scientific figures. Nature 554 , 133–134 (2018).

Weaver, S. & Gleeson, M. P. The importance of the domain of applicability in QSAR modeling. J. Mol. Graph. Model. 26 , 1315–1326 (2008).

Sutton, C. et al. Identifying domains of applicability of machine learning models for materials science. Nat. Commun. 11 , 4428 (2020).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Meyer, H. & Pebesma, E. Predicting into unknown space? Estimating the area of applicability of spatial prediction models. Methods Ecol. Evol. 12 , 1620–1633 (2021).

Pearl, J. & Bareinboim, E. Transportability of causal and statistical relations: a formal approach. In 2011 IEEE 11th International Conference on Data Mining Workshops https://doi.org/10.1109/ICDMW.2011.169 (IEEE, 2011).

Munthe-Kaas, H., Nøkleby, H. & Nguyen, L. Systematic mapping of checklists for assessing transferability. Syst. Rev. 8 , 22 (2019).

Dekkers, O. M., von Elm, E., Algra, A., Romijn, J. A. & Vandenbroucke, J. P. How to assess the external validity of therapeutic trials: a conceptual approach. Int. J. Epidemiol. 39 , 89–94 (2010).

Schloemer, T. & Schröder-Bäck, P. Criteria for evaluating transferability of health interventions: a systematic review and thematic synthesis. Implement. Sci. 13 , 88 (2018).

Fernandez-Hermida, J. R., Calafat, A., Becoña, E., Tsertsvadze, A. & Foxcroft, D. R. Assessment of generalizability, applicability and predictability (GAP) for evaluating external validity in studies of universal family-based prevention of alcohol misuse in young people: systematic methodological review of randomized controlled trials. Addiction 107 , 1570–1579 (2012).

Avellar, S. A. et al. External validity: the next step for systematic reviews? Eval. Rev. 41 , 283–325 (2017).

Bareinboim, E. & Pearl, J. A general algorithm for deciding transportability of experimental results. J. Causal Inference 1 , 107–134 (2013).

Degtiar, I. & Rose, S. A review of generalizability and transportability. Preprint at https://doi.org/10.48550/arXiv.2102.11904 (2021).

Bareinboim, E. & Pearl, J. Meta-transportability of causal effects: a formal approach. J. Mach. Learn. Res. 31 , 135–143 (2013).

Jamieson, D. Scientific uncertainty: how do we know when to communicate research findings to the public? Sci. Total Environ. 184 , 103–107 (1996).

Article   CAS   Google Scholar  

Burchett, H. E. D., Mayhew, S. H., Lavis, J. N. & Dobrow, M. J. When can research from one setting be useful in another? Understanding perceptions of the applicability and transferability of research. Health Promot. Int. 28 , 418–430 (2013).

Forscher, P. et al. Build up big-team science. Nature 601 , 505–507 (2022).

Whalen, M. A. et al. Climate drives the geography of marine consumption by changing predator communities. Proc. Natl Acad. Sci. USA 117 , 28160–28166 (2020).

Moshontz, H. et al. The Psychological Science Accelerator: advancing psychology through a distributed collaborative network. Adv. Methods Pract. Psychol. Sci. 1 , 501–515 (2018).

Marschner, I. C. A general framework for the analysis of adaptive experiments. Stat. Sci. 36 , 465–492 (2021).

Clark, M. Shrinkage in Mixed Effects Models https://m-clark.github.io/posts/2019-05-14-shrinkage-in-mixed-models/ (2019).

Gurevitch, J. & Hedges, L. V. Statistical issues in ecological meta-analyses. Ecology 80 , 1142–1149 (1999).

Mengersen, K., Gurevitch, J. & Schmid, C. H. in Handbook of Meta-analysis in Ecology and Evolution (eds Koricheva, U. et al.) 300–312 (Princeton Univ. Press, 2013).

Hudson, L. N. et al. The database of the PREDICTS (Projecting Responses of Ecological Diversity In Changing Terrestrial Systems) project. Ecol. Evol. 7 , 145–188 (2017).

Dornelas, M. et al. BioTIME: a database of biodiversity time series for the Anthropocene. Glob. Ecol. Biogeogr. 27 , 760–786 (2018).

Salguero-Gómez, R. et al. The COMPADRE Plant Matrix Database: an open online repository for plant demography. J. Ecol. 103 , 202–218 (2015).

Salguero-Gómez, R. et al. COMADRE: a global data base of animal demography. J. Anim. Ecol. 85 , 371–384 (2016).

Pastor, D. A. & Lazowski, R. A. On the multilevel nature of meta-analysis: a tutorial, comparison of software programs, and discussion of analytic choices. Multivar. Behav. Res. 53 , 74–89 (2018).

Download references

Acknowledgements

We thank J. Chase for informative discussion of concepts. R.S. is grateful for funding from the German Centre for Integrative Biodiversity Research – iDiv - Halle-Jena-Leipzig. J.M.B. was funded under UKCEH National Capability project 06895. C.T.C. was supported by a Marie Skłodowska-Curie Individual Fellowship (no. 891052).

Author information

Authors and affiliations.

School of Biological Sciences, University of Reading, Reading, UK

Rebecca Spake

School of Ecosystem and Forest Sciences, University of Melbourne, Melbourne, Victoria, Australia

Rose E. O’Dea

Evolution & Ecology Research Centre and School of Biological, Earth and Environmental Sciences, University of New South Wales, Sydney, New South Wales, Australia

• Shinichi Nakagawa

School of Biological Sciences, University of Southampton, Southampton, UK

C. Patrick Doncaster

Leibniz Centre for Agricultural Landscape Research (ZALF), Müncheberg, Germany

Masahiro Ryo

Brandenburg University of Technology Cottbus–Senftenberg, Cottbus, Germany

German Centre for Integrative Biodiversity research – iDiv - Halle-Jena-Leipzig, Leipzig, Germany

Corey T. Callaghan

UK Centre for Ecology & Hydrology, Wallingford, UK

James M. Bullock

You can also search for this author in PubMed   Google Scholar

Contributions

R.S. conceived the idea and developed a first draft with J.M.B., R.E.O., S.N., C.P.D., M.R. and C.T.C. contributed to idea development and paper writing.

Corresponding author

Correspondence to Rebecca Spake .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Peer review

Peer review information.

Nature Ecology & Evolution thanks the anonymous reviewers for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary information.

Appendix S1 is a table of the aims and scope of the 50 most cited journals in ecology. Appendix S2 provides details of an analysis (a meta-regression of sapling abundance on thinning intensity in coniferous forests of Japan).

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Cite this article.

Spake, R., O’Dea, R.E., Nakagawa, S. et al. Improving quantitative synthesis to achieve generality in ecology. Nat Ecol Evol 6 , 1818–1828 (2022). https://doi.org/10.1038/s41559-022-01891-z

Download citation

Received : 15 December 2021

Accepted : 26 August 2022

Published : 03 November 2022

Issue Date : December 2022

DOI : https://doi.org/10.1038/s41559-022-01891-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Quantitative evidence synthesis: a practical guide on meta-analysis, meta-regression, and publication bias tests for environmental sciences.

• Yefeng Yang
• Malgorzata Lagisz

Environmental Evidence (2023)

Analytic transparency is key for reproducibility of agricultural research

• Gudeta W. Sileshi

CABI Agriculture and Bioscience (2023)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

• Methodology
• Open access
• Published: 24 April 2023

Quantitative evidence synthesis: a practical guide on meta-analysis, meta-regression, and publication bias tests for environmental sciences

• Shinichi Nakagawa   ORCID: orcid.org/0000-0002-7765-5182 1 , 2 ,
• Yefeng Yang   ORCID: orcid.org/0000-0002-8610-4016 1 ,
• Erin L. Macartney   ORCID: orcid.org/0000-0003-3866-143X 1 ,
• Rebecca Spake   ORCID: orcid.org/0000-0003-4671-2225 3 &
• Malgorzata Lagisz   ORCID: orcid.org/0000-0002-3993-6127 1  

Environmental Evidence volume  12 , Article number:  8 ( 2023 ) Cite this article

9042 Accesses

25 Citations

30 Altmetric

Metrics details

Meta-analysis is a quantitative way of synthesizing results from multiple studies to obtain reliable evidence of an intervention or phenomenon. Indeed, an increasing number of meta-analyses are conducted in environmental sciences, and resulting meta-analytic evidence is often used in environmental policies and decision-making. We conducted a survey of recent meta-analyses in environmental sciences and found poor standards of current meta-analytic practice and reporting. For example, only ~ 40% of the 73 reviewed meta-analyses reported heterogeneity (variation among effect sizes beyond sampling error), and publication bias was assessed in fewer than half. Furthermore, although almost all the meta-analyses had multiple effect sizes originating from the same studies, non-independence among effect sizes was considered in only half of the meta-analyses. To improve the implementation of meta-analysis in environmental sciences, we here outline practical guidance for conducting a meta-analysis in environmental sciences. We describe the key concepts of effect size and meta-analysis and detail procedures for fitting multilevel meta-analysis and meta-regression models and performing associated publication bias tests. We demonstrate a clear need for environmental scientists to embrace multilevel meta-analytic models, which explicitly model dependence among effect sizes, rather than the commonly used random-effects models. Further, we discuss how reporting and visual presentations of meta-analytic results can be much improved by following reporting guidelines such as PRISMA-EcoEvo (Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Ecology and Evolutionary Biology). This paper, along with the accompanying online tutorial, serves as a practical guide on conducting a complete set of meta-analytic procedures (i.e., meta-analysis, heterogeneity quantification, meta-regression, publication bias tests and sensitivity analysis) and also as a gateway to more advanced, yet appropriate, methods.

Evidence synthesis is an essential part of science. The method of systematic review provides the most trusted and unbiased way to achieve the synthesis of evidence [ 1 , 2 , 3 ]. Systematic reviews often include a quantitative summary of studies on the topic of interest, referred to as a meta-analysis (for discussion on the definitions of ‘meta-analysis’, see [ 4 ]). The term meta-analysis can also mean a set of statistical techniques for quantitative data synthesis. The methodologies of the meta-analysis were initially developed and applied in medical and social sciences. However, meta-analytic methods are now used in many other fields, including environmental sciences [ 5 , 6 , 7 ]. In environmental sciences, the outcomes of meta-analyses (within systematic reviews) have been used to inform environmental and related policies (see [ 8 ]). Therefore, the reliability of meta-analytic results in environmental sciences is important beyond mere academic interests; indeed, incorrect results could lead to ineffective or sometimes harmful environmental policies [ 8 ].

As in medical and social sciences, environmental scientists frequently use traditional meta-analytic models, namely fixed-effect and random-effects models [ 9 , 10 ]. However, we contend that such models in their original formulation are no longer useful and are often incorrectly used, leading to unreliable estimates and errors. This is mainly because the traditional models assume independence among effect sizes, but almost all primary research papers include more than one effect size, and this non-independence is often not considered (e.g., [ 11 , 12 , 13 ]). Furthermore, previous reviews of published meta-analyses in environmental sciences (hereafter, ‘environmental meta-analyses’) have demonstrated that less than half report or investigate heterogeneity (inconsistency) among effect sizes [ 14 , 15 , 16 ]. Many environmental meta-analyses also do not present any sensitivity analysis, for example, for publication bias (i.e., statistically significant effects being more likely to be published, making collated data unreliable; [ 17 , 18 ]). These issues might have arisen for several reasons, for example, because of no clear conduct guideline for the statistical part of meta-analyses in environmental sciences and rapid developments in meta-analytic methods. Taken together, the field urgently requires a practical guide to implement correct meta-analyses and associated procedures (e.g., heterogeneity analysis, meta-regression, and publication bias tests; cf. [ 19 ]).

To assist environmental scientists in conducting meta-analyses, the aims of this paper are five-fold. First, we provide an overview of the processes involved in a meta-analysis while introducing some key concepts. Second, after introducing the main types of effect size measures, we mathematically describe the two commonly used traditional meta-analytic models, demonstrate their utility, and introduce a practical, multilevel meta-analytic model for environmental sciences that appropriately handles non-independence among effect sizes. Third, we show how to quantify heterogeneity (i.e., consistencies among effect sizes and/or studies) using this model, and then explain such heterogeneity using meta-regression. Fourth, we show how to test for publication bias in a meta-analysis and describe other common types of sensitivity analysis. Fifth, we cover other technical issues relevant to environmental sciences (e.g., scale and phylogenetic dependence) as well as some advanced meta-analytic techniques. In addition, these five aims (sections) are interspersed with two more sections, named ‘Notes’ on: (1) visualisation and interpretation; and (2) reporting and archiving. Some of these sections are accompanied by results from a survey of 73 environmental meta-analyses published between 2019 and 2021; survey results depict current practices and highlight associated problems (for the method of the survey, see Additional file 1 ). Importantly, we provide easy-to-follow implementations of much of what is described below, using the R package, metafor [ 20 ] and other R packages at the webpage ( https://itchyshin.github.io/Meta-analysis_tutorial/ ), which also connects the reader to the wealth of online information on meta-analysis (note that we also provide this tutorial as Additional file 2 ; see also [ 21 ]).

Overview with key concepts

Statistically speaking, we have three general objectives when conducting a meta-analysis [ 12 ]: (1) estimating an overall mean , (2) quantifying consistency ( heterogeneity ) between studies, and (3) explaining the heterogeneity (see Table 1 for the definitions of the terms in italic ). A notable feature of a meta-analysis is that an overall mean is estimated by taking the sampling variance of each effect size into account: a study (effect size) with a low sampling variance (usually based on a larger sample size) is assigned more weight in estimating an overall mean than one with a high sampling variance (usually based on a smaller sample size). However, an overall mean estimate itself is often not informative because one can get the same overall mean estimates in different ways. For example, we may get an overall estimate of zero if all studies have zero effects with no heterogeneity. In contrast, we might also obtain a zero mean across studies that have highly variable effects (e.g., ranging from strongly positive to strongly negative), signifying high heterogeneity. Therefore, quantifying indicators of heterogeneity is an essential part of a meta-analysis, necessary for interpreting the overall mean appropriately. Once we observe non-zero heterogeneity among effect sizes, then, our job is to explain this variation by running meta-regression models, and, at the same time, quantify how much variation is accounted for (often quantified as R 2 ). In addition, it is important to conduct an extra set of analyses, often referred to as publication bias tests , which are a type of sensitivity analysis [ 11 ], to check the robustness of meta-analytic results.

Choosing an effect size measure

In this section, we introduce different kinds of ‘effect size measures’ or ‘effect measures’. In the literature, the term ‘effect size’ is typically used to refer to the magnitude or strength of an effect of interest or its biological interpretation (e.g., environmental significance). Effect sizes can be quantified using a range of measures (for details, see [ 22 ]). In our survey of environmental meta-analyses (Additional file 1 ), the two most commonly used effect size measures are: the logarithm of response ratio, lnRR ([ 23 ]; also known as the ratio of means; [ 24 ]) and standardized mean difference, SMD (often referred to as Hedges’ g or Cohen’s d [ 25 , 26 ]). These are followed by proportion (%) and Fisher’s z -transformation of correlation, or Zr . These four effect measures nearly fit into the three categories, which are named: (1) single-group measures (a statistical summary from one group; e.g., proportion), (2) comparative measures (comparing between two groups e.g., SMD and lnRR), and (3) association measures (relationships between two variables; e.g., Zr ). Table 2 summarizes effect measures that are common or potentially useful for environmental scientists. It is important to note that any measures with sampling variance can become an ‘effect size’. The main reason why SMD, lnRR, Zr, or proportion are popular effect measures is that they are unitless, while a meta-analysis of mean, or mean difference, can only be conducted when all effect sizes have the same unit (e.g., cm, kg).

Table 2 also includes effect measures that are likely to be unfamiliar to environmental scientists; these are effect sizes that characterise differences in the observed variability between samples, (i.e., lnSD, lnCV, lnVR and lnCVR; [ 27 , 28 ]) rather than central tendencies (averages). These dispersion-based effect measures can provide us with extra insights along with average-based effect measures. Although the literature survey showed none of these were used in our sample, these effect sizes have been used in many fields, including agriculture (e.g., [ 29 ]), ecology (e.g., [ 30 ]), evolutionary biology (e.g., [ 31 ]), psychology (e.g., [ 32 ]), education (e.g., [ 33 ]), psychiatry (e.g., [ 34 ]), and neurosciences (e.g. [ 35 ],),. Perhaps, it is not difficult to think of an environmental intervention that can affect not only the mean but also the variance of measurements taken on a group of individuals or a set of plots. For example, environmental stressors such as pesticides and eutrophication are likely to increase variability in biological systems because stress accentuates individual differences in environmental responses (e.g. [ 36 , 37 ],). Such ideas are yet to be tested meta-analytically (cf. [ 38 , 39 ]).

Choosing a meta-analytic model

Fixed-effect and random-effects models.

Two traditional meta-analytic models are called the ‘fixed-effect’ model and the ‘random-effects’ model. The former assumes that all effect sizes (from different studies) come from one population (i.e., they have one true overall mean), while the latter does not have such an assumption (i.e., each study has different overall means or heterogeneity exists among studies; see below for more). The fixed-effect model, which should probably be more correctly referred to as the ‘common-effect’ model, can be written as [ 9 , 10 , 40 ]:

where the intercept, \({\beta }_{0}\) is the overall mean, z j (the response/dependent variable) is the effect size from the j th study ( j  = 1, 2,…, N study ; in this model, N study  = the number of studies = the number of effect sizes), m j is the sampling error, related to the j th sampling variance ( v j ), which is normally distributed with the mean of 0 and the ‘study-specific’ sampling variance, v j (see also Fig.  1 A).

Visualisation of the three statistical models of meta-analysis: A a fixed-effect model (1-level), B a random-effects model (2-level), and C a multilevel model (3-level; see the text for what symbols mean)

The overall mean needs to be estimated and often done so as the weighted average with the weights, \({w}_{j}=1/{v}_{j}\) (i.e., the inverse-variance approach). An important, but sometimes untenable, assumption of meta-analysis is that sampling variance is known. Indeed, we estimate sampling variance, using formulas, as in Table 2 , meaning that vj is submitted by sampling variance estimates (see also section ‘ Scale dependence ’). Of relevance, the use of the inverse-variance approach has been recently criticized, especially for SMD and lnRR [ 41 , 42 ] and we note that the inverse-variance approach using the formulas in Table 2 is one of several different weighting approaches used in meta-analysis (e.g., for adjusted sampling-variance weighing, see [ 43 , 44 ]; for sample-size-based weighting, see [ 41 , 42 , 45 , 46 ]). Importantly, the fixed-effect model assumes that the only source of variation in effect sizes ( z j ) is the effect due to sampling variance (which is inversely proportional to the sample size, n ; Table 2 ).

Similarly, the random-effects model can be expressed as:

where u j is the j th study effect, which is normally distributed with the mean of 0 and the between-study variance, \({\tau }^{2}\) (for different estimation methods, see [ 47 , 48 , 49 , 50 ]), and other notations are the same as in Eq.  1 (Fig.  1 B). Here, the overall mean can be estimated as the weighted average with weights \({w}_{j}=1/\left({\tau }^{2}+{v}_{j}^{2}\right)\) (note that different weighting approaches, mentioned above, are applicable to the random-effects model and some of them are to the multilevel model, introduced below). The model assumes each study has its specific mean, \({b}_{0}+{u}_{j}\) , and (in)consistencies among studies (effect sizes) are indicated by \({\tau }^{2}\) . When \({\tau }^{2}\) is 0 (or not statistically different from 0), the random-effects model simplifies to the fixed-effect model (cf. Equations  1 and 2 ). Given no studies in environmental sciences are conducted in the same manner or even at exactly the same place and time, we should expect different studies to have different means. Therefore, in almost all cases in the environmental sciences, the random-effects model is a more ‘realistic’ model [ 9 , 10 , 40 ]. Accordingly, most environmental meta-analyses (68.5%; 50 out of 73 studies) in our survey used the random-effects model, while only 2.7% (2 of 73 studies) used the fixed-effect model (Additional file 1 ).

Multilevel meta-analytic models

Although we have introduced the random-effects model as being more realistic than the fixed-effect model (Eq.  2 ), we argue that the random-effects model is rather limited and impractical for the environmental sciences. This is because random-effects models, like fixed-effect models, assume all effect sizes ( z j ) to be independent. However, when multiple effect sizes are obtained from a study, these effect sizes are dependent (for more details, see the next section on non-independence). Indeed, our survey showed that in almost all datasets used in environmental meta-analyses, this type of non-independence among effect sizes occurred (97.3%; 71 out of 73 studies, with two studies being unclear, so effectively 100%; Additional file 1 ). Therefore, we propose the simplest and most practical meta-analytic model for environmental sciences as [ 13 , 40 ] (see also [ 51 , 52 ]):

where we explicitly recognize that N effect ( i  = 1, 2,…, N effect ) >  N study ( j  = 1, 2,…, N study ) and, therefore, we now have the study effect (between-study effect), u j[i] (for the j th study and i th effect size) and effect-size level (within-study) effect, e i (for the i th effect size), with the between-study variance, \({\tau }^{2}\) , and with-study variance, \({\sigma }^{2}\) , respectively, and other notations are the same as above. We note that this model (Eq.  3 ) is an extension of the random-effects model (Eq.  2 ), and we refer to it as the multilevel/hierarchical model (used in 7 out of 73 studies: 9.6% [Additional file 1 ]; note that Eq.  3 is also known as a three-level meta-analytic model; Fig.  1 C). Also, environmental scientists who are familiar with (generalised) linear mixed-models may recognize u j (the study effect) as the effect of a random factor which is associated with a variance component, i.e., \({\tau }^{2}\) [ 53 ]; also, e i and m i can be seen as parts of random factors, associated with \({\sigma }^{2}\) and v i (the former is comparable to the residuals, while the latter is sampling variance, specific to a given effect size).

It seems that many researchers are aware of the issue of non-independence so that they often use average effect sizes per study or choose one effect size (at least 28.8%, 21 out of 73 environmental meta-analyses; Additional file 1 ). However, as we discussed elsewhere [ 13 , 40 ], such averaging or selection of one effect size per study dramatically reduces our ability to investigate environmental drivers of variation among effect sizes [ 13 ]. Therefore, we strongly support the use of the multilevel model. Nevertheless, this proposed multilevel model, formulated as Eq.  3 does not usually deal with the issue of non-independence completely, which we elaborate on in the next section.

Non-independence among effect sizes and among sampling errors

When you have multiple effect sizes from a study, there are two broad types and three cases of non-independence (cf. [ 11 , 12 ]): (1) effect sizes are calculated from different cohorts of individuals (or groups of plots) within a study (Fig.  2 A, referred to as ‘shared study identity’), and (2) effects sizes are calculated from the same cohort of individuals (or group of plots; Fig.  2 B, referred to as ‘shared measurements’) or partially from the same individuals and plots, more concretely, sharing individuals and plots from the control group (Fig.  2 C, referred to as ‘shared control group’). The first type of non-independence induces dependence among effect sizes, but not among sampling variances, and the second type leads to non-independence among sampling variances. Many datasets, if not almost all, will have a combination of these three cases (or even are more complex, see the section " Complex non-independence "). Failing to deal with these non-independences will inflate Type 1 error (note that the overall estimate, b 0 is unlikely to be biased, but standard error of b 0 , se( b 0 ), will be underestimated; note that this is also true for all other regression coefficients, e.g., b 1 ; see Table 1 ). The multilevel model (as in Eq.  3 ) only takes care of cases of non-independence that are due to the shared study identity but neither shared measurements nor shared control group.

Visualisation of the three types of non-independence among effect sizes: A due to shared study identities (effect sizes from the same study), B due to shared measurements (effect sizes come from the same group of individuals/plots but are based on different types of measurements), and C due to shared control (effect sizes are calculated using the same control group and multiple treatment groups; see the text for more details)

There are two practical ways to deal with non-independence among sampling variances. The first method is that we explicitly model such dependence using a variance–covariance (VCV) matrix (used in 6 out of 73 studies: 8.2%; Additional file 1 ). Imagine a simple scenario with a dataset of three effect sizes from two studies where two effects sizes from the first study are calculated (partially) using the same cohort of individuals (Fig.  2 B); in such a case, the sampling variance effect, \({m}_{i}\) , as in Eq.  3 , should be written as:

where M is the VCV matrix showing the sampling variances, \({v}_{1\left[1\right]}\) (study 1 and effect size 1), \({v}_{1\left[2\right]}\) (study 1 and effect size 2), and \({v}_{2\left[3\right]}\) (study 2 and effect size 3) in its diagonal, and sampling covariance, \(\rho \sqrt{{v}_{1\left[1\right]}{v}_{1\left[2\right]}}= \rho \sqrt{{v}_{1\left[2\right]}{v}_{1\left[1\right]}}\) in its off-diagonal elements, where \(\rho \) is a correlation between two sampling variances due to shared samples (individuals/plots). Once this VCV matrix is incorporated into the multilevel model (Eq.  3 ), all the types of non-independence, as in Fig.  2 , are taken care of. Table 3 shows formulas for the sampling variance and covariance of the four common effect sizes (SDM, lnRR, proportion and Zr ). For comparative effect measures (Table 2 ), exact covariances can be calculated under the case of ‘shared control group’ (see [ 54 , 55 ]). But this is not feasible for most circumstances because we usually do not know what \(\rho \) should be. Some have suggested fixing this value at 0.5 (e.g., [ 11 ]) or 0.8 (e.g., [ 56 ]); the latter is a more conservative assumption. Or one can run both and use one for the main analysis and the other for sensitivity analysis (for more, see the ‘ Conducting sensitivity analysis and critical appraisal " section).

The second method overcomes this very issue of unknown \(\rho \) by approximating average dependence among sampling variance (and effect sizes) from the data and incorporating such dependence to estimate standard errors (only used in 1 out of 73 studies; Additional file 1 ). This method is known as ‘robust variance estimation’, RVE, and the original estimator was proposed by Hedges and colleagues in 2010 [ 57 ]. Meta-analysis using RVE is relatively new, and this method has been applied to multilevel meta-analytic models only recently [ 58 ]. Note that the random-effects model (Eq.  2 ) and RVE could correctly model both types of non-independence. However, we do not recommend the use of RVE with Eq.  2 because, as we will later show, estimating \({\sigma }^{2}\) as well as \({\tau }^{2}\) will constitute an important part of understanding and gaining more insights from one’s data. We do not yet have a definite recommendation on which method to use to account for non-independence among sampling errors (using the VCV matrix or RVE). This is because no simulation work in the context of multilevel meta-analysis has been done so far, using multilevel meta-analyses [ 13 , 58 ]. For now, one could use both VCV matrices and RVE in the same model [ 58 ] (see also [ 21 ]).

Quantifying and explaining heterogeneity

Measuring consistencies with heterogeneity.

As mentioned earlier, quantifying heterogeneity among effect sizes is an essential component of any meta-analysis. Yet, our survey showed only 28 out of 73 environmental meta-analyses (38.4%; Additional file 1 ) report at least one index of heterogeneity (e.g., \({\tau }^{2}\) , Q , and I 2 ). Conventionally, the presence of heterogeneity is tested by Cochrane’s Q test. However, Q (often noted as Q T or Q total ), and its associated p value, are not particularly informative: the test does not tell us about the extent of heterogeneity (e.g. [ 10 ],), only whether heterogeneity is zero or not (when p  < 0.05). Therefore, for environmental scientists, we recommend two common ways of quantifying heterogeneity from a meta-analytic model: absolute heterogeneity measure (i.e., variance components, \({\tau }^{2}\) and \({\sigma }^{2}\) ) and relative heterogeneity measure (i.e., I 2 ; see also the " Notes on visualisation and interpretation " section for another way of quantifying and visualising heterogeneity at the same time, using prediction intervals; see also [ 59 ]). We have already covered the absolute measure (Eqs.  2 & 3 ), so here we explain I 2 , which ranges from 0 to 1 (for some caveats for I 2 , see [ 60 , 61 ]). The heterogeneity measure, I 2 , for the random-effect model (Eq.  2 ) can be written as:

Where \(\overline{v}\) is referred to as the typical sampling variance (originally this is called ‘within-study’ variance, as in Eq.  2 , and note that in this formulation, within-study effect and the effect of sampling error is confounded; see [ 62 , 63 ]; see also [ 64 ]) and the other notations are as above. As you can see from Eq.  5 , we can interpret I 2 as relative variation due to differences between studies (between-study variance) or relative variation not due to sampling variance.

By seeing I 2 as a type of interclass correlation (also known as repeatability [ 65 ],), we can generalize I 2 to multilevel models. In the case of Eq.  3 ([ 40 , 66 ]; see also [ 52 ]), we have:

Because we can have two more I 2 , Eq.  7 is written as \({I}_{total}^{2}\) ; these other two are \({I}_{study}^{2}\) and \({I}_{effect}^{2}\) , respectively:

\({I}_{total}^{2}\) represents relative variance due to differences both between and within studies (between- and within-study variance) or relative variation not due to sampling variance, while \({I}_{study}^{2}\) is relative variation due to differences between studies, and \({I}_{effect}^{2}\) is relative variation due to differences within studies (Fig.  3 A). Once heterogeneity is quantified (note almost all data will have non-zero heterogeneity and an earlier meta-meta-analysis suggests in ecology, we have on average, I 2 close to 90% [ 66 ]), it is time to fit a meta-regression model to explain the heterogeneity. Notably, the magnitude of \({I}_{study}^{2}\) (and \({\tau }^{2}\) ) and \({I}_{effect}^{2}\) (and \({\sigma }^{2}\) ) can already inform you which predictor variable (usually referred to as ‘moderator’) is likely to be important, which we explain in the next section.

Visualisation of variation (heterogeneity) partitioned into different variance components: A quantifying different types of I 2 from a multilevel model (3-level; see Fig.  1 C) and B variance explained, R 2 , by moderators. Note that different levels of variances would be explained, depending on which level a moderator belongs to (study level and effect-size level)

Explaining variance with meta-regression

We can extend the multilevel model (Eq.  3 ) to a meta-regression model with one moderator (also known as predictor, independent, explanatory variable, or fixed factor), as below:

where \({\beta }_{1}\) is a slope of the moderator ( x 1 ), \({x}_{1j\left[i\right]}\) denotes the value of x 1 , corresponding to the j th study (and the i th effect sizes). Equation ( 10 ) (meta-regression) is comparable to the simplest regression with the intercept ( \({\beta }_{0}\) ) and slope ( \({\beta }_{1}\) ). Notably, \({x}_{1j\left[i\right]}\) differs between studies and, therefore, it will mainly explain the variance component, \({\tau }^{2}\) (which relates to \({I}_{study}^{2}\) ). On the other hand, if noted like \({x}_{1i}\) , this moderator would vary within studies or at the level of effect sizes, therefore, explaining \({\sigma }^{2}\) (relating to \({I}_{effect}^{2}\) ). Therefore, when \({\tau }^{2}\) ( \({I}_{study}^{2}\) ), or \({\sigma }^{2}\) ( \({I}_{effect}^{2}\) ), is close to zero, there will be little point fitting a moderator(s) at the level of studies, or effect sizes, respectively.

As in multiple regression, we can have multiple (multi-moderator) meta-regression, which can be written as:

where \(\sum_{h=1}^{q}{\beta }_{h}{x}_{h\left[i\right]}\) denotes the sum of all the moderator effects, with q being the number of slopes (staring with h  = 1). We note that q is not necessarily the number of moderators. This is because when we have a categorical moderator, which is common, with more than two levels (e.g., method A, B & C), the fixed effect part of the formula is \({\beta }_{0}+{\beta }_{1}{x}_{1}+{\beta }_{2}{x}_{2}\) , where x 1 and x 2 are ‘dummy’ variables, which code whether the i th effect size belongs to, for example, method B or C, with \({\beta }_{1}\) and \({\beta }_{2}\) being contrasts between A and B and between A and C, respectively (for more explanations of dummy variables, see our tutorial page [ https://itchyshin.github.io/Meta-analysis_tutorial/ ]; also see [ 67 , 68 ]). Traditionally, researchers conduct separate meta-analyses per different groups (known as ‘sub-group analysis’), but we prefer a meta-regression approach with a categorical variable, which is statistically more powerful [ 40 ]. Also, importantly, what can be used as a moderator(s) is very flexible, including, for example, individual/plot characteristics (e.g., age, location), environmental factors (e.g., temperature), methodological differences between studies (e.g., randomization), and bibliometric information (e.g., publication year; see more in the section ‘Checking for publication bias and robustness’). Note that moderators should be decided and listed a priori in the meta-analysis plan (i.e., a review protocol or pre-registration).

As with meta-analysis, the Q -test ( Q m or Q moderator ) is often used to test the significance of the moderator(s). To complement this test, we can also quantify variance explained by the moderator(s) using R 2 . We can define R 2 using Eq. ( 11 ) as:

where R 2 is known as marginal R 2 (sensu [ 69 , 70 ]; cf. [ 71 ]), \({f}^{2}\) is the variance due to the moderator(s), and \({(f}^{2}+{\tau }^{2}+{\sigma }^{2})\) here equals to \(({\tau }^{2}+{\sigma }^{2})\) in Eq.  7 , as \({f}^{2}\) ‘absorbs’ variance from \({\tau }^{2}\) and/or \({\sigma }^{2}\) . We can compare the similarities and differences in Fig.  3 B where we denote a part of \({f}^{2}\) originating from \({\tau }^{2}\) as \({f}_{study}^{2}\) while \({\sigma }^{2}\) as \({f}_{effect}^{2}\) . In a multiple meta-regression model, we often want to find a model with the ‘best’ or an adequate set of predictors (i.e., moderators). R 2 can potentially help such a model selection process. Yet, methods based on information criteria (such as Akaike information criterion, AIC) may be preferable. Although model selection based on the information criteria is beyond the scope of the paper, we refer the reader to relevant articles (e.g., [ 72 , 73 ]), and we show an example of this procedure in our online tutorial ( https://itchyshin.github.io/Meta-analysis_tutorial/ ).

Notes on visualisation and interpretation

Visualization and interpretation of results is an essential part of a meta-analysis [ 74 , 75 ]. Traditionally, a forest plot is used to display the values and 95% of confidence intervals (CIs) for each effect size and the overall effect and its 95% CI (the diamond symbol is often used, as shown in Fig.  4 A). More recently, adding a 95% prediction interval (PI) to the overall estimate has been strongly recommended because 95% PIs show a predicted range of values in which an effect size from a new study would fall, assuming there is no sampling error [ 76 ]. Here, we think that examining the formulas for 95% CIs and PIs for the overall mean (from Eq.  3 ) is illuminating:

where \({t}_{df\left[\alpha =0.05\right]}\) denotes the t value with the degree of freedom, df , at 97.5 percentile (or \(\alpha =0.05\) ) and other notations are as above. In a meta-analysis, it has been conventional to use z value 1.96 instead of \({t}_{df\left[\alpha =0.05\right]}\) , but simulation studies have shown the use of t value over z value reduces Type 1 errors under many scenarios and, therefore, is recommended (e.g., [ 13 , 77 ]). Also, it is interesting to note that by plotting 95% PIs, we can visualize heterogeneity as Eq.  15 includes \({\tau }^{2}\) and \({\sigma }^{2}\) .

Different types of plots useful for a meta-analysis using data from Midolo et al. [ 133 ]: A a typical forest plot with the overall mean shown as a diamond at the bottom (20 effect sizes from 20 studies are used), B a caterpillar plot (100 effect sizes from 24 studies are used), C an orchard plot of categorical moderator with seven levels (all effect sizes are used), and D a bubble plot of a continuous moderator. Note that the first two only show confidence intervals, while the latter two also show prediction intervals (see the text for more details)

A ‘forest’ plot can become quickly illegible as the number of studies (effect sizes) becomes large, so other methods of visualizing the distribution of effect sizes have been suggested. Some suggested to present a ‘caterpillar’ plot, which is a version of the forest plot, instead (Fig.  4 B; e.g., [ 78 ]). We here recommend an ‘orchard’ plot, as it can present results across different groups (or a result of meta-regression with a categorical variable), as shown in Fig.  4 C [ 78 ]. For visualization of a continuous variable, we suggest what is called a ‘bubble’ plot, shown in Fig.  4 D. Visualization not only helps us interpret meta-analytic results, but can also help to identify something we may not see from statistical results, such as influential data points and outliers that could threaten the robustness of our results.

Checking for publication bias and robustness

Detecting and correcting for publication bias.

Checking for and adjusting for any publication bias is necessary to ensure the validity of meta-analytic inferences [ 79 ]. However, our survey showed almost half of the environmental meta-analyses (46.6%; 34 out of 73 studies; Additional file 1 ) neither tested for nor corrected for publication bias (cf. [ 14 , 15 , 16 ]). The most popular methods used were: (1) graphical tests using funnel plots (26 studies; 35.6%), (2) regression-based tests such as Egger regression (18 studies; 24.7%), (3) Fail-safe number tests (12 studies; 16.4%), and (4) trim-and-fill tests (10 studies; 13.7%). We recently showed that these methods are unsuitable for datasets with non-independent effect sizes, with the exception of funnel plots [ 80 ] (for an example of funnel plots, see Fig.  5 A). This is because these methods cannot deal with non-independence in the same way as the fixed-effect and random-effects models. Here, we only introduce a two-step method for multilevel models that can both detect and correct for publication bias [ 80 ] (originally proposed by [ 81 , 82 ]), more specifically, the “small study effect” where an effect size value from a small-sample-sized study can be much larger in magnitude than a ‘true’ effect [ 83 , 84 ]. This method is a simple extension of Egger’s regression [ 85 ], which can be easily implemented by using Eq.  10 :

where \({\widetilde{n}}_{i}\) is known as effective sample size; for Zr and proportion it is just n i , and for SMD and lnRR, it is \({n}_{iC}{n}_{iT}/\left({n}_{iC}+{n}_{iT}\right)\) , as in Table 2 . When \({\beta }_{1}\) is significant, we conclude there exists a small-study effect (in terms of a funnel plot, this is equivalent to significant funnel asymmetry). Then, we fit Eq.  17 and we look at the intercept \({\beta }_{0}\) , which will be a bias-corrected overall estimate [note that \({\beta }_{0}\) in Eq. ( 16 ) provides less accurate estimates when non-zero overall effects exist [ 81 , 82 ]; Fig.  5 B]. An intuitive explanation of why \({\beta }_{0}\) (Eq.  17 ) is the ‘bias-corrected’ estimate is that the intercept represents \(1/\widetilde{{n}_{i}}=0\) (or \(\widetilde{{n}_{i}}=\infty \) ); in other words, \({\beta }_{0}\) is the estimate of the overall effect when we have a very large (infinite) sample size. Of note, appropriate bias correction requires a selection-mode-based approach although such an approach is yet to be available for multilevel meta-analytic models [ 80 ].

Different types of plots for publication bias tests: A a funnel plot using model residuals, showing a funnel (white) that shows the region of statistical non-significance (30 effect sizes from 30 studies are used; note that we used the inverse of standard errors for the y -axis, but for some effect sizes, sample size or ‘effective’ sample size may be more appropriate), B a bubble plot visualising a multilevel meta-regression that tests for the small study effect (note that the slope was non-significant: b  = 0.120, 95% CI = [− 0.095, 0.334]; all effect sizes are used), and C a bubble plot visualising a multilevel meta-regression that tests for the decline effect (the slope was non-significant: b  = 0.003, 95%CI = [− 0.002, 0.008])

Conveniently, this proposed framework can be extended to test for another type of publication bias, known as time-lag bias, or the decline effect, where effect sizes tend to get closer to zero over time, as larger or statistically significant effects are published more quickly than smaller or non-statistically significant effects [ 86 , 87 ]. Again, a decline effect can be statistically tested by adding year to Eq. ( 3 ):

where \(c\left(yea{r}_{j\left[i\right]}\right)\) is the mean-centred publication year of a particular study (study j and effect size i ); this centring makes the intercept \({\beta }_{0}\) meaningful, representing the overall effect estimate at the mean value of publication years (see [ 68 ]). When the slope is significantly different from 0, we deem that we have a decline effect (or time-lag bias; Fig.  5 C).

However, there may be some confounding moderators, which need to be modelled together. Indeed, Egger’s regression (Eqs.  16 and 17 ) is known to detect the funnel asymmetry when there is little heterogeneity; this means that we need to model \(\sqrt{1/{\widetilde{n}}_{i}}\) with other moderators that account for heterogeneity. Given this, we probably should use a multiple meta-regression model, as below:

where \(\sum_{h=3}^{q}{\beta }_{h}{x}_{h\left[i\right]}\) is the sum of the other moderator effects apart from the small-study effect and decline effect, and other notations are as above (for more details see [ 80 ]). We need to carefully consider which moderators should go into Eq.  19 (e.g., fitting all moderators or using an AIC-based model selection method; see [ 72 , 73 ]). Of relevance, when running complex models, some model parameters cannot be estimated well, or they are not ‘identifiable’ [ 88 ]. This is especially so for variance components (random-effect part) rather than regression coeffects (fixed-effect part). Therefore, it is advisable to check whether model parameters are all identifiable, which can be checked using the profile function in metafor (for an example, see our tutorial webpage [ https://itchyshin.github.io/Meta-analysis_tutorial/ ]).

Conducting sensitivity analysis and critical appraisal

Sensitivity analysis explores the robustness of meta-analytic results by running a different set of analyses from the original analysis, and comparing the results (note that some consider publication bias tests a part of sensitivity analysis; [ 11 ]). For example, we might be interested in assessing how robust results are to the presence of influential studies, to the choice of method for addressing non-independence, or weighting effect sizes. Unfortunately, in our survey, only 37% of environmental meta-analyses (27 out of 73) conducted sensitivity analysis (Additional file 1 ). There are two general and interrelated ways to conduct sensitivity analyses [ 73 , 89 , 90 ]. The first one is to take out influential studies (e.g., outliers) and re-run meta-analytic and meta-regression models. We can also systematically take each effect size out and run a series of meta-analytic models to see whether any resulting overall effect estimates are different from others; this method is known as ‘leave-one-out’, which is considered less subjective and thus recommended.

The second way of approaching sensitivity analysis is known as subset analysis, where a certain group of effect sizes (studies) will be excluded to re-run the models without this group of effect sizes. For example, one may want to run an analysis without studies that did not randomize samples. Yet, as mentioned earlier, we recommend using meta-regression (Eq.  13 ) with a categorical variable of randomization status (‘randomized’ or ‘not randomized’), to statistically test for an influence of moderators. It is important to note that such tests for risk of bias (or study quality) can be considered as a way of quantitatively evaluating the importance of study features that were noted at the stage of critical appraisal, which is an essential part of any systematic review (see [ 11 , 91 ]). In other words, we can use meta-regression or subset analysis to quantitatively conduct critical appraisal using (study-level) moderators that code, for example, blinding, randomization, and selective reporting. Despite the importance of critical appraisal ([ 91 ]), only 4 of 73 environmental meta-analyses (5.6%) in our survey assessed the risk of bias in each study included in a meta-analysis (i.e., evaluating a primary study in terms of the internal validity of study design and reporting; Additional file 1 ). We emphasize that critically appraising each paper or checking them for risk of bias is an extremely important topic. Also, critical appraisal is not restricted to quantitative synthesis. Therefore, we do not cover any further in this paper for more, see [ 92 , 93 ]).

Notes on transparent reporting and open archiving

For environmental systematic reviews and maps, there are reporting guidelines called RepOrting standards for Systematic Evidence Syntheses in environmental research, ROSES [ 94 ] and synthesis assessment checklist, the Collaboration for Environmental Evidence Synthesis Appraisal Tool (CEESAT; [ 95 ]). However, these guidelines are somewhat limited in terms of reporting quantitative synthesis because they cover only a few core items. These two guidelines are complemented by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Ecology and Evolutionary Biology (PRISMA-EcoEvo; [ 96 ]; cf. [ 97 , 98 ]), which provides an extended set of reporting items covering what we have described above. Items 20–24 from PRISMA-EcoEvo are most relevant: these items outline what should be reported in the Methods section: (i) sample sizes and study characteristics, (ii) meta-analysis, (iii) heterogeneity, (iv) meta-regression and (v) outcomes of publication bias and sensitivity analysis (see Table 4 ). Our survey, as well as earlier surveys, suggest there is a large room for improvement in the current practice ([ 14 , 15 , 16 ]). Incidentally, the orchard plot is well aligned with Item 20, as this plot type shows both the number of effect sizes and studies for different groups (Fig.  4 C). Further, our survey of environmental meta-analyses highlighted the poor standards of data openness (with 24 studies sharing data: 32.9%) and code sharing (7 studies: 29.2%; Additional file 1 ). Environmental scientists must archive their data as well as their analysis code in accordance with the FAIR principles (Findable, Accessible, Interoperable, and Reusable [ 99 ]) using dedicated depositories such as Dryad, FigShare, Open Science Framework (OSF), Zenodo or others (cf. [ 100 , 101 ]), preferably not on publisher’s webpages (as paywall may block access). However, archiving itself is not enough; data requires metadata (detailed descriptions) and the code needs to also be FAIR [ 102 , 103 ].

Other relevant and advanced issues

Scale dependence.

The issue of scale dependence is a unique yet widespread problem in environmental sciences (see [ 7 , 104 ]); our literature survey indicated three quarters of the environmental meta-analyses (56 out of 73 studies) have inferences that are potentially vulnerable to scale-dependence [ 105 ]. For example, studies that set out to compare group means in biodiversity measures, such as species richness, can vary as a function of the scale (size) of the sampling unit. When the unit of replication is a plot (not an individual animal or plant), the aerial size of a plot (e.g., 100 cm 2 or 1 km 2 ) will affect both the precision and accuracy of effect size estimates (e.g., lnRR and SMD). In general, a study with larger plots might have more accurately estimated species richness differences, but less precisely than a study with smaller plots and greater replication. Lower replication means that our sampling variance estimates are likely to be misestimated, and the study with larger plots will generally have less weight than the study with smaller plots, due to higher sampling variance. Inaccurate variance estimates in little-replicated ecological studies are known to cause an accumulating bias in precision-weighted meta-analysis, requiring correction [ 43 ]. To assess the potential for scale-dependence, it is recommended that analysts test for possible covariation among plot size, replication, variances, and effect sizes [ 104 ]. If detected, analysts should use an effect size measure that is less sensitive to scale dependence (lnRR), and could use the size of a plot as a moderator in meta-regression, or alternatively, they consider running an unweighted model ([ 7 ]; note that only 12%, 9 out of 73 studies, accounted for sampling area in some way; Additional file 1 ).

• Missing data

In many fields, meta-analytic data almost always encompass missing values see [ 106 , 107 , 108 ]. Broadly, we have two types of missing data in meta-analyses [ 109 , 110 ]: (1) missing data in standard deviations or sample sizes, associated with means, preventing effect size calculations (Table 2 ), and (2) missing data in moderators. There are several solutions for both types. The best, and first to try, should be contacting the authors. If this fails, we can potentially ‘impute’ missing data. Single imputation methods using the strong correlation between standard deviation and mean values (known as mean–variance relationship) are available, although single imputation can lead to Type I error [ 106 , 107 ] (see also [ 43 ]) because we do not model the uncertainty of imputation itself. Contrastingly, multiple imputation, which creates multiple versions of imputed datasets, incorporates such uncertainty. Indeed, multiple imputation is a preferred and proven solution for missing data in effect sizes and moderators [ 109 , 110 ]. Yet, correct implementation can be challenging (see [ 110 ]). What we require now is an automated pipeline of merging meta-analysis and multiple imputation, which accounts for imputation uncertainty, although it may be challenging for complex meta-analytic models. Fortunately, however, for lnRR, there is a series of new methods that can perform better than the conventional method and which can deal with missing SDs [ 44 ]; note that these methods do not deal with missing moderators. Therefore, where applicable, we recommend these new methods, until an easy-to-implement multiple imputation workflow arrives.

Complex non-independence

Above, we have only dealt with the model that includes study identities as a clustering/grouping (random) factor. However, many datasets are more complex, with potentially more clustering variables in addition to the study identity. It is certainly possible that an environmental meta-analysis contains data from multiple species. Such a situation creates an interesting dependence among effect sizes from different species, known as phylogenetic relatedness, where closely related species are more likely to be similar in effect sizes compared to distantly related ones (e.g., mice vs. rats and mice vs. sparrows). Our multilevel model framework is flexible and can accommodate phylogenetic relatedness. A phylogenetic multilevel meta-analytic model can be written as [ 40 , 111 , 112 ]:

where \({a}_{k\left[i\right]}\) is the phylogenetic (species) effect for the k th species (effect size i ; N effect ( i  = 1, 2,…, N effect ) >  N study ( j  = 1, 2,…, N study ) >  N species ( k  = 1, 2,…, N species )), normally distributed with \({\omega }^{2}{\text{A}}\) where is the phylogenetic variance and A is a correlation matrix coding how close each species are to each other and \({\omega }^{2}\) is the phylogenetic variance, \({s}_{k\left[i\right]}\) is the non-phylogenetic (species) effect for the k th species (effect size i ), normally distributed with the variance of \({\gamma }^{2}\) (the non-phylogenetic variance), and other notations are as above. It is important to realize that A explicitly models relatedness among species, and we do need to provide this correlation matrix, using a distance relationship usually derived from a molecular-based phylogenetic tree (for more details, see [ 40 , 111 , 112 ]). Some may think that the non-phylogenetic term ( \({s}_{k\left[i\right]}\) ) is unnecessary or redundant because \({s}_{k\left[i\right]}\) and the phylogenetic term ( \({a}_{k\left[i\right]}\) ) are both modelling variance at the species level. However, a simulation recently demonstrated that failing to have the non-phylogenetic term ( \({s}_{k\left[i\right]}\) ) will often inflate the phylogenetic variance \({\omega }^{2}\) , leading to an incorrect conclusion that there is a strong phylogenetic signal (as shown in [ 112 ]). The non-phylogenetic variance ( \({\gamma }^{2}\) ) arises from, for example, ecological similarities among species (herbivores vs. carnivores or arboreal vs. ground-living) not phylogeny [ 40 ].

Like phylogenetic relatedness, effect sizes arising from closer geographical locations are likely to be more correlated [ 113 ]. Statistically, spatial correlation can be also modelled in a manner analogous to phylogenetic relatedness (i.e., rather than a phylogenetic correlation matrix, A , we fit a spatial correlation matrix). For example, Maire and colleagues [ 114 ] used a meta-analytic model with spatial autocorrelation to investigate the temporal trends of fish communities in the network of rivers in France. We note that a similar argument can be made for temporal correlation, but in many cases, temporal correlations could be dealt with, albeit less accurately, as a special case of ‘shared measurements’, as in Fig.  2 . An important idea to take away is that one can model different, if not all, types of non-independence as the random factor(s) in a multilevel model.

Advanced techniques

Here we touch upon five advanced meta-analytic techniques with potential utility for environmental sciences, providing relevant references so that interested readers can obtain more information on these advanced topics. The first one is the meta-analysis of magnitudes, or absolute values (effect sizes), where researchers may be interested in deviations from 0, rather than the directionality of the effect [ 115 ]. For example, Cohen and colleagues [ 116 ] investigated absolute values of phenological responses, as they were concerned with the magnitudes of changes in phenology rather than directionality.

The second method is the meta-analysis of interaction where our focus is on synthesizing the interaction effect of, usually, 2 × 2 factorial design (e.g., the effect of two simultaneous environmental stressors [ 54 , 117 , 118 ]; see also [ 119 ]). Recently, Siviter and colleagues [ 120 ] showed that agrochemicals interact synergistically (i.e., non-additively) to increase the mortality of bees; that is, two agrochemicals together caused more mortality than the sum of mortalities of each chemical.

Third, network meta-analysis has been heavily used in medical sciences; network meta-analysis usually compares different treatments in relation to placebo and ranks these treatments in terms of effectiveness [ 121 ]. The very first ‘environmental’ network meta-analysis, as far as we know, investigated the effectives of ecosystem services among different land types [ 122 ].

Fourth, a multivariate meta-analysis is where one can model two or more different types of effect sizes with the estimation of pair-wise correlations between different effect sizes. The benefit of such an approach is known as the ‘borrowing of strength’, where the error of fixed effects (moderators; e.g., b 0 and b 1 ) can be reduced when different types of effect sizes are correlated (i.e., se ( b 0 ) and se ( b 1 ) can be smaller [ 123 ]) For example, it is possible for lnRR (differences in mean) and lnVR (differences in SDs) to be modelled together (cf. [ 124 ]).

Fifth, as with network meta-analysis, there has been a surge in the use of ‘individual participants data’, called ‘IPD meta-analysis’, in medical sciences [ 125 , 126 ]. The idea of IPD meta-analysis is simple—rather than using summary statistics reported in papers (sample means and variances), we directly use raw data from all studies. We can either model raw data using one complex multilevel (hierarchical) model (one-step method) or calculate statistics for each study and use a meta-analysis (two-step method; note that both methods will usually give the same results). Study-level random effects can be incorporated to allow the response variable of interest to vary among studies, and overall effects correspond to fixed, population-level estimates. The use of IPD or ‘full-data analyses’ has also surged in ecology, aided by open-science policies that encourage the archival of raw data alongside articles, and initiatives that synthesise raw data (e.g., PREDICTS [ 127 ], BioTime [ 128 ]). In health disciplines, such meta-analyses are considered the ‘gold standard’ [ 129 ], owing to their potential for resolving issues regarding study-specific designs and confounding variation, and it is unclear whether and how they might resolve issues such as scale dependence in environmental meta-analyses [ 104 , 130 ].

Conclusions

In this article, we have attempted to describe the most practical ways to conduct quantitative synthesis, including meta-analysis, meta-regression, and publication bias tests. In addition, we have shown that there is much to be improved in terms of meta-analytic practice and reporting via a survey of 73 recent environmental meta-analyses. Such improvements are urgently required, especially given the potential influence that environmental meta-analyses can have on policies and decision-making [ 8 ]. So often, meta-analysts have called for better reporting of primary research (e.g. [ 131 , 132 ]), and now this is the time to raise the standards of reporting in meta-analyses. We hope our contribution will help to catalyse a turning point for better practice in quantitative synthesis in environmental sciences. We remind the reader most of what is described is implemented in the R environment on our tutorial webpage and researchers can readily use the proposed models and techniques ( https://itchyshin.github.io/Meta-analysis_tutorial/ ). Finally, meta-analytic techniques are always developing and improving. It is certainly possible that in the future, our proposed models and related methods will become dated, just as the traditional fixed-effect and random-effects models already are. Therefore, we must endeavour to be open-minded to new ways of doing quantitative research synthesis in environmental sciences.

Availability of data and materials

All data and material are provided as additional files.

Higgins JP, Thomas JE, Chandler JE, Cumpston ME, Li TE, Page MJ, Welch VA. Cochrane handbook for systematic reviews of interventions. 2nd ed. Chichester: Wikey; 2019.

Book   Google Scholar  

Cooper HM, Hedges LV, Valentine JC. The handbook of research synthesis and meta-analysis . 3rd ed. New York: Russell Sage Foundation; 2019.

Google Scholar  

Schmid CH, Stijnen TE, White IE. Handbook of meta-analysis. 1st ed. Boca Ranton: CRC; 2021.

Vetter D, Rucker G, Storch I. Meta-analysis: a need for well-defined usage in ecology and conservation biology. Ecosphere. 2013;4(6):1.

Article   Google Scholar  

Koricheva J, Gurevitch J, Mengersen K, editors. Handbook of meta-analysis in ecology and evolution. Princeton: Princeton Univesity Press; 2017.

Gurevitch J, Koricheva J, Nakagawa S, Stewart G. Meta-analysis and the science of research synthesis. Nature. 2018;555(7695):175–82.

Article   CAS   Google Scholar  

Spake R, Doncaster CP. Use of meta-analysis in forest biodiversity research: key challenges and considerations. Forest Ecol Manag. 2017;400:429–37.

Bilotta GS, Milner AM, Boyd I. On the use of systematic reviews to inform environmental policies. Environ Sci Policy. 2014;42:67–77.

Hedges LV, Vevea JL. Fixed- and random-effects models in meta-analysis. Psychol Methods. 1998;3(4):486–504.

Borenstein M, Hedges LV, Higgins JPT, Rothstein H. Introduction to meta-analysis. 2nd ed. Chichester: Wiley; 2021.

Noble DWA, Lagisz M, Odea RE, Nakagawa S. Nonindependence and sensitivity analyses in ecological and evolutionary meta-analyses. Mol Ecol. 2017;26(9):2410–25.

Nakagawa S, Noble DWA, Senior AM, Lagisz M. Meta-evaluation of meta-analysis: ten appraisal questions for biologists. Bmc Biol. 2017;15:1.

Nakagawa S, Senior AM, Viechtbauer W, Noble DWA. An assessment of statistical methods for nonindependent data in ecological meta-analyses: comment. Ecology. 2022;103(1): e03490.

Romanelli JP, Meli P, Naves RP, Alves MC, Rodrigues RR. Reliability of evidence-review methods in restoration ecology. Conserv Biol. 2021;35(1):142–54.

Koricheva J, Gurevitch J. Uses and misuses of meta-analysis in plant ecology. J Ecol. 2014;102(4):828–44.

O’Leary BC, Kvist K, Bayliss HR, Derroire G, Healey JR, Hughes K, Kleinschroth F, Sciberras M, Woodcock P, Pullin AS. The reliability of evidence review methodology in environmental science and conservation. Environ Sci Policy. 2016;64:75–82.

Rosenthal R. The “file drawer problem” and tolerance for null results. Psychol Bull. 1979;86(3):638–41.

Nakagawa S, Lagisz M, Jennions MD, Koricheva J, Noble DWA, Parker TH, Sánchez-Tójar A, Yang Y, O’Dea RE. Methods for testing publication bias in ecological and evolutionary meta-analyses. Methods Ecol Evol. 2022;13(1):4–21.

Cheung MWL. A guide to conducting a meta-analysis with non-independent effect sizes. Neuropsychol Rev. 2019;29(4):387–96.

Viechtbauer W. Conducting meta-analyses in R with the metafor package. J Stat Softw. 2010;36(3):1–48.

Yang Y, Macleod M, Pan J, Lagisz M, Nakagawa S. Advanced methods and implementations for the meta-analyses of animal models: current practices and future recommendations. Neurosci Biobehav Rev. 2022. https://doi.org/10.1016/j.neubiorev.2022.105016:105016 .

Nakagawa S, Cuthill IC. Effect size, confidence interval and statistical significance: a practical guide for biologists. Biol Rev. 2007;82(4):591–605.

Hedges LV, Gurevitch J, Curtis PS. The meta-analysis of response ratios in experimental ecology. Ecology. 1999;80(4):1150–6.

Friedrich JO, Adhikari NKJ, Beyene J. The ratio of means method as an alternative to mean differences for analyzing continuous outcome variables in meta-analysis: A simulation study. BMC Med Res Methodol. 2008;8:5.

Hedges L, Olkin I. Statistical methods for meta-analysis. New York: Academic Press; 1985.

Cohen J. Statistical power analysis for the beahvioral sciences. 2nd ed. Hillsdale: Lawrence Erlbaum; 1988.

Senior AM, Viechtbauer W, Nakagawa S. Revisiting and expanding the meta-analysis of variation: the log coefficient of variation ratio. Res Synth Methods. 2020;11(4):553–67.

Nakagawa S, Poulin R, Mengersen K, Reinhold K, Engqvist L, Lagisz M, Senior AM. Meta-analysis of variation: ecological and evolutionary applications and beyond. Methods Ecol Evol. 2015;6(2):143–52.

Knapp S, van der Heijden MGA. A global meta-analysis of yield stability in organic and conservation agriculture. Nat Commun. 2018;9:3632.

Porturas LD, Anneberg TJ, Cure AE, Wang SP, Althoff DM, Segraves KA. A meta-analysis of whole genome duplication and theeffects on flowering traits in plants. Am J Bot. 2019;106(3):469–76.

Janicke T, Morrow EH. Operational sex ratio predicts the opportunity and direction of sexual selection across animals. Ecol Lett. 2018;21(3):384–91.

Chamberlain R, Brunswick N, Siev J, McManus IC. Meta-analytic findings reveal lower means but higher variances in visuospatial ability in dyslexia. Brit J Psychol. 2018;109(4):897–916.

O’Dea RE, Lagisz M, Jennions MD, Nakagawa S. Gender differences in individual variation in academic grades fail to fit expected patterns for STEM. Nat Commun. 2018;9:3777.

Brugger SP, Angelescu I, Abi-Dargham A, Mizrahi R, Shahrezaei V, Howes OD. Heterogeneity of striatal dopamine function in schizophrenia: meta-analysis of variance. Biol Psychiat. 2020;87(3):215–24.

Usui T, Macleod MR, McCann SK, Senior AM, Nakagawa S. Meta-analysis of variation suggests that embracing variability improves both replicability and generalizability in preclinical research. Plos Biol. 2021;19(5): e3001009.

Hoffmann AA, Merila J. Heritable variation and evolution under favourable and unfavourable conditions. Trends Ecol Evol. 1999;14(3):96–101.

Wood CW, Brodie ED 3rd. Environmental effects on the structure of the G-matrix. Evolution. 2015;69(11):2927–40.

Hillebrand H, Donohue I, Harpole WS, Hodapp D, Kucera M, Lewandowska AM, Merder J, Montoya JM, Freund JA. Thresholds for ecological responses to global change do not emerge from empirical data. Nat Ecol Evol. 2020;4(11):1502.

Yang YF, Hillebrand H, Lagisz M, Cleasby I, Nakagawa S. Low statistical power and overestimated anthropogenic impacts, exacerbated by publication bias, dominate field studies in global change biology. Global Change Biol. 2022;28(3):969–89.

Nakagawa S, Santos ESA. Methodological issues and advances in biological meta-analysis. Evol Ecol. 2012;26(5):1253–74.

Bakbergenuly I, Hoaglin DC, Kulinskaya E. Estimation in meta-analyses of response ratios. BMC Med Res Methodol. 2020;20(1):1.

Bakbergenuly I, Hoaglin DC, Kulinskaya E. Estimation in meta-analyses of mean difference and standardized mean difference. Stat Med. 2020;39(2):171–91.

Doncaster CP, Spake R. Correction for bias in meta-analysis of little-replicated studies. Methods Ecol Evol. 2018;9(3):634–44.

Nakagawa S, Noble DW, Lagisz M, Spake R, Viechtbauer W, Senior AM. A robust and readily implementable method for the meta-analysis of response ratios with and without missing standard deviations. Ecol Lett. 2023;26(2):232–44

Hamman EA, Pappalardo P, Bence JR, Peacor SD, Osenberg CW. Bias in meta-analyses using Hedges’ d. Ecosphere. 2018;9(9): e02419.

Bakbergenuly I, Hoaglin DC, Kulinskaya E. On the Q statistic with constant weights for standardized mean difference. Brit J Math Stat Psy. 2022;75(3):444–65.

DerSimonian R, Kacker R. Random-effects model for meta-analysis of clinical trials: an update. Contemp Clin Trials. 2007;28(2):105–14.

Veroniki AA, Jackson D, Viechtbauer W, Bender R, Bowden J, Knapp G, Kuss O, Higgins JPT, Langan D, Salanti G. Methods to estimate the between-study variance and its uncertainty in meta-analysis. Res Synth Methods. 2016;7(1):55–79.

Langan D, Higgins JPT, Simmonds M. Comparative performance of heterogeneity variance estimators in meta-analysis: a review of simulation studies. Res Synth Methods. 2017;8(2):181–98.

Panityakul T, Bumrungsup C, Knapp G. On estimating residual heterogeneity in random-effects meta-regression: a comparative study. J Stat Theory Appl. 2013;12(3):253–65.

Bishop J, Nakagawa S. Quantifying crop pollinator dependence and its heterogeneity using multi-level meta-analysis. J Appl Ecol. 2021;58(5):1030–42.

Cheung MWL. Modeling dependent effect sizes with three-level meta-analyses: a structural equation modeling approach. Psychol Methods. 2014;19(2):211–29.

Bolker BM, Brooks ME, Clark CJ, Geange SW, Poulsen JR, Stevens MHH, White JSS. Generalized linear mixed models: a practical guide for ecology and evolution. Trends Ecol Evol. 2009;24(3):127–35.

Lajeunesse MJ. On the meta-analysis of response ratios for studies with correlated and multi-group designs. Ecology. 2011;92(11):2049–55.

Gleser LJ, Olkin I. Stochastically dependent effect sizes. In: Cooper H, Hedges LV, Valentine JC, editors. The handbook of research synthesis and meta-analysis. New York: Russell Sage Foundation; 2009.

Tipton E, Pustejovsky JE. Small-sample adjustments for tests of moderators and model fit using robust variance estimation in meta-regression. J Educ Behav Stat. 2015;40(6):604–34.

Hedges LV, Tipton E, Johnson MC. Robust variance estimation in meta-regression with dependent effect size estimates (vol 1, pg 39, 2010). Res Synth Methods. 2010;1(2):164–5.

Pustejovsky JE, Tipton E. Meta-analysis with robust variance estimation: expanding the range of working models. Prev Sci. 2021. https://doi.org/10.1007/s11121-021-01246-3 .

Cairns M, Prendergast LA. On ratio measures of heterogeneity for meta-analyses. Res Synth Methods. 2022;13(1):28–47.

Borenstein M, Higgins JPT, Hedges LV, Rothstein HR. Basics of meta-analysis: I2 is not an absolute measure of heterogeneity. Res Synth Methods. 2017;8(1):5–18.

Hoaglin DC. Practical challenges of I-2 as a measure of heterogeneity. Res Synth Methods. 2017;8(3):254–254.

Higgins JPT, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med. 2002;21(11):1539–58.

Higgins JPT, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. Brit Med J. 2003;327(7414):557–60.

Xiong CJ, Miller JP, Morris JC. Measuring study-specific heterogeneity in meta-analysis: application to an antecedent biomarker study of Alzheimer’s disease. Stat Biopharm Res. 2010;2(3):300–9.

Nakagawa S, Schielzeth H. Repeatability for Gaussian and non-Gaussian data: a practical guide for biologists. Biol Rev. 2010;85(4):935–56.

Senior AM, Grueber CE, Kamiya T, Lagisz M, O’Dwyer K, Santos ESA, Nakagawa S. Heterogeneity in ecological and evolutionary meta-analyses: its magnitude and implications. Ecology. 2016;97(12):3293–9.

Gelman A, Hill J. Data analysis using regression and multilevel/hierarchical models. Cambridge: Cambridge University Press; 2007.

Schielzeth H. Simple means to improve the interpretability of regression coefficients. Methods Ecol Evol. 2010;1(2):103–13.

Nakagawa S, Schielzeth H. A general and simple method for obtaining R2 from generalized linear mixed-effects models. Methods Ecol Evol. 2013;4(2):133–42.

Nakagawa S, Johnson PCD, Schielzeth H. The coefficient of determination R-2 and intra-class correlation coefficient from generalized linear mixed-effects models revisited and expanded. J R Soc Interface. 2017;14(134):20170213.

Aloe AM, Becker BJ, Pigott TD. An alternative to R-2 for assessing linear models of effect size. Res Synth Methods. 2010;1(3–4):272–83.

Cinar O, Umbanhowar J, Hoeksema JD, Viechtbauer W. Using information-theoretic approaches for model selection in meta-analysis. Res Synth Methods. 2021. https://doi.org/10.1002/jrsm.1489 .

Viechtbauer W. Model checking in meta-analysis. In: Schmid CH, Stijnen T, White IR, editors. Handbook of meta-analysis. Boca Raton: CRC; 2021.

Anzures-Cabrera J, Higgins JPT. Graphical displays for meta-analysis: An overview with suggestions for practice. Res Synth Methods. 2010;1(1):66–80.

Kossmeier M, Tran US, Voracek M. Charting the landscape of graphical displays for meta-analysis and systematic reviews: a comprehensive review, taxonomy, and feature analysis. Bmc Med Res Methodol. 2020;20(1):1.

Intout J, Ioannidis JPA, Rovers MM, Goeman JJ. Plea for routinely presenting prediction intervals in meta-analysis. BMJ Open. 2016;6(7): e010247.

Moeyaert M, Ugille M, Beretvas SN, Ferron J, Bunuan R, Van den Noortgate W. Methods for dealing with multiple outcomes in meta-analysis a comparison between averaging effect sizes, robust variance estimation and multilevel meta-analysis. Int J Soc Res Methodol. 2017;20:559.

Nakagawa S, Lagisz M, O’Dea RE, Rutkowska J, Yang YF, Noble DWA, Senior AM. The orchard plot: cultivating a forest plot for use in ecology, evolution, and beyond. Res Synth Methods. 2021;12(1):4–12.

Rothstein H, Sutton AJ, Borenstein M. Publication bias in meta-analysis : prevention, assessment and adjustments. Hoboken: Wiley; 2005.

Nakagawa S, Lagisz M, Jennions MD, Koricheva J, Noble DWA, Parker TH, Sanchez-Tojar A, Yang YF, O’Dea RE. Methods for testing publication bias in ecological and evolutionary meta-analyses. Methods Ecol Evol. 2022;13(1):4–21.

Stanley TD, Doucouliagos H. Meta-regression analysis in economics and business. New York: Routledge; 2012.

Stanley TD, Doucouliagos H. Meta-regression approximations to reduce publication selection bias. Res Synth Methods. 2014;5(1):60–78.

Sterne JAC, Becker BJ, Egger M. The funnel plot. In: Rothstein H, Sutton AJ, Borenstein M, editors. Publication bias in meta-analysis: prevention, assessment and adjustments. Chichester: Wiley; 2005. p. 75–98.

Sterne JAC, Sutton AJ, Ioannidis JPA, Terrin N, Jones DR, Lau J, Carpenter J, Rucker G, Harbord RM, Schmid CH, et al. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. Br Med J. 2011;343:4002.

Egger M, Smith GD, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. Brit Med J. 1997;315(7109):629–34.

Jennions MD, Moller AP. Relationships fade with time: a meta-analysis of temporal trends in publication in ecology and evolution. P Roy Soc B-Biol Sci. 2002;269(1486):43–8.

Koricheva J, Kulinskaya E. Temporal instability of evidence base: a threat to policy making? Trends Ecol Evol. 2019;34(10):895–902.

Raue A, Kreutz C, Maiwald T, Bachmann J, Schilling M, Klingmuller U, Timmer J. Structural and practical identifiability analysis of partially observed dynamical models by exploiting the profile likelihood. Bioinformatics. 2009;25(15):1923–9.

Matsushima Y, Noma H, Yamada T, Furukawa TA. Influence diagnostics and outlier detection for meta-analysis of diagnostic test accuracy. Res Synth Methods. 2020;11(2):237–47.

Viechtbauer W, Cheung MWL. Outlier and influence diagnostics for meta-analysis. Res Synth Methods. 2010;1(2):112–25.

Haddaway NR, Macura B. The role of reporting standards in producing robust literature reviews comment. Nat Clim Change. 2018;8(6):444–7.

Frampton G, Whaley P, Bennett M, Bilotta G, Dorne JLCM, Eales J, James K, Kohl C, Land M, Livoreil B, et al. Principles and framework for assessing the risk of bias for studies included in comparative quantitative environmental systematic reviews. Environ Evid. 2022;11(1):12.

Stanhope J, Weinstein P. Critical appraisal in ecology: what tools are available, and what is being used in systematic reviews? Res Synth Methods. 2022. https://doi.org/10.1002/jrsm.1609 .

Haddaway NR, Macura B, Whaley P, Pullin AS. ROSES RepOrting standards for systematic evidence syntheses: pro forma, flow-diagram and descriptive summary of the plan and conduct of environmental systematic reviews and systematic maps. Environ Evid. 2018;7(1):1.

Woodcock P, Pullin AS, Kaiser MJ. Evaluating and improving the reliability of evidence syntheses in conservation and environmental science: a methodology. Biol Conserv. 2014;176:54–62.

O’Dea RE, Lagisz M, Jennions MD, Koricheva J, Noble DWA, Parker TH, Gurevitch J, Page MJ, Stewart G, Moher D, et al. Preferred reporting items for systematic reviews and meta-analyses in ecology and evolutionary biology: a PRISMA extension. Biol Rev. 2021;96(5):1695–722.

Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Plos Med. 2009;6(7):e1000097.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, Shamseer L, Tetzlaff JM, Akl EA, Brennan SE, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Plos Med. 2021;18(3): e1003583.

Wilkinson MD, Dumontier M, Aalbersberg IJ, Appleton G, Axton M, Baak A, Blomberg N, Boiten JW, Santos LBD, Bourne PE, et al. Comment: the FAIR guiding principles for scientific data management and stewardship. Sci Data. 2016;3: 160018.

Culina A, Baglioni M, Crowther TW, Visser ME, Woutersen-Windhouwer S, Manghi P. Navigating the unfolding open data landscape in ecology and evolution. Nat Ecol Evol. 2018;2(3):420–6.

Roche DG, Lanfear R, Binning SA, Haff TM, Schwanz LE, Cain KE, Kokko H, Jennions MD, Kruuk LE. Troubleshooting public data archiving: suggestions to increase participation. Plos Biol. 2014;12(1): e1001779.

Roche DG, Kruuk LEB, Lanfear R, Binning SA. Public data archiving in ecology and evolution: how well are we doing? Plos Biol. 2015;13(11): e1002295.

Culina A, van den Berg I, Evans S, Sanchez-Tojar A. Low availability of code in ecology: a call for urgent action. Plos Biol. 2020;18(7): e3000763.

Spake R, Mori AS, Beckmann M, Martin PA, Christie AP, Duguid MC, Doncaster CP. Implications of scale dependence for cross-study syntheses of biodiversity differences. Ecol Lett. 2021;24(2):374–90.

Osenberg CW, Sarnelle O, Cooper SD. Effect size in ecological experiments: the application of biological models in meta-analysis. Am Nat. 1997;150(6):798–812.

Noble DWA, Nakagawa S. Planned missing data designs and methods: options for strengthening inference, increasing research efficiency and improving animal welfare in ecological and evolutionary research. Evol Appl. 2021;14(8):1958–68.

Nakagawa S, Freckleton RP. Missing inaction: the dangers of ignoring missing data. Trends Ecol Evol. 2008;23(11):592–6.

Mavridis D, Chaimani A, Efthimiou O, Leucht S, Salanti G. Addressing missing outcome data in meta-analysis. Evid-Based Ment Health. 2014;17(3):85.

Ellington EH, Bastille-Rousseau G, Austin C, Landolt KN, Pond BA, Rees EE, Robar N, Murray DL. Using multiple imputation to estimate missing data in meta-regression. Methods Ecol Evol. 2015;6(2):153–63.

Kambach S, Bruelheide H, Gerstner K, Gurevitch J, Beckmann M, Seppelt R. Consequences of multiple imputation of missing standard deviations and sample sizes in meta-analysis. Ecol Evol. 2020;10(20):11699–712.

Hadfield JD, Nakagawa S. General quantitative genetic methods for comparative biology: phylogenies, taxonomies and multi-trait models for continuous and categorical characters. J Evol Biol. 2010;23(3):494–508.

Cinar O, Nakagawa S, Viechtbauer W. Phylogenetic multilevel meta-analysis: a simulation study on the importance of modelling the phylogeny. Methods Ecol Evol. 2021. https://doi.org/10.1111/2041-210X.13760 .

Ives AR, Zhu J. Statistics for correlated data: phylogenies, space, and time. Ecol Appl. 2006;16(1):20–32.

Maire A, Thierry E, Viechtbauer W, Daufresne M. Poleward shift in large-river fish communities detected with a novel meta-analysis framework. Freshwater Biol. 2019;64(6):1143–56.

Morrissey MB. Meta-analysis of magnitudes, differences and variation in evolutionary parameters. J Evol Biol. 2016;29(10):1882–904.

Cohen JM, Lajeunesse MJ, Rohr JR. A global synthesis of animal phenological responses to climate change. Nat Clim Change. 2018;8(3):224.

Gurevitch J, Morrison JA, Hedges LV. The interaction between competition and predation: a meta-analysis of field experiments. Am Nat. 2000;155(4):435–53.

Macartney EL, Lagisz M, Nakagawa S. The relative benefits of environmental enrichment on learning and memory are greater when stressed: a meta-analysis of interactions in rodents. Neurosci Biobehav R. 2022. https://doi.org/10.1016/j.neubiorev.2022.104554 .

Spake R, Bowler DE, Callaghan CT, Blowes SA, Doncaster CP, Antão LH, Nakagawa S, McElreath R, Chase JM. Understanding ‘it depends’ in ecology: a guide to hypothesising, visualising and interpreting statistical interactions. Biol Rev. 2023. https://doi.org/10.1111/brv.12939 .

Siviter H, Bailes EJ, Martin CD, Oliver TR, Koricheva J, Leadbeater E, Brown MJF. Agrochemicals interact synergistically to increase bee mortality. Nature. 2021;596(7872):389.

Salanti G, Schmid CH. Research synthesis methods special issue on network meta-analysis: introduction from the editors. Res Synth Methods. 2012;3(2):69–70.

Gomez-Creutzberg C, Lagisz M, Nakagawa S, Brockerhoff EG, Tylianakis JM. Consistent trade-offs in ecosystem services between land covers with different production intensities. Biol Rev. 2021;96(5):1989–2008.

Jackson D, White IR, Price M, Copas J, Riley RD. Borrowing of strength and study weights in multivariate and network meta-analysis. Stat Methods Med Res. 2017;26(6):2853–68.

Sanchez-Tojar A, Moran NP, O’Dea RE, Reinhold K, Nakagawa S. Illustrating the importance of meta-analysing variances alongside means in ecology and evolution. J Evol Biol. 2020;33(9):1216–23.

Riley RD, Lambert PC, Abo-Zaid G. Meta-analysis of individual participant data: rationale, conduct, and reporting. BMJ. 2010;340:c221.

Riley RD, Tierney JF, Stewart LA. Individual participant data meta-analysis : a handbook for healthcare research. 1st ed. Hoboken: Wiley; 2021.

Hudson LN, Newbold T, Contu S, Hill SLL, Lysenko I, De Palma A, Phillips HRP, Alhusseini TI, Bedford FE, Bennett DJ, et al. The database of the PREDICTS (projecting responses of ecological diversity in changing terrestrial systems) project. Ecol Evol. 2017;7(1):145–88.

Dornelas M, Antao LH, Moyes F, Bates AE, Magurran AE, Adam D, Akhmetzhanova AA, Appeltans W, Arcos JM, Arnold H, et al. BioTIME: a database of biodiversity time series for the anthropocene. Glob Ecol Biogeogr. 2018;27(7):760–86.

Mengersen K, Gurevitch J, Schmid CH. Meta-analysis of primary data. In: Koricheva J, Gurevitch J, Mengersen K, editors. Handbook of meta-analysis in ecology and evolution. Priceton: Princeton university; 2013. p. 300–12.

Spake R, O’Dea RE, Nakagawa S, Doncaster CP, Ryo M, Callaghan CT, Bullock JM. Improving quantitative synthesis to achieve generality in ecology. Nat Ecol Evol. 2022;6(12):1818–28.

Gerstner K, Moreno-Mateos D, Gurevitch J, Beckmann M, Kambach S, Jones HP, Seppelt R. Will your paper be used in a meta-analysis? Make the reach of your research broader and longer lasting. Methods Ecol Evol. 2017;8(6):777–84.

Haddaway NR. A call for better reporting of conservation research data for use in meta-analyses. Conserv Biol. 2015;29(4):1242–5.

Midolo G, De Frenne P, Holzel N, Wellstein C. Global patterns of intraspecific leaf trait responses to elevation. Global Change Biol. 2019;25(7):2485–98.

White IR, Schmid CH, Stijnen T. Choice of effect measure and issues in extracting outcome data. In: Schmid CH, Stijnen T, White IR, editors. Handbook of meta-analysis. Boca Raton: CRC; 2021.

Lajeunesse MJ. Bias and correction for the log response ratio in ecological meta-analysis. Ecology. 2015;96(8):2056–63.

Download references

Acknowledgements

SN, ELM, and ML were supported by the ARC (Australian Research Council) Discovery grant (DP200100367), and SN, YY, and ML by the ARC Discovery grant (DP210100812). YY was also supported by the National Natural Science Foundation of China (32102597). A part of this research was conducted while visiting the Okinawa Institute of Science and Technology (OIST) through the Theoretical Sciences Visiting Program (TSVP) to SN.

Australian Research Council Discovery grant (DP200100367); Australian Research Council Discovery grant (DP210100812); The National Natural Science Foundation of China (32102597).

Author information

Authors and affiliations.

Evolution & Ecology Research Centre and School of Biological, Earth and Environmental Sciences, University of New South Wales, Sydney, NSW, 2052, Australia

Shinichi Nakagawa, Yefeng Yang, Erin L. Macartney & Malgorzata Lagisz

Theoretical Sciences Visiting Program, Okinawa Institute of Science and Technology Graduate University, Onna, 904-0495, Japan

Shinichi Nakagawa

School of Biological Sciences, Whiteknights Campus, University of Reading, Reading, RG6 6AS, UK

Rebecca Spake

You can also search for this author in PubMed   Google Scholar

Contributions

SN was commissioned to write this article so he assembled a team of co-authors. SN discussed the idea with YY, ELM, RS and ML, and all of them contributed to the design of this review. ML led the survey working with YY and ELM, while YY led the creation of the accompanying webpage working with RS. SN supervised all aspects of this work and wrote the first draft, which was commented on, edited, and therefore, significantly improved by the other co-authors. All authors read and approved the final manuscript.

Corresponding authors

Correspondence to Shinichi Nakagawa or Yefeng Yang .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Contest for publication

The authors provide consent for publication.

Competing interests

The authors report no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1:.

The survey of meta-analyses in environmnetal sciences.

Additional file 2:

The hands-on R tutorial.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Nakagawa, S., Yang, Y., Macartney, E.L. et al. Quantitative evidence synthesis: a practical guide on meta-analysis, meta-regression, and publication bias tests for environmental sciences. Environ Evid 12 , 8 (2023). https://doi.org/10.1186/s13750-023-00301-6

Download citation

Received : 13 January 2023

Accepted : 23 March 2023

Published : 24 April 2023

DOI : https://doi.org/10.1186/s13750-023-00301-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Hierarchical models
• Robust variance estimation
• Spatial dependency
• Variance–covariance matrix
• Meta-analysis of variance
• Network meta-analysis
• Multivariate meta-analysis

Environmental Evidence

ISSN: 2047-2382

• General enquiries: [email protected]

Quantitative Methods for Current Environmental Issues

• Conference proceedings
• © 2002
• Clive W. Anderson 0 ,
• Vic Barnett 1 ,
• Philip C. Chatwin 2 ,
• Abdel H. El-Shaarawi 3

School of Mathematics and Statistics, University of Sheffield, Sheffield, UK

You can also search for this editor in PubMed   Google Scholar

School of Mathematical Sciences, University of Nottingham, Nottingham, UK

National water research institute, burlington, canada.

Features the most recent quantitative methods for studying environmental issues

Includes contributions from the foremost researchers in the field

8485 Accesses

236 Citations

This is a preview of subscription content, log in via an institution to check access.

Access this book

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime
• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Other ways to access

Licence this eBook for your library

Institutional subscriptions

About this book

Similar content being viewed by others, the insignificance of thresholds in environmental impact assessment: an illustrative case study in canada, basic mathematical errors may make ecological assessments unreliable.

What Is Sociological About Environmental Sociology?: Qualitative Methods in an Era of Rapid Environmental Change

• environment
• environmental science
• environmental sciences

Table of contents (12 papers)

Front matter, spatial and temporal models and methods, modeling spatio-temporally misaligned areal and point process environmental data.

• Bradley P. Carlin, Andrew S. Mugglin, Li Zhu, Alan E. Gelfand

Space and Space-Time Modeling using Process Convolutions

• Dave Higdon

Multivariate Kriging for Interpolating with Data from Different Sources

• H. Wackernagel, L. Bertino, J. P. Sierra, J. González del Río

Environmental Sampling and Standards

Distance sampling: recent advances and future directions.

• S. T. Buckland, L. Thomas, F. F. C. Marques, S. Strindberg, S. L. Hedley, J. H. Pollard et al.

Setting Environmental Standards: A Statistical Approach

• Vic Barnett, Marion Bown

Atmosphere and Ocean

The interpretation and validation of measurements of the ocean wave directional spectrum.

• Lucy R. Wyatt

Thermal Energy Emission and Propagation from Accidents

• A. Pelliccioni, F. Altavilla, S. Berardi

Development and Application of an Extended Methodology to Validate Short-Range Atmospheric Dispersion Models

• Harry Eleveld, Harry Slaper

Uncertainty and Sensitivity of Dispersion Model Results to Meteorological Inputs: Two Case Studies

• Joseph C. Chang

Risk and Uncertainty

Statistics and the environmental sciences: approaches to model combination.

• Gudmund Høst

Bayesian Analysis of Computer Code Outputs

• Marc C. Kennedy, Anthony O’Hagan, Neil Higgins

The Realities of Decision Making on Risks

• Jim McQuaid

Back Matter

Editors and affiliations.

Clive W. Anderson, Philip C. Chatwin

Vic Barnett

Abdel H. El-Shaarawi

Bibliographic Information

Book Title : Quantitative Methods for Current Environmental Issues

Editors : Clive W. Anderson, Vic Barnett, Philip C. Chatwin, Abdel H. El-Shaarawi

DOI : https://doi.org/10.1007/978-1-4471-0657-9

Publisher : Springer London

eBook Packages : Springer Book Archive

Copyright Information : Springer-Verlag London Limited 2002

Hardcover ISBN : 978-1-85233-294-5 Due: 15 January 2002

Softcover ISBN : 978-1-4471-1171-9 Published: 15 September 2011

eBook ISBN : 978-1-4471-0657-9 Published: 06 December 2012

Edition Number : 1

Number of Pages : X, 270

Topics : Statistical Theory and Methods , Math. Appl. in Environmental Science , Statistics for Engineering, Physics, Computer Science, Chemistry and Earth Sciences , Applications of Mathematics

• Publish with us

Policies and ethics

• Find a journal
• Track your research

• [email protected]
• +1 646 475 7865
• REQUEST FOR CALL BACK

What is the Importance of Quantitative Research in the Environment?

Owing to the inclination of millennials and GenZ towards sustainability and environmental protection, many researchers are turning towards environmental studies and working along with business owners to facilitate the development of new products for consumer use.

Quantitative research is a methodology that involves the collection and analysis of numerical data using statistical methods to make inferences about a population. It is essential in many fields, including the environment. It provides a rigorous, systematic approach to collecting and analyzing data that enables researchers to identify patterns, trends, and relationships that would be difficult to detect otherwise.

The importance of quantitative research in the environment cannot be underestimated. Let’s explore the methods and data collection techniques used in quantitative research and discuss the significance of accuracy and reliability in the article below for a better understanding of the importance of quantitative data analysis services in the environment.

Understanding Quantitative Research

Quantitative research uses methods such as surveys , experiments, and observational studies to collect data. These methods are used to collect data from a large sample size, which enables researchers to generalize their findings to a larger population.

Data collection techniques used in quantitative research include structured questionnaires, structured observations, and standardized tests. These techniques help to ensure that the data collected is accurate and reliable as they are crucial in quantitative research. Researchers must also ensure that the data collected is representative of the population they are studying and that the methods used are consistent across all participants.

The Importance of Quantitative Research in the Environment

Quantitative market research services are vital in identifying environmental problems such as air change. Through data analysis, researchers can identify trends and patterns that may not be immediately apparent, and use this information to develop solutions to mitigate or solve upcoming problems.

The right approach to research can also provide policymakers with data-driven insights into the potential impact of proposed policies. For example, quantitative research can be used to determine the potential cost-benefit of implementing a particular environmental regulation or to identify the areas of the population that would be most affected by a proposed policy change.

Another benefit of implementing quantitative research about environmental science is that it can be used to evaluate the effectiveness of environmental interventions, such as the impact of pollution reduction efforts or conservation programs. This type of research is critical in determining whether interventions are working as intended and can help to identify areas for improvement.

Example of Quantitative Research About Environment

Gathering quantitative data can give you insights into the most important areas of the environment. The right use of the data can help mitigate future risks and eliminate potential problems. Let’s talk about some of the examples where quantitative research can be used.

1. Air Quality Monitoring

Quantitative research is used to monitor air quality in many parts of the world. Data is collected using sensors that measure the concentration of pollutants in the air, such as particulate matter and nitrogen dioxide. This quantitative data is then analyzed to identify trends and patterns in air quality, which can be used to develop policies to reduce air pollution.

2. Water Quality Testing

One of the most significant examples of quantitative research about the environment is monitoring water quality in rivers, lakes, and oceans. Data is collected using water quality meters that measure parameters such as pH, dissolved oxygen, and temperature. This can help in analyzing the trends and patterns in water quality, which can be used to develop policies to protect water resources

3. Climate Change Modelling

Quantitative research about the environment and climate change is also used to study the effects of climate change on ecosystems, wildlife, and human populations. This data is collected using climate models that simulate the effects of climate change on various aspects of the environment.

Quantitative Research Methodologies in Environmental Studies

Quantitative research is a scientific method used to gather and analyze numerical data. This type of research is often used in environmental studies to measure and quantify different aspects of the natural world.

By using these methods, researchers can gather and analyze data in a scientific way, helping them to better understand the complex systems that make up our environment. Get in touch with the expert of research at Insights Opinion to get accurate data.

Challenges and Limitations of Quantitative Research in the Environment

Studying the environment is challenging as the data may differ at every geographical location and the respondents may have different views depending on the area they live. However, with the help of a quantitative market research company , you can enjoy smooth sailing and ensure that your final result comes from the data of a team of global panelists. They can guide you through every step of the way and give you expert opinions whenever required.

1. Difficulty in Measuring Complex Environmental Factors

Quantitative research about the environment can be challenging when measuring complex environmental factors such as biodiversity or ecological diversity. The complexity of these factors makes it difficult to develop accurate data, which can alter the conclusions.

2. Ethical Concerns Surrounding Data Collection and Analysis

Data collection and analysis can raise ethical concerns, particularly when studying vulnerable populations or when collecting data that is personal or sensitive in nature. Researchers must ensure that they obtain informed consent from participants and that the data collected is kept confidential.

3. Effect of Bias on Research Findings

Bias can have a significant impact on the findings of quantitative research, particularly when researchers have preconceived notions or biases. Researchers must take steps to minimize bias, such as using randomized sampling techniques and ensuring that their methods are consistent across all participants.

Let the Professionals of Market Research Help you in Quantitative Data Research

Quantitative research plays a vital role in understanding and addressing environmental issues. It provides policymakers with data-driven insights into the potential impact of proposed policies and helps to evaluate the effectiveness of environmental interventions.

As one of the most experienced quantitative market research agencies , Insights Opinion understands how important it is to collect data from various niches with the right practices to bring out the right results. With a team of expert researchers and 8 million+ panelists, we can help you gain maximum advantage from the data. Get in touch with our team today to know more about our services.

Q. 1 How can a market research company help my business?

Ans: A market research company can provide you with information about your target market which can help you make informed decisions about product development, pricing, and marketing strategies. It can help you identify areas of growth and opportunity in your industry.

Q. 2 What types of businesses can benefit from market research?

Ans: Any business that wants to understand its target market can benefit from market research. New businesses can identify their target market and develop effective marketing strategies and established businesses can stay ahead of industry trends and changes in consumer preferences.

Q. 3 How can I choose a market research company?

Ans: When choosing a market research company, it is important to consider their expertise in your industry, their experience conducting research for businesses similar to yours, and their reputation in the industry. You should also consider the types of research methods they use and their ability to provide actionable insights based on their findings.

• Tags example of , importance of quantitative research , quantitative data , quantitative research , quantitative research about environment , quantitative research about environmental science

Team Insights

• Quantitative Research
• Qualitative Research
• Global Panel
• Mystery Shopping
• Survey Audit
• Programming & Hosting
• Translation
• Data Processing & Analytics
• Online Community Building

Request for call back

Are you seeking one-on-one college counseling and/or essay support? Limited spots are now available. Click here to learn more.

50 Best Environmental Science Research Topics

May 31, 2023

Environmental science is a varied discipline that encompasses a variety of subjects, including ecology, atmospheric science, and geology among others. Professionals within this field can pursue many occupations from lab technicians and agricultural engineers to park rangers and environmental lawyers. However, what unites these careers is their focus on how the natural world and the human world interact and impact the surrounding environment. There is also one other significant commonality among environmental science careers: virtually all of them either engage in or rely on research on environmental science topics to ensure their work is accurate and up to date.

In this post, we’ll outline some of the best environmental science research topics to help you explore disciplines within environmental science and kickstart your own research. If you are considering majoring in environmental science or perhaps just need help brainstorming for a research paper, this post will give you a broad sense of timely environmental science research topics.

What makes a research topic good?

Before we dive into specific environmental science research topics, let’s first cover the basics: what qualities make for a viable research topic. Research is the process of collecting information to make discoveries and reach new conclusions. We often think of research as something that occurs in academic or scientific settings. However, everyone engages in informal research in everyday life, from reading product reviews to investigating statistics for admitted students at prospective colleges . While we all conduct research in our day-to-day lives, formal academic research is necessary to advance discoveries and scholarly discourses. Therefore, in this setting, good research hinges on a topic in which there are unanswered questions or ongoing debates. In other words, meaningful research focuses on topics where you can say something new.

However, identifying an interesting research topic is only the first step in the research process. Research topics tend to be broad in scope. Strong research is dependent on developing a specific research question, meaning the query your project will seek to answer. While there are no comprehensive guidelines for research questions, most scholars agree that research questions should be:

1) Specific

Research questions need to clearly identify and define the focus of your research. Without sufficient detail, your research will likely be too broad or imprecise in focus to yield meaningful insights. For example, you might initially be interested in addressing this question: How should governments address the effects of climate change? While that is a worthwhile question to investigate, it’s not clear enough to facilitate meaningful research. What level of government is this question referring to? And what specific effects of global warming will this research focus on? You would need to revise this question to provide a clearer focus for your research. A revised version of this question might look like this: How can state government officials in Florida best mitigate the effects of sea-level rise?

 2) Narrow

Our interest in a given topic often starts quite broad. However, it is difficult to produce meaningful, thorough research on a broad topic. For that reason, it is important that research questions be narrow in scope, focusing on a specific issue or subtopic. For example, one of the more timely environmental science topics is renewable energy. A student who is just learning about this topic might wish to write a research paper on the following question: Which form of renewable energy is best? However, that would be a difficult question to answer in one paper given the various ways in which an energy source could be “best.” Instead, this student might narrow their focus, assessing renewable energy sources through a more specific lens: Which form of renewable energy is best for job creation?

 3) Complex

As we previously discussed, good research leads to new discoveries. These lines of inquiry typically require a complicated and open-ended research question. A question that can be answered with just a “yes” or “no” (or a quick Google search) is likely indicative of a topic in which additional research is unnecessary (i.e. there is no ongoing debate) or a topic that is not well defined. For example, the following question would likely be too simple for academic research: What is environmental justice? You can look up a definition of environmental justice online. You would need to ask a more complex question to sustain a meaningful research project. Instead, you might conduct research on the following query: Which environmental issue(s) disproportionately impact impoverished communities in the Pacific Northwest? This question is narrower and more specific, while also requiring more complex thought and analysis to answer.

4) Debatable

Again, strong research provides new answers and information, which means that they must be situated within topics or discourses where there is ongoing debate. If a research question can only lead to one natural conclusion, that may indicate that it has already been sufficiently addressed in prior research or that the question is leading. For example, Are invasive species bad? is not a very debatable question (the answer is in the term “invasive species”!). A paper that focused on this question would essentially define and provide examples of invasive species (i.e. information that is already well documented). Instead, a researcher might investigate the effects of a specific invasive species. For example: How have Burmese pythons impacted ecosystems in the Everglades, and what mitigation strategies are most effective to reduce Burmese python populations?

Therefore, research topics, including environmental science topics, are those about which there are ample questions yet to be definitively answered. Taking time to develop a thoughtful research question will provide the necessary focus and structure to facilitate meaningful research.

10 Great Environmental Science Research Topics (With Explanations!)

Now that we have a basic understanding of what qualities can make or break a research topic, we can return to our focus on environmental science topics. Although “great” research topics are somewhat subjective, we believe the following topics provide excellent foundations for research due to ongoing debates in these areas, as well as the urgency of the challenges they seek to address.

1) Climate Change Adaptation and Mitigation

Although climate change is now a well-known concept , there is still much to be learned about how humans can best mitigate and adapt to its effects. Mitigation involves reducing the severity of climate change. However, there are a variety of ways mitigation can occur, from switching to electric vehicles to enforcing carbon taxes on corporations that produce the highest carbon emission levels. Many of these environmental science topics intersect with issues of public policy and economics, making them very nuanced and versatile.

In comparison, climate change adaptation considers how humans can adjust to life in an evolving climate where issues such as food insecurity, floods, droughts, and other severe weather events are more frequent. Research on climate change adaptation is particularly fascinating due to the various levels at which it occurs, from federal down to local governments, to help communities anticipate and adjust to the effects of climate change.

Both climate change mitigation and adaptation represent excellent environmental science research topics as there is still much to be learned to address this issue and its varied effects.

2) Renewable Energy

Renewable energy is another fairly mainstream topic in which there is much to learn and research. Although scientists have identified many forms of sustainable energy, such as wind, solar, and hydroelectric power, questions remain about how to best implement these energy sources. How can politicians, world leaders, and communities advance renewable energy through public policy? What impact will renewable energy have on local and national economies? And how can we minimize the environmental impact of renewable energy technologies? While we have identified alternatives to fossil fuels, questions persist about the best way to utilize these technologies, making renewable energy one of the best environmental science topics to research.

3) Conservation

Conservation is a broad topic within environmental science, focusing on issues such as preserving environments and protecting endangered species. However, conservation efforts are more challenging than ever in the face of a growing world population and climate change. In fact, some scientists theorize that we are currently in the middle of a sixth mass extinction event. While these issues might seem dire, we need scientists to conduct research on conservation efforts for specific species, as well as entire ecosystems, to help combat these challenges and preserve the planet’s biodiversity.

4) Deforestation

The Save the Rainforest movement of the 1980s and 90s introduced many people to the issue of deforestation. Today, the problems associated with deforestation, such as reduced biodiversity and soil erosion, are fairly common knowledge. However, these challenges persist due, in part, to construction and agricultural development projects. While we know the effects of deforestation, it is more difficult to identify and implement feasible solutions. This is particularly true in developing countries where deforestation is often more prevalent due to political, environmental, and economic factors. Environmental science research can help reduce deforestation by identifying strategies to help countries sustainably manage their natural resources.

Environmental Science Topics (Continued)

5) urban ecology.

When we think of “the environment,” our brains often conjure up images of majestic mountain ranges and lush green forests. However, less “natural” environments also warrant study: this is where urban ecology comes in. Urban ecology is the study of how organisms interact with one another and their environment in urban settings. Through urban ecology, researchers can address topics such as how greenspaces in cities can reduce air pollution, or how local governments can adopt more effective waste management practices. As one of the newer environmental science topics, urban ecology represents an exciting research area that can help humans live more sustainably.

6) Environmental Justice

While environmental issues such as climate change impact people on a global scale, not all communities are affected equally. For example, wealthy nations tend to contribute more to greenhouse-gas emissions. However, less developed nations are disproportionately bearing the brunt of climate change . Studies within the field of environmental justice seek to understand how issues such as race, national origin, and income impact the degree to which people experience hardships from environmental issues. Researchers in this field not only document these inequities, but also identify ways in which environmental justice can be achieved. As a result, their work helps communities have access to clean, safe environments in which they can thrive.

7) Water Management

Water is, of course, necessary for life, which is why water management is so important within environmental science research topics. Water management research ensures that water resources are appropriately identified and maintained to meet demand. However, climate change has heightened the need for water management research, due to the occurrence of more severe droughts and wildfires. As a result, water management research is necessary to ensure water is clean and accessible.

8) Pollution and Bioremediation

Another impact of the increase in human population and development is heightened air, water, and soil pollution. Environmental scientists study pollutants to understand how they work and where they originate. Through their research, they can identify solutions to help address pollution, such as bioremediation, which is the use of microorganisms to consume and break down pollutants. Collectively, research on pollution and bioremediation helps us restore environments so they are sufficient for human, animal, and plant life.

9) Disease Ecology

While environmental science topics impact the health of humans, we don’t always think of this discipline as intersecting with medicine. But, believe it or not, they can sometimes overlap! Disease ecology examines how ecological processes and interactions impact disease evolution. For example, malaria is a disease that is highly dependent on ecological variables, such as temperature and precipitation. Both of these factors can help or hinder the breeding of mosquitoes and, therefore, the transmission of malaria. The risk of infectious diseases is likely to increase due to climate change , making disease ecology an important research topic.

10) Ecosystems Ecology

If nothing else, the aforementioned topics and their related debates showcase just how interconnected the world is. None of us live in a vacuum: our environment affects us just as we affect it. That makes ecosystems ecology, which examines how ecosystems operate and interact, an evergreen research topic within environmental science.

40 More Environmental Science Research Topics

Still haven’t stumbled upon the right environmental science research topic? The following ideas may help spark some inspiration:

• The effects of agricultural land use on biodiversity and ecosystems.
• The impact of invasive plant species on ecosystems.
• How wildfires and droughts shape ecosystems.
• The role of fire ecology in addressing wildfire threats.
• The impact of coral bleaching on biodiversity.
• Ways to minimize the environmental impact of clean energies.
• The effects of climate change on ocean currents and migration patterns of marine species.

Environmental Justice and Public Policy

• Opportunities to equalize the benefits of greenspaces for impoverished and marginalized communities.
• The impact of natural disasters on human migration patterns.
• The role of national parks and nature reserves in human health.
• How to address inequalities in the impact of air pollution.
• How to prevent and address the looming climate refugee crisis.
• Environmentally and economically sustainable alternatives to deforestation in less developed countries.
• Effects of environmental policies and regulations on impoverished communities.
• The role of pollutants in endocrine disruption.
• The effects of climate change on the emergence of infectious diseases.

AP Environmental Science Research Topics (Continued)

Soil science.

• Effects of climate change on soil erosion.
• The role of land management in maintaining soil health.
• Agricultural effects of salinization in coastal areas.
• The effects of climate change on agriculture.

Urban Ecology

• How road construction impacts biodiversity and ecosystems.
• The effects of urbanization and city planning on water cycles.
• Impacts of noise pollution on human health.
• The role of city planning in reducing light pollution.

Pollution and Bioremediation

• The role of bioremediation in removing “forever” chemicals from the environment.
• Impacts of air pollution on maternal health.
• How to improve plastic recycling processes.
• Individual measures to reduce consumption and creation of microplastics.
• Environmental impacts of and alternatives to fracking.

Environmental Law and Ethics

• Ethical implications of human intervention in the preservation of endangered species.
• The efficacy and impact of single-use plastic laws.
• Effects of religious and cultural values in environmental beliefs.
• The ethics of climate change policy for future generations.
• Ethical implications of international environmental regulations for less developed countries.
• The impact and efficacy of corporate carbon taxes.
• Ethical and environmental implications of fast fashion.
• The ethics and efficacy of green consumerism.
• Impacts of the hospitality and travel industries on pollution and emissions.
• The ethical implications of greenwashing in marketing.
• Effects of “Right to Repair” laws on pollution.

Final Thoughts: Environmental Science Research Topics

Environmental science is a diverse and very important area of study that impacts all aspects of life on Earth. If you’ve found a topic you’d like to pursue, it’s time to hit the books (or online databases)! Begin reading broadly on your chosen topic so you can define a specific research question. If you’re unsure where to begin, contact a research librarian who can connect you with pertinent resources. As you familiarize yourself with the discourse surrounding your topic, consider what questions spring to mind. Those questions may represent gaps around which you can craft a research question.

Interested in conducting academic research? Check out the following resources for information on research opportunities and programs:

• Research Opportunities for High School Students
• Colleges with the Best Undergraduate Research Programs
• College Success
• High School Success

Emily Smith

Emily earned a BA in English and Communication Studies from UNC Chapel Hill and an MA in English from Wake Forest University. While at UNC and Wake Forest, she served as a tutor and graduate assistant in each school’s writing center, where she worked with undergraduate and graduate students from all academic backgrounds. She also worked as an editorial intern for the Wake Forest University Press as well as a visiting lecturer in the Department of English at WFU, and currently works as a writing center director in western North Carolina.

• 2-Year Colleges
• ADHD/LD/Autism/Executive Functioning
• Application Strategies
• Best Colleges by Major
• Best Colleges by State
• Big Picture
• Career & Personality Assessment
• College Essay
• College Search/Knowledge
• Costs & Financial Aid
• Data Visualizations
• Dental School Admissions
• Extracurricular Activities
• Graduate School Admissions
• High Schools
• Homeschool Resources
• Law School Admissions
• Medical School Admissions
• Navigating the Admissions Process
• Online Learning
• Outdoor Adventure
• Private High School Spotlight
• Research Programs
• Summer Program Spotlight
• Summer Programs
• Teacher Tools
• Test Prep Provider Spotlight

“Innovative and invaluable…use this book as your college lifeline.”

— Lynn O'Shaughnessy

Nationally Recognized College Expert

College Planning in Your Inbox

Join our information-packed monthly newsletter.

Breadcrumbs Section. Click here to navigate to respective pages.

Research Methods for Environmental Studies

DOI link for Research Methods for Environmental Studies

Get Citation

The methodological needs of environmental studies are unique in the breadth of research questions that can be posed, calling for a textbook that covers a broad swath of approaches to conducting research with potentially many different kinds of evidence.　　

Written specifically for social science-based research into the environment, this book covers the best-practice research methods most commonly used to study the environment and its connections to societal and economic activities and objectives. Over five key parts, Kanazawa introduces quantitative and qualitative approaches, mixed methods, and the special requirements of interdisciplinary research, emphasizing that methodological practice should be tailored to the specific needs of the project. Within these parts, detailed coverage is provided on key topics including the identification of a research project; spatial analysis; ethnography approaches; interview technique; and ethical issues in environmental research.　

Drawing on a variety of extended examples to encourage problem-based learning and fully addressing the challenges associated with interdisciplinary investigation, this book will be an essential resource for students embarking on courses exploring research methods in environmental studies.

TABLE OF CONTENTS

Chapter | 14  pages, introduction to research methods in environmental studies, chapter | 25  pages, a brief history of knowledge and argumentation, chapter | 20  pages, general research design principles, chapter | 12  pages, general principles of quantitative research, chapter | 22  pages, quantitative data and sampling, chapter | 24  pages, basic quantitative methods and analysis, chapter | 28  pages, more advanced methods of quantitative analysis, chapter | 17  pages, spatial analysis and gis, chapter | 19  pages, general principles of qualitative research, the case study method, the ethnographic approach, chapter | 15  pages, actor-network theory, chapter | 18  pages, environmental discourse analysis, chapter | 13  pages, action research, mixed methods, data collection i, data collection ii, ethical issues in environmental research, chapter | 23  pages, writing a research proposal.

• Privacy Policy
• Terms & Conditions
• Cookie Policy
• Taylor & Francis Online
• Taylor & Francis Group
• Students/Researchers
• Librarians/Institutions

Connect with us

Registered in England & Wales No. 3099067 5 Howick Place | London | SW1P 1WG © 2024 Informa UK Limited

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is Quantitative Research? | Definition, Uses & Methods

What Is Quantitative Research? | Definition, Uses & Methods

Published on June 12, 2020 by Pritha Bhandari . Revised on June 22, 2023.

Quantitative research is the process of collecting and analyzing numerical data. It can be used to find patterns and averages, make predictions, test causal relationships, and generalize results to wider populations.

Quantitative research is the opposite of qualitative research , which involves collecting and analyzing non-numerical data (e.g., text, video, or audio).

Quantitative research is widely used in the natural and social sciences: biology, chemistry, psychology, economics, sociology, marketing, etc.

• What is the demographic makeup of Singapore in 2020?
• How has the average temperature changed globally over the last century?
• Does environmental pollution affect the prevalence of honey bees?
• Does working from home increase productivity for people with long commutes?

Table of contents

Quantitative research methods, quantitative data analysis, advantages of quantitative research, disadvantages of quantitative research, other interesting articles, frequently asked questions about quantitative research.

You can use quantitative research methods for descriptive, correlational or experimental research.

• In descriptive research , you simply seek an overall summary of your study variables.
• In correlational research , you investigate relationships between your study variables.
• In experimental research , you systematically examine whether there is a cause-and-effect relationship between variables.

Correlational and experimental research can both be used to formally test hypotheses , or predictions, using statistics. The results may be generalized to broader populations based on the sampling method used.

To collect quantitative data, you will often need to use operational definitions that translate abstract concepts (e.g., mood) into observable and quantifiable measures (e.g., self-ratings of feelings and energy levels).

Note that quantitative research is at risk for certain research biases , including information bias , omitted variable bias , sampling bias , or selection bias . Be sure that you’re aware of potential biases as you collect and analyze your data to prevent them from impacting your work too much.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Once data is collected, you may need to process it before it can be analyzed. For example, survey and test data may need to be transformed from words to numbers. Then, you can use statistical analysis to answer your research questions .

Descriptive statistics will give you a summary of your data and include measures of averages and variability. You can also use graphs, scatter plots and frequency tables to visualize your data and check for any trends or outliers.

Using inferential statistics , you can make predictions or generalizations based on your data. You can test your hypothesis or use your sample data to estimate the population parameter .

First, you use descriptive statistics to get a summary of the data. You find the mean (average) and the mode (most frequent rating) of procrastination of the two groups, and plot the data to see if there are any outliers.

You can also assess the reliability and validity of your data collection methods to indicate how consistently and accurately your methods actually measured what you wanted them to.

Quantitative research is often used to standardize data collection and generalize findings . Strengths of this approach include:

• Replication

Repeating the study is possible because of standardized data collection protocols and tangible definitions of abstract concepts.

• Direct comparisons of results

The study can be reproduced in other cultural settings, times or with different groups of participants. Results can be compared statistically.

• Large samples

Data from large samples can be processed and analyzed using reliable and consistent procedures through quantitative data analysis.

• Hypothesis testing

Using formalized and established hypothesis testing procedures means that you have to carefully consider and report your research variables, predictions, data collection and testing methods before coming to a conclusion.

Despite the benefits of quantitative research, it is sometimes inadequate in explaining complex research topics. Its limitations include:

• Superficiality

Using precise and restrictive operational definitions may inadequately represent complex concepts. For example, the concept of mood may be represented with just a number in quantitative research, but explained with elaboration in qualitative research.

• Narrow focus

Predetermined variables and measurement procedures can mean that you ignore other relevant observations.

• Structural bias

Despite standardized procedures, structural biases can still affect quantitative research. Missing data , imprecise measurements or inappropriate sampling methods are biases that can lead to the wrong conclusions.

• Lack of context

Quantitative research often uses unnatural settings like laboratories or fails to consider historical and cultural contexts that may affect data collection and results.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Chi square goodness of fit test
• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Mixed methods research
• Non-probability sampling
• Inclusion and exclusion criteria

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to systematically measure variables and test hypotheses . Qualitative methods allow you to explore concepts and experiences in more detail.

In mixed methods research , you use both qualitative and quantitative data collection and analysis methods to answer your research question .

Data collection is the systematic process by which observations or measurements are gathered in research. It is used in many different contexts by academics, governments, businesses, and other organizations.

Operationalization means turning abstract conceptual ideas into measurable observations.

For example, the concept of social anxiety isn’t directly observable, but it can be operationally defined in terms of self-rating scores, behavioral avoidance of crowded places, or physical anxiety symptoms in social situations.

Before collecting data , it’s important to consider how you will operationalize the variables that you want to measure.

Reliability and validity are both about how well a method measures something:

• Reliability refers to the  consistency of a measure (whether the results can be reproduced under the same conditions).
• Validity   refers to the  accuracy of a measure (whether the results really do represent what they are supposed to measure).

If you are doing experimental research, you also have to consider the internal and external validity of your experiment.

Hypothesis testing is a formal procedure for investigating our ideas about the world using statistics. It is used by scientists to test specific predictions, called hypotheses , by calculating how likely it is that a pattern or relationship between variables could have arisen by chance.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Bhandari, P. (2023, June 22). What Is Quantitative Research? | Definition, Uses & Methods. Scribbr. Retrieved August 29, 2024, from https://www.scribbr.com/methodology/quantitative-research/

Is this article helpful?

Pritha Bhandari

Other students also liked, descriptive statistics | definitions, types, examples, inferential statistics | an easy introduction & examples, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”