How we reviewed the evidence for our story on marine protected areas

 

~ By Amy Fensome

In order to establish whether marine protected areas (MPAs) are an effective conservation tool, we primarily looked for studies that compared conditions (surveyed or perceived) within an MPA with unprotected areas outside of the MPA, or that compared conditions before and after MPA establishment. We also included studies that compared the current state of reserves (for example, habitat representativeness) to standards outlined in international agreements (for example, the Convention on Biological Diversity).

We began building our evidence base using search protocols recommended for systematic reviews by Pullin & Stewart 2006.  First, to find relevant research, we used specific search terms on the literature search engine Google Scholar.

Our search terms included synonyms for the focal conservation strategy (“Marine Protected Area” OR “MPA” OR “Marine Nature Reserve” OR “Marine Conservation Zone” OR “Marine” AND “Protected area” OR “National Park” OR “reserve” OR “Special Area of Conservation” OR “Special Areas of Conservation” OR “Special Protection Area” OR “Nature Conservation Zone” OR “conservation zone”) as well as terms related to environmental (“Biodiversity” OR “species” OR “population” AND “abundance” OR “richness” OR “diversity” OR “assemblage” OR “threatened” OR “red list” OR “rare” OR “fish stocks” OR “habitat”), social (“community” OR “indigenous” OR “local” OR “social” OR “government” AND “participation” OR “wellbeing” OR “health” OR “conflict” OR “cohesion” OR “benefit” OR “education” OR “collaboration”) and economic (“wealth” OR “jobs” OR “economy” OR “livelihood” OR “Direct economic benefit” OR “income” OR “financial” OR “opportunity costs”).

This search returned ~300,000 results, sorted by relevance. We then went through the 1,000 most relevant Google Scholar search results and found ~200 potentially relevant article titles.

Next, we read the abstracts of these potentially relevant articles, and identified studies that 1) measured one or more specific outcomes connected with the marine protected areas; 2) were peer-reviewed; and 3) were published in 2000 or after. We considered outcomes that fell into three broad themes: environmental, social and economic.

We included meta-analyses and systematic reviews if they calculated overall effect size, and highlighted them as such (see Types of Evidence here), and we did not further include the individual studies on which the reviews were based. We excluded purely theoretical, opinion, and modeling studies.

Following consultation with experts in the field, an additional three studies were included in the final review, and these are highlighted by an asterisk in the reference list.

From each study, we extracted the following information:

1) first author; 2) study title; 3) year of publication; 4) other studies covering the same case; 5) whether the study is peer-reviewed; 6) brief description of the methodology (for example, whether the researchers use remote sensing or interviews); 7) type of evidence; 8) continent; 9) country; 10) conservation strategy (i.e. Marine Protected Area); 11) type of management that the impact is being compared to (e.g. no management); 12) thematic group of the variable that is assessed (environmental, social, or economic); 13) variable that is being compared (for example, diversity, community wellbeing, harvestable species); 14) outcome (positive, neutral, negative); 15) detailed outcome (verbal description of the main finding).

The literature review was carried out by a primary reviewer, and at least 20 percent of the studies were checked by a second reviewer.

Caveats

  • The purpose of this review is to display a sample of the existing evidence, rather than to draw any definite conclusions on individual variables. Not all the individual comparisons used are independent, as some studies contributed multiple comparisons.
  • The individual studies were carried out with different degrees of rigor and this should be kept in mind when interpreting the results.
  • We only reviewed peer-reviewed literature written in English and we acknowledge that an important body of evidence might exist in other languages.
  • Our literature review is not exhaustive. It was beyond the scope of our work to execute an exhaustive, systematic review.
  • Further, when extracting information from studies, we necessarily introduce a certain amount of bias and error. For example, while we might be able to unequivocally say which country the study was performed in, there may be several possible interpretations of the results we find (positive, neutral, or negative), depending on how one defines conservation success. Wherever possible, we attempt to provide additional information for the readers to judge whether their interpretation agrees with ours. We have provided links to every original study for the readers to refer to.