Monitoring and Evaluation of Supplementation Projects

Introduction

In this report, the Independent Scientific Review Panel (ISRP) and Independent Scientific Advisory Board (ISAB) identify for the Council, Council staff, and regional managers, the critical uncertainties of supplementation, outline monitoring data needed to evaluate supplementation, and provide options for coordinating projects throughout the basin to produce an experimental design sufficient to resolve these uncertainties. The genesis for the report is conversations with the Council when the ISRP Retrospective Report (ISRP 2005) was presented in Spokane on September 14, 2005, and questions that have arisen in recent three-step reviews of Master Plans that include supplementation projects. The ISRP and ISAB intend that this information will serve to assist in evaluating the effectiveness of this Fish and Wildlife Program strategy at the subbasin (project), provincial, and basin level.

Background and Context for this Report

The 1996 amendment to the Northwest Power Act charges the ISRP with providing scientific review of projects funded through the Northwest Power and Conservation Council Fish and Wildlife Program (FWP). This charge includes scientific review for artificial production (AP or “hatchery”) projects. New artificial production initiatives are evaluated through a formal three-step review process, first implemented in 1999.

The three-step review includes an evaluation of whether a Master Plan is consistent with the ten Artificial Production Review policies (APR; NWPPC 99-15) adopted by Council into the 2000 FWP. A recurring ISRP criticism in their three-step reviews of hatchery projects is a failure to adequately address the second APR policy: artificial production must be implemented within an adaptive management framework including (1) a rigorous experimental design to evaluate the risks and benefits of the proposed project and address the associated scientific uncertainties, and (2) a set of decision rules for adjusting management in response to the experimental results. Furthermore, the 2000 FWP explicitly directs an experimental approach to all projects (see page 29, Final 2000 Fish and Wildlife Program):

In recognition of the risk and uncertainty associated with artificial production, each artificial production activity must be approached experimentally with a plan detailing the purpose and method of operation, the relationship to other elements of the subbasin plan, including associated habitat and other projects within the subbasin plan, specific measurable objectives for the activity, and a regular cycle of evaluation and reporting of results. This approach will allow the region to address the remaining uncertainties on a case-by-case basis and quickly make adjustments in artificial production activities where warranted.

Fulfilling this directive requires specifying quantifiable objectives for each project, identifying the pertinent uncertainties, and designing an effectiveness monitoring and risk monitoring plan to yield data that will permit an evaluation of whether the project is achieving its objectives without causing harm and is addressing the uncertainties.

Consistent with the Council’s program language, the ISRP has been especially concerned about monitoring and evaluation of supplementation, an artificial production strategy where fish of hatchery origin are placed in streams to increase the rate of recovery of naturally spawning populations. In the judgment of the ISRP and the ISAB, the uncertainty concerning both the benefits and the risks of supplementation is sufficiently great to put the merit of supplementation into question as a recovery strategy. Given such uncertainties (reviewed in the APR and the ISAB’s Review of Salmon and Steelhead Supplementation; ISAB 2003-3), the ISRP concludes that some supplementation projects likely provide no actual conservation benefit and some supplementation projects may also pose a sizeable obstacle to recovery of ESA-listed stocks. Thus, when a decision is made to go forward with a supplementation project, to address the ISRP’s concerns and the Council’s program, a strong and lasting commitment would be warranted for the monitoring, provided that the monitoring adequately addresses and resolves the critical uncertainties.

In fact, resolving the uncertainties about supplementation will require precise experimental and sampling designs and a clear scientific understanding of complex ecological and genetic concepts. This report provides ISRP and ISAB recommendations to Council and Council staff, and regional managers on the critical uncertainties, data (performance metrics) needed to monitor and evaluate supplementation projects, and options for experimental designs to resolve these uncertainties. These recommendations are drawn largely from the ISAB Review of Salmon and Steelhead Supplementation (ISAB 2003-3), the ISRP Retrospective Report (ISRP 2005), and Goodman (2004).

Supplementation versus Conventional Hatchery Production

Supplementation programs lie at one end of an artificial production continuum, with conventional production hatchery programs at the other.

Conventional production hatcheries are operated with the intent to release juvenile fish to provide harvest when they return as adults (i.e., put-take or put-grow-take; see Utter and Epifanio (2002) for further description and discussion of various production and release models). This goal is often implemented through development of a separate hatchery stock, drawing most or all of the hatchery broodstock in each generation from returning hatchery fish.

Supplementation programs, by contrast, are artificial propagation programs operated with the intent that a significant fraction of the released fish will escape harvest to successfully reproduce in the wild. Having these hatchery-origin fish reproduce naturally is to provide a “demographic boost” that eventually leads to increased abundance of natural-origin adults in the salmon or steelhead population. To increase the chances that the released fish will be ecologically adapted to reproduce in the basin where they are released, supplementation projects often draw a large fraction, or all, of the hatchery broodstock in each generation from fish that originated in the wild. The result is that the supplementation strategy creates an “integrated” population where hatchery spawning and wild spawning are both represented in all the pedigrees.

The hope is that in the short-term, the numerical or demographic contribution of the hatchery fish to reproduction will more than offset any genetic or ecological negative impacts of hatchery releases on wild stocks. It is also hoped that the longer-term result of supplementation will be to accelerate rebuilding of depressed populations and the recovery of endangered populations to the point where they no longer need supplementation to assure survival.

Formally, supplementation was defined by the Regional Assessment of Supplementation Project (RASP 1992):

Supplementation is the use of artificial propagation in an attempt to maintain or increase natural production, while maintaining the long-term fitness of the target population and keeping the ecological and genetic impacts on non-target populations within specified biological limits.

The ISAB (2003) recommended using this as the operational definition within the Columbia River Basin, and most project sponsors have adopted it.

From this a more precise definition of the primary objective of supplementation is the conservation of the target population, which is further defined as the natural-origin individuals originating from within a particular watershed.

Management Control of Supplementation

The operation of a supplementation program influences its effect on the original wild population that is absorbed into the integrated program. Scientific advisory committees have recommended specific constraints on supplementation operations, including: the use of local broodstocks; limits on the fraction of wild populations that are collected for use as broodstock; limits on the proportion of hatchery-origin adults that are allowed to mix with natural-origin adults on spawning grounds; and limits on the use of hatchery-origin adults in hatchery spawning (ISAB 2003, HSRG 2005). These constraints are intended to result in an integrated program that poses lower levels of genetic and demographic risk compared to less restrained supplementation programs while retaining some prospect of a net conservation benefit. These recommendations have not yet been adopted as required policy and the degree of human intervention in the life cycle of Columbia River Basin salmonids by supplementation programs varies greatly, from minor additions to a population with substantial natural production to captive broodstock/captive rearing programs with nearly complete replacement of natural reproduction and rearing.

Supplementation Risks and Uncertainties

The critical uncertainties are whether supplementation provides a demographic increase in natural production (the potential benefit) and whether supplementation leads to decreased natural-spawning fitness (the potential harm) in the integrated population. Supplementation entails demographic, genetic (fitness), ecological, and disease risks.

Demographic Risk

Increasing the abundance of salmon using supplementation is uncertain. The immediate net demographic benefit or harm to population abundance from supplementation depends on the intrinsic biological parameters of the stock (growth rates, reproductive rates, and survival rates) in its natural environment and in the hatchery, and it also depends on management control of the program (broodstock removal rates, proportion of hatchery-origin fish on the natural spawning grounds, size of smolt release, compared to the size of the natural population). The long-term fitness consequence of supplementation arises from different natural selection in the hatchery and natural environments.

A necessary condition for realizing a demographic boost is achieving an increase in the combined hatchery-origin plus natural-origin adult population. The basis for anticipating this desired effect is that the survival advantage for early life stages in the hatchery environment is expected to generate a number of hatchery-origin adults returning from the ocean than is larger than would have resulted from natural spawning by the same number of parents. Then, some of these progeny of hatchery spawning can be allowed to augment the natural-origin population on the spawning grounds for a net increase in abundance in that generation.

The demographic risk is that the number of natural-origin progeny lost due to the removal of natural-origin adults for hatchery broodstock is not replaced by the survival and reproduction of the hatchery-origin smolts that return as adults and reproduce naturally. Even the desired increase in numbers during supplementation is not guaranteed. Increasing the number of adults on the spawning ground (whether of natural spawning or hatchery origin) requires that the smolt-to-adult survival rate (in the wild) of the released hatchery-origin juveniles not be so much smaller than that of the natural-origin juveniles, so as to nullify the hatchery survival advantage in the egg-to-smolt stage. In any case, merely increasing the numbers of adults on the spawning ground may not increase the number of adults in the next generation. If the hatchery-spawned fish are less effective at surviving and reproducing in nature (judged on an adult to adult basis) than the naturally spawned fish, there could be a net demographic loss from supplementation. At this time empirical evidence has not demonstrated either demographic loss from supplementation or evidence that hatchery fish have contributed to natural fish recruitment.

Genetic Risk

The primary genetic risk is that matings in the wild involving one or more hatchery-origin parents result in the production of offspring with reduced fitness. The effect of supplementation on long-term fitness of the target population depends on the extent of disruptive selection between the hatchery and natural environment. Disruptive selection occurs when traits that prove advantageous in the life cycle that begins with hatchery spawning are disadvantageous in the life cycle that begins with natural spawning, and vice versa. The quantitative extent of disruptive selection is modulated by management controls on broodstock withdrawal rates, the fraction of the naturally spawning population consisting of hatchery-origin adults, the fraction of the hatchery population consisting of hatchery-origin adults, and harvest selectivity. These genetic effects could persist for some number of generations after supplementation is terminated. Therefore, even if supplementation increases the numbers of naturally spawning fish of natural origin, there will be a net conservation loss if natural spawning fitness has been eroded in the process.

Ecological Risk

The primary ecological risk is that hatchery-origin fish compete with or may even prey upon natural-origin fish from the target population or other species. A secondary ecological risk is that the presence of the hatchery fish may increase the predator base by providing a more consistent and abundant food supply than might occur in their absence.

Ecosystem Limiting Factors

Finally, increasing numbers through supplementation without addressing the factors that caused the population to decline in the first place, will likely result in a decline in the target population upon termination of supplementation because ecological function has not been restored. If the supplementation diminishes the natural spawning fitness in the population, the population will decline more precipitously than before, upon termination of supplementation. On the other hand, if supplementation has little effect on the fitness of the naturally spawning population, the increased number of fish through hatchery rearing might provide the demographic conditions necessary to assist survival during a catastrophic event or until better conditions for survival of wild fish are realized or developed.

All the mechanisms involved in the potential demographic benefit and the potential ecological and genetic harm are known to be real, but their relative magnitudes have not been quantified well enough for a secure prediction of the balance between benefit and harm. Uncertainty exists regarding both the potential for achieving the demographic objectives and the assurance of not harming the fitness of the target population. For these reasons supplementation is considered experimental. Consequently, monitoring data (performance metrics) and evaluation are needed to determine the pertinent quantities and provide information for adjusting the deployment of supplementation to keep the harm within bounds and verify that costs are warranted by the actual benefits.

With all the uncertainty and risk attached to supplementation as a measure to mitigate for impaired habitat, the direct alternative strategy of restoring the habitat deserves careful consideration. If habitat can be restored in time, the consequences for the natural population do not involve major uncertainties or risks. The element of time arises because of the possibility that the decline of the population may reach a critical level before the habitat restoration takes effect.

Evaluating Supplementation

The RASP definition of supplementation produces two standards for the use of supplementation programs. First, intervention should be required to conserve a population. According to this standard, supplementation programs should be directed toward areas where natural production alone results in a declining or barely stable population under present or anticipated near-term habitat conditions. In more technical terms, the average annual population growth rate (lambda) of the naturally producing population is less than 1. Second, supplementation should not reduce “the long-term fitness of the target population” and should keep “the ecological and genetic impacts on non-target populations within specified limits.”

Clear data (performance indicators) are required to evaluate whether or not these standards are being met and to provide a technically sound basis for management decisions. From the RASP definition performance indicators are needed in three areas, at a minimum:

1. target population abundance and productivity, and capacity;

2. target population long-term fitness, and;

3. non-target population impacts.

For evaluation of supplementation to be informative, the correct parameters need to be monitored, using an adequate experimental design, and sample sizes large enough to allow detection of the true effects of supplementation.

The recommended performance metrics (response variables) and evaluation designs for fully monitored sites that would comprise a core evaluation of supplementation are described below:

1. Target population abundance, productivity, and capacity.

Abundance and productivity of natural-origin adults in supplemented streams during supplementation, needs to be compared to those of natural-origin adults in unsupplemented reference streams. We define productivity as the maximum smolt or adult recruits per spawner produced at low spawner density, and capacity as the maximum recruits at the asymptote.

Abundance and productivity estimates of natural-origin fish are required because the natural fish are the target population that supplementation is intended to conserve. Measuring total abundance (hatchery-origin plus natural-origin adults) in supplemented streams is not adequate, because hatchery-origin adults could be replacing natural-origin fish and hatchery-origin fish may not be reproducing well in the wild.

The appropriate evaluation of target population abundance is the trend in abundance of natural-origin adults in a supplemented stream contrasted with the trend in abundance of natural-origin adults in unsupplemented reference locations. The very large temporal variation in salmon survival would make evaluation using comparison of abundance and productivity of the target population before and after supplementation a poor design choice because the true effects of supplementation would be confounded with naturally occurring temporal variation in survival. The evaluation must estimate the numbers of adult returns by origin (natural and hatchery) and age. Details of an appropriate data collection are provided in Chapter 5 of the ISAB Supplementation Report (ISAB 2003).

To determine whether the natural-origin juveniles lost due to the removal of natural-origin adults for hatchery production are replaced by the survival and reproduction of the hatchery-origin smolts that return as adults and reproduce naturally, requires evaluation of target population natural spawning replacement rate. This performance metric requires that the monitoring in the supplemented stream must reliably distinguish, in the wild, a) returning adults that are the progeny of natural spawning by fish that were themselves spawned in the wild, versus b) returning adults that are the progeny of natural spawning of fish that were products of the hatchery, versus c) returning adults that were progeny of the two possible crosses. In practice this will require rigorous and extensive marking, most likely by genetic sampling and pedigree analysis, but perhaps by physical tagging, and perhaps involving control of which fish are allowed to return to spawn in which tributary. This design will carry a substantial implementation effort, but it is the only way to answer the crucial questions about effectiveness of supplementation.

2. Target population long-term fitness

The density corrected replacement rate of naturally spawning natural-origin adults from supplemented streams needs to be compared to those of natural-origin adults from unsupplemented reference streams.

The appropriate test of the change in natural spawning fitness owing to supplementation is calculated as the difference between the measured natural spawning fitness in a population that has undergone several generations of supplementation and the measured natural spawning fitness of a population of the same stock that has not been supplemented (Goodman 2004). A basic measure of natural spawning fitness is the “density corrected” female replacement rate (number of females returning to spawn that are the offspring of a female who spawned in the previous generation). Measuring and contrasting life-history attributes (e.g., age and size at maturation, fecundity, etc) or relative reproductive success of hatchery-origin adults and natural-origin adults returning to, and spawning in, a supplementation stream will not provide an evaluation of supplementation’s effect on natural spawning fitness. The density corrected replacement rate in the supplemented and reference streams should be estimated over the same years to control for temporal variation in survival.

3. Non-target population impacts

Abundance or productivity (replacement rates) of non-target species (e.g., bull trout) in the target supplemented areas, needs to be compared with abundance or productivity of non-target species in unsupplemented reference streams.

Once a set of standards has been established for these performance indicators, measuring progress toward achieving those standards would then provide a mechanism to evaluate supplementation.

Challenges and Recommendations for the Evaluation Design

In ideal circumstances the indicators for target population abundance, productivity, and capacity; target population long-term fitness; and non-target population ecological effects would be measured in every stream being supplemented. Logistically this might be daunting, and a sufficient evaluation of the efficacy of supplementation might be achieved without complete evaluations in each stream. The number of locations that need to be fully monitored needs to be determined as an overall Columbia River basin experiment, and this has not been done.

The suggested evaluation of target population abundance, productivity, and capacity involves contrasting trends in treatment and reference locations. Treatment and reference locations will undoubtedly differ from each other beyond the supplementation treatment. With a sufficient number of treatment and reference locations, it may be possible to account for the effect of supplementation. There are several possible designs for a large-scale, basin-wide experiment to assess supplementation. Treatment-control, before-after treatment control, or within system detailed life-stage monitoring and genetic sampling are options to be considered.

Evaluating long-term fitness effects of supplementation by comparison of the female replacement rate from supplemented and unsupplemented populations also poses practical difficulties to execute and interpret. One option would be to compare the productivity of these fish in the treatment and reference locations. This is problematic because of uncontrolled differences between the streams and a need to standardize density in both streams. A second option would be to compare the productivity of the supplemented and unsupplemented populations in a common experimental setting. Various considerations for implementing this design are outlined in Goodman (2004).

As a practical matter, it is not an ISRP or ISAB responsibility to make the final decision among the possible experimental designs that could be employed to evaluate supplementation in the Columbia River basin. Rather, it is our responsibility to identify that one does not now exist – and that one is needed in order to evaluate supplementation effects at the project and Columbia River basin scales. This situation was discussed at a joint ISRP and ISAB meeting with the NOAA Fisheries and CRITFC Ex-Officio board members. The suggested resolution from those discussions was that NOAA Fisheries, CRITFC, and perhaps the Council jointly organize a very small workshop/work group of invited attendees that includes the sponsors of supplementation projects and biostatisticians to establish a basin level evaluation. This workshop/work group is not envisioned to be a public forum to debate supplementation, but an implementation group to execute a cooperative management experiment. This workshop approach may be useful towards selection of designs within the Columbia Basin that utilize data on population demographics and recruitment to assess the effectiveness and impact of supplementation.

As a contribution to preparing for such a workshop, the ISRP and ISAB provide the following recommendations:

Determine which projects to include in the basin level evaluation. Based on projects that were reviewed by the ISAB (2003) and the ISRP in various provincial and three-step reviews, likely candidates to contribute to this basin level evaluation include Hood River steelhead, Yakima River spring-run Chinook, Umatilla spring-run Chinook and summer-run steelhead, Tucannon River spring-run Chinook, Johnson Creek summer-run Chinook, Chiwawa River (Wenatchee River) spring-run Chinook, Grand Ronde and Imnaha River spring-run Chinook and summer steelhead, and spring/summer-run Chinook in the Lemhi and Pahsimeroi Rivers in Idaho. This list is not intended to be exhaustive, there are likely others known to the managers and scientists in the basin.

Establish defined protocols for select projects. For effective evaluation, supplementation in individual projects must be implemented with a disciplined protocol, following defined management rules for the broodstock collection rates, proportion of hatchery fish spawning naturally, and the proportion of hatchery fish used in serial hatchery production. Each evaluation would then test supplementation under these specific management protocols. Goodman (2005) has shown that the strength of the forces that could give rise to harmful genetic effects is directly related to these management parameters, so varying any of them within a single “treatment” system will reduce the interpretability of the experiment. Comparisons across the individual supplementation projects that are employing different protocols should reveal if there are critical thresholds for these management parameters, such that below the threshold the risk is slight, whereas above the threshold the risk is considerable. Such a result would provide powerful guidance for future deployment of supplementation.

Establish more reference locations. A fundamental requirement for the experimental design is the presence of a number of unsupplemented reference sites that can serve as a valid basis for comparison to the supplemented “treatment.” To facilitate evaluation of the supplementation strategy, effort is needed to establish more reference streams. These are locations where the abundance of adults can be reasonably determined and hatchery-origin adults can be reasonably excluded, so a wild line can be maintained. The natural spawning grounds of the reference streams should be near the natural spawning grounds of the supplemented population and should be chosen for similar habitat characteristics. For any new projects, potential pairs of supplementation and reference locations should be identified and the ones to be supplemented should be randomly selected. These reference locations are likely to serve monitoring and evaluation needs other than supplementation.

Establishing an evaluation of fitness should be a high priority. Several supplementation programs’ experimental designs have been improved through iterative review exchanges between project sponsors and the ISRP. The projects above are likely collecting the necessary data to assess the abundance, productivity, and capacity of the populations in the supplemented streams. With sufficient reference locations, a reasonable assessment of the near-term demography of a supplemented stream is likely. At this time the ISRP and ISAB are not aware of a suitable evaluation of the effects of supplementation on natural spawning fitness of the target population. Addressing an evaluation of the relative fitness of salmon from supplemented populations compared to unsupplemented populations should be a high priority.

Monitoring and Evaluation Plans in Three Step Reviews

All supplementation projects need some data collection for evaluation, risk assessment, and adaptive management. The ISRP and ISAB recommend that Master Plans submitted for three-step reviews identify what types of assessment are logistically practicable for each population proposed to be supplemented. Each project should identify measures of success and failure, and what is needed to collect the data to make the measurement. Projects that can only collect limited data because the populations are not easily sampled should be required to indicate how the basin level evaluation will substitute for a more thorough assessment, and the decision tree that leads to program adjustments. If it would be helpful, and requested, the ISRP can follow this report with a short review/implementation checklist of monitoring and evaluation needs for supplementation projects to be included in three-step reviews.

Concluding Comment

Monitoring and evaluation of supplementation projects is critically important. For the monitoring to be effective, a very rigorous design is needed, and the scale and logistics of implementation will carry costs that are significant. The scientific issues underlying the definitions of performance metrics and the necessary controls in the design are genuinely complicated. Some of the scientific tools for measuring performance are new, and involve a level of knowledge of population and molecular genetics which until recently has not been part of the standard fisheries curriculum.

The consequences of not conducting these studies and continuing to assume no deleterious impacts from supplementation, and being wrong, are much greater than short-term changes in salmon abundance. The natural populations that may be lost if supplementation actually decreases their fitness are irreplaceable. On the other hand, if supplementation proves an aid to natural population during distress, further application may be warranted. Both outcomes remain uncertain without adequate monitoring and evaluation, which will likewise guide best management practice and cost effectiveness.

Literature Cited

Goodman, D. 2004. Salmon supplementation: demography, evolution, and risk assessment. Pages 217 – 232. in M. J. Nickum, P. M. Mazik, J. G. Nickum, and D. D. MacKinlay, editors. Propagated fish in resource management. American Fisheries Society, Symposium 44, American Fisheries Society, Bethesda, Maryland.

Goodman, D. 2005. Selection equilibrium for hatchery and wild spawning fitness in integrated breeding programs. CJFAS 62:1-16.

Hatchery Scientific Review Group (HSRG). 2005. Hatchery Reform in Washington State: Principles and Emerging Issues. Fisheries 30(6):11-23.

Independent Scientific Advisory Board (ISAB) 2003. Review of salmon and steelhead supplementation. ISAB 2003-3. Northwest Power and Conservation Council, Portland, Oregon.

Independent Scientific Review Panel (ISRP) 2005. Retrospective Report 1997 - 2005. ISRP 2005-14. Northwest Power and Conservation Council, Portland, Oregon.

Regional Assessment of Supplementation Project (RASP) 1992. Supplementation in the Columbia basin. Bonneville Power Administration. Contract DE-AC06-75L0183. Portland, Oregon.

Utter, F. and J. Epifanio. 2002. Marine aquaculture: genetic pitfalls and potentialities. Reviews in Fish Biology and Fisheries 11:59-77.

________________________________________

w:\em\ww\isrp projects and reports\1 final isrp reports\isrp&isab 2005-15 supplementation monitoring.doc

Contributing ISRP and ISAB Members