As the authors say, evaluation of our ecosystems plans, goals, objectives, and actual implementation can be painful, and is often avoided despite the obvious value.

I’ve said in class (and the previous set of notes) that we all resist evaluation, we prof’s worry about FCQs and outcomes assessments. But any significant environmental or ecosystems action is a public action that entails accountability, so evaluation is the right thing to do.

Evaluation is a part of the Strategic Planning Process.

It should look at:

Policy: the general guidelines and goals for EM are set by policy, like the Endangered Species Act and the rules and regulations that federal agencies adopt to implement the act. EM efforts might has less formalized policies, which would be something more like precepts or principles: like "do no harm" or make no one person worse off by implementing the plan.

The program: This is the programmatic approach: maybe having to do with land acquisition (you might institute conservation easement program for private land). A significant EM effort might have several programs, comprising bundles of similar strategies.

Projects: This is the implementation action, and needs to be evaluated to assess if it really furthers the goals and objectives of the program, and whether it was implemented in an appropriate manner.

Formative:

This is something like the Assessment/Inventory in the strategic thinking model. Do we need a program? It includes:

Assessing the logic and rationality of the proposed program (a cost-benefit study, or some form of technical review by outsider experts might help)

Assessing likely outcomes, potential for success (modeling can help here)

Formative evaluation is prospective, usually qualitative, and normative (e.g., based on, and compared to, ideals, models, and theoretical notions of what should be).

Tracks implementation to make sure it is done correctly.

It might look at: budget, actions, targets being met or not; conforms with the plan and regulations and legislative requirements.

Is a chance to adjust the program

This is outcomes assessment. How effective was the program? It is mostly a retrospective look based on actual data, especially before and after data, but can also include important qualitative outcomes (e.g., did the stakeholders feel properly involved?)

The indicators of effectiveness should be chosen in advance.

The summative step is often hardest in any non-trivial ecosystems management effort, and avoided, or descends into squables about what is considered good or bad, a success or not.