This section will help you to:
- Refine your evaluation purpose and objectives.
- Reflect on what you are trying to evaluate and the logic behind this.
- Draw out, understand and re-evaluate your assumptions.
What is the purpose of my evaluation?
To get the most out of the process it is vital to understand the reasons for the evaluation, what you hope to get out of it and what type of evaluation you are planning.
- More information on types of evaluation.
Questions to consider:
- What is the purpose of your evaluation and what would you like to learn?
- How might you maximise the synergies between these purposes or manage conflicting purposes?
- What trade-offs might you have to make and can these be justified?
- Have you defined the learning objectives of your evaluation – who should be learning what and how?
- What are the benefits of formative and summative evaluation approaches for your purposes? See Types of evaluation.
- What balance do you need to strike between these two types of evaluation? What can you learn from other types of evaluation?
The purpose of your evaluation will relate closely to your objectives; why you do something is likely to inform what you want to achieve.
It is important that you are clear about the purpose and objectives of your evaluation before designing your approach, and maintain a focus on these throughout the work.
The reasons for your evaluation may be complementary or conflicting – understanding these synergies and tensions at an early stage will help you to develop a balanced and effective approach.
Some of the most common reasons for evaluations are explored below:
- To evaluate effectiveness
- To assess efficiency
- To understand equity
- To provide accountability
- To assess outcomes
- To improve learning
- To improve future activities or interventions
- To compare with other similar activities or interventions
Evaluators often seek to assess whether or not an intervention has been effective in achieving its original aims. This requires that the objectives (outputs and outcomes) are clearly specified at the outset. Any evaluation of effectiveness should consider whether these objectives have been achieved but also whether what was intended was actually appropriate or needed.
An assessment of the efficiency of an intervention will look at the costs, benefits and risks involved, and the timeliness of actions. Economic evaluation techniques may be used to calculate costs and benefits in financial terms. This is important in adaptation where additional investments need to be assessed and justified.
The impacts of climate change will not occur evenly across time or geographical area, and will affect individuals and communities differently according to their vulnerability to those impacts. Equity and justice are therefore important factors to consider in evaluating your adaptation interventions. This may raise questions about:
- What are the effects of the project on different social groups and their ability to engage in and benefit from the intervention?
- Has the intervention has targeted the right people?
- Are certain groups exposed to disproportionate risks, bear additional costs or be hindered by the intervention?
An evaluation may be required to ensure that commitments, expectations, and standards are met, particularly where public money has been invested and evidence of achievements and challenges are needed. Accountability may overlap with efficiency considerations, for example to account for the costs and benefits of an investment.
Your evaluation may help you to understand the outcomes of an intervention and its impacts. It can be challenging to disentangle outcomes attributable to the intervention from those resulting from other variables – the long time periods involved in climate change impacts add to the complexity. In adaptation, the avoidance of negative consequences can be a successful outcome, yet can be hard to measure and assess, precisely because they have been avoided! The assessment of outcomes tends to be associated with summative evaluation approaches and the use of impact indicators.
It is important to improve our understanding of adaptation interventions, what works and why. However, the creativity and time invested in learning can vary considerably between evaluations. This can be the result of:
- A tension between learning (what happened and why?) and accountability (have we done what we said we would?).
- The limitations placed upon monitoring and evaluation processes.
Recognising these tensions and identifying who should be learning what and how can help you to achieve your learning objectives.
While some information may be commercially sensitive, sharing knowledge and experience of adaptation makes sound business sense, helping to make future adaptation interventions more efficient and cost effective.
Your evaluation may be designed to help strengthen future activities and interventions, suggesting a strong focus on learning and, where appropriate, use of the formative methodology. Given we are at an early stage in adapting to climate change this should be a strong consideration for all evaluation processes.
You may undertake a comparative evaluation to understand how the impact of an adaptation intervention varies in different locations or communities, or to compare the implementation and outputs of one adaptation option with those of another.
- A formative evaluation focuses on ways of improving a project or programme while it is still happening, and is often associated with forecast and mid-term evaluations.
- A summative evaluation judges the overall effectiveness of an intervention, usually after a project or programme has been.
The forward looking nature of formative evaluations provides a rich source of techniques to inform future adaptation strategies, while for accountability purposes, summative approaches provide a valuable evidence base by reflecting on past experience.
The type of evaluation is linked to the purposes of the evaluation. For example:
If efficiency and value for money is a key factor, then an economic evaluation with a focus on the quantification of costs and benefits of an adaptation intervention may be the best approach.
However, such an approach may be less effective at considering who bears the costs or experiences the benefits, or what the non-economic costs and benefits might be, unless this is factored into your evaluation design. Designing an evaluation usually results in trade-offs being made between competing and, at times, conflicting purposes and objectives.