It is important to recognise any limitations on your evaluation at the outset.
Questions to consider:
- List the existing limitations of your evaluation. Does the proposed evaluation appear proportionate?
- Consider which limitations are likely to have the greatest adverse effect in understanding what has worked and why? Can these be overcome (e.g. by adjusting the timescale of the evaluation) and how might you make a convincing case to decision-makers (e.g. to expand the scope of an evaluation)?
- What trade-offs have been made? Can these be justified?
These might include:
Financial constraints
Any evaluation must be proportionate to the activity investment, for example, it would make no sense to spend £50,000 on evaluating a £20,000 project.
Time constraints (total days available and timeframe for delivery)
A well-designed evaluation allows sufficient time for it to be completed effectively, with delivery of the findings at an appropriate point, and should dovetail with key decision-making processes. This is especially important where a decision may lock in an organisation to a particular investment for a long period of time. Consider how much time participants will be able to give – ensure the methodology uses their time wisely.
Limitations of scope, linked to resources or the interest and responsibilities of the commissioning body
An evaluation may also be limited by the scope of interest of the commissioning body. However, it is important that more peripheral impacts and outcomes are documented wherever possible, even if they cannot be explored in detail. This will contribute to our learning about how best to adapt, and the potential impacts of adaptation. A well-designed evaluation must balance a pragmatic approach focussed on its specific purpose, but with the flexibility to explore unexpected avenues.