Contact Us
Outcome evaluation overview
Outcome evaluations measure the effect a programme has had on a target population by assessing the degree to which it has met desired outcomes. Outcomes are positive changes the programme intends to achieve. Examples include changes in knowledge, skills, behaviours, capacity, or health status of the target population. Outcomes are typically categorised by short-term, medium-term, and long-term goals. What these temporal categories mean depend on the programme, but short-term could refer to 1–3 years, medium-term to 2–5 years, and long-term to 5+ years.1, 2
How to perform an outcome evaluation
The following stages, while presented chronologically, are not always linear. Depending on the evaluation, stages might need to be completed simultaneously or be revisited.
Stage 1: Create an advisory group
The first step of an outcome evaluation is to create an advisory group. This group consists of evaluators and key stakeholders who will plan for and oversee the evaluation. Advisory groups lead to better quality and useful evaluations.3, 4
Note: Advisory groups may also be called reference groups, working groups, or advisory panels.
Stage 2: Determine the scope of the evaluation
The advisory group should clarify what is to be evaluated, develop or revisit a theory of change, discuss possible evaluation methods, and agree on the audience of the evaluation. These evaluation characteristics should be used to create a terms of reference and/or an evaluation plan.
Stage 3: Create an evaluation plan
Create an evaluation plan to outline what the evaluation is trying to do and how it will be performed. At minimum, the plan should list evaluation questions, determine what data are required, choose methods for obtaining and analysing the data, and include a schedule of dates, milestones, and responsibilities. Other information, such as a theory of change or Gantt chart, can also be included in the plan.5
Some examples of outcome evaluation questions include: 6
- How well did the programme work?
- Did the programme contribute to the intended outcomes?
- What unintended outcomes were produced?
- To what extent can changes be attributed to the programme?
When selecting data collection and analysis methods, consider the following: 7
- Are the methods appropriate for the required data?
- Can the methods ensure accuracy and validity?
- Will technical assistance be required?
Stage 4: Perform the evaluation
Collect and analyse the data as outlined in the evaluation plan.
Stage 5: Interpret and report on the findings
Interpret the data, make a list of recommended actions, and present your findings. Results are typically presented in a final written report, but slide shows or informational videos can also be used. Visual aids, such as short videos or infographics, are great methods for communicating evaluation results in a clear and simple way. Results should be shared with all stakeholders.8
Stage 6: Apply the findings
The advisory group should review the recommendations, discuss what has been learned from the evaluation, and identify next steps for applying the results. Prioritise the actions and create an action plan to ensure the results are used.8
Outcomes vs. impacts
The terms “outcome” and “impact” are sometimes used interchangeably in evaluation. However, impacts typically refer to the broad, long-term outcomes of a programme. For example, an outcome of a gambling self-exclusion programme might be to help gamblers feel more in control of their gambling behaviour, whereas an impact of that same programme might be to reduce long-term rates of harmful gambling and improve population health.9
Programme impacts can be difficult to measure and attribute to a single activity. Likewise, impact evaluations can be time consuming and difficult to perform. For these reasons, outcome evaluations are more common.
Additional resources
For more information on how to design and conduct outcome evaluations, visit the resources below, or contact evaluation@greo.ca to see how Greo can help.
- Guidance: Outcome evaluation (Public Health England)
- Strategies for Causal Attribution (United Nations International Children's Emergency Fund)
- Evaluation Handbook (W.K. Kellogg Foundation)
1 Centers for Disease Control and Prevention. (n.d.). Types of Evaluation. https://www.cdc.gov/std/Program/pupestd/Types%20of%20Evaluation.pdf
2 W.K. Kellogg Foundation. (1998). Evaluation Handbook. https://www.wkkf.org/resource-directory/resources/1998/01/evaluation-handbook
3 BetterEvaluation. (2020). Advisory Group. https://www.betterevaluation.org/en/evaluation-options/advisory_group
4 Child Trends. (2008). Process evaluations: A guide for out-of-school time practitioners. https://www.childtrends.org/wp-content/uploads/2008/01/Process-Evaluation.pdf
5 BetterEvaluation. (2018). Evaluation Plan. https://www.betterevaluation.org/en/evaluation-options/evaluation_planning_templates
6 BetterEvaluation. (2016). Specify the Key Evaluation Questions (KEQs). https://www.betterevaluation.org/en/rainbow_framework/frame/specify_key_evaluation_questions
7 International Program for Development Evaluation Training. (2007). Module 8: Data Collection Methods. IPDET Handbook. https://www.measureevaluation.org/resources/training/capacity-building-resources/data-quality-portuguese/DATA_COLECTION.pdf
8 Ontario Agency for Health Protection and Promotion (Public Health Ontario). (2015). At a glance: The ten steps for conducting an evaluation. Toronto: Queen’s Printer for Ontario. https://www.publichealthontario.ca/-/media/documents/A/2015/at-a-glance-10step-evaluation.pdf
9 Public Health England. (2018). Guidance: Outcome evaluation. https://www.gov.uk/government/publications/evaluation-in-health-and-well-being-overview/outcome-evaluation