This lesson is an introduction to outcome measurement. An outcome measure is used to assess the effects of an intervention.
For each outcome that you identify in the logic model, you will need to determine an indicator that will be used to measure the outcomes. You will also need to decide where the indicator data will come from and how it will be measured.
Below is an example of an outcome measurement framework for a public health intervention looking to improve the health of Canadians.
|OUTCOME||INDICATOR||DATA SOURCE||DATA COLLECTION
/ TOOL / METHOD
|Improved physical health||How often participants exercise||Participants||Participant exercise log|
|Number of trips to the hospital||Hospital records||Review of hospital data|
|Quality of life||Participants||Participant survey|
Table 5. Sample outcomes matrix.
When choosing outcomes measures, don’t reinvent the wheel! Several open access tools are available. Try these websites to start with:
- Ontario Centre of Excellence for Child and Youth Mental Health Measures database
- YouthRex youth-friendly outcome measures https://youthrex.com/wp-content/uploads/2019/10/The-Social-Connectedness-Scale-Revised.pdf
How many outcomes should you be monitoring?
The fact is you can’t measure everything. Fortunately, you don’t need many outcomes to gain valuable insights as long as they are clearly defined. One of the ways that program evaluation fails to provide usable results is in not having a clear focus. You learned in an earlier lesson how logic models help to define clear outcomes. Getting clear about outcomes is important during the planning stage. If there are too many outcomes, you risk making the process too complicated to yield clear results.
Outcomes can be grouped into short-term, medium-term, and long-term. Some examples of short- to mid-term outcomes are:
- Greater knowledge of tenancy rights.
- Improved financial management skills.
- More effective, self-directed coping skills.
- Attaining employment.
- Improved housing stability.
- Enhanced awareness of community networks.
|Questions to consider when determining outcomes:|
|1. Is it reasonable to believe the program can influence the outcome in a non-trivial way, even though it can’t control it?|
|2. Would measurement of the outcome help identify program successes and help pinpoint and address problems or shortcomings?|
|3. Will the program’s various stakeholders (staff, volunteers, participants, collaborating organizations, funders, the general community) accept this as a valid outcome of the program?|
Now that you have defined your outcomes, the next step is figuring out how to measure the outcomes you have described. To do so, you need to define your outcome indicators.
Outcome indicators are specific, measurable pieces of information that you can collect to keep track of the difference that your work is making. They tell you whether or not you are achieving your outcomes and how much change has occurred. Outcome indicators should be phrased as ‘level of’, ‘number of’, ‘type of’ or ‘how often’. They should not use a word that indicates change, such as ‘increased’ or ‘improved’.
They should also include numbers (quantitative data) and descriptions or narratives (qualitative data).
This data should combine subjective (i.e., self-reported perspective of participants) and objective (i.e., number of people achieving stable housing) indicators.
|Questions to consider when choosing indicators:|
|1. Is there at least one indicator for each outcome?|
|2. Does each indicator measure some important aspect of the outcome that no other indicator measures? (Try to avoid duplication)|
|3. Is the wording of each indicator sufficiently specific? Does it tell you what characteristic or change you will count?|
|4. Does each indicator identify the statistic(s) that will summarize the program’s performance on the outcome? Will the statistic(s) convey your level of achievement effectively?|