Program Evaluation for the Homelessness Sector

Evaluation Strategy

This lesson is on developing an evaluation strategy.

One of the most important steps in evaluation is determining your evaluation question. You will recall in Part 1, Focus the evaluation, that the evaluation design is determined by the questions you want to answer. We have already seen examples of common evaluation questions such as:

  1. Is my program or service being implemented as intended?
  2. Are we doing this program the way we said we would do it?
  3. Are my program or service participants achieving desired outcomes?

Your question will help you decide on the type of evaluation design that will be the best fit. 

Determining the Evaluation Strategy:

FORMATIVE PROCESS OUTCOME
Is my program being implemented as intended? Are we delivering this program the way we said we would do it? Are program participants achieving the desired outcomes?
METHOD:
Logic models, key informant interviews, focus groups, client interviews
METHOD:
Key informant interviews, focus groups, client interviews
METHOD:
Surveys with clients over time

You need to know the type of evaluation design for your specific questions before you can  think about how you will collect the data. Data sources could include documentation such as administrative records, interviews with program staff or participants, or focus groups. A good way to map out this process is through an evaluation matrix. An evaluation matrix outlines three major categories: your evaluation question, the method you will use to answer the question, and your data source. An example of the structure is shown in the table below:

Evaluation Question Method Data Source
(1) Is my program or service being implemented as planned? Choose method

Choose data source

(2) What were the facilitators and barriers the program or service experienced?    
(3) What are the housing outcomes of clients in the program or service?    

Table 4. Evaluation Matrix

Before you decide on how you will gather your evidence it will be important to determine how much time and resources can be dedicated to the project. For example, is this something that can be done internally? Are external evaluators required? You should develop a timeline and include that in your evaluation plan.

A visual representation of this timeline is helpful, such as a GANTT chart. A very basic example can be found below, but much more sophisticated examples online and templates are available for various software applications. 

GANTT Chart examples

 

As you have probably figured out, determining a strategy to answer your evaluation questions requires thoughtful planning.

The Research Question video

In the video above, Dr. Bernie Pauly discusses the relationship between the evaluation question and the approach and she compares program evaluation with other research projects.