Program Evaluation for the Homelessness Sector

Analyzing the Data

This lesson covers basic strategies to analyze your data. 

Once you have collected your data, you will want to compile and analyze it. For example, from the survey you may want to know how many people were satisfied with current services and how many people suggested they wanted an increase in services. From the focus groups and interviews you may want to know what were the most common themes to emerge from the interviews.

Based on the results of your analysis you will decide what your findings mean. In other words you want to know if the change that you have seen (if there has been change) is valuable. You can compare your findings against your expectations or program standards. With your results in hand you can decide to continue, expand, change or redesign the program. In some instances, though not often, you may decide that the change is not valuable. If that’s the case, then it’s likely that you will end the program.

In this section, we look at some basic strategies to give you an idea of how to analyze your data. To analyze your data using more advanced techniques, you will likely need to get some additional training on data analysis or enlist some expert support. There are many online resources available, but it can be challenging if you don’t have some basic knowledge.

To make sense of your data, start by seeing if you can spot any errors that may have been made during collection. Sort to find missing values and data that has been entered incorrectly. It is important to start with a clean data set before proceeding further.

Analyzing Quantitative Data

You can analyze your data by hand or use statistical software. The method you use will depend on how much data you have, your evaluation questions and the types of analysis you need. Begin with basic analysis of frequency, percentages and averages for each data category. 

Review each item/category and create a pattern for your analysis based on the relationships among categories (i.e., variables) and how important they are in answering the evaluation questions or measuring the indicators. For example, if you need to know how gender may affect responses to a training program with regard to knowledge test scores or level of satisfaction, you will need to run an analysis to describe the relations among these variables.

When you compile the results, it will make sense to arrange them so that they relate to both the evaluation questions and indicators.

Software programs such as SPSS and SAS are great tools for analyzing quantitative data, but don’t forget about Excel. Most people have Excel as part of an MS Office Suite. There are some websites that provide Excel tutorials and also YouTube videos if you prefer. You may also find low-cost courses on Excel data analysis on Coursera, LinkedIn Learning and Udemy

Analyzing Qualitative Data

The method you use to analyze field observation notes will be different than analyzing data from interviews and case studies. Read over all the collected data that related to your questions and indicators. 

  1. Sort the data based on common themes. For example, search for verbs or phrases and use codes to organize them (e.g., action verbs, affective verbs).
  2. Find similarities among responses across participants (e.g., positive feelings toward a situation). Check whether the identified similarities can address your highlighted evaluation questions and indicators, and document them.
  3. Identify the level of occurrence for each similarity: high (mentioned by many respondents), medium (mentioned by some respondents), or low (mentioned by few respondents).
  4. Repeat the steps above to find differences among responses across participants (e.g., positive vs negative feelings toward a situation). Identify the level of occurrence for each difference.
  5. Search for responses, incidents, or stories that are odd and distinguished in any way or introduce a new perspective.
  6. Prepare the results of your analysis by explaining evaluation questions and indicators on one side and the themes and categories, general and particular patterns of responses, similarities and differences, and distinguished or innovative perspectives on the other side. 

Make that you don’t get tripped up by these common pitfalls:

  • Don’t assume that the program is the only cause of positive changes. 
  • Other factors may be responsible for changes you have observed. Acknowledge these other factors in your final report.
  • Don't assume your method is 100% reliable.

The same methods can give different results when used by different people. Respondents may tell the evaluator what they think they want to hear.

There are software programs that will make the tasks of sorting and categorizing qualitative data easier. This list provides some that offer free, freemium or paid versions.

Also check out these resources: