Evaluation and Dwight Hall at Yale

Dwight Hall at Yale requires its member organizations to complete an annual assessment of the group's impact in the community and evaluate the quality of services and resources provided by Dwight Hall at Yale.  The 2009-2010 Member Group Inventory template may be downloaded here:

2009-2010 Member Group Inventory.doc

Dwight Hall at Yale established an evaluation pilot program in 2004. The implementation plan may be viewed here:

2004 Evaluation Pilot Program.pdf.

As one of Dwight Hall at Yale's institutional programs, the Urban Fellows program pairs a survey for student participants and a survey for host agencies to evaluate the impact of the program on student participants and the community at large.

2008 Urban Fellow Supervisor Evaluations.doc

2007-2008 Urban Fellow Evaluation Questions.doc

Completed surveys are reviewed to ensure that the program's expected outcomes are achieved.

External Resources for Program Evaluation

 W.K. Kellogg Foundation

The W.K. Kellogg Foundation website features a Knowledge Center with resources for evaluation and examples of logic models with template forms. 

http://www.wkkf.org/knowledge-center/resources/2010/Logic-Model-Development-Guide.aspx

Dwight Hall at Yale provides a logic model worksheet adapted from United Way of America materials:

Logic Model Worksheet.doc

The W.K. Kellogg Foundation also provides an Evaluation Handbook that can be downloaded here:

http://www.wkkf.org/knowledge-center/resources/2010/W-K-Kellogg-Foundation-Evaluation-Handbook.aspx 

This handbook provides a framework for thinking about evaluation as a relevant and useful program tool. It was written primarily for project directors who have direct responsibility for the ongoing evaluation of W.K. Kellogg Foundation-funded projects.

Evaluation Center at Western Michigan University

The Evaluation Center at Western Michigan University provides a variety of resources related to program evaluation.  The Evaluation Center has developed a valuable set of high quality checklists that groups may find useful:

http://www.wmich.edu/evalctr/checklists/

Site Description:

This site provides refereed checklists for designing, budgeting, contracting, staffing, managing, and assessing evaluations of programs, personnel, students, and other evaluands; collecting, analyzing, and reporting evaluation information; and determining merit, worth, and significance.  Each checklist is a distillation of valuable lessons learned from practice.

The site’s purpose is to improve the quality and consistency of evaluations and enhance evaluation capacity through the promotion and use of high-quality checklists targeted to specific evaluation tasks and approaches.

Theory of Change

ActKnowledge hosts a website at www.theoryofchange.org that includes information about a valuable tool that helps organizations and initiatives identify basic assumptions about their program and articulate the interventions that will lead to intended outcomes.

TOC maps out your initiative through 5 stages:

• Identifying long-term goals and the assumptions behind them
• Backwards mapping and connect the preconditions or requirements necessary to achieve that goal.
• Identifying the interventions that your initiative will perform to create your desired change.
• Developing indicators to measure your outcomes to assess the performance of your initiative.
• Writing a narrative to explain the logic of your initiative.

The site also features an example of a Theory of Change for a specific program.  Background on this program (named Project Superwomen) is cited from the website:

Project Superwomen is a real program that started as a collaboration between a social service provider, a non-profit employment training center and a domestic violence shelter to help female abuse survivors to create long-term, livable wage employment opportunities for women who had been victims of domestic violence. The three organizations began their program with two basic assumptions (which are integral to their theory of change):

1. Non-traditional jobs, such as electrical, plumbing, carpentry, building management provide better wages and more opportunities for upward mobility and are more likely to have unions. Therefore, job stability and good wages are more likely if women are trained in these areas.
2. Women who have been through domestic abuse need more than job training to move to economic stability. They need to develop coping skills, workplace behaviors, and have child care available. They also need to be able to manage crises in their lives and such events as court appearances and dealing with the foster care system. If these aspects of their lives are not taken into account, any job training will not likely lead to permanent employment.

The collaboration subsequently brought in ActKnowledge to evaluate the program and its design. New Destiny Housing Corporation, the lead agency for Project Superwomen, has graciously allowed ActKnowledge to use their program as an example for this website.

Note: Project Superwomen is a program. Theories of change are often used for single programs like this. However, a strength of the theory of change approach is that it can be used for initiatives that may comprise many programs and partners. For the purposes of a tutorial to convey the basic processes and concepts, we will stick with a single program.