Intro to Program Evaluation for Public Health
Introduction to Program Evaluation for Public Health Program: A Self Study Guide
Checklists for the 6 steps:
- Engage Stakeholders
- Describe the Program
- Focus the Evaluation Design
- Gather Credible Evidence
- Justify Conclusions
- Ensure Use of Evaluation Findings and Share Lessons Learned
Checklist for Step 1: Engage Stakeholders
- Identify stakeholders, using the three broad categories discussed: those affected, those involved in operations, and those who will use the evaluation results.
- Review the initial list of stakeholders to identify key stakeholders needed to improve credibility, implementation, advocacy, or funding/authorization decisions.
- Engage individual stakeholders and/or representatives of stakeholder organizations
- Create a plan for stakeholder involvement and identify areas for stakeholder input
- Target selected stakeholders for regular participation in key steps, including writing the program description, suggesting evaluation questions, choosing evaluation questions, and disseminating evaluation result.
Checklist for Step 2: Describing the Program
- Compile a comprehensive program description including need, targets, outcomes, activities, and resources.
- Identify the stage of development and context of the program.
- Convert inputs, activities, outputs, and outcomes into a simple global logic model
- Elaborate the model as needed
- Develop more detailed models from the global model as needed
Checklist for Step 3: Focusing the Evaluation Design
- Define the purpose(s) and user(s) of your evaluation.
- Identify the use(s) of the evaluation results.
- Consider stage of development, program intensity, and logistics and resources.
- Determine the components of your logic model that should be part of the focus given these utility and feasibility considerations.
- Formulate the evaluation questions to be asked of the program components in your focus (ie. implementation, effectiveness, efficiency, and attribution questions.
- Review evaluation questions with stakeholders, program managers, and program staff.
- Review options for the evaluation design, making sure that the design fits the evaluation questions.
Checklist for Step 4: Gathering Credible Evidence
- Identify indicators for activities and outcomes in the evaluation focus.
- Determine whether existing indicators will suffice or whether new ones must be developed
- Consider the range of data sources and choose the most appropriate one
- Consider the range of data collection methods and choose those best suited to your context and content.
- Pilot test new instruments to identify and/or control sources of error.
- Consider a mixed-method approach to data collection.
- Consider quality and quantity issues in data collection
- Develop a detailed protocol for data collection
Checklist for Step 5: Justifying Your Conclusions
- Check data for errors
- Consider issues of context when interpreting data
- Assess results against available literature and results of similar programs.
- If multiple methods have been employed, compare different methods for consistency in findings.
- Consider alternative explanations.
- Use existing standards (e.g. Health People 2010 objectives) as a starting point for comparisons
- Compare program outcomes with those of previous years.
- Compare actual with intended outcomes.
- Document potential biases
- Examine the limitations of the evaluation.
Checklist for Step 6: Ensuring That Evaluation Findings Are Used and Sharing Lessons Learned
- Identify strategies to increase the likelihood that evaluation findings will be used.
- Identify strategies to reduce the likelihood that information will be misinterpreted.
- Provide continuous feedback for the program.
- Prepare stakeholders for the eventual use of evaluation findings.
- Identify training and technical assistance needs.
- Use evaluation findings to support annual and long-range planning.
- Use evaluation findings to promote your program.
- Use evaluation findings to enhance the public image of your program.
- Schedule follow-up meetings to facilitate the transfer of evaluation conclusions.
- Disseminate procedures used and lessons learned to stakeholders.
- Consider interim reports to key audiences.
- Tailor evaluation reports to audience(s)
- Revisit the purpose(s) of the evaluation when preparing recommendations
- Present clear and succinct findings in a timely manner.
- Avoid jargon when preparing or presenting information to stakeholders
- Disseminate evaluation findings in several ways.