| |
Abstract/Syllabus:
|
380.611 Fundamentals of Program Evaluation
Spring 2006

Description
Fundamentals of Program Evaluation familiarizes students in different types of program evaluation, including needs assessment, formative research, process evaluation, monitoring of outputs and outcomes, impact assessment, and cost analysis. Students gain practical experience through a series of exercises involving the design of a conceptual framework, development of indicators, analysis of computerized service statistics, and development of an evaluation plan to measure impact. This course covers experimental, quasi-experimental, and non-experimental study designs, including the strengths and limitations of each.
Syllabus
Course Description
Familiarizes students in different types of program evaluation, including needs assessment, formative research, process evaluation, monitoring of outputs and outcomes, impact assessment, and cost analysis. Students gain practical experience through a series of exercises involving the design of a conceptual framework, development of indicators, analysis of computerized service statistics, and development of an evaluation plan to measure impact. Covers experimental, quasi-experimental, and non-experimental study designs, including the strengths and limitations of each.
Course Objectives
By the end of this course, students will be able to:
1. Explain the major concepts in program evaluation:
- Types of evaluation and their purpose
- Levels of measurement: population-based vs. program-based
- Sources of data
- Study designs, including randomized control trials, and threats to validity
2. Perform skills required in conducting program evaluation:
- Design of a conceptual framework
- Develop objectives and indicators
- Conduct of a focus group
- Pretest of a communication
- Processing of service statistics
- Use of participatory evaluation techniques
3. Write an evaluation plan
Readings
One classic evaluation textbook is Rossi, Lipsey, and Freeman. Evaluation: A Systematic Approach. 7th edition. Thousand Oaks, CA: Sage Publications, 2004.
It is not required reading for this class, but students intending to specialize in program evaluation may want to purchase this textbook and read relevant chapters that parallel the syllabus.
Course Requirements
Each student in the class will prepare an evaluation plan to fulfill the requirements for this class. The plan will contain four parts, each of which represents a separate assignment counting 25% toward the final grade. The topics to cover in each section are as follows:
- Defining the problem and describing the intervention
- Development of indicators
- Process evaluation
- Summative evaluation
Note: an evaluation plan does not necessarily have to conform to this outline, but we will use this outline as one approach to developing such a plan.
Students will have the option to select either a domestic program (tuberculosis control in Baltimore ) or an international program (Stop AIDS Love Life in Ghana ) for this paper. However, the four parts to the assignment must all use the same program; no switching back and forth between programs on the different parts of the assignment.
OCW offers a snapshot of the educational content offered by JHSPH. OCW materials are not for credit towards any degrees or certificates offered by the Johns Hopkins Bloomberg School of Public Health.
For information on for-credit courses go to: http://commprojects.jhsph.edu/courses. Unlike for-credit courses, OpenCourseWare does not require registration and does not provide access to the School's faculty.
Schedule
|
1 |
Overview of evaluation concepts |
Lecture |
2 |
Overview of evaluation concepts (cont.); TB Control Program in Baltimore City: Presentation of a domestic program |
Lecture |
3 |
Overview of evaluation concepts (cont.); Stop AIDS Love Life in Ghana: Presentation of a n international program
|
Lecture |
4 |
Developing a conceptual framework and introduction to formative research
|
Lecture |
5 |
Communication pretesting, needs assessment (U.S.); MCH needs assessment: An overview
|
Lecture |
6 |
Development of indicators and participatory evaluation methods |
Lecture; Assignment 1 due |
7 |
Process evaluation |
Lecture |
8 |
Routine health information systems: concepts and methods
|
Lecture |
9 |
Monitoring outputs and outcomes and introduction to study design |
Lecture |
10 |
Experimental, non-experimental, and quasi-experimental designs |
Lecture; Assigment 2 due |
11 |
Using qualitative methods for monitoring and evaluation
|
Lecture |
12 |
Randomized control trials
|
Lecture; Assignment 3 due |
13 |
Cost effectiveness analysis |
Lecture |
14 |
Monitoring and evaluation in action
|
Panel discussion |
15 |
Review session; Discussion of final exercise
|
Review |
16 |
Submit final paper |
Assigment 4 due |
|
|
|
Further Reading:
|
Readings
Textbooks
One classic evaluation textbook is Rossi, Lipsey, and Freeman. Evaluation: A Systematic Approach. 7th edition. Thousand Oaks, CA: Sage Publications, 2004.
It is not required reading for this class, but students intending to specialize in program evaluation may want to purchase this textbook and read relevant chapters that parallel the syllabus.
|
1 |
Overview of evaluation concepts |
No reading |
2 |
Overview of evaluation concepts (cont.); TB Control Program in Baltimore City: Presentation of a domestic program |
Jane T. Bertrand and Gabriela Escudero. Compendium of Indicators for Evaluating Reproductive Health Programs. Chapel Hill, NC: Carolina Population Center , MEASURE Evaluation Project, 2004: Pp. 1-14.
Jane T. Bertrand. Evaluating Health Communication Programs. Drum Beat, June 2005. Issue 302.
|
3 |
Overview of evaluation concepts (cont.); Stop AIDS Love Life in Ghana: Presentation of a n international program |
Andrew Fisher, James Foreit, et al. Designing HIV/AIDS Intervention Studies: An Operations Research Handbook. New York: Population Council, 2002: 1-44, 73-84
"Context and Content of Condom Negotiation and Sex Refusal Skills among Youth in Ghana." Report on Qualitative Research Using a "Storyline" Methodology. November 1997.
|
4 |
Developing a conceptual framework and introduction to formative research |
J.A. Earp, and S.T. Ennett. Conceptual models for health education research and practice. Health Educ Res 1991;6:163-171.
Assessing the Need for Programs. In: J. McDavid and L. Hawthorn, Program Evaluation and Performance Measurement. Thousand Oaks , CA : Sage Publications. 2006. |
5 |
Communication pretesting, needs assessment (U.S.); MCH needs assessment: An overview |
SS Weir, et al. A pilot study of a rapid assessment method to identify places for AIDS prevention in Cape Town , South Africa. Sex Transm Infect 2002;78(suppl 1):i106-i113
CC Johnson, et al. Profiles of the adolescent smoker: models of tobacco use among 9th grade high school students. Acadiana Coalition of Teens against Tobacco (ACTT). Preventive Medicine 2004;39:551-558.
|
6 |
Development of indicators and participatory evaluation methods |
Jane T. Bertrand and Gabriela Escudero. Compendium of Indicators for Evaluating Reproductive Health Programs. Chapel Hill, NC: Carolina Population Center , MEASURE Evaluation Project, 2004: (vol 1) pp. ix-xiv; and Appendix B. (vol 2, pp. B-462 to 463).
Who Are the Question Makers? - A Participatory Evaluation Handbook, UNDP.
|
7 |
Process evaluation |
RP Saunders, MH Evans, J Praphul. Developing a Process-Evaluation Plan for Assessing Health Promotion Program Implementation: A How-To Guide. Health Promotion Practice 2005;6: 134-147.
CI Viadro. Designing a Process Evaluation for a Comprehensive Breast Cancer Screening Intervention: Challenges and Opportunities. Evaluation and Program Planning 1997;20:237-249.
|
8 |
Routine health information systems: concepts and methods |
Theo Lippeveld. Design and Implementation of HIS. Geneva: World Health Organization, 2000. Chapters 1, 2, 3, and 14. |
9 |
Monitoring outputs and outcomes and introduction to study design |
Andrew Fisher, James Foreit, et al. Designing HIV/AIDS Intervention Studies: An Operations Research Handbook. New York: Population Council, 2002: pp. 45-71
B Krevor. Preventing Illegal Tobacco and Alcohol Sales to Minors Through Electronic Age-Verification Devices: A Field Effectiveness Study. J Pub Health Policy 2004;24(3/4):251-268.
|
10 |
Experimental, non-experimental, and quasi-experimental designs |
DL Kincaid, et al. Impact of a Mass Media Vasectomy Promotion Campaign in Brazil. International Family Planning Perspectives 1996;22:169-175. |
11 |
Using qualitative methods for monitoring and evaluation |
Debus, Mary. Handbook for Excellence in Focus Group Research. Washington, DC: Academy for Educational Development, 1998. |
12 |
Randomized control trials |
RN Shain, JM Piper, et al. A Randomized Controlled Trial of a Behavioral Intervention to Prevent Sexually Transmitted Disease Among Minority Women. The New England Journal of Medicine 1999;340:93-100.
GC Smith, JP Pell. Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomized controlled trials. BMJ 2003;327:1459-1461.
|
13 |
Cost effectiveness analysis |
Irene Huse, James C. McDavid, and Laura Hawthorn. Concepts and Issues in Economic Evaluation. In: McDavid and Hawthorn (eds) Program Evaluation & Performance Measurement: An Introduction to Practice. Thousand Oaks, CA: Sage Publications, 2006. pp. 241-278.
Thielman et al. Cost-Effectiveness of Free HIV Voluntary Counseling and Testing Through a Community-Based AIDS Service Organization in Northern Tanzania. Am J Pub Health, 2006;96:114-125.
|
14 |
Monitoring and evaluation in action |
Victora et al. Evidence-based public health: beyond randomized trials. Am J Pub Health 2004;94:400-405.
JP Habicht, CG Victora, and JP Vaughn. Evaluation designs for adequacy, plausibility and probability of public health programme performance and impact. Int J Epidemiol 1999;28:10-18.
|
|
|
|
Rating:
0 user(s) have rated this courseware
Views:
25601
|
|
|
|
|