London School of Hygiene & Tropical Medicine

The module focuses on evaluating public health interventions. These interventions are often “complex”, with many interacting components. We teach approaches to conceptualising, describing and developing logic models for this type of intervention. The course teaches methods for outcome evaluation, including both randomised and nonrandomised designs. We teach about process evaluation, used to describe implementation, mechanisms of change and relevant contextual factors. Finally, you will meet experienced evaluators throughout the course who will describe some of the practical, ethical and technical challenges in undertaking such evaluations.

The overall module aim is to help students understand what evaluation is, when it can be used, and to provide students with the knowledge and skills to design an evaluation of a public health intervention, programme or policy tailored to the research question and context.

The module illustrates how theories of change can be used within an evaluation, and illustrates various approaches to estimating the causal effects of programmes and policies.

The module highlights the importance of integrated/mixed process evaluation methods in order to open the ‘black box’ by measuring implementation, process and pathways to outcomes and relevant contextual factors. The module emphasis is on evaluation design, however, an overview of analytical methods will also be provided.

Intended learning outcomes

Upon successful completion of the module a student should be able to:

1. Define outcome and process evaluation and differentiate these from related activities such as M&E (monitoring and evaluation) and clinical trials;
2. Describe how and why outcome and process evaluation are relevant to situations where public health interventions are complex in form, and are evaluated in real life settings where context is likely to be relevant to the success of the intervention;
3. Understand the key aims of process evaluation - to study questions related to the feasibility and fidelity of intervention implementation; to assess mechanisms of change including understanding how participants respond to interventions and concepts such as acceptability and accessibility; and to document key aspects of context relevant to understanding intervention success and failure and thus inform understanding of the potential for transfer or scale-up;
4. Assess the strengths and weaknesses of methodological approaches that can be used to collect data in process evaluations;
5. Propose the design for a process evaluation of a complex public health intervention;
6. Describe the core purposes of outcome evaluation;
7. Understand the idea of counterfactual thinking and describe how this relates to the questions answered by, and the strengths of, a randomised controlled trial design;
8. Describe the principles, data requirements, approaches to counterfactual estimation and strengths and limitations of a range of non-randomised outcome evaluation study designs: including, interrupted time-series, difference-in-difference, and designs based on matching;
9. Interpret results from studies using the designs detailed in the two points above;
10. Understand how outcome and process evaluations are usually designed in collaboration between evaluators and implementers.

Mode of delivery

This module is delivered predominantly face-to-face. Where specific teaching methods (lectures, seminars, discussion groups) are noted in this module specification these will be delivered by predominantly face-to-face sessions. There will be a combination of live and interactive activities (synchronous learning) as well as self-directed study (asynchronous learning).

Assessment

For the summative assessment, the assignment is in two parts:

1. A process evaluation proposal report (1500 words), worth 70% of the grade and individually completed. Students will produce a theory of change and process evaluation proposal for one of a choice of evaluation scenarios. The assignment will be conducted individually.
2. An impact evaluation proposal presentation prepared as a group, for 30%. Students will prepare a group presentation and report on an impact evaluation for a given scenario. Students will have the chance to ask questions at a dedicated session with the module organisers.

Credits

• CATS: 15
• ECTS: 7.5

Module specification

For full information regarding this module please see the module specification.