Course Details

Evaluation Methodologies for Complex Programs Training Course

INTRODUCTION

This essential training course provides comprehensive knowledge and practical skills for designing and implementing evaluations of complex programs. Recognizing that traditional linear evaluation approaches often fall short in capturing the nuances and emergent properties of interventions in dynamic, interconnected systems, this program equips participants with a systematic understanding of advanced methodologies tailored for complexity. Participants will gain deep insights into systems thinking, causal inference in complex contexts, participatory approaches, and adaptive learning, all crucial for generating robust evidence, fostering organizational learning, and enhancing the effectiveness of intricate development and social initiatives. This course is ideal for M&E specialists, program managers, researchers, policymakers, and funders grappling with complex interventions.

DURATION

10 days

TARGET AUDIENCE

This course is specifically designed for professionals involved in evaluating or managing programs characterized by multiple interacting components, diverse stakeholders, unpredictable environments, or emergent outcomes, including:

• Monitoring and Evaluation (M&E) Specialists.
• Program Managers and Directors overseeing complex interventions.
• Researchers and Academics in development, public health, or social change.
• Policymakers and Strategists dealing with systemic challenges.
• Funders and Grant Managers supporting complex programs.

OBJECTIVES

Upon completion of this course, participants will be able to:

• Understand the characteristics of complex programs and their implications for evaluation.
• Differentiate between various evaluation methodologies suitable for complexity.
• Apply systems thinking principles to design more insightful evaluations.
• Navigate challenges in establishing causality and contribution in complex settings.
• Develop adaptive evaluation strategies that foster learning and program improvement.

MODULES

Module 1: Understanding Program Complexity in Evaluation

• Define what makes a program "complex" (e.g., non-linear, emergent, adaptive, multi-stakeholder).
• Explore the limitations of traditional, linear evaluation models in complex environments.
• Understand key concepts from complexity theory relevant to evaluation (e.g., feedback loops, emergence, tipping points).
• Discuss the implications of complexity for evaluation design, data collection, and analysis.
• Examine different types of complex programs (e.g., systems change, capacity building, advocacy).

Module 2: Systems Thinking for Evaluation

• Understand the principles of systems thinking and how to apply them in evaluation.
• Discuss methods for mapping program systems and identifying key actors and linkages.
• Explore techniques for identifying feedback loops (reinforcing and balancing) within a program.
• Examine the importance of boundaries and context in understanding system dynamics.
• Learn about leveraging systems maps to formulate more comprehensive evaluation questions.

Module 3: Theory-Based Evaluation Approaches for Complexity

• Understand how Theory of Change (ToC) can be adapted for complex programs.
• Discuss the application of Process Tracing for understanding causal mechanisms in complex pathways.
• Explore Realist Evaluation for explaining "what works, for whom, in what circumstances, and why."
• Examine Contribution Analysis for assessing plausible contribution in multi-causal contexts.
• Learn about combining theory-based approaches for robust evidence in complexity.

Module 4: Participatory and Collaborative Evaluation in Complex Settings

• Understand the rationale for involving diverse stakeholders in evaluating complex programs.
• Discuss participatory evaluation methods that foster shared learning and ownership.
• Explore techniques for facilitating collaborative sense-making workshops with stakeholders.
• Examine the role of evaluators as facilitators and learning partners.
• Learn about managing power dynamics and ensuring genuine inclusion in complex evaluations.

Module 5: Qualitative Methodologies for Capturing Complexity

• Understand the strengths of qualitative methods in uncovering emergent outcomes and contextual factors.
• Discuss advanced qualitative data collection techniques (e.g., in-depth interviews, ethnographic observation, narrative inquiry).
• Explore methods for qualitative data analysis suitable for complexity (e.g., thematic analysis, grounded theory, discourse analysis).
• Examine techniques for capturing unintended consequences and emergent behaviors.
• Learn about ensuring rigor and trustworthiness in qualitative evaluation of complex programs.

Module 6: Quantitative Approaches and Data Innovations for Complexity

• Discuss the appropriate use of quantitative methods within a complex evaluation strategy.
• Explore network analysis for understanding relationships and influence within systems.
• Examine social network analysis (SNA) for mapping connections between actors.
• Learn about the potential of big data, remote sensing, and AI for tracking complex changes.
• Understand the challenges and ethical considerations of using large-scale quantitative data in complex settings.

Module 7: Adaptive Evaluation and Learning Systems

• Understand the concept of "adaptive evaluation" and its role in supporting adaptive management.
• Discuss how to design evaluation processes that generate rapid feedback loops.
• Explore methods for facilitating regular reflection and learning cycles within programs.
• Examine how to integrate evaluation findings for ongoing program adjustment and improvement.
• Learn about developing a learning agenda for complex interventions.

Module 8: Managing and Communicating Complex Evaluations

• Discuss the management challenges unique to evaluating complex programs (e.g., scope creep, evolving objectives).
• Explore strategies for structuring evaluation reports for complex findings.
• Understand how to communicate uncertainty and nuanced findings to diverse audiences.
• Examine the role of visual tools and storytelling in conveying complexity.
• Learn about building evaluation capacity for dealing with complex programs internally.

CERTIFICATION

• Upon successful completion of this training, participants will be issued with Macskills Training and Development Institute Certificate

TRAINING VENUE

• Training will be held at Macskills Training Centre. We also tailor make the training upon request at different locations across the world.

AIRPORT PICK UP AND ACCOMMODATION

• Airport pick up and accommodation is arranged upon request

TERMS OF PAYMENT

Payment should be made to Macskills Development Institute bank account before the start of the training and receipts sent to info@macskillsdevelopment.com