Evaluation

TRAINING LEVEL: Beginner

Training Objectives

In this document you will learn:

• What evaluation means, what is its purpose and why it is an important part of participatory processes
• How to approach evaluation and how to create an evaluation plan for your project
• Basic methods of data collection for evaluation

Table of Contents:

1. Key terms definition
2. The importance of evaluation for participatory processes
3. How to approach participatory process evaluation
4. Data collection for evaluation
5. Theory of Change

1. KEY TERMS DEFINITION

EVALUATION 

Evaluation, derived from the French word évaluer, means a systematic assessment of the quality, value and success of a process or project based on data that have been collected throughout its duration. It is an essential but often neglected part of any participatory process. The purpose of evaluation is to identify which aspects of the participatory process worked well and what were the weak points. These findings enable you to propose specific steps to improve or readjust the process for its subsequent cycles or for a similar process in the future.

2. THE IMPORTANCE OF EVALUATION FOR PARTICIPATORY PROCESSES 

Identifying the strengths and weaknesses of the process and how to improve them 

Evaluation serves to assess what aspects of the participatory process went well, and what parts did not work and why. Identifying weak points enables you to readjust the process, either for its continuation or for a similar process or project in the future. Evaluation thus serves as a tool for improving participatory processes. 

Measuring project results and impact 

Evaluation helps you to measure and define the results of the evaluated participatory process, allowing you to determine what changes and how much impact the process has achieved. In addition, public grant offers are often conditional upon the achievement of measurable results. 

Evaluation of the overall process and its individual aspects 

Monitoring the participatory process throughout its duration will allow you to assess whether it is progressing according to the plan and thus leading to the desired outcomes. Moreover, monitoring allows the project team to thoroughly map the given process and the methods and tools used, review and learn from the experience, optimize future processes and establish good practice that can be replicated or inspire future projects.

3. HOW TO APPROACH PARTICIPATORY PROCESS EVALUATION  

Approaches to evaluation range from simple monitoring that can be done internally within your institution to more robust procedures based on extensive data and information collection during the participatory process that may require hiring a third-party evaluator.  

The first step is to clarify the purpose of the evaluation of your participatory process and what type of findings it should provide, and then determine the appropriate format and scope of the evaluation.

Examples of what to focus on when evaluating participatory processes: 

1. Process mapping: capturing individual process steps for optimization, replication or inspiration for future processes
2. Identifying weak and/or strong aspects of the process: what worked well and what did not, to adjust future processes and their goals
3. Optimization of the process based on its evaluation
4. Measuring and quantifying process results
5. Capturing the impact of the project
6. Focusing on evaluating certain aspects of the process, e.g.:
   • Involvement of the target group, e.g.:
     • Was the target group sufficiently involved? Were the correct methods used to engage them? Did their involvement meet the expected goals (e.g. did you receive the necessary information/data, was there a sufficient degree of deliberation, was there a sense of belonging to the process among the target group, etc.)?
     • Were all the necessary target groups involved? Were some groups excluded from the process?
   • Efficiency of use of allocated resources
   • Human capacities
   • Project management
   • Communication
   • Quality of results
   • And more

4. DATA COLLECTION FOR EVALUATION 

For a successful evaluation, it is essential to decide on how to evaluate the project in the early stages of project planning, in order to know what data and reports are needed for the final evaluation and what information to record throughout the duration of the project.

Examples of data collection methods:

• In-depth interviews (e.g. with the project team, representatives of the target group, stakeholders, etc.)
• Quantitative survey (questionnaires)
• Observation
• Input documents (meeting minutes, progress reports from contractors, output reports from individual activities, etc.)

5. THEORY OF CHANGE 

One of the more complex and detailed process evaluation methods is the Theory of Change. This method is used to map out how and why a desired change is expected to happen in a particular context. It does this by first defining the desired long-term goals and then works back from these to identify all the conditions (outcomes) that must be in place (and how these relate to one another) for the goals to occur. It explains the mutual connections and time continuity of the individual steps. Theory of Change requires you to define the expected results even before you begin to plan the course-of-action to achieve them and to identify connections between the individual steps: causes and consequences (“if A, then B”) and chronology (“first A, then B”). This leads to better planning, in that activities are linked to a detailed understanding of how change actually happens. Theory of Change helps explain why goals have or have not been achieved by understanding causes and effects.

This method can be used to determine: 

• Project goals and how to achieve them
• Metrics to measure achievement of goals
• Inputs needed to meet objectives (finance, experts, partners, etc.)
• Partial activities that are part of the project (e.g. background creation, training, use of tools, etc.)
• Project outputs/outcomes
• Project results