Monitoring and Evaluation of EbA

Effective monitoring and evaluation (M&E) of adaptation activities is critical for building a strong, global evidence base around EbA measures and for assessing the wide, diverse range of interventions being implemented to address it. At the global level, it is a tool for identifying and documenting successful projects and approaches and tracking progress toward common indicators.  At the project level, the purpose is to track implementation and outputs systematically, and measure the effectiveness of projects, while strengthening understanding around the many multi-layered factors underlying EbA. By doing so, it can also prevent future implementation problems (such as mal-adaptation).

A comprehensive framework and training on M&E of overall climate change adaptation has been developed by the German Development Cooperation (GIZ).

However, specific guidance on M&E of EbA is still rare. The project ‘Strategic Mainstreaming of Ecosystem-based Adaptation’ thus developed a concept note on M&E of EbA which is based on the above documents, but transfers the existing knowledge into an EbA context. The concept note served as a basis for the development of a monitoring and evaluation system in the pilot provinces Ha Tinh and Quang Binh.

In the guidebook ‘Adaptation made to measure’, GIZ uses a five-step model for creating an M&E framework for climate change adaptation. This model is a tool to guide practitioners in setting up a framework to monitor and evaluate implementation of activities as well as the impact, output and outcomes related to these activities and interventions based on ‘SMART’ (Specific, Measurable, Attainable, Relevant and Time bound) indicators .

For monitoring EbA, the complexity of a social-ecological system and the environmental, economic and social impacts of climate change need to be taken into account each step of the model. Specifically, project activities and their impacts on ecosystem services (short, medium and long-term), as well as the resilience, vulnerability and adaptive capacities of humans and the environment have to be assessed.

Environment and climate factors (e.g. temperature, precipitation, information on climatic development) and the non-climatic factors (e.g. socio-economic trends, policies, capacities to deal with climate variability) make up the overall context and influence vulnerability, resilience, adaptive capacity and risks.

The standard procedure for assessing context within EbA is a vulnerability assessment. This tool is used to measure the vulnerability and resilience of a specific ecosystem (and its services), as well as the vulnerability, resilience and adaptive capacity of human communities. It forms the basis for outlining options and barriers to EbA measures. In order to capture the complexity and interaction of social, economic and environmental factors that underlie EbA, a vulnerability assessment of socio-ecological systems (VASES) as has been conducted for the project ‘Strategic Mainstreaming of Ecosystem-based Adaptation’ is recommended.

Click here to learn more about the VASES-approach.

For identifying the contribution of a measure to adaptation, ‘Adaptation made to measure’ suggests making use of the three dimensions Building adaptive capacity, Measure for reducing identified risks/vulnerabilities and Successful development despite climate change (sustained development). When taking economic, environmental and social factors into account, this step is also applicable to an EbA context.

By defining activities, outputs, outcomes, results/impacts and underlying assumptions in a results framework (also referred to as ‘logframe’), successful contribution to EbA can be monitored. However results frameworks often show processes in linear terms (as can be seen in the figure below), whilst in reality, processes of climate change and impact of EbA-measures are not linear. EbA requires an iterative, flexible and adaptive process to prevent mal-adaptation. Due to this complexity and dynamic character of EbA measures, it is recommended to take the results framework further and work with a Theory of Change approach to develop outputs, outcomes and impacts. This model allows for more intermediate re-evaluation, based on monitoring during the project, which needs to be an option during the project lifetime, as activities may change.

Based on the results framework, indicators can now be identified for short-term outputs, medium-term outcomes and long-term impacts. Here, it is important to include both qualitative and quantitative indicators, and to define all of these according to ‘SMART’ criteria. This can be achieved by first, defining the subject (taken from the afore developed results framework); second, specifying the quantity of change; third, specifying the quality of change; fourth, defining a time horizon; fifth, specifying disaggregation (i.e. by gender, geographical reference) if applicable; and finally, combining all five steps into one subject-specific indicator for short, medium- and long-term time frames. This procedure is repeated for each theme as defined in the results framework. The definition of indicators is crucial for the M&E process and thus needs to be done extremely thoroughly.

Baseline data should be gathered in the initial phase of a project implementation.

For useful operationalization of the M&E system, it is important to systematically monitor the change process. For this, data needs, data sources, the data collection method, data analysis method and responsibilities need to be identified. In addition to generating the above named criteria, an M&E plan with and for project or partner staff at different levels as well as training for project or partner staff needs to be developed to ensure the sustainability of the measures and their effects when a project is phased out.

When operationalizing a monitoring system, multiple challenges arise: First, EbA is often also related to changes in people’s awareness and capacity in terms of knowledge. Measuring this is only possible to a limited degree, as assessments can solely be done through qualitative interviews, which still will only reveal people’s actual knowledge on EbA-related topics (or lack of it) to a certain extend. Second, EbA measures often only prove effective after many years, and regularly in a time frame that lies outside of a project scope. It is thus highly important to prepare thoroughly described indicators, and to ensure a timely and all-encompassing handover to stakeholders who can monitor the activities over a longer time period and who will work with the results of the M&E .

Furthermore, unexpected changes and divergences from planned developments are normal and inevitable when working with a complex approach like ecosystem-based adaptation, where elements of vulnerability and resilience of nature, economy and society all need to be taken into consideration. This point is factored in when making use of a results framework in style of a Theory of Change (step 3) which allows for changes in planned outputs, outcomes and impacts. Here, it is core to be open and pay attention to such changes, as well as to understand their origins.