1.0 INTRODUCTION

Monitoring and Evaluation Approaches and Methods:
CABMACC Programme will ensure that the evaluation methodologies reflect the highest professional standards as outlined in the Standards for Evaluation in the UN System (UNEG, 2005). In addition, evaluation processes, will ensure that evaluation is conducted in an objective, impartial, open and participatory manner, based on empirically verified evidence that is valid and reliable, with results being made available. The programme will further ensure that the evaluation function is dynamic, adapting to new developments and changing needs both within and outside the organization. Most Significant Change: The programme will also use Appreciative Inquiry (AI) and Most Significant Change (MSC) Approaches. AI is preferred because, it analyses what has worked best in the past and envision what an organization and/or institution, want in future and then build from what has worked best in order to achieve the proposed goal. MSC deals with the unexpected by drawing meaning from actual events, rather than being based on indicators. The method involves systematically collecting stories which are then analyzed, discussed and verified. The stories capture changes in the lives of ‘beneficiaries’, their colleagues and in the character of their participation. The method also helps to identify why change happens. Therefore a critical analysis of institutional strengths as well as collected stories will highlight the successes and major lessons from the programme. Appreciative Inquiry is about the coevolutionary search for the best in people, their organizations, and the relevant world around them. In its broadest focus, it involves systematic discovery of what gives “life” to a living system when it is most alive, most effective, and most constructively capable in economic, ecological, and human terms. AI involves, in a central way, the art and practice of asking questions that strengthen a system’s capacity to apprehend, anticipate, and heighten positive potential. It centrally involves the mobilization of inquiry through the crafting of the “unconditional positive question” often-involving hundreds or sometimes thousands of people. In AI the arduous task of intervention gives way to the speed of imagination and innovation; instead of negation, criticism, and spiraling diagnosis, there is discovery, dream, and design. AI seeks, fundamentally, to build a constructive union between a whole people and the massive entirety of what people talk about as past and present capacities: achievements, assets, unexplored potentials, innovations, strengths, elevated thoughts, opportunities, benchmarks, high point moments, lived values, traditions, strategic competencies, stories, expressions of wisdom, insights into the deeper corporate spirit or soul-- and visions of valued and possible futures. Taking all of these together as a gestalt, AI deliberately, in everything it does, seeks to work from accounts of this “positive change core”—and it assumes that every living system has many untapped and rich and inspiring accounts of the positive. Link the energy of this core directly to any change agenda and changes never thought possible are suddenly and democratically mobilized.

2.0 Objectives of the CABMACC Results Based M & E plan

The objectives of the M & E plan are to:

  1. Guide the systematic data collection, analysis, reporting, use and feedback at all project levels.
  2. Facilitate the standardization of M&E methodologies and tools among project implementation partners
  3. Define the selected M&E indicators in line with project objectives, outcomes/results, and targets.
  4. To enhance effective project management by creating a platform for partnership, networking and collaboration for sharing and utilization of information among stakeholders
  5. Identify capacity needs for the full implementation of the M&E Plan

The monitoring and evaluation shall be at all levels of the CABMACC management, such as at beneficiary, programmes Coordinating Office, Programme Advisory Committee. All projects will be feeding to the broader goals at the Programme level. This system will be implemented in a collaborative effort between all implementing partners from joint planning activities and programme development where key performance indicators and M&E functions will be put place. Unlike the traditional implementation monitoring which focuses on whether activities have been done, Results Based M&E focuses on what happens next after outputs have been achieved. The results based M&E plan has been conceived for the CABMACC project to ensure partners are also playing an active part in monitoring of their activities. Key elements related to this M&E plan are described in the sections below:

3.0 Roles and Responsibilities

The implementation of the CABMACC results based M&E plan is a responsibility of PCO in collaboration with the implementing partners (Projects). However all partners will play an important role in ensuring the following:
  1. Effective data collection, collation, analysis and reporting
  2. Regular M&E training and backstopping of partners
  3. Dissemination of M&E results to all stakeholders including communities
  4. Data use to inform programme strategies and allocation of funds
  5. Sufficient funding for M&E activities
  6. Periodic review of the M&E plan in alignment with changes in programme direction
4.0 Levels of CABMACC M & E

4.1 Programme level M & E

Programme M&E will be coordinated at the Programmes Coordinating Office. The M&E specialist will receive all quarterly and annual reports from the project grantees and shall synthesise and compile formal reports and submit to the PCO who will in turn submit to PAC. In line with the mandates of the PAC the PCO will float issues coming from field beneficiaries, compile them in a format ready for discussion and provision of direction by the PAC. The programme will first develop a reporting guideline for the programme management team and at grants level. The guidelines will be developed to show progress at all levels of implementation that will include; at impact level, outcome level, output level, activity level and input levels. The PCO will commission scheduled field implementation audits to be undertaken to ascertain the extent to which planned activities are actually taking place. This will ensure timely and responsive action at points where problems have arisen. Other external counterparts from the PAC institutions shall be involved in special monitoring and evaluation field visits. These will be responsible in making sure that CABMACC is in line with national policies. The major purpose / objective indicators which the programme will use to monitor and evaluate its achievement are listed below:
  1. Enhanced capacity by the University towards emerging local and global climate change perspectives.
    Indicators
    • Proportion of staff engaged /participation in climate change and related projects by end of the project.
    • Proportion of staff and students trained on climate change related issues by end of the project.
    • Proportion of government contribution towards higher education by end of the project.
    • Proportion of staff retained by LUANAR after training by end of the project.
    • Number of new projects on climate change attracted through increased skills/empowerment by end of the project.
  2. Increased knowledge, technologies and systems for climate change mitigation and adaptation
    Indicators
    • Proportion of new technologies generated (and approved) for climate change by end of the project.
    • Proportion of research areas generated that feed into policy and practice by end of the project.
    • Proportion of research prioritized agenda for climate change developed, and supported by government by end of the project.
    • Proportion of beneficiaries utilizing generated technologies for climate change adaptation and mitigation by end of project.
  3. Increased capacity on advocacy, networking and mainstreaming of climate change within policies/plans.
    Indicators
    • Proportion of climate change issues that have been considered in government policies and programmes by end of project.
    • Proportion of stakeholders/institutions carrying out climate change outreach and advocacy work.
    • Number of functional study circles, research and technology transfer nodes, committees and other innovative structures instituted by the end of the project.


4.2 Projects level M & E

M&E plans for all CABMACC supported projects (grantees) will be mirrored to the main programme’s M&E plan. The first level of monitoring falls under the responsibility of the project team leaders and project administrators, who will keep administrative and management records on an on-going basis. All projects will develop their own log frame which will be a basis for project monitoring and evaluation. Indicators for measuring some of projects will be adapted from “Impact and output indicators for agriculture, food security, nutrition, natural resources and fisheries/aquaculture projects/programmes in Malawi” produced by Ministry of Agriculture and Food Security. Data and information will be reported through management records that will compare the activities planned in their Logical framework matrix with those carried out in the field, with particular attention at the participation of beneficiaries.

Each project shall have its own M&E focal point person who shall be trained together with other team members in M&E aspects for easy following up of progress based on performance indicators. Projects will use performance tracking matrix in capturing all the indicators from the field. However, in each project it shall the responsibility of the focal point person to collect and disseminate all important project progress in collaboration with the project team leader.

Project reporting shall be on quarterly basis to the granter (Bunda – PCO) and the reports will be copied to the implementing EPA through the DADO. These quarterly and annual reports shall include financial and technical issues. These will be stipulated in the guidelines on Competitive Grants. A quality mechanism shall be put forward to assure proper accountability of funds. This mechanism will allow PCO not to release funds where anomalies have been identified in the project implementation. The PCO will commission scheduled short (one week to ten days) field implementation audits to be undertaken to ascertain the extent to which planned activities are actually taking place. This will ensure timely and responsive action at points where problems have arisen. For the field visits to be productive, standard clear guidelines will be established in order to help managers and desk officers to make the most of their time and carry out effective monitoring activities.

4.3 M&E plans for long term and short training

Under the thematic pillars of teaching and research in the CABMACC programme there will be a lot of long and short term trainings. Beneficiaries for these trainings will be staff and students under LUANAR as well as other implementing partners of CGS projects. LUANAR will identify students to be trained based on the project guidelines and its needs. The other category of students will be those directly linked to CGS projects. Under all these categories, students will be submitting progress reports on quarterly and annual basis to the University Registrar, Programmes Coordinator, Deans and their heads of department.

Research for these graduate students will at the end of their studies yield into development of technologies for adaptation and mitigation of climate change in Malawi. Therefore, students will enter a contract with the institution which will act as a bond between the student and the institution. This will ensure retention of these members of staff after completion of their studies. At the same time supported students will need to submit their research proposals which CABMACC will endorse if it befits contribution towards attainment of the programme goal. At the end of the studies supported students will be expected to publish two research articles in peer reviewed journals and disseminate their findings widely while a copy of their thesis and dissertation will be submitted to the PCO library as part of records. After short courses students will be expected to submit a report and supporting certificates of completion of their studies. 5.0 CABMACC Data Collecting Strategies

Collection of information in the field is not an end in itself and should not be seen as a passive exercise. Data and information regularly collected must be analyzed and this can be done at different levels. The first one, as seen, is the coordination offices level, where information coming from the field is further analyzed.

The Indicator Matrix will be developed clearly defining indicators for the different project components and outcomes, sources of data to generate the indicators, measurement tool, frequency of data collection, responsible body for data collection, and the method of measurement.

The programme will use various results based monitoring tools including the following: Performance tracking table; Project Beneficiaries listings; Experimental data collection sheets; Global Information System; Field visits; Formal Surveys; Case studies; Mid-term evaluation; Internal Impact Assessment Surveys and Focus Group Discussions (FGDs).

6.0 Reporting System

The data reporting system is structured at different levels to necessitate a high degree of information sharing and feedback. Routine data will be generated and recorded at the farmer/Clubs by Field Technicians and their Government Counterparts in the targeted districts. The data shall be collected in accordance with the designated report forms. Project holders will from time to time compile the data for project reporting based on the project outcomes/results and targets. Particular care will be taken to identify and resolve double counting issues on project beneficiaries over time, gender disaggregated data and gender sensitive indicators. Finalized collated reports are sent to the Monitoring and Evaluation Specialist in a standardized format. Feedback at all levels should be provided for clarification of project objectives and improvement.

7.0 Evaluation

The M&E plan for CABMACC offers a number of strategies for evaluation to assess the effectiveness of the whole project interventions as outlined below:

7.1 Process Evaluation

The process evaluation will help to determine activities implemented as planned, what resources are used, how many people are reached, and who the project is reaching. This will be carried out at project level through routine data collection at field level, district and country level. 7.2 Mid Term Evaluation

A midterm evaluation will be conducted midway through the project’s first phase to evaluate implementation and most of all draw lessons to improve implementation in achieving programme objectives. The midterm evaluation will be facilitated by an external consultant based on the NORADs terms of reference. 7.3 End and Impact Evaluation

CAMACC will follow a counter-factual impact evaluation method in order to better measure the changes emanating from implementation of the project. The counter-factual evaluation will enable the project to appropriately attribute the impact to either project interventions or other factors. To this effect, the project will conduct a baseline survey which will collect baseline data on all farmers, students, staff, researchers who will benefit from the project and a control group for which comparison will be made against before and after the project implementation. Internally the project may sanction an internal impact assessment otherwise a final evaluation will be facilitated by an external consultant again based on drafted NORAD terms of reference. 7.4 Data Storage and use

The programme will develop a Management Information System (MIS) that will be used by all project holders (grantees) for data storage and access. In order to feed into the MIS, the programme will carry out yearly outcome and impact assessment using the designed tools (PM&E). The assessments will involve qualitative and quantitative data collection, analysis and reflection during project implementation, which will help to identify best practices and measure progress. The starting point for data storage shall be an excel sheet capturing quantitative data on the key project indicators in line with a Performance tracking table. Similarly, qualitative information including case stories, best practices among others, shall be stored both electronically and in hard copies. The data shall be used for all partners and stakeholders as per requirements of the respective institutions.

8.0 The Logical Framework Matrix

PROGRAMME STRATEGY
OBJECTIVELY VERIFIABLE INDICATORS
MEANS / SOURCES OF VERIFICATION

GOAL:
Improved livelihoods and food security through innovative responses and enhanced capacity for climate change adaptation and mitigation in Malawi.

Key indicators:

  • Number of farming families capacitated for climate change adaptation and mitigation.
  • Number of families improve food sufficiency during the lean periods of December to February
  • 80 % LUANAR staff retained after being trained by the programme.
  • Number of beneficiaries adopting CABMACC technologies.

  • Baseline report
  • Programme reports
  • Institutional reports
  • Media reviews
  • Research reports
  • Projects resports

PROGRAM : PURPOSE/OBJECTIVE:

  1. Enhanced capacity of the University towards emerging local and global climate change perspectives.

  • 60% of staff engaged /participated in climate change activities.
  • Number of secondary school pupils participate in outreach programme on climate change .
  • 20 % of academic staff trained on climate change related issues .
  • Number of teaching materials on climate change issues developed.
  • 20 % of academic staff trained up to MSc & PhD .
  • Number of technical, administrative, procurement, library, ICT and support staff trained
  • 10 % of female students supported with scholarships

  • Baseline report
  • Project Reports
  • M&E reports
  • Media reports
  • Policy reports
  • Students assessment reports
  • External assessors reports

  1. Increased knowledge, technologies and systems for climate change mitigation and adaptation

  • 5 new technologies generated under CABMACC and approved by government by end of the programme.
  • 5 of peer reviewed journal articles published.
  • Number of people reached with IEC materials.

  • Project and M&E reports
  • Policy reports, crop estimates reports, NSO reports, NRCM reports
  • Research study reports

  1. Increased capacity on advocacy, networking and mainstreaming of climate change within policies/plans

  • Number of govt. depts. / organizations with climate change issues in their strategic plans .
  • Number of people attending public debates.
  • 15 functional community based structures (dialogue forums, functional study circles, research and technology transfer nodes, committees and other innovative structures) formed .
  • 40% of stakeholders/institutions carrying out climate change outreach and advocacy work .

  • Project reports
  • Rapid Assessments
  • Event reports (e.g. workshop, meetings)
  • Newsletters
  • Policy briefs, brochures, fliers
  • Policy review reports
  • Journal articles, publications
  • Training reports

PROGRAM OUTPUTS:
1.1 Improved capacity of LUANAR and key stakeholders on climate change adaptation and mitigation

  • 10 staff involved in study exchange visits between LUANAR and UMB .
  • 3 MSc and 2 PhD programs developed and implemented
  • Eight (8) members staff to attend short courses at UMB
  • Number of staff participating in 8 mentoring sessions for proposal development conducted by the end of the programme
  • Guidelines for mainstreaming gender into teaching, research and outreach programmes developed
  • 3 MSc and 2 PhD programs developed and implemented

  • Capacity building reports
  • Revised curriculum
  • Project reports/Fieldvisit reports
  • Training reports
  • Training needs assessment reports
  • Progress Reports

1.2 Gender mainstreamed within teaching and research programmes

  • Number of primary and secondary school teachers trained on climate change related issues
  • Three (3) career talks to promote climate change among girls in secondary schools conducted . (Number of students reached with career talks)
  • Number of extension workers trained on gender and climate change

  • Revised training curriculum/modules
  • Training reports
  • Completion Reports
  • Project reports
  • Manuals for gender and climate change

1.3 HIV/AIDS issues are mainstreamed into teaching and research programmes across LUANAR

  • Number of people using the HIV and Aids guide book.
  • Number of patients attending VCT per month week
  • Guidelines for mainstreaming HIV in teaching research and outreach programmes approved
  • Number of people reached with sensitization meetings conducted to the university wide community .
  • Number of people (or students?) participating in HIV and AIDs social learning forums

  • Revised training curriculum/modules
  • Training reports
  • Project reports
  • Manuals for HIV/AIDS and climate change

1.4 Improved information access, documentation and ICT services

  • Reduced theft of library books.
  • Reduced unauthorized access to library materials
  • Steady/reliable internet connectivity at Luanar
  • Subscription to e-journals done by the end of the programme
  • Number of learners accessing E-learning system;
  • Learning Management system (LMS) established and maintained .
  • Number of resource test books published
  • University Journal produced

  • Project reports
  • Delivery notes for the equipment.
  • MIS reports
  • Installation reports for equipment.
  • Equipment usage reports
  • Tendering documents

2.1 Innovative research, best bet practices and technologies developed on climate change adaptation and mitigation undertaken

  • Number of villages on carbon credits interventions
  • 10 project proposals developed and implemented with international collaborators .
  • 1 project proposal developed and implemented for efficient and effective renewable energy. (number of farming families using effective renewable energy)
  • Baseline indicators for CABMACC collected and institutionalized

  • Project progress reports
  • Dissemination reports
  • Journal articles
  • M&E reports
  • Meeting minutes
  • Certificates/Theses/ Dissertation
  • Baseline report

2.2 Commissioned & Postgraduate Research Grants on climate change established and promoted.

  • 6 research areas on carbon credits and other financing mechanisms commissioned
  • Number of identified new research areas for emerging issues
  • 5 research grants provided to PhD students on reviewed and prioritized research themes .
  • Number of project grantees trained in project management.
  • Number of promotional conservation of indigenous crops and landraces gardens established

  • Project reports
  • Research Grant reports
  • Proposals
  • Training reports

2.3 Research grants effectively managed

  • Stakeholders planning workshop conducted
  • Ratification, validation and sensitization of governance structures
  • Review criteria for selection of proposals and scholarship beneficiaries developed
  • Research agenda developed
  • Number of success responses to call for concept notes and proposals
  • Number of peer review meetings conducted

  • Workshop reports
  • Programme reports
  • Activity reports

3.1 Capacity of key stakeholders enhanced for climate change adaptation and mitigation

  • Number of policy makers trained on climate change issues
  • Number of stakeholders attending dissemination conferences
  • 4 policy briefs disseminated
  • Number of programme and research findings disseminated through radios and TVs
  • 150 farmers, government officials and NGOs trained on climate change by the end of the project.
  • Knowledge management systems for policy processes developed
  • Number of lead farmers trained
  • Open learning and short courses for policy makers developed (Number of learners on open learning courses)
  • CABMACC website hosted (Volume of traffic to the CABMAC website)

  • Project Reports
  • Newsletters
  • Policy briefs
  • Fliers, brochures
  • M&E reports
  • Policy review reports
  • Media reports
  • Field days/Agricultural fairs
  • Knowledge Management System

4.1 Programme Monitoring and Evaluation

  • Mid-term and end of programme evaluation and annual surveys conducted
  • Functional M and E system developed
  • Regular technical and financial monitoring and evaluation conducted

  • Evaluation reports
  • Annual reports
  • Quarterly reports