G l o b a l E n v i r o n m e n t F a c i l i t y
GEF/ME/C.27/1
October 12, 2005
GEF Council
November 8-10, 2005
Agenda Item 7 (a)
THE GEF
MONITORING AND EVALUATION POLICY
(Prepared by the GEF Office of Monitoring and Evaluation)
Recommended Council Decision
The Council, having reviewed document GEF/ME/C.27/1, The GEF Monitoring and
Evaluation Policy, decides:
- To approve the Policy subject to incorporation of comments made by Council at this
meeting and decisions on the GEF Management Action Record and the interaction
between the GEF Office of Monitoring and Evaluation and Council.
- To request the Office of M&E to develop proposals fully reflecting the independence
of the Office in the main documents of the GEF such as the Instrument.
- To request the Secretariat, the Implementing and Executing Agencies to implement
the strengthened minimum requirements for M&E that have been adopted through this
policy.
- That the final version of the Policy will be published as a self standing policy note on
the GEF website; the proposed final version will be forwarded to the Council for
approval on a no-objection basis before the end of 2005.
- To approve the change of name of the Office of Monitoring and Evaluation to the GEF
Evaluation Office.
- That this Policy replaces the Terms of Reference for an independent M&E Unit in
GEF/C.21/12/Rev.1.
- To request the GEF Office of Monitoring and Evaluation to develop appropriate
guidelines and procedures to implement the Policy.
- To request the GEF Office of Monitoring and Evaluation to continue and formalise its
consultative process with M&E partners in the GEF.
- To request the GEF Office of Monitoring and Evaluation to develop a proposal for an
M&E training program to be presented to the GEF June 2006 Council, in order to
introduce the new policy and minimum requirements for M&E to the appropriate staff.
Executive Summary and cover note
1.
Further to the November 2004 Council request, the GEF Policy on Monitoring and
Evaluation is presented attached to this cover note for review and approval by the Council. The
Policy addresses, among other issues, a new division of labor on M&E, a change in the name of
the Office and the promotion of internationally recognized standards for M&E in the GEF.
2.
The Policy was developed through a consultative process with the appropriate partners
involved in monitoring and evaluation at various levels in the GEF. Based on the GEF
Instrument, the GEF Operational Strategy and the Terms of Reference for an Independent
Monitoring and Evaluation Unit, the Policy was informed by a series of brainstorming
workshops, extensive interaction and meetings with partners, the Norms and Standards for
Evaluation in the UN System, the OECD Development Assistance Committee Criteria for
Evaluating Development Assistance and the development banks' Evaluation Consultation Group
good practice standards for evaluation. The Policy also reflects the experience of other donors
and similar agencies in developing evaluation policies.
3.
The Policy would become effective upon approval of the GEF Council, and remain valid
for the duration of the GEF-4 replenishment period. The minimum requirements will be
obligatory for all projects presented to Council from 1 July 2006 onwards.
4.
By the nature of the GEF network, this Policy has to cover more than one organization,
many of which have their own evaluation policies established by their respective evaluation
offices. It is expected to cover monitoring as well; an element which is normally addressed
through operational policies. Complexity is added by various levels at which monitoring and
evaluation takes place in the GEF- at project, portfolio, country, corporate levels, and by agency
and focal area.
5.
In June 2005, the Council endorsed a shift in portfolio monitoring responsibilities from
the GEF Office of M&E to the GEF Secretariat. The implications for monitoring and evaluation
of the recently approved Results Allocation Framework (RAF) are not yet clear. The Policy
leaves room for more details on portfolio monitoring and performance management in the
context of results-based resource allocation, to be incorporated further to discussions of the
operational consequences in the coming months. Further clarity in actual tasks and
responsibilities may lead to standards for monitoring at higher portfolio levels.
6.
The Policy is derived from past GEF Council discussions and decisions on minimum
requirements for monitoring and evaluation, including standards for project M&E plans;
performance indicators; data availability; reporting; and project terminal evaluations. For
information purposes, Annex A of the Policy contains an overview of how the Policy addresses
the Terms of Reference for an Independent M&E Unit. (GEF/C.21/12/Rev.1). Annex B contains
an overview of the minimum standards for monitoring and evaluation as adopted in the TOR and
how these relate to the minimum requirements in the Policy. Once the final version of the Policy
is approved, it will be issued as a stand-alone document, without annexes.
7.
Given that the Implementing and Executing Agencies have their own systems of rules
and regulations governing monitoring and evaluation of these activities, the Policy does not
prescribe norms and standards for them, but contains minimum monitoring and evaluation
requirements GEF-funded activities that they implement. For those parts of the GEF for which
the Council is directly responsible, i.e. the GEF Secretariat and the Office of Monitoring and
Evaluation, evaluation and monitoring norms and standards are proposed. Key principles that
reflect professional sound ways of conducting M&E are also included.
8.
Monitoring and evaluation form part of systems of oversight and accountability. Whereas
it covers development effectives and organizational performance, the Policy does not address
aspects of Trustee management of the GEF Trust Fund, financial and managerial audit or
investigation mechanisms, which are subject to other regulations of the GEF Instrument.
9.
The GEF Policy is expected to facilitate, over time, the enhancement and enforcement of
international standards for monitoring and evaluation, and promote synergies that increase
efficiency and effectiveness of GEF operations. The consultative process has already led to
increased participation of Agency Evaluation Offices in GEF evaluation, both in project
evaluation and corporate evaluation. Several GEF Agencies developed or revised their own
evaluation polices in parallel with the GEF monitoring and evaluation policy, thus providing an
excellent opportunity for increased collaboration.
10.
The process has also led to a clearer definition of roles and responsibilities in the system.
Within several agencies, responsibilities have shifted to differentiate between management and
monitoring, on the one hand and evaluation, conducted independently, on the other. As the
lynchpin of the system, the Office conducts independent evaluation. The actual monitoring
would be undertaken by the GEF Secretariat and the Agencies coupled with setting norms and
supporting oversight of both monitoring and evaluation throughout the system.
11.
It is proposed that the words "Monitoring and" be omitted from the name of the Office in
order to describe more accurately its core business. This would also bring its name in line with
the evaluation offices of other international institutions. Many of these offices have the same
responsibilities as regards setting minimum requirements and oversight of M & E systems on the
project level, yet do not reflect this in their names.
12.
The GEF faces particular challenges in aggregation and attribution of results, at the focal
area, strategic priority, operational program or country level. The M&E partners have agreed that
future portfolio monitoring would have to look for realistic approaches beyond roll-up of project
indicator data. The introduction of the Results Allocation Framework will also have implications
for portfolio monitoring which should be incorporated into the GEF Monitoring and Evaluation
Policy.
13.
By bringing together the various requirements and making them explicit and transparent,
the Policy will promote greater incentives for accountability for monitoring and evaluation in the
system. The new and strengthened minimum requirements for monitoring and evaluation cover
project design, the application of M&E at the project level, and project evaluation. The
consultative process has led to new agreements on the quality and independent validation of
project evaluation of full-sized projects, while requirements for cost-effective evaluation of
medium-sized projects are to be addressed by the Joint Evaluation of the activity cycle and
modalities. Until then, the current requirements to undertake MSP evaluations remain in effect.
2
14.
In preparing its next four-year rolling work program, the Office was requested by the
Council to "take into account the outcomes of the consultative process, the new policy, and the
new division of labor on monitoring and evaluation, which may lead to efficiencies in actual
costs". Thus far, the process has not led to direct cost reduction since the expected efforts for
M&E are strengthened. However, some economies of scale are likely though the increase of
quality of M&E, value for money and coverage of GEF in evaluation throughout the system.
Streamlining of responsibilities for M&E would also, over time, reduce inefficiencies and
diseconomies of scale. The Office will report on any budgetary gains or shortfalls in its annual
(four year rolling) work plan and budget.
15.
Proposals on the Management Action Record and interaction with the GEF Council are
presented separately to the Council November 2005 session. These elements will be included in
the final version of the Policy depending on Council decisions. Thus the final version of the
M&E Policy will be forwarded to Council members for approval on a no-objection basis before
the end of 2005.
16.
The Policy will be operationalized through additional guidelines and good practice
examples on specific subjects, such as operational procedures of the Office, systems for rating of
progress and results; use of indicators and baselines and M&E plans; the annual performance
review and terminal evaluations. All relevant guidelines will be made available under a policy
and procedure resource page of the website of the Office, and developed through the regular
consultative M&E mechanisms. The independence of the GEF Office of M&E is not yet fully
reflected in the main documents of the GEF such as the Instrument. For example, the Office is
not recognized as a separate entity within the GEF.
17.
To ensure enhancement of capacities on monitoring and evaluation, the GEF partners
have also identified the need for training and support on monitoring and evaluation. This would
specifically address the minimum requirements for M&E and norms and standards, and may be
targeted to Agency staff, project staff and country stakeholders. Given the range of GEF
activities and partners involved, any such training program would have to provide for innovative
ways of outreach, maximum use of electronic communication channels and materials, and
seizing existing opportunities for interaction with partners. The GEF Office of M&E would be
able develop a proposal for an M&E training program for the GEF June 2006 Council.
18.
In the implementation of the M&E Policy, the Office will continue to work with the
Agencies in the consultative process to identify potential gaps in policy and practice. The
consultative process on M&E issues will be formalized through regular consultations to be
convened at appropriate moments in the year, for example before or after other evaluation
meetings (such as the UN Evaluation Group or the Evaluation Coordination Group of the
International Financial Institutions). Optimum use will be made of existing opportunities to meet
and discuss M&E issues.
19.
This Policy does not contain the full budgetary requirements to be implemented, since
these are not yet known on all levels. The Office of Monitoring and Evaluation is fully funded by
the Council through the annual discussions of its budget and (four year rolling) work plan. The
GEF Secretariat also discusses its budget and work plan annually with Council. The
Implementing and Executing Agencies pay for M & E issues out of the corporate budget (for
3
Implementing Agencies) and project fees and/or allocations (for both). Neither the corporate
budget nor the project allocations/fee contain specific references to what should or could be
spent on M&E. In some cases, the Policy and its minimum requirements may represent
additionalities to the Agency's own systems and practices with organizational or financial
consequences. These may be brought to the attention of the GEF Council through the appropriate
channels.
4
G l o b a l E n v i r o n m e n t F a c i l i t y
THE GEF
MONITORING AND EVALUATION
POLICY
DRAFT
Table of Contents
1. Monitoring and Evaluation in the GEF...................................................................................................1
1.1
Background.......................................................................................................................................1
1.2 Evaluation in the GEF.......................................................................................................................2
1.3 Monitoring in the GEF......................................................................................................................4
2. Roles and Responsibilities ......................................................................................................................5
2.1 M&E Partners in the GEF.................................................................................................................5
2.2 GEF Council .....................................................................................................................................6
2.3 GEF Evaluation Office .....................................................................................................................6
2.4 GEF Secretariat.................................................................................................................................7
2.5 Implementing Agencies and Executing Agencies ............................................................................8
2.6 Scientific and Technical Advisory Panel (STAP).............................................................................9
2.7 Participating Countries......................................................................................................................9
2.8
Stakeholders....................................................................................................................................10
3. Evaluation Criteria and Minimum Requirements .................................................................................11
3.1 International Criteria and Minimum Requirements ........................................................................11
3.2 Monitoring and Evaluation Criteria ................................................................................................12
3.3 Minimum Requirements and Key Principles ..................................................................................12
4. The GEF Evaluation Office ..................................................................................................................16
4.1 Mission Statement and Core Principles ..........................................................................................16
4.2 Key Functions .................................................................................................................................16
4.3 Tasks and Procedures......................................................................................................................17
5. Use of Evaluations ................................................................................................................................19
5.1
Follow-up........................................................................................................................................19
5.2 Knowledge Sharing.........................................................................................................................19
Annexes
ii
1. MONITORING AND EVALUATION IN THE GEF
1.
Monitoring and evaluation in the GEF have the following overarching objectives:
a.
Promote accountability for the achievement of GEF objectives through the assessment of
results, effectiveness, processes and performance of the parties involved in GEF activities.
GEF results will be monitored and evaluated for their contribution to global environmental
benefits;
b.
Promote learning, feedback and knowledge sharing on results and lessons learned among
the GEF and its partners, as basis for decision-making on policies, strategies, program
management, and projects and to improve knowledge and performance.
1.1 BACKGROUND
2.
The Global Environment Facility is a financial mechanism for international cooperation, based on
partnerships, that provides new and additional grant and concessional funding to meet the agreed
incremental costs of measures to achieve agreed global environmental benefits in six focal areas:
biological diversity, climate change, international waters, land degradation (primarily desertification and
deforestation), ozone layer depletion, and persistent organic pollutants in developing countries and
countries with economies in transition. The GEF receives strategic and policy guidance from the GEF
Council on these six focal areas and from the Conference of Parties of the global environmental
conventions that the GEF serves as the financial mechanism1. Two of the ten GEF Operational Principles
for GEF work, as described in the 1996 GEF Operational Strategy, require GEF, among other things, to
ensure that its programs and projects are monitored and evaluated on a regular basis, and to maintain
sufficient flexibility to respond to changing circumstances and experience gained from monitoring and
evaluation activities.
3.
Monitoring and evaluation play an important role in the GEF. The GEF's mission in the global
environment requires it to be innovative or experimental and puts the network in a position to assess
global dimensions of environmental and development policies. The GEF is also pioneering institutional
relationships among International Finance Institutions, United Nations agencies in partnership with the
participant countries, international conventions, NGOs, and other organizations. A policy on monitoring
and evaluation must make full use of the combined capacities of the expansive GEF network, and the
respective comparative advantages of each GEF partner. The multiplicity of stakeholders also places a
premium on learning and improvement, by continuously sharing knowledge from monitoring and
evaluation, both within and among the GEF partners and with external stakeholders. The active
engagement of all key stakeholders will enhance capacity for monitoring and evaluation as well as its
utility.
4.
Monitoring and evaluation feedback allows the GEF to track progress in fulfilling its mission of
delivering global environmental benefits in its six focal areas. GEF projects are more likely to capitalize
on their innovative and catalytic role when they are fully integrated with results-based management and
where management activities are based on feedback from systematic monitoring and evaluation findings.
Monitoring and evaluation processes can help strengthen partnerships, participation and ownership
around GEF projects and concerns, which are essential principles of GEF operations and policies. As a
1 For more details on the GEF see: "Instrument for the Establishment of the Restructured Global Environment
Facility" ("GEF Instrument"), last updated in May 2004, GEF Operational Strategy (February 1996) and GEF
Operational Programs.
consequence, the GEF will emphasize the quality of monitoring and evaluation systems and ensure that
their findings are disseminated widely.
5.
A strong performance management system is essential in building confidence among partners
both stakeholders and beneficiaries - in the reliability of information on development effectiveness. By
making requirements and expectations more explicit, consolidated and transparent, the M&E policy
should provide incentives and consequences for conducting good monitoring and evaluation at various
levels of programming. This is particularly important given the specific challenges in measuring and
aggregating GEF global results.
6.
The monitoring and evaluation functions of the GEF were established after the GEF restructuring in
1994, when the GEF Council was entrusted with the responsibility for developing, adopting, and
evaluating the operational policies and programs for GEF activities. A framework for monitoring and
evaluation was approved in May 1997 as the Framework and Work Program for GEF's Monitoring,
Evaluation and Dissemination Activities (GEF/C.8/4). As a result of the Second Overall Performance
Study and Replenishment of the GEF Trust Fund, the GEF Monitoring and Evaluation Unit was made
independent in 2003 and now reports directly to the GEF Council. In November 2004, the GEF Council
renamed the Unit as the GEF Office of Monitoring and Evaluation and requested it to proceed with
developing a new policy for monitoring and evaluation in the GEF.
7.
This Policy aims to explain the concept, role and use of monitoring and evaluation within the GEF
network and define the institutional framework and definition of responsibilities. Specifically, it
establishes requirements for how GEF activities should be monitored and evaluated in line with
international standards, and assigns roles and responsibilities for these tasks. The Policy does not address
aspects of Trustee management of the GEF Trust Fund, financial and managerial audit or investigation
mechanisms, which are subject to other regulations of the GEF Instrument.
8.
The GEF Policy takes effect when approved by the GEF Council and shall remain valid for the
duration of the GEF-4 replenishment period. To ensure that the Policy remains relevant to evolving
circumstances and captures international trends in monitoring and evaluation, it may be revised
periodically. Any proposals for changes in the Policy will be presented to the Council by the GEF
Evaluation Office.
9.
The Policy will be operationalized through guidance on specific issues and standards, developed by
the GEF Evaluation Office in consultation with partners. The Office is authorized to publish and revise
such guidelines, as required. The Policy and related guidelines will be shared with the GEF network and
the public through the GEF Evaluation Office website.
1.2 EVALUATION IN THE GEF
10. Definition. An evaluation is a systematic and impartial assessment of an activity, project, program,
strategy, policy, sector, focal area or other topics. It aims at determining the relevance, impact,
effectiveness, efficiency and sustainability of the interventions and contributions of the involved partners.
An evaluation should provide evidence-based information that is credible, reliable and useful, enabling
the timely incorporation of findings, recommendations and lessons into the decision-making processes.
11. Use of evaluation. Evaluation feeds into management and decision making processes regarding the
development of policies and strategies, and the programming, implementation and reporting of activities,
projects and programs. Thus evaluation contributes to institutional learning and evidence based policy-
making, accountability, development effectiveness and organizational effectiveness. Evaluation informs
2
the planning, programming, budgeting, implementation and reporting cycle. It aims at improving the
institutional relevance and the achievement of results, optimizing the use of resources, providing client
satisfaction and maximizing the impact of the contribution provided.
12. Types. The evaluation approach and method must be adapted to the nature of the undertaking.
Within the context of the GEF, the main types of evaluations include:
a.
Project evaluations of projects under implementation; at the end of the intervention
(terminal evaluation); and after the project end (ex-post evaluation) or before project start (ex
ante quality at entry).
b.
Program evaluations of a set of interventions to attain specific global, regional, country or
sector objectives. These
evaluat
include
ions or studies of the GEF focal areas, Operational
Programs or strategic priorities.
c.
Country program evaluations of one or more agency's portfolio of projects and activities,
and the assistance strategy behind them, in a partner country. These include country portfolio
evaluations or reviews assessing GEF achievements in countries.
d.
Impact evaluations of the long term effects produced by an intervention, intended or
unintended, direct or indirect. Impact may be assessed at project, program, portfolio and
country levels, and includes global environmental benefits.
e.
Cross-cutting and thematic evaluations of a selection of interventions, all of which address
a specific concern to all or several countries, regional and sectors. These include studies that
assess topics
rati
GEF ope
onal programs, such as participation, gender, capacity building,
policy, or technology.
f.
Process evaluations of the internal dynamics of participating organizations, instruments,
mechanisms and management practices. These include evaluations of institutional and
procedural issues acros G
s EF focal areas; and assessments of experience with GEF policies,
criteria and procedures.
g.
Overall Performance Studies (OPS) of the GEF, connected to the GEF replenishment and
Assembly cycles. These address overriding issues like global impact and benefits of GEF
programs, as well as the GEF institutional arrangements, policies, strategies, programs and
priorities.
13. Purposes of evaluation include understanding why and the extent to which intended and
unintended results are achieved, and their impact on stakeholders. Evaluation is an important source of
evidence of the achievement of results and institutional performance and contributes to knowledge and to
organizational learning. Evaluation should serve as an agent of change and play a critical role in
supporting accountability. Evaluation can be used to improve the design and performance of an ongoing
program (a formative evaluation); to make an overall judgment about the effectiveness of a completed
program, often to ensure accountability (a summative evaluation); and to generate knowledge about good
practices. It should help GEF to position itself to better address the pursuit of global environmental
benefits. Evaluation differs from other oversight mechanisms such as investigation and audit that focus on
the adequacy of management controls; compliance with regulations, rules and established policies; and
the adequacy of organizational structures and processes.
3
1.3 MONITORING IN THE GEF
14. Definition. Monitoring is a continuous or periodic function that uses systematic collection of data,
qualitative and quantitative, for the purposes of keeping activities on track. It is first and foremost a
management instrument.
15. Use of monitoring. Monitoring provides management and the main stakeholders of an ongoing
development intervention with indications of the extent of progress and achievement of objectives and
progress in the use of allocated funds. It provides regular feedback on program performance taking into
account the external environment. Information from systematic monitoring serves as a critical input to
evaluation.
16. Levels. Within the context of the GEF, monitoring may take place on three levels:
a.
Project level mainly of implementation process and activities, the delivery of outputs, and
progress towards outcomes.
b.
Portfolio level mainly of trends in implementation; and outcome, namely the short- or
medium term effects of an intervention's outputs. This includes monitoring of focal areas and
overall results for the GEF as well as monitoring of institutional issues.2
c.
National and global level mainly of global environmental impact, based on independent
data gathering and analysis by national bureaux of statistics and/or international bodies and
organizations.
17. Purposes of Monitoring include providing early information on progress or lack thereof towards
achieving the intended objectives, outcomes and impacts. By tracking progress, monitoring helps to
identify implementation issues that warrant decisions at different levels of management. A good
monitoring system combines information from various levels organizational, portfolio and project in
such a way that it provides a comprehensive picture of performance and allows periodic reports to the
management that facilitate decision-making and learning.
2 The systems for portfolio monitoring will be further defined over the coming months, following the approval of the
Results Allocation Framework and the shift of portfolio monitoring responsibilities to the GEF Secretariat, and
issued in guidelines and/or updates to the Policy.
4
2. ROLES AND RESPONSIBILITIES
2.1 M&E PARTNERS IN THE GEF
18. Monitoring and evaluation are a shared responsibility in the GEF network. On different levels, for
different partners and involving different functions within the GEF, a complex picture emerges of who is
involved and what will be done. This has been termed the "M&E" pyramid of the GEF network: A wide
base of project monitoring and evaluation taken care of by the Implementing and Executing Agencies and
their partners; a mid-level tier of indicators in focal areas, portfolio reviews, thematic and cross-cutting
evaluations and annual performance reports in which the Secretariat and the Evaluation Office add their
efforts to those of other GEF partners; and at the top emerging environmental and development trends and
the GEF results within these trends, as reported on in the Overall Performance Study.
Table: M&E levels and responsible agencies in the GEF
Overal l
Per formance
Stud y
Thematic and
cros s-cutti ng
eval uations,
impact assessments,
country portfolio reviews
APR
Portfolio and program reviews
Focal ar ea indi cators
Annual PIR, Projects-at-r isk systems,
Super vi si on
Project indicators
Monitoring
Mid-term and final evaluations
19. The essence of a network of partners is that each has his own system of governance and rules and
regulations governing the implementation of activities, as well as the monitoring and evaluation of these
activities. The GEF Council can adopt principles, norms and standards for those parts of the network for
which it is directly responsible, such as the Secretariat, the Evaluation Office and STAP, but it cannot
enter into the sovereign responsibilities of the partners in the GEF and adopt principles, norms and
standards for them. However, the GEF can require minimum standards and minimum procedures to be
applied to the activities it (co-)funds. For this reason this policy contains principles, norms and standards
for the work of the Secretariat in monitoring and for the work of the Evaluation Office, but not for the
Implementing and Executing Agencies and other partners. For these organizations, so called minimum
requirements on monitoring and evaluation are formulated for the GEF projects that they implement.
5
20. This chapter contains a brief description of the roles and responsibilities of each of the GEF
partners in monitoring and evaluation, reflecting mandate, comparative advantage and the recent shift in
M&E responsibilities3. Under the overarching guidance and oversight of the GEF Council to ensure that
the monitoring and evaluation functions in the GEF are properly assigned and conducted, the GEF
Evaluation Office undertakes corporate evaluation work and supports the application of internationally
accepted norms and standards. The Office works closely with the evaluation departments of the Executing
and Implementing Agencies to enhance the combined capacity of the GEF system to effectively and
efficiently fulfill evaluation needs. In fulfilling their management functions, the Agency operational
departments and the GEF Secretariat ensure monitoring of and reporting on progress and results at the
project and consolidated portfolio levels, respectively. In line with the GEF Instrument, both monitoring
and evaluation processes must fully draw on the capacities and knowledge of scientific advisers, program
governments, local stakeholders and beneficiaries.
2.2 GEF COUNCIL
21. The governance structures of monitoring and evaluation reside with the GEF Council. The Council
develops, adopts and evaluates the operational policies and programs for GEF-financed activities, in
conformity with the GEF Instrument and fully taking into account reviews carried out by the GEF
Assembly. It also reviews and approves the policy for monitoring and evaluation.
22. The GEF Council shall provide an enabling environment for monitoring and evaluation activities in
line with internationally accepted standards. The GEF Council is responsible for ensuring that adequate
resources are allocated to enable the evaluation function to operate effectively and with due
independence; that evaluators have the freedom to conduct their work without repercussions for career
development; and for appointing a professionally competent Director of Evaluation. It promotes
transparency, participation and disclosure in monitoring and evaluation findings.
23. The GEF Council ensures accountability and oversight of GEF performance. It keeps under review
the operation of the GEF with respect to its purposes, scope and objectives; ensures that the GEF policies
and work program, including operational strategies and projects, are monitored and evaluated on a regular
basis. The Council uses monitoring and evaluation to complement a larger system of financial oversight
and accountability within the GEF Trustee and Agencies. On behalf of the Council, the GEF Trustee
ensures the maintenance of appropriate records and accounts of the Fund, and providing for their audit, in
accordance with the rules of the Trustee.
24. The GEF Council, together with the CEO and the Director of Evaluation, are responsible for
ensuring active use of monitoring and evaluation products for decision making and management, through
an M&E planning system; systematic consideration of findings, conclusions and recommendations and
repositories of lessons learnt.
2.3 GEF EVALUATION OFFICE
25. Within the GEF network, the GEF Evaluation Office has the central role of ensuring the evaluation
function within the GEF, of setting minimum requirements for monitoring and evaluation, of ensuring
oversight of the quality of M&E systems on program and project level and of sharing evaluative evidence
within the GEF network.
3 GEF Council decision based on GEF/ME.C.24, Elements of a new M&E Policy.
6
26. The Office has the responsibility for undertaking independent evaluations which involve a set of
projects from more than one Implementing or Executing Agency. These evaluations are typically on a
strategic level, or on focal areas, or on cross-cutting themes. Furthermore, institutional evaluations are
undertaken. Where possible and to prevent duplication, the Office will collaborate in these evaluations
with (independent) evaluation offices of the Implementing and Executing Agencies.
27. Within the GEF system, the GEF Evaluation Office facilitates cooperation with and between the
GEF partners on matters of evaluation and monitoring. This includes the establishment of procedures and
guidelines on evaluation of GEF matters, based on internationally recognized standards, and collaboration
with the GEF Secretariat and Agencies to establish requirements for portfolio monitoring.
28. In support of the Council's oversight role, the GEF Evaluation Office provides the Council with
periodic information on the quality of M&E systems in the GEF. This information is presented in an
annual performance report and is based on evaluative evidence developed by the GEF Evaluation Office,
the Agency evaluation departments, or by operational units and reviewed by independent quality
assurance mechanisms. The Office also reviews project terminal evaluations submitted by the Agencies.
The report focuses on the ex-post results of GEF projects
ds in the co
and tren
mpliance with the minimum
requirements on project design of M&E, application of project M&E and on project evaluation. It may
also cover trends in the quality of portfolio monitoring in future.
29. The GEF Evaluation Office supports knowledge sharing and follow-up of evaluation
recommendations. It works with the GEF Secretariat, the IAs and EAs to establish systems to disseminate
lessons learned and best practices emanating from the monitoring and evaluation activities, and provides
independent evaluative evidence to the GEF knowledge base.
30. The GEF Director of Evaluation is accountable directly to the GEF Council for the work of the
Office, and may propose to the Council any measure that he or she believes is necessary to ensure
evaluation independence. In line with the Terms of Reference4, the Director manages the GEF Evaluation
Office and its budget by implementing strategic decisions by the GEF Council; providing overall
agement of re
direction and man
strengt
sources; and
hening institutional relationships. The Director is
solely
GE
responsible for personnel decisions in the
F Evaluation Office in accordance with staff rules.
2.4 GEF SECRETARIAT
31. The GEF Secretariat is responsible for monitoring the overall GEF portfolio that covers all focal
areas and Executing or Implementing Agency projects. This may entail aggregation of findings for project
focal areas, operational programs, agency or project type. Based on information for the Executing and
Implementing agencies on their on-going projects during the y
the GEF Secretariat
ear,
produces an
implementation review in which it presents an overview of progress towards results, including outcomes,
implementation issues and portfolio-wide trends to the GEF Council.
32. In support of effective monitoring, the GEF Secretariat takes the lead in the identification of
portfolio- or program-level indicators and the use of data for monitoring performance on program
indicators. It reviews all projects prior to their appr al to ensur
ov
e that they meet GEF monitoring and
evaluation standards and policies. It also works with the GEF Evaluation Office in establishing
monitoring standards for project and portfolio levels.
4 GEF/C.21/12.Rev.1, Terms of Reference for an Independent Monitoring and Evaluation Unit, July 28, 2003, Annex I
7
33. The GEF Secretariat takes the lead in developing GEF knowledge management systems that uses
monitoring and evaluation information. It facilitates cooperation in comprehensive monitoring and
learning at portfolio level, by bringing together relevant partners in task forces; establishing mechanisms
and systems for knowledge capture and dissemination.
34. In support of evaluation, the GEF Secretariat responds promptly and fully to GEF Evaluation Office
requests for information relating to GEF projects, coordinates the GEF system management response to
corporate evaluations; provides certain administrative support for the GEF Evaluation Office and consults
with the Evaluation Office when conducting reviews of GEF-related aspects.
35. In its managerial capacity, the GEF Secretariat ensures that findings and recommendations
emanating from evaluation and monitoring activities are followed up with regard to GEF policies,
programs and procedures, and that related Council decisions are implemented. The Secretariat ensures
that results and lessons identified through monitoring and evaluation activities are adequately reflected in
public information about the GEF. Its activities of governance, program management, and relations with
constituents are covered by the GEF corporate budget. This includes preparation of the annual GEF
Project Implementation Review; activities to gather and disseminate best practices to improve portfolio
quality
ster replicatio
and fo
n; providing information required by the Evaluation Office and preparing joint
management responses to evaluations. 5
2.5 IMPLE
N
ME TING AGENCIES AND EXECUTING AGENCIES
Operational Units
36. The Agencies are responsible for monitoring and supervising the project activities, the production
of outputs and the progress towards outcomes. Through their internal monitoring systems, the relevant
Agency operational departments ensure periodic assessment of trends and issues in their GEF Agency
portfolio, and periodic reporting to the GEF Secretariat on project implementation. They also develop and
use project indicators, and work with the GEF Secretariat in developing program indicators in focal areas
where operational policies and programs have been endorsed.
37. The Agencies are responsible for ensuring that projects are evaluated periodically and in line with
internationally recognized standards, and that any project or portfolio evaluations conducted are shared
with the GEF Evaluation Office. The Agencies supports the GEF Evaluation Office by responding
promptly and fully to requests for information or support relating to monitoring or evaluation of GEF
projects, and by making project evaluations publicly accessible.
38. The Agencies work with the departments of the other GEF partners to exchange lessons learned and
information, and incorporate lessons learned into their operational policies, program
ojects as
s or pr
appropriate. They also encourage public involvement in all stages of the project cycle, by fully consulting,
informing and briefing GEF participating countries and stakeholders regarding M&E activities.
39. The three Implementing Agencies receive a GEF corporate budget. All Agencies receive project
allocations and project fees. Project allocations cover the costs of goods, work and services procured by
GEF grant recipients as part of the preparation and implementation of projects, including specific
activities to undertake monitoring and evaluation. Project fees allow Implementing and Executing
Agencies to provide project cycle management services related to the GEF projects they manage. These
services include: portfolio development and management by regional and operational units; project
5 GEF/C.25/7, Corporate Budget, FY06
8
identification; assistance to recipient countries in their project development and preparation; appraisal of
project proposals and negotiation of GEF co-financed operations; supervision of GEF projects,
preparation of implementation completion reports; reviews by the agency's evaluation units. The
Agencies also provide inputs to the GEF Evaluation Office and prepare joint management responses. 6
Evaluation Units
40. The evaluation departments of Agencies shall exchange mutually their evaluation agendas or work
plans with the GEF Evaluation Office to seek possible areas of common interest and cooperation, and
possible joint evaluations. They shall encourage optimal coverage of environmental related issues in their
evaluation plans. For relevant evaluations covering issues of GEF concerns and GEF portfolio, the
evaluation departments shall provide opportunities to the GEF Evaluation Office to interact with regard to
TOR, approach and scope. Where a notable GEF portfolio exists, the Agency corporate evaluations shall
integrate and reflect this as much as possible, for example in their country portfolio evaluations, impact
evaluations and thematic evaluations. The Agency evaluation offices will also cooperate on norms,
standards and quality of evaluations. Bilateral consultations will be organized between the GEF
Evaluation Office and Agency evaluation offices to address any systemic issues, including budgetary
issues. If additional funding is required for these units to be involved in the GEF, this will either be met
through internal procedures of the Agency concerned or through inclusion of the necessary funding in the
corporate budget or the project allocations or fees, without in any way detracting from the independence
of these evaluation units.
2.6 SCIENTIFIC AND TECHNICAL ADVISORY PANEL (STAP)
41. The GEF Scientific and Technical Advisory Panel (STAP) will provide timely and relevant advice
on scientific and technical matters related to monitoring and evaluation activities. The Chair of STAP
takes part in meetings and consultations on monitoring and evaluation in the GEF.
42. STAP provides advice on the work program of the GEF Evaluation Office related to evaluations
with components on science and technology, and suggestions on such subjects to evaluate. It may also
provide opinions on the evaluability of scientific aspects and related methodologies for measuring global
environmental impacts, in response to evaluation approach papers, Terms of Reference or reports. STAP
members may also be called upon to support directly an evaluation, while respecting the independence of
both STAP and the GEF Evaluation Office.
43. The Panel will also support monitoring of scientific and technical aspects of the GEF, through
knowledge management and information sharing; for scientific and technical evaluation of the portfolio.
h
STAP shall support t e GEF Secretariat in the development and use of scientific indicators to measure
impact at national and portfolio levels. Its work is covered by the GEF corporate budget.
2.7 PARTICIPATING COUNTRIES
44. A number of entities in GEF participating countries are involved in monitoring and evaluation in
different ways. The GEF participating countries are responsible for national monitoring, evaluation and
assessment systems on global environmental benefits and for establishing a favorable national monitoring
and evaluation framework. This may include efforts to improve basic census and other data in partner
countries; establishing national and project baselines; using national communications and inventories of
global environmental benefits; participating in various global initiatives such as the Global Reporting
6 GEF/C.25/7, Corporate Budget, FY06
9
Initiative and monitoring of the Millennium Development Goals, with the support of development
partners as appropriate.
45. In line with the GEF Operational Principles, GEF M&E activities shall be country-driven and
provide for consultation and participation. Staff members of the cooperating governments or institutions
will be expected to support evaluations by responding promptly and fully to evaluation office requests for
information relating to GEF projects, portfolio or policies, to participate actively in project monitoring as
set out in the project M&E plan, and to share relevant experiences. The GEF Focal Point have a particular
responsibility for use, follow-up and action on evaluation recommendations related to GEF matters and
directed at the regional, national, local and project levels and for integrating lessons into project
proposals. The Focal Point also plays a key role in keeping stakeholders fully consulted, informed and
involved about the plans, the implementation and the results of country-related GEF monitoring and
evaluation activities.
2.8 STAKEHOLDERS
46. A number of locally and internationally-based stakeholders are involved in GEF monitoring and
evaluation activities, including sub-national authorities, cooperating institutions, local community groups,
non-governmental organizations, civil-society organizations, the private sector, academic institutions, and
co-financiers that supplement GEF resources in particular projects. GEF-financed projects shall provide
for full disclosure of all non-confidential information, and consultation with major groups and local
communities in monitoring and evaluation.
47. Consistent with provisions in the Instrument, there should be transparency in the preparation,
conduct, reporting, and evaluation of public involvement activities in all projects, including for
monitoring and evaluation. Monitoring and evaluation in the GEF shall involve project stakeholders and
beneficiaries, both as participants and contributors and as users and beneficiaries. The stakeholders have a
particular responsibility in providing feedback. They use monitoring and evaluation to assess progress,
raise issues or confirm the achievement of results, to improve performance and learning. In the design of
monitoring systems and in the terms of reference for evaluations, the specific possibilities for interaction
with stakeholders and participation of the stakeholders will be identified, taking account of country
conditions, such as cultural, political, and project-specific factors. Any budgetary requirements will be
addressed in the relevant project proposals.
10
3. EVALUATION CRITERIA AND MINIMUM
REQUIREMENTS
3.1 INTERNATIONAL CRITERIA AND MINIMUM REQUIREMENTS
48. The work of the Implementing and Executing Agencies in monitoring and evaluation is in various
degrees guided by internationally recognized principles, norms and standards. Almost all IAs and EAs
have well formulated policies and regulations, which contain norms and standards. Although there is a
general convergence towards internationally recognized norms and standards, there is also a divergence
caused by the specific goals and objectives of the IAs and EAs. These different goals call for differences
in emphasis and for differences in application of standards between agencies. This means that it is
impossible to formulate precise principles, norms and standards which are common throughout the GEF
network. Nevertheless it is expected that in future more convergence may appear, due to developments in
the UN evaluation system and in the system of the Banks.
49. The UN Evaluation Group has recently adopted professional norms and standards for evaluation.
These norms and standards have been compiled taking into account the "state of the art" in evaluation in
the bilateral community (in the OECD DAC Evaluation Network) and in the Evaluation Coordination
Group of the Banks. UNEG proposes that each UN agency adopts an evaluation policy in which the
norms and standards will be translated to the specific situation of that agency. Furthermore, a system of
peer reviews will be developed to help each agency achieve better performance and better adaptation of
the UNEG norms and standards.
50. The Evaluation Coordination Group of the Banks is following a different route. This Group has not
adopted any professional norms and standards, but has benchmarked "best practices" in evaluation in
several subject areas, in order to harmonize and improve evaluation performance throughout the Banks. A
special issue is the independence of evaluation, which has received strong attention and which has led to
formulation of a template for independence and a peer review on independence, which allows each Bank
to fine-tune its organizational set-up, if and when necessary.
51. The OECD/DAC Evaluation Network has for more than a decade been the most active and
authoritative forum for discussing professional norms and standards in evaluation of development and
grant related issues. The DAC Principles for Evaluation of Development Assistance, which were adopted
by the DAC High Level Meeting in 1991, remain to this day the internationally best known principles and
criteria for evaluating grants. Currently, the DAC Evaluation Network is working on minimum standards
for evaluations and on a peer review process to assess the quality of evaluations and the quality of
evaluation systems. This peer review system will be extended beyond the bilateral evaluation community
to the UN system and eventually to the Banks.
52. On monitoring, no professional norms and standards have been formulated in the bilateral, UN or
Bank communities. However, it is common to formulate minimum requirements for monitoring systems:
that projects shall have them; that they need to be tied into the logical framework targets and indicators as
much as possible, and so on. However, it is also recognized that in general monitoring systems are
"project specific", i.e. need to be designed to fit into the specific circumstances of the projects.
53. A key international norm concerns the adequate provision of resources to enable the monitoring and
evaluation functions to operate effectively. Planning for monitoring and evaluation must be an explicit
part of planning and budgeting at the project level and for the organization as a whole. Monitoring and
evaluation in the GEF should be managed to ensure cost-effectiveness in terms of adding value to the
11
portfolio. The costing and budgeting of monitoring and evaluation activities shall be addressed, as
appropriate, in the budgetary planning of the independent GEF Evaluation Office; the GEF Corporate
Budget, the Agency fee system and in project budgets. This would include any additional financial
implications of addressing the minimum requirements and responsibilities of this Policy.
3.2 MONITORING AND EVALUATION CRITERIA
54. GEF projects and progra s
m will adopt monitoring systems, including planning for relevant
performance indicators, that are:
a.
Specific: The system captures the essence
ir
of the des ed result by clearly and directly relating
to achieving an objective and only that objective.
b.
Measurable: The monitoring system and indicators are unambiguously pecif
s
ied so that all
parties agree on what it covers and there are practical ways to measure it.
c.
Achievable and Attributable: The system identifies what changes are anticipated as a result
of the intervention and whether the result(s) ar
n requires that changes
e realistic. Attributio
in
the targeted developmental issue can be linked to the intervention.
d.
Relevant and Realistic: The system establishes levels of performance that are likely to be
achieved in a practical manner, and that reflect the expectations of stakeholders.
e.
Time-bound, Timely, Trackable and Targeted: The system allows progress to be tracked
in a cost-effective manner at desired frequency for a set period, with clear identification of
the particular stakeholder group to be impacted by the project or program.
55. In general, evaluations in the GEF explore five major criteria, not all of which need to be
systematically reviewed in all cases:
a.
Relevance: The extent to which the activity is suited to local and national development
priorities and organizational policies, including changes over time.
b.
Effectiveness: The extent to which an objective has been achieved or how likely it is to be
achieved.
c.
Efficiency: The extent to which results have been delivered with the least costly resources
possible. Also called cost-effectiveness or efficacy.
d.
Results: The positive and negative, and foreseen and unforeseen, changes to and effects
produced by a development intervention. In GEF terms, results include direct project outputs,
short- to medium term outcomes, and longer-term impact including global environmental
benefits, replication effects and other, local effects.
e.
Sustainability: The likely ability of an intervention to continue to deliver benefits for an
extended period of time after completion. Projects need to be environmentally as well as
financially and socially sustainable.
3.3 MINIMUM REQUIREMENTS AND KEY PRINCIPLES
56. The following minimum requirements shall be applied to monitoring and evaluation on the project
level.
12
Minimum requirement 1: Project Design of M&E
All projects will include a concrete and fully budgeted monitoring and evaluation plan by the time of CEO approval.
This monitoring and evaluation plan will contain as a minimum:
·
SMART indicators for project implementation, or if no indicators are identified, an alternative plan for monitoring
which will deliver reliable and valid information to management
·
SMART indicators for results (outcomes and if applicable impacts), and where appropriate indicators identified at
the corporate level
·
baseline for the project, with description of problem to address, with indicator data, or if major baseline indicators
are not identified, an alternative plan for addressing this within one year of implementation
·
identification of reviews and evaluations which will be undertaken, such as mid-term reviews or evaluations of
activities
·
organizational set-up and budgets for monitoring and evaluation.
57. GEF project objectives and intended results should be specific and measurable, so as to make it
possible to effectively monitor and evaluate the project. The baseline data would be developed for the key
results indicators. In rare cases, further development of some baseline indicators may be completed during
the first year of implementation. The presence of the M&E plan and baseline would be considered as a
performance measure of satisfactory M&E in the first Project Implementation Report. Agencies may
encourage attention at the PDF stage to ensure timely M&E planning.
Minimum requirement 2: Application of Project M&E
Project monitoring and supervision will include implementation of the M&E plan, comprising:
·
SMART indicators for implementation are actively used, or if not, a reasonable explanation is provided
·
SMART indicators for results are actively used, or if not, a reasonable explanation is provided
·
the baseline for the project is fully established and data compiled to review progress reviews and evaluations are
undertaken as planned
·
the organizational set-up for M&E is operational and budgets spent as planned.
58. GEF project monitoring provides Agency management with a basis for decision-making on
progress and the GEF with information on results. In order to be used for conclusions and decisions,
monitoring would use both qualitative and quantitative data to report accurately on the production of
outputs and progress towards outcomes; identify key implementation issues and proposes actions to solve
these. Periodic reports should be based on a principle of continuity to allow for tracking of results and
progress. To be valid, monitoring should be based on periodic observation visits, capture the views of
stakeholders, and explain any methodological limitations of its use of sources and data. M&E plans are
dynamic tools and should be revised if the project scope changes significantly.
Minimum requirement 3: Project Evaluation
Each Full Sized Project will be evaluated at the end of implementation. This evaluation will have the following
minimum requirements:
·
the evaluation will be undertaken independent of project management, or if undertaken by project management,
will be reviewed by the evaluation office of the Implementing or Executing Agency or by independent quality
assurance mechanisms of the Agency
·
the evaluation will apply the norms and standards of the Implementing or Executing Agency concerned
·
the evaluation will assess at a minimum:
o achievement of outputs and outcomes, and provide ratings for targeted objectives and outcomes
o likelihood of sustainability of outcomes at project termination, and provide a rating for this
o whether minimum requirements for M&E 1 and 2 were met provide a rating for this
·
the report of this evaluation will contain as a minimum:
o basic data on the evaluation:
when the evaluation took place
who
was
involved
the
key
questions
methodology, including application of the five evaluation criteria
13
o basic data of the project, including actual GEF and other expenditure
o lessons of broader applicability
o the TOR of the evaluation (in an annex)
·
the report of the evaluation will be sent to the GEF Evaluation Office immediately when ready, and at the latest
within 12 months of completion of project implementation.
59. Project evaluations should serve to provide lessons learned and recommendation for future projects,
policies or portfolios. Agencies will apply their internal arrangements for the conduct of evaluations and
their cost to ensure that evaluation reports of GEF projects are credible, unbiased, consistent and well
documented in line with the requirements above. Each evaluation will assess results, namely outputs,
outcomes and impact) according to the criteria of relevance, effectiveness, efficiency (or cost-
effectiveness), sustainability, as applicable. Future GEF Council decisions on the concept of cost-
effectiveness may lead to minimum requirements for GEF projects, to be incorporated into the M&E
policy. The GEF medium sized projects are more limited in duration and budget, and therefore merit
consideration for tailored minimum evaluation requirements. The Joint Evaluation of the GEF Activity
Cycle and Modalities will address the experience with MSPs and provide recommendations in this
regard.7
60. Furthermore, monitoring and evaluation in the GEF will be guided by the following key principles,
which have been identified as common denominators in the GEF network, and which will be further
developed through specific guidelines or procedures in the consultative process of the GEF Evaluation
Office with its partners in network. These key principles are not "minimum requirements" as such, but are
internationally recognized professional ideals that need to be applied to the specific evaluations and
monitoring systems that the GEF undertakes, or in which GEF network partners collaborate.
a.
Independence. Members of evaluation teams should not in person have been engaged in the
activities to be evaluated, or been responsible in the past for the design, implementation or
supervision of the project, program or policy to be evaluated. For self-evaluation, specific
review mechanisms may help verify impartiality and rigor.
b.
Impartiality. Evaluations must give a comprehensive and balanced presentation of strengths
and weaknesses of the policy, program, project or organizational unit being evaluated. The
evaluation process should reflect impartiality in all stages and taken account of views of all
stakeholders. Units commissioning evaluations should endeavour to ensure that evaluators
selected are impartial, and unbiased. The principle of absence of bias also applies to self-
evaluations, self-assessments, internal reviews and reports and monitoring actions.
c.
Transparency. Transparency and consultation with the major stakeholders are essential
features in all stages of both monitoring and evaluation processes. This involves clear
communication concerning the purpose of the evaluation or monitoring activity, the criteria
applied and the intended use of the findings. Documentation emanating from monitoring and
evaluations in easily consultable and readable form should also contribute to both
transparency and legitimacy. Evaluation and monitoring reports shall provide transparent
information on its sources, methodologies and approach.
d.
Disclosure. The lessons from monitoring and evaluation shall be disseminated by
establishing effective feedback loops to policymakers, operational staff, beneficiaries and the
general public. In the spirit of partnership, the GEF partners shall share with each other GEF-
related evaluation reports; monitoring reports and other internal periodic reviews of progress
7 Until such time, current requirements to undertake MSP evaluations remain in effect.
14
and implementation and make findings and lessons available to project management for
improved effectiveness. The GEF Evaluation Office shall be provided access to all project
documentation of the Implementing and Executing Agencies relating to GEF-financed
activities.
e.
Ethical. Monitoring and evaluation shall provide due regard for the welfare, beliefs and
customs of those involved or affected, avoiding conflict of interest. Evaluators must respect
the right of institutions and individuals to provide information in confidence. If evidence of
wrongdoing is uncovered, the evaluator or manager shall report such cases discreetly to the
GEF Evaluation Director who will take appropriate action such as informing the investigative
body of the Agency concerned. Ethical monitoring and evaluation require that management
and/or commissioners of evaluations remain open to the findings.
f.
Partnership. GEF activities are being implemented through various partnerships of
international organizations, national or non-governmental entities, as well as bilateral donors
involved through co-financing. The GEF Evaluation Office and GEF partner agencies shall
actively explore the possibility of joint evaluations which would provide the GEF with
insights and feedback which might not be reached through a stand-alone evaluation. The GEF
partners shall help further GEF evaluation work though their participation in international
groups and associations for monitoring and evaluation and the research community. GEF
monitoring and evaluation activities shall be carried out with the participation of in-country
stakeholders, including project management and NGOs involved in project implementation,
to enable the beneficiaries to participate in the learning process with the GEF and to enable
the GEF network to learn from them.
g.
Competencies and Capacities. Depending on the subject, monitoring and evaluation
activities require a range of expertise that may be technical, environmental, within social
science or the evaluation profession. Units commissioning evaluations are responsible for
selecting independent-minded, experienced and sufficiently senior evaluators, and adopting a
rigorous methodology for the assessment of results and performance. Evaluations of GEF
activities shall make the best possible use of local expertise, both technical and evaluative.
The GEF partners shall, as feasible, cooperate to stimulate evaluation capacity development
at the local level, with a specific focus on environmental evaluation concerns.
h.
Credibility. Monitoring and evaluation shall be credible and based on reliable data or
observations. This implies that monitoring and evaluation reports shall reflect consistency
and dependability in data, findings, judgments and lessons learned, with reference to the
quality of instruments and procedures and analysis used to collect and interpret information.
Monitoring and evaluation at project and portfolio levels shall use, as much as possible,
dynamic and pragmatic techniques and indicators for measurement of results and progress.
i.
Utility. Monitoring and evaluation must serve the information needs of intended users.
Partners, evaluators and units commissioning evaluations shall endeavor to ensure that the
work is well-informed, relevant and timely, and is clearly and is concisely presented, so as to
be of maximum benefit to stakeholders. Monitoring and evaluation reports should present in a
complete and balanced way the evidence, findings or issues, conclusions and
recommendations. They shall be both results-and action-oriented.
15
4. THE GEF EVALUATION OFFICE
4.1 MISSION STATEMENT AND CORE PRINCIPLES
61. In accordance with GEF Council decision8, the GEF Evaluation Office operates as an
organizational unit that is independent of Agency or GEF Secretariat management in the conduct of the
evaluations that it undertakes. In its work, the Evaluation Office applies its mission statement of:
Enhancing Global Environmental Benefits through
Excellence, Independence and Partnership
in Monitoring and Evaluation.
62. The GEF Evaluation Office will work in close partnership with other entities in the GEF and extend
this collaboration to the global evaluation community in order to remain on the cutting edge of emerging
and innovative methodologies and derive maximum benefits from collaboration. It will dialogue, consult
and collaborate with all relevant partners, to foster a network of monitoring and evaluation professionals
that may add value to GEF operations and results.
63. The Office will conduct its work under the core principles of 9:
a.
Impartiality - conducting evaluations and arriving at the findings in a balanced and unbiased way
b.
Professionalism - applying the latest evaluation knowledge and skills with integrity, accountability, and
respect
c.
Transparency - ensuring full disclosure and active communication with stakeholders on evaluation
priorities, terms of reference of evaluations, methodology and the formulation of findings and
recommendations, at appropriate times.
4.2 KEY FUNCTIONS
64. Within the GEF network, the GEF Evaluation Office pursues the goals of improved accountability
and learning, through three main functions:
a.
An evaluative function. The main function of the Office is to independently evaluate the effectiveness of
GEF programs and resource allocations on project, program, country, portfolio and institutional levels.
b.
A normative function. The Evaluation Office is tasked to set minimum monitoring and evaluation standards
within the GEF family, in order to ensure improved and consistent measurement of GEF results.
c.
An oversight function. The Office provides quality control of the minimum requirements of in monitoring
and evaluation practices in the GEF, full cooperation with relevant units in the Implementing and Executing
Agencies.
65. In the exercise of these functions, the Director of Evaluation participates in Council, Assembly and
replenishment preparatory and regular meetings on monitoring and evaluation issues, and responds to
Council requests on any related matters. The Council has direct access to the Director of Evaluation and
8 GEF/C.21/12.Rev.1, Terms of reference for an Independent Monitoring and Evaluation Unit
9 GEF/ME/C.24/1, Elements for a New GEF Monitoring and Evaluation Policy
16
his/her staff, and the Director of Evaluation may communicate directly with Council members, as deemed
appropriate and without prior clearance from anyone outside the GEF Evaluation Office. 10
66. The GEF Evaluation Office will be independent from both the policy-making process and the
delivery and management of assistance, to guarantee that data gathering and analysis; judgments on
criteria; findings and recommendations will not be influenced through conflicts of interest or undue
interference by management at any level. The Director will issue final evaluation reports directly and
simultaneously to the Council and the CEO without any prior clearance. The Office will be staffed with
independent-minded, experienced and sufficiently senior evaluators to underpin the exercise of
independence and excellence.
67. To avoid conflict of interest, an evaluation will not be entrusted to an Office staff member who has
been responsible in the past for the design, implementation or supervision of the project, program or
policy to be evaluated. The Office will not engage consultants who have worked previously on the design
or implementation of a project program or policy to conduct evaluation analysis or prepare evaluation
reports.
68. The Director of Evaluation formulates independently from management the four year rolling
program of work and budget with all related expenses and submits these directly to the Council for
approval, whereas the budgetary needs of the Agencies and the GEF Secretariat are addressed in the GEF
corporate budget or through project fees. As detailed in the Office Work Program11, evaluation
programming will be developed based on transparent criteria and reflect a phased approach over a GEF
Replenishment period to ensure adequate evaluation coverage for promoting accountability and learning.
For every major evaluation, the GEF Evaluation Office will prepare an approach paper which will be
shared for comments with all the parties involved to allow for stakeholder feedback.
69. Options for the interaction of the GEF Evaluation Office with the Council will be presented to the
GEF Council in November. The Council decision on this issue will be reflected in the final M&E policy
document, to be published after the Council meeting in November.
4.3 TASKS AND PROCEDURES
70.
The GEF Evaluation Office functions will be implemented through the following tasks:
a.
Developing policies, operational guidelines and standards relating to monitoring and
evaluation components in projects, portfolio, program or to corporate monitoring and
evaluation activities, and providing related support.
b.
Conducting evaluations of GEF Operational Programs and Strategic Priorities; country
portfolio and impact evaluations; cross-cutting and thematic studies; and institutional and
procedural issues, and any other matter approved by the Council.
c.
Managing comprehensive independent evaluations of GEF's overall performance in
preparation for the GEF replenishment and Assembly every four years.
d.
Tracking and reporting on implementation of evaluation recommendations and related
management responses.
10 Details on mechanisms for interactions between the GEF Council and the Evaluation Office will be available in a
separate document for the November 2006 Council meeting and any decisions incorporated into the Policy.
11 GEF/ME/C.25/3, Four-year Work program and Budget of the Office of M&E FY06-09
17
e.
Collaborating with GEF partners and taking part in international professional fora to further
monitoring and evaluation in the GEF.
f.
Supporting performance measurement within the GEF by establishing criteria for measuring
performance, results, and impact; and providing methodological support to the development
of program indicator systems in partnership with IAs and the GEF Secretariat.
g.
Conducting validation exercises, as required, for oversight of monitoring and evaluation
systems, including project reviews, reviews of evaluations, verification of progress towards
targets and/or implementation of recommendations.
h.
Providing monitoring and evaluation data and information to assist the GEF in meeting its
informational responsibilities to the GEF Council, the Assembly, the conventions which GEF
serves, other partners, and the general public, and responding to related Council requests.
i.
Working with the Secretariat, the Agencies and other partners to disseminate lessons learned
and best practices emanating from the monitoring and evaluation activities, including through
a formal feedback loop.
18
5. USE
OF
EVALUATIONS
5.1 FOLLOW-UP
71. Satisfactory follow up of monitoring and evaluation reports requires active engagement by all GEF
entities. In all cases, the issuing office will take responsibility for the quality of the final product, with
acknowledgement of inputs and responses from stakeholders.
72. A management response will be required for all evaluation and performance reports presented to
the GEF Council by the GEF Evaluation Office. The GEF Chief Executive Officer coordinates the
preparation of the management response with Agency stakeholders for GEF Council consideration,
tailored to each evaluation report. The Agencies ensure that recommendations from GEF-related
evaluations conducted by the GEF Evaluation Office or departments within the Agency, are submitted for
decision making and action within the Agencies.
73. The Council discusses and reviews GEF monitoring and evaluation reports, the recommended
actions and the evaluation management responses, takes any necessary decisions and gives guidance to
GEF on policies or an appropriate plan of action within specific timeframes.
74. There will be a systematic follow-up on the implementation of the evaluation recommendations that
have been accepted by management and/or the GEF Council, with periodic reporting on the status of the
implementation of the evaluation recommendations. The GEF Evaluation Office and the Secretariat will
report annually to the Council on the follow-up of the Council decisions compiled in a Management
Action Record.
5.2 KNOWLEDGE SHARING
75. Monitoring and evaluation contributes to knowledge building and organizational improvement.
Findings and lessons should be accessible to target audiences in a user-friendly way. Evaluation reports
should be subject to a dynamic dissemination strategy tailored to the audience of that specific report, and
described in the relevant evaluation approach paper and Terms of Reference.
76. For the purposes of this Policy, knowledge management is considered as the process through which
organizations generate value and improve performance from their intellectual and knowledge-based
assets. Knowledge sharing enables partners to capitalize on lessons learned by gaining insight and
understanding from experience, and by applying this knowledge to generate new knowledge. It helps the
GEF create and transform knowledge into action, innovation and change. Knowledge management is
closely linked to performance enhancement and results-based management.
77. The main purposes of knowledge creation and sharing of monitoring and evaluation information in
the GEF are to: (a) Promote a culture of learning through better outreach to project and country level by
providing easily accessible learning products; (b) Promote the application of lessons learned to improve
the performance of GEF activities.
78. Monitoring and evaluation are closely linked to policy-making, more informed management and
decision making for strategic planning. Evaluations can provide a highly-cost effective way to improve
the performance and impact of development policies, programs and projects, especially where evaluations
are conducted at the right time, with a focus on key issues of concern to policy makers and managers.
19
Furthermore, knowledge management supports policy-making by building a comprehensive body of
evidence, lessons learned and good practices from a number of evaluations and monitoring reports.
79. All GEF partners are responsible for actively and transparently contributing to knowledge and
learning. Knowledge management and lessons learned dissemination strategies should be based on user
needs and priorities and the latest technologies and approaches. They are based on the principle of
integration with existing knowledge system in the Agencies, to allow the partners to integrate and
promote relevant learning from GEF monitoring and evaluation across respective portfolios, and for the
GEF to benefit from wider knowledge bases. The development of and participation in knowledge
management systems and communities of practice should increase access to knowledge, enhance
knowledge sharing, collaboration and innovation.
80. The GEF Secretariat coordinates the overall knowledge management strategy of the GEF family,
and promotes mechanisms to disseminate lessons learned and best practices emanating from monitoring
activities in the GEF, through an appropriate repository of knowledge. Evaluation departments contribute
evaluative and empirical evidence, independent validation and causal analysis.
81. Lessons from monitoring and evaluation activities should in particular be made available to
stakeholders directly involved in project formulation and implementation at country level for improved
effectiveness. GEF partners will seek dynamic and interactive ways of disseminating findings from
monitoring and evaluation activities to a wide audience, including environmental entities, academia,
research institutions, civil society and the public. By sharing findings and lessons widely, monitoring and
evaluation may contribute to increased awareness of the importance of global environmental benefits;
confidence in GEF work and leveraging of support.
82. The GEF Evaluation Office specifically supports knowledge sharing by ensuring the highest
standards in accessibility and presentation for its published reports; providing additional learning products
based on evaluations; using a range of channels to reach target audiences; participating in knowledge
management activities; and facilitating interagency sharing of experiences relevant to the GEF. The
Office will take full advantage of possible dynamic means of sharing lessons learned with a broader
audience, including electronic and interactive channels, knowledge networks and communities of
practice.
20
Annex A: Comparison Table
Terms of reference of an independent M&E Unit and the GEF M&E Policy
#
TOR paragraph text
Policy paragraph
reference
1
Under the policy recommendations of the Third Replenishment of the GEF Trust Fund it was
Introduction; historical
2
agreed that the Monitoring and Evaluation Unit, for purposes of evaluation, should be made
background; no need to
3
independent and report directly to the GEF Council. Some of the other key policy
reflect this fully in new
recommendations are that a process for Council oversight be established; that a formal feed-
Policy which is supposed
back loop from evaluation findings should ensure more systematic use of lessons in subsequent to "stand alone".
activities; that enhanced procedures to disseminate lessons be put in place and that more
rigorous minimum standards for M&E be pursued.
This paper proposes a way forward on how the GEF's Monitoring and Evaluation function will
be made independent and strengthened compared to the policies and procedures that are
currently in place. In addition, this paper contains guidelines for the involvement of other
members of the GEF partnership in M&E efforts and a draft Terms of Reference for a proposed
Director of the M&E Unit.
Earlier versions of this document were presented in October 2002 and May 2003. This version
reflects comments from the Council, both in writing and at the May Council Meeting. Council
members have especially recommended that the GEF M&E function should be strengthened
and made independent. An M&E Director should be appointed by the Council and would
report directly to the Council on monitoring and evaluation matters. Regular and timely
communication and consultation modes with the GEF Council should be developed.
4
Council members further recommended a good partnership approach to M&E by all GEF
Covered. Partnership with
entities. There should be an effective collaboration with the GEF Secretariat on administrative
evaluation units
matters, program issues and knowledge management. Both the GEF coordination units and the
expanded, para 18, 62,
evaluation divisions of the Implementing and Executing Agencies should be engaged in
section 2.5, 60 (f)
enhancing systems for measurements of results, the implementation of reviews and studies and
Measurement para 60
in sharing experiences and lessons within the GEF. The Scientific and Technical Advisory
(h), 64 (b)
Panel (STAP) should be involved on scientific and technical issues.
STAP section 2.6
5
Further, there should be enhanced feedback loop mechanisms from monitoring and evaluation
Covered. Section 5.2,
to policy, program and project design especially by the development of a knowledge
para 79, 60 (d)
management system.
6
Upon approval of this document, the M&E unit will so revise "Monitoring and Evaluation
New Policy and
Policies and Procedures" (January 2002).
Guidelines replace the
M&E brochure. Policy
updates covered in
paragraphs 8, 9
7
The objectives of the M&E unit are to:
Changed. Now objectives
· Independently monitor and evaluate, on a continuing basis, the effectiveness of GEF
of M&E apply in general,
programs and resource allocations on project, country, portfolio and institutional
not just to Unit.
bases.
Objective 1 in para 26, 64
· Provide a basis for decision-making on amendments and improvements of policies,
(a) (for Unit, without
strategies, program management, procedures and projects;
monitoring conduct)
· Promote accountability through the assessment of processes, results, impacts and the
Previous objectives 2-4
performance of the parties involved in GEF activities.
are streamlined for all, in
· Provide feedback to subsequent activities, and promote knowledge management on
para 1
results, performance and lessons learned.
8
In accordance with Council's recommendations, the M&E Unit will be independent and report
Covered in Policy; details
directly to the Council in a regular and timely way. The Council will:
will be covered in
· Appoint the M&E Director, upon nomination by the CEO/Chair, for a 5 year term,
administrative
21
#
TOR paragraph text
Policy paragraph
reference
which may be renewed only once. The recruitment process will be conducted in a
operationalization (such
transparent way and in consultation with GEF Council. The M&E Director can be
as Director appointment
removed only by decision of the Council.
procedures in para 22)
· Review, discuss and approve M&E policies and procedures, four year rolling M&E
Review/approve in para
work plans, the annual work program and budget;
68
· Receive directly from the M&E unit all of its reports, either prior to Council meetings Receive directly para
or at any other time.
66, 68, 30
· Arrange for meetings or informal briefings with the M&E team in conjunction with
Meetings in para 65,
regular Council meetings (without going through the CEO/Chairman), encourage
Nov05 Council document
exchanges of views throughout the year, and include at every Council meeting an
Direct access - para 65
agenda item to discuss monitoring and evaluation reports and any recommendations
Annual Review para 72
put to the Council by the M&E unit.
· Have direct access to the M&E Director and his/her staff.
· Review on an annual basis the progress of the implementation of M&E
recommendations and action plans approved by the Council.
RESPONSIBILITIES OF THE M&E DIRECTOR AND M&E UNIT
9
The GEF M&E Unit will be responsible for all corporate M&E activities, including
Covered.
development of M&E policies and guidelines on M&E methodology and implementation of the Unit responsibilities in
annual Project Performance Review, focal area and cross-cutting evaluations, country portfolio Para 25, 26, 27
reviews and especially the evaluation of GEF's overall performance. It will also include a
PPR- para 31 (GEFSec)
review of progress made towards the implementation of the Replenishment recommendations.
Replenishment para 70(g)
10 The M&E Director will be solely responsible for personnel decisions in the M&E Unit. The
Covered.
GEF Secretariat and the IAs and EAs will cooperate closely with the M&E unit on the
Director para 30
development of M&E systems, implementation of reviews and studies, and the development of Cooperation -para 27, 38,
knowledge management systems, including dissemination of M&E results. The budget for the 77
M&E Unit includes provisions for all related expenses, including salaries, consultants' fees,
Budget para 68
office accommodations, equipment and staff services.
11 The practices of the M&E Unit will conform to the highest internationally accepted principles
Covered.
and standards regarding the evaluation of development results, such as those followed by the
International Standards
UN, the multilateral development banks, OECD-DAC, and others. The M&E staff and
para 19, 20, 27, 37, 48, 52
consultants preparing evaluation findings and recommendations will be selected and guided to
Selection para 66, 67,
make fair and consistent assessments based on their best professional and independent
60 (g) (h)
judgments, following GEF Council guidance and oversight.
12 Among the key overarching principles and standards for M&E are:
Covered and expanded to
· Independence: The M&E Unit will be impartial and independent from both the
include issues such as
policy-making process and the delivery and management of assistance. This means,
ethics, transparency,
inter alia, that the unit will be separate from and not report to GEF management, that
impartiality, disclosure.
it should establish policies to ensure that there are not conflicts of interest that could
Independence para 60
weaken its objectivity, that it should have a separate budget, that it should have
(a)
authority for its own personnel decisions, and that the Director should report directly
to the Council.
Accountability para 66
· Accountability: The unit will be staffed with independent-minded, experienced and
on staffing, 60 (h) (i) on
sufficiently senior evaluators, adopting a rigorous methodology for the assessment of
methodology, 12 (d)
results and impacts and the performance of the partners concerned, as well as
highlighting successes, unexpected results, shortcomings and failures and disclosing
Partnerships - Para 60 (f)
them to relevant stakeholders and the general public without interference from any
vested interest.
Feedback para 60 (d)
· Partnerships: The unit will endeavor to ensure that its work is useful, well-informed,
relevant and timely, and is clearly and is concisely presented, so as to be of maximum
22
#
TOR paragraph text
Policy paragraph
reference
benefit to the partners.
· Feedback: The lessons from evaluation will be disseminated by establishing effective
feedback loops from evaluation to policymakers, operational staff and the general
public.
IMPROVED MEASUREMENT OF GEF RESULTS
13 Improved policies, procedures, standards for measurements of results are essential building
Covered.
blocks to an efficient and effective M&E system. This pertains to all levels from the individual Para 60 (d) and 64 (b)
projects and country portfolios to the level of GEF itself as a global environment facility.
14 GEF M&E Policies and Procedures. The GEF M&E policies and procedures will be
Covered.
continuously reviewed and updated in dialogues with all GEF entities to make them dynamic
Para 8 and 9
and relevant to the current circumstances.
15 1.
Improved Monitoring and Evaluation Plans. The M&E Unit will ensure that both
Covered in MR-1 (para
regular and medium-size project designs include satisfactory M&E plans. These plans will
56)
identify how performance information from monitoring and evaluation activities can guide
See Annex B.
project management towards accomplishing project objectives, and report on the
accomplishments.
16 Minimum M&E Standards At entry, all projects must have M&E plans, which will, as a
Revised; more rigorous
minimum, include: definition of performance indicators and unit of measurements; description and strengthened version
of the data source(s) for the indicator; identification of baseline data and methods for data
included. See Annex B.
collection and processing; scheduling frequency of data collection and designating officials
responsible for ensuring data availability. Performance indicators need to be specific,
measurable, achievable, relevant and time-bound. The data should be available on a timely
basis at intervals consistent with management requirements (at least annually). Any ongoing
project which is found to be inadequately performing in this regard will be retrofitted to meet
such minimum M&E standards. Outcomes should be reported to the Council, at a minimum,
on an annual basis. The M&E unit will also establish minimum standards for terminal
evaluations, e.g., conflict of interest rules. Progress on strengthening M&E standards will be
reported to the Council on an annual basis.
17 Program Indicators. In cooperation with teams of the GEF Secretariat, the Implementing
Responsibilities clarified.
Agencies and STAP, the M&E Unit will intensify its work on developing, testing redrafting
GEFSec para 32, STAP
and using program indicators in all focal areas where operational policies and programs have
para 43, EO para 26,
been endorsed.
Agencies para 36
MONITORING PROGRAM
18 The results of the annual Project Performance Report (PPR) will firstly include the outcome of
Covered, responsibilities
the annual Project Implementation Review (PIR) based upon reporting by the GEF
streamlined, para 31, 36,
Implementing Agencies (IA) and Executing Agencies on all ongoing projects. In FY 2002 the 28, 70 (g)
PPR has been complemented by two additional review modalities: the Specially Managed
SMPR discontinued
Project Review (SMPR) and the Terminal Evaluation Review (TER).
(GEF/ME/C25/3)
PROGRAM AND POLICY EVALUATIONS
19 Program evaluations cover essential featuresor the full scopeof GEF operational
Summarized in Para 12
programs. Cross-cutting evaluations provide the opportunity to assess topics of concern to all
(b) (c) (e)
operational programs, e.g. participation/gender, capacity building, policy formulation,
technology comparisons, funding arrangements, management and operational responsibilities,
best practices in M&E arrangements, and application of lessons learned. Country portfolio
reviews enable the assessment of GEF achievements in countries, particularly how GEF
responds to country policies and priorities, and how country policies and practices affect GEF
23
#
TOR paragraph text
Policy paragraph
reference
project results.
In preparation for the GEF replenishment and Assembly every four years, the M&E Unit
Covered. Definition OPS
arranges for a comprehensive external study of GEF's overall performance (OPS), which
in para 12 (g)
addresses overriding issues like global impact and benefits of GEF programs, as well as the
Responsibility para 70(c)
appropriateness of GEF's institutional arrangements, policies, strategies, programs and
Follow-up generic in
priorities. The GEF Secretariat reports on the follow up of the decisions made by the GEF
para 71, 72
Council and the Assembly on the basis of the overall performance study.
FEEDBACK AND KNOWLEDGE MANAGEMENT
21 An essential and integral part of M&E is the feed-back to the decision-making processes in the
Covered.
GEF at policy, program and project levels. The GEF Replenishment called for a formal
Feed-back - Para 78, 60
"feedback loop" to be established between evaluation findings and management activities to
(d) (i)
ensure more systematic use of the results and outputs of the GEF projects for the improvement
of planning and subsequent activities. Evaluation findings and recommendations will be
provided in a timely and readily accessible form to have a positive impact in the GEF activities. Evaluation findings - Para
Their implementation will be reported on an annual basis. The efforts to ensure feed-back and
29, 72
follow-up will partly be done in a coordinated GEF manner, and partly by the Implementing
and Executing Agencies, as appropriate.
22 Activities for feedback and knowledge management will be designed with clearly identified
Covered.
tasks, resources for their implementation and designated dissemination responsibilities. It
Para 29, 79, 82
further requires using techniques that promote and facilitate the integration of findings, and
lessons and recommendations into GEF's programs and projects and, more broadly, into all
Section 5.2
related efforts that further the attainment of global benefits. The GEF M&E Unit will, in
collaboration with other teams of the GEF Secretariat and the Implementing and, when
appropriate, Executing Agencies, develop a knowledge management strategy based on user
needs and priorities and the latest technologies and approaches, subject to budget resource
availability.
ANNEX I: TERMS OF REFERENCE FOR THE GEF MONITORING AND EVALUATION UNIT
DIRECTOR
1.
The GEF Monitoring and Evaluation Unit Director is accountable directly to the GEF
Director TOR remain
Council on M&E matters including work program and budget, and reports directly to the
valid. Broad outlines are
Council The M&E Director will have authority to issue final evaluation reports directly and
covered in Policy.
simultaneously to the Council and the CEO without prior clearance from anyone outside the
M&E Unit. The Secretariat, IA and EAs and other affected parties may receive, comment and
TOR will be part of
respond to the draft and final reports, but will not have the right to approve, hold back, request
administrative
changes to or otherwise modify such draft and final evaluation reports. The Director will not
operationalization.
be reemployed by the GEF upon completion of his or her term; nor will s/he be employed in
any capacity involving or having authority for GEF projects at any of the implementing or
Accountability- Para 66,
executing agencies after the completion of his or her term. The Director will be a senior
68
official, comparable to the level of independent evaluation heads in other international
Reports 5.1
institutions.
2.
The Director will be responsible for managing the unit independently. Among these
responsibilities, the Director:
a.
Implements Strategic Direction and Oversight Provided by Council:
(i)
Prepares and submits draft M&E policies, guidelines and procedures for GEF
Council decision, after requesting inputs from relevant GEF entities.
Para 30
(ii)
Prepares and submits draft four year rolling work plans and annual work
Policies para 8, 27
programs and budgets for Council approval, after requesting inputs from
24
#
TOR paragraph text
Policy paragraph
reference
relevant GEF entities.
(iii) Provides individual M&E reports, report summaries, and annual summary
Work plan para 67
reports on M&E activities and results, including information and follow-up of
evaluation findings and recommendations. Arranges special meetings and
communicates directly with Council members, as deemed appropriate and
Reporting para 65, 69
without prior clearance from anyone outside the M&E unit.
(iv) Participates in Council meetings on monitoring and evaluation issues, and
Council relations para
responds to Council requests on such matters.
65, 69
b.
Provides Overall Direction and Management of Resources:
(i)
Formulates independently from management the annual M&E program of work
and budget and submits it directly to the Council for approval. Plans and work
programs cover methodology development as well as monitoring, review and
evaluation at the project, program, country, policy, operational and overall GEF
level.
(ii) Implements the work program and manages its component activities within
budget, staff and time constraints.
c.
Manages and Coordinates Staff and Team Performance:
(i)
Makes all personnel and operational decisions concerning M&E staff in
accordance with staff rules.
(ii)
Ensures that the unit is staffed by independent-minded, experienced and
Personnel decisions
sufficiently senior evaluators.
para 30
(iii) Manages GEF M&E staff, assigns tasks and coordinates the inputs of GEF
Staffing para 66
Secretariat, Implementing and Executing Agencies.
(iv) Evaluates and monitors team and individual performance.
Coordination para 34
(v)
Provides coaching and mentoring; and recommends suitable training.
(vi) Maintains open communications, fosters team work and resolves conflicts.
Points (iv) to (vii) will be
(vii) Nurtures and protects the independence of mind of M&E evaluators.
covered in administrative
(viii) Establishes conflict of interest rules for M&E staff. In this connection, the
operationalization.
Director will make certain that the engagement of any individual in an
evaluation exercise will not generate a conflict of interest. In particular, an
evaluation will not be entrusted to a unit staff member who has been
responsible in the past for the design, implementation or supervision of the
project, program or policy to be evaluated. A consultant who has worked
Conflict of interest para
previously on the design or implementation of a project program or policy may
60 (b) and (e), 67
being engaged as a resource person for providing information to the evaluation
team but not as a consultant entrusted with the conduct of the evaluation
analysis and preparation of the evaluation report.
(ix) Unit staff other than the Director will be entitled to seek employment in other
units of the GEF, but must inform the Director prior to such job search in order
to avoid conflicts of interest. GEF Secretariat will treat unit staff who apply for
positions outside the monitoring and evaluation unit on a fully equal basis with
other GEF staff.
d.
Strengthens Institutional Relationships:
(i)
On M&E matters strengthens the relationship between the GEF and
cooperating partners.
(ii)
Seek full cooperation with the M&E units in the Implementing Agencies and
Executing Agencies, to help ensure that their M&E practices are in accordance
with minimum GEF standards, and facilitates interagency sharing of
Cooperation and
operational lessons learned.
relationships para 61,
(iii) Reports to Council.
62, 60 (f)
(iv) Proposes to the Council any measure that he or she believes is necessary to
ensure evaluation independence.
25
#
TOR paragraph text
Policy paragraph
reference
(v)
Cooperates with the heads of evaluation units of IAs, EAs, other international
institutions and bilateral development agencies.
Measures para 30
e.
Ensures Feed-back and Knowledge Management:
(i)
Ensures feed-back of M&E results to the decision-making processes in the GEF Cooperate para 26
at policy, program and project levels, and works with the secretariat and
Implementing and Executing Agencies in establishing a formal feedback
mechanism.
(ii)
Ensures follow-up on all M&E matters which require decisions.
· Participates in the development and maintenance of a comprehensive
Feedback - para 63c, 60
knowledge management system based on M&E and other findings and
(d)
lessons.
· Reports to Council on outstanding M&E Unit recommendations and
replenishment recommendations relating to M&E, with a focus on those
Follow-up section 5.1
relevant to the overall action plan.
and 5.2
· Communicates evaluation results to stakeholders and the general public
ANNEX II: GUIDELINES FOR PARTICIPATION OF ALL RELEVANT GEF ENTITIES IN M&E
EFFORTS
INTRODUCTION
Covered.
1.
M&E is a shared responsibility within the GEF partnership. The GEF Secretariat,
the Implementing and Executing Agencies and the Scientific and Technical Advisory Panel
Para 18 to 20
(STAP) will cooperate closely with the GEF M&E Unit and will promptly respond to requests
from the GEF M&E Unit for documents, data and any other information necessary to perform
its functions. Management and technical staff members participating in GEF-funded activities
will also be asked to participate in M&E activities. Therefore a coordinated approach is
required.
THE GEF M&E UNIT DUTIES AND RESPONSIBILITIES
2.
The GEF M&E Unit will have the following primary duties and responsibilities:
Covered.
a. Prepare minimum M&E guidelines, standards and procedures relating to M&E
Section 2.3
components in project documents and all monitoring and evaluation activities.
a para 64 (b), 70 (a)
b. Manage evaluations of GEF's overall performance.
b para 70 (c)
c. Conduct evaluations of strategic and operational topics, programs, project impacts,
c para 26, 64 (a)
country portfolios, and cross-cutting issues, and any other matter approved by the
d para 28, 70 (g)
Council.
e Moved to GEFSec
d. Coordinate the annual Project Implementation Reviews (PIR), the Specially Managed
responsibility, monitoring
Project Reviews (SMPR) and the Terminal (project) Evaluation Reviews (TER).
f- para 29, 79, 80
e. Report annually on GEF Project outcomes.
g para 70 (b)
f. Work with the Secretariat, the IAs and EAs to establish procedures to disseminate
lessons learned and best practices emanating from the monitoring and evaluation
activities, including through a formal feedback loop.
g. Respond to Council requests.
GEF SECRETARIAT DUTIES AND RESPONSIBILITIES
3.
The CEO and Chairman will conduct the recruitment process for the M&E Director in Partly covered. Details
an open and transparent way and nominate a candidate for the Council's consideration. The
will be covered in
GEF Secretariat will provide certain administrative support for the M&E unit. The
administrative
secretariat's program and technical staff will cooperate with the M&E unit on a wide range of
operationalization.
M&E activities including knowledge management and dissemination. In particular, the GEF
Secretariat will:
Section 2.4
26
#
TOR paragraph text
Policy paragraph
reference
a. Participate through the provision of requested information and inputs in all monitoring Administration para 34
and evaluation reviews, studies and evaluations and the annual PIR and SMPR.
a para 34
b. Ensure that findings and recommendations emanating from evaluation and monitoring b para 35, 72
activities are followed up with regard to GEF policies, programs and procedures, and
c para 32
that Council decisions are implemented.
d para 35
c. Participate in the identification of program-level indicators and the use of data for
monitoring performance on program indicators.
d. Ensure that results and lessons identified through M&E activities are adequately
reflected in public information about GEF, including the publication of all monitoring
and evaluation reports.
IMPLEMENTING AND EXECUTING AGENCY DUTIES AND RESPONSIBILITIES
Covered and streamlined.
4.
The GEF Implementing and Executing Agencies have lead responsibility for:
Section 2.5
a. Designing monitoring and evaluation plans and performance indicators for projects
a para MR 1 and 2, para
and adequately monitoring the implementation of projects against the indicators.
57, 56
b. Reporting annually on project performance in the context of the PIR.
b para 36
c. Conducting terminal evaluations of projects and programs, and impact evaluations of
c para 40, 59, MR3
capacity building and enabling activities. When appropriate, conducting mid-term and d para 72
phase evaluations during project implementation.
e para 38, 60 (d)
d. Ensuring that recommendations from evaluations are submitted for decision making
f- para 40
within the Agencies, as appropriate, and reporting to the Council the follow-up on
g para 37
M&E recommendations.
h para 9, 40, 38
e. Making project evaluations publicly accessible and project documents available to the
M&E team.
f. Conducting impact evaluations of GEF projects after project completion as agreed
with the M&E Director.
g. Respond promptly and fully to GEF M&E Unit requests for information pursuant to
any monitoring and evaluation study, evaluation or review.
h. Finally, the three GEF Implementing Agencies will support the M&E team and others
within the GEF partnership on the following, as requested by the M&E team and
agreed:
(i)
Participating in developing M&E policies and identifying program
indicators.
(ii)
Participating in program studies, cross-cutting evaluations, SMPR and
TERs.
(iii)
Participate in the development of and the implementation of knowledge
management.
(iv)
Assisting with the facilitation of studies of GEF's overall performance.
SCIENTIFIC AND TECHNICAL ADVISORY PANEL DUTIES AND RESPONSIBILITIES
5.
The GEF Scientific and Technical Advisory Panel (STAP) will cooperate on scientific Expanded, Section 2.6,
and technical matters with the M&E Unit on key aspects of the evaluation and monitoring
para 41-43
work program, including the development of program indicators and scientific measurements
of GEF results.
PARTICIPATING COUNTRIES' ROLES
Expanded
Section 2.7
6.
The GEF participating countries directly involved in an M&E activity (i.e. studies of
Briefing - para 38
a particular project) will be fully consulted, informed and briefed about the plans, the
Responding - Para 45
implementation and the results of the evaluation activity. Staff members of the cooperating
Participant para 60 (c, f)
27
#
TOR paragraph text
Policy paragraph
reference
governments or institutions will be expected to participate in evaluations and respond promptly
and fully to GEF M&E Unit requests for information relating to GEF projects. They will also
be requested and encouraged to share relevant experiences prior to, during and following
evaluations.
STAKEHOLDERS
Expanded
7.
M&E activities will be conducted with the participation of project stakeholders and
Section 2.8, para 46-47
beneficiaries, including, as appropriate, those that carry out project activities (e.g., IAs, EAs,
Participant para 60 (c, f)
national and subnational governments, local community groups, NGOs, the private sector,
academic institutions, and cofinanciers).
ANNEX III: GEF-3 RECOMMENDATIONS ON MONITORING AND EVALUATION
1.
A strengthened monitoring and evaluation function within the GEF, built upon the
Covered.
monitoring and evaluation systems of the Implementing and Executing Agencies, is a
Integrated into whole
necessary corollary to improved measurement of GEF outcomes and results. The
policy and work planning.
establishment of a framework for monitoring and evaluation with clear indicators and the
extension of monitoring and evaluation tasks to more strategic and programmatic issues should
be integral components of the GEF monitoring and evaluation activities. The monitoring and
evaluation framework should provide for the incorporation of the views of, and lessons
emanating from, the recipient countries. Cross-learning within the GEF should be strengthened
and accelerated so that GEF resources can be used more effectively.
2.
Participants recommend that a high priority be placed on strengthening monitoring
Covered.
and evaluation of GEF projects. Participants also recommend that the roles and responsibilities
for monitoring and evaluation among the GEF Monitoring and Evaluation Unit, the Secretariat
and the Implementing and Executing Agencies be reviewed by the Council together with
recommendations aimed at developing a partnership approach to monitoring and evaluation
responsibilities in order to increase complementarity. Drawing upon its technical expertise, the
GEF Secretariat and the Monitoring and Evaluation Unit should have a more participatory role
in the Implementing and Executing Agencies' project implementation reviews with regard to
determining progress toward achieving GEF objectives while recognizing that accountability
for project monitoring and supervision of implementation lies with the Implementing and
a TOR, 60 (a), 61, 66,
Executing Agencies. More specifically, Participants recommend that the following actions be
26
undertaken:
b para 28, 23, 69,
a.
the GEF monitoring and evaluation unit, for purposes of evaluation, should be made Nov05 Council document
independent, reporting directly to the Council, with its budget and work plan c para 38, 33, 79, 80
determined by the Council and its head proposed by the GEF CEO and appointed by d section 5.1, para 78
the Council for a renewable term of five years;
e mainstreamed into
b.
a process for Council oversight of monitoring and evaluation should be established;
management response ,
c.
the GEF Secretariat and Implementing and Executing Agencies should establish a section 5.1
procedure to disseminate lessons learned and best practices emanating from the f- section 3, para 56- 59
monitoring and evaluation activities;
g para 27, 29, 80, 79
d.
a formal "feedback loop" should be established between evaluation findings and h section 3, para 56-59
management activities to ensure more systematic use of the results and outputs of i para 28
GEF projects for the improvement of planning and subsequent activities;
j para 37, 60 (d)
e.
the GEF Secretariat and the Implementing and Executing Agencies are called upon to
report annually to the Council on their response to relevant recommendations of OPS2
and the replenishment documents;
f.
the monitoring and evaluation unit should establish more rigorous minimum standards
for GEF-specific aspects of projects relating to GEF policies and strategies expected
of monitoring and evaluation units of the Implementing and Executing Agencies;
g.
as each of the Implementing and Executing Agencies has its own system for drawing
lessons from operational experiences, the GEF monitoring and evaluation unit should
facilitate more intensive interagency sharing of experiences relevant to the GEF;
h.
all projects should include provisions for monitoring the impacts and outcomes of
28
#
TOR paragraph text
Policy paragraph
reference
projects, and those existing projects which do not have such provisions and which
have more than two years left in their implementation should be retrofitted to meet
such monitoring standards;
i.
the monitoring and evaluation unit should report annually to the Council on its work;
j.
the monitoring and evaluation unit should be provided access to all project documents
of the Implementing and Executing Agencies relating to GEF-financed activities.
3.
Taking into account the above two paragraphs, Participants recommend that the
Covered.
monitoring and evaluation unit prepare a note for consideration by the Council at its meeting in
October 2002 on the terms of reference for the independent monitoring and evaluation unit.
29
Annex B: Comparison Table
Minimum Standards in the Terms of Reference of an independent M&E Unit and
Minimum requirements in the GEF M&E Policy
Para
TOR
Policy
15
The M&E Unit will ensure that both regular and medium-
Minimum requirement 1 (MR-1, para 56) expanded to
size project designs include satisfactory M&E plans.
provide more detail provided on the meaning of
`satisfactory'. Responsibility for `ensuring' design lies with
GEFSEC/Agencies.
15
These plans will identify how performance information
Deleted. Use of M&E information should be part of
from monitoring and evaluation activities can guide
management, not of M&E plan.
project management towards accomplishing project
objectives, and report on the accomplishments.
16
At entry, all projects must have M&E plans.
MR-1, para 56: Specified that entry means CEO approval,
and added that plans must be "concrete, fully budgeted".
16
M&E plans, will, as a minimum, include:
· Expanded to reflect a comprehensive M&E plan which
· definition of performance indicators and unit of
includes indicators). MR-1 and 2 made more rigorous
measurements
by specifying that performance indicators include
indicators for implementation, results (outcomes, if
applicable impacts) and corporate indicators
·
Added requirement that if no indicators are identified,
an alternative plan for monitoring which will deliver
reliable and valid information to management
· `Unit of measurements' included in SMART indicator
M&E plans, will, as a minimum, include:
· Included in SMART indicator.
· description of the data source(s) for the indicator
M&E plans, will, as a minimum, include:
· MR-1 and 2 expanded to reflect that baseline can also
· identification of baseline data and methods for data
have qualitative aspects (including description of
collection and processing
problem to address with indicator data)
· Added: if major baseline indicators are not identified,
an alternative plan for addressing this within one year
of implementation (as PIR performance measure)
reflecting that for complex projects PDF funds are
insufficient to develop satisfactory baseline before start
· Methods for data collection and processing included in
SMART indicator system
M&E plans, will, as a minimum, include:
· Included in SMART indicator, adding that the system
· scheduling frequency of data collection
should allow progress to be tracked in a cost-effective
manner
M&E plans, will, as a minimum, include:
· Expanded to "organizational set-up and budgets for
· designating officials responsible for ensuring data
monitoring and evaluation". Ensuring data availability
availability.
is one of many M&E tasks.
· New: "identification of reviews and evaluations which
will be undertaken". Project evaluation was previously
not covered as requirement in M&E plan.
Performance indicators need to be specific, measurable,
MR-1 and MR-2, para 56 and 57, for both implementation
achievable, relevant and time-bound.
and results indicators.
More detail on meaning of SMART in para 54.
The data should be available on a timely basis at intervals
Included in SMART indicator and M&E system.
consistent with management requirements (at least
Annual data not realistic if dealing with some long-term
annually).
results indicators. If data is available, it is generally timely.
Need to differentiate between periodicity of indicator data
(according to M&E plan) and periodicity of reporting (to
management and/or Council.)
30
Para
TOR
Policy
Para 58 now reflects that `Periodic reports should be based
on a principle of continuity to allow for tracking of results
and progress'
16
Any ongoing project which is found to be inadequately
There was lack of clarity on "inadequately performing",
performing in this regard will be retrofitted to meet such
though some past projects were retrofitted. Now propose a
minimum M&E standards.
`transition period' of making MR-1 and 2 applicable for
Replenishment period. (Previous standards and current
practice of M&E plans are adequate in the meantime for
projects in workplan but not started, given the requirement
that full plan can be developed within one year.)
16
Outcomes should be reported to the Council, at a
Not a minimum requirement at project level; projects report
minimum, on an annual basis.
a variety of results to Agency management who reports
overall results and trends to GEF. Results include outputs,
outcomes, impact, and it takes time for a project to generate
outcomes cannot be annually.
Reporting to Council will now be undertaken by GEFSec
through "implementation review in which it presents an
overview of progress towards results, including outcomes,
implementation issues and portfolio-wide trends" (para 31).
Evaluations will also report outcomes, but no periodicity
can be fixed.
16
The M&E unit will also establish minimum standards for
New: MR-3, para 58 now containing project evaluation
terminal evaluations, e.g., conflict of interest rules.
requirements. Para 60 (a ) and (b ) cover independence and
impartiality. Conflict of interest rules apply to GEF Office
of M&E (para 66); not applicable to self-evaluations. Will
be further developed in guidelines
16
Progress on strengthening M&E standards will be reported Not a minimum requirement at project level. Progress in
to the Council on an annual basis.
strengthening M&E standards themselves is reported to the
Council on an annual basis in GEF Office of M&E
progress reports. Progress in strengthening application of
M&E standards is reported in Annual Performance report of
the Office (para 28)
31
Document Outline