G l o b a l E n v i r o n m e n t F a c i l i t y

GEF/ME/C.30/1
November 6, 2006
GEF Council
December 5-8, 2006

Agenda Item 6





GEF EVALUATION OFFICE:
PROGRESS REPORT FROM THE DIRECTOR






(Prepared by the GEF Evaluation Office)

---

Recommended Council Decision

The Council, having reviewed document GEF/ME/C.30/1 "GEF Evaluation
Office: Progress Report from the Director," takes note of the work on on-going
evaluations, the implementation of the GEF monitoring and evaluation policy, and the
selection criteria for country portfolio evaluations. On the basis of the information given,
Council requests the Evaluation Office to prepare, manage and implement an independent
Fourth Overall Performance Study (OPS4) in time for submission to the negotiations on the
fifth replenishment of the GEF Trust Fund, to better integrate evaluation findings in OPS4
and to enhance the reporting on results.

The Evaluation Office is invited to redesign the annual budget of the Office for
discussion in June 2007, incorporating the necessary expenses for OPS4 throughout the
coming years, ensuring overall cost savings as compared to OPS3.


2

---

EXECUTIVE SUMMARY
1.
The GEF Evaluation Office is presently conducting a series of evaluations on cross-
cutting issues, institutional procedures and principles, country level support and impact. The
Office found some of the evaluations proposed for this year more complex than initially thought,
requiring development of appropriate methodologies and approaches.
2.
On the request of Council, the criteria for selecting countries for the GEF Country
Portfolio Evaluations are presented. These criteria aim to first of all ensure that all recipient
countries of the GEF have a chance to be selected, and then proceed to apply the programming
criteria of the Office for its four year rolling work plan. Countries will be chosen according to
region first, after which the final decision will be taken per fiscal year in view of potential
synergies with other evaluations and the financial means available to the Office for this kind of
evaluation.
3.
In fiscal year 2006 this selection process has led to the identification of the Philippines
and Samoa. Combining these two evaluations means that they can be done for the price of one.
The Philippines has a full portfolio and will need a full review ­ however, Samoa has a relatively
low GEF profile, while at the same time highlighting the difficulties that Small Island Developing
States are facing. The on-going evaluations of the Small Grant Programme and the Evaluation of
Capacity Development will ensure that the Country Portfolio Evaluations will have the benefit of
their field work as well.
4.
In coming fiscal years other countries will be chosen on the basis of a re-run of the last
steps of the selection process. In FY08 Africa will be the focus; in FY09 Eastern Europe and
Central Asia; in FY10 Middle Eastern & Northern Africa will follow and in FY11 the process
will return to Latin America and the Caribbean.
5.
This report also contains further information on how the Evaluation Office sees its role
and function in OPS4. Council approved a four year rolling work plan in June 2006 which
incorporates the principles of building up evaluations to culminate in OPS4 and asked for further
information before asking the Office to implement OPS4 itself.
6.
Major advantages for the Office to implement OPS4 are that this would substantially
reduce learning costs of an external team of evaluators and it would strengthen the reporting on
results in OPS4. Disadvantages could be a "perceived" lack of independence of the Office from
the GEF as a whole and the lack of a fresh outsiders perspective. These disadvantages can be
overcome if the Office has been "peer reviewed" as independent in role and function and if
OPS4 ensures processes to incorporate a fresh perspective on the GEF.
7.
It is proposed that the Evaluation Office prepare, manage and implements OPS4, while
ensuring that the terms of reference for OPS4 are approved by Council and taking care that two
elements of OPS4 are undertaken independently of the Office: a stakeholder consultation
process and the evaluation of M&E in the GEF. The Office will reflect this in its proposal for a
restructured and integrated annual budget in June 2007.

---

INTRODUCTION
8.
At the Council meeting of June 2005 a Progress Report was presented for the first time
to Council as an Information Document. The report was discussed and Council requested that
the second progress report would be a Working Document. This report was presented to
Council in June 2006, but was not discussed due to time constraints. Council members
suggested "that there was overlap between the progress report of the Director and the Four
Year Work Plan and that this could be prevented by staggering these reports in time so that
only one of the two is on the agenda of each Council session".1 This progress report meets that
suggestion and is an up-date of the report of June 2006, including information on several issues
that the Council asked for.
9.
In its June 2006 meeting, Council asked for a short note on the criteria to select
countries for future Country Portfolio Evaluations.2 Furthermore, the Office was requested to
prepare for Council consideration a more detailed proposal on the suggested role of the
Evaluation Office in OPS4. This would feed into proposals to provide an integrated budget with
clearer guidance and priorities for special initiatives. Lastly, more information was required on
how external funding relates to the independence of the Office and how potential conflicts of
interest are prevented.3
10.
This report is thus structured as follows: a chapter on on-going work, including
information on how potential conflicts of interest are prevented, followed by chapters on the
criteria for selection of countries for Country Portfolio Evaluations and on OPS4 as well as the
preparation of a new budget format. Given the fact that many expectations have been voiced
during the Assembly and at other meetings on the mid-term review of the RAF, the last chapter
also contains a first look at how the emerging key questions could be tackled.
ON-GOING WORK
11.
The Office is presently conducting a series of evaluations on cross-cutting issues,
institutional procedures and principles, country level support and impact. In addition, the Office
has established a network of partners across the Implementing and Executing Agencies as well
as throughout the world interested in pursuing the evaluation principles of accountability and
lessons learning. The Office found some of the on-going evaluations more complex than initially
thought, requiring development of appropriate methodologies and approaches. For example, the
evaluations of GEF impacts and capacity building have been delayed until the approach is fully
developed. The evaluation of the catalytic role of the GEF will also start with a phase of
methodology development.
12.
In FY06 the Office began to develop an approach to impact evaluation within the
GEF, and initially it was envisioned that the pilot impact evaluation would be completed by the

1 Joint Summary of the Chairs, June 22, 2006 ­ paragraph 75
2 Joint Summary of the Chairs, June 22, 2006 ­ paragraph 10 (d)
3 Joint Summary of the Chairs, June 22, 2006 ­ paragraph 14

2

---

end of FY06. After initial steps to develop the approach paper for the evaluation, it was
determined that an extensive additional effort would be necessary to develop a methodology
appropriate for the challenge of evaluating impacts within the context of the GEF. It was
necessary to ensure that impact evaluation of GEF support be cost-effective and produce
findings in a manner that satisfies the needs of all GEF stakeholders. The resulting methodology
paper proposes to adopt a theory based approach, which would work through logic models
and results chains to ensure that the evaluation focuses on the intervention mechanisms that will
be driving the final impact projects. On the basis of this methodology, a first impact evaluation
will start in the coming months.
13.
The approach paper on the Capacity Development Evaluation was shared with
GEF partners in August 2006 and the finalized version is published on the website. It proposes
to evaluate the results of all GEF capacity development support in two (or more) neighboring
countries, which share one or more common environmental challenges of global importance.
The study will examine in each country the nature and results of the national and regional
interventions and relate these to policy, institutional and individually-focused capacity
development targets. In addition, the evaluation will develop a set of protocols, which will
enable forthcoming Annual Performance Reports, Country Program Evaluations and OPS4 to
evaluate the achievements of capacity development activities on a broader scale.
14.
Work has started on the Evaluation of the Catalytic Role of the GEF. Given the
fact that only a handful of evaluations have been done internationally on replication,
mainstreaming and up-scaling, as well as leverage and co-funding, a methodological phase is
necessary to explore possibilities for this evaluation. This methodological phase will be done
together with other evaluation offices in agencies that also perceive their own role as catalytic or
innovative, such as the International Development Research Center in Canada, IFAD and
UNDP. Furthermore, collaboration with STAP will be actively promoted.
15.
After the Assembly meeting in Cape Town, work has started on an issues and options
paper for the possible Evaluation of the Role of Science, in collaboration with STAP. This
issues and options paper will feed into the four year rolling work plan of the Evaluation Office,
which will be proposed to Council in June 2007.
16.
Other on-going activities and evaluations are: the preparation of the 2006 Annual
Performance Report, Country Portfolio Evaluations, knowledge sharing, support to
development of indicators, and preparations for the special initiatives that Council has approved
in June 2006: the international workshop on environmental and sustainable development
evaluation and the junior evaluation professional from a developing country, to be hired by the
Office in 2007.
Implementation of the New M & E policy

17.
The new GEF Monitoring and Evaluation Policy was approved by Council on a no-
objection basis through mail on February 6, 2006. The Office is mainstreaming the new M&E

3

---

policy thought-out the GEF system in four ways: dissemination, development of guidance and
administrative procedures, support of monitoring, and oversight and performance feedback.
18.
Dissemination of the new policy. The Office has made the new policy available on
the internet and has published and distributed hard copies among the key GEF stakeholders.
The Office also presented and discussed the new policy at the Sub-regional Consultations with
GEF Focal Points. The Office will also work with the GEF Secretariat to incorporate M&E and
GEF M&E minimum requirements in various training programs and the focal points training
workshop.
19.
Development of guidance and administrative procedures. The GEF M&E policy
will be further developed and implemented through specific guidelines and administrative
arrangements and procedures. Two guidelines have recently been finalized. The ethics
guidelines provide guidance to the Evaluation Office and evaluators in the GEF, both staff and
consultants, on ethical behavior to ensure that evaluations are free of bias, transparent, take
stakeholder rights and interests into account and are of the required quality. The guidelines
contain provisions on preventing conflict of interest on three levels: institutional, staff, and of
consultants hired to contribute to evaluations. Secondly, guidelines to conduct terminal
evaluations have been finalized as well.
20.
Support to the establishment of portfolio and program monitoring systems.
During FY06 and early FY07 the Office has provided support to the GEF Secretariat in various
aspects related to the Secretariat's new monitoring roles. The Office also supports the
International Waters Task Force to define scientific based indicators for environmental results
and catalytic impact for nutrient reduction projects and to define baseline indicators for ground
water projects. The Office has also worked with the Land Degradations Task Force to further
define the global environmental benefits in this focal area and to put in place an indicator system
conceptual framework for the focal area.
21.
Oversight and performance feedback. The Annual Performance Reports have in just
two years become an important instrument to assess the performance of the GEF partners,
especially on M&E issues. The 2005 APR has established a baseline on projects compliance of
M&E minimum requirements that will be used to track progress in the implementation of the
policy during project design. The Office will continue to track and provide feedback on the
efficacy of the policy and processes that GEF agencies put in place to implement the new GEF
M&E Policy. Furthermore, the APR will over the years look into various aspects of
performance and present overviews in the form of an Agency Performance Matrix.
International Workshop on Environmental and Sustainable Development Evaluation

22.
The Evaluation Office is currently convening an international partnership to prepare an
international workshop on evaluation of environmental and sustainable development issues in
developing countries and countries with economies in transition, focusing on climate change. The
workshop will allow a sharing of experiences in evaluating sustainable development projects and

4

---

programs. The workshop will focus on climate change issues: both on mitigation and adaptation.
Scheduled for early 2008, the workshop is planned to take place at the Bibliotheca
Alexandrina, in Alexandria, Egypt.
23.
The workshop is expected to: 1) enhance the quality of evaluations and interventions; 2)
facilitate the identification of best practices; and 3) lead to the development of a network of
experts and institutions on the covered areas. Evaluators from partner agencies, research
community, donors and civil society will be invited to join. Active interaction with regional
organizations and conferences will ensure participation of environmental evaluators with focused
papers. At present, the workshop has received financial contributions from the GEF Council
(25k), Denmark (20k), Norway (circa 50k), whereas Switzerland, the Netherlands and France
have not yet made definite contributions but are positively inclined.
SELECTION CRITERIA FOR THE COUNTRY PORTFOLIO EVALUATIONS
24.
Council requested the Office to present a short note to its December meeting on how
countries are selected for GEF Country Portfolio Evaluations. The objectives of these
evaluations is to provide the Council with an assessment of how the GEF is implemented at the
country level, of results achieved at the project level and of how these projects are linked to
national environmental and sustainable development agendas as well as to the GEF mandate and
focal areas programs. They aim to evaluate the relevance and efficiency of the GEF in the
country, as well as the effectiveness and results of completed projects and to provide feedback
and knowledge to all GEF partners, including the country concerned.
25.
Furthermore, these evaluations bring to the attention of Council different experiences
and lessons of how the GEF is implemented at the national level from a wide variety of
countries. These evaluations do not have the objective of evaluating the performance of
Implementing Agencies, Executing Agencies or national governments or individual projects. The
standard terms of reference for these evaluations have been published on the website of the
Office.
Country Selection Criteria
26.
The principles behind the country selection criteria include:
·
Clarity and transparency of selection process
·
Fair chance for all GEF recipient countries to be selected
·
Programmatic criteria ­ using the seven criteria developed and used by the
Evaluation Office to program its activities (policy relevance, financial weight,
stakeholder opinion and demand, public and/or media debate, evaluation
coverage, evaluability and international collaboration (synergy)
·
Synergy with other evaluations conducted in the Evaluation Office or by the
evaluation offices of the Implementing and Executing Agencies.

5

---

27.
Countries were selected using a stepped approach with the goal to select two to three
countries per year. Given the large number of countries that are eligible, a stratified random
selection was adopted to produce a list of 8 countries per geographic region. On this selection a
set of criteria was applied, which are used by the Evaluation Office to determine programming
priorities in the four year rolling work plan. These criteria are both quantitative (i.e., using an
index system) and qualitative (i.e., providing room for expert judgment). The final step to fine-
tune the selection of the final 1 or 2 countries will consider the evaluability and synergy with on-
going evaluations as well as with GEF corporate activities and will be conducted at the end of
the fiscal year, regarding the Country Portfolio Evaluations for the following fiscal year, so
preparation can start well in advanced.
Steps of the selection process
28.
First step: regional distribution. The first step in the process was to group countries
according to geographic regions (as defined by the World Bank). The proposal is to conduct 1
or 2 CPEs per year in a particular region according to the following sequence: Asia in FY07;
Africa in FY08; Eastern Europe and Central Asia in FY09; Middle East and North Africa in
FY10; and Latin America and the Caribbean in FY11. The main reason to select countries
from the same region in each year is to save costs in travel to the region by combining missions
and resources.
29.
Second step: ranking according to contribution to GEF mandate. The second step was
to group countries within each region according to their potential contribution to the mandate of
the GEF: generation of global environmental benefits using the RAF indexes for biodiversity and
climate change as a proxy. The list of countries ranked according to their RAF funding
allocation for biodiversity and climate change4 was used to group the countries into four groups:
above $50 million, between $21 and $49, not in the group but less than $20 and country in
group allocations (the country's highest allocation, either in biodiversity or climate change, was
used to assign a country to a group).
30.
Third step: stratified random selection. Random numbers were generated by Excel for
each of the 4 groups and 2 countries are selected per group to a maximum of 8 countries per
region.5
31.
Fourth step: application of programmatic criteria. The GEF Evaluation Office uses
seven criteria for programming its activities that is also relevant for selecting countries for CPEs:
policy relevance; financial weight, stakeholder opinion and demand, public and/or media debate,
evaluation coverage, evaluability and international collaboration (synergy). The first five were
used to further narrow the number of possible countries for Country Portfolio Evaluations from

4 GEF Resource Allocation Framework: Indicative resources allocation for GEF-4, for the biodiversity and
climate change focal areas (September 15, 2006)
5 An alternative to using a stratified random selection could have been to use the selection criteria presented
in step 4 for all 160 countries. This was considered not appropriate given that there are only 7 criteria which
will not allow to have enough difference between countries to be able to select 8 countries per region.

6

---

those selected in Step 3 (the other two are used for the final selection, see step 5 below). A
system of indicators and scores to measure each of the five criteria was developed with a
possible highest score of 7.5 (see Table 2 for definitions and indicators). A score of 4 or above
was selected to determine the final countries to be considered each year for further exploration.
32.
Final step: integration with synthetic opportunities. The final step in the selection
process will be done each year before the end of the fiscal year, to select the final 2-3 countries
for the next fiscal year, and will consist of reviewing the countries selected in step 4 according to
two criteria: evaluability and synergies/international collaboration. Evaluability is measured by
the quality of available baseline information regarding the context in which the GEF projects
were prepared and implemented. For example, countries that have been the subject of previous
country evaluations conducted by evaluation offices of the partner GEF agencies and those that
have or are preparing GEF and national environmental strategies, have conducted their RAF
prioritization exercise, have completed the National Capacity Self-Assessment projects and
have had a country dialogue workshop would receive higher priority. The synergy criterion
measures the relevance of the particular country to on-going or future evaluations proposed by
the GEF Evaluation Office or evaluation offices of GEF partner agencies (a higher preference
will be given to countries that are relevant to any of these evaluations).
Selection of Countries for Fiscal Year 2007

33.
For Fiscal Year 2007 (July 2006 through June 2007), 5 countries in Asia were
selected up to step 4: Bangladesh, China, Philippines, Cambodia and Samoa). As described
above, the last step (Step 5) needs to be applied at this point to select the final two countries.
From the point of view of evaluability China, Philippines and Samoa seem to be the best
candidates. Although the Implementing and Executing Agencies country assistance programs
have been evaluated by the relevant Implementing and Executing Agencies evaluation offices in
all 5 cases, the three mentioned above have prepared national environmental strategies, have
conducted an initial RAF prioritization, have concluded their National Capacity Self-Assessment
(NCSA) and have participated in a country dialogue workshop. All of these activities provide
good baseline information for the country evaluations. From the point of view of synergies with
other evaluations conducted by the GEF Evaluation Office, Philippines, Cambodia and Samoa
seem to be the best fits since they have a Small Grants Programme (in particular Philippines,
which has one of the biggest and more established programs) and their portfolio would be very
useful for the capacity building and replication evaluations.
34.
There are several reasons not to select China, using these two criteria presented above.
First, although China has an extensive and very diverse portfolio there is no Small Grants
Programme. Given the limited financial resources provided by Council for this type of evaluation
it would be very difficult to conduct a Country Portfolio Evaluation in such a large country with a
huge portfolio. Finally, the Chinese government has conducted an assessment of its participation
in the GEF as well as of its portfolio. The GEF Evaluation Office proposes to consult with the
Chinese government on how this type of studies could be brought up to the attention of Council.
The case of Cambodia was not as strong as the Philippines (importance of its participation in the

7

---

GEF) or Samoa (member of two very important groups in the GEF partnership, SIDS and
LDC) and therefore was not selected.
35.
Council agreed to finance one Country Portfolio Evaluation in FY06. However, by
choosing the Philippines and Samoa for FY06, it is possible to combine two countries for the
price of one evaluation. These countries require different levels of inputs (i.e., number of
consultant/days and management of the exercise) to produce a Country Portfolio Evaluation. In
the case of the Philippines, the Country Portfolio Evaluation would require extensive input
because of the complexity of the GEF portfolio in that country. On the other hand, in the case of
Samoa, the portfolio is limited, so the evaluation could be prepared with limited investment in
consultant time and management from the GEF Evaluation Office. The annex provides a
summary of the information on Philippines and Samoa.
Fiscal Years 2008 ­ 2011

36.
For the next four fiscal years (FY08 ­ FY11), the application of steps 1 through 4
generated a list presented in Table 5. The final selection of countries for each year (1 or 2) will
be done, as explained above, following step 5, to ensure that the selection of countries reflects
the best synergies with the evaluations proposed for that particular year by the GEF Evaluation
Office. The selection criteria, including overviews of the steps taken, the implementation of these
steps and the overview of countries emerging for the FY08-FY011, have been included in a
document published on the website of the Evaluation Office.
FOURTH OVERALL PERFORMANCE STUDY AND BUDGET IMPLICATIONS
37.
The GEF has undergone four overall independent evaluations (including the one for the
Pilot Phase in 1993, OPS1 in 1997, OPS2 in 2001 and OPS3 in 2005). Each of these
evaluations contributed to the decision-making processes of the GEF Replenishment and
Assembly. These overall independent evaluations have so far all been outsourced to teams of
independent evaluators. For OPS3, the Council prepared and approved the terms of reference.
The hiring process of the evaluators was done through an open international tender, which was
awarded by the independent Operations Evaluation Department (OED) of the World Bank.
This was done to ensure full independence, which at that moment in time was not yet in place at
the GEF Monitoring & Evaluation Unit, given the fact that the Director took up his duties in
September 2006, after the tender procedure was completed.
38.
The Fourth Overall Performance Study (OPS4) will have to be completed in FY10 to
contribute to the fifth replenishment of the GEF. It will need to be perceived as a fully
independent evaluation, building on the existing independent evaluation evidence in the GEF and
producing an independent overview of results and institutional issues, which would feed into the
fifth replenishment process.
39.
Now that the GEF Evaluation Office is fully independent and a new GEF M&E policy is
in place, the Office proposed to Council in June 2006 that the Fourth Overall Performance
Study would be carried out by the Office, to ensure a smooth incorporation of evaluation

8

---

reports into OPS4 and to reduce the costs of the overall exercise. In its proposal to Council, the
Office mentioned that two elements of OPS4 would need to be outsourced in any case: an
evaluation of the M&E system of the GEF and a stakeholder consultation with all stakeholders
of the GEF. Furthermore, the Terms of Reference for OPS4 would need to be approved by
Council. The Council reacted positively, approving the underlying principles of fully integrating in
the Office's four year rolling work program the necessary inputs (i.e. evaluations) for OPS46,
but requested more information from the Evaluation Office to make a final decision at its
December meeting7.
40.
The proposal of the Evaluation Office emerged out of a meeting with its advisors in
January 2006 in The Hague and is based on the experiences with OPS3, including its cost-
overrun. It was widely felt that the OPS3 had to spend a relatively high amount of resources on
becoming familiar with the GEF and understanding the system ­ money and energy which could
have been used to deepen the empirical basis of OPS3. Furthermore, the OPS3 team was not
able to add any substance to what had already been found in the program studies, which were
the main inputs of the Evaluation Office into OPS3. The independence of OPS3 in gathering
stakeholder opinions was very much appreciated, as was the new perspective of the GEF as a
network. However, the pressing time limits and steep learning curve led to an overrun of costs
on the side of the OPS3 team, whereas the Evaluation Office part of the budget was spent
according to plan, with some minor savings.
41.
The advice of The Hague meeting was to ensure that the four year rolling work plan of
the Evaluation Office would build up towards OPS4. Throughout the years evaluations would
cover the various subjects, themes, programs and focal areas in the GEF and would thus allow
for a better integration of findings in the overall synthetic view that OPS4 would need to present
to the fifth replenishment. This vision of how the four year rolling work plan would interact with
OPS4 was presented at the Council meeting in June and the underlying principles were
accepted by Council.8 The remaining element is now to ask Council to approve the proposal
that OPS4 will be prepared, managed and implemented by the GEF Evaluation Office.
42.
On the positive side it should be noted that this will enable a better integration of
evaluation reports into OPS4 for a considerably lower price, since no external consultant team
will need to go through a learning period. Secondly, the work in OPS4 itself can be focused on
any additional empirical or qualitative work that would be needed to update findings or fill in the
gaps between reports. Lastly, it would show that the GEF has confidence in the improvements it
has made in its own M&E system, with the move towards an independent Evaluation Office at
internationally acceptable level and the adoption of a GEF M&E policy.

6 Joint Summary of the Chairs, June 22, 2006 ­ paragraph 12.
7 Joint Summary of the Chairs, June 22, 2006 ­ paragraph 75: "Many Council Members agreed with the
suggestion to ask the Evaluation Office to implement OPS4; it was also felt that further information would be
useful before a final decision would be taken."
8 Joint Summary of the Chairs, June 22, 2006, paragraph 12.

9

---

43.
On the negative side, the GEF Evaluation Office could be perceived by outsiders as not
sufficiently independent from the GEF as a whole. It is after all financed by the Council and
reports to the Council. Furthermore, the Office may be seen as having been involved too
intimately in the system and no longer having a fresh outsider perspective. The GEF may even
appear self-congratulatory if OPS4 would be positive in tone and this tone would be seen as
emanating from part of the GEF itself.
44.
The Evaluation Office feels that the advantages outweigh the disadvantages, especially
because cautionary measures can be taken to prevent negative effects. First of all, it should be
noted that any independent team of consultants that would undertake OPS4 would do this
funded by the Council and reporting to the Council, as was done for OPS3. In this regards,
hiring independent experts does not bring extra independence. Secondly, although the Office
aims to bring an unbiased perspective to its evaluations, it would in addition be worthwhile to
ensure that fresh outsider perspectives will be brought into OPS4, and mechanisms can be
incorporated that will do that. Thirdly, if OPS4 is positive in tone, it will not be considered to be
self-congratulatory if the findings and conclusions are based on solid empirical and qualitative
data and analysis and if the independence of the Evaluation Office is recognized through an
external peer review.
45.
Consequently, the GEF Evaluation Office proposes that the Fourth Overall
Performance Study of the GEF should be managed and prepared by this Office. If necessary,
the Office will contract outside specialized expertise to cover specific elements, issues or tasks
of OPS4. This would mean that no independent team of consultants would be hired for OPS4,
but that the work would be carried out by the Office and consultants hired by the Office. As
was the case for OPS3, the TOR for OPS4 would be decided by the Council, on the basis of a
first draft prepared by the Office. Nevertheless, two aspects of OPS4 should still be contracted
out to be fully independent: a worldwide stakeholder consultation and an independent evaluation
of monitoring and evaluation in the GEF.
46.
It is expected that with inflation increase over the years an OPS4 on the same basis as
OPS3 (outsourced to a team of consultants) would amount to more than US$2.7 million (a
reference point may be the Independent External Evaluation of FAO, which is currently running
for a budget of more than US$4 million, for an organization that has a lower annual budget than
the GEF). By integrating substantial portions of the necessary work for OPS4 into its regular
work program, the Office expects to achieve efficiencies of US$0.5 million to $0.7 million,
meaning that by the end of OPS4, the Office could have conducted the overall study for $2
million to $2.25 million. This will imply that the Office's budget from FY08 onwards will include
a relative portion of this cost within its budget, to reflect increases in the activities building up to
OPS4. A more detailed plan and a justification of why the efficiencies can be achieved while
increasing the scope and depth of OPS4 will be presented as part of the next four year rolling
work plan and budget to be submitted to Council for its June 2007 session.

10

---

The Budget Structure
47.
Council requested the Office in June 2006 to provide an integrated budget with clearer
guidance and priorities for special initiatives. This referred especially to the fact that some of the
special initiatives proposed, such as an extra Country Portfolio Evaluation, were considered by
many Council members to be a regular activity of the Office rather than a special, ad hoc
initiative coming on top of the regular work.
48.
The Office proposes to develop a new, integrated budget on an activity basis, which
would not follow the budgetary discipline of adding a 3 % inflation compensation increase
annually, but would adopt a four year perspective, in which activities will gradually increase
towards the year that OPS4 will be fully implemented, and will decrease in the year afterwards.
In other words: the budget would no longer show OPS4 as a special initiative, but would
incorporate the funding in its regular budget. Over the four years, budgetary discipline will be
ensured through following the four year pattern rather than fixed increases per year and by being
fully accountable for the relationship between budget and actual products delivered. In overall
terms the budget will be lower in total over four years than an annual regular budget increased
with 3 % and a special initiative to cover OPS4.

11

---

Annex I ­ Selection Criteria Applied to the Philippines and Samoa

Philippines
Samoa
RAF Allocation


Biodiversity
$21.3 mil
group (up to $3.4 mil)
Climate Change
$6.6 mil
group (up to $3.1 mil)
GEF portfolio


Completed projects
7 project - $56.07 mil
2 project - $1.10 mil
On-going projects
18 projects - $65.09 mil
5 projects - $1.00 mil
Focal Areas (projects)


Biodiversity
13 projects
4 projects
Climate Change
9 projects
1 project
International Waters
none
none
Land Degradation
1 project
1 project
POPs
2 project
1 project
Type of project


Full size
15 projects
none
Medium size
4 projects
1 project
Enabling Activity
6 projects
6 projects
IA/EAs


World Bank
9 projects - $83.95 mil
1 project - $.92 mil
UNDP
15 projects - $27.87 mil
6 projects - $1.17 mill
ADB
1 project - $9.34 mil
Selection Criteria


Policy Relevance


Place in RAF
More than $50 mil
groups
Number of Focal Areas
4-6
4-6
Number of IAs/EAs
More than 3
0-1
Types of projects
More than 3
2
Financial weight


Amount of GEF $
More than $100 mil
More than $5mil
approved
Stakeholder opinion &


demand
Large RAF recipient
SIDS and LDC
SIDS/LDC
Public and media debate


Media coverage
Often in the news -renewable
Rarely in the news
energy, ecoturism
Evaluation coverage by


GEFEO
JE, LBS, CCPS, SMPR
No EO evaluations
Evaluability


Info baseline; national
IAs/EAs country strategies and
National environmental
environmental
evaluations; NCSA completed,
management strategy, NCSA, UNDP
strategies, NCSAs, RAF
environmental strategy, RAF
country program, World Bank IEG
prioritization, previous
prioritization, IA/ExA sector review
review
IAs/EAs evaluations and
sector work
Synergy/International


collaboration
Small Grants Program evaluation
Small Grants Program (more recent
Planned evaluations
(large program and one of the
program)
more mature)
Some projects with capacity building
Extensive portfolio with capacity
and replication objectives
building and replication objectives.


12

---

13

---