Final Report


23 June 2008








Final Evaluation


RER/01/G33

Black Sea Ecosystem Recovery Project


PIMS Number: 3065
Atlas Project Number: 00039160

---

PREFACE
This report sets out the findings, lessons learned and recommendations for
the final evaluation of the UNDP/GEF Black Sea Ecosystem Recovery Project
(BSERP). The report is delivered in compliance with the Terms of Reference
developed by UNOPS, who are tasked with managing the BSERP on behalf of
UNDP. The evaluation is based upon collected reference materials from the
project, as well as a series of interviews of key stakeholders and participants
during February ­ May 2008.
The evaluators would like to thank the BSERP project implementation unit
and country team leaders for their assistance with evaluation mission logistics
and project documentation, and to thank the many project stakeholders for
taking the time discuss project achievements.
















Contacts:

Alan Fox
Transboundary Consulting, LLC
7 Hampton Rd.
Scarsdale, New York 10583

Paul Buijs
Buijswater
Sandenburg 25
3813 LX Amersfoort
The Netherlands



2

---

EXECUTIVE SUMMARY
This constitutes the final evaluation of the Black Sea Ecosystem Recovery Project (BSERP).
The evaluation is designed to enable key stakeholders to assess the relevance, efficiency,
effectiveness, impact and sustainability of the project at its conclusion. This final
evaluation report constitutes the combined outcome of a literature review and interviews
of key stakeholders and participants.
The BSERP was developed under the auspices of the Global Environmental Facility (GEF)
International Waters Program, implemented by the United Nations Development Program
(UNDP) and executed by the United Nations Office for Project Services (UNOPS). Partners
include the six coastal Black Sea countries: Bulgaria, Georgia, Romania, the Russian
Federation, Turkey and Ukraine. The participating countries are members of the Black Sea
Commission, created through the Bucharest Convention. The BSERP was launched in
2002 as one of three projects in the GEF Strategic Partnership on the Black Sea and
Danube Basin, constituting a joint capacity building and investment program aimed to
reduce nutrient loading and restore Black Sea / Danube ecosystems.
The $10 million (GEF contribution) BSERP builds upon earlier GEF support in the Black Sea
region, launched to assess and mitigate a host of human-induced threats, including
excessive nutrient loading, over-exploitation of fish stocks, invasive species outbreaks, ill-
conceived coastal development, the dumping of untreated solid waste and sewage, and
periodic oil spills.
The BSERP was developed to assist in the implementation of practical measures to restore
and protect the Black Sea environment as agreed by the coastal countries n the BSSAP
(1996). The most important transboundary issue indentified in the Transboundary
Diagnostic Analysis (TDA) and addressed in the SAP was the need to reduce nutrient
inputs to the Black Sea from riverine and land-based sources. In preparation for the 2nd
phase of the BSERP (2005), project outputs were revised out of concern that the
conditions of the Black Sea and the principal causes of perceived degradation were still not
well understood, consequently, more information was needed in the form of an updated
and expanded Transboundary Diagnostic Analysis. There was also recognition of
shortcomings in the Strategic Action Plan (BSSAP ­ 1996), previously agreed to, but not
well implemented by the Black Sea countries. Consequently, the Project included as a key
2nd phase outcome a revised SAP, providing practical measures to restore and protect the
Black Sea environment. Throughout both project phases the overall development aim was
to help the Black Sea countries develop policies, laws and programmes to reduce the loads
of nutrients and hazardous substances discharged to such levels as necessary to permit
Black Sea ecosystems to recover to conditions similar to those observed in the 1960s.
BSERP also was designed to help strengthen the capabilities of the BSC Permanent
Secretariat.
The project was funded in two phases due to financial considerations for GEF. The BSERP
Phase 2 Project Document and Logical Framework Matrix list five specific objectives:
1. Support for the consolidation and operation of institutional mechanisms for
cooperation under the Black Sea Convention
2. Development of policy guidelines, legal and institutional instruments for pollution
reduction from land based activities (LBA) and protection of ecosystems of the
Black Sea and coastal zones.
3. Development of economic instruments and promotion of investment opportunities
in coastal zones for pollution control and protection of Black Sea ecosystems.
4. Development of operational systems for monitoring, information management and
research under the Black Sea Convention
5. Strengthening of public participation in environmental protection through access to
information, stakeholder training and awareness raising and implementation of
community actions (Small Grants Programme).

3

---

Findings
Some of the key indicators for BSERP success were related to regional and national policy
and legislative developments. These were based on technical assistance to the Black Sea
Commission and the six countries to establish a land based activities protocol, a
negotiated fisheries agreement, and coastal zone management strategies. The progress
made in these policy areas was significantly less than had been expected at project
inception; however, while there were some difficulties in project implementation at the
PIU, it is the countries themselves that must be held accountable for the slow pace of
policy change.
In areas where the project team was less dependent on government decision-making
there is a much more positive story to tell. Good work was done by the project team to
better understand the status of the Black Sea ecosystem, to involve NGOs and enhance
their capabilities, to establish monitoring and information systems, and to build public
awareness and appreciation for Black Sea issues, especially through the annual Black Sea
Day celebrations.
During Phase 1, the BSERP encountered management problems, necessitating the
replacement of two Chief Technical Advisors (CTAs). During this period the project
suffered especially due to a dysfunctional and counterproductive relationship with the BSC
Permanent Secretariat. The BSERP ­ BSC relationship suffered in part from a large gap in
expectations concerning the nature of the BSERP role and the extent to which it should act
as a subsidiary body to the Commission. Working relationships were resolved during the
2nd phase.
In 2004 UNDP and UNOPS made the decision to further link the Danube and Black Sea
projects by appointing a Regional Manager (the CTA of the Danube Regional project). The
idea was for the strong deputy CTAs for each project to take on greater day to day
responsibilities and the Regional Manager to play a coordinating and policy level role. This
hybrid approach enabled the BSERP to move forward without a protracted CTA selection
process; however the approach was not ideal given the extensive need for high level
attention to turn the BSERP around. The arrangement ended up succeeding largely due to
the policy and managerial strengths of the selected regional Manager, as well as the
capabilities of experts in both PIU staffs.
The time frame for project task completions experienced some drift. The TDA revision
effort commenced during Phase 2, and the TDA was finalized in May, 2007, six months
later than planned. Phase 1 inputs and problems with the quality of data received from
the countries through the BSC meant considerable time was required in the early months
of Phase 2 to get the TDA effort moving. Delays in receiving data and comments from
several Advisory groups also held up the TDA development process. The completion of a
revised strategic action plan (SAP) slipped, and the process is still underway as the BSERP
PIU closes. Extension contracts for consulting assistance to the BSC through May ­ June
2008 have succeeded to revise and complete a document that (informally) is now agreed
to by five of the six contracting parties. Several other expected outputs were also issued
only at the end of the project, including training for and transfer of the Black Sea
Information System (BSIS).
The overall budget for the project was appropriate, recognising the geographic size of the
Black Sea region, the capacities of the countries involved, the significant pollution
pressures addressed, the presence of the two additional support projects within the GEF
Strategic Partnership, and a parallel support framework through the European Commission
(Tacis).
A leading cause of nutrient loading into the Black Sea is from agricultural sources,
however the BSERP was designed only to survey and inventory the regions agricultural
situation and provide a small amount of training on best agricultural practices.
The BSERP can point to successful adaptations, for instance in the decision to couple the
BSC Commissioner and BSERP Steering Committee Member roles during the project
second phase and organising back-to-back Commission meetings and Project Steering
Committee meetings. This change helped forge closer cooperation between the project

4

---

and Commission, and reduced steering meeting time and cost. Adaptive management is
evident in the decision to downgrade certain activities where there was not enough
financial weight to make much impact, for instance concerning studies on air-borne
dispersion of nitrogen.
The BSERP during its second phase put significant time and effort towards strengthening
the BSC and its Secretariat, through financial support for meetings, etc, and in particular
through developing an institutional review and exit strategy, both designed to ensure
management effectiveness and sustainability of the BSC after the conclusion of the BSERP.

Changes in the staffing of the Permanent Secretariat have increased its technical
capability, and while still understaffed have increased its ability to meet planned
objectives. Individual country contributions have also strengthened the future viability of
the Commission and its Secretariat, in particular the host country ­ Turkey has indicated
its willingness to contribute considerably more than the negotiated $60,000 per country
commitment. Nevertheless, difficult decisions over the role and functioning of the Advisory
Groups still need to be made, and increased funding of the PS from all countries will be
required. Future funding of the Secretariat from the EU may be a future option; however
this may hinge on achieving consensus of the 6 countries to invite the EU as a contracting
party, and will likely require the Secretariat to demonstrate improved effectiveness
Lessons and Recommendations
Included in section 3.4 and 3.5 of the full report are a series of lessons learned and
recommendations for the Black Sea counties and for UNDP/GEF. Highlights include:
After 16 years, its time to call a hiatus to GEF support in the Black Sea ­ on a
regional basis. In particular, it will be important for the Black Sea Countries to
show a good faith effort in achieving the SAP agreements and ratifying and
implementing protocols, before the GEF and other funders provide more assistance
at the regional level. There should be opportunities for GEF to continue providing
support at the national and local levels, for example on integrated coastal zone
management, nutrient discharge reduction and climate change adaptation. The GEF
small grants program should be used as a mechanism to continue providing
support to local NGOs and building on previous public awareness efforts.
The revised TDA is a significant improvement over its predecessor; however it took
on too much importance and was perceived by several stakeholders to be an
external / consultant driven exercise. Future TDA revision support efforts should
become part of the ongoing efforts of the transboundary waters commissions and
their subsidiary bodies, with GEF projects playing more of a supporting role. Of
course, to take on this task requires that commissions are adequately funded and
have the requisite technically-competent staff.
The BSERP provides important lessons on the use of independent scientific advisory
groups like the ISG, which can bring top regional scientists into projects, and help to
keep applied research efforts focused on scientific discovery. A key challenge is to
develop a clear set of agreed objectives and timetables for ISG deliverables, closely
aligned to the overall project objectives.
The management and outcomes of research cruises is another area where the
BSERP provides important lessons. The four cruises carried out during the BSERP
took a considerable amount of time and financial resources ­ more than originally
anticipated. While the cruises were of scientific use ­ for instance in helping
substantiate the gradual recovery of the Black Sea western shelf and Phyllophora
fields, greater use could have been made of the data obtained. In addition, much
more attention should be paid to the opportunities for media, public awareness and
educational opportunities derived from these cruises.
Future GEF projects will encounter a similar debate as the BSERP regarding hiring
Country Team leaders. While the CTL hiring has been seen by many stakeholders
as being beneficial to the implementation of the BSERP, there are significant
downside risks:

5

---

o This can constitute a major cost factor for multi-country projects. What is
typically expected as a co-financing cost born by the partner countries
becomes an administrative cost to the project ­ thereby reducing available
funds for demonstration projects and other technical assistance.
o There is an important issue at stake concerning country ownership and
sustainability. If the countries are not financially responsible for in-county
project management, they are less likely to have a stake in the outputs and
productivity of the CTLs and project as a whole. Also, if the in-country
coordination is paid for by the project ­ then at the end of the project there
are likely to be problems in terms of sustaining the effort. The BSERP
devised a phase out strategy over the final 18 months to scale back CTL
financial support ­ with the expectation that the countries would escalate
their support.
o If CTLs are to play a technical role they need technical skills. This requires a
transparent CTL selection process that can identify persons with the
requisite technical capabilities.
o CTLs paid for by the project but selected by the responsible Ministry officials
encounter loyalty and ,,chain of command pressures. Who are they
responsible to and how can they be replaced if they fail to perform their
proscribed duties?
The BSERP included an institutional review for the BSC as well as an exit strategy.
These are useful tools to consider for GEF projects that are aiding transboundary
waters commissions, especially to help pave the way for post-project financing and
sustainability. Key to success is a real commitment on the part of the Commission
to cooperate with the management consulting effort and implement the resulting
recommendations. The timing of exit strategy development is also crucial. The
BSERP/BSC effort would have benefited from an earlier start to this process ­ soon
after the start of the 2nd phase. 2 years is the minimum time needed to put such
strategies in place to secure replacement funding and to decide on how best to
proceed with activities launched and supported by the GEF project.
The BSC should revise the Secretariat work plan and set a more realistic
expectation for activities, commensurate with its budget.
The BSC should act on the BSERP recommendations to revise the Bucharest
Convention, SAP and other instruments to reduce the number of Advisory/Expert
Groups to three and to dispense with the Regional Activity Centres. The staff size at
the Secretariat is not sufficient to coordinate the work of 16 subsidiary bodies and
the existing budget does not provide enough support for these bodies to meet.
Further regional studies should be conceived and funding sought to improve the
accuracy of calculations on the extent of air born and non point source runoff
problems in the Black Sea.
The interministerial coordination effort through the BSERP has not led to a
noticeable expansion in the involvement of other local and national government
ministries, beyond the identified responsible agencies. The lack of progress is
mostly due to a lack of interest, and in some cases strong resistance, from the
responsible agencies to this type of assistance. Future projects should avoid an
overarching interministerial expectation, but rather to include interministerial
coordination within actual planned project outputs and activities. For instance, a
planned output to negotiate a fisheries protocol should include the engagement of
ministries responsible for fisheries. Efforts to develop integrated coastal zone
management should include meetings, training and demonstration/replication at
the local government level.


6

---

TABLE OF CONTENTS
EXECUTIVE SUMMARY .................................................................................................................................. 3
ACRONYMS .......................................................................................................................................................... 8
1
INTRODUCTION TO THE EVALUATION ...................................................................................... 9
1.1
PURPOSE OF THE EVALUATION.............................................................................................................. 9
1.2
STRUCTURE AND METHODOLOGY OF THE EVALUATION ...................................................................... 9
2
THE BSERP AND IT'S DEVELOPMENT CONTEXT ................................................................... 9
2.1
PROJECT BACKGROUND ......................................................................................................................... 9
2.2
PROBLEMS ADDRESSED ....................................................................................................................... 10
2.3
PROJECT OBJECTIVES .......................................................................................................................... 11
2.3.1
Project Phases ........................................................................................................................ 12
2.3.2
Outcomes & Indicators ....................................................................................................... 12
3
FINDINGS ................................................................................................................................................. 13
3.1
PROJECT FORMULATION ....................................................................................................................... 13
3.1.1
Appropriateness of the project concept and design ................................................ 14
3.1.2
Project contribution to overall development objectives ......................................... 15
3.2
PROJECT IMPLEMENTATION ................................................................................................................. 15
3.2.1
Project management ........................................................................................................... 16
3.2.2
Quality of inputs and activities ........................................................................................ 16
3.2.3
Budgeting ................................................................................................................................. 17
3.2.4
Major factors facilitating/impeding project implementation ................................. 18
3.2.5
Adaptive management ........................................................................................................ 19
3.2.6
BSC Management and execution .................................................................................... 19
3.2.7
Danube /Black Sea Strategic Partnership ................................................................... 20
3.3
PROJECT IMPACTS ................................................................................................................................ 21
3.3.1
Achievements of the project against objectives, outputs and activities.......... 22
3.3.2
Country ownership ...................................................................................................................... 35
3.3.3
Stakeholder and public awareness. ............................................................................... 36
3.3.4
Cost-effectiveness ................................................................................................................ 36
3.3.5
Sustainability of project impacts..................................................................................... 37
3.4
BEST PRACTICES AND LESSONS LEARNED .......................................................................................... 41
3.5
RECOMMENDATIONS ............................................................................................................................ 43
3.5.1
Black Sea Countries and Commission ........................................................................... 43
3.5.2
UNDP/GEF ................................................................................................................................ 45
3.5.3
Ratings ...................................................................................................................................... 46

Annexes
Annex 1 -- TOR for Terminal Evaluation
Annex 2 -- Mission Interviews
Annex 3 ­ List of Documents Reviewed

7

---

ACRONYMS
APC/EG
Accident Prevention and Control Expert Group
AG CBD
Black Sea Commission Advisory Group on the Conservation of Biological
Diversity
AG ECO/EG
Black Sea Commission Advisory Group on Ecology
AG ESAS
Black Sea Commission Advisory Group on Environmental and Safety Aspects of
Shipping
AG F/MLR
Black Sea Commission Advisory Group on Fisheries and other Marine Living
Resources
AG ICZM
Black Sea Commission Advisory Group on the Development of Common
Methodologies for Integrated Coastal Zone Management
AG IDE
Black Sea Commission Advisory Group on Information and Data Exchange
AG LBS
Black Sea Commission Advisory Group on Control of Pollution from Land Based
Sources
AG PMA
Black Sea Commission Advisory Group on Pollution Monitoring and Assessment
APR
Annual Project/Program Report
AQA
Analytical Quality Assurance
AQC
Analytical Quality Control
BAP
Best Agricultural Practices
BAT
Best Available Technology
BOD
Biological Oxygen Demand
BSC
Black Sea Commission
BSERP
Black Sea Ecosystem Recovery Project
BSIMAP
Black Sea Integrated Monitoring and Assessment Programme
BSIS
Black Sea Information System
BSNN
Black Sea NGO Network
BSSAP
Black Sea Strategic Action Plan
CAP
Common Agricultural Policy
DABLAS TF
EU Danube Black Sea Task Force
DRP
Danube Regional Project
EC
European Commission
EU
European Union
EUR
Euro
GDP
Gross Domestic Product
GEF
Global Environment Facility
GIS
Geographical Information System
GPA
Global Program of Action, UNEP
HELCOM
Baltic Marine Environment Protection Commission (Helsinki Commission)
HoD
Head of Delegation
ICPDR
International Commission for the Protection of the Danube River
IFI
International Financing Institution
IPPC
Integrated Pollution Prevention and Control Directive
ISG
International Study Group (of the BSERP)
IW
International Waters
LBD
Legally Binding Document
LFM
Logical Framework Matrix
M&E
Monitoring and Evaluation
MOU
Memorandum of Understanding
MTE Report
Mid-Term Evaluation Report
NGOs
Non Government Organizations
OP8
Operational Program 8
PIU
Project Implementation Unit
PIR
Project Implementation Review
ProDoc
Project Document
RBM
River Basin Management
RAC
Regional Activity Centre
REC
Regional Environmental Centre
SAP
Strategic Action Plan
TACIS
EU Support Program for Russia and the former CIS
TDA
Transboundary Diagnosis Analysis
UNDP
United Nations Development Program
UNEP
United Nations Environmental Program
UNOPS
United Nations Office for Project Services
WFD
Water Framework Directive

8

---

1 INTRODUCTION TO THE EVALUATION
1.1 Purpose of the Evaluation
The final evaluation is designed to enable key stakeholders to assess the relevance,
efficiency, effectiveness, impact and sustainability of the BSERP at its conclusion. The
evaluation provides an assessment of achievements against objectives, including a re-
examination of the relevance of the objectives and project design. The report considers
factors that have facilitated and impeded the achievement of objectives and provides
lessons learned and recommendations for the UNDP, GEF and participating countries.
1.2 Structure and Methodology of the Evaluation
The evaluation has been structured in accordance with UNDP Guidelines for Evaluators. It
covers the issues set out in the Terms of Reference for this evaluation, and takes into
account the expectations of UNOPS.
This final evaluation report constitutes the combined outcome of a literature review and
interviews of key stakeholders and participants. The evaluation team was present at the
closing BSERP Steering Committee meeting in late February, 2008 and then travelled to
each of the 6 countries that are signatories to the Black Sea Convention to conduct
interviews with key stakeholders. Subsequent interviews were carried out in New York,
Washington DC and The Hague, as well as follow on phone and email contacts. In Annexes
2-4 are the mission itinerary, persons interviewed and documents reviewed.
The use of stakeholder interviews as the lead vehicle for evaluation has been done
recognizing that the BSERP is a capacity building and "influencing" project, designed to
build stakeholder support for improved coastal and marine resources management.
Evaluation Report Components
The evaluation report has been developed consistent with the expectations set out in the
Terms of Reference for the Assignment. The TOR is included as Annex 1 to this report.
Following the standard UNDP evaluation format, the report sets out the development
context for the project and followed by an assessment of the project design and project
implementation. Lessons learned and conclusions based upon the project results are
then provided. Per the expectations set out in the TOR, the evaluators have utilised a
four step rating system (highly satisfactory, satisfactory, marginally satisfactory and
unsatisfactory) on the following criteria: a) outcomes/ achievement of objectives; b)
implementation approach; c) Stakeholder participation / public involvement; d)
Sustainability; and e) Monitoring & Evaluation.
2 THE BSERP AND IT'S DEVELOPMENT CONTEXT
2.1 Project Background
The full title of the project is the Black Sea Ecosystem Recovery Project. . The Project was
developed under the auspices of the Global Environmental Facility (GEF) International
Waters Program, implemented by the United Nations Development Program (UNDP) and
executed by the United Nations Office for Project Services (UNOPS).
The history of coordinated efforts to stem the accelerating degradation of the Black Sea
environment dates to the early 1990s. A Diplomatic Conference on the Protection of the
Black Sea, held in Bucharest Romania, April 1992, paved the way for adoption of the
Convention on the Protection of the Black Sea against Pollution (Bucharest Convention - in
force since 1994 after ratification by the Black Sea riparian states: Bulgaria, Georgia,
Romania, Russia, Turkey and Ukraine). The Convention objectives are to undertake all
necessary measures consistent with international law to prevent, reduce, and control
pollution discharges into the Black Sea in order to protect and preserve the marine
environment. The Bucharest Convention was further elaborated and implemented through
the Declaration on the Protection of the Black Sea, signed in Odessa Ukraine, April 1993.

9

---

An analysis of root causes, termed the Black Sea Transboundary Diagnostic Analysis
(TDA), was completed in June 1996. Based from the TDA, a Strategic Action Plan for the
Rehabilitation and Protection of the Black Sea (BSSAP) was agreed to by the riparian
states during their meetings in Istanbul, Turkey, October 1996. This meeting also formally
launched the Black Sea Commission (aka BSC, or Istanbul Commission) and its subsidiary
bodies (Advisory Groups and Regional Activity Centres). At a meeting of the Istanbul
Commission, held April 2000, agreement was reached on institutionalizing a Secretariat to
the Commission.
External support to the Black Sea Commission commenced soon after its founding. The
Black Sea Environmental Program (BSEP), launched in June 1993, included a series of
projects supported by the Global Environmental Facility (GEF), under its International
Waters (IW) program. Other donors have also provided strong support to the Black Sea
Commission, especially the European Union, through its Phare and Tacis programmes.
The BSEP was established in response to regional and global concern about the critically
degraded environmental conditions in the Black Sea and the positive policy initiatives
taken by the six Black Sea governments at their Odessa meeting. An initial three-year GEF
Pilot Phase project, entitled "Environmental Management and Protection of the Black Sea
(RER/93/G31)", was launched to: (1) create and strengthen regional capacities to manage
the Black Sea ecosystem, (2) develop an appropriate policy and legislative framework for
the assessment, control and prevention of pollution and maintenance and enhancement of
biodiversity, and (3) facilitate the preparation of sound environmental investments.
Under the GEF, assistance continued with a second regional project, implemented until
1998. GEF support was maintained at a low level during an interim period while the Black
Sea riparian countries developed national strategic action plans, and while the launch of a
BSC Secretariat was negotiated.
Recognizing the interrelated nature of the Black Sea and Danube River basin ecosystems,
a combined GEF IW technical support and investment mechanism was launched in 2001.
The "GEF Strategic Partnership on the Black Sea and Danube Basin" is a US$ 97 million
capacity building and investment program aimed to restore Black Sea / Danube
ecosystems to their 1960s condition. The Framework Brief for the partnership, submitted
to GEF Council in May 2001, established the "Investment Fund for Nutrient Reduction in
the Black Sea / Danube Basin" (Implementing Agency: World Bank; budget: US$ 70
million); the "Danube Regional Project (UNDP; US $17 million) and the "Black Sea
Ecosystem Recovery Project" (UNDP, US$ 10 Million). The project includes an
implementation unit (PIU) in Istanbul Turkey, housed together with the Permanent
Secretariat to the International Commission of the Black Sea (BSC).
2.2 Problems Addressed
The UN Environmental program (UNEP) was instrumental in helping to develop the Black
Sea Transboundary Diagnostic Analysis TDA (1996). Its characterization of the
environmental degradation at that time provided a suitable frame for considering the
problems that the BSERP then sought to address:
"In a period of only three decades, the Black Sea has suffered the catastrophic
degradation of a major part of its natural resources. Increased loads of nutrients from
rivers caused an overproduction of tiny phytoplankton, which in turn blocked the light
reaching the sea grasses and algae, essential components of the sensitive ecosystem of
the north-western shelf. The entire ecosystem began to collapse. This problem, coupled
with pollution and irrational exploitation of fish stocks, started a sharp decline in fisheries
resources. To make matters worse, in the mid 1980s, a jellyfish-like species (Mnemiopsis
leidyi), which was accidentally introduced to the Black Sea from the eastern seaboard of
America in the ballast water of a ship, invaded the Black Sea. Poor planning has destroyed
much of the aesthetic resources of the coastlines. Uncontrolled sewage pollution has led to
frequent beach closures and considerable losses in the tourist industry. In some places
solid waste is being dumped directly in the sea or on valuable wetlands. Tanker accidents
and operational discharges have often caused oil pollution. All of this came at a time when

10

---

five of the Black Sea countries were facing an economic and social transition and were
unable to take the necessary urgent remedial actions".
The above scenario remained relevant as a basis for developing and launching the BSERP
in 2001-2002. The underlying problems still remained, although a decade of
environmental progress coupled with shifting economic patterns and reduced agricultural
production had slowed the eutrophication and ecological deterioration of the previous
period.
2.3 Project Objectives
As indicated when the project was in its initial stages (PDF-B), the main purpose of the
BSERP was to assist in the implementation of practical measures to restore and protect
the Black Sea environment, as agreed by the coastal countries in the BSSAP (1996). The
most important transboundary issue identified in the Transboundary Diagnostic Analysis
(TDA) and addressed in the SAP, was the reduction of nutrient inputs to the Black Sea
from riverine and land-based sources. The SAP also provided a policy framework to
implement priority actions and address transboundary environmental concerns associated
with, amongst others, sustainable fisheries, tourism, conservation of living marine
resources and coastal landscapes, and pollution from shipping and land-based sources.
Immediate and development objectives for the BSERP built on the main objective to "take
measures to reduce the loads of nutrients and hazardous substances discharged to such
levels as necessary to permit Black Sea ecosystems to recover to conditions similar to
those observed in the 1960s". A corresponding intermediate goal was for urgent
measures "to be taken in the wider Black Sea Basin in order to avoid that the loads of
nutrients and hazardous substances discharged into the Seas exceed those that existed in
the mid 1990s". It is important to note that the verification of success in meeting these
two goals was made difficult by the substantial information gaps concerning the health of
Black Sea ecosystems during these target periods (mid-1990s and 1960s).
Overall the project was expected to significantly contribute to the recovery of the Black
Sea marine ecosystem by assisting the Black Sea countries to:
Reduce nitrogen and phosphorus loading into the Black Sea;
Enhance the service function of wetlands and benthic (seabed) plant communities
for the assimilation of nutrients;
Improve management of critical habitats to permit economic recovery of fisheries in
parallel with improvements to the ecosystem;
Reduce transboundary contamination by hazardous substances, particularly where
these have similar sources to nutrients.
The project aimed to help the Black Sea countries achieve these objectives by:
1. Reinforcing institutional mechanisms of the Black Sea Commission, building
cooperation amongst the Black Sea countries, and achieving efficient
implementation of joint policies and actions, and the operation of common
management and control mechanisms.
2. Expanding knowledge of the environmental status of the Black Sea, and monitoring
trends.
3. Taking actions to mitigate and reduce inputs leading to eutrophication, and other
environmental threats such as hazardous substances and over fishing;
4. Supporting the development of legal instruments for national efforts to manage and
protect Black Sea natural resources;
5. Establishing mechanisms to build stakeholder awareness and support for the Black
Sea within the 6 riparian states;
6. Designing and applying economic instruments to pay for environmental
interventions needed to improve Black Sea water quality.

11

---

2.3.1 Project Phases
The project was funded in two phases due to financial considerations for GEF. This two
phased approach required development and approval of a second ProDoc at the
completion of the first phase (2004). There was also a mid term evaluation carried out
after the Phase 2 ProDoc had been approved. The MTE recommendations were then taken
into account by the project team and implemented through the Project Implementation
Plan and revisions to the LogFrame.
2.3.2 Outcomes & Indicators
As required, the BSERP Team developed a Logical Framework Matrix for its Phase 2
activities which sets out the long term and specific objectives of the effort, the verifiable
indicators of achievement, sources of verification as well as key assumptions and risks.
The project was organised around a set of five specific objectives, which served as the 5
major project components, to which were attached corresponding outputs and activities.
There is an ongoing effort at GEF to improve and standardize indicators, and draft
guidance has been developed that differentiate between five types of outcome indicators:
project process, catalytic process, project stress reduction, catalytic stress
reduction and environmental status. The BSERP Phase 2 ProDoc was developed prior
to this guidance. The Project Logical Framework Matrix contains a set of verifiable
indicators, which are presented below using the above typology. It is important to note
that all of the indicators set out for the project, both at the objective and output level, are
process related. There is no indication that project activities were expected to have a
direct impact by reducing stress to the ecosystem or changing its environmental status.
Objective 1: Supporting the consolidation and operation of institutional mechanisms for
cooperation under the Black Sea Convention.
Verifiable indicator for objective 1: At the end of the Project the institutional
mechanisms of the Black Sea Commission are reinforced and fully operational
ensuring cooperation between all Black Sea countries to efficiently implement joint
policies and actions and operate common management and control mechanisms.
(Catalytic process)
Objective 2: Development of policy guidelines, legal and institutional instruments for
pollution reduction from land based activities (LBA) and protection of ecosystems of the
Black Sea and coastal zones.
Verifiable indicator for objective 2: Policies and legal and institutional
instruments in all Black Sea countries are revised and reinforced to assure
sustainable coastal zone and marine resource management while reducing nutrients
and hazardous substances through the application and translation into concrete
actions of revised policies and legislation in the agricultural, industrial, transport and
municipal sectors. (Catalytic process)
Objective 3: Development of economic instruments and promotion of investment
opportunities in coastal zones for pollution control and protection of Black Sea ecosystems.
Verifiable indicator for objective 3: Economic analysis taking into account the
principles of EU WFD guidelines is carried out in all Black Sea countries and most
cost-effective measures for pollution control and water use are identified and control
systems (incl. pollution charges, fines and incentives) are developed and accepted at
the national level in the Black Sea countries. (Project and catalytic processes)
Objective 4: Development of operational systems for monitoring, information
management and research under the Black Sea Convention.
Verifiable indicator for objective 4: Institutional and organizational mechanisms
for transboundary cooperation in water quality monitoring and information
management including GIS are established and fully operational at the regional and
national level by 2006 to assess water quality and nutrient reduction to the Black
Sea; at the same time, results from scientific research in nutrient reduction and

12

---

eutrophication are available to enhance reporting on the status of the Black
Sea.(project and catalytic processes)
Objective 5: Strengthening of public participation in environmental protection through
access to information, stakeholder training and awareness raising and implementation of
community actions (Small Grants Programme).
Verifiable indicator for objective 5: The civil society and in particular national
NGOs in all Black Sea countries are at the end of the project informed and
proactively participating in national programmes for nutrient reduction, coastal zone
management and protection of coastal and marine ecosystems (project and catalytic
process).
3 FINDINGS
3.1 Project formulation
The BSERP project design had direct relevance within the GEF guidelines as well as in
terms of global concerns for the Black Sea. During ProDoc development there were
concerns that the conditions of the Black Sea and the principal causes of perceived
degradation were not well understood, consequently, more information was sought, in the
form of an updated and expanded Transboundary Diagnostic Analysis, bolstered by a fairly
robust (applied) research program. There was further recognition of shortcomings in the
Strategic Action Plan previously agreed to, but not well implemented by the parties, so the
Project included plans for a revised SAP. Central to any updating process should have
been an assessment of the successes and failures of the previous TDA/SAP process, and a
gap analysis on the status of achievement for the existing BSSAP. Such ,,lessons learned
were not emphasised in the first Project Document, however in 2000, the original SAP
implementation was only 4 years in progress, so it was perhaps a bit premature to already
anticipate the slow pace of country efforts to meet agreed actions. However, the second
phase ProDoc was developed after the Ministerial Meeting in Sofia on June 14, 2002,
where decisions were made to revise and extend most of the deadlines for country action.
This should have been a signal that the countries were not prepared to meet their
obligations, and this risk should have been more fully explored within the ProDoc and LFM.
The project was designed in a period when it was known that a substantial part of the
Black Sea basin was relatively ,,data and information poor. For instance most national
monitoring programmes were under-performing during the 1990s, generating meagre,
sometimes erroneous and often inaccessible data. At the start of the new millennium
there was little basis for optimism that this would rapidly improve, given further reductions
in marine research budgets, and a continuing reluctance on the part of governments and
institutes in the region to share their data. Yet several project components hinged on the
sharing of reliable and robust data, for example to assess the environmental status of the
Black Sea and properly link the status with actual pressures and then to propose effective
measures. Although the project design included activities that would generate new data
and information, it could have been anticipated that the results of field work, for instance
from the research cruises, would take time to produce and assess and would only be
available late in the project cycle.
There was genuine concern that the Black Sea was experiencing major eutrophication
problems, largely due to discharges from the major tributary rivers, in particular the
Danube, caused by excess nutrient loading. Consequently, the project focused on nutrient
issues and drew attention to non point source as well as point sources. Unfortunately, this
focus did not translate into specific activities designed to reduce the leading cause of
nutrient loading: agriculture. All that was expected under the BSERP was to survey and
inventory the region concerning agriculture nutrient sources and to do a bit of training on
best agricultural practices.
The project design reflects a major concern about the state of fisheries in the Black Sea. It
proposed as an output the development of a draft fisheries protocol to the Black Sea
Convention. The problem in this design is that the Black Sea Commission and

13

---

participating countries were under no compulsion to do anything with the draft protocol
that was created, and consequently no real progress was made towards the formation of a
binding agreement on fisheries. The 2nd tranche log frame indicted as a risk that the
parties would draw out their negotiations on how best to develop a legally binding
document on fisheries (LFA Output 2.6). This indeed occurred, and the fisheries
component was largely unsuccessful. It must be recognised that this delivery ,,failure had
as its major cause a barrier that was beyond the project to overcome. The key barrier to
moving this initiative forward was the new legal situation once Romania and Bulgaria
became members of the European Union. Both countries lost sovereignty over the
fisheries agenda, and the EC had no standing within the Black Sea Commission to
negotiate with the other four countries on behalf of Romania and Bulgaria. Unless or until
this legal barrier is overcome, through approval of the EU as a contracting party, the Black
Sea commission will have limited authority on Black Sea fisheries issues
The project included institutional strengthening and capacity building components, notably
for the Black Sea Commission and its Permanent Secretariat. This was a critical part of the
project, as it was designed to help with long term sustainability of the Black Sea protection
effort. The capacity building aspects were under-whelming in the 1st tranche document
and the approved 2nd tranche ProDoc. It was significantly strengthened only after the
project mid-term evaluation, which suggested including an Institutional Review for the
BSC and an Exit Strategy.
The Project Design development succumbed to a typical problem for large GEF IW
projects: an overly ambitions array of expected outputs and activities, each of which was
important, but each of which was given insufficient financial support to have a real and
lasting impact. The BSERP Phase 2 ProDoc had 14 planned outputs each with 5 ­ 7
related activities, covering applied research (joint sea research cruises), information
system development (BSIS), institutional strengthening, small grants and NGO support,
media and communications strategies and policy development. The Phase 2 project
document was developed in full awareness of the overly ambitious project expectations;
however the modified LogFrame nevertheless still implied considerable achievements in
areas where there were insufficient resources.
The team considerably reduced Output 3.1 activities focused on economic analysis, yet
with a very limited budget still indicated several deliverables. The resulting reports have
been disseminated at the end of the project seminar; however it is difficult to imagine they
will get much traction in the Black Sea countries without significant additional funding to
adapt the information to local economic conditions. The BSERP was not designed and
budgeted in a way that would enable much economic work to be done, especially
recognising the complexity of the subject and the difficulties that many Black Sea
countries will face in generating the kind of data required. This situation points to a
greater issue for UNDP/GEF, and that is the generally low level of attention paid to
economic studies, and cost-benefit analysis in transboundary waters projects. A key
question to try and answer in all of the transboundary projects is what are the economic
consequences of greater ecosystems protection? And its corollary: What are the impacts if
nothing gets done? Future projects of similar magnitude to the BSERP should employ a
dedicated economist to work with the contracting parties on making an economic case for
the policy changes being advocated.

As indicated in the following sections of the evaluation report, there were some
shortcomings in the quality and extent of implementation and achievement of the BSERP.
These are problems that had their germination in the project formulation phase, as they
are the results of an overly-ambitious ToR, under-resourcing to meet the ToR and a failure
to agree with the Black Sea Commission on project priorities. In particular, more attention
should have been paid during project formulation to structure and clarify the working
relationship between the Project Team and the BSC Secretariat.
3.1.1 Appropriateness of the project concept and design
It would have been difficult for any project document developer to envision the significant
institutional changes that would transform the region during the project time period. At

14

---

the time of its creation, the concept was relevant. The period 2000-2008 has been quite
dynamic from socio-economic and political points of view. In several countries,
governments changed frequently; the status of Bulgaria and Romania altered from ,,pre-
accession to full EU Member States; the economy in most countries has not prospered
during the project period.
Most Black Sea countries have been oscillating between greater centralisation of political
power and greater regional and local autonomy. As with other GEF IW project efforts, the
BSERP was designed with a central government focus, yet the key day to day decision
making with the greatest impact on coastal ecosystems is occurring at the local level. A
stronger local government focus of the project design, especially for the integrated coastal
zone management effort, may have enabled the project to achieve greater replication
success.
The projects Logframe for Phase 2 takes note of many assumptions and risks, all of them
relevant in the context of the Black Sea basin. Some risks indeed were substantiated, like
the (lack of) timely and reliable data. Several assumptions either directly or indirectly
referred to the commitment of the Black Sea basin countries as being a significant risk.
This risk was likewise well-substantiated.
3.1.2 Project contribution to overall development objectives
The overall objective aimed to support the participating countries in the development of
policies and legislation and the definition of priority actions to reduce the discharge of
nutrients. The emphasis was on regional, overarching action plans and protocols rather
than national policies and legislation. There were some activities designed to assist with
defining priority national and local actions, for instance ICZM strategies and demonstration
projects (see Output 2.2) and feasibility studies for treatment works investments (output
3.2) however there was no replication strategy envisioned to build on these targeted
efforts.
3.2 Project implementation
In general the BSERP faced major difficulties in its implementation and management. At
one point just prior to the launch of Phase 2 it was at risk of being cancelled due to
performance problems. At the time of the mid term, the relationship between the Project
PIU and BSC Secretariat was highly dysfunctional. During the project second phase,
changes in personnel of the Project and BSC contributed to a significant improvement in
cooperation and a significant improvement in project implementation. This cooperation
was augmented by mid-project changes which placed even greater importance on
institutional strengthening and program assistance to the BSC.
The BSERP outputs were supposed to especially focus on policy development, including a
land based sources protocol, a Legally Binding Document (LBD) on Fisheries,
improvements in coastal zone management, and steps to reduce nutrient loading. While
there were clearly some difficulties in project implementation that can be set at the feet of
the PIU, it is the countries themselves that must be held accountable for deficiencies in
project implementation. A lack of country ownership, and in some cases country
opposition to the BSERP effort, made it extremely difficult for the PIU to effectively carry
out its assignments.
The BSERP was developed as part of a strategic partnership that included the UNDP ­
implemented Danube Regional Project and the World Bank ­ implemented nutrient
reduction facility. Special effort was made to link these three initiatives, including annual
combined meetings. In the case of the DRP and BSERP, the linkage was strongly made
through a commingling of PIU management during the final several years of both projects.
It is clear that for a variety of reasons a productive link between the capacity building
UNDP effort and the investment oriented World Bank efforts was never forged. The
rationale for linkage was strong.

15

---

3.2.1 Project management
Management of GEF IW projects is often considered an issue primarily for the PIU staff. In
fact, the PIU is only one of the key responsible actors. Others include management
representatives at the UNDP, GEF and UNOPS and in particular the steering committee
members and focal points from the participating countries
During Phase I, the BSERP encountered problems with the functioning of the PIU, and
there was significant staff turnover, including the replacement of two Chief Technical
Advisors (CTAs). In 2004 UNDP and UNOPS made the decision to link the Danube and
Black Sea projects through appointing a Regional Manager (the DRP CTA). The idea was
for the strong deputy CTAs for each project to take on greater day to day responsibilities
and the Regional Manager to play a coordinating and policy level role. Under the
circumstances this decision made good sense. The hybrid approach provided a much
closer linkage between the two connected projects. It also enabled the BSERP to move
forward without a protracted search and negotiation for hiring a new CTA. The approach
was aided by the policy and managerial strengths of the selected regional Manager, as
well as the capabilities of other PIU management and experts.
Interviews during the evaluation pointed out that many participants would have preferred
having a full time CTA selected, who could have put more time and energy directly
towards issues in the Black Sea. Stakeholders understood and accepted the reasons for
the regional manager approach, yet still felt that the Black Sea effort deserved a full time
high level manager. This concern was further heightened by the fact that the Regional
Manager took a new GEF assignment prior to completion of the BSERP.
During the BSERP first phase, a decision was reached with the project steering committee
to employ country team leaders (CTLs) in each of the Black Sea countries. These persons
were tasked as full time project staff, responsible for coordinating BSERP activities in their
country, in addition to wider issues responsibility (i.e. ICZM and Pollution Monitoring and
Assessment). Under the circumstances, with the BSERP getting only limited support from
several of the participating countries, a decision to hire CTLs was made by the Steering
Committee, based from the urging of the CTA at the time. As noted in the lessons learned
section of this evaluation (section 3.4 #4), the decision to hire CTLs carries risks that were
borne out by the BSERP experience, for example:
The CTL effort constituted a major cost factor for BSERP, and freed the responsible
agencies of their agreed co-funding commitment. The cost of hiring and managing
these six full time CTAs reduced available funds for demonstration projects and other
technical assistance in the midst of a project that was already under-funded based on
expected outputs.
CTLs were selected by the responsible agencies in the governments, based on project-
developed terms of reference. The technical capabilities of the CTLS were not
necessarily the key determinant on who got selected.
The arrangement of having the responsible agencies in the countries select the CTLs
while the project paid their salaries created some difficulties in terms of loyalty and
,,chain of command issues. For instance if a CTL was found to be ill-suited to the
assignment, it was difficult for the PIU to negotiate a replacement.
3.2.2 Quality of inputs and activities
Staff inputs were generally of good quality. The PIU experts (including the ones deployed
through UNEP/GPA) were qualified for their tasks. Their co-operation with the major
project counterparts, stakeholders and other experts overall has been appreciated. The
project employed a wide range of regional experts, whose inputs were indispensable for
many activities and outputs. The performance varied among individuals, but was generally
satisfactory.
The Project Team developed annual work plans which were reviewed and approved by the
Project Steering Committee. As exhibited by the work plan for 2007, the plans explained
expected achievements on each output and activity. The Work Plans did not go into detail

16

---

on the sequencing of sub-activities and did not specifically indicate which project staff
person would be responsible.
The time frame for project task completions experienced some drift. The TDA revision
effort commenced during Phase 2, and the TDA was finalized in May, 2007, six months
later than planned. Phase 1 inputs and problems with the quality of data received from
the countries through the BSC meant considerable time was required in the early months
of Phase 2 to get the TDA effort moving. Delays in receiving data and comments from
several Advisory groups also held up the TDA development process. The completion of a
revised strategic action plan (SAP) slipped, and the process is still underway as the BSERP
PIU closes. Extension contracts for consulting assistance to the BSC through May ­ June
2008 have succeeded to revise and complete a document that (informally) is now agreed
to by five of the six contracting parties. Several other expected outputs were also issued
only at the end of the project, including training for and transfer of the Black Sea
Information System (BSIS).
3.2.3 Budgeting
The overall budget for the project was appropriate, recognising the geographic size of the
Black Sea region, the capacities of the countries involved, the significant pollution
pressures addressed, the presence of the two additional support projects within the GEF
Strategic Partnership, and a parallel support framework through the European Commission
(Tacis). Whether the budget was sufficient opens up a wider discussion of objectives and
expectations. Developing draft policies and plans, and training ministry staff, do not
guarantee a reduction in pollution. It is only if and when these draft policies are
implemented by the countries and lead to increased budgets and greater compliance, that
the adequacy of the BSERP budget can be measured.
Budgeting for the project was made difficult by the deflating value of the dollar against the
Euro and currencies in the Black Sea region, creating a nominal 20% reduction in available
financing. The BSERP was also hit with unforeseen UNOPS headquarters charges (in this
case for ASHI ­ After Service Health Insurance, for long term UN employees). These
charges were assessed after project approval and on top of the expected UNDP
administrative fee. Future projects, should be configured to include this and any other
administrative costs within the budgeted administrative fee.
The PIU had some difficulties determining their budget during the final project year.
Apparently, there was some minor overlapping within UNOPS for the budgeting of the
Danube and Black Sea projects, largely due to the joint assignment of the Regional
Manager and how his costs would be apportioned. However this should not have been a
difficult line item to track. Most projects keep a simple spread sheet "shadow" budget that
allows them to gauge their funding levels on a day to day basis, which then is matched
and revised periodically against the more cumbersome ATLAS budget.
The budgets available for activities such as the reduction of land based sources of
chemical pollution, the analysis of air-born nitrogen deposition, improved municipal
services budgeting, and priority investments to deal with coastal pollution hot spots were
not at levels where major impacts and scientific breakthroughs could be reasonably
expected. Funding was sufficient only for small scale studies. Future projects need to
hold to a tighter, more rigorous budgeting process. There needs to be enough funding
and effort placed behind fewer activities, to ensure that they build into an effort that
outlives the project. One-off studies that end in a report sitting on the shelf should be
avoided. The PIU and Steering Committee made some progress on narrowing the focus
during the 2nd phase, yet they were of chained to the over-ambitious objectives of the
original concept and expansive outputs obligated through the Phase 1 & 2 project
documents.
The project did not budget sufficiently for translation. Oddly, with 6 countries included,
speaking 6 different languages, there was very little done to translate reports and internet
pages so they could be understood in each of the countries. Especially in projects that
seek to build public awareness and support, there must be consideration given to
translating all key documents, at the very least their summaries, into each of the national

17

---

languages. Otherwise, outputs are available only to a limited audience. An alternative to
a large translation budget is to include upfront commitments from each of the countries to
take responsibility for translation of key project documents.
The budgeting and pay out procedures used by the PIU were appropriate and effective.
Participants were satisfied with budgeting procedures for conferences and meetings. Also
the budgeting for the research cruises was generally handled well, notwithstanding one
occasion where the deputy project manager used his own money to get one of the
research cruises underway.
3.2.4 Major factors facilitating/impeding project implementation
The BSERP built upon existing structures established by the Black Sea countries. This
included in particular the BSC and Secretariat, as well as advisory groups established
during the previous GEF project (BSEP).
The strategic partnership provided an opportunity to extend cooperation across the
entire Danube/Black Sea basin, and to draw important connections between upstream
pollution in the Danube and its impact ­ through eutrophication - in the Black Sea.
The membership of Bulgaria and Romania in the European Union helped to further
cement the participation and support of the EU for Black Sea ecosystem protection.
Economic changes in the region can be seen both as facilitating and impeding the Black
Sea effort. On the one hand, the downturn in the regional farm economy had a
beneficial effect in reducing nutrient loading and enabling some recovery of Black Sea
ecology. On the other hand, economic upheaval had a seriously negative effect on the
capacity of marine research institutes in many of the countries.
Political upheaval in the region was an impediment to the BSERP. Rapidly changing
political fortunes amongst leading political parties in some of the Black Sea Coastal
Countries has led to frequent changes in environmental and natural resource ministry
officials. In some cases, new governments have also made policy shifts that revise the
levels of responsibility for development and environmental protection at the local,
regional and central government. This evolving situation with respect to institutional
responsibility has created a perpetual sense of impermanence and reluctance to take
decisions on environmental issues, including long term actions to reduce environmental
degradation in the Black Sea. There have been numerous and frequent changes
amongst project counterparts and stakeholders in the participating countries. Advisory
group members changed frequently throughout the project period, making it difficult to
build and retain continuity. The delay in commencing the actual national negotiations
one the draft LBA Protocol partially can be explained by changes in the people
involved.
The production of the updated TDA was seriously hampered by the limited amount of
available data that were not always of a proper quality and sometimes hard to be
retrieved. The combination of factors added to a delay in the compilation of the TDA.
Lack of data and information also hampered other tasks and activities.
The PIU handled well the management of the Country Team Leaders. There was
significant effort made to develop and revise their terms of reference during the early
stages of employment, and the PIU managers were wise to include in their remit not
only country-based activities but also cross-sectoral responsibilities.
The project tranche 2 ProDoc was approved prior to conducting a mid term evaluation
of the project. This was an error in process and necessitated another project revision
once the MTE recommendations came out.
As noted in the mid term evaluation, closer oversight by UNOPS and UNDP could have
helped to identify and rectify problems in project management at an earlier stage
during the project first phase.

18

---

3.2.5 Adaptive management
Adaptive management is a general term meant to have positive connotations. The hope
and expectations are that project managers will recognise that shifting circumstances call
for some flexibility in the implementation of project plans. Social and economic upheavals
may change the basis for action, the Project Document can be seen to have flaws, there
may be changes in government, there may be changes in the project team, there may be
new pressures on the environment, for instance due to severe weather. The key to
adaptive management is to ensure that changes based on circumstance continue to lead
the effort toward achieving its stated objectives.
The BSERP faced quite a few changes in circumstance, such as changes in the economic
structure of most of the Black Sea countries, changes in ministry personnel and
responsibilities, changes in project team composition and reconsidered project outcomes
and activities. The BSERP can point to successful adaptations, for instance in the decision
to couple the BSC Commissioner and BSERP Steering Committee Member roles during the
project second phase and organising back-to-back Commission meetings and Project
Steering Committee meetings. This change helped forge closer cooperation between the
project and Commission, and reduced steering meeting time and cost. The decision to hire
Country Team leaders was also a successful adaptation. Adaptive management is also
evident in the decision to downgrade certain activities where there was not enough
financial weight to have much impact for instance concerning air-borne dispersion of
nutrients.
3.2.6 BSC Management and execution
The BSERP ­ BSC relationship suffered from a large gap in expectations concerning the
nature of the BSERP role and the extent to which it should act as a subsidiary body to the
Commission. Was the PIUs number one priority to achieve its terms of reference? Or was
it to assist the BSC Secretariat to carry out its mission? Should the GEFs funds have been
made available in response to Secretariat demands? Or rather should they have been tied
only to activities expressly set forth in the Project Document? This issue strikes at the
core of how GEF IW projects should be organised when there is an international
commission in place that the project seeks to support. Future projects need to spell out
clearly ­ both in the project Document and in a memorandum of agreement, what are the
parameters of the relationship.
The institutional set up of the BSC and its subsidiary bodies has been evolving. The BSC
structures set in place by the Bucharest Convention included Regional Activity Centres
(RACs) and Advisory Groups (AGs). The AGs provide expertise, information and support to
the Commission for implementation of the BS SAP, and were established pursuant to
approval of the BS SAP. Activity Centres are authorized in Resolution 4 of the Bucharest
Convention, indicating that ,,certain activities concerning technical matters such as
organisation of training courses, formulation of pollution control guidelines and joint
Intercalibration and inter-comparison exercises, etc shall be carried out by the research
institutes of the Contracting Parties as activity centres" (pg 28). The RACs were politically
identified and most were not the premier regional or national institutes in the disciplines
they were chosen for. At this point, only two continue to receive financial resources from
their countries to carry out minor duties on behalf of the BSC. The RACs were an ill-
conceived concept that has done little in terms of providing expertise to the BSC as
national in-kind contributions.
The BSERP during its second phase put significant time and effort towards strengthening
the BSC and its Secretariat, through financial support for meetings, etc, and in particular
through developing an institutional review and exit strategy, both designed to ensure
management effectiveness and sustainability of the BSC after the conclusion of the BSERP.
Despite this effort, at the conclusion of the BSERP, the future viability of the BSC and its
PS are at risk. Funding provided by the countries is likely to be insufficient for the
Secretariat to carry out its expected assignments. Future advisory group meeting are in
doubt due to a lack of funding, as are future efforts to promote the Black Sea Day.


19

---

The future sustainability of the Black Sea monitoring program is more optimistic. National
sections of the Black Sea Integrated Monitoring and Assessment Programme (BSIMAP) are
operated and paid for at the national level by the countries. BSIMAP includes fractions of
the national monitoring systems, which countries report on. However, for those stations,
which are included in BSIMAP, there are additional requirements from a methodological
and analytical point of view. These systems can be expected to continue, and in the case
of some countries (Turkey and Romania), expanded geographically and in terms of media
monitored.
The BSC Secretariat has been established along similar lines as other International Waters
Secretariats. The structure is workable however staffing is insufficient. The budget for the
BSC is insufficient if the Commission and Secretariat are expected to lead regional
cooperation efforts on such aspects as water quality monitoring, fisheries management,
and reduction of land based sources of pollution. The current agenda for the Secretariat
includes a long list of activities that would be challenging to achieve even with triple the
existing staff and budget.
The Exit Strategy provides a recommendation to increase annual financing commensurate
with the amount that has been provided during the previous three years with BSERP
support, indicated as between 1.5 million and 1.7 million, (we assume this is dollars
although the Exit Strategy does not clarify). This level of increased funding that the
authors suggest to achieve by 2010 is unrealistic given the low level of country ownership
and the difficulties faced by the Commission to build a regular budget. During their
February 2008 Commission meeting, the Commissioners agreed to increase funding to
$60,000 USD per year, although a final agreement on this increase from $48,000 had to
be deferred to enable the Georgia Commissioner to consult with officials back home. The
Commissioners were adamant to keep the funding levels equal, so this means a $300,000
budget. Turkey indicated a willingness to provide additional funding, of over $100,000,
which will be needed for the pending move of the Commission offices. Even with this
additional contribution, the budget for the Commission will not exceed $500,000 per year;
indicating a shortfall of more than 1 million USD annually.
3.2.7 Danube /Black Sea Strategic Partnership
There were explicit linkages between the BSERP and DRP and the World Bank Investment
Fund for Nutrient Reduction (NRF), in the frame of the GEF ­ World Bank Danube/Black
Sea Partnership Program. The DRP and BSERP are similar in structure and content ­ as
they focus on regional TDA/SAP development and capacity building. The NRF is a $75
million investment fund for projects to reduce nutrient loading.
BSERP/DRP cooperation was formalised in the BSERP ProDoc, supporting implementation
of the MOU signed between the Danube and Black Sea Commissions. In particular, a Joint
Technical Working Group (JT WG) was established for implementation of the MOU and a
work programme was devised. In all, there were four annual JT WG meetings organised
from 2002 onwards.
Close collaboration was not seen as a high priority by the project teams during their
formative years. During 2004, a Strategic Partnership Stock-taking meeting was held to
include 80 high-level country representatives of the ICPDR, BSC, GEF, UNDP and other
experts. Subsequently, a closer association was forged, especially between the DRP and
BSERP.
The DRP and BSERP projects became closely aligned at the end of 2004, when a decision
was reached by UNDP/UNOPS to have the DRP CTA take on responsibility for both
projects. This decision was precipitated by management issues at the BSERP. The decision
was aided by a conviction that the DRP management team was sufficiently strong, and the
project moving smoothly enough, to enable a sharing of the CTAs time.
Cooperation between the BSERP and NRF remained infrequent throughout project
implementation. It was originally hoped that the TDA/SAP procedures and then
subsequent monitoring and capacity building efforts under the DRP and BSERP could help
to define project priorities and pipelines for investments under the NRF. In practice, the

20

---

timing of the NRF programme and WB requirements for investments proved to be
impediments to this ideal relationship. In the end, the priority setting and then
implementation of capacity building and investment projects proceeded independently,
although there has been a concerted effort to share information, with annual meetings
held between the Strategic Partnership members during the three years 2005 ­ 2007.
Ideally, the BSERP would have identified major pollution concerns, including nutrient ,,hot
spots, that then became the focus for investments under the NRF. At the end of the
BSERP there is no evidence that this type of linkage occurred. The NRF team carried out
their investment project ID efforts without consideration for the findings coming out of the
BSERP. The parallel timing of assignments carried out by two separate UN agency
projects with different criteria (World Bank ,,bankable project considerations for example)
are indicated as reasons for the difficulties in successfully building a coordinated sequence
of fact finding, capacity building and then investments.
3.3 Project impacts
The BSERP has succeeded in making progress in expanding knowledge, awareness and
support for ecosystems protection in the Black Sea region. The Project has had a terms
successful impact by expanding understanding of the status of the Black Sea ecosystem
and identifying compelling early signs of ecosystem recovery. Impacts are also to be
considered with respect to the growing cadre of concerned and involved citizens ­ as seen
through NGO support for the successful small grants effort and growing interest in the
Black Sea Day. While there has been an increase in funding for environmental controls on
sites listed as hot spots during the project period, this support cannot be attributed to the
BSERP effort.
In terms of measures to be taken by the states to reduce nutrients and other hazardous
substance loading into the Black Sea, the project has been able to map out some options,
supported by a causal chain analysis. The identified options are still rather generic and
consequently in most cases not yet costed out. Acknowledging that details are to be
elaborated by the countries themselves, the project mostly rendered generic guidance and
tools for doing so. For several issues, like hazardous substances, there has been
insufficient data shared by the governments and industries to allow for an external
assessment and options development. Most of the Black Sea countries still face some
problems in their capacity for analysing hazardous and other substances (despite the
national and international efforts for increasing these capacities).
From the standpoint of environmental improvement, the impacts of the BSERP are at this
point difficult to discern, although it must be acknowledged that the BSERP project had
neither the expectations nor the financial resources for measures directly reducing
pollution loading from municipal and agriculture sources.
The impacts that were expected, which remain unfulfilled are largely focused towards
policy changes at the regional and national levels, and therefore largely in the hands of
the BSC contracting parties to achieve. These include:
the revised SAP is not yet signed (although five countries have informally indicated
their approval);
the LBA Protocol is still under revision (with all but one contracting party ready to
agree);
A coastal zone strategy for the region has been developed but coastal zone plans have
not been implemented;
no areas have been set aside as no fishing zones or marine protected areas (although
Bulgaria and Romania have begun to negotiating a joint MPA);
no changes have been made to fishing quotas or restrictions in the catch of any specific
species;
no particular industrial projects have been altered or closed down as a result of project
efforts;

21

---

no coastal wetlands or other fragile ecosystems have been restored or protected;
no revisions to agricultural policy have been instituted to reduce non-point source
runoff;
no agreements have been reached regarding specific regional measures to reduce
marine based invasive species.
3.3.1 Achievements of the project against objectives, outputs and activities
Objective 1. Support the consolidation and operation of institutional mechanisms
for cooperation under the Black Sea Convention
Outputs / Indicators
Results/ Impacts
1.1: Operational
Achievement under this output can be considered satisfactory, in
structures and
particular considering the strong efforts made during the BSERP 2nd
management tools of
Phase to increase its assistance to the BSC secretariat, including through
the Black Sea
the Institutional Strengthening Review and the Exit Strategy. What is not
Commission further
clear yet is the extent to which the Commission will agree to act on the
developed and
good recommendations developed.
functioning.
Operational structures for the BSC have improved to the extent that the
Advisory Groups are acting with significantly more independent initiative
than was the case early in the project.
Management tools have been developed with BSERP assistance and
turned over to the Secretariat ­ in particular the BSIS and BSIMAP.
Suggestions and recommendations for the structure of these
management tools came out of the capacity building workshops and
inter-calibration exercises, and were agreed upon or developed together
with current and former BSC Secretariat members. It is not clear whether
the Secretariat will have the means to fully utilise these tools.
Objective 2: Development of policy guidelines, legal and institutional instruments
of pollution reduction from LBA and protection of ecosystems of the Black Sea and
its coastal zone
Outputs / Indicators
Results/ Impacts
2.1: Update/Renew
Transboundary Diagnostic Analysis
the Transboundary
Diagnostic Analysis
The TDA has been both updated and renewed, reflecting the recent GEF
(TDA) and Strategic
,,best practises. The TDA has been published as a report as well as in a
Action Programme
web-based version, accessible via the website of the BSC. Achievement for
(Black Sea Strategic
this output can be considered satisfactory.
Action Plan, BSSAP)
The TDA though has been issued rather late. A preliminary first draft was
to account for the
available in autumn 2006; the final draft was issued in January 2007. A
changes in the
combination of factors is responsible for the delays, including the time
pressures on and the needed by the PIU for planning, for identification of local experts and for
state of the Black
issuing subcontracts; the time needed for acquiring the data (there were 66
Sea ecosystem.
contributing specialists); data being provided in various formats (despite
prescribed templates) and with a varying quality (requiring extra checks).

The 1st phase BSERP activities, while not specifically focused on TDA/SAP
development, nevertheless were supposed to provide deliverables that
should have greatly aided in the TDA revision process, including a detailed
study on emergent issues in the Black Sea region and their root causes
(Phase 1 LFM 2.2); national and regional commitments by the countries and
regionally to develop technical administrative and legal measures to control
land based sources and other ,,emergent problems (2.3); and the
development and adoption of process, stress reduction and environmental
status indicators (4.2). At the start of the TDA effort, PIU staff viewed
these and other 1st phase deliverables to be inadequate for the TDA revision
process, so a great deal of time and effort first went into acquiring
improved data. The updated TDA differs from the 1996 TDA especially in

22

---

that information and analysis is now based on collected national, regional
and external sources of data with clear policy outcomes, whereas the
previous version was entirely based on expert judgments and indirect
observations.
Although more and better data and information are now included, there are
still several important gaps. For instance:
There are uncertainties considering the source apportionment for
nutrients, notably nitrogen. A tentative estimate indicates that the
nitrogen loads carried with atmospheric deposition might be of similar
order of magnitude as the nitrogen loads carried with the Danube
River.
There is limited information about trace metals and organic micro
pollutants (including pesticides) in the Black Sea itself (water,
sediment, and biota) as well as in the various pollution sources (like
direct point sources, rivers and diffuse sources). The latter is mainly
due to the fact that micro pollutants are still not routinely monitored
by most countries; most information has been collected with the
BSERP surveys (see also output 4.1 further below).
Regional fish stock data is missing entirely, due to no agreements
yet reached on a regional assessment methodology, and the data
gathering to support it.
The programmes of measures are rather generic and without cost
estimates for the Black Sea Countries.
Generally, the TDA contains more details for the north-western and
southern parts of the basin, compared to the eastern sections
(notably GE and RU). Most gaps merely reflect the still existing lack
of sufficient data and information for the Black Sea Basin; and in the
case of some countries, Russia in particular, a decision not to supply
requested data.
Hot Spots
The BSERP Final Technical report indicates (pg 55) that of the 50 capital
investment projects identified in the 1996 TDA, only 12 were completed 11
years later, with 2 no longer required, and work in progress on another 10
sites. The revised TDA, section 5, provides a hot spot analysis which goes
into detail on these findings, noting that there are no expectations for
government actions involvement 26 of the original hot spots. It would have
been useful in the TDA to review and revise the hot spot designations,
taking into consideration the actual monitoring/effluent data for the 50
identified hot spots (the 1996 SAP merely used generic denominators) and
to consider areas with significant diffuse sources.
The hot spot analysis in the revised TDA provides no indication of additional
sites that should be added. The PIU recognised there were serious flaws in
the methodology used for the original hot spot analysis setting up an
uneven ,,playing field amongst the 6 countries and ensuring that national
issues were prioritised over transboundary issues. The PIU staff proposed
an alternative methodology to the LBSA AG, however the AG was unable to
reach consensus to agree or even comment on the proposed methodology.
The Project Steering Committee then decided not to update the list of hot
spots, and instead included just an assessment of the work undertaken to
review the existing list. Agriculture has been added as a ,,primary suspect
but without real quantification or prioritisation (e.g. whether livestock
should be prioritised over the use of artificial fertilisers, for example).
Some of the TDA conclusions were considerably weakened through the
process of revising the document based on national comments. While the
review process increased country buy-in, in some cases it did so at the
expense of providing a clear and accurate picture of the pressures on the
Black Sea ecosystem. At one stage the TDA made it very clear that the
Istanbul population constituted a greater load than all of the direct point
source loads to the Sea combined, but the BSERP Steering Committee
required that this strong statement be removed before accepting the

23

---

document.
Opinions about the updated TDA vary, with some key stakeholders more
satisfied than others. The project seems to have produced the ,,best
available TDA, taking into account the time and resources available, in
combination with the available data and their accessibility. Despite the
contributions by local experts, including participation in Technical Task
Teams for each transboundary issue, some interviewees consider the TDA
to be more of a BSERP product
Strategic Action Plan
Achievement for this output can be considered satisfactory.
The SAP revision effort was carried out in the project 2nd phase,
commencing after the TDA was completed in January 2007. The
development process utilised a SAP Drafting Team including participants
from each of the Black Sea countries.
The 2007 SAP revision was accompanied by the 1996 SAP implementation
analysis conducted by the consultant hired by the Secretariat on behalf of
the European Commission. The principle innovation of the revised SAP is
the introduction of the concept of Ecosystem Quality Objectives for the
Black Sea with a series of accompanying phased, step-by-step
short/medium and long term targets. The reports (regional and national
ones) were used at the process of drafting the new SAP and helped to
confirm the deficiencies in implementation of the old SAP and the
usefulness of the new methodology.
The SAP was being significantly revised and expanded during the evaluation
mission. Given that it constitutes a continuing work in progress, it is
difficult to render a full assessment on the SAP.
The technical draft version of the SAP reviewed by the evaluation team
contains a ,,wish list of all together 107 short-, mid- and long-term targets,
(later drafting based on country comments is apparently revising this
number of targets downward). A high number of targets carry a potential
risk of lacking focus. The realisation of 57 short term (1-5 years) targets
may be considered as rather ambitious (the 1996 SAP has been typified as
overly ambitious with very few of the targets being accomplished on time).
The format of the revised SAP differs from the original one issued in 1996.
While the format reflects the latest GEF requirements, it does not meet one
of the partner country expectations. Because of their inclusion of the first
SAP in the legislation adopted in 2001, Ukraine has indicated it would prefer
a comparable/compatible structure.
The technical draft included an Annex 5: Details of Agreed Measures at the
National and Regional Levels (Including National Policy/Legal/Institutional
Reforms and Investments) and their Implementation Mechanisms, which
had not yet been developed. This section requires the input of the Black Sea
Commission and the participating countries during the political process of
SAP formulation.
There are different opinions about the importance of having an updated
SAP. Some interviewees felt that many things still could be done with the
SAP of 1996. One interviewee mentioned to have been more interested in
digging deeper into the explanations why many of the original SAP
obligations were not implemented (on time), instead of preparing an
updated SAP.
It is not yet clear whether the updated Black Sea Strategic Action Plan will
be ready for signing during the Ministerial Meeting scheduled for 31 October
2008. It has been reported that the continuing BSC/BSERP revision process,
carried out while the final evaluation was in progress, has led to circulation
of a draft revised SAP with initial favourable response from five of the six
contracting parties. Significant revisions and amendments to the SAP draft
have been submitted by the Russian Federation.


24

---

2.2: Adoption of the
Achievements under BSERP can be considered satisfactory. While the
Protocol for Land-
Protocol is still not approved and moving through the process of ratification,
based Activities
this is essentially a problem of BSC and country decision making. It is not
(LBA) (concluded and an issue of proper Protocol preparation but rather the unwillingness of some
adopted as a draft in
countries to move forward.
Phase I) is facilitated
through the national
The BSC at its 11th Meeting (November 2004) decided to approve the
and regional
proposed Draft Revised LBA Protocol, including its annexes, submit it for
negotiation process
national consultations and possible adoption at the Ministerial Meeting to be
in order to ensure
held in 2007 (and postponed until march 2008).
the adoption of the
More elaboration/consultation was deemed necessary regarding the
revised text at the
annexes to the Protocol, so this was included in the 2nd phase of the BSERP.
Ministerial
In August 2006, UNEP/GPA1 issued a 3rd version of the Draft Revised LBA
Conference (2007).
Protocol, which included the comments that were received and discussed in
the meantime. UNEP/GPA presented this version as the official document
that the Black Sea countries may wish to use for the internal procedures for
approval and the official diplomatic negotiations scheduled for 2007. In
August 2006 (Russia) and in the period January ­ June 2007 (other BS
countries), National Workshops were organised in support of capacity-
building of national and local authorities responsible for the implementation
of the LBA Protocol. Comments on the Draft Revised LBA Protocol were
contained as an annex to the reports that were prepared by UNEP/GPA on
the National Workshops. During the 18th Extraordinary Commissioners
Meeting in February 2008, it was agreed to organise an expert meeting,
tentatively in May 2008, to finalise the draft and conclude the preparatory
work on the Draft Revised LBA Protocol.
A leaflet "land-based sources of pollution in the Black Sea. Protecting our
sea" has been published in all six BS languages and in English.
As can be derived from the above summary of events, there has been a
delay of at least more than one (August 2006) up to almost three years
(November 2004) in terms of initiating the internal country procedures for
approval of the Draft Revised LBA. The unfinished annexes of the 2004
Draft Revised LBA Protocol actually turned out to have opened the door for
further discussion about the core text of the protocol as well. This situation
has been mainly due to neither the BSC nor the PS having had a proper
understanding about the procedures that commonly apply according to
international practises; the year 2006 furthermore coincided with the staff
changes in the PS.
Because of their advisory role, the BSERP project, including UNEP/GPA, are
no longer active parties in the process (Inter-Agency Agreement (IAA)
between UNEP/GPA and UNOPS. From a facilitation point of view, the
project has performed well, including the continuation of the support by
processing comments that were received throughout the years and the
support for meeting scheduled for May 2008.
2.3: Strengthen
A series of 6 activities were envisioned in the ICZM output, including
Integrated Coastal
carrying out an inventory of ICZM legislation and policies, developing a
Zone Management in
regional strategy as part of the revised BSSAP, developing a feasibility
line with EU
study on a new ICZM Protocol to the Bucharest convention, implementing a
Directives and in
pilot project testing ICZM concepts and disseminating the results of the
testing concept for
pilot. There was also a specific activity included to assist in the negotiation
Best Practices for
process to establish a transboundary marine protected area for Vama-
ICZM as developed
Veche (Bulgaria ­ Romania border).
by BSC/TACIS, to
assure reduction of
The output has been satisfactorily achieved in terms of BSERP PIU
nutrients and
deliverables. The BSERP has provided appropriate tools to the BSC for their
hazardous
continuing efforts to develop a regionally consistent approach to coastal
substances from
zone management.
coastal areas into the Implementation at the national level remains a work in progress. Two of
Black Sea.
the Black Sea Coastal States, Romania and Bulgaria, have national laws and

1 An Inter-Agency Agreement was signed between the UNEP/GPA Coordination Office and UNOPS in August 2003 to undertake the
activities under component 2 related to the LBA protocol.

25

---

management instruments in place specifically on ICZM. Two other states,
Georgia and Ukraine have draft ICZM laws in hand, at early stages in the
parliamentary review process. Turkey has a Coastal Law, but no integrated
legislation covering multiple aspects of coastal zone management. The
verifiable indicators for 2.3 anticipated that all six states would be
developing ICZM strategies and legislation, with 3 countries having adopted
and started to implement ICZM policies.
The ICZM pilot carried out in Turkey offers useful lessons in the
development of ICZM. The pilot was carried out during a very compressed
11 month period in 2007. Starting the effort earlier would have enabled a
longer development period as well as a chance during the project to assist
in implementation and work towards wider replication in other areas. Even
with the short duration, prospects for replication look promising. During the
evaluation period the ICZM pilot team met with Turkish environmental
ministry officials to review outcomes and push for replication elsewhere.
The demonstration project team noted there were already ICZM guidelines
developed under the Mediterranean Action Plan (MAP) and utilised on the
Turkish Mediterranean coastline. The MAP experience should have been
reviewed as a guide and lessons learned for the Black Sea effort.
Notably, all six countries indicated their support for a regional ICZM
protocol, and expectation that the BS ICZM Plan would be incorporated into
the overall BSSAP. The BSERP Feasibility Study on an ICZM Protocol takes
this support into account, but stands it up against the difficulties faced by
the BSC with its other protocols: the Biodiversity Protocol (ratified so far by
2 of the 6 states) and the draft Land Based Sources Protocol, now in the
midst of a protracted 4 year review and revision process. There has also
been failure to move forward on a fisheries protocol on other binding
instrument. The Feasibility Study suggests rather putting forward ,,soft law
instruments: ICZM Declaration, Code of Practice and Action Plan.
The Vama-Veche cross-border marine reserve effort continues its slow
development. A report developed in November 2007 indicates that the
Romanian led-effort now includes joint efforts between Bulgaria and
Romania to map the reserve. On the Romanian side, the Romanian
National Institute for Marine Research and Development (INCDMN) is
custodian of the marine reserve development effort ­ since 2004. An
information centre has been constructed and a Junior Ranger Club has been
created. In 2006 a Management Plan and Regulations for the Marine
Reserve were submitted to the Ministry of Environment and Sustainable
Development for approval.
2.4: Agricultural
Achievements under this output were marginally satisfactory. The
policies reviewed for
reports published on the PIU DVD under the Agriculture component mainly
application at
comprise (country) reports on Livestock Numbers and Potential
national level to limit
Nutrient/Organic Loads to the Black Sea from Riparian Countries. None of
export of nutrients
the reports deal with application of inorganic fertilisers or with hazardous
and hazardous
substances originating from agriculture (notably plant protection agents).
substances.
Neither do the reports contain a review of agricultural policies (also options
for reducing nutrient emissions from livestock breeding are not addressed).
The "Case study on calculating cost-effective measures to tackle nutrient
pollution from the agricultural, municipal and industrial sectors in the Black
Sea" published under Objective 3.1 (see further below) contains rather
generic information that is not really tailored to the Black Sea Basin
countries.
The LogFrame mentions in terms verifiable Indicators / Results: "National
experts are trained to introduce Agricultural BAP in their countries.
Representatives from relevant ministries, municipalities and local
Governments are trained in the development and implementation of
sectoral policies and NAPs". These indicators were not realised, as training
on agriculture BAPs were not carried out.
It is primarily a shortcoming in the project design that a project focused on
nutrient reduction focuses so little attention to the most significant land-
based source of nitrogen loading ­ agriculture.

26

---

2.5:
The industrial/transport and municipal component included 4 sets of
Industrial/transport
activities including the establishment of inventories of industrial, transport
and municipal
and municipal pollution sources, developing criteria and then revising hot
policies and
spot identification, reviewing policies and legislation, and organising
legislation reviewed
workshops to introduce BAT and financial support mechanisms.
for application of BAT
(best available
Achievements under this output are deemed to be marginally
techniques, including
satisfactory. The Project team was limited in its achievement in particular
cleaner technologies) by the difficulty in getting reliable data from the countries. Initial
towards reduction of
expectations were that the BSC Secretariat would provide useful data, yet
nutrients (N and P)
data quality checking was a problem and some data was purposefully
and hazardous
withheld. Secretariat staff and some of the responsible country ministries
substances.
insisted that their industrial point sources discharge data was confidential
and could not be viewed or utilised by the project Team. As clearly stated
by the BSERP PIU, the information provided through this exercise, and
included in the BSIS, is deeply flawed and serves mostly to over-report the
extent of pollution loading from the Danube.
The updated TDA contains an assessment of progress made in addressing
the original list of 50 ,,hot spots identified in the first TDA of 1996 in terms
of undertaking the capital investments originally identified. The "Case study
on calculating cost-effective measures to tackle nutrient pollution from the
agricultural, municipal and industrial sectors in the Black Sea" published
under Objective 3.1 (see further below) contains a chapter "Case study on
nutrient reduction in industry" that seems to have been written more for the
Danube Basin rather than the Black Sea basin.
Data were provided from each of the countries on industrial and municipal
dischargers above 1000 m3 daily discharge. This data suggests that nutrient
loading from industrial and municipal sources directly discharging into the
Black Sea are equivalent to only 2% of river borne DIN and 13% of river-
borne PO4 - Phosphorus loading into the Sea. As noted by the BSERP in its
report, these loading figures seriously under report possible other coastal
pollution sources - for instance data on the discharge amounts from
Istanbul into the Bosporus are not included. The BSERP in its final report
notes that much of the data were weak and full of omissions, noting that
there remain problems in the country monitoring and analytical quality
assurance programmes. Improved QA/QC for load estimation were
recommended. It remains clear that the Black Sea countries are quite
reluctant to report and share data concerning their industrial and municipal
discharges directly into the Black Sea and most of its tributaries.
No data were compiled concerning pollution loadings from the transport
sector

27

---

2.6: Legally binding
The Fisheries component included 5 separate activities: assisting the BSC
Document on
on a legally binding document (LBD) for fisheries protection, carrying out a
Fisheries finalised
study on sensitive habitats and nursing grounds ­ with recommendations on
and proposals for
fisheries ­ free zones and MPAs, annexes to the Bucharest Convention
establishment of
,,Biodiversity Protocol, elaborate stock assessment approaches on migratory
fisheries-free zones
fish, and to organise a regional workshop on regional stock assessments.
and marine protected
areas developed
A working draft for the LBD was developed. Its formulation involved
consultations with representatives of Ministries of Foreign Affaires from all
countries at a special meeting in Sile (Turkey, 2003). The draft LBD on
Fisheries was then approved at a regular meeting of the BSC (2004). Since
then, the effort to develop a legally binding document (LBD) on fisheries
has not moved forward. One of the critical issues in the last several years
has been that with Romania and Bulgaria now EU members, it is the
European Union that must negotiate fisheries treaties on behalf of the
member states. The EU is not a party to the Bucharest Convention (despite
its expressed interest to join), so the Commission is not in a position to
serve as a negotiating forum for fisheries, until such time as the EU
becomes party to the convention.
The BSERP together with BSC Secretariat collected maps from each of the
countries, together with supplementary reports, setting out nursery and
spawning grounds. This compiled information is planned to be added to
Annex IV of the Biodiversity Protocol.
The TDA includes a generally well-developed section on fisheries, including
a causal chain analysis on the decline of commercial fish species / stocks.
It is worth noting that there are serious omissions in the data ­ for Russia
provided no data on seafood production consumption and employment,
while annual catch of various species typically excluded some countries
from the data (see TDA pg 72). Evidence reported in the TDA and from the
research cruises in the western shelf hint at a recovery in marine
ecosystems stemming from a reduction in nutrient loadings after the post-
Soviet economic upheaval. The trend data for fisheries indicates an
increase in sprat and anchovy catch, although other key commercial species
such as Whiting, horse mackerel and mullet remain depressed.
For the TDA, the BSERP utilised information from fishing fleet statistics and
fish landings. The BSERP team has made clear that the information
received from the countries on fisheries is deeply flawed. There is no
common methodology used to report landings, weight of catches goes
underreported, and no consideration is given to the extent of illegal fishing,
(viewed as a likely considerable percentage of total catch). As noted in the
BSERP final technical report (pg 48), "The reality is that we are no nearer to
identifying sustainable catches for the Black Sea than we were a decade
ago".
Achievements in this output were unsatisfactory, however most of the
responsibility for this lack of progress rests with the partner countries, not
with the BSERP team.


28

---

Objective 3: Development of economic instruments and promotion of investment
opportunities in coastal zones for pollution control and protection of Black Sea
ecosystems
Outputs / Indicators
Results/ Impacts
3.1: Overall
The achievements with respect to this output are marginally satisfactory.
economic analysis
carried out analyse
Black Sea country national policies and programmes have not been analysed
national policies and
in consideration of cost effectiveness in reducing the discharge of nutrients
programmes and
and hazardous substances. Instead, a generic study of potential measures
cost-effectiveness of
to reduce these pollutants was developed. The study appears to have been
measures in respect
,,borrowed or at least built largely from activities within the UNDP/GEF
to reduction of
Danube Regional Project. The deliverables included three "pilot" studies
nutrients and
that were submitted to the BSERP by its consultant, focusing on nutrient
hazardous
discharge reduction measures for farmers, municipalities and fertiliser
substances (results
factories. These studies are not specific to any particular facilities or even
of the analysis are
to the Black Sea region, although some Danube and EU information is
incorporated in the
included for Romania and Bulgaria. The recommendations provide little
revised TDA -
practical guidance for the Black Sea countries, as they are a compilation of
Activity 2.1.1).
possible measures which would need to be selected based upon the specific
considerations for any particular municipality, farm or fertiliser plant.

The generic nature of this information is unfortunate, in particular because 5

of the 6 participating countries came through a major economic
transformation over the previous decade that decimated their agricultural

production. As this sector of the economy rebuilds, there are opportunities
to establish new management measures that are more environmentally

benign, yet they must be built upon an understanding of the conditions in
the countries, and utilising real world examples.

There is no indication that any of the information from the reports has been

translated and disseminated, so it is at this point of negligible utility to
farmers, municipalities and the fertiliser industry in the Black Sea basin.

An analysis has been included in the TDA that provides background data on

the socio-economic situation amongst the 6 countries in the basin.

A cost-effectiveness analysis of existing national policies to reduce nutrient
and hazardous substance pollution was also an expected output, but not

achieved. Such analyses are complex and expensive to carry out, requiring
use of a source apportionment model, validated by monitoring data. The
budget established for output 3.1 was not sufficient, nor was the quality of
data available to effectively carry out this analysis.


29

---

3.2 Investment
Achievements under this output can be considered satisfactory. The
programme for
original 2nd Phase ProDoc included preparing regional investment
nutrient
programmes for municipal, industrial and other infrastructure projects, a
pollution/load
donor conference in 2005, and further development of interactions between
reduction measures
the private sector and GEF. Recognising that regional investment programs
in the Black Sea
and a donor conference duplicated efforts of the DABLAS Task Force, the
coastal zones is
Logframe and PIP for Phase 2 replaced these with a) country- specific
revised for further
guidance notes on Russian and Ukrainian water utility financial analysis, and
submission to the
b) preparation of two pilot water utility investment projects: Mikolaiv W&S
IFIs and bi-lateral
DABLAS Pilot and Yalta W&S DABLAS Pilot.
donors.
The BSERP has successfully assisted the Mykolayiv City and Utility to
develop a proposed short term investment program (STIP), incorporating
recommendations for financial and operational performance improvement.
The BSERP support has built upon initial work from the EC DABLAS Project
Broker and forms part of the DABLAS Task Force Pipeline. The final report
of activities was submitted in January, 2008.
The same consultant to BSERP provided assistance in developing the Crimea
Regional W&S Investment Project. Their results, finalised in January 2008,
include a technical, financial and institutional review of the Crimean Water
Supply and Sewerage Sector, a short term investment project (STIP)
proposal (focused on Leninskiy District), and TOR for further technical
assistance were developed. It has been indicated to the evaluation team
that the Government in the Crimea region (ARC) has requested that $5
million funding from the stalled World Bank project: Crimea Coastal Zone
Management Project be redirected to the Crimea Regional water and
Sewerage Investment Project. No response from UNDP GEF has been
provided concerning this request,
The investment assistance, including STIP development, appears well
conceived and carried out. Both of the assignments build from the original
DABLAS priority list. It is noted that both projects are focused on basic
upgrading of wastewater treatment and sewerage systems and priority
water system improvements. Neither project, at this stage, includes
planning for technologies and techniques specifically focused on the
reduction of nutrient discharge.
The investment programme under BSERP was not coordinated with the
World Bank nutrient reduction facility, despite the periodic convening of
stock tacking meetings between these two parts of the Danube ­ Black Sea
Strategic Partnership. It is clear that the World Bank developed its project
investment pipeline without consideration of the BSERP TDA and other
deliverables.
Objective 4: Development of operational systems for monitoring, information
management and research under the Black Sea convention.
Outputs / Indicators
Results/ Impacts
4.1 The Black Sea
Achievements in the BSIMAP effort can be considered satisfactory.
Integrated
Monitoring and
Manuals have been prepared for sampling and analysis of: nutrients,
Assessment
chlorophyll, phytoplankton, zooplankton and zoobenthos. The manuals also
Programme
contain procedures for quality management. The manuals have been
(BSIMAP) is
discussed, tested (including training) and approved in workshops in which
operational,
representatives of all countries participated.
providing tools and
Pilot monitoring exercises (PMEs) were undertaken by 5 countries (BG,
indicators to
GE, RO, RU, UA) in two rounds. The sets of analysed parameters were not
evaluate changes
the same for all countries. Not all countries appear to have applied the
over time in the
sampling and analysis methods prescribed in the abovementioned manuals.
coastal and marine
The results are reported in separate reports prepared by each of the
environment.
countries, with varying content (i.e. the Russian report is incomplete).
None of the reports contain a discussion /reflection of the PME in terms of
,,lessons-learned for the BSIMAP.
BSIMAP is getting mature, meanwhile containing standardised sampling,

30

---

storage, analytical techniques, and assessment methodologies and
reporting formats for most parameters. There are clear intentions
concerning quality management procedures, including proficiency testing. A
major gap is that agreed methods for sampling and analysis of micro
pollutants in water, sediment and biota are still lacking. Furthermore, there
are no agreed sets of environmental indicators yet. Activities in the
Advisory Groups concerning development of indicators commenced in 2003,
but are still not finished. For the hydrobiological parameters there are
furthermore no complete lists with Black Sea species available yet.
Activities for compiling such list have been started during the project, but
are not yet completed.
BSIMAP is not fully operational, since the agreed monitoring and
assessment programmes are not yet implemented by all BS countries. A
number of factors apply. Getting sufficient finances for a BSIMAP compliant
monitoring programme seems to be a problem in all countries2. The
capacity for laboratory analysis of micro pollutants is not yet fully
developed in all countries, due to either lack of equipment or of sufficient
expertise in the analyses. Some countries also do not have the appropriate
microscopes required for analysis of several hydrobiological parameters.
The mandatory requirements for national monitoring programmes are not
always in line with BSIMAP. The organisational arrangements (which
institute, laboratory will conduct which part of the monitoring) are not yet
settled in all countries. These factors have been mainly out of the reach of
BSERP.
VTOPIS
The VTOPIS (Vessel Traffic Oil Pollution Information System) software has
been installed on computers in the Bulgarian Marine Administrations in
Burgas and Varna in the autumn of 2007. The Marine Administrations are
said to use the system on a daily basis and to be happy with the system.
The VTOPIS software is not ,,plug-and-play, meaning that it is not
immediately operational when installed on computers of maritime
administrations in other countries. Required adjustments include: localised
electronic maps/charts of the harbours and coastal area, localised statistics
on prevailing current and wind directions, and general integration with the
existing systems, like the VTMIS (Vessel Traffic Management Information
System). According to the developer, the key for exchange is in the
underlying database. As long as data are stored in compatible formats
(templates), administrations have means of exchanging data such that they
more easily can be processed mutually. Here, the ESAS AG can play an
important role.
The perspectives on replication of VTOPIS in other countries are uncertain.
The discussions during AG ESAS meetings in the last years show that
signing of an agreement on implementation of a centralised system will be
difficult due to political and technical issues. The final VTOPIS meeting in
October 2007 was attended by representatives of Bulgaria and Romania
only. The Romania Naval Authority is enthusiastic about VTOPIS, notably
the backtracking module, and is interested in introducing it in their own
systems.
With respect to oil spill prevention and remediation, in addition to the
VTOPIS demonstration, a number of coordinating activities were carried
out during the 2 project phases: sensitive area have been mapped, a
Contingency Plan has been developed, and dry run exercises are
implemented on a regular basis. During accidents in the Kerch Straits in
2007, the contingency protocols and communication routes were utilised
and considered effective.
4.2: Black Sea
The achievements in the BSIS development can be considered marginally
Information System
satisfactory.
including tools for

2 An unintended side-effect of the PME has been that it actually provided some water quality data for the year 2006 for Ukraine. Due to lack of
finances, the Ukrainian Scientific Centre Ecology of the Sea (UkrSCES) was not able to perform their routine monitoring work, so otherwise
there would not have been any data for 2006 at all.

31

---

GIS, mapping and
The major part of the development of the BSIS by BSERP took place during
remote sensing
Phase 1 and the initial stages of Phase II. The original members of the PS
developed to support
were not able to clearly outline the expected outputs of the BSIS. After
the activities of the
having adjusted the system several times for a series of (sometimes
BSC and
contradictory) requests, the team decided to put the system development
implementation of
on a hold. During Phase II BSIS has been populated with data until 2006,
the BSSAP.
prepared it to become accessible via web-based Intranet and added
mapping facilities.
The structure of the current BSIS very much reflects the structure of the
SAP of 1996. Data are entered and can be retrieved using templates that
follow the numbered articles of the SAP. The current Technical Draft of the
revised SAP is structured quite differently; possible ramifications for the
BSIS have not been made.
The new Secretariat staff did not start actively using the BSIS for their
activities (like for compilation of annual topic reports). The current
Secretariat PMA officer mentioned that the expected outputs are not yet
defined, such as indicator-based reporting. The system has not yet been
made accessible to other bodies, like the Advisory Groups or Focal Points;
they merely provided their data in the required formats.
The BSERP mid term review indicated: "The BSIS format and templates are
very good, although more effort is needed now to provide help aides and
training to the BSC PS and other users." Up until the last weeks of the
project this remained still an issue. The BSIS software was installed on the
server of the BSC Secretariat only in April 2008, at which time a full set of
the documentation was prepared and training exercises took place (both for
database management, and database users).
Recognising that the BSIS was already essentially developed by 2004, it
would have been very useful to put this system to use as the information
repository for both the BSERP and the BSC. It is very unfortunate, and a
significant procedural shortcoming that the training and transfer for BSC
use took another 4 years. It is recognised that part of the delay stemmed
from differences of opinion with previous members of the BS Secretariat on
how the database should be configured, and also there were issues of data
sensitivity that previous Secretariat management indicated required
separate database development. The use of incompatible information
systems, and the lack of effective data sharing greatly impeded both BSERP
and BSC performance.
4.3: Research
The results of the research programme can be considered satisfactory.
Programme designed
and implemented to
Research cruises
assess input of
In addition to the research cruise of Phase 1, three more cruises have been
nutrients and
organised under Phase 2 (2004, 2005, and 2006). The research cruises
hazardous
covered various components of the Black Sea ecosystem, including water,
substances in the
sediment, benthic communities, Phyllophora (including the Phyllophora
Black Sea
fields in Odessa Bay) and other macrophytes, phytoplankton and
zooplankton. Especially the opportunity to conduct measurements in the
open sea has been appreciated very much (most national surveys are
confined nearer to their coasts). The cruises had a strong focus on the
northern and western parts of the Black Sea, with the third cruise
extending some of the measurements to the south. This leaves the eastern
part towards Georgia and Russia less studied.
The draft Summary Report: BSERP Research Cruises and Pilot Monitoring
Activities 2003-2006 (published on the PIU 2008 DVD) contains a chapter
with a critical overview of lessons learned. A number of issues are
addressed: the 2005 expedition was hampered by problems with the vessel
and a storm; the post-cruise costs of processing/analyzing samples
collected during the cruises were not sufficiently well accounted for in the
budgets (sometimes leading to no or late delivery of reports); some of the
scientists delivered high quality reports on time, but there were also others
who were very late in delivering; there has been relatively little cross-
fertilisation of results between different scientists; the ISG wished to
pursue more fundamental research, investigating energy and nutrient

32

---

transfer between different trophic levels, while UNDP-GEF was interested
primarily in the ,,environmental status.
Numerous (raw) data have been obtained with the research cruises that, as
just mentioned, only partially have been integrated. The Summary Report
(cited above) has not been completed. A long awaited report on the cruise
findings from the chair of the ISG was published in January 2008.
Some major results of the research cruises were incorporated into the
updated TDA. Generally, the scientific Black Sea community has been
nourished by the expeditions, leading to a better understanding of the very
complex Black Sea ecosystem, including eutrophication phenomena. Some
of the results of the cruises could be used to indicate some improvements
in the status of the Black Sea ecosystem.
Modelling
The Kamchiya River pilot project comprised the modelling of nitrogen and
phosphorous emissions in this Bulgarian River. The model developed under
the Kamchiya River pilot project is rather straightforward and completely
contained in Excel spreadsheets. It uses several generic coefficients for
estimating emissions in case no actual load data are available. Because of
limitations in the available monitoring data, the comparison with the actual
river loads also had to be approximated with several artificial assumptions.
Generally, there is no lack of such nutrient models, varying in their
complexity and therewith in their demand for data required for running the
models. It is outside the reach of the evaluation to assess the overall
robustness of the model developed under the Kamchiya River project. It
seems to be a useful tool for some first tentative order of assessments. But
its performance is not yet sufficiently known to decide whether it could be
proposed as the model to be used by the Black Sea Basin countries.
Atmospheric deposition
A pilot study was implemented in Phase I within ISG activities; however, it
was decided by the project Steering Committee not to continue in Phase II,
since there was not enough funding to make it properly, and a very general
and report-only-oriented activity was not viewed to be useful.
Scientific conference
BSERP facilitated (also financially) the First Biannual Scientific Conference:
Black Sea Ecosystem 2005 and Beyond. May, 2006. The Second Conference
in 2008 has been co-financed by BSERP.

Objective 5: Strengthening of public participation in environmental protection
through access to information, stakeholder training and awareness raising and
implementation of community actions (small grants programme)
Outputs / Indicators
Results/ Impacts
5.1: NGOs structures
Achievements for this output and set of activities can be considered
and activities
satisfactory. The LogFrame indicates four activities, including
reinforced though
development of an NGO registry, support to NGOs to participate in
support for
periodic forums, training on coastal environmental management and
institutional
support for translation and dissemination of publications.
development and
community actions in
NGO registries were developed for each of the six countries. They appear
awareness raising,
to be quite comprehensive. They are available as six separate files. The
training and education
registries were not periodically updated, nor were they compiled into a
on the issues related
database / address book that could have served as a regular list for
to the management of
sending out information and informing of upcoming events.
nutrients and
During the BSERP the Black Sea NGO Network was created and came to
hazardous substances. serve as the key regional NGO liaison for the BSERP and BSC. The
network has demonstrated a professional approach and has been able to
independently obtain additional grant funding, providing the hope and
expectation that it can be sustained after the BSERP is concluded. A key

33

---

question for NGOs in the future will be the extent of their participation in
subsequent Black Sea Commission meetings. The BSC has no funding
capacity to assist with NGO participation. It can be assumed that the
Network will utilise its Turkish affiliates to enable a low cost continuing
presence.
The BSERP team made real efforts to involve NGOs in the many activities
they carried out under the project. In particular, NGOs participated in the
SAP development process, and played key roles in public participation and
small grants efforts. The effort to engage NGOs was enhanced through
development of a BSERP NGO Work Plan, containing activities and
timetables.
In June 2006, 30 NGOs from all six Black Sea Coastal countries
participated in an NGO workshop sponsored by BSERP, to identify and
prioritise NGO activities in the region and identify NGO capacities and
skills.
NGO training sessions were held in all of the Black Sea countries except
Russia during 2007. The training workshops helped to: promote
environmental education, raise awareness on environmental issues in the
Black Sea region, improve NGO networking and improve communications
skills.
Two publications received translation assistance support from the BSERP:
a book on the comb jelly fish (Mnemiopsis), and a book on Black Sea
Ecology.

5.2: Community
The small grants output was highly satisfactory. Two grant sessions
actions for awareness
were run, the first during Phase 1, including 17 projects carried out in
raising and
2003. The second series was carried out in 2006/2007 and involved 36
environmental
projects in total distributed fairly evenly across the 6 Black Sea Coastal
protection
Countries. These 2nd phase projects received awards of between $4900
implemented with
and $10,720, with a total 2nd phase allotment of $308,802.
funding from GEF
"Small Grants
The 2nd phase portfolio was subject to a separate evaluation, which was
Programme" targeted
carried out in February and March, 2008. The evaluators assessment was
specifically at the
that virtually all of the projects achieved success, and the effort was
support/participation
especially effective in general awareness raising. NGOs used their
in the management of
funding efficiently; with tens of thousands of pages of information
nutrients and
materials published and disseminated more than 100 public events, and
hazardous substances
dozens of local community actions triggering hundreds of media reports.
and protection of
The evaluator further indicated that while only a few of the projects were
coastal zones and
targeted towards specific environmental impacts (such as beach cleanups)
marine ecosystem.
the totality of the effort laid the groundwork for measurable environmental
improvement in the future.
The involvement of the CTLs in the small grants administration was
considered very useful, especially in the initial launching of the effort and
screening of applications, and then in the monitoring of the effort. It has
been indicated that there were some issues raised by the participants
concerning the rapid and brief project period, the small amount of grant
money available, and some problems with bank transfers.
5.3: Public
Achievements under this output can be considered satisfactory. This
information on
output included 6 activities: develop a communications strategy for the
reduction of nutrients
BSERP and BSC, launch and administer public information and awareness
and hazardous
raising campaigns, develop press materials, support environmental
substances, their
education and assist in developing and producing information material on
effect on the Black Sea the management of coastal zones and marine ecosystems. There was also
ecosystem, and the
to be an evaluation of the effects and impacts of the communications
recovery measures are programme at the end of the BSERP.
disseminated to the
public at large (i.e., by The overarching effort was put into Black Sea Day. In particular, effort
means of the
was geared up for the 2006 Black Sea Day, which also commemorated the
Communication
10th year anniversary of the BSSAP. The 2006 Black Sea Day was a major
Strategy, Educational
success: with over 200 events and activities, media coverage reaching an

34

---

Programme, Public
audience of 8 million, significant in kind contributions from NGOs, local
awareness campaigns, companies and media and the distribution of 27,000 branded items. The
media coverage).
2007 Black Sea Day received somewhat less support and exposure but
nevertheless continued to successfully draw attention to Black Sea
ecosystem and water quality issues.
An excellent education study pack has been developed during the BSERP
Phase 1. Phase 2 support provided an update of the design and content
and creation of an interactive CD. Unfortunately, the Study Pack remains
a great resource that is largely untapped. There are no indications that
school systems across the region have utilised the Study Pak as part of
their science curriculum. It is also in need of further support to translate
into the other regional languages and to continue to update and expand its
contents.
With the help of Coca Cola (its Every Drop Matters project with UNDP) a
22 minute BBC TV documentary was developed: "The sea that Nearly
Died". The documentary was shown on BBC World in May, 2007. The
documentary provided a well-developed laymens look at the current
situation in the Black Sea, and the positive signs of recovery.
One of the expected deliverables within this output was a communications
strategy. This should have been fully developed during Phase 1 inception
and then recalibrated following the revised focus of activities during Phase
2. A strategy was developed in 2003 by the Public Participation Specialist
(Staff Member) of the PIU and widely discussed at one of the SCMs.
However, it was not implemented in Phase I. Effort was then put into this
activity in Phase II, with a revised strategy developed and implemented
from June 2006.
The involvement of Coca Cola in supporting promotional efforts for the
Danube and Black Sea basin constitutes a useful public private
partnership. The Every Drop Matters programme for Coca Cola is a multi-
year effort that promises as much as $500,000 to the Black Sea
Commission for public awareness raising efforts. Inexplicably, the BSC has
not followed up to cement its partnership with Coca Cola and the financial
support for the Black Sea is in jeopardy. Certainly there must always be
caution and a clear set if guidelines for private sector involvement in
funding environmental awareness campaigns, however the precarious
situation of BSC future financing suggests that it is critical for the
Commission to look to a variety of non-traditional sources if it has any
hope of continuing to build public awareness.
The project included an English web site, yet did not include any
translation of that site, nor were there national sites. This significantly
restricted the web audience for this project. Given that the project
employed full time CTLs in each country, and had a sizeable NGO small
grant budget, it should have been possible to do more with respect to local
language web sites.
3.3.2 Country ownership
The major involvement of the participating countries has been through environmental
ministries and their subordinate bodies. These Ministries rarely have responsibility for
resource issues such as fisheries, mining and agriculture, so activities with the greatest
impact on water quality are largely outside of their control. This situation has been
recognised in the BSERP and other GEF IW projects. The solution has been to push for the
countries to set in place interministerial coordinating mechanisms. This effort has made
minimal headway amongst the Black Sea countries. The project results concerning Inter
Ministerial Coordination Mechanisms (IMCM) are contained in a rather generic report that
does not elaborate on Ukraine or Russia yet contains sections on Serbia and Moldova,
while Romania and Bulgaria merely get mentioned as having IMCMs in place. On a positive
note, the interactions with Georgia seemed to have encouraged officials to assign the role
of environmental coordinating body to the State Commission on Water Supply and
Sanitary Policy Development.

35

---

Some interviewees labelled outputs like the updated TDA as being more of a BSERP
product than something of high priority to the countries. This could be considered
indicative for the overall perception of the project. While generally being highly
appreciated for its facilitation, like organising and co-financing international meetings and
research expeditions, not necessarily all of its activities and envisaged outputs received
similar interest. Representatives interviewed in Bulgaria and Romania furthermore
explained there was less interest in BSERP because a substantial part of its envisaged
activities and results already are covered by these two countries as they strive to meet EU
requirements, especially the Water Framework Directive.
There has been a noticeable lack of progress made on the BSSAP (1996). Subsequently,
the six states have had difficulties reaching consensus on biodiversity, land based sources
and fisheries protocols. There are still major problems in the sharing of data, especially
concerning point sources of pollution, and in reaching agreements on monitoring
methodology. Even the methodologies for determining hot spots are still called into
question, 12 years after the first hot spot designations. Other country ownership issues
arise with respect to the lack of Commission follow through on most of the
recommendations for institutional strengthening that were developed with BSERP support,
including in its exit strategy. Taken together, the lack of progress and follow through
points to a decided lack of country ownership and interest in working jointly to protect and
improve the Black Sea ecosystem. This lack of country ownership poses serious risks to
the future effectiveness and sustainability of the Black Sea Commission.
It is premature to assess the possible BSERP long term impacts on the policies and
strategies of the countries. Some key outputs, like the revised SAP and LBA Protocol, were
still in the process of being completed while preparing this evaluation report. The possible
adoption of the revised SAP and LBA Protocol during the Ministerial meeting scheduled for
October 2008 would be a major positive project outcome, and evidence of increasing
country ownership and commitment.
3.3.3 Stakeholder and public awareness.
It is difficult to gauge the level of awareness in each of the countries concerning project
outputs due to the wide ranging variety of these outputs, and the close relationship
between BSERP outputs and BSC outputs. No specific activities have been undertaken to
actually gauge stakeholder awareness of project outputs.
Within a fairly narrow band of government ministry officials involved in marine
environmental protection for the Black Sea, there can be assumed a generally good
awareness of the BSERP efforts on the revised TDA and SAP. Within the marine science
community, there was an appreciation for some of the scientific efforts, including the joint
research cruises. For NGOs, the small grants effort was widely known and appreciated.
For the general public, the Black Sea Day (especially 2006) can be considered successful
in calling attention to Black Sea issues. Activities like the National Workshops on the LBA-
protocol and the stakeholder analysis among 42 stakeholder groups, plus the involvement
of the Advisory Groups made a broader audience aware and acquainted with the projects
results. Key stakeholders from the private sector, including industry and agriculture, were
not specifically engaged during the project implementation.
3.3.4 Cost-effectiveness
The BSERP has taken 6 years and cost GEF $ 10.3 million. An additional $9 million is the
estimated co-financing, although the participating countries have not estimated their in-
kind contribution at project end, so these numbers are speculative at best. As noted in
the above sections discussing each of the project outputs, there is a decided lack of
success in meeting the verifiable indicators of environmental and policy reform progress as
a result of this project.

36

---

3.3.5 Sustainability of project impacts
Sustainability is dependent on the countries continuing to take joint and separate actions
to reduce nutrient and other pollution loading and to enhance fisheries recovery. Country
buy in has been and will remain a major point of concern. Current financial contributions
to the BSC merely allow for running a Permanent Secretariat with several staff members,
and without funding for expert/advisory groups, further joint research and monitoring
efforts or public awareness activities. Furthermore, the lack of progress made on the first
SAP and the slow pace of approval and ratification of the Biodiversity and LBA Protocols
suggest that agreement and then implementation of common policies are likely to remain
very difficult.
Objective 1. Support the consolidation and operation of institutional
mechanisms for cooperation under the Black Sea Convention
Outputs / Indicators
Sustainability
1.1: Operational structures and
There remain structural and financial weaknesses in the
management tools of the Black
Black Sea Commission, suggesting that it will be difficult for
Sea Commission further
the BSC carry on much of the work that has been provided
developed and functioning.
by the GEF and the EU for legal and policy development and
institutional capacity building. .
BSIS and BSIMAP provide useful tools that the BSC can be
expected to continue using.
While the project provided many policy and institutional
recommendations and some useful information
management tools, uptake by the Commission has been
limited
Many of the other activities that have developed due to
BSERP support, including small NGO grants, applied
research on ecosystem health, efforts to harmonise and
improve monitoring quality control, and raise the
prominence of Black Sea day, are unlikely to be continued.
1.2: Black Sea Project
Not really applicable with respect to sustainability.
Implementation Unit of the
BSERP (BSERP-PIU) fully
operational for implementing
Tranche II of the Project.
Objective 2: Development of policy guidelines, legal and institutional
instruments of pollution reduction from LBA and protection of ecosystems of
the Black Sea and its coastal zones.
Outputs / Indicators
Sustainability

37

---

2.1: Update/Renew the
Transboundary Diagnostic Analysis
Transboundary Diagnostic
Analysis (TDA) and Strategic
Recognizing its deficiencies, in particular with respect to data
Action Programme (Black Sea
accuracy and the exclusion of potentially important pollution
Strategic Action Plan, BSSAP) to
sources, such as Istanbul, the revised TDA still represents a
account for the changes in the
significant improvement over the previous TDA and provides a
pressures on and the state of the clear and up to date consideration of the pressures and
Black Sea ecosystem.
measures of concern to the Black Sea ecosystem. The text
has received favourable comments from the Commissioners,
Secretariat and other key stakeholders and can be expected to
provide the Commission and participating governments with a
useful tool for identifying cooperation priorities.
Strategic Action Plan
The Technical Draft SAP was compiled during the later stages
of the project, and is now being revised and further jointly
developed by the BSERP and BSC. A consultant working for
the Black Sea Commission has been detailed to make
revisions, supported by a full-time BSERP PIU staff member.
There have been extensive negotiations with all of the
countries, and major revisions to the draft based upon country
comments. Nevertheless the current document bears a strong
similarity to the Technical Draft SAP produced in December
2007. The PIU has high expectations that an updated SAP will
be signed by all parties later this year
2.2: Adoption of the Protocol for
The draft revised LBA Protocol is likely at some point to be
Land-based Activities (LBA)
approved at the Commission level, given the significant
(concluded and adopted as a
amount of effort put in. The greater question and concern is
draft in Phase I) is facilitated
whether it will succumb to the fate of the Biodiversity and
through the national and
Landscape Conservation Protocol, which was signed in 2002
regional negotiation process in
but remains not in force, with only two of the six parties,
order to ensure the adoption of
Turkey and Ukraine, having ratified it.
the revised text at the Ministerial
Conference (2007).

2.3: Strengthen Integrated
The sustainability of ICZM efforts in the region has benefited
Coastal Zone Management in line from the BSERP efforts together with the ECBS (EU) project.
with EU Directives and in testing
The ICZM pilot in Turkey is creating interest there for
concept for Best Practices for
additional coastal zone planning activities and with the help of
ICZM as developed by
the ECBS there is an ICZM law under consideration now in
BSC/TACIS, to assure reduction
Georgia. Based from interview comments of several Black Sea
of nutrients and hazardous
Commissioners, there are likely to be difficulties getting
substances from coastal areas
coastal zone protocol through the Black Sea Commission at
into the Black Sea.
this juncture, a draft Regional ICZM Strategy document was
approved by the Commission in 2004.
2.4: Agricultural policies
While the achievements under this agricultural output were
reviewed for application at
minimal, there are quite a number of important agricultural
national level to limit export of
management targets in the draft SAP, with 5 of the countries
nutrients and hazardous
having so far agreed.
substances.
2.5: Industrial/transport and
There were no achievements under the industrial/transport
municipal policies and legislation and municipal policies effort that are likely to have a
reviewed for application of BAT
sustainable impact, however without discharge data, it was
(best available techniques,
impossible for the PIU to determine issues such as cost
including cleaner technologies)
effectiveness and which industries should be prioritised.
towards reduction of nutrients (N
and P) and hazardous

substances.

38

---

2.6: Legally binding Document
There were no achievements under the fisheries output that
on Fisheries finalised and
are likely to have a sustainable impact. If the BSC refuses to
proposals for establishment of
accept the European Commission as a member, the BSC is
fisheries-free zones and marine
unlikely play a major role in fisheries policy for the region.
protected areas developed

Objective 3: Development of economic instruments and promotion of
investment opportunities in coastal zones for pollution control and protection
of Black Sea ecosystems
Outputs / Indicators
Sustainability
3.1: Overall economic analysis Sustainability of Objective 3 outcomes concerning economic
carried out analyse national development and investment opportunities is unlikely. The
policies and programmes and projects outputs were rather shallow to start with.
cost-effectiveness of measures Furthermore, there is a tendency in GEF IW projects to pay ,,lip
in respect to reduction of service to cost/benefit analysis and the use of economic tools,
nutrients
and
hazardous however, the finances set out for these studies and activities
substances
(results
of
the is meagre, there is insufficient linkage to available investment
analysis are incorporated in the finance, and the project identification efforts rarely extent into
revised TDA - Activity 2.1.1).
full feasibility studies ­ so they are of limited use to IFIs. The
generic nature of the information developed suggests that it
will be of marginal utility to the Black Sea Countries in the
future.
3.2 Investment programme for
There can be expected real, tangible and sustainable
nutrient pollution/load reduction
achievement as a result of the efforts to assist with feasibility
measures in the Black Sea
studies for Mykolayiv City and Crimea regional Water and
coastal zones is revised for
Sewerage Investment Project. These are both firmly within
further submission to the IFIs
the DABLAS pipeline and supported by the Ukrainian
and bi-lateral donors.
Government.
A major challenge for the Black Sea Commission will be able
to disseminate and yield upon such pilot projects.

Objective 4: Development of operational systems for monitoring, information
management and research under the Black Sea convention.
Outputs / Indicators
Sustainability
4.1 The Black Sea Integrated
BSIMAP holds out promise for sustainability. The standardised
Monitoring and Assessment
sampling, storage, analytical techniques, and assessment
Programme (BSIMAP) is
methodologies and reporting formats have been developed
operational, providing tools and
and agreed. However, BSIMAP is not fully operational, since
indicators to evaluate changes
the agreed monitoring and assessment programmes are not
over time in the coastal and
yet implemented by all BS countries. Full implementation of
marine environment.
BSIMAP by all countries is not something to be expected in the
short term.
Sustainability and replication of VTOPIS are uncertain. Signing
of an agreement on implementation of a centralised system is
a difficult issue due to political and technical issues. The
Romania Naval Authority is enthusiastic about VTOPIS,
notably the backtracking module, and is interested in
introducing it in their own systems.
4.2: Black Sea Information
The formal turnover of BSIS to the BSC Secretariat took place
System including tools for GIS,
after the evaluation, so training and operational manuals are
mapping and remote sensing
only now in the hands of the Secretariat staff. It can be
developed to support the
assumed that the database will be utilised by the Secretariat,
activities of the BSC and
especially for updating of information for the TDA and SAP.
implementation of the BSSAP.

39

---

4.3: Research Programme
Some major results of the research cruises were incorporated
designed and implemented to
into the updated TDA. Generally, the scientific Black Sea
assess input of nutrients and
community has been nourished by the expeditions, leading to
hazardous substances in the
a better understanding of the ­very complex­ Black Sea
Black Sea
ecosystem, including eutrophication phenomena. The results
of the cruises have been used to verify improvements in the
status of the Black Sea ecosystem, in particular signs of some
recovery in the Phyllophora fields. Future joint cruises require
donor support. Currently there is a joint cruise being
organised within one of the Framework Programme projects of
the European Commission (ELME project).
The Kamchiya River pilot project comprised the modelling of
nitrogen and phosphorous emissions in this Bulgarian River.
The model seems to be a useful tool for some first tentative
order of assessments, but its performance is not yet
sufficiently known to decide whether it could be proposed as a
model to be used elsewhere by the Black Sea Basin countries.
Atmospheric deposition
No activities were carried out beyond using existing
publications to estimate loading.
Scientific conference
BSERP facilitated (also financially) the First Biannual Scientific
Conference: Black Sea Ecosystem 2005 and Beyond. May,
2006. The Second Conference in 2008 is also co-financed by
BSERP. The BSC financial situation suggests that external
donor financing will be needed for any future scientific
conferences.
Objective 5: Strengthening of public participation in environmental protection
through access to information, stakeholder training and awareness raising
and implementation of community actions (small grants programme)
Outputs / Indicators
Sustainability
5.1: NGOs structures and
There appear to be good opportunities for the continuation of
activities reinforced though
the NGO network, especially with indications that they have
support for institutional
received additional funding from other sources to continue
development and community
their coordinating efforts amongst NGOs in the Black Sea
actions in awareness raising,
Region. The previous history, whereby NGO umbrella
training and education on the
organisations were supported under the GEF projects only to
issues related to the
dissolve once funding ended, appears to have now been
management of nutrients and
overcome. What is far less certain is the extent of future
hazardous substances.
participation of the BSNN in BSC Annual Meetings. There
appears to be a genuine interest from the BSC for a continued
input from the NGO community, but financing that
participation may be a problem. It can be assumed that the
Network will utilise its Turkish affiliates to enable a low cost
continuing presence.
The NGO training efforts hold promise as having continued the
process of increasing their capabilities, especially on the
management of international support financing.
The translation of scientific texts was very modest and unlikely
to have much impact, especially with the translation effort
only into English (from Russian).

40

---

5.2: Community actions for
The small grants effort holds out hope of sustainability, as the
awareness raising and
effort enabled more than 30 NGOs to participate and it can be
environmental protection
expected to have a successful impact in terms of building
implemented with funding from
public awareness. What is unknown at this point is the extent
GEF "Small Grants Programme"
to which the BSERP small grants effort has enabled involved
targeted specifically at the
NGOs to increase their success in obtaining funding and public
support/participation in the
support for their work. The small grants evaluation provides
management of nutrients and
some anecdotal information of successful continuing efforts
hazardous substances and
amongst the 53 projects.
protection of coastal zones and
marine ecosystem.
5.3: Public information on
Black Sea Day is in danger of receding back into the
reduction of nutrients and
background of events ­ or then to become more of an NGO
hazardous substances, their
driven, localised effort. Inexplicably, the BSC Secretariat has
effect on the Black Sea
so far chosen not to follow up with the offer from Coca Cola to
ecosystem, and the recovery
continue supporting the Black sea day effort, and they have
measures are disseminated to
not solicited funds from other sources, so the means to
the public at large (i.e., by
organise events, establish a media presence and deliver
means of the Communication
branded items is not there, It is also not clear yet whether
Strategy, Educational
the Black Sea Commission will hold its 2008 Annual Meeting in
Programme, Public awareness
Odessa around the Black Sea Day, as they continue to
campaigns, media coverage).
struggle with the agenda and approval of key deliverables like
the LBA Protocol.
3.4 Best practices and lessons learned
1. The TDA/SAP approach in the GEF IW Programme has proven its value across many
projects. Lessons from the BSERP suggest that the process of revising TDAs and
SAPs should not be the rational for a continuation project. It is important to
reassess and update environmental status, and to identify possible changes in
pressures, and the TDA provides a very good format for this. However TDA
revisions and SAP updates should be periodic/ongoing activities managed by the
transboundary waters commissions, with project PIUs playing a supporting role.
Taking this responsibility to an internationally funded project runs the real risk of
lowering country ownership and responsibility. Of course, there needs to be a well-
managed and adequately funded Commission in place to take on TDA/SAP
development responsibilities
2. The BSERP utilized an independent scientific body (e.g BSERP International Study
Group) to identify the agenda for applied research through the joint research
cruises. This concept should be considered for replication in other projects. An
ISG enables top scientists in the region to engage with the project, and ensure that
applied research efforts are science rather than policy driven. The BSERP ISG ran
into some difficulties with the publication of information culled from the research
cruises. It took far too long to get information out, and there have been reported
instances where published works from the research effort did not cite the financial
contribution of UNDP/GEF/BSERP.
3. The Black Sea region during the last 10 years has witnessed a significant reduction
in support for marine sciences and regional research institutes. Previously well-
staffed and funded institutes were stripped of their funding and in a precarious
financial position. IW projects like the BSERP have become a critical lifeline for
research on marine and riverine issues. This suggests a two-fold consideration:
first the extent to which national support for the participating research institutes
should be a pre-condition for GEF support, and then consideration on the types of
support ­ such as training and capacity building, that should be included in the
project activities. Central and eastern European countries are full of laboratories
with high priced internationally-funded machinery but no spare parts or
consumables, few technicians with the skills to operate the machinery, and few
assignments where the machinery is needed.

41

---

4. The BSERP Steering Committee made a strategic decision during the project first
phase to hire full time paid Country Team Leaders in each of the six countries to
help coordinate activities. The decision was made based at the urging of the
participating countries and reflecting some difficulties in getting the involved
Ministries to follow through on expected commitments. The CTL effort received
generally strong support from the stakeholders interviewed, raising the question
whether this approach should be replicated in other projects.
There can be some benefit to taking a CTL approach to project management in
special cases, however this approach should be avoided where possible due to
significant downside risks. On the positive side, hiring CTLs can enable projects to
expand country activities and improve coordination in situations where the
responsible agency is not managing the effort well. As to the downside risks:
o CTLs constitute a major cost factor for multi-country projects. What is
typically expected as a co-financing cost born by the partner countries
becomes an administrative cost to the project ­ thereby reducing available
funds for demonstration projects and other technical assistance.
o There is an important issue at stake concerning country ownership and
sustainability. If the countries are not financially responsible for in-county
project management, they are less likely to have a stake in the outputs and
productivity of the CTLs and project as a whole.
o If the in-country coordination is paid for by the project ­ then at the end of
the project there are likely to be problems in terms of sustaining the effort.
The BSERP devised a phase out strategy over the final 18 months to scale
back CTL financial support ­ with the expectation that the countries would
escalate their support.
o If CTLs are to play a technical role they need technical skills. This requires a
transparent CTL selection process that can identify persons with the
requisite technical capabilities.
o CTLs paid for by the project but selected by the responsible Ministry officials
encounter loyalty and ,,chain of command pressures. Who are they
responsible to and how can they be replaced if they fail to perform their
proscribed duties?
The reality in some countries is that the responsible Ministries and their staff are
already overburdened and have little capacity to take on more obligations, and
budgets are not increased to cover the in-kind contribution obligations made when
signing up for the GEF project. It may be that future GEF projects can address this
issue through carrying out a more extensive needs assessment of each
participating country during project development, and generating a more detailed
and ,,honest assessment of expected country in-kind contributions. Such an
approach could lead to a differential support structure for projects ­ where
countries may be eligible for direct support for in-country coordination if it is clear
they cannot participate otherwise. More capable countries would then be
supported only through indirect mechanisms such as demonstration projects. The
UNDP/GEF Danube Project experienced this situation during the second phase and
elected to hire a full time CTL only for Bosnia-Herzegovina, which was well-justified
on the basis of the split government structure in that country.
5. Exit Strategies can help countries focus on the eventual closure of a GEF support
project and consideration of what they will do to sustain and replicate activities.
The BSERP/BSC Exit Strategy is quite brief and lacks an array of options (based on
available budget) nonetheless, it includes useful comments and recommendations,
and it builds upon the institutional review developed for the BSC, so the two taken
together constitute a reasonable consideration of how to strengthen the capacity of
the BSC to carry out its intended mission(s) over the short, medium and long
terms.

42

---

6. One of the key concepts behind Exit Strategies is to set in motion a gradual phase
out of project support, especially for Commission and Advisory Group meetings,
with the partner countries increasing their budgets to compensate. The BSERP
Exit Strategy establishes no such recommendations for funding phase down, and
there is no indication that discussions were held with the Commission members (in
their dual role as Steering Committee Members) on a phased transfer of financial
responsibility.
7. The institutional strengthening aspects of GEF IW projects, especially when they
include support for a Permanent Secretariat, need further consideration and
revision. In the Black Sea project, similar to the Danube experience, the
relationship between project PIU and Commission Secretariat is complicated and
fraught with potential problems ­ especially relating to span of control and
decision-making authority on how budgets are allocated. Project Documents need
to clarify in far greater detail this relationship. For the Black Sea, the problem was
initially exacerbated by having a separate steering committee structure for the
Project and the Commission. During the 2nd phase, this was altered, with
Commission members also participating as Project Steering Committee members.
8. The BSERP small grants effort builds upon successful small grants sub-programmes
in several of the GEF full size projects. In light of these achievements, UNDP and
GEF should consider expanding the budget for NGO small grants components, but
better integrating them into the strategic planning for many of the planned project
outputs. So, for example, NGO small grants should be a key aspect of the
communications and public awareness strategy, with NGOs providing specific
inputs that help the project meet its strategic aims. Likewise, there should be
room for NGO participation in pilot studies, environmental monitoring, education
and training activities.
3.5 Recommendations
3.5.1 Black Sea Countries and Commission
9. Agree to allow international/regional organizations into the Commission, paving the
way for membership of the European Commission. This is the only means for
enabling this Commission to play a significant role in Black Sea fisheries, as
Romania and Bulgaria can no longer negotiate on these issues independent of the
European Union. EU membership can also be expected to aid in financial support
to the organization.
10. It is unlikely that the countries will provide much in the way of additional funding or
resources to the Black Sea Commission and its Secretariat. As a consequence, it is
important to set realistic expectations for Commission and Secretariat activities.
The existing Secretariat work plan includes a wide array of activities that cannot
possibly be achieved by two technical experts and an IT expert. The work plan
should be drastically pared down, commensurate with the extent of interest and
support being shown by the participating countries.
The Secretariat should be able to accomplish the following:
Support for the Commissioners in terms of setting the agenda providing briefing
documents for annual meetings, and tracking progress on SAP and other
agreements and country obligations.
Assisting Commissioners in the revisions and then approval and ratification
efforts for the LBS Protocol, revised SAP and Bucharest Convention
amendments
Develop State of the Environment reports for the Black Sea
Coordinate communications and outreach to linked Commissions and
organizations (MAP, ICPDR, HELCOM, European Commission, European
Environmental Agency).

43

---

Serve as a repository for news, information and applied research on the Black
Sea, using the BSIS and made available through the BSC website.
Organize an annual Black Sea Day celebration.
11. Expert/Advisory Groups can and should supplying technical and policy expertise
and country support to the BSC. In the case of other Commissions, such as
HELCOM and the ICPDR, the AGs (or expert groups) function as the driving forces
for policy recommendations and the drafting of legal instruments such as protocols.
In the case of the Black Sea, the AGs have seriously underperformed. AG
performance was hampered by: participants changing frequently and many
participants lacking adequate English language skills. The BSC Institutional Review
identified 16 subsidiary bodies to the BSC: 6 regional activity centres, 7 standing
advisory groups and 3 ad hoc working groups (for WFD, State of the Environment
Report, and Danube / Black Sea Joint Technical Working Group). The Institutional
review provides a recommendation to reconfigure this subsidiary arrangement, and
reorganize support into three "units", organized thematically around Science and
Information, Policy and Law, and Economic Sustainability / Project Management.
While the reorganization of the advisory groups into these three units is a sound
idea, the further suggestion to have countries assign and pay for experts to staff
these units is not likely to get much traction with the responsible ministries. A
more modest approach would be to reconfigure the expert groups into the three
thematic areas, and then push the countries to fund an annual meeting for each of
them.
12. Revise the Black Sea Convention, SAP and other instruments to strip away the role
and responsibilities of Regional Activity Centres (RACs). Two of the six named
RACs no longer function. The RACs were selected through a closed door process
without competitive bidding and with politics more then expertise playing the
deciding role on the designated activity for each country. In the future, the BSC
and country decisions on granting contracts for carrying out research should be
done on a case by case basis, using standard Terms of Reference and Requests for
Proposals.
13. It is assumed based on discussions with UNDP and GEF officials that no further
support will be provided, at least in the near term, and on a region-wide basis, to
the Black Sea. This is appropriate. After 16 years, its time to call a hiatus to GEF
support. In particular, it will be important for the Black Sea Countries to show a
good faith effort in achieving their SAP and other policy targets as a precondition
for further international funding.
There may be opportunities at the national level, however, to continue providing
support on integrated coastal zone management, especially as a planning tool for
climate change adaptation. There should also be opportunities at the national and
local level for the application of small grants programs, in particular to capitalize on
the previous efforts of NGOs in the region to expand public awareness and promote
the Black Sea Day. Small grants, coupled with the Coca Cola ,,Every Drop Matters
support frame, could have a real impact and greatly help the BSC.
It is also possible that negotiations within the BSC will provide a seat for the
European Union, which should stimulate additional EU financial support. There is
strong interest at the European Commission to see the BSC strengthened and
successfully coordinating regional marine policy for the Black Sea region.
14. Phase 1 of BSERP generated some modelling studies on air deposition, however not
narrowly focused on nitrogen loading. Based on a decision of the ISG, the BSERP
re-directed funding from a Phase 2 atmospheric deposition study to other activities
because the available funding was considered insufficient for delivering a robust
study with monitoring data collection. This is a subject that begs for further review
as scientists have suggested that air born dispersal of nutrients is a major
contributor to Black Sea eutrophication. At its core, this is an issue relating to the
application of fertilisers and manure for agriculture, as well as the combustion of
fossil fuels. The subject links closely with non-point source (NPS) runoff, another

44

---

subject that received some attention under the BSERP but where much more work
is required.

While further studies can help determine the extent of air born and NPS problems,
achieving stress reduction outcomes will take project assistance aimed at farm
management, such as training farmers in best agricultural practices, generating
investment and loan finance for environmentally-beneficial farm investments,
helping to establish marketing opportunities for organic farming and designing
projects that can utilise green house gas credit trading markets. The agriculture
sector in the Black Sea countries is slowly rebounding. Without changes in farm
management practices, the eutrophication problems of the previous decade will
surely return in the coming years
3.5.2 UNDP/GEF
15. Monitoring and evaluation methodology for full size projects should be reviewed in
light of the Black Sea experience. In particular, guidance on the timing of mid term
evaluations should be met. The BSERP Phase 2 was approved prior to conducting
the MTE, which in the future should be avoided. Recommendations included in the
(belated) mid term included suggestions to plan for closer monitoring of projects
during their first year, which should now be possible with the expanded UNDP/GEF
regional advisory setup. Consideration should also be given to requiring mid term
and final reports by the project teams, forming the basis upon which the external
evaluators do their review. The Black Sea PIU made a very good effort on project
reporting ­ with CDs developed at mid term and final as well as a final summary of
achievements.
16. PIU reports at the mid term and final stages should include an accounting from the
project partners of their in-kind and cash contributions to date. Otherwise, it is
exceedingly difficult for the evaluation team to determine whether promised
contributions were achieved or even exceeded. The track record on tabulating
partner contributions has been significantly better when the projects are being
considered for a continuation grant from GEF. The IMO/UNDP/GEF GloBallast
project, for example, tabulated partner contributions and could then show a
significant catalytic impact. The BSERP and Danube projects, in contrast, were
unable to formulate these figures. To create accurate figures for in-kind
contributions, this would need to be done on an annual basis. This should be
considered for inclusion in the APR/PIR reporting procedures.
17. The TDA/SAP development process provides a strong basis for assessing pressures
and measures, to protect water resources. Its emphasis in GEF IW projects may
need to be reconsidered, in particular to reduce the time and cost of development.
A preliminary TDA should be part of the ProDoc development exercise, with a SAP
already outlined as part of the project objectives. A revised and expanded TDA and
SAP can then form part of the mid term deliverables, with another set of revisions
forming the final project deliverables. In this way, the SAP can avoid being
construed as a one-off negotiated agreement, instead serving as a working plan for
cooperative efforts to improve environmental status (SAP) based upon a
periodically modified analysis (TDA).
18. The prolonged friction between the PIU and Secretariat, lasting throughout the
BSERP Phase 1 and part of Phase 2, created a serious drag on project achievement.
Had the two teams worked in greater harmony, and had the countries worked more
diligently to achieve agreed project objectives, this could have been a highly
successful project. Anticipating this kind of problem will be critical for the success
of future Commission support projects. During the project design phase there
needs to be consideration given to whether the main focus is on the resource, or
the regional body (Commission) set up to protect that resource. Will success be
measured by the number of outputs completed? Or whether the effort has helped
to ensure the sustainability and effectiveness of the Commission/Secretariat. This
decision needs to be clarified upfront and stated clearly in the ProDoc. In addition,
Memorandums of Agreement between the parties should be considered. In this
case, there would have been an MOA between the BSC and UNDP spelling out the

45

---

responsibilities of the BSERP PIU and the particular support mechanisms that would
be provided to the Secretariat. There would also have been MOAs signed with the
responsible ministries in each government on their specific in kind and financial
contributions. . .
19. Interministerial coordination is very important for furthering the aims of integrated
water resources management and coastal zone management. The difficulty is that
GEF IW projects are typically managed through the environmental/water/natural
resources ministries with little involvement of other ministries. The Black Sea
experience showed the difficulty in getting interministerial involvement. Future
projects need to include interministerial coordination as part of the Project
Document expectations, built from actual planned outputs and activities. For
instance, a planned output to negotiate a fisheries protocol needs to include the
engagement of ministries responsible for fisheries. The key is to get these
commitments prior to project approval.
20. Research cruises can be time and money intensive, the negotiations preceding
them on where to study are often arduous and saddled with geo-political baggage,
the scientific reporting is often slow, the outcomes are often of marginal use in TDA
development, and there is rarely enough budget to edit, summarise, translate and
publish the materials for a wider audience. So, why carry on these exercises? The
answer is that when managed well, they provide a useful addition to the project
effort. The two main aims should be: a) to develop as informative source of
knowledge in areas where historical environmental monitoring has been weak, and
b) to enhance communications strategies by providing another platform for media,
public awareness and education.
3.5.3 Ratings
The evaluation team has rated various criteria from the project, based on a four step
system: highly satisfactory, satisfactory, marginally satisfactory and unsatisfactory. The
ratings set out below are necessarily subjective, yet based on a thorough review of project
achievements, taking into consideration the comments of persons interviewed, an
understanding of the challenges of forging consensus on environmental protection in the
region and also considering the achievements of other GEF/IW projects. It is important to
emphasize that the ratings reflect the total sum of: project design, PIU performance, plus
efforts of the partnering countries and the project steering committee/Black Sea
Commission.
A rating of satisfactory has been determined for the outcomes and achievement of
objectives. On the one hand, the project succeeded to advance an improved
understanding of the status and trends in Black Sea ecosystem health, and the
establishment/re-establishment of linkages amongst the regions scientific community.
The project also delivered an improved Transboundary Diagnostic Analysis and a draft
revised SAP. On the other hand, wider project objectives including policy reforms and
improved collaboration amongst the Black Sea coastal country governments to deal with
shared problems in fisheries, land based sources of pollution, and coastal biodiversity
protection/planning, did not advance as far as expected during the project period and will
face continuing difficulties to achieve in the future.
A rating of marginally satisfactory is indicated for the implementation approach. The
implementation approach refers both to how the project was conceived and then how it
was managed ­ and the adaptive management that occurred. Some of the problems
experienced during project implementation had their germination in the project
formulation phase, resulting from an overly ambitious ToR, under-resourcing to meet the
ToR and a failure to agree with the Black Sea Commission on project priorities. The PIU
operated under difficult circumstances, in particular, the strained working relationship
between the Project team and Secretariat through the beginning and middle period of the
project was a critical problem that took several years to resolve. Staff turnover was clearly
a factor, especially during the BSERP 1st phase but also continuing through the second.
There was a strong rebound in the later stages of the project in terms of PIU/Secretariat
cooperation and the achievement of outputs. Stakeholders gave high marks for the

46

---

management of the small grants program, and the Black Sea Day campaigns (especially
2006) as well as the management of various seminars and workshops.
Stakeholder participation outcomes were satisfactorily achieved. There was room for
improvement in terms of developing and implementing a communications strategy from
project start up, and the educational and public awareness efforts, while innovative (i.e.
shell palace), lacked planning and follow through. The Black Sea Day efforts, especially in
2006 and 2007, were quite successful, and the TV documentary well conceived. The
broadening of NGO participation went well, and the small grants programme was a
success.
A rating of marginally satisfactory is indicated for project sustainability. On the positive
side, the project team succeeded in producing outputs that can help to promote long-term
regional cooperation for ecosystem protection and to forward the objectives of the
Bucharest Convention. The project team produced a revised TDA and SAP, LBA protocol,
legally binding document for fisheries, and ICZM strategies; they also endeavoured to
improve the capacity of the Black Sea Commission and its subsidiary bodies. On the
negative side, these outputs have so far had negligible impact on national policy setting
amongst the Black Sea countries. Budgets are not sufficient for effectively managing the
BSC Secretariat. Protocols are not being ratified. The (current) SAP targets keep getting
shifted back rather than achieved. There has been little progress made on fisheries
management. Other than the two EU countries, the rest of the Black Sea countries have
made only minimal progress in passing legislation and implementing programmes for
integrated coastal zone management. The October 2008 Black Sea Commission annual
meeting will provide a litmus test for the commitment of the contracting parties and the
sustainability of this partnership.
Monitoring and evaluation was marginally satisfactory. As noted, the project was
pushed through to a second phase prior to its mid term review and despite serious
concerns at the GEF over its first phase accomplishments. The (revised) project team
responded well to the MTE and Danube/Black Sea stock taking recommendations and the
Black Sea Commission also reconfigured its Secretariat, and starting in late 2005, the
project was back on track.


47

---