Final Evaluation Report
on the UNDP/GEF Project
"Development and Implementation of the Lake Peipsi /
Chudskoe Basin Management Programme"
RER/02/G35
April 2006
Evaluator:
Lisa Supeno, MSc.
Contents
ABBREVIATIONS
3
ABSTRACT
4
EXECUTIVE SUMMARY
4
1.
INTRODUCTION
14
1.1
Project Background
14
1.2
Purpose of the evaluation
14
1.3
Key issues addressed
14
1.4.
Methodology of the evaluation
15
1.5
Outputs of the final evaluation
16
2.
THE PROJECT AND ITS DEVELOPMENT CONTEXT
17
3.
FINDINGS AND CONCLUSIONS
18
3.1
Introduction
18
3.2
Key Findings
18
3.3.
Some conclusions and recommendations
19
4.
LESSONS LEARNED
27
5.
RECOMMENDATIONS
29
6.
ANNEXES:
31
Annex 1
32
Annex 2
38
Annex 3
41
Annex 4
43
Annex 5
44
Annex 6
45
Annex 7
56
Abbreviations
APR
Annual project report
CB
Capacity building
CBA
Cost and benefit analysis
CBC
Cross border cooperation
EC
European Commission
EU
European Union
GEF
Global Environment Facility
IMS
Information Management Systems
IWP
International water project
JWC
Joint Water Commission
LBMP
Lake Basin Management Programme
LFA
Logical framework approach
M&E
Monitoring and evaluation
N/A
Not available
NGO
Nongovernmental organizations
OVI
Objectively verifiable indicator
PCM
Project cycle management
PHARE
EC aid programme for Central and Eastern Europe
PIT
Project implementation team
PIU
Project Implementation Unit
PM
Project manager
PPP
Public Participation Plan
PPR
Project progress report
SC
Steering Committee
SMART
Specific, measurable, achievable, relevant, time-limited
SME
Small and medium enterprises
SWOT
Analysis of strengths, weaknesses, opportunities and threats
TACIS
Technical Assistance for Commonwealth of Independent States
TDA
Transboundary diagnosis analysis
ToR
Terms of Reference
UNDP
United Nations Development Programme
USD
U.S. dollars
WG
Working group
Abstract
The scope of this report is the final evaluation of the UNDP-GEF project "Development and Implementation of the Lake
Peipsi/Chudskoe Basin Management Programme" as pre-defined by the terms of reference (ToR). The report contains the
results of the evaluation process including the desk studies, interviews, site visits and mid-term evaluation results. It
identifies the key findings, conclusions and recommendations. The evaluation's findings could be used as an overall input
for the improved design of the UNDP/GEF future programs/projects, and as an interim input for the on-going similar
projects. The report was prepared by request of the contracting agency, Peipsi Center for Transboundary Cooperation, and
also will be used by the UNDP/GEF as well as other stakeholders, beneficiaries indirectly.
EXECUTIVE SUMMARY
Project background
A well-known fact is that 50% of the earth's land surface made up of transboundary basins, and 70% of the total surface
oceans, the majority of the world's water resources must be managed internationally as the transboundary ecosystems.
The cooperation in managing the transboundary waters is one of the four focal areas of the GEF activities. More than 50
international water projects have been implemented and/or are ongoing with the aid of the GEF funding and reflect the
request from more than 100 countries. Hereto, Estonia and Russia joined several international conventions for the
protection of the environment, i.e. the U.N. Convention on the Protection and Use of Transboundary Watercourses and
International Lakes (Water Convention, Helsinki 1992) and the Convention for the Protection of the Marine Environment
of the Baltic Sea (Helsinki 1992). The cooperation between Estonia and Russia started on the basis of the
intergovernmental agreement on the protection and sustainable use of transboundary water bodies signed in 1997. Same
year the Joint Estonian-Russian Transboundary Water Commission (JWC) was formed.
Implemented since January 2003, the overall objective of the project "Development and implementation of the Lake
Peipsi/Chudskoe Basin Management Programme" (LBMP) was to develop and start implementation of the Lake
Peipsi/Chudskoe Basin Management Programme that included the practical recommendations for the Lake
Peipsi/Chudskoe nutrient load reduction and prevention as well as the sustainable conservation of habitats and eco-
systems in the cross-border region. This served to provide an effective framework for prevention, control and reduction of
transboundary impacts on the lake's basin environment through the cross-border cooperation.
The project substituted the uncoordinated small-scale projects that could be otherwise implemented separately by the
Estonian and Russian parts without sufficient coordination, education and public information components with possible
omission of key interest of the local stakeholder groups and public at large.
The main expected outputs of this project were as follows:
Management Programme (Strategy Document) and Action Plan prepared and approved by all relevant
Estonian and Russian authorities
Strengthened Capacity of the key regional stakeholders including the environmental monitoring
infrastructure, data collection and maintenance system
Established networking and information exchange
Two demonstration projects implemented in Estonia: one for the eutrophication reduction through planning
water systems in small community, and the second for ecotourism and nature protection (ecological route).
The project was executed by the international NGO - Peipsi Center for Transboundary Cooperation and two Project
Implementation Units located in Tartu, Estonia and Pskov, Russia. In accordance with the ToR, the project was carried
out by the project partners: Ministry of Environment of Estonia, Ministry of Natural Resources of the Russian Federation
and also UNDP/GEF as GEF Implementing Agency. The overall project management was performed by the Project
Manager located in St. Petersburg office, Russia. For implementation of this project an NGO-execution mode was chosen.
The project stands out due to its cross-border context. Certain key elements of cooperation were established and
developed within the project framework and additionally, have long-term impact and strategic importance for the future
regional development. The cooperation dimensions entailed such issues as: (a) relations between two countries in
transition, (b) development of new EU border cooperation aspects, (c) factors of small and large state affairs, and others.
Several bi-national, regional and local agreements taken place, considerable efforts and time added to the general climate
of building trust between two riparian countries in different spheres of life. It resulted in the development of the cross-
border cooperation networks and institutional strengthening. Furthermore, it's worth mentioning the cooperation
developed among the local authorities, small businesses for environment and tourism, and NGO networks.
Although the sustainability is early to assess it is evident that the project's immediate results may have long-term impacts
in the coming years. A direct benefit of this project is the enhanced institutional structures developed within the project
that enable to address the existing and future transboundary environmental risks, eutrophication, groundwater pollution,
depletion of biodiversity resources including the lake's fish stock, other issues related to air, land and water pollution.
Context and Purpose of the Final Evaluation
Context
The intention of the final evaluation is to assess the relevance, performance and success of the project. Its results can be a
contribution to a continuous improvement in the projects/programs' design, performance and management of GEF
International Projects portfolio. Usually, the project's design, performance and management are assessed on the basis of
five established evaluation criteria, i.e. relevance, efficiency, effectiveness, impact and sustainability. The use of the
logframe with the objectively verifiable indicators (OVI) at the stage of a project's design and implementation contribute
to the quality of monitoring and evaluation (M&E).
In accordance with the UNDP/GEF policy for the M&E processes at the project level four main objectives are
encompassed, i.e. 1) monitoring and evaluation of results and impacts; 2) provision of a basis for decision making on
necessary amendments and improvements; 3) promotion of accountability for resource use; and 4) documentation,
feedback, and dissemination of lessons learned.
Purpose
The purpose of the final evaluation is to provide an overall assessment of the project and an opportunity critically to
assess administrative and technical strategies and issues. The evaluation will provide the recommendations to improve the
potential of future projects funded by the UNDP/GEF and EU.
The overall objective of this Final Evaluation to review progress towards the project's objectives and outcomes, assess
the efficiency and cost-effectiveness of how the project has moved towards its objectives and outcomes, identify strengths
and weaknesses in project design and implementation, and provide recommendations on design modifications that could
have increased the likelihood of success, and on specific actions that might be taken into consideration in designing future
projects of a related nature.
Some cross-cutting issues
·
Cross-border cooperation between Estonia and Russia, i.e. political commitment, institutional strengthening,
capacity building, public relations & public awareness, networking
·
Management & Operation of the Peipsi/Chudskoe Lake Basin Management Programme
·
Harmonized standards, norms and procedures
·
Technical support mechanism, e.g. IMS (information management systems), data base for networking and
information exchange, required hard and software, technology transfer
·
Socio-economic improvements in the local/regional communities of the Lake Basin.
Most cross-cutting issues were sufficiently addressed during the project implementation. It is apparent that the political
commitment of both parties has gradually progressed and very likely to result in imminent approval and signing of the
Lake Basin Management Programme in the nearest future.
Based on the concept "learning by doing" the stronger institutions and human resource development evolved, e.g. the
local authorities and NGOs built new partnership arrangements with roles and responsibilities, levels of coordination
among relevant players. In terms of the capacity building the UNDP/GEF project contributed to setting the ground for the
Management Programme development and implementation, e.g. a working version of the Management Programme as a
Strategy Document, Environmental Monitoring Programme, three joint monitoring expeditions conducted within the
project implementation, gained experience in working together at the JWC's level, the Peipsi Forum activities, Peipsi
Council concept, joint resource mobilization, etc.
Two new UNDP/GEF approaches for capacity development were also introduced, i.e. the public awareness campaigns
and dissemination of project results and information via the Web site. In Estonia through the area media, the awareness
campaigns for the Lake Peipsi and the environment became a first time big success. Moreover, the project served to build
the grass-roots networks and strengthen the institutional networks, e.g. local municipalities, local community
representatives, NGOs from lake Peipsi/Chudskoe region, SMEs.
Although the JWC may require some further strengthening and empowerment, the Peipsi CTC demonstrated successful
network operational modality, network sustainability, and establishment of international partnership linkages. The
introduced operational activities aimed at the four Es (a) exchange of best practices and lessons learned, (b)
enhancement of local capacity for the development and implementation of cross-border cooperation and lake basin
management, e.g. Lake Peipsi Programme, (c) expansion of NGOs for grass-roots cross-border cooperation and
transboundary basin management through replication of Peipsi CTC working models, and (d) establishment of new
partnerships with concerned stakeholders towards sustainable lake basin management.
The harmonization of standards, norms and procedures is still on-going process where there is a need of the agreement on
such issues as joint water quality monitoring, similar and comparable monitoring systems, use of agreed parameters and
timely data collection and sharing after validation.
Furthermore, the harmonization of the norms, standards and procedures as an instrument of the LBMP implementation
will ensure a sustainable use and protection of transboundary water resources through (i) maintenance and improvement
of the lake's water quality, (ii) sustainable use of transboundary waters, (iii) restoration of fish stock and sustainable use
of fish resources, (iv) protection of the lake's wildlife, and (v) joint management of the Lake's Basin via coordinated
activities, environmental monitoring, information exchange, additional research and studies, joint surveillance, public
participation, financing, etc.
For the Management Programme implementation the development and introduction of the IMS, database for networking
and information exchange, application of relevant hard and software, technology transfer are still required. The initial
basic installation of some components provides the general framework but further enhancement is needed.
Key Findings
3 out of 4 development objectives and outputs of the IWP were completely achieved, Objective I LBMP
approval and signing is near completion
Management processes were flexible, responding to conditions encountered during implementation, and were
appropriate in content and time
The level of cooperation and coordination among relevant players during project implementation was well-
established
The LBMP developed during project implementation has a good scientific basis but not fully clear on structured
definition of roles and responsibilities, presentation of targets and objectives, communications and coordination
mechanism definitions, allocated sources and resources needed, benchmarks and milestones since some of these
issues are already fixed under bilateral agreements or arrangements.
The relations and partnerships with other projects and donors were well-developed and proved to be resourceful
The projected level of public involvement in the Management Programme elaboration was appropriate although
in implementation phase it should be defined in more detail
The stakeholders' cooperation and efforts in project implementation proved to be successful
The project made an impact on the beneficiaries, e.g. improved capacity, strengthened institutions, network and
communications established, spin-off effect, although the project will likely have more impacts with the LBMP
implementation in medium and long-term
It's early to assess the sustainability though there is an indication proving that the project continuation is
necessary and likely occurs
The project brought good examples for replication in terms of best practices in capacity building, institutional
strengthening, grass roots network development, public awareness, building trust, establishing cross-border
cooperation, although the stronger commitment, openness, transparency and ongoing dissemination will be
required for the outcomes sustainability
Although M&E was a requirement for the project and conducted periodically but nascent in terms of joint
cooperation and timely data sharing. For future more emphasis on the M&E procedures should be given.
Results achieved
The project under evaluation had four main objectives that encompassed ten key activities (see Part IV, Work Plan, Annex
K of the Proposal/Agreement). Based on the desk study review of the project documents, consultations with Peipsi CTC,
UNDP, PIU's, field visits and interviews with key stakeholders, the qualitative assessment of the completion of each
objective has been made. Furthermore, the project manager (PM) and the project implementation team (PIT) also have
contributed to this assessment. Since the logframe was incomplete, the quantitative assessment became hardly feasible. It
is obvious that the implementation of the project on the Estonian and Russian side also differ on both local and regional
levels.
Although both sides had parallel project activities, the outputs on the Estonian side were higher than on the Russian side
for several reasons, i.e. difference in administrative capacities, restricted human and financial resources (allocated budget
for the Estonian side was much more than for the Russian counterpart), data availability and reliability, decision-making
procedures and pace, general communication practices. Nevertheless, the PIT did its outmost in order to manage the
identified and appearing risks while observing the situation critically, using the results of mid-term evaluation and others.
All these enabled to complete the project in due time and within the allocated budget, activate the JWC pace of operation
and involve other officials in process of consensus building, contribution to the LBMP development and approval.
The obtained results represent the following:
Objective I Management Programme The objective included five activities with such deliverables as (i) the
draft Lake Basin Management Programme and Action Plan, (ii) Monitoring Programme, (iii) Transboundary
Diagnosis Analysis (TDA), (iv) Nutrient Reduction Plan, (v) Public Participation Plan for the Basin Management
Programme. In general, the objective was achieved and all tasks completed although with some delays caused by
the external factors. The level of completion could be considered within the range of 85-90 % based on the expert
assessment and the stakeholders' estimation.
The draft of LBMP was completed only in the 4th Quarter of 2005 that left no time for its partial implementation.
However, some project activities greatly contributed to the agreed actions under the Programme (for example,
joint monitoring etc.). The Nutrient Reduction Plan started as scheduled but also experienced the delay due to the
extension of the two satellite projects, i.e. the EC Life Environment Viru-Peipsi CAMP project (Estonia) till
September 2005 and the EU Tacis CBC Baltic Line 2000 project "Environmental Management of
Lake
Chudskoe" till the end of 2005. Both projects are further referred as LIFE project (Estonia) and the EC Tacis
project (Russia). The UNDP/GEF project expected significant inputs from those two satellite projects both for the
TDA, Monitoring Programme and the LBMP. The delays had a serious impact on the timeliness and quality of
this component. Despite the earlier signs and preconditions, this risk was underestimated even after the mid-term
evaluation with no contingency plan set.
The TDA was prepared by Akvaplan- NIVA A/S in cooperation with other institutions from Estonia and Russia in
February 2005. Since the satellite projects were still on-going it brought certain limitations to apply to full extent
the final data, findings and recommendations drawn by those projects and relevant to the UNDP/GEF project.
Whilst preparing the TDA there was still an uncertainty related to the pollution sources and loads from the
Russian part of the Lake Peipsi due to the delay with the EU Tacis project implementation.
The draft of Monitoring Programme was developed in the 3rd Quarter of 2004. There were three joint expeditions
carried out in 2003 and 2004. Furthermore, previous joint monitoring results were used for data analysis. The
Draft Monitoring Programme was sent to the officials for the comments and feedback in order to improve the
quality of monitoring. The results are still pending. To summarize, a lot of work was done to reach this objective.
Nevertheless, the official approval of both the LBMP and Monitoring Programme and as well as their partial
implementation did not occur during the project life span.
Two feasibility studies (on eco-farming and eco-tourism) were completed in 2005 on both sides of the lake in
order to evaluate potential development in this field and possible negative effects on water quality.
For Estonia, the eco-farming feasibility study was also supported by an introductory level of farmers' training on
environment and pollution issues. Different sites for eco-farming were assessed and identified for crop potential.
On the Russian side, the eco-farming pilot project encompassing an assessment of impact made by agriculture on
the Lake Peipsi, and eco-farming concept development could be considered fully completed. The research
included the analysis of the available sites, current utilization of the agricultural lands, recent developments in
plants cultivation and livestock breeding, fertilizers use, assessment of forest reserves and others.
Under the eco-tourism study, some cities, towns, and areas were assessed and prioritized for potential eco-tourism
development based on the local surveys. The concept of eco-tourism development was based on eight
mainstreams of tourism development. In fact, this concept served as the basis of the overall Lake Peipsi eco-
tourism development plan to be developed and implemented under the LBMP.
Objective II Strengthening Capacity This objective included four main activities focused on institutional
strengthening of the Joint Water Commission, the Lake Peipsi Basin Authorities (National & Regional
Environmental Agencies), NGOs, local authorities and stakeholder groups.
This component was an important building block that should ensure a successful implementation of the Lake
Basin Management Programme. The level of completion could be considered within the range of 90-95 % based
on the expert assessment and the stakeholders' estimation. Some activities like a Peipsi Council/Forum or Peipsi
Museum in Russia have not been completed to full extent by December 2005 though the wrap-up is expected in
the 1st Quarter of 2006.
Nevertheless, further demonstration of the success ratio on the Estonia side for project scope and replication is
reflected through the synergy of its activities. For instance, many large-scale projects were initiated in Lake Peipsi
region through using outcomes of the project capacity building and strengthening (such as trainings, joint
seminars, roundtables etc. for discussing the needs and possible tools for implementation), e.g. INTERREG IIIB
BIRD project with a budget of 4,0 M , INTERREG IIIB TRABANT project with budget of 1,2 M , BEN,
PHARE CBC Waterway, PHARE CBC Community Philanthropy to highlight some spin-off activities. Thus, the
UNDP/GEF Lake Peipsi Project directly and indirectly resulted in new projects, and strengthened contacts among
NGOS, local authorities, and stakeholder groups.
·
Objective III Networking & Information Exchange This objective was completely achieved. The Web site
was designed and operational. The dissemination of project information to general public, press and other
stakeholders is still on-going process . Project partners also expressed the will to distribute main project results
via their own web-sites. The level of completion could be considered as 100 % based on the expert assessment
and the stakeholders' estimation derived from the interviews carried out during the evaluation site-visits.
Additionally, the UNDP/GEF project supported the development of a robust multi-stakeholder network in the
transboundary region that resulted in multiple cross-border cooperation projects including, for instance,
TACIS/INTERREG project on sustainable tourism executed by Tartu County Union of Local Authorities, a bike
tour project around Peipsi conducted regularly by the Russian and Estonian NGOs, environmental education
projects such as on volunteer monitoring, etc
·
Objective IV Two Demonstration Projects The pilot eco-tourism projects started as planned and considered
to be fully completed by December 2005. An average level of completion could be considered 100 % based on
the expert assessment and the stakeholders' estimation.
Infrastructure pilot project: "Development plan for water supply and waste water treatment systems for Rapina
rural municipality" was also fully completed even early than planned due to the wish of local stakeholders to ger a
Plan early in order to be eligible for the coming EU investments.
The Lake Peipsi Museum has a small facility in Kallaste, new light board exhibit and a virtual museum on the
Internet in Pskov.
Conclusion
In accordance with the terms of reference, the final evaluation mostly focused on three main project components: 1)
project delivery, 2) implementation, and 3) project finances. Each project component was evaluated on the basis of 3
established criteria, i.e. effectiveness, efficiency, and timeliness.
The final evaluation was held within the period of January 22 through February 10, 2006 (Annex 2) by the international
expert Ms. Lisa Supeno, MSc. who undertook the field missions to Estonia and Russia. During the field mission from
January 22 until February 4, 2006 the evaluation expert reviewed the key project documents, conducted the fact-finding
and interviews with all partied involved in the project. During this period, the expert also visited Peipsi CTC (host) in
Tartu, Estonia, stakeholders' offices in Tallinn, and the project office in St. Petersburg. A few telephone interviews were
organized and managed, .i.e. the UNDP/GEF office in Moscow, PIU office in Pskov, Tacis project manager in Finland.
To summarize the results of this evaluation, the project is scored for each criterion on a five point scale, with 1 on the
scale represents a highly satisfactory project, 2 satisfactory, 3 marginally satisfactory, 4 unsatisfactory, and 5 N/A,
lowest score. More details on the final evaluation are presented in Annex 7. Project Information Evaluation Sheet
Main Components
Effectiveness Efficiency Timeliness
Overall Score
Project Delivery
2.1
2.2
2.1
2.1
Project Implementation
2
1.9
2
2.0
Project Finances
2
2
2
2.0
Overall Score (average)
2.0
2.0
2.0
2.0
Overall, the UNDP/GEF project `Development and Implementation of the Lake Peipsi / Chudskoe Basin Management
Programme' is evaluated as satisfactory despite some difficulties mostly coming from:
(a)
A time lag between programming and actual implementation, e.g. the programme dated 1996-1997, the
project started in January 2003 that means that the project cycle was long and the budget became less realistic
in both currency and initially allocated amount. This issue is already under consideration by UNDP/GEF and
potential improvements are at their designation. It is obvious that for future UNDP/GEF projects there could
be shorter term between programming and implementation if possible.
(b)
Dependence of the UNDP/GEF project on the inputs from other two EU funded satellite projects
(c)
Delays in implementation of some project components due to external factors, e.g. extension of the satellite
projects' duration and late presentation of the required data to the UNDP/GEF project
(d)
Different priorities on political, economic, environmental, cultural and technical issues with regard to
transboundary cooperation in both countries whereby the coordinated measures can be made
(e)
Slow pace in reaching a joint agreement on the key issues like the monitoring systems, methods, tests,
sampling, etc. experienced by both countries
(f)
Staff changes on either side of the Joint Water Commission, e.g. changes or replacements
(g)
Differences in decision-making processes on both sides of the JWC
(h)
Complexity of some tasks and some ambitious expectations regarding the outcomes for the Objective I, e.g.
the LBMP development, approval and signing, agreement on monitoring programme, etc.
(i)
Some internal factors like too flexible project planning, non-structured project communication scheme, lack
of operational logframe and its update, use of M&E could be better and more efficient as a management tool
Hereby, the score indicates the results of the final evaluation in terms of the project performance, i.e. delivery,
implementation and finances. Most tasks and objectives were timely and well accomplished, the others experienced some
delays and changes in scope of work, e.g. less available open tenders/contracts as originally planned, due to the financial
reasons caused by considerable currency exchange fluctuation and other external factors. Among them also were the
cross-border cooperation issues, timely deliverables from the EU LIFE and EU Tacis projects. Most those factors were
mentioned earlier in the Proposal/Agreement (see Annex G. Risks, Prior Obligations and Sustainability) although they
were not properly elaborated and addressed during the project implementation. Thus, more intensive use and revision of
the operational logframe, updated risk management program, and results of M&E could have prevented the experienced
problems and delays as well brought the timely and efficient intervention in the project process.
On the one hand, the project team controlled the internal factors, i.e. project administration, finances, local/regional
partnership, reporting, etc. On the other hand, some external factors, e.g. the level of cross-border cooperation,
international/national partnership, the satellite projects' progress and deliverables, and others were outside of the PIT's
control.
The project team implemented the project had appropriate experience and expertise in the required fields.
Drawn from the interviews conducted in St. Petersburg and Tallinn, the modality of the NGO led project was considered
highly successful and replicable for other GEF projects. The summary of conclusions, recommendations and lessons
learned is presented below.
Conclusions
Recommendations
Lessons Learned
1
2
3
The project was executed satisfactorily. As an
Expand the `best practices' for other
Despite different hurdles during the
NGO-run project the modality proved
projects in regions/countries context.
execution the project gain of improved
successful for UNDP/GEF.
Notably with transboundary context and
cooperation, better understanding of current
multi-task issues and many stakeholders
situation in the Lake Basin area,
levels.
willingness to work together on prescribed
solutions, and certain level of commitment
to the overall objective.
3 out of 4 development objectives and outputs
Accelerate the process of negotiations and
Developing cross-border cooperation,
of the project were achieved, Objective I
discussions aimed to shorten the time
building trust, establishing a shared vision
LBMP approval and signing is near
between the LBMP agreement and signing,
is a process. The timeframe for
completion.
commence the implementation.
implementation and unforeseen bottlenecks
should not be underestimated in future
activities.
Lake Basin Management Programme and
After all comments received, it is advisable
Pre-conditions of risk management are to
Monitoring Programme developed in situ
that the documents both contents and
be properly considered at the early stage.
committee and discussed by stakeholders, yet
quality-wise are corresponding adjusted
Proper SWOT analysis is required at the
finalization is still pending due to the official
and/or corrected, and whereby the JWC
programming stage in order to avoid any
feedback and approval.
revisit the discussions aimed at final
political, social or economic complications
The level of cooperation & coordination
approval of both documents.
during the implementation phase.
among relevant players during project
Review the content of LBMP, structure it in Based on the existing experience gained
implementation was well established,
terms of objective(s), targets, the roles,
through other cross-border water projects,
although the structured definition of roles and
responsibilities and coordination
there is a number of good examples for the
responsibilities, coordination mechanism was
mechanisms, ,benchmarks &milestones,
Management Programme development.
not clearly mapped in the LBMP since it is
required or/and available resources.
partially covered by other documents.
Incorporate the adjustments in the final
version of LBMP.
The TDA and other prepared documents
Set the timeframe for the LBMP discussion, The first steps were already made, common
sufficiently contributed to the LBMP drafting.
corresponding adjustments and signature.
understanding established. A faster pace in
In general, the LBMP clearly reflects the main Priorities could be the harmonized norms,
the decision-making process is required.
issues in environmental, economic, financial,
procedures as well as agreed national
social and political context. The LBMP is
recourses allocation.
feasible for implementation after official
approval by both countries.
The relations and partnerships with other
Further dissemination of best practices on
The well-built level of cooperation ensured
projects and donors were well developed, and
broader scale, e.g. regional, international,
the expected outputs and outcomes being
proved to be resourceful.
conferences, forums, workshops and
more reliable and risk level of the project
trainings.
was thereby satisfactorily lowered.
Institutional strengthening was conducted in
When the LBMP is approved and final
Initial steps to cooperate, plan and
due time with visible outcomes. Although the
decision of responsible agencies for
implement took place. However, the
JWC was founded and operational it still
implementation made, the JWC
modality to operate successfully and
required more empowerment and equal
composition must be reviewed, adjusted,
efficiently is still to be developed.
resource allocation from both sides.
the status bestowed and the required
Exchange programs from other similar joint
resource allocated. JWC could also learn
water committees enrich the knowledge
more from the experience of other
and experience and contribute to the
international water committees.
networking and cooperation.
The project brought good examples for
Create for inter-agency coordination
The best practices developed during the
replication in terms of best practices in
documental and verifiable activities that
UNDP/GEF project implementation
capacity building, institutional strengthening,
would be coordinated with the past,
already have spin-offs and evolved into
grass roots network development, public
ongoing and prospective work of the
multi-country, multi-stakeholder projects
awareness, building trust, establishing cross-
Implementing Agencies and other relevant
replication in the near future.
border cooperation, although the stronger
authorities and stakeholders. These will
commitment, openness, transparency and
include experience gained, lessons learned,
ongoing dissemination will be required for
dissemination based on the project
IWP outcomes sustainability.
activities. The know-how will comprise the
experience of multilateral, bilateral, and
private institutions, international & national
NGO, regional/ national research centers
and academic organizations, municipal
authorities, volunteer network, other
relevant partners.
Other issues related to capacity building show
A system of continuous training and public
Project has commenced identifying and
still meager on the respective institutional
awareness building should progress.
involvement of champions (individuals in
capacities while stronger on the grass roots
Planned activities should continue to
government and civil society who could
sustainability side. On the community level,
exceed the expected outcome of inputs,
promote sustainability of project
good work was performed for raising
costs, budgets and deliverables. Obvious
outcomes). However, there is a concern. If
awareness, training, and information
that without the governmental funds such
the key documents are not approved soon,
dissemination. At mid-level of local &
programs are hardly feasible. The
the impact of these players and momentum
regional authorities, there is understanding
willingness and open-mindedness of the
could be lost. It means that the level of the
and willingness (systems, structures, staff, and populace with new sights and a vision of
built capacity so far, now needs more
expertise) yet full commitment is yet partial.
overall community improvement should not expansion and practical utilization.
Generating funds regionally and locally can
be overlooked. Future incomes can be
actually transpire with short-medium term
generated while preventing the pollution.
pilot project implementations, for example.
This also can be an objectively verifiable
That will certainly add to the sustainability
indicator of progress for both Russian and
On the national level, absorption capacity is
Estonian sides.
there, however, needs further development i.e.
market transforming, better economic,
Use project-trained individuals, institutions
The project results is in the `best practices'
financial instruments, and human, social
or companies to replicate the project's
category and bears replication locally,
capital utilization.
outcomes in other regions.
regionally, nationally and internationally.
The projected level of public involvement in
The revised public participation plan (PPP)
The PPP was recently revised and
the LBMP elaboration was appropriate
must be in incorporated in LBMP in more
contained all activities relevant for the
although in implementation phase it should be
detailed than a general descriptive way.
LBMP implementation.
defined in more detail.
The networking and information exchange
Somehow, the relations between different
It was a very extensive networking and
was sufficiently established. It built
stakeholders should be kept active through
information exchange and took on its own
partnerships among different project
the seminars, workshops, etc. before the
qualities of reliability, operational
stakeholders and proved operational and
LBMP implementation starts. The
effectiveness and accountability due to
ongoing. The Web site and intranet together
established network should be sustained
quality management. It mostly reached the
with printed materials facilitated the
through participatory activities rather than
project objective that gave the added value-
communications and fast information
abandoned. In the future, the JWC and
to all resulting communications.
exchange. The future of the project website
other relevant stakeholders should decide
was discussed at the SC and the partners
what website is to be used or created for the
agreed to place the results and other relevant
LBPM implementation.
information on some official web pages in
order to preserve the data for the LBMP
implementation
Demonstration projects on eco-tourism and
Preparation of the inventory of
With the LBMP implementation, more
infrastructure development in both Estonia
local/regional skills, sites for eco-tourism,
emphasis is to put on promotion activities
and Russia resulted in concrete actions which
use of volunteers in the planned activities.
in order to activate and/or bring the
contribute to water protection activities and
Establishment of contacts and conduct of
investors to the region.
bring additional income to the regions. Other
regular meetings with both the Western and
pilot projects could be initiated in the region
area entrepreneurs for tourism development
and include aquaculture industry growth or
including the infrastructure upgrade, e.g.
fish stocking, farming crop rotations, and
hotels, restaurants, road, etc.
others.
Overall project management is considered
Use and update the logframe during the
In addition to the initial project
satisfactory despite the external factors that
project life. M&E must be used as a
communication it's advisable to have a
caused the delays of some outcomes and
management tool and conducted on a
detailed and updated project
changes in scope of work.
regular basis following the contracting
communication scheme as a project
authorities' procedures and guidelines.
management tool. A combination of mixed
Although M&E was a requirement for the
tools, like LFA, and PCM enable to keep
project and conducted periodically but nascent There is still a room for improvement i.e.
project on track with necessary & timely
in terms of feedback and timely data sharing.
introduction of Information Management
intervention. Use of these tools enables to
In general, management processes were
Systems (IMS) where probabilities of
have a holistic vision of the project
flexible, responding to conditions and changes outcomes are regularly or cyclically timed,
developments.
encountered during implementation, were
late project deliverables are few, and
appropriate and timely.
product/project cycles are reliable and
Bringing together both sides, establishing
continuous with less variances or deviations the level of trust, developing level of
providing more confidence in successful
cooperation, maintaining regular meetings,
project results from all contributing
meant increased networking cooperation,
partners.
and communication of the JWC with other
relevant commissions, international,
regional organizations and should not be
underestimated. Therefore, it can become a
cornerstone for future concerted productive
activities.
Generally, project implementation is
Despite the interdependence evolved during Identification of potential sources of co-
considered satisfactory although the nature of
UNDP/GEF project implementation, it is
financing as well as leveraged and
the complex project and externalities has
advisable that in future projects with the
associated financing must be properly
proved to all stakeholders that shortcomings
involvement of different donors' projects,
balanced. The project was implemented
could impede project e.g. satellite projects'
the clear preconditions should be defined,
within the timeframe and allocated budget.
late inputs to UNDP /GEF project. The
combination of flat/vertical hierarchy
project completed the activities & met or
established, as well as official
exceeded the expected outcomes in terms of
commitments, conditions and deadlines
achievement of the GEF objectives according
determined.
to schedule, and as cost-effective as initially
planned.
Overall stakeholders' cooperation and efforts
Activities of stakeholders' cooperation as
Stakeholders' maintaining a presence
in project implementation proved to be
they relate to the project implementation
within the project as exhibited by a very
successful.
can be used as a means to compare and use
good level of cooperation were a role
to coordinate activities with other relevant
model for the rest of the project and can be
institutions and organizations when the
replicated for future projects.
LBMP starts.
Transboundary cooperation between two
With the developed transboundary
Cooperation patterns developed within this
countries in terms of political commitment,
cooperation there is always a room for
project could be an example for other
institutional strengthening, capacity building,
continuous improvement in the cooperation
regions, countries, encountering similar
public relations & public awareness,
mechanism. More joint activities on both
situations with small and bigger country
networking have all developed identity and
sides of the lake conducted by different
border relations, differences in language,
presence along with the Lake Basin as an
stakeholders in different sectors e.g.
institutional structures, legislation, lake
,
environmental focal area.
regular volunteer monitoring, Lake Day
management practices and developing
campaign, joint community projects
strategies, administration procedures,
cleanup, fundraising, fairs, etc.
standards and norms, etc.
The project made certain impacts on the
However, it's somewhat early to assess
The project had positive spin-offs resulted
beneficiaries, e.g. improved capacity,
impact now the overall objective will likely
in initiation of a number of new projects
strengthened institutions, established network
be reached in future. Positive impact on
inside and outside the region, funded by
& communications, spin-off effect, although
socio-economic and environmental issues is different donor agencies.
the project will likely have more impacts with
expected for both sides. More integration of
LBMP implementation in the medium and
socio-economic and environmental
long-term.
components in regional programs/projects
will ensure better impacts.
It is rather soon to estimate the sustainability
It is likely that the project will have a
The built capacity, institution strengthening
though there is an indication proving that the
continuation. Since the development and
and public awareness raising will likely
project will have a continuation
introduction of tailor made activities under
contribute to sustainability after the project
umbrella of this project and follow-up
completion and during the possible follow-
projects is necessary, more emphasis on the
up.
socio-economic advancement issues should
frame the future activities.
1.
Introduction
1.1
Project Background
The overall objective of the project "Development and implementation of the Lake Peipsi/Chudskoe Basin Management
Programme" was to develop and start implementation of the Lake Peipsi/Chudskoe Basin Management Programme that
included the practical recommendations for the Lake Peipsi/Chudskoe nutrient load reduction and prevention as well as
the sustainable conservation of habitats and eco-systems in the cross-border region.
The project substituted the uncoordinated small-scale projects that would be otherwise implemented separately by the
Estonian and Russian parts without sufficient coordination, education and public information components as well as
possible omission of key interest of local stakeholder groups and public at large.
The main envisaged outputs of this project are as follows:
·
Management Programme ( Strategy Document) and Action Plan prepared and approved by all relevant Estonian
and Russian authorities
·
Strengthened Capacity of the key regional stakeholders including the environmental monitoring infrastructure,
data collection and maintenance system
·
Established networking and information exchange
·
Two demonstration projects implemented in Estonia: one for the eutrophication reduction through planning
water systems in small community, and the second for ecotourism and nature protection (ecological route).
The project was implemented by the international NGO - Peipsi Center for Transboundary Cooperation and two Project
Implementation Units located in Tartu, Estonia and Pskov, Russia. In accordance the ToR, the project was carried out by
the project partners: Ministry of Environment of Estonia, Ministry of Natural Resources of the Russian Federation and
also UNDP/GEF. The overall project management was performed by the Project Manager located in St. Petersburg office,
Russia. For implementation of this project an NGO-execution mode was chosen.
1.2
Purpose of the evaluation
The final or terminal evaluation is intended to assess the project achievement of objectives and impacts as well as provide
an objective estimate of administrative and technical strategies, issues and constrains associated with large international
and multi-partner initiative. The evaluation will provide the recommendations to improve the potential of future projects
funded by the UNDP/GEF and EU. Finally, the evaluation has one more objective, i.e. a continuous improvement in the
project/programme's programming and design, performance and management.
The overall objective of this Final Evaluation to review progress towards the project's objectives and outcomes, assess
the efficiency and cost-effectiveness of how the project has moved towards its objectives and outcomes, identify strengths
and weaknesses in project design and implementation, and provide recommendations on design modifications that could
have increased the likelihood of success, and on specific actions that might be taken into consideration in designing future
projects of a related nature.
1.3
Key issues addressed
In line with the overall objectives, the following key issues were addressed during the final evaluation:
·
Assessment of the extent to which the project achieved its overall objective
·
Assessment of the extent to which the development objectives and outputs of the IWP were achieved
·
Description of the management processes, i.e. how the project activities changed in response to new conditions
encountered during implementation, and whether the changes were appropriate
·
Review of the clarity of roles and responsibilities of the various institutional arrangements for the Management
Programme's implementation and the level of coordination between relevant players
·
Review of any partnership arrangements with other donors and related projects, and comments on their strengths
and weaknesses
·
Assessment of the level of the public involvement in elaboration of the Management Programme and
recommendations on whether the public involvement was appropriate to the goals of the project
·
Description and assessment of the stakeholders' efforts to support of the project`s implementation
·
Review and evaluation of the extent to which the project had the impact on the intended beneficiaries
·
Assessment of the likelihood of continuation and sustainability of project outcomes and benefits after its
completion
·
Description of the key factors that require the attention in order to improve the prospects for sustainability of IWP
outcomes and the potential for a replication of the approach
·
Review of the implementation of the project monitoring and evaluation
·
Description of the main lessons that emerged in terms of:
o
country ownership/drive
o
regional cooperation and inter-governmental cooperation
o
stakeholder participation
o
adaptive management processes
o
efforts to secure sustainability; and
o
role of M&E in project implementation.
·
Distinction between the lessons applicable to this project and the ones that may be valuable in broad terms
including other, similar UNDP/GEF and EU environmental project initiatives.
1.4.
Methodology of the evaluation
The final evaluation was conducted by the international expert, Ms Lisa Supeno, MSc., selected on the base of open
tender.
The final evaluation was carried out as a combination of the activities such as the desk study, site visits and interviews
with selected stakeholders, e.g. Peipsi CTC, UNDP, Government officials of Estonia and Russia on different levels, local
municipalities, local NGOs, communities, etc.
The methodology for the final evaluation also covered the following areas:
Desk study review of all relevant project documentation
Assessment of the project's performance based on `indicators of success' available in the initial project
documentation
Consultations with Peipsi CTC, UNDP , PIUs and offices
Site visits to the Lake Peipsi and its adjacent areas, Tartu, St. Petersburg and Tallinn
Interviews (including phone interviews) with the key stakeholders
o
Ministry of Environment of Estonia and its sub-departments
o
Russian Ministry of Natural Resources and its territorial bodies
o
The Estonian-Russian Transboundary Water Commission
o
Local Municipalities
o
Local community representatives
o
NGOs from Lake Peipsi / Chudskoe region
o
Other projects in the Peipsi / Chudskoe region
Preparation of the draft and final evaluation report in compliance with Annex 1 of the ToR
Feedback from the PIT and stakeholders, incorporation of the remarks and comments in the final report
Presentation of the findings and conclusions of the final evaluation at the Steering Committee (SC) meeting held at
the end of February 2006.
The key issues addressed during the final evaluation were mentioned above in section 1.3. In accordance with the ToR
three main components were evaluated: 1) project delivery, 2) implementation, and 3) project finances. Each component
was evaluated on the basis of three criteria, i.e. effectiveness, efficiency and timeliness where:
Effectiveness means the extent to which the project's objective has been achieved or will likely be achieved
Efficiency will assess the outputs in relation to inputs whilst keeping in mind the costs, implementation timeframe,
economic and financial results.
Timeliness means the delivery/obtaining the inputs and results
Furthermore, the final evaluation included the ratings for three mentioned criteria. The summary of the evaluation's
results were presented for each criterion on a five point scale, with 1 on the scale presenting a highly satisfactory, 2
satisfactory, 3 marginally satisfactory, 4 unsatisfactory, and 5 N/A, lowest score. A proposed module for evaluation
results presentation was as follows:
Following the UNDP/GEF policy for the monitoring and evaluation (M&E) processes at the project level four main
objectives were encompassed, i.e. 1) monitoring and evaluation of results and impacts; 2) provision of a basis for decision
making on necessary amendments and improvements; 3) promotion of an accountability for resource use; and 4)
documentation, feedback, and dissemination of lessons learned. This mix of tools ensured an appropriate project
evaluation despite the lack of the logfarme and detailed monitoring reports.
1.5
Outputs of the final evaluation
Since the final evaluation is an assessment of the relevance, performance and achievements of the project, it also gives a
notification of potential impact and sustainability of the results including a contribution to the capacity development and
attainment of global environmental goals. Furthermore, it serves a dual function of instrument independent to project
process, yet also considered as an intrinsic part of project. The evaluation provides an accountability and transparency
while assessing the project's design, performance and management. The outcomes of the final evaluation can be used by
the Contracting Agency as well as the Executing Agency and project stakeholders for the current project analysis, similar
on-going projects and future project/programmes' design, implementation and management.
The assessment of the project's outcomes and outputs was conducted on the basis on the key project documents' review
(see Annex 5 of the Report) and the interviews (Annex 3 and 6) with the key stakeholders and the PIT. The evaluation
was based on the UNDP and GEF M&E rules, as well as the approved methodology elaborated and approved in the
inception report (see also sub-section 1.4 above). The results of the final evaluation of the UNDP/GEF project
"Development and implementation of the Lake Peipsi/Chudskoe Basin Management Programme" are presented in the
Table below.
Main Components
Effectiveness Efficiency Timeliness
Overall Score
Project Delivery
2.1
2.2
2.1
2.1
Project Implementation
2
1.9
2
2.0
Project Finances
2
2
2
2.0
Overall Score (average)
2.0
2.0
2.0
2.0
For more details of the project evaluation refer to the Annex 7 of this report.
2.
The project and its development context
Project
Context
Project start and its duration
January 2003, duration - 36 months
Implementation status
Completed since January 1, 2006
Problems that the project
Management Programme(Strategy Document) and Action
sought to address
Plan prepared and approved by all relevant Estonian and
Russian authorities
Strengthened Capacity of key regional stakeholders (including
environmental monitoring infrastructure and data collection and
maintenance system)
Networking and information exchange established
Two demonstration projects implemented in Estonia: one on
eutrophication reduction through planning water systems in
small community and the second one on ecotourism and nature
protection (ecological route).
Immediate and development
Strengthening of the regional/local capacity in order to
objectives of the project
implement the agreed and approved international Management
Programme for the protection and sustainable use of Lake Peipsi
/ Chudskoe.
Executing Agency
Peipsi Center for Transboundary Cooperation, International
NGO
Main stakeholders
Joint Estonian-Russian Transboundary Water Commission
Local and regional NGOs
Local and regional authorities
SME
Project partners
Ministry of Environment Estonia
Ministry of Natural Resources Russian Federation
UNDP, GEF
Results expected
Management Programme is agreed and approved by Estonia and
Russia, staff from responsible institutions/organizations well-
trained, enhanced stakeholders involvement in the Programme
implementation. Empowered JWC with sufficient human,
financial and other resources for the PM implementation.
3.
Findings and Conclusions
3.1
Introduction
The final evaluation of the project's performance was completed as described in Section 1 of this report. The report
follows the lay-out specified in the ToR, and covers three main components:
Project Delivery
Project Implementation
Project Finances
3.2
Key Findings
The final evaluation was undertaken for the UNDP/GEF project officially completed on December 31, 2005 with the
wrap-up in March 2006. The key findings, grouped according to the main components, were as follows:
Project Delivery
The project was completed within the defined timeframe and allocated budget
Most development objectives and outputs of the project were completely achieved except objective I LBMP
approval and signing is pending
Key environmental issues were clearly addressed in the TDA and reflected in the LBMP
The developed LBMP contains a solid scientific background but is less stronger in its structure in terms of
detailed definition of roles and responsibilities, communications and coordination mechanism definitions since
some of the issues are already covered by other agreements
The UNDP/GEF project is likely to make a sound impact on the situation in this transboundary water basin in the
coming years via supporting cooperation and sustainable water management in the basin
The projected level of public involvement in the LBMP elaboration was appropriate and during its
implementation phase it should be maintained as well
The project made an impact on the beneficiaries, e.g. improved capacity, strengthened institutions, network and
communications established, the project will likely have other impacts with the LBMP implementation in medium
and long-term
The sustainability is early to be assessed, though there is an indication proving that both countries demonstrate
their commitment in term of good political will, further cooperation and financing.
Project Implementation
Management processes were flexible, responding to the changes encountered during implementation, and can be
considered appropriate and timely
Transboundary cooperation was gradually established and proved to be viable though some key issues like the
agreement and approval of LBMP, Monitoring Programme and some other documents are still in process
Also the level of cooperation and coordination among the stakeholders was well established during the project
execution
The relations and partnerships with other projects and donors were well-developed and proved to be useful
The project brought good examples for replication with regard to best practices in capacity building, grass roots
network development, public awareness, building trust and establishing cross-border cooperation
M&E was a project requirement and conducted periodically, however, in the future more emphasis should be
given to this issue
Administrative and operational capacity to manage the project is sufficient
Coordination between the national agencies, Peipsi CTC, PIUs, UNDP/GEF was satisfactory
Project Finances
The project was completed within according to the allocated budget and generated the synergy with the EU funds
coming from two satellite projects
The time lag between the project programming and implementation was too long, it affected first of all the project
budget
3.3.
Some conclusions and recommendations
Hereby, the detailed summary of the conclusions and recommendations drawn from the desk studies, interviews and
consultations is presented below. It also encompasses the three project components under evaluation such as project
delivery, project implementation, and project finances on the basis of 3 established criteria, i.e. effectiveness, efficiency
and timeliness. The rating system represents a five point scale from 1 highly satisfactory, 2 satisfactory, 3 marginally
satisfactory, 4 unsatisfactory, and 5 N/A, lowest score.
CONCLUSSIONS
RECOMMENDATIONS
General. This project is rated as `satisfactory'
Following the UNDP/GEF guidelines, it is recommended
per the final evaluation. The delivery of the
to develop, use and update the logframe during the project
outputs was appropriate even with some delays,
implementation. That enables the Executing Agency to
but within the allocated budget.
interfere timely and efficiently in the project processes.
Timeline of requirements for the benchmark goals,
monitoring results and other key factors are needed for
appropriate measurability. Application of the SMART
principles can facilitate to project quality improvement
both in qualitative and quantitative contexts.
The project has mostly reached the immediate
The time between programming and implementation
objectives. It remains still relevant since the
should be much shorter. In addition, it is advisable to
forming in 1996-1997 although the allocated
involve the key stakeholders in project design on the early
budget seems less realistic due to the long
stage.
project cycle and considerable currency
exchange fluctuation.
Although the level of cooperation between both
Official approval of the LBMP and relevant agreement on
countries on transboundary issues has improved
the Monitoring Programme are needed. The strong role of
there are some concerns related to the official
the JWC in these issues is indispensable.
approval of the Lake Basin Management
In general, the LBMP clearly reflects the main issues in
Programme and Joint Monitoring Programme
environmental, economic, financial, social and political
by the Russian side (highest level of authority is
context. The LBMP is feasible for implementation after
unclear).
official approval by both countries
The TDA and other prepared documents
sufficiently contributed to the LBMP drafting.
The harmonization of standards, norms and
LBMP implementation should ensure a sustainable use
procedures is still on-going process where there
and protection of transboundary water resources through
is a need of the agreement on such issues as
(i) maintenance and improvement of the lake's water
joint water quality monitoring, similar and
quality, (ii) sustainable use of transboundary waters, (iii)
comparable monitoring systems, use of agreed
restoration of fish stock and sustainable use of fish
parameters and timely data collection and
resources, (iv) protection of the lake's wildlife, and (v)
sharing after validation. The project partially
joint management of the Lake's Basin via coordinated
addressed the issues, however, official
activities, environmental monitoring, information
agreement in this field under the Commission is
exchange, additional research and studies, joint
crucial.
surveillance, public participation, financing, etc.
Project Delivery. On average it is rated as
Proposed recommendations include a use of the PCM
satisfactory. Project Delivery includes the
principles with the LFA that ensure a good project design,
institutional arrangement, outcomes,
implementation as well as monitoring and evaluation.
partnership, risk management and M&E. The
The quality of the project performance is much higher
planning, most preparatory work and other
when the logframe is properly used and revised during the
strategies were built on the results of previous
whole project life. Moreover, it enables to draft a realistic
activities dated 1995-2000. The Work Plan and
Time Schedule or Work Plan, make adjustments or
Results Framework (see Part II, Results
corrections if needed, and have the timely implementation
Framework, and Part IV Legal Context. Work
of the activities.
Plan of Proposal/Agreement, 4/2002) were
correspondingly elaborated at the early stage.
During the project span, the plan experienced
the changes due to the external factors. The
Project Work Plan was not properly detailed and
presented graphically at the very beginning. The
split on the project components definitely helps
but when it is not detailed it is difficult to keep
track on all planned activities and follow up
their timely implementation. The lacking part is
the holistic view of the whole project for the
whole project life. Nevertheless, the project
planning was sufficient and satisfactory during
the implementation phase despite the
externalities that caused some hurdles and
delays with some activities.
Technical support was quite sufficient during
For future the recommendation is for quarterly financial
the project implementation although there were
statements per Moscow to the project rather than simply
some problems with the changes of the UNDP
annual reconciliations and this was agreed during this
financing management system. This was aided
project and carried out successfully.
by timely cooperation of Moscow UNDP/GEF
unit to facilitate absorption capacities
Institutional strengthening was conducted in due With the LBMP is approved and final decision of
time with visible outcomes. Although the JWC
responsible agencies for implementation made, the JWC
was founded and became operational, it still
composition must be reviewed, adjusted, the status
requires more empowerment and equal resource
bestowed and the required resource allocated. JWC could
allocation from both sides.
also learn more from the experience of other international
water committees.
Capacity building (CB) initiatives were
JWC as the designated body for the PM implementation
developed quite satisfactorily. Overall gain
should be more empowered and given more human,
through the whole project was active
financial and other resources. Contemporary water
involvement of the stakeholders and
commission bodies such as Black Sea, Danube or Rhine
development and strengthening of the grass
Commissions could be useful example.
roots. The JWC and new Peipsi Council/Forum
In the future, it can be recommended to add strengthening
are expected to play a key role in the LBMP
regional centers of excellence for further CB.
implementation. The project utilized both
traditional and newer approaches in CB, i.e.
traditional one - training workshops, technical
assistance, supporting scientific research, and
newer - multidisciplinary implementation teams
and steering committees, public meetings,
strengthening networks, info dissemination
through website, awareness campaigns. CB
resulted in public meetings, strengthening
networks, info dissemination through trainings
and awareness campaigns with some numbers:
64 seminars-workshops in Estonian side, dozens
on the Russian side. A number of booklets,
newsletters, manuals, postcards, leaflets.
Updated Web site with a high number of visits.
Mass media received press releases in every
event that was bigger than small workshop, for
the grassroots NGOs; several articles in
magazine/newspaper published with first time
ever environment full scale press attention;
radio/TV broadcasts in Estonia and Russia. The
small grant facility aspect of CB meant that with
the first call for proposals 12 were submitted
and all received money; a total of 78 proposals
were submitted during the whole period and out
of 78 proposals 40 were awarded with a total
overall budget of $5.8K USD for mostly
educational programmes
Although the project outputs were delivered, the It is recommended to have less dependence on the
delays caused by the external factors and
external inputs and factors through use of proper project
unaddressed risks took place. Due to the tight
design.
schedules prompted by delays both content and
The LBMP content could be better elaborated with main
quality of some outputs might suffer. Details,
focuses on, e.g.:
standards and executions consequently were
·
Targets and approach
somewhat lacking the key outputs, e.g. the
·
Ecosystem improvement
LBMP, PPP and TDA.
·
Water quality improvement
·
Groundwater protection
·
Instruments and public relations
·
Success control
·
Implementation and costs
·
Appointed responsible agencies
·
Planned funding
The Management Programme should not exceed 30 pages
for the decision-makers review and approval.
Assumption & risks were preliminary mentioned Proposed recommendations include: more independence
in the Proposal/Agreement (see Annex G of the
of the projects from potential satellite projects in terms of
original Agreement). Out of 10 line items, only
inputs and deliverables. On horizontal level the problems
one was somewhat higher with 9 assessed as
happened due to the flat hierarchy of Project Management
low risks. Several key or critical risks were
Structure.
initially overlooked, e.g. delays related to the
satellite projects' inputs, slower pace of
Communication scheme could be better developed at the
appraisal and approval of the key documents,
outset of the project, and adjusted during the
changeover in Russian administration added
implementation. It should be not only `top-down'
some changes and delays in the JWC work
hierarchy but also `bottom-up' from grassroots and
including adding further complexities to the
NGO's for maximum effectiveness and efficiency.
personnel responsibilities and accountabilities,
further the communication priorities, and
With administrative changes on both shores of the lake
perceptions changed among the JWC members
each 2-3 years the JWC will experience problems with
etc. Finally, the outset budget allocation quite
regard to the composition, timeliness and effectiveness.
changed time value with conversion costs from
It may be advisable in future, to keep aside political
USD at outset to local currency at least 5 years
issues/changes from the actual project work of JWC, et.
later. Hence, project scope was
al, so that the Programme can be managerial, scientific
changed/narrowed in line with the existing
and task-oriented generally. Therefore, maintaining future
realistic budget and less tasks/tenders were
ongoing business and science relations while minimizing
performed.
external risks that could affect the Management
Programme implementation
With the LFA in place at the outset, all key risks could be
addressed timely and precisely while keeping the project
progress on track.
Budget recommendations entail a shorter project cycle
with more realistic and comprehensive allocations.
Directly resulting from the Lake Peipsi there
For future, what can be recommended on the
were some complimentary activities which grew complimentary activities might also include:
into useful and satisfactory future outlets i.e. per 1. Additional funding via
the Peipsi CTC. They are:
a)Application to the available programme funds e.g., EU
a) Lake basin management activities
Twining Programme, Foundation Matra, Netherlands,
b) Public participation/water programme
(eco-farming, eco-tourism up to 200,000 grants)
management activities
b)Partners for Water, Netherlands (small grants)
c) NGO and other development fostering
c)others in USA, Canada, EU
work activities for funding and
2. Establish a better network with other international
transboundary projects
NGOs (through World Water Forum and like)
Development of eco-tourism and eco-farming in 3. Some commercial project like `environmentally
both Estonia and Russia might bring further
friendly fishery' together with EU entrepreneurs and
income to the local communities, in some cases
Estonian & Russian businesses. Some with eco-farming
replacing the former livelihood of fishing, in the (health food stores supply in Estonia, Russia, EU)
short, medium terms, if the required
4. Public Participation
infrastructure is in place.
a) Extensive further educational measures e.g. lectures,
seminars, TV/radio broadcastings on the regular basis.
Volunteers programs, e.g. road race (Tartu Marathon),
Sponsors for Lake Peipsi, Youth Hostels, Wilderness
Training Weekends, Walking and Hikes, Parks,
5. Eco-tourism, e.g. sailing, boating, canoe, biking, water
sports and river sports, ice fishing, ice vehicles.
The project outcomes can be considered as
More attention to the issues like problems/solutions,
satisfactory despite the changes in the original
lessons learned and graphic presentation at least the
scope and again experienced delays. Project
milestones of the project progress. It enables to show the
meetings and various reporting could be
project dynamics.
assessed as satisfactory. Even the project
reporting was timely and detailed to certain
Stronger position of the JWC with enough allocated
extent, the graphic presentation of the project
resources, better coordination and cooperation between
progress as well as the problem/solution
both countries responsible agencies, timely allocation of
identification and lesson learned were light
the required resources, approval of the Management
weighted.
Programme & Monitoring Plan.
Activities related to Peipsi CTC performance
during this project were efficient, effective and
timely, especially in execution and coordination.
It can be a good example for future UNDP/GEF
and EU funded projects.
The partnership during the project
A good example for development and fostering new
implementation has improved on all levels, i.e.
partnership could be the joint water monitoring
international, regional and local including all
expeditions undertaken in 2003-2004, whereby they had
key stakeholders. The partnership has grown
two countries, one vessel, cooperation and willingness to
and fostered, at the higher levels to only a mild
accomplish the mission. For the Programme
extent in contrast to the regional and local
implementation increased involvement of other
levels. The common ground for partnership was
local/regional stakeholders from industries and business,
soundly established with the activities and final
e.g. use the Venn diagram of the stakeholder relationships,
agreements lagging behind. On both Estonian
is envisaged.
and Russian sides new stakeholders'
involvement made significant contributions to
the projects,.
The risk management score bordered on
More use of the LFA with practical problem/solution tree
satisfactory and marginally satisfactory. It was
exercise completed. Better identification of problems and
already mentioned in risks and assumption area
risks during the project design.
above. Most identified risks at the early stage
were to certain extent underestimated. Although
some risks were partially overlooked and
appeared as hurdles for the project
implementation, the PIT did the best to manage
them correspondingly. It enabled to complete
the project in due time, within the budget with
the required intervention of the JWC in the
process, consensus building, provision of inputs
for the LBMP development and approval.
The lack of the logframe with the risks and
assumptions table revised and updated made it
difficult to deal with identified and unprojected
risks appropriately. Therefore, no risk
management measures were in place.
Project monitoring & evaluation has scored
While the monitoring entails to check progress, take
marginally satisfactory rate. Although it was
remedial actions, update plans, and especially link to the
performed in accordance with the UNDP/GEF
logframe objective hierarchy, i.e. inputs, activities, results,
procedures, i.e. field visits, annual project
it is proposed to enhance the existing practices with
reports, outcome groups based on the
tangible regarding the basic building blocks of the
coordination mechanism, and annual reviews, it
management systems.
did not become a real project management tool
to full extent. The M&E results should be
distributed among the key stakeholders for
comments and feedback. The Steering
In future, the follow-up procedures should include regular
Committee for the most part handled the
standard monitoring conducted by an independent agent
adaptive management in the project
familiar with the UNDP/GEF procedures, and the results
implementation. It was somewhat effective
accordingly disseminated among the stakeholders. The
since most objectives were accomplished. No
mid-term evaluation should be conducted more rigorously
project specific monitoring plans were available. delivering the SMART results (specific, measurable,
The reporting framework was appropriately set,
achievable, relevant, time-limited). It enables to keep the
and timely conducted. The M&E was not a real
project firmly on track and add to better quality of the
management tool since the project experienced a final evaluation.
number of delays and scope changes. The
results of the mid-term evaluation looked very
optimistic and general.
Project Implementation. Overall score for this
Better communication scheme and risk management
project component is satisfactory.
programme are required in order to avoid the main pitfalls
There is an opinion that the modality of the
e.g. emerging risks, poor communication, other diversions
NGO led project, i.e. Peipsi CTC office, is very
in project progress and management.
successful pilot project example by itself. It
alleviates a great deal of the work load of the
The strong Steering Committee will provide steady
Contracting Agency. Evidently, most project
strategic guidance for the LBMP implementation in
administrative procedures were in place and
future.
timely conducted. Development of reliable &
The SC should be "checkpoint" to ascertain not only the
effective communication for both PIUs &
delivery of milestones but also the continuing relevance or
stakeholders contributed appropriately to project otherwise of the context and content of the deliverables.
Although some negotiations may have
This applies particularly for internal risks and
progressed slower, the key decisions and outputs assumptions.
were managed nonetheless. The SC faced the
necessity to modify the scope and the budget in
order to stay within the fixed limits and time
frame.
The project generated a number of documents,
The Intranet portal management containing the project
e.g. monthly, PPRs, APR/PIRs, annuals plans,
documents as well as the project web site and related links
monthly/quarterly financial reports, SC agendas
have contributed to the results dissemination. Since the
and minutes, other strategic documents,
programme implementation is envisaged in 2006-2015
newsletters, press releases, brochures and
there should be the solution made with regard to the web
booklets, etc. Mostly they were produced
site and other relevant documents that could be used
timely, efficiently and effectively, with or
during implementation. Disseminating information has a
without templates. Via the Intranet portal they
far-reaching effect for project success and management
could be accessed digitally as well.
and should be considered as one of the vital project
components in the future.
Project oversight. In general the project
Programmatically most of the mechanisms envisaged at
oversight with regard to the institutions involved the outset worked very well together. Perhaps the bridge
i.e. Peipsi CTC, both PIUs, St. Petersburg
building of the TACIS component with GEF/UNDP
office, UNDP, SC could be considered
component could have been stronger for better relations
satisfactory on the basis of three established
with Estonia and Russia in successful negotiating
criteria. Peipsi CTC was very active, St.
practices. The JWC, as a successor of the SC can count
Petersburg Office was best noted for ability to
on the experience gained by the SC and continue to
build bridges with other agencies, PIU Pskov
cultivate the steady albeit slower progress of improved
was best noted for strong local ties building,
relations for the cross border issues. Substantial inroads
UNDP/GEF noted for timely and supportive
have indeed been made and ought to be capitalized on
interventions and SC held a key programmatic
systematically.
role taking all pre-conditions and changes into
account for best actions.
Project execution. The modality of the NGO-
Due to the large number of objective verifiable success
run with Peipsi CTC as executing agency can be and results based indicators for UNDP/GEF standards, the
viewed as a successful precedent and pilot
project has proven to be relevant and will likely aid to
project that proved its capacity, experience and
achieve global objectives. Its array of relevant operations
expertise offsetting much of the responsibility of for planned policies and actions is useful for other NGO
the Contractor and freeing for new aspects of
run projects and relevant UNDP/GEF requirements.
the projects to be added on to implementation
measures and objectives. The staff's high level
of the professionalism can be touted and used to
train other similar units in fact.
Project implementation. The project was
The institutional capacity of PIUs was satisfactory and
executed within the budget and allocated time.
there was a high degree of commitment and willingness to
The Executing Agency managed to address
continue. However, in future, the PIU composition,
most technical and administrative issues timely
resources, scope of work should be realistically
and efficiently. The logistics was done very
considered..
satisfactorily.
Project Finances. While the new Management
The Financial Manager established a system covering
System per the UNDP/GEF office had been
operational and administrative financial aspects and
assimilated successful, it entailed that the
quality control aspects of the project. Budget and
financial planning was based on the direction of
personnel have being properly administered and
the Project Manager along with Project Advisor, supervised.
the two Project Coordinators (Tartu and Pskov)
and the Financial Manager. Converting from
The financial outputs of the project were of a very high
line item detailed account system to the new
standard. The Operational Guidelines for managing the
account based general category system took
finances of project could be used as a benchmark and
concerted efforts and sufficient coordination of
distributed to all stakeholders.
Financial Manager and ever-willing
Moscow/UNDP/GEF unit but success was
achieved.
The newly adopted procedures appeared to be For future projects, before starting the project it is very
intact and operational to provide a normalized important to provide the executing agency with proven
financial position in line with the project and reliable Management Software which enables the
activities.
agency to not only manage the project activities and
deliverables but to fully manage the finances of the
Actual spending versus budget expectations project.
were in line with the original specifications.
The disbursement analysis proved that the
project funds were used effectively efficiently.
Efficiency is the focus on the value-for-money
axiom or the input-output ratio implied by the
achievement of results by the means identified
by activities per result in the logframe. By
definition, this takes into account both the
preconditions to the project and any endogenous
risks and assumptions identified at the level of
activities.
Efficiency in the broad context for the project
was very positive although this conclusion is
derived from qualitative rather than precise
quantitative OVIs, except for balanced accounts
and financial reports, as well as from the
initially agreed detailed plan of activities.
It is assumed that the UNDP/GEF Moscow unit
auditor had fully audited and recommended any
changes or related issues for the project. So all
reviews were done in a timely manner and in
compliance with the existing requirements.
As known there were minor revisions of the
budget and fund re-allocation from the initial
intentions due to the fluctuation of the interest
rates, for example. All this activities were just
in full compliance with the UNDP and GEF and
had authorized changes accordingly.
Though there were some concerns regarding
potential budget change approval and, at the
beginning, successful assimilation/integration of
project management/financial management
software, the new system did not provide
sufficient detail to manage the project. These
problems were acknowledged and solved.
Banking was another issue in that there is no
bank common to both agencies which could
cause delays in transactions and such. The
financial risks were considered to be low except
the USD exchange rates. The audit procedures
were in place and operational.
Coordinating structures and mechanisms did
For coordination mechanism improvement it is necessary
work in this project and is rated as satisfactory.
to increase/enhance the use of communication plans, and
Working relations and coordination with other
also periodic stakeholder relationship analyses (Venn
donors, international organizations were
diagram). Maybe this could include the improvement of
generally good in that were necessary structures
government and non-governmental relations through
and mechanisms for the implementation of this
common goals and activities. Following are some of the
strategy.
outer lying concepts related to the project.
Inherent to the coordinating structures and mechanisms
are certain focal areas such as:
Country context and institutions: encompassing
`enabling' environment and water policy reform, designs
and mandates and maybe directives or maybe legal
frameworks, and the role of stakeholders
Balancing cross-sectoral and cross-boundary interests, i.e.
national and international conflict mitigation, water (use)
rights, international data bases, and cross-boundary
networking
Organization management: i.e. organizational
performance, management of external relations, and
incentive systems and career development
Knowledge and institutional resource base, i.e. education
and training, data bases and information management, the
use of networks and expertise pools.
4.
Lessons Learned
Project design:
Project cooperation between both countries is like a `pilot project' in itself'. There are different legislations,
standards, procedures, agendas of where to go further and somewhat similar positions at the state level. The
JWC's activities proved to be ground breaking, pivotal, instrumental and paving the way for the future key
decisions and outputs for the LBMP implementation.
By keeping political issues separately from the global environmental issues there is less risk as part of the project
and tasks can better reach completion.
Project delivery:
The LBMP has been developed and mostly ready for implementation while some minor structuring is needed. The
LBMP is viable and feasible in its content and scope.
The Management Programme was viewed by many interviewees as a primary output of the project. Expediting
the pace of negotiations between two parties on the LBPM official approval will be a crucial step since the
Programme is designed for implementation in the period of 2006 and 2015
Though the Monitoring Programme was prepared in 2004 the decision to agree is still pending due to the need to
harmonize national monitoring systems and requirements.
The findings, lessons learnt, best practices are recommended to consider in the follow-up project and for future
programmes/projects. While planning there should be feasible correlation between scope of work and budget. A
comprehensive logframe and monitoring procedures should be outlined in the tender dossier as requirements. The
monitoring and evaluation bodies as well as procedures should be defined in advance.
Feasibility studies on eco-tourism were developed and CB has been done so in fact developing the areas for the
future is now much easier. There are some investors that would like to develop business plans. It is for
entrepreneurs, how fast are they willing to develop more. Added bonus is that it can also maybe in future replace
former fishing industry as a means of livelihood for many in the region. Notably, the Russian side is especially
interested in water tourism.
Project implementation:
Management process/procedures are a primary output in the new capacities for each side respectively.
Both Estonian and Russian side found the personnel efforts useful with agendas distributed before Joint
Committee and Steering Committee meetings. Additionally, minutes, quarterly reports and others were found to
be efficient as monitoring and progress indicators to certain extent.
UNDP/GEF and EU Tacis and Life Project Committees, as well as personnel, did discuss respective targets and
methods during planning phase to avoid replications in each project scope as part of the complex.
The extensions of satellite projects and delays with the input to UNDP/GEF project brought hurdles for partial
implementation of newly designed Management Programme as well as its approval by country's high level
authorities. The extensions in both case were caused by different internal and external factors, e.g. Tacis project
got two extensions caused by different reasons.
Networking and information exchange was well-established and facilitated through printed materials or Web
portal of the project. There were several sorts of publications, including the booklets, newsletters, manuals,
guides, postcards, other printed materials. There were extensive trainings, seminars and workshops, i.e. almost
100 in total.
Stakeholders involvement:
The Committees' (SC and JWC) perceptions enhanced awareness and activities is a signal to many stakeholders
and target groups of strengthened capacity and more networking and information exchange established. With
continual meetings the perceptions of Russia and Estonia Joint Committee Members did indeed progress towards
mutual collaboration, cooperation, participation; albeit more negotiating and less concrete activities transpired
with many smaller agreements building to larger agreements.
Committees (SC and JWC) build own management know-how and practices. There were practical and productive
sessions with both Russian and Estonian sides aimed at the cost /benefit analysis of problems, projects and
agendas including consideration of advantages and disadvantages of potential actions and developments.
Public awareness and the wellspring of new perception for both Russia and Estonia . The project contributed to
the perception, that now the Pskov/Chudskoe/Peipsi region is looked at a whole unit; with both Estonian and
Russian counterparts. Consideration now looks to the Lake Peipsi Region in total and it's qualities and
characteristics, features and attributes.
For both Estonian and Russian communities networking and information exchange has begun to build new public
awareness for terms "eco, environment, nature, including the lake as an entire ecological area" along with new
business and/or growth ideas correspondingly.
Environment concerns of eutrophication, measuring and monitoring became a GEF funded project and will
become joint programmes for future projects/best practices/know-how transfer in: Lake Basin Management
Community Schemes public participation, NGO and development project funding advisory for multi-country
approaches and transboundary. NGO's know-how can actually provide: idea generating, cooperation building,
information sharing (including lectures, seminars, best practices, etc.) for the new regions, countries, areas, nature
and eco-projects development.
Farmers and school teachers are more open minded about why not to pollute, using environmental law, getting
new technologies in future and how to consider changes in polluting farming behaviors. It can not be
implemented immediately, yet it would consider some options to add technology or practices since they had
education seminars. Yet a new `open-mindedness' and willingness to get involved based on
affirmations/acknowledgements from regional authorities on their using their own `know-how' with eco or
environment solutions. This institutional acknowledgement to the farming community, of new better practices is
useful and inspiring for them.
5.
Recommendations
The project `Development and Implementation of the Lake Peipsi / Chudskoe Basin Management Programme' was
completed as of January 2006 within the allotted timeframe and allocated budget. The Final Evaluation was conducted by
the international expert, Lisa Supeno, MSc. who has the required experience and expertise in both technical and project
management issues. Although there were some concerns encountered during the fact-finding and interviews of the final
evaluation of the project, still the main issue here remains within the realm of cooperation between the Governments of
Russia and Estonia on transboundary water cooperation issues. However, this is outside of the project control.
Nevertheless, there were a number of points raised by stakeholders and PIT that could build on current lessons learned
and contribute to the future regional and national benefits under different funding. This section contains a summary of
general and specific recommendations in order to improve the future quality, performance and management of similar
projects.
General Recommendations
During the programming phase, more attention should be paid to realistic resource allocation, specific of the
region conditions, e.g. political, economic, social, cultural relations, preconditions, i.e. readiness and willingness
to cooperate, potential hurdles and risks, mitigation measures, etc.
Transboundary cooperation developed during the UNDP/GEF project implementation still leaves a room for
further improvements and developments, i.e. the structure of cooperation could be better off with a combination
of vertical and horizontal level, where the vertical level represents a country's political decision making
hierarchy (Ministry of Environment, Department of Environment, etc.) including the Joint Water Committee,
the horizontal level is the Peipsi Council, JWC's WG (working groups) and ad hoc groups, implementation
organizations, stakeholders -NGO's, businesses, SME's, community groups, etc
The cooperation can be also enhanced and expanded via introduction of more joint activities on both sides of the
lake conducted by NGO's, local communities, SMEs, e.g.a joint action - Lake Peipsi Day with multiple
programs and volunteers to clean the lakeshore from litter, joint student monitoring day, etc.
Since there is no universal prescription or model for lake or river basin management committee or authority,
perhaps a common goal could be standardization of procedures and cooperation enabling a stable and constant
information exchange, transparency and openness between countries, different sectors of water users and experts
, as well as use of best practices from other regions.
The identified key environmental problems, i.e. eutrophication, fishery management, and groundwater pollution
have the transboundary context. They must be rigorously addressed in short-term by both countries through the
joint consultations and discussion, and incorporated in the national and regional action plans.
For the transboundary risks management, there should be a combined approach of pollution prevention and
cooperation. In this case the cooperation can be channeled via allocation (water resources, appropriate
legislation and agreements, `equitable utilization' and `no harm principle' and financial, social and human
capital), salvation (cooperative efforts, sound water policies, potential bottlenecks resolution jointly),
opportunity (common management results, benefits sharing, holistic approach for ecosystem preservation,
efforts to jointly administer the Peipsi Lake in a more flexible, inclusive and cooperative framework approach
of integrated management based on the ecosystem of the basin, co-financing), stakeholders' participation
(commitment, cooperation and coordination, dissemination, transparency and public awareness)
The fundamentals of PCM and LFA must be strictly followed as required by the contracting authorities
Further thematic evaluations/studies on project owner administrative capacity, particularly for projects needs
should be undertaken as an ex-ante input for all programmes
Each new project should have clear and specific preconditions
Specific Recommendations
The time between programming and implementation should be much shorter. This can make the project budget
and cycle more viable. In addition, it is advisable to involve the key stakeholders in project design on the early
stage.
The sooner the agreement and approval of the Joint Monitoring Programme by both countries happens, the better
the platform for the Water Management Programme to be launched as the next step scenario.
For new projects, it is advisable to prepare a timeline of requirements for the benchmark goals, monitoring results
and other key factors needed for appropriate measurability. Application of the SMART principles can facilitate to
project quality improvement both in qualitative and quantitative contexts. Include benchmark checkpoints and
institution responsible, for example along with LFA and PCM principles.
Better commitment and willingness to reach the goals should be inherent with all or both parties concerned across
the Lake. It can be achieved through mature dialogue and negotiations logically taking into account each's pre-
conditions and risks, information exchange, joint activities whereby businesses can be driving forces to bring the
needed finances not only for the programme implementation but to raise the income of the local communities.
The role and position of the Joint Water Committee definitely must be enhanced and empowered with required
resource allocation, e.g. human, financial and social capital to implement successfully this Lake Basin
Management Programme.
It is advisable in the future to address the issues related to the JWC composition robustness so that no personnel
changes can hinder its functioning or operations. When the JWC encounters committee member changes for either
country, it is advisable that a more specific joint process is introduced, i.e. the existing JWC receives the detailed
curricula vitae of the proposed candidates, study them, discuss them, and make jointly the decision of approval or
disapproval. This is the means to voice the collective opinion on the prospects jointly. In this case, it brings the
required quality of the personnel working at the committee. While it is up to the respective governments to make
new appointees it should be considered that it is a bilateral committee, thus the participatory component for input
and voting, for example.
Involvement of other stakeholders after the stakeholder relationship analysis is imperative during the LBMP
implementation. This can help in part to ensure robustness and sustainability of the project
When programming future projects like Lake Peipsi/Chudskoe the contracting agency (UNDP/GEF, etc) should
properly consider the complex of issues e.g. existing countries' relations (economic, political, social, cultural),
common language of cooperation, institutional structures and legislations, absorption capacities, existing
standards, norms and procedures, decision-making mechanisms, level of public participation and information-
sharing, potential co-financing, and availability and reliability of existing data and other resources
Future complimentary activities in the Lake Basin could include eco-farming (fish and agriculture i.e. cereals and
berries), eco-tourism (water tourism, ice tourism, sailing, Lake Day, etc.). Better business relations with small and
large businesses in the area or outside enable to bring definite changes in the status of infrastructures overall
Better risk management precondition outlines and better monitoring and evaluation needs to be included in all
projects for robustness and sustainability
The project data should be archived and stored. The chosen agency, responsibilities and other procedures are the
decision reserved by the JWC.
Although the draft Lake Basin Management Programme was presented to the stakeholders for review and
comments it is recommended to have a fresh overview of the Table of Contents and pay attention to the definition
of `General Objective, Specific Objectives, Targets and Approaches, Instruments and Public Relations, Success
Control, Implementation and Costs, Appointed Responsible Agencies, Planned Funding, and better structure this
document. Such document shouldn't exceed more than 30 pages and be prepared for decision-makers (who are
not scientists per se), written in plain language.
Public participation plan should be better presented in the LBMP.
6.
ANNEXES:
Annex 1 ToR
Annex 2 Itinerary
Annex 3 List of Persons interviewed
Annex 4 summary of field visits
Annex 5 list of documents reviewed
Annex 6 questionnaire used and summary of results (I - III)
Annex 7 project information evaluation sheet
Annex 1
Terms of Reference
1. Introduction
Standard UNDP/GEF Monitoring and Evaluation requirements
The Monitoring and Evaluation (M&E) policy at the project level in UNDP/GEF has four objectives: i) to monitor and
evaluate results and impacts; ii) to provide a basis for decision making on necessary amendments and improvements; iii)
to promote accountability for resource use; and iiii) to document, provide feedback on, and disseminate lessons learned. A
mix of tools is used to ensure effective project M&E. These might be applied continuously throughout the lifetime of the
project e.g. periodic monitoring of indicators -, or as specific time-bound exercises such as mid-term reviews, audit
reports and final evaluations.
Final evaluation is intended to assess the relevance, performance and success of the project. It should look at early signs
of potential impact and sustainability of the results, including the contribution to capacity development and the
achievement of global environmental goals. It will also identify/document lessons learned and make recommendations
that might improve design and implementation of both other UNDP/GEF and EU financed environmental projects.
Project objectives
The overall objective of the project is to develop and start implementation of a Lake Peipsi/Chudskoe Basin Management
Programme including practical recommendations for the Lake Peipsi/Chudskoe nutrient load reduction and prevention
and the sustainable conservation of habitats and eco-systems in the cross-border region. The project is substituting
uncoordinated small-scale projects that would be otherwise implemented separately on the Estonian and Russian sides
without sufficient coordination, education and public information component, and without taking into account interest of
local stakeholder groups and wider public.
The development objective of the project is to prepare, adopt and launch Lake Peipsi/Chudskoe Basin Management
Programme for the nutrient load reduction and the sustainable development of the cross-border region.
The main outputs of the project are:
Management Programme (Strategy Document) and Action Plan prepared and approved by all relevant Estonian and
Russian authorities;
Strengthened Capacity of key regional stakeholders (including environmental monitoring infrastructure and data
collection and maintenance system);
Networking and information exchange;
Two pilot projects implemented in Estonia: one on eutrophication reduction through planning water systems in small
community and the second one on ecotourism and nature protection (ecological route).
Project is executed by international NGO Peipsi Center for Transboundary Cooperation. Due to the nature of the project
the implementation of project activities are done by two Project Implementation Units situated in Tartu, Estonia and
Pskov, Russia. Overall management of the project is responsibility of Project Manager situated in St. Petersburg liaison
office, Russia.
2. Objective and purpose of the Final Evaluation
The overall objective of this Final Evaluation is to review progress towards the project's objectives and outcomes, assess
the efficiency and cost-effectiveness of how the project has moved towards its objectives and outcomes, identify strengths
and weaknesses in project design and implementation, and provide recommendations on design modifications that could
have increased the likelihood of success, and on specific actions that might be taken into consideration in designing future
projects of a related nature.
In pursuit of the overall objectives, the following key issues will be addressed during the Final Evaluation of the project:
Assess the extent to which the project achieved its overall objective
Assess the extent to which the development objectives and outputs of the IWP were achieved;
Describe the management processes how did project activities change in response to new conditions encountered during
implementation, and were the changes appropriate?
Review the clarity of roles and responsibilities of the various institutional arrangements for Management Programme
implementation and the level of coordination between relevant players;
Review any partnership arrangements with other donors and related project and comment on their strengths and
weaknesses;
Assess the level of public involvement in the elaboration of the Management Programme and recommend whether public
involvement was appropriate to the goals of the project;
Describe and assess efforts of stakeholders in support of the implementation of the project;
Review and evaluate the extent to which project impacts have reached the intended beneficiaries;
Assess the likelihood of continuation and sustainability of project outcomes and benefits after completion of the project;
Describe key factors that will require attention in order to improve prospects for sustainability of IWP outcomes and the
potential for replication of the approach;
Review the implementation of the project monitoring and evaluation;
Describe the main lessons that have emerged in terms of:
· country ownership/drivenness;
· regional cooperation and inter-governmental cooperation;
stakeholder participation;
· adaptive management processes;
· efforts to secure sustainability; and
· the role of M&E in project implementation.
In describing all lessons learned, an explicit distinction needs to be made between those lessons applicable only to this
project, and lessons that may be of value more broadly, including to other, similar UNDP/GEF and EU environmental
project initiatives.
The Report of the Final Evaluation will be stand-alone document that substantiates its recommendations and conclusions.
The Report will be targeted to meet the evaluation needs of all key stakeholders (GEF, UNDP, MOE, MNR, Peipsi CTC
and other stakeholders of Estonia and Russia).
3. Scope of the Final Evaluation
The main components to be evaluated include project delivery, implementation and project finances. Each component will
be evaluated using three criteria: effectiveness, efficiency and timeliness.
Project Delivery:
The Final Evaluation will assess to what extent the project has achieved its immediate objectives? It will also identify
what have been produces and what impact products have achieving overall objective.
The section will include an assessment of the following priority areas:
Institutional arrangements
Strategic planning, preparatory work and implementation strategies,
Technical support,
Capacity building initiatives,
Project outputs,
Assumptions and risks, and
Project-related complementary activities.
Outcomes:
Efficiency of project activities,
Progress in the achievement of immediate objectives (level of indicator achievements when available)
Partnerships
Assessment of collaboration between governments, intergovernmental and non-governmental organizations,
Assessment of national-level involvement and perceptions
Assessment of local partnerships, and
Involvement of other stakeholders
Risk Management
Were problems/ constraints, which impacted on the successful delivery of the project, identified at project design?
Were there new threats/risks to project success that emerged during project implementation?
Were both kinds of risk appropriately dealt with?
Monitoring and evaluation
Assess the extent, appropriateness and effectiveness of adaptive management in project implementation
Has there been a monitoring and evaluation plans for the project?
Is the reporting framework effective/appropriate?
Has M&E been used as a management tool in directing project implementation in a timely manner?
Is this framework suitable for replication/ continuation for any future support?
Project Implementation
Review the project management structure and implementation arrangements at all levels, in order to provide an opinion on
its efficiency and cost-effectiveness. This includes:
Processes and administration:
Project-related administration procedures;
Key decisions and outputs;
Major project implementation documents prepared with an indication of how the documents and reports have been useful.
Project oversight
Peipsi CTC
PIU in Pskov Chudskoe Projekt
Liaison office in St. Petersburg
UNDP
Tripartite Review Process
Steering Committee
Project execution
Peipsi CTC as Executing Agency (under the UNDP National Execution (NEX) modality)
Project implementation
Project Manager's office
Project Implementation Units (PIU's): Peipsi CTC and Chudskoe Project
Project Finances
How well and cost-effective did financial arrangements of the project worked? This section will focus on the following
three priority areas:
Project disbursements. Specifically:
Provide an overview of actual spending vs. budget expectations
Critically analyze disbursements to determine if funds have been applied effectively and efficiently.
Budget procedures
Did the Project Document provide enough guidance on how to allocate the budget?
Review of audits and any issues raised in audits; and subsequent adjustments to accommodate audit recommendations;
Review the changes to fund allocations as a result of budget revisions and provide an opinion on the appropriateness and
relevance of such revisions.
Coordinating mechanisms
Evaluate appropriateness and efficiency of coordinating mechanisms between national agencies, Peipsi CTC, PIU's
(including internal coordination), UNDP and the GEF.
4. Methodology for Evaluation Approach
The Final Evaluation will be done through a combination of processes including a desk study, selected site visits and
interviews - involving all stakeholders (but not restricted to): Peipsi CTC, UNDP, Government officials of Estonia and
Russia on different levels, local municipalities, local NGO's, communities etc.
The methodology for the evaluation is envisaged to cover the following areas:
Desk study review of all relevant Project documentation
Consultations with Peipsi CTC, UNDP , PIUs and offices,
Site visits to Lake Peipsi and its adjacent areas, Tartu, Pskov and St.Petersburg and, optional, to Tallinn and Moscow
Interviews with stakeholders
Ministry of the Environment of Estonia (incl. Sub-departments)
Russian Ministry of Natural Resources and its territorial bodies
The Estonian-Russian Transboundary Water Commission
Local Municipalities
Local community representatives
NGO's from Lake Peipsi/Chudskoe region
5. Products
The main product of the Final Evaluation will be:
Final Evaluation Report based on the general format outline at Annex 1.
Final Evaluation Report
The Final Evaluation report will include:
Findings and conclusions in relation to issues to be addressed identified under sections 2 and 3 of this TOR;
Assessment of gaps and/or additional measures needed that might justify further funding to Peipsi/Chudskoe lake region;
Recommendations for future project interventions in the region.
The draft and final report will be written in the format outlined in Annex 1 of this TOR. The draft report will be submitted
to Peipsi CTC no later than 31 January 2005.
Based on the feedback received from stakeholders a final report will be prepared by 28th of February 2006.
The report will be submitted both electronically and in hard copies.
The report will be supplemented by: Project Information Evaluation Sheet
Presentation of basic information on the project and evaluators' rating and textual assessment in free format.
Summary presentation of findings to be presented in final evaluation meeting
Team leader will conduct a final meeting for selected stakeholders and prepares summary presentation of conclusions and
findings of the Final Evaluation.
The presentation will be followed by questions & answers session and round-table discussion on effective implementation
of evaluation recommendations. Expected time for a final meeting is March 2006.
6. Evaluation Expert
The Final evaluation will be carried out by highly qualified individual expert.
Expert is responsible for:
overall management and successful completion of the final evaluation as well as for drafting, finalizing and presenting
Final Evaluation Report to stakeholders (incl. Steering Committee).
drafting evaluation documents (questionnaires, interview lists etc)
conducting interviews with Estonian and Russian stakeholders.
review and analyze of project related documents (based in desk study)
Participation in the working meetings and Steering Committee meeting to present evaluation results and recommendations
Other tasks that are important to successful completion of final evaluation, etc.
Above are described some indicative tasks to be performed by evaluator. Expert will be responsible in further assignments
of tasks and division of responsibilities according to objective and methodology set by Terms of Reference.
Professional requirements:
Expert is expected to be familiar with the region (both Estonian and Russian side of Lake Peipsi basin) and have basic
knowledge of the project activity areas (such as transboundary waters, transboundary cooperation etc.)
Expert must have expertise on areas of environmental and/or water management, (incl. issues of eutrophication and non-
point source pollution reduction approaches), additional knowledge on NGO community, business administration and
public participation would be an asset.
The experience on monitoring and evaluation of international projects is required.
7. Implementation Arrangements
Evaluation management arrangements
Role of Peipsi CTC office in Tartu
Overall coordination of evaluation activities in Estonia and Russia
Overall administrative arrangement of evaluation
Organization of site visits in Estonia
Organization of meetings with selected stakeholders in Estonia
Role of Project Manager (located in St. Petersburg liaison office)
Coordination of evaluation activities in Estonia and Russia
Administrative and logistical support to Evaluation Team members during their mission in Russia
Role of Chudskoe Projekt in Pskov
Administrative and logistical support to Evaluation Team members during their mission in Russia
Coordination of evaluation activities in Pskov
Role of UNDP
Coordination of evaluation activities in Moscow
Administrative and logistical support for Evaluation Team in Moscow
Tentative timeframe
Tender
December 2005
Selection of evaluator
January 2006
Briefings for evaluator
January 2006
Desk review
January 2006
Visits to the field (including allocation for travel), interviews, questionnaires
Estonia
January/February 2006
Russia
January/February 2006
Debriefings
January/February 2006
Validation of preliminary findings with stakeholders through circulation of initial reports for comments, meetings, and
other types of feedback mechanisms - January/February 2006
Preparation of draft evaluation report
January/February 2006
Comments form stakeholders, input of comments - February 2006
Presentation of report & roundtable meeting
27-28 February 2006
Exact dates will be set with service contract. Final evaluation report must be presented to project Steering Committee on
27-28 February 2006 in Tartu, Estonia.
8. Report Submission
The report will be simultaneously submitted to:
Ms. Lea Vedder
and Ms. Natalia Alexeeva
Director
Project Manager
Peipsi Center for Transboundary Cooperation
Center for Transboundary Cooperation - St.Petersburg
Aleksandri 9, Tartu 51004, Estonia
Kozhevennaya line 34 - 411, 199006, St.Petersburg,
lea.vedder@ctc.ee
Russia
natalia@ctcspb.ru
Annex 1.1: Outline of Final Evaluation Report
Executive summary:
Brief description of the project
Context and purpose of the evaluation
Main conclusions, recommendations and lessons learned
Introduction:
Project background
Purpose of the evaluation
Key issues addressed
The outputs of the evaluation and how will they be used
Methodology of the evaluation
Structure of the evaluation
The Project and its development context:
Project start and its duration
Implementation status
Problems that the project seeks to address
Immediate and development objectives of the project
Main stakeholders
Results expected
Findings and Conclusions
Project delivery
Progress of the project as a whole in achieving its stated objectives
Effectiveness, efficiency and timeliness of project implementation
Stakeholder participation, partnerships
Project implementation
Project oversight
Project execution
Project implementation
Project administration
Project planning
Monitoring and evaluation
Risk management
Project finances
Budget
UNDP/GEF budget
Governments contribution
EU/LIFE and TACIS budgets
Other funds
Financial planning
Budget procedures
Disbursements
Effectiveness of funding mechanism
Risks
Recommendations
Corrective actions for the design, implementation, monitoring and evaluation of the project
Proposals for future directions underlining main objectives
Lessons learned
Best and worst practices in addressing issues relating to relevance, performance and success
Annexes:
TOR
Itinerary
List of persons interviewed
Summary of field visits
List of documents reviewed
Questionnaire used and summary of results
Annex 2
Itinerary
Date/day
Location
Activity
Person involved
Expected outcome
Comments
1
2
3
4
5
6
January 22
Amsterdam
International
L. Supeno
Arrival to project
Sunday
Tallinn - Tartu
travel
location
January 23
Tartu, Peipsi CTC
Briefing
S.Talvet @CTC
Instructions,
Monday
10 am
documents for desk
ditto
Data collection,
study
analysis.
Desk study
Aija Kosk@CTC
Interview
3 pm
Background data
January 24
Tartu, Peipsi CTC
Desk study
Data study
Based on the
Tuesday
Interviews
L. Vedder@CTC
Background data
obtained data,
11 am
Based on the
the inception
Iti Aavik@CTC
questionnaire
report is to be
Inception report
2 pm
Draft inception
done before
preparation
report
25.01.06
L. Supeno
January 25
Tartu, Peipsi CTC
Interviews
E. Soosaar
Project background
Wednesday
@office 3 pm
data
M. Nisu@CTC
Desk study
11:30 AM
Background data
Inception report
L. Supeno
Inception report
January 26
Tartu, Peipsi CTC
Site visits
J. Maandel@CTC
Project background
Thursday
Nature museum
11:30 am
data
Gollaste Museum, Journalist
G. Roll
M. Nisu
Fjodor Daspanov
January 27
Tallin Interviews
Interviews
H. Liiv, A Jaani
Project background
Friday
Travel to/from
A Laane
data
Tallin
Desk study
M Metsur
Project background
370 km
Site visits
L. Supeno
data
planning
S. Talvet
January 28
Tartu, Peipsi CTC
Desk study
L. Supeno
Project background
Saturday
data analysis
January 29
Tartu -St.
Briefing
L. Supeno
Project background
Sunday
Petersburg
data analysis
travel
January 30
St. Petersburg
Desk study
L. Supeno
Data on the project
Monday
Data collection
St. P. Office
January 31
St. Petersburg
Interviews
N Alexeeva @
Data on the project
For final report
Tuesday
St. Ptrsbrg PIU
12:30
L. Supeno
February 1
St. Petersburg
Site visit
Vladimir Popov,
Data on the project
For final report
Wednesday
Tartu travel
Leningrad
.
Water Ministry
L. Supeno
February 2
Tartu
Data analysis
L. Supeno
Data on the project
Thursday
Draft evaluation
for the final
report
evaluation
preparation
February 3
Tartu
Trip
S. Talvet@CTC 10
Data on the project
Friday
Interviews
am
for the final
G. Roll 11 am
evaluation
O.Vassilenko@telcon
V. TuurbelCTC 2 pm
Ulo Sults CTC 3 pm
February 4
Tartu, travel to
International
L. Supeno
Saturday
Tallin to
Travel
Amsterdam
February 5
The Hague, NL
Draft final
L. Supeno
Sunday
report
preparation
February 6
The Hague, NL
Draft final
L. Supeno
Monday
report
preparation
February 7
The Hague, NL
Draft final
L. Supeno
Tuesday
report
preparation
February 8
The Hague, NL
Draft final
L. Supeno
Wednesday
report
preparation
February 9
The Hague, NL
Draft final
L. Supeno telcon
Thursday
report
TACIS TL
preparation
Lasse Koivunen,
Telcon
Finland
Interview
w/TACIS
Rep:
Lasse Koivunen,
Finland 4 pm.
February 10
The Hague, NL
Draft final
L. Supeno
Friday
report
submission
Draft final
S. Talvet
Ms Talvet will send
evaluation
the report to the
report
stakeholders
distribution to
the stakeholders
for the feedback
within 10
working days
Comments &
S. Talvet
The stakeholders
feedback to be
will send their
sent to L.
comments to Ms.
Supeno by
Talvet by February
February 21
21. Ms Talvet's
office will forward
the comments to L.
Supeno
February 11
Saturday
February 12
Sunday
February 13
Monday
February 14
Tuesday
February 15
Wednesday
February 16
Thursday
February 17
Friday
February 18
Saturday
February 19
Sunday
February 20
Monday
February 21
Tuesday
February 22
Wednesday
February 23
Feedback and
S. Talvet
Thursday
comments on
the draft report
February 24
Friday
February 25
Saturday
February 26
Amsterdam
International
L. Supeno
Arrival to the
Sunday
Tallinn
travel
project location
February 27
Tallinn
Workshop
Stakeholders
Discussions
Monday
presentation
Final Workshop
Presentation
February 28
Tallinn
Workshop
Stakeholders
Discussions
Tuesday
presentation
Final Workshop
Presentation
March 1
Tallinn -
International
L. Supeno
Wednesday
Amsterdam
travel
Annex 3
List of persons interviewed
CTC Office Interviews
Management
Natalia Alexeeva & Sille Talvet
Gulnara Roll
Aija Kosk & Olga Vassilenko (telcon)
Lea Vedder
Staff:
1.
Iti Aavik, Peipsi CTC Small Grants Coordinator
2.
Eilika Molder, Web-Master
Questions:-
Interviewee role, organization, level of involvement, project start, time frame
Project implementation and overall status
Problems and main risks encountered
Immediate and development objectives of the project; bottlenecks/resolutions
Stakeholders
Results expected
Project delivery
Progress of the project as a whole in achieving its stated objectives
Effectiveness, efficiency and timeliness of project implementation
Stakeholder participation, partnerships
Project implementation
Project oversight, execution, implementation, administration
Project design and planning and initial changes, if any
Project Cycle Management, Log Frame Analysis familiarity and to what extent used; and if not, why?
Future improvement of deliverables (quantity and quality)
Monitoring and evaluation any agency, feedback given/used; resulting improvements
Risk management how identified risks were managed
Lessons learned or improvements and/or recommendations for future project plans
Project finances
Financial planning
Budget procedures, Disbursements
Effectiveness of funding mechanism
Risks
Plus `Success Indicators'
Pilot Project:
Ulo Sults
Virve Tuubel
Lasse Koivunen, TACIS component, (telcon)
Tallin Interviews
Mr. Jalmar Mandel, Head of the Tartu County Environmental Department of Estonian Ministry of the
Environment (Viru-Peipsi WMP, LIFE)
Mr. Endel Soosaar, Tartu County Administration, Deputy - Head of Economic Development Dept. (Steering
Committee member, eco-farming, eco-tourism)
Mr. Ain Lääne, Project manager of the EU LIFE Viru-Peipsi CAMP
Mr. Harry Liiv, Deputy Secretary of Environmental Management Department, Head of the Integrated Water
Management WG of the Transboundary Commission
St. Petersburg PIU Interviews:
Armand Elena, UNDP, Head of the Environmental Unit; St. Pete' telcon
Popov Vladimir, Former SC; Leningrad Complex Regional Water Admin
Questions:-
Background to component
Timetable
Activities undertaken
Results/ achievements
Feed-back on Results
Problems & solutions
Lessons learnt
Recommendations
Awareness of overall project objectives
Project Management issues
Any other points
Annex 4
Summary of field visits
The Evaluator selected was:
Lisa Supeno, MSc.
International Independent Expert
Programme
The Evaluator completed a number of meetings and discussions with key stakeholders suggested by the PIT including the
project offices in Tartu, Tallinn, Estonia and St. Petersburg, Russia (22 January 10 February, 2006) followed by desk
study. The results presentation was made at the SC meeting held on February 27-28 in Tallinn, Estonia.
Activities and dates are indicated in Annex 2 Itinerary and Annex 3 correspondingly.
Estonia:
1. Tartu CTC office location and CTC - PIU Implementation Unit (January 22-29, 2006 and
February 1 -4, 2006)
2. Kallaste County Nature Museum (January 26, 2006)
3. Tallinn various staff interviews @ office locations (January 27, 2006)
Russia:
1. St. Petersburg PIU Implementation Unit (January 29-February 1, 2006)
Estonia:
1.
Tallinn Evaluation results presentation at the SC (February 27 28, 2006)
Annex 5
List of Documents Reviewed
1. Peipsi-Chudskoe UNDP/GEF project "Development and implementation of the Lake Peipsi/Chudskoe Basin
Management Program" description with the Annexes A - L.
2. Project Performance Reports (period review from the project start upto its end, i.e. January 2003 December
2005) with Annexes.
3. Technical/Financial Proposal/Agreement, April 11, 2002
4. Annual Project Reports 2003, 2004
5. Lake Peipsi/Chudskoe Basin Management Program
6. Public Participation Plan, updated version
7. Financial Agreement with the UNDP, 2002
8. Budgeting change, 2004
9. Pepsi/GEF budget proposal, 2001
10. Annual Plans 2004, 2005
11. Quarter financial reports, 2003-2006
12. Annual Financial Reports, 2003-2005
13. Minutes of the Steering Committee meetings, five in total
14. Transboundary Diagnosis Analysis (TDA), 2005
15. GEF Monitoring and Evaluation Policies and Procedures, 2002
16. GEF Guidelines for Implementing Agencies to conduct Terminal Evaluation, March 4, 2005
17. UNDP Handbook on Monitoring and Evaluating for Results, 2002
18. Preliminary Report of the EU Tacis CBC Baltic Line 2000 project "Environmental Management of Lake
Chudskoe"
19. Mid-Term Evaluation Report on the UNDP/GEF project, June 2004
20. Project's newsletters and press releases, 2004-2005.
Annex 6
I.
Questionnaire used and summary of results
UNDP/GEF Lake Peipsi Project Final Evaluation Stakeholder Assessment Questionnaire
An important component of this mid-term review is an assessment of stakeholder perception of the progress of this project
and their involvement in the project. The stakeholders fall into a number of categories:
1. International Stakeholders
·
UNDP/ GEF
·
Peipsi CTC
·
Estonian Russian Trans-boundary Water Commission
·
International NGOs
·
EC Tacis
·
Other international projects
2. Government Officials
·
Estonian Government officials
·
Russian Government officials
3. Local interest
·
Local municipalities
·
Local Community representatives;
·
Local NGOs
·
Local projects
This assessment will evaluate the level of collaboration between
·
the project and identified stakeholders
·
between different government agencies
·
international organization;
·
local organizations
The purpose of the interviews will be to establish the level of contact between stakeholders and the project, to establish
the `value' placed on the project by the stakeholders and to obtain feedback on the execution of the project. The
questionnaire should be divided into several sections to establish the role of the person being interviewed and their
involvement with Lake Peipsi/Chudskoe and then to establish their perception of the progress of the project, the
information being provided by the CTC and the involvement of the public in general in the improvements to the
environment as a result of this project.
QUESTIONNAIRE
1. Please mention your name, position in Project, organization, level of involvement and time frame of being
involved.
2. During the planning phase did any changes occur in the project plan? To your opinion how did the project go on?
3. What were the problems and main risks encountered?
4. How the bottlenecks were resolved and by whom and what means?
5. Did the project reach its 4 main objectives and to what extent?
6. What can be done better in the future to improve the quality and quantity of deliverables (comments and
recommendations in other words)
7. Did the interviewee have familiarity with PCM (Project Cycle Management) and/or LFA (Logical Framework
Approach) methods e.g. GEF?
8. If any, how was the Project monitoring conducted, and how? Any feedback received from the organization
conducted the monitoring and how the feedback was used to improve the performance?
9. How the identified risks were managed?
10. Did the project planning experience any changes during the implementation phase e.g. change scope of work
shrink/change during the project? And why?
11. Were the PCM principles and the Logframe used at all? And if not why?
SUMMARY OF RESULTS
Is there already the Agreement on Water Quality Status between Russia and Estonia officially signed?
1.
Was the Monitoring Programme agreed by Estonia and Russia already, if not when?
2.
What is the opinion on the UNDP/GEF Management Software ? Is it good, efficient or inflexible and useless?
(the question is to Peipsi CTC and Pskov PIUs)
3.
How many seminars and workshops were organized and conducted during the whole project (I could not find
it on the Web)? It is even better to know the split Estonia-Russia
ANSWER:
64 seminars-workshops in Estonian side
4.
How many publications were produced including the booklets, newsletters, manuals, guides, postcards (it was
only one with the fish as I saw), other printed materials, etc during the whole project?
ANWER:
booklets - 3
newsletters - 3
manuals - 2
guides - 0
postcards - 1
other printed materials (leaflets) - 1
What language dominated?
ANSWER:
booklets 3 (Estonian, Russian, English)
newsletters 3 (Estonian, Russian, English)
manuals 2 (Estonian)
guides - 0
postcards 1 (Estonian, English)
other printed materials (leaflets) 1 (English)
Were the recommendations made by the Mid-Term Evaluation Team taken into account, e.g. produce the published
materials as one document but into languages (means two languages in side one document)?
ANSWER:
Unfortunately we have published all booklets and manuals before mid-term evaluation. In case of project documents
we followed this advice.
What kind of publications were produced?
ANSWER:
All publication are region (Lake Peipsi/Chudskoe) specific and/or on problems/solutions concerning surface waters.
5.
How often the project Web site was updated?
ANSWER:
Updating of Web-site based on information availability.
What is its future after the project completion?
ANSWER:
After two years it will be closed
Did the Web master have any idea how many people visited the site (there is a special counter)?
ANSWER:
There is counter.
If yes, what is the number?
6.
How often did the public at large receive the information about the project and from the project?
ANSWER:
Information on the project was issued before and after every event organized in the frame of the project. Before every
event was issued the following information: what kind of event is coming; where; whom; main objectives. After every
event was issued the following information: main objectives and main results of the event. This information was
available on the project homepage and also sent out by lists.
What did the common people in both countries think about this project?
ANSWER:
Speaking with project team local people was very positive on this project. At the same time they mentioned that
mainly it is project to prepare policy/management documents.
Did the mass media make the project visible, if yes how and why in both countries?
ANSWER:
Mass media received press releases on every event that was bigger that small workshop for grassroots' NGOs.
How many articles in magazines, newspapers, etc were published and why?
ANSWER:
3 times in Estonia
How many radio and TV broadcasting were made in both countries?
ANSWER:
5 times in Estonia
7.
What is the story with the Russian museum?
ANSWER:
I am not competent to answer this question.
The Estonian is operational already.
ANSWER:
In Estonia Lake Peipsi Life Room (Museum) is operating since June, 2004. It is in Kallaste, in the region with a lot of
small museums for educational purposes.
How is the Russian part going to go on with the public awareness? What are the plans of the Estonian part with regard
to the same questions?
ANSWER:
I am not competent to answer these questions.
Are there any solid plans with certain financing found?
ANSWER:
Peipsi CTC has some projects (running already) that would be consider as follow-up activities of the UNDP/GEF
project.
8.
Were the results from the Tacis project and EU LIFE project incorporated in the Lake Basin Management
Program
ANSWER:
EU TACIS and EU LIFE projects results were incorporated in the Lake Basin Management Program. Lake Basin
Management Programme was drafted based on results of these projects plus expert opinion.
and when
ANSWER:
EU TACIS and EU LIFE projects results were available in November and September 2005, respectively.
and to what extent?
ANSWER:
Lake Basin Management Programme was drafted based on results of these projects plus expert opinion.
What is the status of the Monitoring Program?
ANSWER:
Monitoring Programme was submitted to Water Quality and Monitoring WG of the Estonian-Russian Joint
Commission on Transboundary Waters on June, 2005. It is accepted by the Secretaries of both side of the Joint
Commission as draft document that will be developed by the WG in the future.
9.
How was the local industries (except farmers and fishery) on both sides (Estonia & Russia) were involved in
the project?
ANSWER:
Local industries were not involved into the project actively.
What were the practical results of this participation/involvement?
10.
Please give the brief information of the two pilot projects, i.e. eco-tourism and eco-farming. When started and
ended?
How was the process and attitude? What was the budget allocated? Was it sufficient? Was the specific objective(s)
achieved? What are the practical results? Any lessons learned and recommendations for future?
11.
What was the stakeholders' involvement (passive/indifferent/partially active/very active)?
ANSWER:
In the regions with significant environmental problems (Ida and Lääne Virumaa) stakeholders participated in different
seminars and workshops very actively. In the regions without significant environmental problems (Põlvamaa,
Jõgevamaa, Võrumaa) stakeholder were rather indifferent.
Any lessons learned, if yes What will be the necessary steps in the future?
ANSWER:
NGO as Peipsi CTC should deal mainly with educating local people and providing information on the state of
environment in language understandable for local people.
12.
Were any other NGOs involved in the project except those that ran the project Peipsi CTC, Tartu and Pskov?
ANSWER:
Other NGOs were involved as stakeholders for information dissemination or consultation. Also they were encouraged
to apply small grant money.
13.
What was the level of support from the local governments ( passive/indifferent/partially active/very active)?
ANSWER:
Representatives of local governments participated actively in different seminars, workshops and meeting. Preparation
of water management plan as well as water supply and sewerage treatment development plan were in agenda of all
local governments according to the Estonian legislation at the same time. Therefore, they were interested to get
information.
Any concrete examples?
14.
How did the Small grant facility within the project work?
ANSWER:
Within the first call for application we received 12 applications and all these applications received money.
What was the total number of the proposals submitted during the whole period?
ANSWER:
78
How many were awarded?
ANSWER:
40
What was the total budget spent in total
ANSWER:
5845 USD
Per small project in average?
ANSWER:
150 USD
What kind of projects were they?
ANSWER:
95% of these projects were educational type. Main topic: practical environmental protection exercises in local
communes.
What were the practical results?
ANSWER:
Cleaned surrounding (collection of abandon waste)
Local people learned traditional handicraft from natural materials
Different practical studies for school pupils in nature
II
Interviews summary of results
Findings per Management and Staff Consultations and Interviews
Component
Problems
Lessons Learned
Future Recommendations
Project
1. While there is a joint
1. Through GEF communication kept
1. A joint protocol
Delivery
Water Management
open and facilitated between 2
implementation with step by
Programme for E and R
governments.
step measures is needed for
pending final signatures,
2. One special meeting was held for
the Joint Water Commission
currently there is no
TACIS, Life and GEF in January 2004.
for the two countries.
reason for E and R
governments to meet as
2 By keeping political issues separate
2. To write a final report for
their respective and
there is less risk as part of the project and project in future have criteria
mutual water
tasks can better usually reach
and indicators, who is
management programmes completion; insofar as precise science
responsible, before writing,
are nearing completion so terms or management divisions may not
e.g. at state and ministerial
far.
fit bureaucratic needs yet it may be
level so to better progress
understood, informally, at perhaps higher
the completed work without
2. JWC Russian members government levels but not in project
setbacks.
appointees or officials
specifics
may change on
3. Most important result was
committee requiring new
to develop with R and E a
set of relations, issues
3 Management planning is reflected in
common understanding of
and priorities each time
the Leningrad Water Admin./Project
the ecological state of the
causing slowdown of
Database good data correlations and
lake and methods to manage
JWC finalization
flows between Russia and Estonia. This
and evaluate. The common
progress. Change from
includes success of joint monitoring
understanding was pivotal
environment committee
mission financed by R oblast for one
goal.
rep to lawyer rep detracts
vessel, 2 countries, and one team taking
from quality of SC and
samplings. Although initially unplanned 4. Estonia should look for
JWC interactivities
it was very successful example.
state funds, EU and others.
technically. Change of
Will also try to find other
perception and priorities
4 Very good and meaningful R and E
funds themselves.
for R side may deter
discussions of Cost Benefit Analysis of
Russian local authorities
project goals with
problems, projects and agendas with
have to lobby for funds to
Estonian side.
plus and minuses of each problem at the
get money for environmental
Joint E and R Committee level.
issues so not at top of
3. With added political
Independently, this also occurred at the
priority list for Federal
agendas it is much more
Farmer level whereby they were
financial budgeting. $ in St.
difficult to
instructed how to calculate sufficient
Petersburg not in Leningrad
create/complete concrete
gains in order to be in business
oblast or Pskov region.
tables of deadlines,
altogether.
Entrepreneurs are invited to
timelines, tasks and
NGO meetings also.
needed materials.
5.Pilot project NGOs along with Peipsi
CTC, Tartu and Pskov and other NGOs
5. One team member is
4. Coordinated state
were involved as stakeholders for
optimistic about monitoring
monitoring delivery could information dissemination or
coordination on water
be improved from its
consultation and also encouraged to
quality for E side but less
existing flow between the apply for small grant monies for their
optimistic for Russian side.
three agencies with data
NGOs related to the Project outcomes.
and processes.
6. One Russian SC member
6. For eco-farming, Estonia 6-7%
was delighted to participate
5. Also difference
agricultural land managed by eco-
in the funding of the vessel
between Russian
farming and Russia also eco-farming
that set out for combining
ministries means different project shows just initial steps taken
the sample work with the
responsibilities to river
less than 1%. In Pskov, Leningrad
two country reps and equally
water quality and
oblast, there is abundant land, especially
as enthusiastic about the
problems may differ.
near Lake Peipsi, but few farmers
managerial level of the
interested in agricultural activities. This
database that shares the
6. It took many small
is since most R agricultural activity
various environment data for
agreements and
there is in Eastern Pskov, the opposite
all 3 of the projects.
implementations before
side to Lake Peipsi and far off.
perceptions of
7. Evaluator suggested
committees did change
7. Awareness raising from study means aquaculture inquiry for eco-
and negotiating moved
Jogevamaa county considers itself as a farming/fish stocking of the
towards definite
whole unit and not just one city only for lake area i.e. Prince Edward
resolutions.
tourism so better planning.
Island/mussels/Canada
7. If Russia has: env't
8. Maybe certain regions need eco-
Keeping the web materials
standards, different
protection, fragile for big tourism and for broad dissemination in 3
norms, monitoring
some city ordinances for preserving languages served well:
procedures based on end
historical looks.
The languages being:
of pipe solutions and;
Russian, Estonian and
If Estonia has to comply
9. The project meant that now the
English .
with the Water
region is looked at a whole too; not only
Framework Directive and
Estonia and Russia counterparts and in
8. Regional benefit example
monitors the state of
joint efforts; but also the Lake Peipsi
is a great benefit for Region
water from sample points
Region in total and it's qualities and
all the problems with the
along the river area
characteristics, features and attributes.
environment was brought
Then Estonia and Russia
out and published in the
need to agree to the
Eco-tourism for a clean environment media and got response from
Transboundary Water
new mission.
the local people and new
Commission monitoring
actions.
group so both can accept
10. Can also replace former fishing
to monitor the same way.
industry as a means of livelihood for
In the future both must
many in the region.
Remarkably, this is the first
not only sign but ratify or
time such an initiative
enact.
11.Lake Peipsi 5,000 km sq. on Russian (problems, media, folk
This risk is not managed
side and 4,300 km sq on Estonian side. response) has occurred in the
yet.
May 2005 study results in R. Study Region of Estonia. Almost
8. The co-dependency of
involved communities just that bordered like a pilot project itself.
Viru Life and TACIS
the Lake.
affected timeliness and
efficiency of project
Russian side especially interested in 9. Offshoots and good
including the inability
results from GEF project
water tourism.
for all the risks to be
Already have new project:
managed
Transboundary River Basin
May 2005 seminar in R. reps,
entrepreneurs, local reps, experts :
Management Program is
9. Kallaste Lake Peipsi
could discuss how could work.
pending:
museum risk is less
Future ongoing cooperation
CTC is from Estonian side
funding for advertising
entrepreneurs and personal contacts.
and Russian side is CTC St.
and new location for
Petersburg is a partner
growth & tourism.
Met in August: Eco-tourism like NGO
Latvia, Lithuania, Finland,
Project web site will be
in Estonia for possibilities
Estonia , Russia and Sweden
closed after two years
New terminology in Estonia e.g. eco,
TRABANT with the Water
sans funding.
nature
Framework Directive. Deals
with most public
10. Russian environment
So far tourism is not suitable for big participation issues that
data quality and quantity
tourist groups and you need a guide originated with the Mantra
is much less available
especially.
East/UNDP section and
than Estonian and slows
developed successfully.
progress.
Working out standards for eco-tourism Some monitoring issues and
guides
also the know-how transfer
11. No formal monitoring
to other river basins too is
of project process. Yet
12.With continual meetings the
quite good.
SC members overlap with perceptions of Russia and Estonia Joint
working group members
Committee Members did indeed
Can provide development
who can informally see
cooperation and
some degree of quality
progress towards mutual collaboration,
development assistance to
control. No use of PCM
cooperation, participation; albeit more
other countries for rivers,
and Logframe.
negotiating and less concrete activities
lakes, water management
transpired with many smaller
issues, public participation:
Enabling /providing
agreements building to larger
Kyrgyzstan, Moldova,
laptop to an Estonian
agreements
Kazakhstan and Uzbekistan
member necessary for
and other countries is funded
effective interchanges, in
by Estonian Ministry of
addition to PIU offices.
Foreign Affairs the
13. Environment concerns of
development cooperation.
12. Pilot project eco-
eutrophication, measuring and
tourism area in Estonia
monitoring became a GEF funded
10. It may be best to have
may need conservation
project and will become joint
one central organizing unit
ordinances/historical
programmes for future projects/best
for the three projects rather
preservation at some
practices/know-how transfer in Lake
than separate areas.
areas.
Basin management Community
13. Pilot project eco-
Schemes public participation, NGO
11. Good practice that
tourism, some R
and development project funding
NGOs can `pool' their
consideration of 8 rules,
advisory for multi-country approaches
resources for applying for
and also consider fragility and transboundary. NGO's can actually
new grants and funding.
of nature in area before
provide: idea generating, cooperation
tourism development.
building, information sharing (including
12. It may be necessary for
lectures, seminars, best practices, etc.)
Estonian Ministry of Foreign
14. Eco-tourism in
for the new regions, countries, areas,
Affairs to handle future
Estonia funded by State
nature and eco-projects development
fundraising efforts that have
priorities are Tallinn and
to do with the Russian and
an island; therefore
Estonian joint water
municipalities must each
management project now.
distinguish own
cities/regions including
gaining own funding.
15. Representatives of
local governments in E
would like to get more
information.
They
participated actively in
different
seminars,
workshops and preparing
water management plan,
water
supply
and
sewerage
treatment
development plans in all
local government agendas
according to environment
legislation at same time.
16. Pilot project eco-
farming info seminars to
farmers need books to
add to environment know
how so to back new
environment protection
measures/equipment.
17. Farm seminars better
held not in summer but in
winter when they are
farming.
18. Eco-farming could
include aqua-culture (i.e.
Prince Edward Island,
mussels culture idea) but
of new stocks or simply
re-stocking rotations
much more difficult to
achieve as post hoc rather
than ad hoc. throughout.
19. Pilot project mapping
took place Estonia eco-
tourism cited all
problems and all work
done in area for mapping
scheme to manage pilot
project. Possibly Russian
too.
20. Small Grants
proposal submitted 78
overall were
submitted and 40
budget of 5,845 with
150 USD. per grant.
Yet, number of active
volunteers was very
high w/o pay also.
21. Pskov office finds it
most useful to involve as
many local stakeholders
before, or earlier in the
project to ensure much
more sustainability.
22. Necessary to write the
final Joint Water
Programme from many
sources of reports after
the fact of meetings and
project over rather than
ongoing or during the
project makes it
23
Installing
new 14. As a result of the eco-tourism study,
13. Moscow GEF unit is
computer to Pskov meant developing the area is now easier. There
always seen as very
additional
hard-
a
re some investors that Virve has access
cooperative and supportive
working juridical team to that would like to develop business
for various PIUs, managers,
just to allocate properly plans. It is for entrepreneurs, how fast
committee members et al.
and
successful
then are they willing to develop more. The
receive/reassign
to development is already visible and is
beneficiary
the
new giving people new direction and things
beneficiary equipment
to do and why.
24. Can't insure that all
15. Already good examples and results,
14. Suggested that GEF and
invited entrepreneurs for
i.e. where a whole community draws
TACIS in the future keep
future aid will attend the
together for tourist before Poltsamaa
more independence from
trainings, info sessions or was just a city and now it has some more
each other in project areas
seminars or GEF
identity as the county itself in addition
better.
presentations.
to the degree of tourism
Now all Jogeva county altogether and
not just a city for tourism
16. Both E and R side found PM great
efforts useful with agendas distributed
before Joint Committee and Steering
Committee meetings. Additionally,
minutes, quarterly reports and others
were found to be efficient as monitoring
and progress indicators to certain extent.
17. The `evolution' of joint R and E
cooperative, communicative, monitoring
and measuring measures seems apparent
with marked results to-date so far.
18 Farmers and schoolteachers are more
open minded about why not to pollute,
using E envt'l law, getting new
technologies in future and how to
consider to change polluting farming
behaviors.
Can't
implement
immediately yet would consider options
to add technology or practices since had
education seminars. Yet a new
willingness to get involved based on
affirmations from regional authorities on
their using their own know-how with
environment solutions. Institutional
acknowledgement
of
new
better
Project
practices is useful.
Implemen-
tation
19. GEF and Life Committees did
discuss targets and methods during
design phase to avoid replications in
each project.
20. Development and training new
projects stem from GEF project:: public
participation in river basin management;
know how on how to apply for funds
and manage NGO's;
Development programme and area
monitoring programme for both Estonia
and Russia sides
b) env't education and NGO
strengthening: seminars, tourism, envt'
issues increase focus
FBBRT - Latvia, Lithuanian, Finnish,
Russian, Estonian and Swedish
Interactive Project for Transboundary
River Basin Management and
Framework Directive. Dealing with
public participation issues (derived from
UNDP and Mantra East) and monitoring
issues (know how transfer).
Talus/CHU Development cooperation
advisory : on public participation and
water management (e.g. Kazakhstan,
Kyrgistan, Moldova, and more)
21. Management process/procedures are
a primary output in the new capacities
for each side respectively.
22. Cooperation: In a sense the
counterpart communication of both
Russian and Estonian is like a `pilot
project' in itself'. There are different
laws, perspective and ideas of where to
go further and different positions at the
State level. So with this in mind and
looking at the cooperation with Estonia
and Russia and stakeholders yes as key
decisions and outputs are useful.
Project
25. Less disbursement
Finances
chance since Russian
financial priorities differ
from Estonian:
Leningrad oblast overall
needs all new financing
for new equipments.
26. There is no Bank
common to both
countries and to their
accounts so have worked
out a R national and E
regional bank solution
Annex 7
Project Information Evaluation Sheet
Criteria
Effectiveness
Efficiency
Timeliness
Priority Areas
(1 = highly satisfactory, (1 = highly satisfactory, 2 (1
=
highly
2 satisfactory,
satisfactory,
satisfactory,
3
marginally 3
marginally 2 satisfactory,
satisfactory,
satisfactory,
3
marginally
t
4 unsatisfactory,
4 unsatisfactory,
satisfactory,
t
e
n
n
5 N/A)
5 N/A)
4 unsatisfactory,
o
C
5 N/A)
1
2
3
4
5
1. Institutional arrangements:
2.1
2.1
2.1
Strategic planning, preparatory work and
2
2
2
implementation strategies
2
2
2
Technical support
2
2
2
Capacity building initiatives
2
2
2
Project outputs
3
3
3
Assumptions and risks
2
2
2
Project-related complementary activities
2. Outcome:
2
2
2
Efficiency of project activities
2
2
2
Progress in the achievement of immediate
2
2
2
objectives (level of indicator achievements
when available)
3. Partnership:
1.7
1.7
2
Assessment
of
collaboration
between
2
2
2
governments, intergovernmental and non-
governmental organizations
e
r
y
Assessment of national-level involvement and
2
2
2
e
l
i
v
perceptions
Assessment of local partnerships
1
1
1
j
e
c
t
D
r
o
P
Involvement of other stakeholders
2
2
2
4. Risk Management:
2.6
3
2.3
Problems/constraints effected the successful
project delivery. Their identification at the
2
3
2
project design phase
New threats/risks emerged during project
implementation
3
3
3
Both kinds of risks dealt appropriately
3
3
2
5. Monitoring and evaluation:
2.2
2.2
2.6
Assessment of the extent, appropriateness and
2
2
2
effectiveness of adaptive management in
project implementation
Availability of a monitoring and evaluation
3
3
3
plans for the project
Reporting framework was effective or
2
2
2
appropriate
M&E used as a management tool in directing
2
2
3
project implementation in a timely manner
Suitability of this framework for replication/
continuation for any future support
2
2
3
6. Process & administration:
2
2
2
Project-related administration procedures
2
2
2
Key decisions and outputs
2
2
2
Major project implementation documents
2
2
2
prepared with an indication of how the
documents and reports have been useful.
7. Project oversight:
2
1.8
2
Peipsi CTC
1
2
2
n
PIU in Pskov Chudskoe Projekt
3
2
2
t
i
o
t
a
Liaison office in St. Petersburg
e
n
2
2
2
l
e
m
UNDP
p
2
2
2
Tripartite Review Process
2
2
2
j
e
c
t
I
m
r
o
Steering Committee
P
2
2
2
8. Project execution:
2
2
2
Peipsi CTC as Executing Agency, under the
2
2
2
UNDP National Execution (NEX) modality
9. Project implementation:
2
2
2
Project Manager's office
2
2
2
Project Implementation Units (PIU's):
2
2
2
Peipsi CTC and Chudskoe Project
10. Project disbursements, specifically:
2
2
2
Overview of actual spending vs. budget
expectations
2
2
2
Disbursement analysis in terms of the funds
used effectively and efficiently
2
2
2
11. Budget procedures:
2
2
2
Sufficient guidance on how to allocate the
2
2
2
budget provided by the Project Document
Review of audits and any issues raised in
audits;
subsequent
adjustments
to
2
2
2
c
e
s
n
accommodate the audit recommendations
a
i
n
Review of the changes to fund allocations as a
result of budget revisions and an opinion
2
2
2
j
e
c
t
F
expressed
on
the
appropriateness
and
r
o
P
relevance of such revisions
12. Coordinating mechanisms:
2
2
2
Assessment of appropriateness and efficiency
of coordinating mechanisms between national
2
2
2
agencies, Peipsi CTC, PIU's including
internal coordination, UNDP and the GEF.