GEF FINAL EVALUATION
Transfer of Environmentally Sound Technologies
(TEST) to Reduce Transboundary Pollution in the
Danube River Basin
PROJECT TITLE:
Transfer of Environmentally Sound Technology (Test) to Reduce
Transboundary Pollution In The Danube River Basin
PARTICIPATING COUNTRIES: Bulgaria, Croatia, Hungary, Romania and Slovak Republic
UNDP PROJECT NUMBER : RER/00/G35
GEF Project Number:
867
Project Evaluator:
David H. Vousden
CONTENTS
Acknowledgements
Acronyms
1. Executive Summary
2. Evaluation Process (Purpose and methodology)
3. Project Background and Landscape
3.1
Objectives
3.2
Justification for the Project
3.3
Project Components and Outputs
4. Findings and Evaluation
4.1 Project Delivery
4.1.1
Outputs and Activities
4.1.2
National Delivery The Demonstrations
4.1.3
Summary of Demonstration Enterprise Delivery
4.1.4
Threats and Root Causes Effective Resolution
4.1.5
Global and National Benefits
4.1.6
Stakeholder Participation and Public Involvement
4.1.7
Capacity Building
4.1.8
Policy and Legislative Reform and Improvement
4.1.9
Replicability
4.1.10
Risks and Sustainability
4.2 Project Management and Implementation
4.2.1
Project Design and Planning
4.2.2
Project Management
4.2.3
Project Execution and Implementation
4.2.4
Country ownership/Drivenness
4.2.5
Workplan and Budget (including cost effectiveness)
4.2.6
Monitoring and Evaluation
4.3 Overall Project Impact
4.3.1
Objective Achievements
4.3.2
Constraints
5. Conclusions of Evaluation
6. Recommendations
7. List of Annexes
Acknowledgements
2
The Evaluator would like to thank all of the people interviewed during this evaluation procedure
(whether in person or by email), all of whom proved to be helpful, courteous and informative, and
all of whom were clearly fully supportive of the project and its delivery.
The Evaluator would like especially to thank UNIDO for its hospitality and for the efforts made by
them to ensure a smooth and successful evaluation, and in particular the Project Coordinator.
David H. Vousden - GEF Project Design Specialist and Evaluator. March 2005.
3
Acronyms
AB
Advisory Board
APR
Annual Progress Review
BAT
Best Available Technology
BOD
Biological Oxygen Demand
CEE
Central and Eastern Europe
CESD
Centre for Environmentally Sustainable Development
COD
Chemical Oxygen Demand
COMFAR
Computer Model for Feasibility Analysis and Reporting
CP
Cleaner Production
CPA
Cleaner Production Assessment
CPC
Cleaner Production Centre
DRP
Danube Regional Programme
EA
Executing Agency (of GEF)
EMA
Environmental Management Accounting
EMIS
Danube Regional Project's Expert Group on Emissions
EMS
Environmental Management System
EST
Environmentally Sound Technology
EU
European Union
GEF
Global Environment Facility
IA
Implementing Agency
ICPDR
International Convention for the Protection of the Danube River
IPPC
Integrated Pollution Prevention and Control
ISO
International Organisation for Standardisation
M&E
Monitoring and Evaluation
M&EB
Mass and Energy Balance
MSP
Medium-Sized Project
NCPC
National Centre for Pollution Control
NGO
Non-Governmental Organisation
NPO
Non-Product Output
OP
Operational Programme (of GEF)
PDF
Project Development Facility (of GEF)
POPS
Persistent Organic Pollutants
QMS
Quality Management System
REC
Regional Environmental Centre
SAP
Strategic Action Plan
SES
Sustainable Enterprise Strategy
SQA
Semi-Quantitative Score
TDA
Transboundary Diagnostic Analysis
TEST
Transfer of Environmentally Sound Technology
UNDP
United Nations Development Programme
UNEP
United Nations Environment Programme
UNIDO
United Nations Industrial Development Organisation
USAID
United States Agency for International Development
VOC
Volatile Organic Compounds
4
1. EXECUTIVE SUMMARY
The GEF Transfer of Environmentally Sound Technologies project was implemented by UNDP
and Executed by UNIDO. In the three-year period of the MSP, the project successfully completed
training and knowledge transfer related to capacity building and institutional strengthening at both
the level of the selected demonstration enterprises and at the level of the national counterpart
institutions (Cleaner Production Centres, Pollution Control Centres, etc). The actual demonstrations
of the TEST approach in 17 enterprises was equally as successfully with considerable investment
made by the selected companies into the adoption of cleaner production processes and
environmentally sound technology.
There are some concerns related to Project Design, which should be noted for future consideration.
Activities for replication and transfer of lessons from the Project's achievements to other
beneficiaries and stakeholders within the countries and the Danube Basin as a whole were weak.
These can be related to the absence of any specific transfer and replication mechanism or linkages,
and the fact that the Project was constrained by its MSP modality and funding limitations (and, to
some extent, 3-year time limitation). Furthermore, the total funding identified in the Project
Document was not fully realised. The Project Document also has no reference to sustainability of
the Project's objectives or of the GEF investment.
In this respect, it must be stated that the Project Management and the Execution Process achieved a
very high level of success from the point-of-view of completion of the Project activities and
delivery of intended outputs. Any criticism has to rest with the Project Design and not its Execution
or Management.
The Terminal Evaluation finds this project to have been most notably successful and a very
worthwhile example of a GEF MSP investment from which many valuable lessons and practices
can be captured. The Evaluation provides a number of recommendations, including the proposal
that serious consideration be given to further investment to transfer these lessons and best practices
and to build on the substantial achievements of the TEST project. The Evaluator applauds the
Executing Agency, the Project Coordinator and the in-country Coordinators and Project Teams for
a praiseworthy achievement.
2. EVALUATION PROCESS (PURPOSE AND METHODOLOGY)
The purpose of a GEF Independent Terminal Evaluation is to enable all of the direct stakeholders
OF the project (Government, private sector, the Implementing Agency, the Executing Agency,
GEF, NGOs, etc.) to review achievements and delivery, and to identify valuable lessons and
practices that need to be captured and sustained. To this end it is important that such an evaluation
gathers as much input and feedback as is possible from a broad spectrum of all project stakeholders
and beneficiaries related to the project objectives.
The Evaluation attempts to determine, as systematically and objectively as possible, the relevance,
efficiency, effectiveness, impact and sustainability of the project. The Evaluation will assess the
achievements of the project against its objectives, including a re-examination of the relevance of
the objectives and of the project design. It will also identify factors that have facilitated or impeded
the achievement of the objectives. While a thorough review of the project design and
implementation is in itself very important in order to explain or justify project trends and/or
amendments, such an in-depth evaluation is ultimately an important tool for providing detailed
recommendations with regard to the current project and its outputs, and for capturing best practices
and lessons which can be used to structure and drive future initiatives.
5
The Evaluation places its emphasis on results and delivery, with reference to any measurable
indicators as defined within the Project Document or subsequent Annual Project Reviews/Project
Implementation Reviews. However, the evaluation also recognises that GEF Projects are, by nature,
dynamic and constantly evolving, and that this requires flexibility and understanding when
reviewing a project in the context of its original objectives and intended outputs.
It is important to see the Evaluation not merely as a monitoring process required by GEF but more
as the final opportunity to scrutinise and review what the project has achieved and learned. The
Evaluation is the instrument that helps all parties to identify valuable lessons, document successes
(and failures) and best practices (as well as those to be avoided). It provides closure to a project
while allocating a degree of achievement, but it should also identify any logical next steps and
potentially valuable follow-up exercises. Of course, most importantly, it also provides guidance for
similar GEF project design and implementation in the future.
Further details regarding the Monitoring and Evaluation requirements of UNDP/GEF and the
Objectives and Purpose of this Evaluation can be found under the Terms of Reference for the
Evaluation (Annex I). Annex II provides a list of the persons/agencies/bodies interviewed during
the course of the evaluation. Annex III gives a list of the documents reviewed.
In looking at the achievements of any Project, it is necessary to review the Outputs against any
measurable indicators provided by various Project documents. This often requires some level of
rating or quantitative scoring. To this effect, the Evaluator has used a Semi-Quantitative
Assessment approach which aims to assess the actual achievements of the project up to the time of
completion against the anticipated achievements as defined in the Project Document and/or
APR/PIR.
This SQA approach assigns a scale of achievement for each output (based on the expected delivery
and the success criteria for measuring that delivery) This provides a useful and quite accurate
guideline to which components were completed, which were not, and what the reason may be for
any lack of delivery. This is not intended to be an exercise in criticism, but more importantly one
whereby valuable lessons and practices can be identified and captured both for the sake of the
current project and for future GEF projects.
This assessment uses a judgement of the percentage of achievement per activity or output against
the original intention of the Project. To smooth-out the subjective nature of this approach, this
percentage is then converted to a scale from 1-5 whereby:
0 1.1 =
Almost No Delivery The Project has effectively failed in its objectives.
1.1-2.0 =
Limited effective delivery - generally poor and unsustainable. Project unlikely to
have secured its objectives.
2.1-3.0 =
Borderline Some notable achievements and delivery in specific areas but weak in
others. Considerable additional effort necessary to secure intended objectives.
3.1-4.0 =
Good, Effective Delivery Most activities or outputs have delivered as expected.
Project has met majority of its objectives and has produced benefits consistent with
GEF operational strategy. Project could have benefited from some changes in design
or realignment of priorities. Some possible areas of weakness that could be
strengthened through follow-up activities.
4.1-5.0 =
Excellent Delivery All outputs delivered as planned. Full stakeholder support.
Project undoubtedly successful and sustainable.
6
The SQA scores for Project Delivery, Management and Implementation are presented and
discussed at the end of each section.
Section 5 (Conclusions of Evaluation) of this report presents the SQA scores for the overall
objectives and components of the Project as defined both in the Project Document and subsequent
APR/PIRs, as well as by the GEF criteria for Projects and discusses their implications. This
includes an extrapolated composite score for the Project Outputs and Activities.
This Evaluation was conducted during the February-March 2005. The Evaluator conducted
interviews and made observations in Bulgaria, Croatia and Slovakia as well as at UNIDO
Headquarters in Vienna, Austria. This included field-trips to selected enterprises in Croatia and
Slovakia as well as the opportunity to observe and ask questions at the final TEST National
Dissemination Workshop in Sofia, Bulgaria. Further follow-up consultations were carried out in the
3 weeks following the Evaluation Mission in order qualify any concerns and to fine-tune the
Evaluation report.
3. PROJECT BACKGROUND AND LANDSCAPE
3.1. Objectives
Overall Project Objective and Description
The UNDP/GEF Pollution Reduction Programme identified 130 major manufacturing enterprises
of concern (known as hot spots) within the Danube River Basin; a significant number of these were
contributing to transboundary pollution in the form of nutrients and/or persistent organic pollutants.
In spite of the environmental problems they were causing, there was a lack of convincing evidence
that it is possible to comply with environmental norms while still maintaining or perhaps enhancing
their competitive position. This project set out to build capacity in existing cleaner production
institutions in five Danubian countries to apply the UNIDO programme on Transfer of
Environmentally Sound Technology (TEST) at selected pilot enterprises that were contributing to
transboundary pollution in the Danube River Basin and the Black Sea. The aim of the assistance
was to bring these pilot enterprises into compliance with environmental norms of the Danube River
Protection Convention while at the same time taking into account their needs to remain competitive
and to deal with the social consequences of major technology upgrading. The enhanced institutional
capacity would then be available to assist other enterprises of concern in these countries as well as
other Danubian countries.
Development Objective from the 2003 APR/PIR
· To improve industrial environmental management by major industrial enterprises in the
Danube River Basin, resulting in Major reductions in pollutant loading and consequently
risk to the Danube River and Black Sea aquatic environments.
· To build capacity in networks of national cleaner production institutions to advise the
enterprises in the five participating countries on how to implement the TEST approach.
3.2. Justification for the Project (Taken from the Project Document)
The Danube River Basin
7
The Danube River basin is the heartland of central Europe. The main river is 2,857 km long and
drains 817,000 sq. km including all of Hungary; most parts of Romania, Austria, Slovenia, Croatia,
and Slovakia; and significant parts of Bulgaria, Germany, the Czech Republic, Moldova and
Ukraine. Territories of FR Yugoslavia, Bosnia and Herzegovina and small parts of Italy,
Switzerland, Albania and Poland are also included in the basin. The Danube River discharges into
the Bla ck Sea through a delta which is the second largest natural wetland area in Europe.
Water Quality Problems in the Danube
The Transboundary Analysis (TDA) for the Danube River Basin (1999) identified the following
main problems that affect water quality use: high load of nutrients and eutrophication;
contamination with hazardous substances, including oils; microbiological contamination;
contamination with substances causing heterotrophic growth and oxygen depletion and competition
for available water. The human activities contributing significantly to these problems are human
settlements, agriculture and industry.
Industry, atmospheric deposition, etc. cause about 20-30 per cent of the problem of excessive
nitrogen and phosphorus in the Danube Basin. Old-fashioned fertilizer factories are major
dischargers of nitrogen and their outdoor piles and lagoons of phosphor-gypsum are a special case
of pollution by nutrients. Even if production on these sites is reduced or stopped, the gypsum
stores will continue to be serious pollution sources in the future.
Industry and mining are responsible for most of the direct and indirect discharges of hazardous
substances into the Danube Basin. Depending on the type of industry, the effluent might contain
heavy metals (smelting, electroplating, chlorine production, tanneries, metal processing, etc.),
organic micro-pollutants (pulp and paper, chemical, pharmaceuticals, etc.) or oil products and
solvents (machine production, oil refineries, etc.). Mining activities result in drainage water from
the mines, run off from tailings and from process water containing metals and sometimes organic
solvents. Data on loadings of hazardous pollutants are available from only a few individual
enterprises. Sewage is a main source of ammonia.
Organic materials discharged by human settlements and industry consume available dissolved
oxygen. The impact is dependent on the total load, the type of organic substances, the water
temperature, the dilution capacity and the initial oxygen concentration of the recipient. Serious
oxygen deficiencies are most likely to occur in slow -flowing and stagnant waters. Downstream of
major outlets, the oxygen concentration may drop below the level that can support aquatic life
forms including fish populations and render the receiving waters unsuitable for drinking water
supply and recreation. Such situations are occurring in the Danube tributaries: for example, the Vit
River in Bulgaria is unable to support fish downstream of the city of Plevin, primarily due to
discharges from a sugar factory, and discharges from the pulp and paper factory in Pietra Neamt
have made one of the Siret tributaries unfit for most uses. The main stream of the Danube,
however, has a very large dilution and oxygen mixing capacity that enables it to cope with heavy
loads of organic materials.
Industrial Polluters
In the frame of the UNDP/GEF Pollution Reduction Programme (PRP) in 1998/1999, country
expert teams under the guidance of the respective country programme coordinators undertook a
new, comprehensive review of the sources of pollution and their effects in the Danube River Basin
and the Black Sea. Each national team developed a national review for their respective countries
based on a common methodology. The results were then compiled and analysed at the regional
8
level in the TDA. Based on the TDA and the ICPDR Emission Expert Group, 130 industrial
enterprises of concern (known as hot spots) were identified within the Danube River Basin.
The specifics of the transboundary pollution problems in the Danube River Basin and Black Sea
originating from the industrial plants in the five countries selected to participate in the TEST
programme can be briefly summarized: Bulgaria -- 8 plants contributing to nutrient loadings of 50
tons/year or greater; Croatia -- 3 plants contributing to nutrient loadings of 50 tons/year or greater
and 2 plants with other pollutant loading affecting a SIA in a neighbouring country; Hungary -- 4
plants contributing to nutrient loadings of 50 tons/year or greater and 2 plants with other pollutant
loadings affecting a SIA in a neighbouring country; Romania--33 plants contributing to nutrient
loadings of 50 tons/year and 5 plants with other pollutant loadings affecting a SIA in a
neighbouring country and Slovakia-- 2 plants contributing to nutrient loadings of 50 tons/year or
greater and 9 plants with other pollutant loadings affecting a SIA in a neighbouring country. Full
details are available in the UNDP UNIDO GEF Project Document.
The major polluting industrial sub-sectors in terms of numbers of enterprises are food; paper,
chemicals, and iron. Together these four sub-sectors account for about 75 per cent of the major
industrial pollutant dischargers.
Thus despite the period of transition in most of Central and Eastern Europe that has lead to serious
changes in the level of industrial and agricultural activity, industrial pollution still remains a
significant problem to be addressed by Danube Countries. Moreover, it can be expected that as
economies in the region recover and industrial production increases, industrial pollution will also
increase unless the source of pollution is adequately addressed.
3.3. Project Components and Outputs:
COMPONENT I. Institutional Strengthening
Objective: To set up national focal points that would facilitate the transfer of ESTs to
industrial enterprises in five Danubian countries
The first step for successful implementation of the project is to strengthen national focal points that
would facilitate the transfer of ESTs to industrial enterprises in five Danubian countries. The focal
points will be working units within an already established NCPC or PCC. Success under this
objective would be strengthened institutional capacity to apply the TEST approach. The availability
of the strengthened capacity would be measured in terms of the availability of trained national team
leaders and their deputies in the TEST approach, of operating information management systems
and of boards of advisors actively involved in enterprises selection and oversight of activities.
COMPONENT II. ENTREPRISE DEMONSTRATIONS
Objective: To apply the TEST approach to at least twenty enterprises located in the Danube
River Basin
The outputs and activities under this objective are the core of the project. Under this objective
national teams will apply the TEST approach in the five countries in order to show 20 enterprises
that it is possible to comply with environmental norms and still remain or perhaps enhance their
competitive positions. Success under this objective would be enterprise application of the TEST
approach, both individual components and of all seven components. Successful application would
be measured in terms of at least 15 out of the 20 participating enterprises applying the full TEST
9
approach to their operations and a larger number of firms applying most of the seven components.
In addition, there should be significant pollutant reduction (at least 30 per cent) by at least ten of the
20 enterprises and some pollutant reduction by the other ten enterprises at the end of the project.
Full compliance with environmental norms will take additional years because of the need to install
the EST packages at the enterprises.
COMPONENT III. Diffusion of Results
Objective: The diffusion of e xperience with the twenty pilot enterprises to other enterprises in
the five participating countries and to other Danubian countries
The ultimate aim of the project is to persuade other polluting enterprises in the Danube that national
institutions are available and capable of assisting them to devise cost effective plans for compliance
with environmental norms. Success under this objective would be wide spread awareness and
demand for the TEST approach among the major industrial enterprises causing pollution of the
Danube.
4. FINDINGS AND EVALUATION
4.1. Project Delivery
The overall objective of the Project was to build capacity in existing cleaner production institutions
to apply the UNIDO Transfer of Environmentally Sound Technology (TEST) procedure to
technology transfer to 17 enterprises that are contributing to transboundary pollution, and primarily
nutrients, in the Danube River Basin and the Black Sea.
Existing CPCs were functional within 3 of the Project countries (Croatia, Hungary and Slovakia) at
the start of the Project. Other pertinent institutions had to be identified for the other two countries
(Bulgaria and Romania). The institutes initially proposed at the national level within these latter
two countries were found to be inappropriate once the project was under way and new counterpart
institutes had to be identified. In the case of Romania this change was almost immediate and did
not cause any significant delays in project activities and delivery. In Bulgaria however, the situation
was much more complicated and the project went through two inappropriate counterpart
institutions before finally identifying the necessary capacity and effective project ownership within
the Technical University of Sofia. In this respect, a preparatory phase would have been helpful (in
this case a PDF A which is the only option available for an MSP) during which the counterpart
institutions could have been identified after working closely with one or two national potential
candidate institutes as part of the process of stakeholder involvement in project preparation. This
would have allowed the Executing Agency and Project Coordinator to `get to know' the institutes
and personalities first before committing the project to a particular counterpart. A more elongated
project preparation phase would have been possible with a Full GEF Project and raises the question
of whether a Full project would have been more appropriate to such a detailed multi-country
demonstration approach. This will be discussed further.
The following statement from the Evaluation Terms of Reference provides some useful guidance
for the evaluation process in this respect:
TEST Project Implementation Strategy
10
The project implementation strategy has been adjusted during project implementation, as indicated
in the UNDP/GEF Project Implementation Report (PIR1) June 2003 (section 3) in order to reflect
country specific conditions and to achieve the project objectives in a timely and cost-effective way.
By replacing the original step-by-step approach with the integrated approach, the revised TEST
project implementation strategy promotes synergies between different and complementary
environmental management tools supporting top management decision-making processes in
medium and long-term planning toward environmental compliance and eco-efficiency.
The TEST integrated approach to industrial environmental management developed by UNIDO, has
been designed to assist enterprises in the developing and transitional countries to effectively adopt
Environmentally Sound Technology (EST). The application of the TEST integrated approach and
its tools, leads to continuous improvement of the economic and environmental profiles of
companies.
The integrated TEST approach is based on three basic principles:
First, it gives priority to the preventive approach of Cleaner Production (systematic preventive
actions based on pollution prevention techniques within the production process) and it considers
the transfer of additional technologies for pollution control (end-of-pipe) only after the cleaner
production solutions have been explored. This leads to a transfer of technologies aimed at
optimizing environmentally and financially optimized elements transfer of technologies: a win-
win solution for both areas.
Second, the integrated TEST approach addresses the managerial aspects of environmental
management as well as its technological aspects, by introducing tools like such as
Environmental Management System (EMS) and Environmental Management Accounting
(EMA).
Third, it puts environmental management within the broader strategy of environmental and
social business responsibilities, by leading companies towards the adoption of sustainable
enterprise strategies (SES).
The schematic diagram below shows the stages of the revised TEST implementation strategy.
1 For additional details on the revised implementation strategy see the related PIR - June 2003
11
Consequently, it should be noted that the original Project Document had proposed to use a stepwise
approach to promoting EST, CP and an EMS. However, once the project started implementation,
the stakeholders discussed the actual merits of this approach versus other options and agreed that a
12
more integrated parallel development approach was necessary rather than the intended A->B->C
serial approach
As a result of these modifications, the project then realigned itself to build capacity and expertise
within the CPCs and related institutes to be able to deliver this new integrated approach, and to
demonstrate the approach through the project implementation and through the individual industrial
enterprise demonstrations. The original approach was defined within the Project Document as a
series of steps through the selection, training and re-education process whereby the industrial
enterprises either met the requirements of the project or dropped out of the demonstration. The
original TEST approach was expensive (requiring almost continuous assessment of many potential
enterprises) and would not capture the investment by GEF and the stakeholders efficiently. It
allowed too many chances for enterprises to drop out or opt out of the demonstration process even
once considerable time and investment had been made in support of those enterprises. This
modification of the project approach was pragmatic and necessary particularly in view of the
shortage of available budget within the constraints of a GEF MSP. Furthermore, the stepwise
approach was technically inefficient. Instead, enterprises were pre-selected on the basis of existing,
published information. Once the enterprises realised that they would also need to contribute time,
financial and human resources to the project aims there was a natural selection process through
attrition and lack of `ownership' for the project concepts and outcomes. In the end 17 enterprises
volunteered for the demonstration process.
The Project was effectively providing capacity building, training and delivery in the following
TEST tools (definitions derived from descriptions in `Increasing Productivity and Environmental
Performance: An Integrated Approach. Know-how and experience from the UNIDO project -
Transfer of Environmentally Sound Technology (TEST) in the Danube river basin' (Authors:
Roberta De Palma, Vladimir Dobes):
Cleaner Production (CP): The continuous application of an integrated preventive environmental
strategy applied to industrial processes, products and services to increase overall efficiency and
reduce risk to humans and the environment. The process includes conservation of raw materials,
water and energy, the elimination of toxic and dangerous raw materials, and the reduction of the
quantity and toxicity of all emissions and wastes. CP generates financial as well as environmental
benefits by encouraging companies to use processes that are more productive and cost-effective.
Environmental Management System: This is partly evolved from the Quality Management
Systems that are commonly developed within companies. It can be defined as that part of the
overall management system that includes the organised structure, planning activities,
responsibilities, practices, procedures, processes and resources for developing, implementing,
achieving, reviewing and maintaining the environmental policy. It should be an integral part of the
existing management system and should be harmonised with any existing quality management
system in the company.
Env ironmental Management Accounting: Monetary EMA is a sub-system of environmental
accounting that deals only with the financial impacts of a company's environmental performance. It
allows management to better evaluate the monetary aspects of products and projects when making
business decisions. EMA assists business managers in making capital investment decisions, costing
determinations, process/product design decisions, performance evaluation and a host of other
forward-looking business decisions. As such, EMA has an internal company-level function and
focus, as opposed to being a tool for reporting environmental costs to external stakeholders. This
gives EMA the flexibility to take into consideration the special needs and conditions of the
company. EMA tends to focus more on materials, energy flow, and environmental cost
considerations.
13
Environmentally Sound Technology: This is a combination of best available techniques and best
available practices. In most cases the introduction of good management practices alone are
insufficient to solve a company's environmental problems and to bring it into compliance with
environmental norms, or to greatly improve environmental performance. Investments in
technological changes or end-of-pipe solutions are also usually necessary. The concept of EST also
builds on the concept of BAT (Best Available Practices) where `Best' refers to best environmental
performance while `available' refers to economic feasibility as well as availability of the
technology on the market.
Sustainable Enterprise Strategy: The purpose of the SES is to assist management of the company
to turn the core strategic environmental/social success factors (as identified during the
implementation of the TEST approach) into formal performance objectives aligned with the
objectives of the company's business strategies (financial, marketing and operational). This means
that the environmental and social objectives of the company are not `stand-alone' objectives, but
are connected to the other objectives of the company and ultimately contribute to achieving the
financial goals of the business. Integration of the added dimensions of environmental and social
considerations should therefore demonstrate a clear competitive advantage within the business plan.
COMFAR: A Computer Model for Feasibility Analysis and Reporting. This is a software tool
developed by UNIDO and intended as an aid in the analysis of investment projects. The program is
applicable for the analysis of investment in new projects, and for the expansion or rehabilitation of
existing enterprises.
The above tools cannot, however, be treated in isolation and should be seen as a series of
overlapping and even interwoven modules that can be plugged in or omitted depending on the
conditions prevailing within the company, and on the assessment and investment needs of that
company. The adopted procedure for TEST within each enterprise would be decided through the
Initial Review prior to initiation of the CPA, but also through the outcomes and recommendations
of the CPA itself.
So, the procedure for building improved production linked to environmentally sound processes was
re-structured prior to its implementation to be a more phased approach. Enterprises were now
expected to complete a logical series of integrated activities before moving onto the next phase. For
example, the first phase focussed on introducing management tools in support of cleaner production
and more environmentally sound approaches, and demonstrating their use and value. The next
phase addressed the need to change the technology through investment in new technology where
appropriate, coupled with on-going changes to company policy and management through improved
awareness. The final phase was to deliver the final outcome of the demonstration package which
was the EST assessment linked into a business plan which effectively constituted a Sustainable
Enterprise Strategy (SES) for the company.
To this effect, the project itself acted now as an overall demonstration exercise for a new integrated
and phased approach to improving productivity while increasing environmental performance and
for evolving an effect mechanism for the adoption of environmental management accounting at the
industrial enterprise level. This resulted in a set of concrete project outputs in the form of two
UNIDO-funded publications:
1. Increasing Productivity and Environmental Performance: An Integrated Approach. Know-
how and experience from the UNIDO project "Transfer of Environmentally Sound
Technology (TEST) in the Danube river basin" (Authors: Roberta De Palma, Vladimir
Dobes)
14
2. Introducing Environmental Management Accounting at Enterprise Level. Methodology and
Case Studies from Central and Eastern Europe. (Authors: Roberta de Palma, Maria
Csutora).
These two publications carry most of the pertinent information and explanation of the revised
TEST process and methodology, how it should be applied, along with a selection of case studies
which demonstrate its effective application. This information has been shared with all of the
countries of the Danube basin along with the national TEST publications. These national
publications identify achievements at each national enterprise by way of economic improvements,
reduced emissions and discharges, low raw material costs, and proposed longer-term TEST-related
investments.
Undoubtedly the project has succeeded in its role of institutional strengthening and capacity
building as is discussed in further detail below. All of the counterpart CPCs and institutes have
received significant training in the TEST procedure both at the desk-top level and through actual
experience in assisting the enterprises themselves. At the end of the MSP this has left all of the
counterpart agencies in a strong position to provide a significant level of national and regional
support to industry in cleaner production assessment and techniques, environmental systems and
environmental management accounting, right through the modular TEST process to the
identification of environmentally sound technologies and the development of long-term sustainable
enterprise strategies. Of course, this capacity building has also been established within the
enterprises that have served as demonstrations. All of the enterprises now have a raised level of
awareness and understanding of the TEST process and associated improvements, not only in
cleaner production, but also in more cost-effective management of resources and wastes. Generally
speaking, the concepts demonstrated to the enterprises by the TEST project have now become
adopted into company policy on a day-to-day basis.
Furthermore, the overall objective of demonstrating reductions in discharges, and more efficient
use of water resources in industrial processes, whilst maintaining economic sustainability and
market competitiveness has been clearly demonstrated. There are still many improvements that can
be made at the individual level of each enterprise that could improve discharge reductions and
water use efficiency, but these are now mostly long-term changes in technology and will require
reconstruction and consequent high investment costs. Serious consideration is necessary as to the
next steps in this respect if GEF's investment to date is to be consolidated and improved. Within
the context of a Medium Sized Project the achievements are considerable and noteworthy. It should
also be noted that the TEST MSP went somewhat beyond its remit at a number of enterprises and
considered the issue of energy efficiency and its implications to the Danube environment. This was
a logical and sensible move as the concepts of energy efficiency dovetail well into the TEST
process and can significantly effect both cleaner production as well as the overall cost-effective
nature of a company's sustainable enterprise strategy.
The diffusion of results has been somewhat less successful at the regional level, although some
significant steps have been taken at the national level. Clearly each country has undertaken
dissemination activities and there has been at least one regional dissemination exercise but there is
a need for a more effective replication and transfer mechanism for lessons and best practices.
However, this should be seen in the context of financial constraints imposed on the project by the
design and modality and does not deter from the significant achievements made in the
implementation of the TEST demonstrations. This is discussed further under the relevant sections
below.
4.1.1 Outputs and Activities
15
Component 1: Institutional Strengthening. Component Outcome - Establishment of Test Focal
Points in each Country
TEST National Focal Points were effectively established for each country either within National
Cleaner Production Centres or in other relevant Pollution Control Centres/Institutes. Three
countries had existing CPCs (Croatia, Hungary and Slovakia).
In Romania, there was a CPC which had been established by USAID. However, the Executing
Agency and Project Management had no knowledge of their capacity and skills. The Project
decided to review several possible candidate institutions and finally settled with ECOIND
(Industrial Ecology Institute) which was considered to be best suited to the requirements of a
National Counterpart Institute and had some of the necessary expertise already in place.
In Bulgaria, the project had an unfortunate start as the original counterpart proved to be
inappropriate and had to be replaced. This regrettable situation was repeated a second time until the
Executing Agency and the Project Coordinator managed to identify an efficient source of expertise
(both for a national counterpart person and for other technical support to the Project) within the
Technical University of Sofia. This caused a delay for Bulgaria of around one year as far a project
implementation was concerned.
One lesson that can be captured here is the need for a more careful and drawn-out process of
selection of national counterparts. In hindsight, a period of preparatory stakeholder input and
assessment (such as a PDF phase of several months) would have been advantageous. This raises a
concern as to whether the MSP (GEF Medium Size Project) modality was appropriate to a regional
project of this nature. Although intended as a `fast-track' mechanism for releasing GEF funds more
efficiently, the MSP modality has two drawbacks that are relevant to the TEST project. One is the
financial limitation (with a ceiling of $1 million in GEF funds). The other is the lack of a prolonged
PDF B phase (MSPs may only access PDF A preparatory funding). This latter concern was
somewhat academic in the case of the TEST project as the MSP actually underwent no formal PDF
phase. This concern regarding the MSP modality is discussed further under the section on Project
Design below.
In the final analysis, strong project management and the selection (ultimately) of appropriate and
committed national counterpart personnel and institutions allowed the TEST Project to implement
an effective programme of capacity building and training for the TEST process which remains
firmly embedded within those personnel and institutions at the closure of the project, and can be
seen as a national and regional asset for further promotion and replication of the TEST process. The
Counterpart institutions have already demonstrated their ability (as well as market demand) to sell
their new expertise and services. A number of the CPCs have already been approached by both the
original demonstration enterprises and those outside of the project to provide further assistance
which the companies are prepared to pay for themselves. Also, a strong level of networking has
been developed between the national counterpart agencies, with countries providing special
expertise to each other and assisting each other in the development of further TEST-related
initiatives. Examples of this include close cooperation between Slovakia and Bulgaria, Hungarian
assistance and funding towards a Croatian project initiative for joint implementation and
dissemination of an Environmental Management Accounting module, and assistance beyond the
TEST countries to non-TEST Danubian partners. The CPC's themselves saw this as a very
effective and important result of the TEST project. Although UNIDO has established such a CPC
network in principle, this was the first time that the CPCs saw it work in action. They were very
complimentary regarding the training and the coordination meetings associated with this
16
networking process. Capacity building is discussed in further detail below under the relevant
section of this report.
The lack of a formal information management system was noted as this was an original output cited
in the Project Document for this Component. However, the project did have an active website that
provided useful information but this could have been improved with links to ICPDR and other
TEST-related initiatives. It was noticeable that there were also no direct links back from the ICPDR
website to TEST.
TABLE 1:
SQA SCORES FOR COMPONENT 1 DELIVERY
%ge
COMPONENT 1: Institutional Strengthening
Verifiable Indicators
Delivery
TEST Focal Point and count erpart institute identified and
All 5 Counterpart Institutes engaged
initiated
and actively delivering TEST products
100
Adoption of TEST Advisory Board
ABs functional with at least 2 meetings
per year
100
Training of TEST Counterpart Team and Consultants in TEST
Intended Number of trainees = 50.
procedures
Actual = 90. Intended number of man-
days of training =300. Actual = 369
100
Information Management System and networking established
Internet linkages to relevant databases
(including EU, ICPDR, etc). Effective
networking established between
CPCs/Institutes and Coordinator and
active sharing of lessons and
expertise between all parties
80
Total for Component
95
Table 1 shows the allocated SQA score for Component 1 delivery as being 95 which is equivalent
to a rating of 4.75 Excellent Delivery. All outputs have been fully successful and have achieved
maximum or very high SQAs. The Information Management System has delivered effectively as
far as the establishment of networks between the CPCs/Institutes and the project is concerned. A
substantial number of Internet linkages have been identified, and the sharing of lessons and
expertise has been very effective. The only missing activity indicator is internet linkage to ICPDR
and the Danube Regional Programme which would have been valuable. The Component has met its
main objectives (institutional strengthening, training of counterparts, establishment of Advisory
Boards, networking and sharing of information, etc) very successfully.
Component 2: Enterprise Demonstrations. Component Outcome - Application of the TEST
Procedure at Demonstration Enterprises within Danube River Basin.
Under the Project Document it had originally been intended to apply TEST to a total of 20
enterprises. However, in the final analysis 17 projects were selected as being suitable and viable
and, in view of the time-constraints imposed by the GEF MSP modality, it was decided to move
ahead with these 17. This is not construed as a shortfall or failure on the part of the Project as the
overall concept of enterprise selection necessitates elimination of non-viable companies.
Furthermore, there was no real basis or justification given within the Project Document for the
selection of 20 companies and this was presumably an arbitrary figures based on 4 demonstrations
per country. It should also be noted that in Slovakia (the only one of the five participating countries
17
to have only two selected enterprises) the official number of hotspots at the start of the project
(2001) was half of the original number included in the SAP (1997). As is discussed later, many of
the hotspots identified by the original SAP were no longer relevant or in existence by the time the
TEST project came to implementation. This situation inevitably affected the enterprise selection
process in some countries, and was unsurprisingly most evident in those countries that first joined
the EU (i.e. Slovakia and Hungary).
The criteria for selecting the enterprises, along with some of the inherent problems in the selection
process, were clearly defined in the 2003 APR/PIR as follows:
Selection of enterprises was a difficult step due to the fact that the participating countries are
characterized by lack of enforcement of environmental legislation and by limited understanding
of environmental concerns. Financial viability of companies was not easy to assess on a
preliminary basis, due to lack of reliable data. Moreover many companies do not have formal
medium-long term strategies, which further complicated the identification of focus areas for
project implementation. This is a common situation in transition economies where companies
are rushing to produce what is requested; and their focus on near-term survival may overcome
that of building a medium-long-term strategy.
In order to properly select the participating enterprises, local counterparts, in cooperation with
the UNIDO project manager, had to identify the right combination of tools for marketing the
project concept to local enterprises trying, on the one hand, to present the expected benefits for
the company and at the same time clarifying the level of commitment and necessary internal
resources. Being a hot spot was never sufficient reason for a company to join the project and as
it happened in several cases, not all hot spots were suitable for the TEST project (most of the
hot spot companies are in very difficult financial circumstances at the moment). Nevertheless
given the difficult situation and the lack of enforcement of environmental legislation, the
required number of enterprises was successfully identified. It seems then that economic drivers
are much stronger then the environmental ones and are pushing companies in the direction of
improving the efficiency of their operations and in acquiring EMS certification.
The 2003 APR/PIR recognises that appropriate marketing tools need to be used to properly select
the demonstration industrial enterprises. The APR/PIR also notes that medium to large size
enterprises fitted the TEST project requirements best (smaller enterprises generally do not have the
human resources necessary to successfully implement TEST procedures), and that the criteria for
selection of the most representative enterprises related to their willingness to cooperate, along with
the influence of regulators, market and community pressures. The Advisory Boards adopted
guidelines on which enterprises should be considered for invitation to take part in TEST.
Discussions were held with relevant government agencies to ensure the shortlist included the best
companies for the demonstrations, and to ensure that every company had been considered and
reviewed.
It is important to note that the changes in the TEST process itself resulted in alterations to the
procedure for CPA and TEST assessment. It has already been mentioned earlier that the project
revised the TEST implementation process in the early stages based on discussion and approval with
all national counterparts and Advisory Boards. This led to a more modular approach to TEST using
an integrated approach to assessment and management rather than a stepwise approach as had been
initially proposed in the Project Document. The original step-by-step approach was considered to
be too linear and time-consuming (as well as generalised and inflexible) with more opportunities
for enterprises to fall out of the process along the way. The modular, integrated approach allowed
TEST to be fine-tuned for the needs of each enterprise. It also included the addition of the
Environmental Management Accounting concept, which all of the counterpart teams and agencies
18
were strongly in favour of adopting into the TEST approach. One noticeable effect that this new
approach had was to alter the emphasis on and within the Sustainable Enterprise Strategy. The
original idea of the SES was to include the business plan for the EST investments (the main output
from the EST assessment), a social action plan, and a negotiated compliance scheme based on the
implementation of the EST solution identified. The EST assessment is the critical component of the
SES. However, it became clear during project implementation that the other two components of the
SES (the social plan and compliance scheme) were not feasible for certain companies.
The social action plan was not always applicable since a) the new technology did not necessarily
imply any redundancy of workers and b) some companies were already undergoing severe
downsizing as a result of the transitional dynamics and associated economics symptomatic within
the CEE (new owners, division of companies, etc). The companies were therefore dealing with
these issues on a larger and more general level with local labour unions.
The environmental compliance plan was already subject to a well-established mechanism for
negotiation between the companies and the local environmental inspectorates on an annual basis.
So, based on these considerations the SES concept was refocused and broadened so that the new
SES tool included elements of strategic management (as well as the EST business plan). This new
SES strategy was implemented in 10 of the 17 enterprises. It was not possible to apply SES to all of
the enterprises. The main reasons for exclusion included a) companies with public ownership where
it was difficult to address top management directly b) lack of a formal business plan c) lack of
interest by top management, often due to on-going restructuring of the company and its assets, and
d) limited resources and time-constraints within the Project. However, it is important to note that
while the SES tool was re-focused, the business plan for the EST investments (the most important
part of the original SES concept) was implemented in all companies.
In effect, this modification of the process gave a much stronger emphasis to identification and
adoption of Environmentally Sound Technologies rather than to the concept of Sustainable
Enterprise Strategy as an end-product of the GEF project. SES was now seen as more of a logical
next step, and in the company's interests as a conclusion to TEST in order to capture the cost-
effective components of the TEST process. All of the enterprises went through the modular
process, and EST options were developed for all 17 enterprises. This is an important change from
the original Project outputs as it presents a proactive and conceptual alteration of the TEST process.
This modification is explained and justified within the 2003 APR/PIR and was clearly supported by
all stakeholders. It does have certain implications for the Evaluation in that some original indicators
needed to be revised also. This requirement for revised indicators is not clearly captured in either
the 2003 or 2004 APR/PIR although it is to some extent implied within the text. The need to
address this oversight in the APR/PIR format is discussed further under the section on Monitoring
and Evaluation.
The concrete delivery from this component were the various options for cleaner production and for
environmentally sound technologies. These were referred to as Type A, B and C options and these
can be defined as follows:
Type A options : Good management practices and process optimisation no cost or low cost.
Type B options : Introduction of cleaner technologies low cost and short-term payback period
Type C options : Larger-scale environmentally sound technologies high cost and long-term
payback period.
A good example of how the TEST process was implemented for Croatia is given below under 4.1.3
National Delivery The Demonstrations.
19
The Project performed extremely well as far as the demonstration of the TEST process is
concerned. Despite some shortcomings in the selection process (related more to project design and
time constraint), all of the enterprises identified valuable CP and EST options, and nearly all of
them implemented a significant number of these, with a clear intention to implement more as and
when investment funding becomes available.
TABLE 2:
SQA SCORES FOR COMPONENT 2 DELIVERY
%ge
COMPONENT 2: Enterprise Demonstrations
Verifiable Indicators
Delivery
Selection of enterprises and viability asses sment
Original intention to select 20
enterprises. Actual selection of 17.
80
Training of the Test Teams in enterprises in TEST procedures
intended number of enterprise staff
trained = 500. Actual = 622. Intended
man-days of training = 1500. Actual =
1673
100
Application of the TEST procedures at selected enterprises
TEST modules selected and
implemented at each enterprise. All 17
enterprises undertaken at least IR,
CPA and EST. EMA and EMS at
selected enterprises.
100
Identification of Type A, B and C improvements
Type A,B and C options specified
through CPA and TEST process,
approved by enterprises and included
in Action Plan
100
Implementation of Type A options for improvements
One or more CP/Housekeeping
measures adopted per enterprise
100
Adoption/implementation of Type B options for improvement
One or more Type B low -cost, early
payback small investments made for
CPA and EST per enterprise
100
Adoption of Type C options for improvement
Type C high-cost, longer payback
measures adopted by company as
part of business strategy, and
investment being actively sort (or, in
some cases, under implementation)
100
Noticeable improvement to identified environmental threat at
Measurable reduction in discharge
each enterprise.
volume and/or toxicity, water usage, or
general waste production for each
enterprise
70
Total for Component
94
Table 2 Shows the SQA scores for this Component. Delivery of outputs has been very high,
equivalent to a rating of 4.5 Excellent Delivery. There could have been improvements in the
enterprise selection process to ensure that CPA and EST options were directly addressing IW
concerns at the level of water pollution, water use efficiency and wastewater management. It is fair
to remember that for the Project Management process, two of the pre-conditions for enterprise
selection were A) Their financial viability and sustainability and B) Their voluntary commitment to
the project. This suggests that the selection procedures should have been more clearly defined in
the project design. Although the actually delivery on the TEST procedure at most enterprises was
commendable, these did not always address real threats at the river basin level. This is further
reflected in the output addressing improvements to identified environmental threats. Furthermore,
20
there is a need for each enterprise to keep more detailed records of improvements, reductions in
pollution and waste discharges, toxicity levels, etc if the data presented to the Project is to be fully
credible. Otherwise, the training of the in-house TEST teams, the application of the TEST
procedures, and the identification and adoption of Type A-C improvements has been remarkable
within the constraints of the time period for the project.
Component 3: Diffusion of Results . Component Outcome - Disseminating Results to other
Enterprises and Countries.
The primary objective of this component was to raise awareness among other industrial enterprises
in the Danube region regarding the TEST process, and to further ensure that they were aware of the
presence of national institutions that could assist them in using the process to develop cost-effective
business plans and Sustainable Enterprise Strategies that would help them to comply with new
environmental directives and regulations.
To this end, every country hosted a National Dissemination Seminar to which at least 10 industrial
companies were invited to review the results of TEST in the Demonstration enterprises.
Furthermore, in certain countries some of the enterprises that attended these seminars and showed a
strong interest were given a demonstration two-day in-plant assessment (4 in Romania and 3 in
Croatia). A regional seminar was also organised in cooperation with ICPDR, and the results of the
TEST Project were disseminated at the national level through many different conference and
workshops.
UNIDO supported the distribution of TEST methodologies and result through the development of
two publications on `Increasing Productivity and Environmental Performance' and on
`Environmental Management Accounting' (as discussed above under Project Delivery). A project
web-page has also been created by UNIDO where the two publications can be downloaded for free
(www.unido.org/doc/26190).
The original Project Design had intended that teams in 4 other Danube countries would be trained
in using the TEST Approach. This was successful in one country (Bosnia) but proved unfeasible for
other countries due to a general lack of project resources along with the difficulties in identifying
national counterparts as there are no CP institutions within the remaining countries of the Danube.
It should be noted that the activities undertaken in Bosnia relating to this component went much
further than was proposed or intended in the original Project Document. As well as the workshop
held in Bosnia which included the participation of several companies and national institutions, a
full project document for the replication of TEST in Bosnia was prepared by UNIDO in
cooperation with the CP centre of Croatia, and the Centre for Sustainable Development (CESD) in
Sarajevo. This project proposal is currently with UNDP for review and funding is being actively
sought to support its implementation. In reality, and considering the constraints of funding and lack
of effective institutional capacity, it would seem that the Project Design has been somewhat
ambitious and optimistic in this respect. A more structured and planned approach to building
further capacity for TEST within the other Danube countries would have been appropriate, and
almost certainly beyond the financial and time constraints of an MSP of this nature.
This Component is the weaker of the 3 project components in that a lot more transfer of
information, lessons and practices could have been done and a more effective replication process
initiated. However, the Evaluator is inclined to see this as a fault in the project design coupled to
the constraints of inadequate funding, and not as a management or implementation fault as such. So
much was achieved in the relatively short project lifecycle that shortfalls in this component should
not be allowed to cloud these achievements. Having said that, there is a risk in losing GEF's
investment here if the lessons and practices are not captured and if the TEST demonstration is not
21
replicated effectively throughout the Danube region. This will be the subject of further discussion
under Conclusions and Recommendations .
TABLE 3:
SQA SCORES FOR COMPONENT 3 DELIVERY
%ge
COMPONENT 3: Diffusion of Results
Verifiable Indicators
Delivery
National Dissemination Seminars to share results
Dissemination seminars completed for
all 5 TEST Project countries. National
reports published
100
TEST training materials and case studies
Test Manual and Case Studies
published and disseminated
100
Introduction of TEST approach to 25 other enterprises in 5
Other (non-demonstration) enterprises
TEST participating countries
engaging the CPCs and Institutes to
undertake TEST process
40
Regional dissemination to other Danube countries
Regional Workshop/seminar (e.g. on
BAT, Industrial Pollution Control,
TEST processes) plus dissemination
of published materials to relevant
institutes in other DRB countries
80
Introduction of TEST approach to other enterprises in Danube
Teams from other Danube countries
River Basin
trained in TEST approach
35
Component Total
71
Table 3 presents the SQA scores for Component 3 which is equivalent to a rating of 3.5 which is
within the Good, Effective Delivery category. Dissemination at the national level has been good as
far as sharing the results of the enterprise demonstrations with the national stakeholders is
concerned. This component has been weaker on delivery than the other 2 components in that a lot
more was expected and intended as far as the transfer of results and lessons and the replication of
the TEST process. The two publications relating to the TEST process are comprehensive and
valuable but there is still a need for a summary report for the entire project which captures all of the
lessons from the 17 enterprises as an overall case study for TEST. Also, there is still a need to
disseminate the overall success and value of the TEST process to more enterprises within the
project Countries. The original Project Document Output identifies the following activities:
· Hold regional seminar for nine countries (including the five participating countries) to
present results and determine interest/capacity building needs for undertaking TEST
programme in other Danubian countries. This has been achieved through the ICPDR two-
day regional workshop.
· Prepare requests for technical assistance as needed (this has been done in the case of
Bosnia).
Less has been achieved at the level of those Danube countries outside of the project. The Project
Document had intended the following activities to occur:
· Identify team members in four other countries with input from country programme
coordinators, NCPC/PPC, UNIDO national focal points and UNIDO staff. This was
achieved in Bosnia
22
· Hold one week training course on TEST approach using selected national experts from the
five participating countries. Croatian CPC experts undertook this training in Bosnia
· Match one team from each participating country with new team from another country.
· Mission to each country to advise on enterprise selection and application of TEST approach.
The Project Coordinator confirmed that this had been addressed during the ICPDR
workshop.
· Provide limited technical advice as requested by new teams from the teams in the five
participating countries.
It is the opinion of the Evaluator that this Component was somewhat optimistic within the
timeframe and funding of the original project design, and in view of the need to complete TEST
procedures at 17 enterprises first before the lessons and best practices could be consolidated into
results for dissemination and training. To this effect, a somewhat lower score and rating for this
component is a reflection of over-ambitious project design rather than an implied criticism of
project management and implementation, or of stakeholder commitment and effort. This is
reflected in the rating which falls just within Good, Effective Delivery rather than Borderline.
The accumulative SQA score for the Project Outputs and Activities comes to 87.6% or a rating
equivalent of 4.5 which amounts to Excellent Delivery for the Outputs and Activities.
4.1.2. National Delivery The Demonstrations
The Evaluator visited three of the 5 countries involved in the TEST Project. Selected stakeholders
from the other two countries were asked to respond to a questionnaire (see Annex IV). The
following descriptions and comments relate to the enterprises in the 3 countries visited during the
mission:
BULGARIA:
There were 3 demonstration enterprises selected for Bulgaria, in alcohol production, fish processing
and textile production. The primary incentive for the alcohol production enterprise was the demand
from government to clean up their wastewater discharge which has been a cause of concern for
some 25 years. The other two enterprises were conscious of the need to get ISO 14000 accreditation
as Bulgaria expects to sign the pre-accession agreement to the EU in early 2007.
In Bulgaria, the project had a poor start due to selection of an inappropriate counterpart. The project
only started to delivery effectively when the Technical University of Sofia was selected as the
counterpart. This institute was chosen because of its staff capabilities and expertise in industrial
production and engineering. The evaluator was unable to visit any of the enterprises in Bulgaria but
did manage to attend the final dissemination workshop where presentations were given on each
enterprise, and was then able to meet with company representatives. The Evaluator also spoke in
detail with the National Counterpart team at the Technical University.
The selection of the companies followed a step-by-step approach. In the first place the project was
advertised among the local industries in order to illustrate the main objective, the methodology, as
well as the expected benefits and the requirements for participation. Secondly, enterprises that
expressed an interest were contacted and visited by the local counterpart. During these visits a more
detailed explanation was given of the TEST Project targets, objectives and expected results.
Finally, enterprises were selected on the basis of their financial viability, magnitude of
environmental problems and management commitment.
23
The Counterpart Institute decided to include energy efficiency and energy auditing into the TEST
process as they had a recognised level of expertise in that field. The Institute staff received several
seminars from outside experts who explained the process of CPA, EST, EMA and SES, etc and
provided general technical assistance and training. The teams from each of the enterprises were
introduced and mixed with each other to exchange ideas and share thoughts. For the EMS module
(carried out at the textile plant) they had assistance from the CPC in Slovakia and for the EMA they
were assisted by the Hungarian CPC experts.
Zaharnia Zavodi AD, a sugar and alcohol production company, carried out all of the TEST modules
except the Environmental Management System module. The company was established in 1913 and
is the biggest sugar and alcohol production company in Bulgaria. Prior to their involvement in the
TEST Project they were unable to sell their alcohol products throughout Europe due to quality
constraints. Furthermore, they had been threatened with closure by the government unless they
cleaned up their waste discharges. The TEST Project was therefore a very timely piece of
assistance for them and the attention within the TEST project was concentrated on the alcohol
production unit which had the greatest environmental problems. The main waste from the process
was the slop from the beer production which was discharged directly into the river, plus liquid
wastes from the distillation process. The energy (thermal) losses were found to be enormous,
especial from the slop and the cooling water. These losses were also going directly into the river.
The Company had no recycling for cooling water, which also went directly into the river.
Through the CP Assessment it was possible to identify the aim of the TEST process at this
enterprises as a) a decreases in organic pollution though reduction of the slop water flow and its
high COD load in to the Yantra river, b) reduction in thermal pollution and re-use of wasted heat
energy, c) reduction of water losses through re-cycling and re-use of cooling water. Various options
to clean up the discharge process were studied including thermal, membrane, biological and
combined processes.
The EST Assessment looked at the investment plans and priorities of the company and its owners,
taking into consideration the results and recommendations from the CPA. This led the EST to focus
on changes to raw materials along with appropriate changes in pure alcohol production technology.
As a consequence the project identified a viable option for the reduction of organic pollution as
being the replacement of molasses with grain in the alcohol production process. As a consequence
of the TEST project, the company made a decision to change from sugar-based to cereal-based
alcohol production which reduced the chemical oxygen demand in the slop and produced a much
higher quality of alcoholic product. These changes also create a value-added by-product, as the slop
could now be used as animal feed. The water used in the process is also recyclable. Step by step the
company has gradually introduced these more cost-effective and environmentally appropriate
changes in process and technology, and is looking for funding to make further improvements. The
total cost for the modifications and reconstruction are estimated to be around $6 million (As of
2003). The company is hoping to introduce more TEST approaches. They have 6 such plants in
total and are planning to build the TEST process into their new construction plans.
The EMA process undertaken at Zaharnia Zavodi AD, was of considerable value to the company
and as a demonstration of hidden losses. Prior to the project, the non-product output (NPO) costs
(effectively the cost to the company to produce waste products) were not considered by the
company in their environmental and financial auditing. In reality, having identified these costs it is
their magnitude and the enormous losses to the company of these `hidden' production costs which
are the greatest incentives to improvement in cleaner production and elimination of waste by-
products. This is discussed further below under the dissemination workshop results.
24
Another enterprises, Slavianka JSC was established in 1948 and is one of the biggest fish
processing companies in Bulgaria and is situated in the town of Bourgas. The production process is
divided into sterilisation and metal packaging. The sterilisation department is equipped with a
production line for the processing of frozen and fresh fish. A second department provides the
necessary metal cans for packaging. The steam boiler house, providing the steam for the production
processes, and the wastewater treatment installation are also part of the facilities. The equipment in
all sections is old and needs replacing and modernising in order to reduce the energy expenses and
to meet the growing requirements related to the treatment of organic matter in the wastewater.
Slavianka also followed all of the TEST procedures except the EMS. This helped them to identify
the extent of their pollution problems. The main problem was fish waste from cleaning, washing
and defrosting, as well as the cleaning of the packing cans, along with an excessively high
consumption of water and energy. The development of the CP and Energy Efficiency opportunities
under the TEST Project therefore focused on three main directions a) water consumption and
related wastewater flows, b) reduction of pollution load and the volume of effluent generated, and
c) improvement of energy efficiency.
The CPA at Slavianka gave rise to two specific requirements 1. the need for a new de-frosting
mechanism and technology ( a major source of wastewater pollution) and 2. The need for a more
efficient energy production unit and boiler house for steam production. The most significant CP
measure that has already been implemented is the installation of the new defroster (a type C
option). This has reduced the electricity, water and steam consumption significantly (the last two by
60%), which has also led to a decrease in the wastewater from the defrosting process. It is
important to note that before TEST they had no idea how big their waste problem was. After TEST
they had reliable figures as well as options for mitigation.
The EST Assessment for Slavianka focused on the reconstruction and modernisation of the basic
production line. Some innovative solutions were used to adapt the old equipment to decrease its
environmental impact. The company is considering the construction of a new boiler house with up-
to-date equipment and greater energy efficiency. Implementation of this measure will give a
significant reduction in hazardous emissions, a reduction in transfer heat losses and reduced fuel
consumption.
The third enterprise selected in Bulgaria was the textile industry, Yuta JSC, which was established
in 1929 and specialised in non-woven textiles. The company uses a large number of technological
methods for textile production. The TEST project concentrated on three production lines 1) thermo-
bonding of non-woven material, 20 adhesive bonded non-woven materials, and 3) anti-dust masks.
The three distinct processes with this area of textile production included thermal bonding, adhesive
bonding and cutting. These processes produce various types of waste including textile fibres with
adhesive bonding substance, waste from cleaning of vessels and other various water discharges to
do with cleaning, cooling and chemical production. The main environmental impacts associated
with the production of non-woven materials at Yuta JSC are the solid wastes and wastewater
discharges. Their biggest problem is latex waste which has been traditionally discharged into the
river. Furthermore, the energy audit of the energy facility revealed significant losses of heat and
electricity resulting from boiler house operations, the distribution pipelines, and the equipment in
the production units.
The CP Assessment set as its goals a) the reduction of solid wastes leaving the plant, b) the
reduction of wastewater released into the Russenski Lom river, c) an increase in energy efficiency
in the generation and distribution of heat and electricity, d) reduction in emissions for the boiler
house, and e) an overall increase in operational efficiency.
25
The EST Assessment identified four EST modifications, two related to material flows (a new
rolling system and a new waste press) and two related to energy conservation measures (a new de-
aerator and a new steam boiler). The new `S' shaped rolling system that reduced solid waste in the
production line was implemented in 2003. The rest were all being considered for a more detailed
assessment including technological appraisal and costs evaluation.
Under the EMA module, the following goals were set at YUTA JSC:
· Training in the field of management accounting
· Development and implementation of a simplified EMA system
· Training for investment project appraisal (including environmental projects)
· Integration of Discounted Cash Flow project appraisal tools into the management practices
of the company along with on-site consultancy assistance for the appraisal of proposed
environmental investments
As the value of the non-product output was the main environmental cost of the enterprise, the effort
of the consultants was directed toward identification and proper calculation of these costs. As with
the alcohol production factory, it was clear that reduction in non-product costs was the primary
incentive for CP and EST improvements.
The EMS module was introduced in Yuta JSC for the entire company and was integrated with the
existing QMS (Quality Management System). The results of the development of the EMS system
can be summarised as follows:
· All elements of an EMS were developed and many of the procedures and instruction were
amended and extended from the ones available in the existing QMS
· Environmental policy was developed, approved and distributed
· EMS training was provided for top management, internal auditors and a certain number of
other relevant employees
· A register of legal and other requirements was filled in accordance with a newly developed
procedure
· A new procedure for internal and external exchange of information was developed
The integration of the CP and EMS modules was an important part of the demonstration of the
TEST Project. CP was viewed as an integral component of EMS and the CP action plan as part of
the EMS. While EMS tended to focus on legal compliance however, CP went further than that and
added efficiency and substance to the environmental performance of the enterprise. EMS added
value by putting formal procedures in place for continual improvement and sustainability.
So, this company undertook all of the TEST procedures including completion of the EMS. This
decision was made based on the fact that this was the only company out of the 3 selected Bulgarian
enterprises that already had ISO 9000 Quality certification and therefore had a quality management
strategy in place already. The EMS was integrated into the company's existing procedures for
quality management. The company is now awaiting further auditing and certification. The company
wants to integrate the quality, environment and health/safety certifications (as a cost-saving
exercise) and get all ISO certifications at once. This is expected to happen later in 2005. It was
mentioned to the Evaluator that cross-communications within the company and between the
Quality Control and the EMS teams could have been better.
All of the Bulgarian demonstration enterprises have made significant improvements under type A,
B and C. Slavianka (Fish processing plant) has undertaken all type A, B and C improvements,
while Zaharni Zavodi (alcohol production) has achieved all type A, most type B and 1of 3 proposed
type C improvements. Yuta (the textile factory) has undertaken most A, all of B and 50% of type C.
26
The experts at the University feel that the whole TEST process and the project implementation has
been enormously educational for all involved.. They have all gained new and valuable knowledge
and have developed a network of colleagues and consultants within the region which they will
maintain and build on.
One recommendation from the Bulgarian CP team was the need for an end-of-pipe technology and
methodology manual. TEST does not really address the need for end-of-pipe technology, yet
sometimes end-of-pipe solutions are the only practical ones both from the point-of-view of cost but
also because the pollutant cannot be removed anywhere else within the process. An example where
this applies is in the case of coal. Bulgarian coal is high in sulphur content and the only options for
reducing pollution emissions is to deal with the actual flue gases (end-of-pipe). Bulgaria cannot
afford to ignore its own coal resources in preference to imported coal but there is no way of
reducing the sulphur content and end-of-pipe technology is the only solution. So, they would like to
see Best Available Technology (BAT) options for end-of-pipe treatment included under TEST.
They also wanted to emphasise that companies need independent and impartial advice on any
TEST products. Commercial companies with innovative techniques will always maintain that their
product is the best. The companies need a source of reliable advice which is where the CPCs and
similar institutes have another important role to play.
The Team at the University also noted that the only way the TEST process would get implemented
in the region was in direct response to the EU directives and IPPC (Integrated Pollution Prevention
and Control) requirements. Penalties for wastewater discharges are far too small to act as a
disincentive (it is cheaper to pay the penalty than to change the process). In view of the obvious
success of the TEST MSP, they would like to see further assistance from GEF in the development
of type A and B improvements, and to assist in identifying realistic loans for investing in type C
improvements.
The National Counterpart noted that during the EMS process they had assistance from the
Slovakian CPC. They undertook a mini-audit of the company's production process. The
recommendations from this audit were passed on to the management who were so impressed that
they adopted them with the result that company production efficiency improved significantly
(including waste reduction etc.). The company was very proud of this.
Although there is currently no CPC as such in Bulgaria, there is interest from the Swiss
Government to finance the creation of a CPC. Obviously it would make sense to try and include the
expertise available at the University in such an enterprise.
As next steps, the Bulgarian team felt that there was a need for greater dissemination of results and
that effective presentation to other companies would attract a lot of interest in the TEST process.
They also felt that there is a need to identify investment capital for type C (as well as some of the
more expensive type B) improvements. They were very keen on the possibility of exploring soft
loan options and suggested that evidence of type A and B improvements could be used as criteria
for gaining access to soft loans for type C. They agreed that it would help a lot if GEF funds could
be used to develop the proposals for soft loans as it takes quite a lot of detail to be able to satisfy
lenders of the feasibility of the proposal and that there loans are secure.
All companies in Bulgaria are now required to do energy audits every 3 years as a new government
requirement. It would be fairly easy to expand these into TEST procedures. The team felt that there
is a need to raise awareness and support within the government regarding TEST and its national
benefits as well as the benefits at the company level.
27
One criticism of the project focussed on the funding and the problems experienced in the exchange
rate. Because of changes in the exchange rates between signing the Project Document and
disbursement of funds, Bulgaria lost nearly 30% of its expected financial support. This would not
have happened if the original figure had been in Euros rather than Dollars as Bulgarian currency
(and most of the European currencies) are now fixed to the Euro. Generally the felt that the funding
was a constraint in view of the large number of workshops and seminars required. The timing (as
per the workplan) was sufficient however. They did feel that a longer preparatory phase would have
helped in allowing more time for selection of companies and for the initial review.
Whilst in Sofia, the Evaluator was able to attend the final project national dissemination meeting.
The Dissemination meeting was attended by over 30 people consisting of representatives from the
national counterpart institute and consultancy team, the participating demonstration enterprises, 7
non-participating enterprises, 3 ministry representatives and 4 academic representatives from
technical departments of different Universities. The meeting was given an overview of the aims and
objectives of the TEST project by the Project Coordinator which gave a strong emphasis on
increased productivity and revenues through reduction in waste products, improved
competitiveness and general reduction in production costs, all with the added benefit of reduced
discharges in line with EU and IPPC directives. The presentation finished with a concise summary
of what every country had achieved. It was clear that very few industries were aware of the IPPC
requirements and BAT potential at the beginning of the Project, but that awareness had been raised
significantly as a result of the GEF initiative.
Following presentations dealt more specifically with how the TEST tools were used in Bulgaria.
This explained the process from Initial Review, through Cleaner Production Assessment (and
Energy Audit), Environmental Management Systems, Environmental Management Accounting,
Environmentally Sound Technology Assessment and Sustainable Enterprise Strategy. It was noted
that one of the significant outputs from the EMA module was the change in management attitude
and culture, with a shift away from intuition driven investment more toward a proper investment
study and appraisal including evaluation of environmental impacts in monetary terms.
One of the final summary presentations gave an excellent explanation of the Environmental
Management Accounting process and the understanding of cost-effectiveness and incentives arising
from the TEST process. EMA is about finding the hidden `factory' within the production process.
EMA demonstrated that 60% of the costs of production at the Zaharnia Zavodi AD, the alcohol
factory, was going into producing wastes which were not of value and not re-used and in fact
required more investment to deal with them as a problem. Only 30% of the production costs were
actually realised as alcoholic product. This is very important concept to get across to the company
as this clearly demonstrates an incentive. They need to reduce their production costs that are going
into non-product outputs (i.e. wastes). Only 1.5% of the company's costs was in fines and penalties
so there was no incentive there for change and these fines and penalties would have to be raised
enormously in order to generate such an incentive.. But there is a massive incentive for the
company to reduce the non-product output. 72% of this non-product output was slop so this was the
obvious area to target first under TEST. The presentation then demonstrated the same `hidden
factory' producing non-product outputs (NPO) within the textile enterprise, and identified this NPO
as being primarily wastes. So EMA demonstrates to the companies that they are actually paying
huge sums of money to produce waste products of no value and of significant concern, rather than
valuable product. EMA uses TEST to show how this can be resolved through investments that are
substantially less than what the company throws away on NPOs.
So, the company incentives that are highlighted by EMA are:
· A reduced amount of investment and cost to the company in producing waste products
· Change of loss to profit if can find way of using or recycling wastes
28
· Improved quality of product
· Increased potential for market as higher quality linked to environmentally-friendly products
· Consequent avoidance of fines and penalties
· Improved company profile
And the disincentives are:
· Only the initial investment (often small at the outset for CPA type A and even type B
investments) but even this is balanced by the returns over time
In reality, this can be seen as one of the most important demonstrations of this Project.
Details of the presentations are available in the joint UNIDO/Technical University of Sofia
publication `An Integrated Approach to Environmental Management in Bulgarian Enterprises The
Experience from the Implementation of the UNIDO TEST Project in Bulgaria'.
CROATIA
Four demonstration enterprises were selected for Croatia specialising in meat rendering, meat
processing, pesticide production and sugar production.
The CPC in Zagreb, Croatia provided a useful example of how the TEST process was implemented
within a country and within the enterprises. Once suitable enterprises had been identified, the
standard process for involving enterprises was for the relevant Ministry to approach the company
and for the company to agree to sign a contractual letter that committed them to the TEST process,
and to make their resources available (particularly human resources) for the duration of the project.
The next step was introductory training for top management to explain the project, provide an
overview of the TEST methodology, to discuss how it related to IPPC and ISO 14000, and to
summarise the benefits of the TEST process to the company. Following this, an initial review of the
company's production process was undertaken and a TEST team was identified within the company
for further training. Standard CP module training was first given to the Project Team within the
enterprises (these were usually people from the production process such as technical directors or
engineers, and occasionally people from the finance department of the company). The methodology
for the standard CP training included 3 collective training sessions in Zagreb for all of the
companies lasting between 2-3 days each time. In between these sessions the CPC worked with the
in-company project team on their specific concerns and needs, undertake an analysis of production,
material use, energy balance, option generation, etc). This process took between 6-7 months. At the
end of this interactive training process, the project team and the CPC prepared a final report on the
proposed methods for cleaner production improvements. This was then presented to the top
management, and then an overall presentation was given to a stakeholder workshop of company
representatives (from all national demonstration enterprises), ministries, the Advisory Board
members and the Project Coordinator. While this CP process was underway, and in parallel with it,
additional analysis was undertaken to see which staff and which areas of the company would be
appropriate for Environmental Management Accounting. CPC staff and Lead Consultants were
given training in COMFAR (Computer Model for Feasibility Analysis and Reporting - UNIDO's
software for company financial appraisal and analysis of investment projects) at UNIDO
Headquarters in Vienna along with training on EMA and EMS. The all of the CPCs from each
country and the demonstration enterprises selected for EMA undertook 2 days EMA training in
Budapest and a further 2 days in Hungary as the CPC in this country has considerable experience
with EMA procedures. Not all enterprises were selected for EMA as this was a fairly innovative
addition to TEST. One company per country was selected to demonstrate the EMA procedure.
29
In Croatia, the CPC then started to work with the company chosen to demonstrate the EMA process
(in this case Herbos, a pesticide producing plant). Throughout this process the Croatian CPC had
constant dialogue with and support from the Hungarian CPC. An EMA expert from the Hungarian
CPC visited Zagreb to provide on-site assistance at the request of the Croatian CPC. The end result
of undertaking the EMA module was a final report prepared jointly by Herbos and the CPC. This
was also used as a case study in TEST Project publications.
While the EMA demonstration at Herbos was underway, an EMA module was being implemented
at Gavrilovic (a meat processing plant) which had a made a commitment to undertake this module
following many discussions regarding the process and methodology. An introductory session was
carried out for the top management. The CPC used a Lead Consultant who was experienced in
EMS, At the same time as these EMA and EMS modules were being implemented in various
companies, the EST module was initiated as part of the integrated package of modules. The CPCs
received training on EST (at UNIDO Vienna) and how it related to IPPC and BAT (as all of the
EST recommendations have to be in accordance with IPCC and BAT). The process in-company
was similar to the EMA and EMS modules. Workshops were held for top management and then for
the in-house project team. These were not necessarily the same people who were involved in the
other modules, but generally the CP team participated in the EMA and EMS also (some of the
results from the CP were necessary to feed into the EMA, and were also important for the EMS).
So every enterprise went through the Initial Review process, the Cleaner Production Assessment
and the Environmental Sound Technology review. Selected companies also did the Environmental
Management System and/or the Environmental Management Accounting modules, and the
Sustainable Enterprise Strategy. The CPC stressed the fact that it was important to bear in mind that
all of these modules are integrated and related to each other. The final module that ties everything
together is the SES which integrates all of the environmental assessments and reviews and the
various modules into one overall management and business strategy for the companies. All of the
National Counterparts attend a workshop on SES in Vienna with training given by UNIDO.
In the case of Croatia, the CPC felt that the selection procedure for the enterprises was one of the
weaker links in the TEST process and they would have preferred to see a more detailed analysis
stage before selection although they recognised that time was a constraint. Again, this reflects the
need for a preparatory phase. However, In Croatia they were impressed with the overall TEST
methodology and the design of the approach. Everything that needed to be done (Project activities)
was clearly defined by the Project Management and there was always a fast and detailed response
to any requests for clarification.
The CPC is now finding that instead of having to be proactive in selling CP and TEST to
companies they are now being approached to assist. They feel the modular approach is important in
this respect as not every company needs the entire TEST process and it needs to be flexible to meet
each individual company' requirements.
The Evaluator was able to visit Gavrilovic, a Croatian company specialising in meat processing and
packaging. Gavrilovic lies some 100 km southwest of Zagreb in an area that suffered fairly badly
during recent conflicts (1991-1995) over the splitting up of Yugoslavia. The company has a long
history (over 300 years) that has seen many changes over the last 80 or so years due to alterations in
the political climate. The company specialises in pig and cattle breeding, slaughtering, and meat
processing and has several brand names that are well known on the international market. The
original owners (who were forced into exile when Yugoslavia became a communist state) are now
back in ownership and running the company very successfully. The company employs over 2,000
people and is responsible for 30% of the domestic budget for the region of Petrinja. The company
aims to produce its own energy on site through a process of co-generation.
30
The company's production levels were severely crippled by the conflicts of 1991-1995 and
production fell from a daily capacity of 200 tonnes to only a few tonnes in 1996. Now they are back
up to 70 tonnes daily with hopes to expand again.
The company became involved when they were approached by the State Directorate for Waters, the
Ministry of Economic Affairs and the CPC. Even before the TEST project came along there had
been a working relationship between Gavrilovic and CPC and some earlier participation in cleaner
production. So, to some extent, part of the selection process was that the company was a known
entity and was willing to work with CPC.
The TEST project has helped Gavrilovic to assess its production approach and its waste treatment
and to come up with a modular and sequential strategy for improvement. TEST has assisted
Gavrilovic in introducing type A housekeeping approaches within its production process. It has also
defined a modular structure for waste treatment units projected over the next 15-20 years for a daily
capacity of not less than 200 tonnes of processed meat. The intention is to introduce these modular
units as they are required and to plan investment around these projections. Each unit had to be
defined in such a way that would enable the company to make a qualified unit investment decision
at the proper time (i.e. increased waste treatment needs as a result of increased production, unit
operational cost, unit investment cost, time of unit construction, and time of introduction into
operation). In addition to the CP training, the company has also installed water meters around the
production plant and the staff have to record water usage in each area. This has raised the
awareness of resource efficiency and waste control on the production floor. Now they can calculate
every month the quantity of water saved per quantity of product. Working with the TEST team and
the Lead Consultant, Gavrilovic has succeeded in reducing it water consumption by 14%.
As a result of the TEST assessment process, Gavrilovic is concentrating now on an integrated waste
treatment facility. Production wastes come from agriculture (Gavrilovic raises some of its own
animals), cattle and pig breeding, the slaughterhouse, meat processing, and general on-site wastes.
The overall waste treatment concept for the company involves energy production (steam and
electricity co-generation). Energy is produced within the incineration unit by burning animal waste
and sludge from waste pre-treatment and biological treatment facilities. This allows for energy
production from renewable resource and efficient treatment of potentially harmful animal wastes.
The EST module has focused on this option of using biogas for the co-generation plant at a total
investment of US$5.2 million. This would allow for some treatment of wastes at the production site
along with a source of renewable energy. The company has already been talking to suppliers about
EST, particularly for biogas, prior to TEST but the costs involved were proving to be very high so
they felt they needed some independent assistance and were therefore able to turn to TEST and the
CPC for advice. The Concept of biogas had been available for some years, but TEST had expanded
this concept to one of integrated water treatment. The introduction of co-generation of steam and
electricity revolutionised the company according to the Production Manager. He is constantly
trying to raise awareness of the cost-efficiency and waste reduction potential of co-generation and
the use of biogas with other companies and with government. Other companies are starting to give
serious consideration to co-generation and are looking at the biogas options also. It is a very simple
process to switch from natural gas as fuel, across to biogas. This system has been pioneered as a
workable strategy by Gavrilovic in cooperation with TEST. The Project provided the incentive and
the tools for the co-generation and biogas technology to become part of the company's new
business sustainability strategy. Although the necessary investments have not been made as yet, a
business plan as been prepared and the investment will pursue a step-by-step logical approach as
and when investment funding can be identified.
31
Every new employee at Gavrilovic has to have an awareness and training session with the
company's own in-house environmental specialist and is instructed in company policy on
environmental issues. Basic training on environmental concerns and TEST has been given to all
production managers. Furthermore, the company has developed an internal training policy for
awareness at all levels (e.g. the Director of the agricultural sector has recently done a Master's
degree in Environmental Management).
The management at Gavrilovic were very supportive of TEST and called it a `win-win' situation as
it helps to resolve their resource use and waste production inefficiencies whilst making savings for
the company overall. A major result for both Gavrilovic and the Project is that now the company
has a water permit that meets the IPPC Directives even though this is not currently required by
Croatian law. The company management was a little concerned about the lack of involvement and
input from certain pertinent government Ministries, particularly in relation to environmental
protection. They felt that not all government departments were supportive, nor did they appreciate
or recognise the benefits of TEST. On the positive side, cleaner production is now mentioned in the
new Croatian laws on waste management (2004) which will become obligatory for all companies.
Generally, Gavrilovic were very pleased with the support that received from both the CPC, the
Project Coordinator and other consultants. They felt that their main incentive for getting involved
was to reduce their wastewater discharge load, to produce biogas for co-generation and to thereby
minimise operational costs. A very real financial incentive was, of course, the fact that GEF was
covering 80% of the costs of the TEST process as part of the demonstration.
Originally, the work schedule for the evaluation had planned for the Evaluator to also visit a sugar
processing company (IPK Tvornica Secera Osijek) but weather conditions and the distances
involved were a cause for concern. Instead it was decided to interview the Lead Consultant in
Zagreb who had worked with the enterprise on behalf of the CPC. The consultant (a chemical
engineer) was from Slovenia but had previous experience of working with and advising sugar
companies and had undertaken an energy audit in this particular company. Therefore he was
familiar with their problems regarding waste and emissions. Initially the consultant visited the
company with the Head of the CPC to meet with the staff. The company had already undertaken
some cleaner production improvements (machines to clean the sugar beet while reducing water
usage and managing soil wastes). The company's first need was identified to be a mass and energy
balance assessment. This represents a detailed study of the industrial process. The results were then
discussed with the company. From this M&EB assessment it was possible to determine where
waste was being generated and in what quantities.
The assessment identified two main problems, 1) from the wastewater and transportation of the beet
cleaning process and, 2) from the consumption of water and production of condensate during
process. The Lead Consultant then looked at proposed solutions and came up with a technical and
economic evaluation. The recommendations to the company include first minimising consumption
of raw materials and energy, and then undertaking some end-of-pipe solutions using settling
lagoons. The water used for washing the beet ends up with a very high soil and sediment content. If
this can be placed in holding basins then the water can be treated and the sediments settled out.
There were two sources of water input identified from the process. From the river and from the
sugar beet processing itself (condensate from the high water content of the plant). By recycling the
water from the cleaning process and condenser, and using it in the cleaning process instead of
freshwater it was estimated that it would be possible to cut down significantly on water abstraction
from the river (by 60%). The benefits of reducing the freshwater intake could be seen to be 1)
reduced pumping costs and water fees, 2) reduction in use of chemicals (which are necessary to
treat the freshwater before it can enter the process but are not necessary for the condensate and
32
recycled water), 3) a higher water temperature can be maintained using recycled water which
improves the cleaning process, 4) an overall reduction in wastewater discharges due to recycling.
The company now has all the information that it needs on equipment suppliers and costs, along
with details of savings and benefits to the company. They now need to go ahead in order to comply
with the new environmental regulations. They are seeking a policy decision at the top management
level, and are also looking for funding to make the investment. One key problem that they also still
have to resolve is formal company privatisation and ownership. They also need permits for
construction.
There are other opportunities for improving processes to achieve cleaner production and more
environmental sound technology within the company. As yet they have not looked at the utilities
side of the company's process (steam plant, freshwater treatment plant, compressor, etc). All of this
produce pollution and could be improved and cleaned up.
The consultant could see clear options to transfer the lessons for this demonstration to other sugar
beet processing companies in Croatia (3) and throughout the Danube river basin. The consultant
emphasised that it is important to undertake the mass and energy balance studies first.
The consultant noted that the CPC in Croatia was doing a good job and that Croatia was fortunate
in having such a Centre to provide guidance to industry in these matters. One big problem he sees is
the lack of investors for the high-cost technological upgrades necessary in some countries to meet
new EU Directive requirements. One important function of the CPC is to identify the benefits
(through the TEST process) and to bring the investors and the enterprises together.
SLOVAKIA
The Evaluator visited ZOS TRNAVA, a Slovakian Company specialising in repair and re-
conditioning of railway rolling stock and currently celebrating its 80th Anniversary in business.
They were originally the largest workshop for Czechoslovakia undertaking 48% of repair work for
all railway wagons. This constituted 17,000 wagons repaired per year. When Czechoslovakia was
split into 2 separate countries ZOS lost a lot of the market and the number of wagons on the railway
system fell significantly by one-third. In 1994 the company was privatised and had to re-think its
marketing approach and production methods. Today the company specialises in the repair and
refurbishment of wagons for customers throughout Europe, with the majority of customers being
from the private sector. ZOS estimates that it now has 50% of the European market. They also
specialise in building and re-designing wagons to suit specialist purposes and to fit specialised
trailers and bogeys. They have managed to become certified and approved for work on railway
stock for Austria, Switzerland, France, Poland and the Czech railways. This certification is a
complex process but they now have official trust of these countries and railway lines to produce
reliable wagons (this even extends to the US market). In 2004 the company repaired over 2000
wagons for foreign companies (each with their own requirements and standards). The total repairs
for 2004 came to over 5,000 including Slovak rolling stock.
The company employs around 1,200 personnel with an average monthly salary of Sk20,900
(US$743). This is in fact about 25% higher than the average national salary. The company is
generally optimistic for the future. There is currently a dramatic expansion and development of
industry in Slovakia and the economy is booming. Furthermore, as railways in the region become
more liberal and privatised then the company's competitive prices will start to give them a keener
edge on the market.
The company has taken a policy decision to invest a substantial amount into improved
environmental practices. With the opportunities offered by the TEST project the company has been
33
able to jointly develop improved policies on both environment and product quality. The great
advantage and incentive provided by the TEST demonstration has been the opportunity to comply
with the ISO 14000 directives from the EU. The Company recognises that as part of its overall
production process it is a `waste-producer' and that the nature of the repair and refurbishment work
on wagons produces pollution. Last year the company logged the production of 32,000 tonnes of
waste, 24,000 of which (3/4) was recycled. Most of the company's waste is metal scrap. Until
recently the environmental record of the company was a disaster (photographs of the processes
shown to the Evaluator confirmed this!). All of the wagons and rolling stock that come into the
factory are cleaned first. This includes the cleaning of tanks that carry chemicals. This used to be
done by hand with enormous waste and discharges and inevitable health implications. Recently this
has been cleaned up by installing an new semi-automatic washing facility at a cost to the company
of Kr 22 million (US$782,000). This included a lot of additional modifications and extra
technology to ensure environmental compliance. The company noted that in order to make such
environmental improvements cost-effective it is a rule-of-thumb that it is necessary to double
overall production at the same time.
Their biggest environmental problem was the paint-shop. New legislation imposed by the EU
regarding VOCs (Volatile Organic Compounds) required them not only to clean-up the paint-shop
operations but to start using non-VOC paints. The cost of building one new paint-shop to comply
with the new standards was Kr 5.5 million ($195,500). Previously, old paint was dumped or buried
but now it is put through a thermal oven for cleaner incineration. This oven system cost kr 5 million
($178,000). In 2007, yet new regulations come into force which will require the company to
upgrade their paint-shops again. They estimate that the cost to upgrade the entire paint line in 2007
to meet the new laws will be in the order of Kr 250 million ($8.9 million).
The company also had to upgrade its HydroMat system which is used for cleaning and degreasing
oily equipment. They contracted a company to explore best available technology with an emphasis
on cleaner production and environmental efficiency as well as water efficiency. At the time BATs
were not defined for this particular process as it is too specialised, so the company urgently needed
external assistance
Over the last 5 years, the company estimates that it has spent some kr550 million ($19.5 million) on
upgrading and replacing processes and technologies to comply with EU IPPC directives and ISO
140000. Unfortunately, ZOS does not qualify for EU subsidy to assist in this process due to its size.
The company finds it unreasonable that foreign companies (such as car manufacturers) can come to
Slovakia and immediately become eligible for Slovakian government funding and subsidy but a
local company is not entitled to any such funding.
So, TEST was very important to ZOS in providing them with an affordable mechanism to meet the
ISO 14000 requirements and to deal more efficiently with their waste products (this included
general small-scale factory floor waste for which they contracted a smaller company to handle).
TEST has helped them to update their policies on waste treatment and discharges and to make
necessary organisational improvements within the company structure. The new processes being
adopted have the added and distinct advantage (and attraction) of being much more efficient and
cost-effective as well as cleaner. The company also now has a waste and water monitoring
laboratory. The company also addressed energy efficiency issues with the assistance of the CPC
and upgraded heating and lighting processes which gave a 40% improvement in energy efficiency.
The management at ZOS was keen to emphasise that the TEST process was not just about buying
or upgrading equipment but also importantly about changing people's attitudes and understanding.
A very recent management review of the EMS system had focused on the need to further improve
waste management and increase individual awareness. One of the important TEST Project
34
contributions was that the company sent all its shop-managers on a cleaner production training
course for 5 days run by the national CPC. The fact that the company allowed its management team
away for 5 days training shows the commitment from the company and the trust in TEST and the
CPC. Furthermore, TEST undertook training of lower management on-site and a further 1 day
training of top management at their own request. Middle management received several days
training in total and line management were also trained on site.
The company felt it was important to stress that it was not enough to implement technical changes
but it was equally as important to adopt a systematic and integrated approach to TEST that linked
into the company's existing Quality Management system. However, the QMS was originally a
process approach whereas EMS has more emphasis on technical knowledge and improvement.
Clearly a more integrated and holistic modular approach was necessary which TEST provided.
ZOS was finally certified for EMS in 2003.
The company noted that throughout the process there was no obvious input or involvement from
the government side. The government adopted new legislation but provided no real support or
guidance to industry. This support and guidance effectively came from TEST. ZOS felt that other
companies are probably aware of the requirement for cleaner production and improvements in
environmental compliance but they do not know what action to take or what tools are available to
assist them.
The company noted that the presence of the TEST project and the direct involvement of the CPC
was important in expediting decisions and speeding up the overall process of cleaner production.
The Project actively persuaded the company directors to move ahead with changes. In fact, the
company was very enthusiastic at an early stage as the offer of assistance from the Project was too
good and too tempting to ignore under the existing climate of EU regulations and directives. They
admit that the 80% contribution by the donor to the project activities was a catalyst in volunteering
themselves as a demonstration enterprise. If they hadn't jumped at this opportunity when offered
they would now be facing more serious problems and time-constraints. The TEST project speeded
up their conversion to cleaner production by some 2-3 years. The management had not originally
been aware of the extent of the environmental problem, the pressing need for cleaner production
and compliance with directives, or the availability of support from GEF. Once this was made clear
they were immediately keen to take part. Now no-one in management regrets this decision and they
have no hesitation in promoting TEST to other enterprises.
Now that the project is completed the company intends to maintain its compliance with directives
for obvious reasons of cost-effectiveness and market competition within Europe. The original
incentive therefore was GEF funding plus the need to meet the EU directives. A new incentive has
been created since in that the shareholders see a significant improvement in company production
and a cleaner image. Furthermore, the company owner is so impressed with what has been achieved
he has made it his policy that ZOS should be a model of cleaner production in Slovakia and the
region. Such personal interest and full engagement by top management is very important to the
continued success of TEST and the company.
As far as next steps within ZOS are concerned, they have identified longer-term higher cost
technological improvements but these require a high level of investment. In this respect, some
assistance with identifying investment possibilities and soft-loans would have been helpful. If they
could find some way to attract low interest investment then it would be much easier for them to
persuade their shareholders to take the next steps (Type C) in cleaner production.
4.1.3 Summary of Demonstration Enterprise Delivery
35
Tables 5-9 provide a summary of the project delivery for each enterprise.
Table 5 presents the list of type A and type B options identified for each enterprise along with the
status of their implementation. As a result of the TEST project, 91% of company-approved Type A
options and 88% of Type B options have already been implemented at the 17 enterprises. A further
$3 million worth of Type B improvements were approved for future investment through company's
business or action plans and policies and similarly over $45 million worth of Type C investments
have been approved. Of the $3 million approved approximately $2.2 million have already been
implemented. This demonstrates that the TEST project has successfully leveraged more than twice
as much co-financing from the private sector as was originally provided as GEF funding.
Table 6 shows the water-related improvements created through the adoption of type A and B
options. Reductions in water consumption vary from as little as 1% to over 50% with actual
volumetric reductions reaching as high as 1.39 million cubic metres per year for one company.
Similarly, wastewater discharge reductions range from 50 cubic meters per year to nearly 2 million
cubic meters per year. The actual reduction in pollution loads in wastewater are highly variable and
dependent on the type of industrial process and the type of pollution (BOD, COD, nutrients, heavy
metals, TSS, etc). In some cases the reduction of load has been as high as 100%.
Table 7 present figures for reduction in non-water related areas (e.g. raw materials, air emissions,
solid waste generation) through the adoption of type A and B options that may still reduce indirect
pollution to the river basin. Again the figures are variable due to the diverse nature of the processes
and pollutants. There are some significant reductions in the use of raw materials and in volatile and
harmful emissions. Reductions in solid waste generation vary from minimal (<1%) to significant
>90%.
Table 8 shows the water-related improvements that can be expected through the adoption of type C
improvements. Some of these are already under implementation and some are planned for future
investment (see Table 5 for estimated dates of implementation). Anticipated reductions in water
consumption and wastewater discharges as a result of the TEST process run as high as 5.5 million
cubic meters per year. Reductions in pollution loads are, as with type A and B options, highly
variable.
Table 9 gives the figures for reduction in non-water related areas as a result of proposed type C
options (again, some of which are already under implementation).
It is not the purpose of this evaluation to go into specific detail, review or discussion regarding
these figures. The improvements speak for themselves and the overall conclusion is that the TEST
process is very effective and the demonstration enterprises have been provided with a powerful tool
with which they can meet the requirements of impending regulations while achieving a more cost-
effective level of productivity and cutting down on the hidden costs of non-products.
36
TABLE 5: TYPE A,B AND C OPTIONS IDENTIFED FOR EACH ENTERPRISE AND STATUS OF IMPLEMENTATION
IMPLEMENTATION OF CLEANER PRODUCTION MEASURES AT
IMPLEMENTATION OF EST
Approved for
DEMONSTRATION ENTERPRISES
MEASURES AT ENTERPRISES
Investment (US$)
COUNTRY AND
Feasible Type A
Status of
Feasible Type B
Status of
Feasible Type C
Estimated
Type B
Type C
ENTERPRISE
Type A
Options Implementation Type B Options Implementation Type C Options Implementation
Options
Options
Options approved of approved
Options approved of approved
Options approved Date for Type
Identified
Type A
Identified
Type B Options Identified
C (EST) large
Options
investment
BULGARIA
Yuta JSC
55
17
88%
28
10
100%
9
2
1 implemented
$32,000
$233,000
Slavianka JSC
12
5
100%
7
3
100%
6
2
2 implemented
$52,000
$222,000
Zaharni Zavodi AD
8
5
100%
11
3
67%
17
3
By Oct 2005
$1,500,000 $4,700,000
CROATIA
Agroprotienka
3
1
100%
5
5
80%
7
4
April 2007
$13,500
$7,500,000
Gavrilovic
2
2
100%
5
2
100%
1
1
Sep 2006
$23,000
$3,500,000
Herbos
1
0
N/A
4
3
100%
6
2
Mid 2006
$14,375
$800,000
Implementation
IPK Tvornica Secera Osijek
1
1
100%
3
3
100%
1
1
delayed
$12,466
$800,000
HUNGARY
Gunter-Tata Kft.
7
5
60%
18
14
92%
1
1
2007
$81,751
$900,000
Indukcios es Vedogazos
6
4
100%
15
10
100%
1
1
2005
$36,710
$18,560
Videoton Audio Company
6
5
70%
8
3
80%
4
3
2007
$2,577
$35,783
Nitrokemia 2000
0
0
N/A
0
0
N/A
2
1
2006
N/A
$266,500
ROMANIA
$100,000 by
Astra Romania
53
21
100%
11
4
85%
12
2
Dec 2004.
$36,000
$3,200,000
$86,000 in
Rulmentul
8
8
100%
5
5
65%
6
1
2004
$29,300
$400,000
$40,000 in
Chimcomplex
13
13
50%
13
13
80%
16
4
2004
$40,100
$27,000
$200,000 in
Somes
8
8
100%
24
14
90%
15
1
2004
$42,800 $11,500,000
SLOVAKIA
$21.7 Mill to
AssiDoman Sturov
23
23
96%
22
21
66%
31
2
date
$1,100,000 $4,050,000
30%
Investment
Zos Trnava
30
10
100%
81
8
100%
6
6
completed
$22,500 $7,200,000
IMPLEMENTED
91%
IMPLEMENTED
88%
APPROVED $3,039,079 $45,352,843
37
TABLE 6:
TYPE A AND B WATER RELATED IMPROVEMENTS FOR EACH ENTERPRISE
REDUCTION IN WATER
REDUCTION OF
CONSUMPTION
WASTEWATER
REDUCTION OF POLLUTION LOAD IN WASTEWATER
DISCHARGE (Cu
(BOD, COD, NUTRIENTS ETC)
COUNTRY AND
M./Year)
ENTERPRISE
%ge Reduction Cu. Metres per
%ge Reduction
Total load Reduction
Year
(tonnes/year unless
otherwise stated)
BULGARIA
Yuta JSC
No Data
No Data
No Data
N/A
N/A
Slavianka JSC
Type C Only
Type C Only
Type C Only
N/A
N/A
Zaharni Zavodi AD
Type C Only
Type C Only
Type C Only
Included in Type C
Included in Type C
CROATIA
Agroprotienka
20%
6,200
62,000
Significant, non-quantified
Significant, non-quantified
Gavrilovic
19%
85,725
85,725
34%
30 (organic load)
Herbos
11%
6,000
6,000
90%
14 (organic load)
IPK Tvornica Secera Osijek
1%
23,177
23,000
3%
305 (organic load)
HUNGARY
Gunter-Tata Kft.
14%
3,064
2,339
No Data
16 m3/y oil
Indukcios es Vedogazos
52%
10.37
10,370
No Data
No Data
Videoton Audio Company
1%
50
50
No Data
No Data
Nitrokemia 2000
No Data
No Data
No Data
No Data
No Data
ROMANIA
Astra Romania
17%
13,200
13,200
98.5% oily products
2,450 oily products
12% COD; 23% BOD; 30%
5.1 COD; 1.1 BOD;
N/A
N/A
N/A
Rulmentul
heavy metals
0.1 heavy metals
28% COD;
275 COD;
47%
7,440
10,725
Chimcomplex
32% Ammonia
71.5 Ammonia
7.5% COD; 6% BOD; 7%
554 COD; 147 BOD; 92 TSS;
11%
1.39 Mill
1.89 Mill
Somes
TSS; 8.6% AOX
20 AOX
SLOVAKIA
AssiDoman Sturov
3.80%
1.13 Mill
1.76 Mill
12%
No Data
Zos Trnava
4%
20
20
100% (oily products)
28.2 (oily products)
38
TABLE 7:
TYPE A AND B NON-WATER RELATED IMPROVEMENTS FOR EACH ENTERPRISE
ENERGY
RAW MATERIAL USE
EMISSIONS OF
WASTE GENERATION
SAVINGS
COUNTRY AND
CO2/H2S/VOC+AC11
ENTERPRISE
%ge Reduction
Total reduction
EQUIVALENT
%ge
Total Reduction
Specific Reduction
in consumption (tonnes/yr unless
(tonnes/year)
Reduction
(tonnes/yr)
otherwise stated)
BULGARIA
fibres 10%
fibres 18.8
Yuta JSC
fuel 15%
fuel 111 m 3/y
80
10%
18.8
Slavianka JSC
255
Zaharni Zavodi AD
fuel 7%
fuel (coal) 36
CROATIA
Agroprotienka
N/A
N/A
None
20%
6,250
Gavrilovic
None
None
914
<1%
2
Herbos
0.27%
8
None
11%
6,000
IPK Tvornica Secera Osijek
0.1%
2,142
1
0.01%
305
HUNGARY
4 oily textiles
80 kWh/yr
Gunter-Tata Kft.
100%
3.3 solvent
0.03
95%
3.3 solvents
196,502 kWh/yr
Indukcios es Vedogazos
124
21,007 m 3/yr gas
30% 67 wood pallets/y
4,880 kWh/yr
Videoton Audio Company
30%
67 pallets/yr
1.58 CO2 equiv
50% 1.5 kg neon tubes
90%
1,343 m3/y sawdust
Nitrokemia 2000
ROMANIA
Astra Romania
0.5%
N/A
961
50%
594
Rulmentul
N/A
N/A
N/A
9%
45.4
Chimcomplex
2.7%
77
2200
N/A
N/A
Somes
12.8%
475
N/A
50%
42 (lost fibre)
SLOVAKIA
AssiDoman Sturov
10%
No Data
1000
5% oil
2.2 oil
0.3%
10%
100
Zos Trnava
2% paint
8.4 paint
39
TABLE 8: TYPE C WATER RELATED IMPROVEMENTS FOR EACH ENTERPRISE
REDUCTION IN WATER REDUCTION OF
CONSUMPTION
WASTEWATER
REDUCTION OF POLLUTION LOAD IN
DISCHARGE
WASTEWATER (BOD, COD, NUTRIENTS ETC)
COUNTRY AND
(Cu M./Year)
ENTERPRISE
%ge
Cu. Metres
%ge Reduction
Total load Reduction
Reduction per Year
(tonnes/year)
BULGARIA
Yuta JSC
N/A
N/A
N/A
N/A
N/A
Slavianka JSC
60%
1,260
1,260
25% COD
No Data
Zaharni Zavodi AD
80%
725,000
95,000
100% COD
No Data
CROATIA
Agroprotienka
None
None
None
95%
600 BOD
Gavrilovic
56%
250,280
168,000
77%
53 BOD
Herbos
None
None
None
10%
1.5 organic load
IPK Tvornica Secera Osijek
62%
1.68 Mill
1.79 Mill
N/A
N/A
HUNGARY
Gunter-Tata Kft.
N/A
N/A
N/A
N/A
N/A
Indukcios es Vedogazos
N/A
N/A
N/A
N/A
N/A
Videoton Audio Company
N/A
N/A
N/A
Below Legal Limits
Below Legal Limits
Nitrokemia 2000
39%
5,322
5,322
31% salt content
472 salt
ROMANIA
Astra Romania
89%
180,000
160,000
N/A
N/A
Rulmentul
N/A
N/A
N/A
N/A
N/A
23% COD;
165 COD;
41%
3,888
4,950
Chimcomplex
58% Ammonia
38.5 Ammonia
22% COD; 17.7% BOD; 1,770 COD; 486 BOD;
65%
5.5 Mill
5.5 Mill
19.5% TSS; 91% AOX
282 TSS; 284 AOX
Somes
SLOVAKIA
AssiDoman Sturov
14.70%
4.42 Mill
261,000
No Data
3,660
Zos Trnava
50%
250
250
100%
53
40
TABLE 9:
TYPE C NON-WATER RELATED IMPROVEMENTS FOR EACH ENTERPRISE
RAW MATERIAL USE
WASTE GENERATION
ENERGY SAVINGS
EMISSIONS OF
COUNTRY AND
CO2/H2S/VOC+AC11
ENTERPRISE
%ge Reduction in
Total reduction
EQUIVALENT
%ge
Total Reduction
Total Reduction
consumption
(tonnes/yr)
(tonnes/year)
Reduction
(tonnes/yr)
BULGARIA
20% fibres 16.8 fibres
Yuta JSC
1% fuel
4.2 m3 fuel
135
10%
11.3
Slavianka JSC
30% fuel
94 fuel
234
400 m3/y water
1,066 Mwh
Zaharni Zavodi AD
1187.5
100% Slop
95,000 m3/y
CROATIA
Agroprotienka
N/A
N/A
none
95%
13,250
Gavrilovic
2%
302
none
85%
7,020
Herbos
0.03%
0.9
none
<1%
2
IPK Tvornica Secera Osijek
none
none
none
55%
1.8 mill
HUNGARY
Gunter-Tata Kft.
Indukcios es Vedogazos
Videoton Audio Company
26% Fe
1.9 dangerous waste
756.1 mill kJ/yr ('cool
2SO4 +Na2C03
495 Na
17% phtalic
2CO3
673-1000 kg CO
13,483 pieces PE
energy' from Ammonia
679 carbamide
2
28%
sack
evaporation)
Nitrokemia 2000
anhydride+carbamide
ROMANIA
Astra Romania
0.68%
N/A
N/A
100% (waste
100% PCE
30
N/A
13.5
Rulmentul
PCE)
Chimcomplex
4.75%
214.5
770
N/A
N/A
Somes
20%
3,100
N/A
31.3%
243
SLOVAKIA
AssiDoman Sturov
N/A
N/A
641
Zos Trnava
5%
24.5
21.76 VOC
10
100
41
4.1.4 Threats and Root Causes Effective Resolution
The primary threats and root causes have been noted under the Project Background and Landscape.
The Project has addressed these concerns in two ways. Firstly at the specific level of 17 enterprises
by reducing their discharges of organic and toxic pollutants, and secondly by providing transferable
demonstrations of the TEST process (with its associated improvements to company management,
cleaner production policy, factory-floor awareness and technological improvements) that can be
replicated throughout other industrial enterprises within the Danubian countries.
One important point that needs to be highlighted is that not every demonstration enterprise has
directly addressed water-related pollution issues. A review of Tables 5-9 shows that, in one or two
cases, a reduction in raw materials (use of toxic products), waste generation, VOCs or air polluting
emissions were the main achievement of the TEST process. Undoubtedly such improvements can
still be linked to a holistic and integrated approach to cleaning up the Danube River Basin and have
added valuable case studies to the TEST approach. The inclusion of these parameters is also
directly consistent with the overall requirements of CPA and EST as defined by UNIDO.
The Project has developed a detailed matrix of indicators (See Tables 5-9) which effectively follow
the TEST process within each enterprise and quantify the changes, improvements and benefits
gained through the various project activities and the TEST modules. These constitute both Process
and Stress Reduction indicators and are considerably more specific than those included in the
original Project Document, providing a more measurable detail of what has been achieved. The
Evaluation gave specific attention to the Process indicators (training, capacity building,
improvements in cleaner production, adoption of other TEST processes, etc) and to the Stress
Reduction indicators (physical changes to handling procedures, construction of waste handling and
reduction facilities, end-of-pipe treatments, etc) identified for each enterprise to ascertain the
accuracy of the indicators. In all cases that were reviewed the measurable indications were seen to
be accurate. These have been discussed further under 4.1.3 Summary of Demonstration
Deliveries.
There are no real Environmental Status indicators for the Project and these would be very difficult
to evaluate and confirm in any case. However, it is the Evaluator's opinion that these would not in
themselves be relevant to this particular demonstration project and are more appropriate to the
larger Danube Regional Project that is following on from the GEF Pollution Reduction Project. The
inclusion of Environmental Status Indicators would have required a fairly large investment in
independent physical monitoring of water quality, water use reduction, etc. In the event, this
information was provided by the enterprises and must be accepted at face value. It is the State
Water Authorities that set the standards for permitted emissions associated with operating licences
for industrial enterprises. Each industry has a specific set of parameters to adhere to and is self-
monitoring although the Authority or Inspectorate may also undertake spot-checks. Financial
penalties are now generally increasing in line with the need to meet the new EU directives.
Industrial enterprises of this nature would be very reluctant to allow outside technical expertise to
collect data from their facilities. However, there is a certain logical assumption that can be made
that would strongly support the argument that confirmed changes at the Process and Stress
Reduction level must result in quantifiable changes at the Environmental Status level. Furthermore,
this needs to be viewed in the context of the objectives of the TEST project which is primarily to
demonstrate how a particular process can achieve cost-effective and sustainable improvements in
cleaner production and environmental management within selected companies in such a manner as
the companies themselves can identify and adopt the benefits. The reality of this situation is that a
42
particular enterprise (as was in fact the case) may actually be reducing its discharge load of a
certain substance below the immediate background measurements a few metres upstream so that
Environmental Status indicator measurements in the actual river environment would hardly be
relevant or appropriate. In most of the Danube countries, government inspectorates are charged
with the responsibility for setting and checking discharge levels. However, the capacity to do this
varies considerably from country to country both at the human resource and at the equipment level.
One consultant that had been closely involved in the development and implementation of the TEST
process did stress the need for the enterprises to develop effective in-house data gathering and
information systems at the earliest stages of the TEST procedure, but noted that this would require
additional resources and investment (particularly for monitoring equipment).
The cost-effective element of the TEST process also works to the Project's advantage as far as
actual discharges and improvements to the Environmental Status of the Danube in proximity to
each enterprise. The TEST process sets out to demonstrate to each enterprise that certain measures
are cost-effective in that a certain level of investment can result in a significant improvement in
their water usage and in their discharge levels. It is therefore not in the company's interest to falsify
the data as they would only be fooling themselves into thinking that their adoption of new
processes and technologies are achieving a significant and cost-effective improvement to their
discharges and water efficiency.
A strong link to other Danube initiatives such as the ICPDR and DRP, which have more
involvement in actual physical monitoring and data handling, would be a valuable improvement to
the TEST process within the region.
It should be noted that the original Project process indicators were revised and rationalised
somewhat in the 2004 PIR following a revision and improvement of the TEST procedure so as to
apply a more integrated approach. The revised indicators are an improvement on the rather
generalised indicators that the project started with. It would have been useful from the Evaluations'
point-of-view if the PIR format used by GEF made allowance for such changes under section 4.
Adjustments to Original Project Strategy. A pertinent question to include in this section in future
would be in reference to any changes in Project indicators as this has a significant bearing on the
evaluation process.
The overall SQA for the Resolution of Threats and Root Causes has been allocated at 80%
equivalent to a Rating of 4. The demonstrations have been fairly effective at showing mechanisms
for addressing threat and causes at the individual enterprise level. The Project could have adopted
more effective indicators for this aspect and these should have been formally revised at the
APR/PIR stage. Also, as previously mentioned, future selection of enterprises could focus fully on
IW related water improvements as well as other issues, which are indirectly related to water quality.
4.1.5 Global and National Benefits
The Project Document notes that the Executing Agency will assist national institutions and
independent experts (counterparts) to enhance their understanding of the need for, and their abilities
to provide, enterprises with the integrated package of services that constitute the TEST programme.
It further notes that perhaps the most important skill developed will be the realization of the
necessity to work in teams to be able to provide the diverse services needed by enterprises to be
able to incorporate ESTs in their production operations.
43
The immediate national target beneficiaries are identified as the 17 enterprises that will be advised
on how to reduce their pollutant discharge at the same time as they improve their production
efficiency. These beneficiaries also include the workers in the enterprises who would have the
prospects of a more secure economic future.
The enterprises used in the TEST project were selected based on the priority hotspots identified
through the Danube Basin Pollution Reduction Programme and SAP and were reconfirmed at the
national level prior to adoption. In this respect, it can be confirmed that the project is addressing
national priorities for developing environmentally sound technologies and for pollution reduction.
These national priorities were well-defined within the Project Brief and the Project Document with
major manufacturing discharges being identified for each country by sector (taken from
information provided by the Danube Pollution Reduction Programme's TDA Report. This shows a
clear linkage and sequence of GEF investment through the TDA and SAP, to the TEST
demonstrations as grass-root delivery of remediation techniques.
It should be further noted that national priorities within the Danube region now extend to the need
to conform to EU directives (as most of the countries are in pre-accession or accession status). The
main goal under these EU directives will be for the countries to develop Integrated Permitting
Procedures and to introduce Best Practices and Technologies. These requirements are all embraced
within the TEST integrated modular procedure.
The ultimate beneficiaries will be those who are adversely affected by the national as well as
transboundary pollution in the Danube River. The Danube River as well as the Black Sea, which is
the ultimate sink for wastes discharged in the Danube, provides vital services such as drinking
water supply and recreation, as well as sustaining aquatic life forms that can survive only in a clean
environment.
Although not mentioned in the Project Document, the Project Brief identifies the target GEF
Operational Programmes to be OP8 (waterbody-based) and OP10 (contaminant-based).
As one of its goals, the waterbody based OP8 includes assistance to countries in making changes in
the ways that human activities are conducted in a number of sectors so a particular waterbody and
its multi-country drainage basin can sustainably support such human activities. Characteristic
interventions listed under OP8 include supporting the incremental cost of technical assistance,
capacity building, limited demonstrations, and certain investments needed to address the priority
transboundary concerns, and to encourage the use of sound science and technological innovation.
The long-term objectives of the contaminant-based OP10 is defined as the development and
implementation of International Waters projects that demonstrate ways of overcoming barriers to
the use of best practices for limiting releases of contaminants causing priority concerns in the
International Waters focal area, and to involve the private sector in utilising technological advances
for resolving these transboundary priority concerns. Listed under the OP10 project outputs are
those that work to implement a number of fast-track demonstration projects of approaches,
techniques, pilot projects, innovative technologies, institutional arrangements, and the use of
economic instruments. Also listed are projects that demonstrate the development of interim best
practices for minimising risk, phase-out of the use of a particular contaminant or of a process that
generates a contaminant, pollution prevent strategies, the development of new technologies to
neutralise priority contaminants and the development of economic instruments to illustrate the
feasibility of measures to abate/prevent priority contaminant release.
44
One of the objectives of the original Danube SAP was the reduction of point-sources of pollution
and hence overall reduction of pollution load into the Danube. In demonstrating cost-effective and
viable processes and technologies to achieve this aim as identified by the SAP, the project is
directly addressing the requirements and priorities of the Danube River Protection Convention and
the recommendations from the Danube River Pollution Reduction Programme.
Based on the Operational Programme guidance and the requirements of the SAP, it is clear that the
TEST Project is directly meeting the needs of the countries and the Danube region as a whole as
well as conforming to the Operational Strategy for International Waters as set out by GEF, and
therefore providing effective global benefits.
The SQA score for Global Benefits is 100%. Undoubtedly the lessons and best practices generated
by the project are of enormous value at the national, regional and global levels.
4.1.6 Stakeholder Participation and Public Involvement
In the opinion of all parties interviewed by the Evaluator, stakeholder involvement in the Project
Design and the Project Implementation was both adequate and effective. The Project Coordinator
confirmed this assessment and noted that there had been no complaints from any party of lack of
dialogue with any potential stakeholders. All Government agencies that were questioned on this felt
there had been appropriate input from their side. Certainly every opportunity seems to have been
offered for full participation by relevant bodies. However, in the opinion of the Evaluator, such
opportunity was not always seized in the case of the Government sectors. In at least one case there
was an evident lack of involvement or understanding of the project from a crucial sector (the
Ministry responsible for Environment). To balance this lack of ownership from one important
Government sector it should be noted that in the same country, the Ministry for Economic Affairs
was a close partner to the project instead. A lack of human resources coupled with a fairly
consistent turn-over of middle to upper management staff within the Ministries of some countries
may partly explain this lack of effective participation in project activities and lack of awareness of
project delivery.
Ordinarily this lack of Government ownership could have created a real problem for the project at
the level of at least one country. However, the unique nature of the TEST project was that the EA
was dealing directly with the private sector through the national counterpart, and not through a
Government agency. It is always difficult to maintain full involvement in project steering and
awareness by government agencies and the level of commitment will always vary from country to
country. In the TEST project, the Upper Danube countries demonstrated a greater level of internal
cooperation and overall stakeholder input. This may be a reflection of the more rapid transition
process within these countries away from the old practice of centralised and dictatorial government.
In any GEF project, it is essential that every effort be made to ensure that government agencies are
always kept abreast of developments and are always invited to appropriate meetings and workshops
even thought Project Management may feel that such effort is a waste of time. These are often the
very same agencies that will be the first to complain if they were left off the invitation list or were
not included in shared information. However, this categorically does not appear to be the case with
the TEST project and all evidence suggest that every effort was made to include all appropriate
stakeholders and that any inadequacies in awareness or involvement lie with the government
agencies themselves and not the Project. Most importantly for the TEST project, the level of
commitment from the private sector was clearly significant and consistent. Sustaining this
45
commitment to project objectives and to the TEST process beyond the project will be the defining
proof for TEST.
As identified in the 2004 APR/PIR, one of the major challenges to implementing the TEST
programme was the identification and selection of the demonstration enterprises. Although the
previous GEF project (Pollution Reduction Programme) had identified many hotspots, being within
one of those hotspots was not in itself considered to be sufficient justification for selection.
Financial viability was a critical selection criteria as only those companies that had long-term
financial viability could be expected to remain with the TEST programme for 3 years, and would
have available funding for investment in TEST processes and technology. Involvement by
enterprises was voluntary and one significant lesson learned from the project was that the
enterprises needed to be convinced that there would be significant economic benefits to themselves
(as well as environmental benefits) as a result of their participation. This undoubtedly provided the
motivation and incentive for their agreement to be involved. Even the environmental aspects were
directly linked to economic consequences (i.e. being able to compete in a European and global
market that demands and legislates for environmentally sound practices).
Although though the TEST process was predominantly funded by the project, the enterprises were
required to make some token financia l contributions and this became a further strategy for
strengthening commitment to the project to be used as part of the selection process. In the final
analysis, the enterprises made a substantial contribution to the Project, both in manpower terms and
in the cost of improvements to their processes, policies and technologies.
The Evaluator has an opportunity to meet with the Technical Expert for Water Management and
Pollution Control at the International Commission for the Protection of the Danube River. The
representative noted that the TEST project had been very beneficial in providing some real
examples of processes and technologies that could be used for pollution reduction and water use
efficiency. The benefit to the Commission and its member's was the actual demonstration of how to
convert hotspots into viable cleaner enterprises and to reduce discharges at specific point-sources.
TEST has taken specific conclusions and recommendations from the Danube Pollution Reduction
Programme (GEF Project) and actually implemented these to demonstrate potential for real
progress in the protection of the Danube River through mitigation of threats and impacts. The
Danube Pollution Reduction Programme's Transboundary Diagnostic Analysis (TDA) and
Strategic Action Plan (SAP) helped to identify the original Hotspots along the Danube river basin.
In this respect, the TEST project is a direct result of, and logical follow-up to, the SAP process.
Now, following on from the Danube Pollution Reduction Programme is the UNDP GEF Danube
Regional Project. One of the activities within this larger regional project is to review and initiate
reform for industrial policy in the region (including identifying Best Available Technologies
BAT) and to assist in strengthening capacity to reduce pollution. Clearly there are direct links here
to the lessons and practices developed through TEST and the case-studies arising from the MSP.
The representative of ICPDR felt that it would be important to follow-up on the TEST project and
to undertake a post-project assessment at a specified date after project closure to ascertain how
effectively the TEST processes and concepts were still being implemented within the 17 selected
enterprises, and how many other enterprises in the region had replicated the TEST process. The
ICPDR representative felt that the incentives for the participating enterprises within the TEST
project were strong and included recognition of the need to comply with EU requirements (ISO
14000 and IPPC), and the competitive advantage that this and a general company policy of cleaner
production would give them within the European and global markets.
46
The Evaluator was also able to communicate with the UNDP GEF Danube Regional Project
regarding mutual dialogue and communications between the two projects. The DRP stated that it
was well aware of the objectives of the TEST project and, to a certain extent, its activities within
the selected Danube countries. DRP noted that it had cooperated with the TEST Project on several
workshops and results dissemination exercises. Relationships had been developed between the two
projects mainly in the area of capacity building. DRP has a component on nutrient pollution control
policies from industrial sources as well as for the promotion of BAT (Best Available
Technologies). The CPC partners in the TEST Project were also cooperating with the DRP
Industrial component and the intention was to continue with this cooperative partnership within the
DRP Project. DRP has been using the case studies developed in cooperation with the TEST Project.
In the opinion of the DRP Project, the dialogue between consultants from both the TEST project
countries and the other Danube countries has been quite intensive. With regard to further follow-up
to the TEST project, DRP would like to see the development of more targeted policies and legal
frameworks focusing on enforcement and the adoption of BAT. They feel that the EU IPPC
provides the framework for such policies and legal reforms.
Where training was provided within the Project activities, the policy was to use experts from within
the region that understood the regional context and needs, and who were therefore true national or
regional stakeholders in their own right. Training needs were identified based on open discussion
with national project teams and at the regional workshops. One CPC noted that this project was
excellent in its involvement of stakeholders as it was "tailor-made for participation".
One area relating to stakeholder participation that the Project Coordinator felt could have been
strengthened was the involvement of more enterprises beyond the selected 17 with regard to
enhancing awareness of the TEST process. Some simple training could have been provided on a
workshop basis, and possibly some opportunities offered for non-demonstration enterprises to
observe at on-ground training sessions. Non-demonstration enterprises were, however, invited to
the dissemination workshops and attendance was generally good. Clearly, more could have been
done with a healthier budget, and it seems that the constraints imposed by an MSP sized budget
coupled with poor delivery of co-financing (see Workplan and Budget section) has caused
constraints on potential project delivery. Also, this involvement of non-demonstration enterprises
could have been assisted through a stronger incentive and ownership throughout all pertinent
government sectors to design national programmes to improve industry incentive. It would also
have been beneficial to project sustainability and government commitment if local Inspectorates
responsible for licensing and enforcement could have been more closely involved in understanding
the objectives and requirements of the project.
The evaluator also had the chance to meet national representatives from the Regional Environment
Centres. These RECs took an active role in the project and sat on the Advisory Boards. They were
involved in project design also. The RECs were consistently supportive of the Project and felt that
the TEST had achieved the best possible end-product that could have been achieved in view of the
constraints. One criticism of project design and implementation was that there should have been
much more time available for project inception and for `preparation of the ground' and sensitisation
of the companies (especially top management). Again, this seemed to be a criticism of the
timeframe and the lack of a proper Preparatory phase for the project. Another suggestion from the
RECs was that it would have been beneficial for them to have been included in the training process.
In general, overall stakeholder participation in the project at the country level has been highly
satisfactory, especially in relation to National Counterpart participation (both the actual counterpart
person, and the associated Institution/Centre). One country felt that more involvement of the
47
industrial branch associations would have been valuable. However, there has been limited
participation in the project by other Danube Countries outside of the 5 project participants. There
was some dissemination of results through the ICPDR but this was fairly minor. There is now an
opportunity and a requirement to actively transfer lessons and best practices across to other Danube
countries in order to capture the true replicability of the TEST process. It is now urgent to find a
way to bring Government, the CPCs/Test Institutes, and the private sector together to address the
replication needs for TEST.
The SQA score allocated to this aspect of the Project is 80% with a consequent rating of 4. The
score is slightly less than perfect as the stakeholder participation by the government is deemed to be
somewhat lacking and the participation outside of the Project countries could have been improved.
The score still remains very high, as the participation within the countries by those directly
involved in the Project aims and objectives was excellent.
4.1.7 Capacity Building
The Project Document and subsequent activities accurately identified and addressed both capacity
building and institutional strengthening. In fact, it is reasonable to say that the Project went beyond
expectations and has provided the NCPCs and other national institutions with sufficient foundation
and capacity that they can now sell their services professionally and offer very effective packages
to industry for Cleaner Production and all of the TEST modules. Tables 10 &11 provide details of
the capacity building efforts and achievements at both the enterprise level and at the level of the
CPCs and Counterpart Institutes. Most of this capacity building was in process and training, with
the only equipment procurements associated with the counterparts being a single computer for
each.
The important lesson will be whether such capacity building is seen to be sustainable. This applies
to both capacity building at the CPC/Institute level, and capacity building at the enterprise level. At
the CPC/Institute level there is significant evidence of sustainability in that further enterprises are
now contacting the CPCs and institutes with a view to contracting them to carry out varies levels
and stages of TEST within their companies in order to meet ISO 14000 and IPPC requirements. At
the enterprise level it would be valuable to see how ingrained and sustainable the TEST process has
become in future years. Again evidence is positive with some trained team members from the
enterprises having been promoted within the company, even at Chief Accountant and Director
level. Some trained personnel have left their posts to take other jobs in other enterprises. It is hoped
that they will carry their new experience and knowledge with them and disseminate it into the new
company policy. In this respect it would be useful to review the TEST project, the CPCs/Institutes
and the demonstration enterprises at a later date to see how sustainable the process has really been.
Table 10 shows the level of capacity building at the demonstration enterprises and Table 11
presents the institutional strengthening and capacity building for the national counterpart
institutions and staff.
At the enterprise level, the number of project modules completed for each enterprise is impressive,
as is the number of employees trained and the amount of man-days dedicated to training and to
undertaking the TEST activities.
At the level of the counterpart institutions, training and therefore associated capacity building has
also been impressive and each country's counterpart institute has benefited significantly from the
new tools and skills that they have developed.
48
Delivery at the capacity-building level of the project has been exemplary and well beyond the
expectations cited in the Project Document. This aspect therefore receives a well-deserved SQA
score of 100%.
49
TABLE 10: CAPACITY BUILDING, TRAINING AND IMPLEMENTED MODULES AT THE SELECTED TEST ENTERPRISES
Number of
Projects Modules undertaken by
Number of
Number of
enterprise
enterprise
INDUSTRIAL
COUNTRY
ENTERPRISE
Employees
man-days of
man-days in
SECTOR
Trained
training
TEST
activities
IR CPA EMS
EMA EST SES
Bulgaria Yuta JSC
Non-Woven Textiles
14
64
140
X
X
X
X
X
X
Slavianka JSC
Fish Processing/Canning
3
5
70
X
X
X
X
Zaharni Zavodi AD
Sugar/Alcohol Production
14
56
210
X
X
X
X
X
Croatia
Agroprotienka
Meat Rendering
6
33
136
X
X
X
X
Gavrilovic
Meat Processing
9
54
290
X
X
X
X
Herbos
Pesticide Plant
6
35
146
X
X
X
X
IPK Tvornica Secera Osijek Sugar production
3
37
190
X
X
X
X
Hungary Gunter-Tata Kft.
Heat Exchange Manufacture
5
35
205
X
X
X
X
X
Indukcios es Vedogazos
Steel Heat Treatment
3
30
90
X
X
X
X
Videoton Audio Company
Electronic Products
2
20
100
X
X
X
X
Nitrokemia 2000
Chemical Production
15
59
243
X
X
X
X
X
X
Romania Astra Romania
Petrochemical Refinery
26
162
1460
X
X
X
X
X
Rulmentul
Bearing Production
50
318
1218
X
X
X
X
Chimcomplex
Chemical Production
52
270
523
X
X
X
X
Somes
Pulp and Paper Bleaching
35
260
770
X
X
X
X
X
Slovakia AssiDoman Sturov
Pulp and Paper Products
55
81
560
X
X
X
X
X
Zos Trnava
Rolling Stock Repairs
82
172
1400
X
X
X
X
X
Project Totals
380
1691
7751
51
TABLE 11:
CAPACITY BUILDING, TRAINING AND NEWS TOOLS/SKILLS DEVELOPED BY COUNTERPART
INSTITUTIONS
New tools/skills developed by counterpart to support
Number of people
Additional jobs
TEST
Number of man-
Number of total
COUNTRY
trained in TEST
created to support
days of training
days of training
Tools
TEST
Social
CPA EMS
EMA EST
COMFAR
Audit
Bulgaria
9
121
83
1
X
X
X
X
X
Croatia
13
92
26
1
X
X
X
X
X
Hungary
5
89
22
2
X
X
X
X
Romania
40
160
31
2
X
X
X
X
Slovakia
23
160
27
2
X
X
X
Project Total
90
622
189
8
52
4.1.8 Policy and Legislative Reform and Improvement
The TEST Project has been unusual in that rather than relying on policy and legislative reform as
an integral part of its delivery and outcome it has actually been driven by parallel needs for policy
and legislative changes vis-à-vis EU requirements for meeting set environmental standards within
industry. In this respect there has been no actual project component or activities directly addressing
such reforms. Instead the entire project is responding to the need for such reforms by demonstrating
how these can be manifested at the individual company/enterprise level. It would be reasonable to
state that the TEST project has provided a working platform to demonstrate such industrial
response and to clarify that such national reforms can be met at the industrial level without
compromising a company's economic viability or jeopardising its place in the market. On the
contrary, TEST demonstrates that such improvements are directly beneficial to both company
economic stability and to market competitiveness. It is expected also that TEST will help to raise
awareness of discharge and emission standards relevant to the new EU legislative requirements at
the enterprise level across all participating countries.
One very important success driver for the TEST Project has been the need to comply with EU
directives on water quality, discharges and air emissions. The EU requirements are very strict as is
the need for compliance and therefore monitoring. A company has to use an accredited agency to
take measurements and report to the government and data can be compared with the hydrological
institutes and other agencies. This means that there are limited opportunities for fraudulent
reporting, and little chance of corruption for those countries that fall within the EU framework.
Obviously some of the less-developed lower Danube countries may need improvement to
enforcement capacity and reporting, and their internal assessment and reporting procedures need
streamlining. Lots of funding is currently being provided for such technical capacity building
through various donor programmes.
One problem experienced by the countries is the `importation' of laws that are more applicable to
countries such as Austria and Germany, but are not really tailor-made for the transition countries.
Also, enforcement is weak in the transition countries. Hopefully, these concerns can be addressed
under the GEF Danube Regional Programme which has a component that specifically deals with
industrial policy reform.
Of interest here to GEF is the fact that this project has partnered the donor (GEF) and its
counterparts at the national level directly with the private sector which is somewhat unique within a
GEF initiative. From this point-of-view it may well be that industry will be providing some of the
standards as far as meeting EU ISO and IPPC directives, and that the public sector will take their
lead from this. In other words, TEST could and should provide the platform from which
government takes the initiative to develop its policies and reform its legislation with respect to
industrial practices and control/compliance on emissions and discharges. It is therefore important
that the Danube Regional Programme reviews the case studies and best practices/lessons from this
TEST project in the context of its industrial policy reform component. If used effectively, the
results of the TEST Project can have a very positive effect on such reforms.
With this consideration in mind it is felt that the TEST project has made a very valuable
contribution to potential Policy and Legislative Reform and is awarded an SQA score of 100%.
53
4.1.9 Replicability
The lessons learned from the TEST project have been shared with the GEF UNDP Danube
Regional Project and with the Emission Inventory group (EMIS) of the International Commission
for the Protection of the Danube River Basin (ICPDR). The TEST project results were presented
during the Workshop on Best Available Technologies and Industrial Pollution Control organised by
the Danube Regional Project in Bratislava in December 2003. However, there is currently a
weakness here that needs to be resolved. Neither the ICPDR nor the DRP websites have direct links
to the TEST website. It took some time to find references to TEST on the DPR site and the
Evaluator was not able to find a direct link to a TEST reference on the ICPDR site. The TEST
project was discussed in the DRP Final Report on Project Component 1.5: Industrial Reform and
the Development of Policies and Legislation towards the Reduction of Nutrients and Dangerous
Substances - June 2004. This report mentions strong support for the development of a website link
between DRP and the main sources of Best Available Technology. There does appear to be a need
for a mechanism at the level of the ICPDR to assist in the provision of the latest data and
information on BAT and TEST-related processes. ICPDR, through its EMIS (the Emissions Group)
component, could provide one strong possibility for encouraging replication. EMIS recognises 4
different kinds of industry (with related emission problems) and has developed a list of
stakeholders for such at both the industrial level and the government level. Certainly, discussions
with representatives from ICPDR (past and present) suggested that dissemination of the TEST
results and encouragement of replication throughout CEE companies was not a high priority. AT
present there are no real tools to push companies to replicate TEST. In this respect the incentive
must still be seen as economic. But this can only be effective if companies/enterprises are aware of
the economic advantages of adopting the TEST process. Some stakeholders felt that this
demonstrated a need for stronger involvement in the Project of those Ministries responsible for
national economies. These Ministries should now be targeted with information on the TEST
process.
Lessons learned were also shared at the national level through national dissemination workshops.
The 2004 APR/PIR also notes in particular that there was an intensive exchange of information
with on-going EU-funded twinning programmes for the implementation of IPPC in Romania, with
the Regional Activity Centre based in Hungary, and with USAID funded ECOLINK programmes
in Bulgaria and Croatia.
The Project Management did note that they would have preferred to see a larger dissemination
phase at the national level, but that funding was constrained both by the MSP modality and by the
shortfall in expected co-financing. Similarly, several countries felt that a replication mechanism
should have been more clearly defined in the Project Design, or that a replication phase should have
been planned to follow the MSP.
Representatives from the RECs found the project to be very worthwhile despite being very
optimistic in its intended achievements and timeframe. They mentioned that although the
COMFAR process is a very good one, for a lot of the companies it may have been too optimistic
and required more time than was available to adapt COMFAR to each company's needs. They felt
that the process should be continued in some way and that now is not the time to close the door on
support to the TEST process. They felt that the best way to raise awareness of the achievements of
the Project and of the TEST process generally within the public sector would be to arrange for
representatives of the demonstration enterprises to present their results at national seminars (or
possibly local seminars within different national regions) and to discuss cleaner production and
54
wastewater. Although this has been done to some extent by the dissemination seminars, a wider
audience is necessary. The RECs would like to play a larger role in this process.
Further TEST projects are already planned in the Mediterranean region and in some of the other
CEE countries such as Bosnia -Herzegovina. In the latter case, plans are under development for
Croatia to provide assistance based on its new expertise in TEST. There are fairly strong cultural,
historic and linguistic linkages between the two countries. The Croatian CPC would provide
assistance to the Centre for Environmentally Sustainable Development in Bosnia-Herzegovina
which has some existing abilities and has already participated in some cleaner production training,
but would benefit greatly from learning more of the TEST process and mechanism.
The CPC in Slovakia is providing bilateral aid to Uzbekistan as part of the Slovakian national
commitment to aid (to address EU requirements) through the Ministry of Food and Agriculture
with assistance from UNDP and UNIDO. In this respect, the CEE states are gradually altering from
being recipient countries to donor countries providing multilateral assistance through the EU. It is
possible that this transition may also provide further opportunities for bilateral assistance within the
region for replication of TEST and to assist in the development of capacity through training.
There is no doubt that the lessons and practices identified through the TEST Project are highly
replicable throughout the region and, indeed, on a global basis. Furthermore, there is a strong need
and urgency for their transfer and replication. However, it appears that there were insufficient funds
within the MSP TEST Project to really undertake an effective replication mechanism for either the
Danube countries or to capture these valuable lessons for GEF and other funding initiatives. The
TEST Manuals and Cases Studies developed by the Project are very valuable commodities and
should be shared with all appropriate institutions and agencies.
There are certain incentives that could help to promote and replicate TEST. In Denmark, for
example, having ISO 14000 accreditation means that licensing fees for that company are cut by
40%. In this respect, government does needed to be more involved so as to provide financial
incentives to companies for cleaner production and compliance with EU directives, all of which are
in the national interest. At the level of the industrial support institutes (CPCs), such institutes could
develop a programme focusing on clusters of related industries with similar needs and
technological requirements. Training and support could be pooled and made more cost-effective
while also being specific to those needs. Experiences would be shared and lessons transferred
between cluster members. To support this cluster approach, one CPC suggested that TEST Toolkits
could be customised to suit individual company/cluster needs.
One approach for transferring the lessons and for further replication of TEST that was suggested by
the Project Coordinator would be to undertake a further phase of TEST but with an altered balance
of subsidy and some small funding input. In the TEST Project, 90% of activities were subsidised by
GEF to prove the demonstration process. A second phase could reduce this to 50% support to a new
set of enterprises, which can now see the market advantages of TEST. Some of this 50% should
represent matched funding from national governments (e.g. national environmental funds) so that
25% would be from GEF and 25% from governments. One major advantage would be that in-
country capacity is already established and ready to assist.
As a next step, now that type A (No investment operational control improvements and
housekeeping) and type B (Low to moderate investment- short payback) implementation measures
have been carried out in most of the enterprises, all stakeholders felt that there was a need to assist
companies in finding realistic investment opportunities to move on to type C measures (Large
55
investment longer term payback). If soft loan agreements could be established with development
banks these could be made conditional to the companies having reached a certain step or module
within the TEST process.
In the final analysis, there could have been more information sharing with other agencies beyond
the TEST project (e.g. with ICPDR, EU, etc) and within and outside of the TEST project countries.
There are justifiable reasons for why this has not occurred as effectively as may have been intended
and these relate mainly to project design, the use of the MSP modality and funding constraints. A
more specific replication phase should have been designed, possibly even as a separate project.
However, any significant criticism under this assessment area has to focus primarily on Project
Design and it is fair to say that there were significant efforts and achievements made within the
constraints mentioned. Thus the allocated SQA score for Replication is still high at 70%.
4.1.10 Risks and Sustainability
Of the 17 enterprises selected, one dropped out in the early stages (Hungary) and was replaced by
another enterprise. Otherwise the Project ran successfully from beginning to end with the 17
selected enterprises.
According to the 2004 APR/PIR, some of the key changes bought about by the Project that should
be maintained into the future include A) Enhanced national capacity in delivering integrated
services to the industrial sector and to the national government on how to address sustainable
industrial production, B) Increased awareness within the industrial sector, C) Increased awareness
within the local environmental authorities.
The enhanced national capacity to deliver integrated services is already demonstrating
sustainability as the demand from the industrial sector for such services is now increasing in all
countries. There are market forces now driving this process that will ensure such sustainability, at
least in the short-term. As EU Directives become more and more pressing, the awareness of both
the industrial sector and the local environmental authorities is becoming increasingly more
focussed on TEST-related issues and the need for formally-recognised and accredited cleaner
production techniques. Certainly the general feedback at the industrial level is that once a company
has successfully attained ISO 14000 accreditation it will make every effort to keep it rather than
have to go through a renewed and costly reappraisal.
The company Nitrokemia (Hungary) presented an interesting case. The company successfully
implemented EMS and EMA, receiving ISO 14001 certification in 2003. However, during the CPA
it was identified that the `good housekeeping' options were marginal and would have had little
effect on the production. The primary options for improvement were at the EST level with a switch
to a new technology. This became the focus of the EST assessment. The new technical solution was
identified and was considered to be acceptable by the company. Implementation was planned for
2004 but had to be postponed due to the on-going restructuring of the company which is being split
into several new companies, some with new owners. Although in some respects this has created a
non-sustainable situation for the TEST process (at least for the time-being) within this company,
Nitrokemia did provide the project with a very good case study for the application of the EMA
process.
As mentioned under capacity building, part of the proof of the effective nature of TEST would be
to review the trained elements of the project at a selected date after the closure of the project to see
how they are using this training. This would apply both to the counterpart institutes and to the
56
demonstration enterprises. This would be a valuable exercise that falls under post-project
evaluation, a concept that is beginning to be included in GEF projects on a more regular basis.
One other issue was raised during the evaluation, which is common to all projects that include
capacity building activities. Once the right people are trained there is a tendency for them to leave
and seek positions that are better paid. Salaries at the technical level throughout the region are very
low. This can be exacerbated at times by international agencies having a policy of preference for
recruiting from the developing countries or those in transition as an act of political correctness. This
tends to act as a drain on the national human resources in such countries. This can inadvertently
result in asset-stripping from such poorer countries in order to maintain a `feel-good' factor within
the international agencies, and to comply with short-sighted policies that do not see the longer term
damage that can be created by such policies.
Several CPCs representative and national counterparts had similar views on sustainability with
respect to next steps. Marketing of the TEST success was seen as a critical factor which needed
more emphasis now that EST was complete. Then financial incentives to A. encourage the
demonstration enterprises to continue to apply TEST processes through Type C improvements
(higher cost long-term technological changes) and to B. bring other enterprises into the TEST
process were seen as a logical progression for the Project objectives. The need for a follow-up
investment phase to consolidate the value of TEST was raised by many stakeholders.
Several stakeholders noted that there was a need to specifically identify individual consultants with
the skills to work with the CPCs and companies in future.
Inevitably the real sustainability of the TEST Project lies not just in the successful demonstration of
the process, but in its transfer and replication as well. This element has already been covered under
Replicability above and further needs will be reviewed under Conclusions .
To summarise, the risks that threatened the project during its implementation were mostly
overcome effectively and efficiently. Sustainability can be viewed in the context of 1. The TEST
concept and process and 2. The efforts within the demonstration enterprises. With respect to 1. This
now very much depends on replication of what has been demonstrated which definitely needs more
effort in order to be seen to be sustainable at either the national or regional level. The second
aspect, sustainability of TEST at the demonstration enterprises level will become clearer with time
and ideally requires re-evaluation some time after closure of the project. As with the replication
assessment above, any major concerns here are focused on Project Design inasmuch as little
consideration was given in the design of the Project Document to Sustainability concerns per se. A
search of the actual Project Document reveals a total absence of the concept of Sustainability in
fact. In fairness, it should be noted that the actual delivery by the Project within the design
constraints has created a reasonable expectation of sustainability inasmuch as awareness and
capacity has been put in place to support sustainability both company, national and regional level
but that this would need further initiatives (see Recommendations ) in order to capture the GEF
investment. SQA score = 75%.
4.2. Project Management and Implementation
4.2.1 Project Design and Planning
Some of the Cleaner Production Centres were involved in the preparation of the Project. At least
one of the CPCs/national counterparts felt that the original Project Document was unsuitable
57
because of 4 years transition between concept and implementation. Many of the hotspots (along
with their factories and machinery) identified by the Pollution Reduction Programme were gone
and new ones had sprung up. There were too many steps involved between UNIDO, UNDP and
GEF and the process of project development was too slow so that the Project Document, when it
was finally approved, was not accurate anymore. This was also one of the reasons behind the
alterations away from the stepwise approach to TEST and the adoption of the EMA approach which
the CPCs supported (and suggested to some extent).
There is a strong case for stating that the constraints of the GEF funding (coupled with unrealised
co-financing) are quite probably responsible for the less-than-successful replication of the TEST
demonstrations and the less-than-perfect transfer of lessons and practices. More appropriate
funding (possibly linked to a longer project timescale) would almost certainly have resolved this
problem and it is the opinion of the Evaluator that the Diffusion component would have been
completely successful in the presence of sufficient time and funds.
There is a strong case for believing that this project should have been submitted as a Full Project
and its submission and adoption as an MSP created a significant risk to its success, especially in
view of the number of countries involved, the transitional nature of those countries, and the fact
that this was the first TEST demonstration project of its kind. The fact that the MSP process was
successful is more a reflection (in the Evaluator's opinion) of efficient and proactive Project
Management and the strong level of ownership and determination by the national counterparts than
of the appropriate selection of GEF project modality. There was a very strong risk that the
demonstration of TEST in at least one country could have failed as a result of these circumstances,
which would have thrown the success of the overall regional project into question.
Another concern related to the MSP modality that has been mentioned previously is the lack of a
formal preparatory (PDF) phase. This project could have benefited greatly from at least a PDF A
and more realistically from a more prolonged PDF B during which a better understanding and
experience of counterpart options could have been developed, and co-financing partnership
obligations could have been confirmed and ensured.
The MSP modality and its funding constraints also did not allow for the inclusion of any technical
or administrative support to the Project Coordinator. The subject matter and structure of the project
was such that the Project Coordinator was required to travel a lot in order to ensure proper
implementation and effective project ownership at the national level, and in order to maintain an
effective dialogue with all national counterparts and the enterprises themselves. Clearly this placed
a heavy burden onto one person for the 3 -year period of the project and reflects both a strong risk
in the choice of the MSP modality, and the need to provide dedicated support staff to senior Project
Management throughout the project lifecycle if they are to perform their duties effectively. It is
insufficient to state that the proof of the effectiveness of the project design lies in the evidence
provided by a successful conclusion to the project. The Evaluator strongly suspects that this has
more to do with the fortuitous selection of a strong and determined Project Coordinator rather then
good project design. This is an all-too-common fault in GEF project design structure and needs
serious consideration for future projects of this nature where efficient project management and
coordination may be undermined by inadequate support or over-optimistic project targets within a
too-limited timeframe.
Another constraint imposed on the project by using the MSP modality was the lack of a significant
project development phase in the participating countries. A PDF B phase is often a valuable
opportunity to seek out strong counterparts and stakeholders. Also in the case of TEST, it would
58
have allowed a more phased approach to enterprise selection, with a pre-project opportunity to raise
awareness in companies at the level of top management before an enterprise was selected. It would
also have allowed more time to confirm co-financing and to ensure formal endorsement of such co-
financing (see Workplan and Budget below).
There was a general opinion amongst most stakeholders that the project would have benefited from
a longer timescale (especially with regard to enterprises implementing CP and EST Option
recommendations), and more funding to include a greater number of enterprises. In some cases it
was felt that several environmentally sound options had been identified but that their integration
into an enterprise's strategy required more time than was available within the time-frame of the
project. This time-frame was also considered to be too optimistic with regard to having an effective
influence on company strategy. However, having said this it does appear that most companies were
impressed with the TEST process and were able to appreciate the improvements that it could make
to productivity and competitiveness.
One area of concern that can be traced back to project design is the issue of demonstrating TEST
processes at companies that produce little by way of water-borne pollution or whose water
consumption is of little consequence. This only really applied to two companies out of the 17. In
fairness to the Project Management, the Project Document itself makes few references specifically
to water-related pollution. In fact, Component 2 Enterprise Demonstrations actually states as one
of its Outputs "Applying the TEST approach to 20 enterprises in order to identify the least cost
solution for complying not only with water related environmental norms, but also with other
environmental norms as required by EU Directive on Integrated Pollution Prevention and
Control". The IPPC Directive clearly includes reduction in the use of raw materials (especially
toxic chemicals), air emissions and energy efficiency among its environmental norms. In future,
Project Documents related to TEST should be specific about which types of enterprises should be
selected in keeping with the aims and Operational Strategy for GEF International Waters. In this
particular situation a case can be made that all forms of pollution, waste generation, or
inappropriate use of resources can be harmful in the long-term to the well-being of the Danube
River Basin.
Although the overall concept of the Project Design was good and the Components were logical
there were a number of design constraints which represented real risks to the project and affected
actual delivery in some areas. These included the fact that the GEF preparation and submission
process was so long that many of the hotspots had changed when the project reached
implementation. The regional Project Management team was effectively one person with no
allowance made for administrative support and the backstopping during the considerable periods of
travel inevitable in a project of this nature. The deliveries were too optimistic for an MSP. The lack
of a preparatory phase and a reasonable inception stage were problematic. Finally, Companies had
barely enough time to adopt and implement their Type A-C options. SQA score = 70%.
4.2.2 Project Management
There is very clear evidence throughout the project of effective and consistent networking between
the Project Management and the various stakeholders (including the national counterparts, selected
enterprises, responsible government agencies, and relevant NGOs). Much of this networking and
communication was at a one-to-one level. The value of this close working relationship and the
consequent strong network was recognised by the counterparts in each country as well as many of
the government and industry stakeholders. The very tangible outcome of this effort by the Project
Management and the Project Coordinator in particular is an established and sustainable network of
59
cooperation and information-sharing throughout the project host countries and beyond into the
other Danube countries. The Regional Coordinator recognised that one possible improvement could
have been the development of community level NGOs to raise awareness and to empower the
community with regard to industrial activities related to specific hotspots. However, this was not
within the remit of the original project document and certainly the constraints of timing and funding
associated with an MSP would not have allowed the flexibility to add such activities without
jeopardising or weakening existing project requirements.
The general consensus from the CPCs/Institutes was that the Project Management had been handled
very capably and very professionally, especially in view of the different levels of capacity in each
country, and the difficulties in early implementation and counterpart selection in at least 2 of the
countries. The early stages of the project required a level of sensitivity and diplomacy, balanced
with firmness and good guidance, and the counterparts felt that this was achieved effectively and
was critical in the overall success of the Project. One CPC, with considerable experience of
working in the region and with UN-type projects, stated that this was "the best-managed project
that they had ever experienced".
The counterparts also valued the regular meetings between themselves and the Project Management
which provided excellent opportunities to share experiences, technological innovations, and new
TEST-related approaches. However, one suggestion for improvement was that the technical
consultants should be included in such meetings in any future Project of this nature, and to arrange
different types of meeting focusing on technical level exchange, training, top management policy,
etc. Although the CPCs recognised the generally weak interaction with government in the TEST
Project they also felt that the project objectives did not require significant input from government
and that TEST can function directly through its relationship with the private sector, with some
simple links to government for information and awareness purposes. The general feeling was that
the project was a successful one because it was directly addressing the private sector as the root
cause of the threat. In the Evaluator's opinion (strongly supported by majority of stakeholders), it is
the strength and determination of the Project Management throughout the implementation of the
Project that has been a major factor in the success of this project. SQA score = 100%.
4.2.3 Project Execution and Implementation
Project Execution was undertaken by UNIDO (United Nations Industrial Development
Organisation). According to the Project Coordinator this UN Agency was the best choice for the
Executing Agency (EA) as they already had considerable experience in cleaner production
techniques and many of the TEST procedures. They also have direct contacts with the existing
CPCs and many related institutions, as UNIDO was instrumental in setting up the CPC network.
The procedures used during TEST (both the technical requirements and the capacity building and
training) are fairly standard approaches for UNIDO. This choice of EA was supported by all
stakeholders, and the Evaluator noted all-round strong support for UNIDO's contribution and
handling of the project execution from all stakeholders questioned.
Project Implementation was through UNDP (the formal Project Implementing Agency IA). No
problems were identified by the project Coordinator and EA regarding the IA/EA relationship nor
did they note any failures on the part of the IA to support the project. One minor criticism of the IA
was aimed at their lack of attendance at any of the appropriate regional project stakeholder
meetings. However, this may also be a reflection of the absence of a formal Steering Committee for
this project as discussed above. In any event it appears to the Evaluator that the Executing Agency
was well in control of Project activities and that there was no apparent need for the IA to involve
60
itself more than it did. There may have been a diplomatic role for the IA with regard to the situation
in Bulgaria where an inappropriate counterpart had been selected, but in the final analysis this
situation was resolved by the Project Coordinator and the project network, without any need to
resort to higher-level diplomatic intervention. Certain CPCs were a little more critical of the IAs
role and stated that they saw little input or interest from UNDP at the national level. Most contracts
were paid directly by UNIDO but in some cases it was through UNDP Country Offices and, where
this was the case, payments were frequently delayed. This was often blamed on `incorrect reporting
and payment request procedures' but the CPCs were given no instructions on how to report or who
to report to and felt that this was unfair. One concern noted previously by the Evaluator that applies
here is the lack of feedback from the IA on the APR/PIR. This is a common criticism within GEF
projects and can have substantially harmful effects where the APR/PIR identifies critical issues
such as the lack of co-financing that was related to this Project. Later discussions with the IA on
this matter revealed that the IA had not provided any response as it considered that these APR/PIRs
were among the best provided by any project each year and required little if any revision. In
particular the IA felt that they were especially strong in reporting on quantifiable indicators.
However, the Evaluator feels that the IAs should provide some feedback to all Project reports (not
just annual but quarterly) even if it is only to inform the Project management that they are satisfied
with the progress indicated by the report and have no specific comments on the content. Lack of
any feedback can be disheartening to an overworked and dedicated project team. Certainly some
response should have been forthcoming in relation to the clear indication that unrealised co-funding
was presenting a problem to the project.
The National Cleaner Production Centres were directly involved in the execution of the project,
while the Regional Environment Centres helped significantly with the dissemination of
information.
Each country had an Advisory Board made up of relevant stakeholders from the CPCs or pollution
control institutes, and from government. For example, in Croatia the AB consisted of a
representative from the Ministry of Environmental Protection and Physical Planning, The Ministry
of Economy, the Ministry of Agriculture and Fisheries, the State Directorate for Water, the
Regional Environmental Centre, and the CPC. The Regional Environment Centre was seen to be as
acting as an `umbrella' representative for NGO organisations. The National Advisory Boards met
2-4 times per year depending on country and need. The national ABs were instrumental in the
selection of the demonstration enterprises for TEST. Preliminary hotspots were taken from the
Danube Pollution Reduction Programme, and then updated and reconfirmed by priority by the State
Directorates for Water. All relevant Ministries were asked to provide input at this stage. The AB
then looked at the long-term viability of the enterprise including its financial situation and the status
of ownership. All candidate enterprises were required to provide a current list of emissions.
Generally, the ABs found that the enterprises could see the overall advantages of cleaner
production in relation to EU requirements and competitive marketing. Some companies were
already aware of the concept of cleaner production through an earlier project financed by USAID.
The Concept of environmental accounting was a fairly new idea and this required some patient
explanation. One stakeholder did raise a concern that although the ABs had a significant role on the
launching of the project (especially identification of the enterprises) they had less of a role in
monitoring and dissemination of the results. Another problem that was noticed during the project
was the continuity of membership of the ABs as a result of changes in Ministry personnel and
responsibilities
61
A final meeting of all National Counterparts was held in Vienna where each country presented its
results. Following this there were the National Dissemination Workshops to which all relevant
national stakeholders were invited.
In conclusion, Project Execution was seen to very effective. Implementation must be considered to
have been less effective and there are concerns here regarding better oversight of Project Design in
relation to standard GEF requirements. These Implementation concerns are reflected in the slightly
lower than perfect SQA score = 80%
4.2.4 Country Ownership/Drivenness
Country ownership of the project resides in the Advisory Boards (ABs) and in the national
counterparts (at the individual and agency level). At the national level, the ABs were directly
involved in the selection process for the enterprises, the identification of which industrial sectors
the project should focus on, and in negotiating the involvement (including signed agreements) by
the selected enterprises. A further function of the ABs was to provide political support to the
project, create national awareness and to promote and disseminate the results. The level to which
the ABS have been effective in this role varies from country to country. The membership of the AB
in each country was discussed and agreed with the national counterparts and with the GEF Focal
Points. The Project tried to ensure a wide selection of stakeholders but with any emphasis on those
directly involved in industrial policy, environmental concerns, and the pertinent EU directives. AB
members were not paid for their involvement so this does represent a direct commitment at the
national level.
All of the TEST project counterparts have been non-profit organisations. Representatives of the
Regional Environmental Centres (which are the most important pertinent NGO in Central and
Eastern Europe) were also members of the national Advisory Boards.
It was notable within one of the 5 project countries that the GEF Focal Point was quite unaware of
the status of the project at the time of the evaluation. Several enquiries on the part of the Evaluator
identified that the fault appeared to lie within the government department responsible and not with
the Project. The department had been consistently appraised and invited to attend project meetings
and workshops but appeared to suffer from either limited human resources or limited awareness
possibly as a result of constant changes in personnel or other priorities. It is however, a fairly
serious concern at the regional level that a GEF Focal Point should be out of touch with a project of
this national significance. This concern should be addressed by UNDP (in coordination with
UNIDO) as the Implementing Agency that has regional responsibility in this case. It should be
noted that where government agencies were actually involved in the TEST project, they were
generally fully committed. However, there was a general opinion among the stakeholders that
government (certainly at the policy level) should be taking a closer interest in the outcome and
lessons from the TEST project. SQA score = 75%
4.2.5 Workplan and Budget (including cost effectiveness)
The Project appears to have stayed fairly close to its original scheduled workplan, although with the
need for some changes in order to embrace the new integrated and modular approach to TEST.
On the topic of cost-effectiveness it has to be stated that the Project objectives and intended
outcomes were overly optimistic within the constrained budget of a GEF Medium-Sized Project.
62
Much of the identified co-funding was never realised as such. If the co-financing had been realistic
and accurate then the MSP approach could have been more appropriate. The Project Management
estimated that only 20% (US$250,000) of the projected co-financing was ever realised. The Project
Coordinator felt that 3 years was not too ambitious although it certainly left little room for
flexibility. But the framework of activities had to be revised to fit within that time constraint. Some
of the activities identified within the stepwise approach (See figure C.1 in the original project
Document) were inappropriately placed, especially under the new modular integrated approach
adopted at the beginning of the Project and needed to be re-shuffled.
The Project Coordinator identified no significant problems with project disbursement of funds. One
recommendation was that fewer contracts with each country would have been safer and more
manageable. A further recommendation was for a preparatory phase for such a project to allow time
to secure co-financing and to develop early working relationships with and between the national
counterparts (especially in those countries which had no CPCs) and to allow for time to discuss and
explain the workplan and budgeting procedures.
The 2003 APR/PIR notes that there is a deficit of -$860,000 between the proposed ($1.48 mill) and
actual ($0.62 mill) co-financing for the project. This deficit is reflected again in the 2004 APR/PIR
but as a slightly higher figure of $930,000. Most of the deficit seems to have been from co-
financing originally expected to be realised through UNIDO. In fact, the actual co-financing
includes additional cash contributions from Governments ($90,000) and Private Sector ($380,000)
that were not in the original Project Document calculations. The actual deficit of contributions is
$1.48 million that was originally expected to come from UNIDO. Although it is not initially clear
from the Project Document or the APR/PIR exactly where the responsibility for the shortfall in co-
financing lies, the Evaluator was informed by Project management that the problem was due to
non-delivery of expected co-financing from the Swiss and Austrian governments to UNIDO, two
countries which had initially been very supportive of the project. Attempts were made through the
Executing Agency to negotiate the delivery of these expected funds but this was unsuccessful. The
Project decided therefore to try to make up the deficit through other potential donors and the Czech
and Hungarian governments came to the project's assistance with co-financing of $90,000. UNIDO
also contributed a further $150,000. It should be noted here that one very positive element with
respect to the co-financing was the significant contribution provided to the project by the
enterprises themselves, both cash and in-kind. The 2003 APR/PIR identifies $308,000 in additional
co-finances representing such enterprise inputs as the allocation of human resources for training in
the various modules and aspects of TEST. Furthermore, the various enterprises between them
contributed $2.2 million by way of adoption of cleaner production and low-cost technological
modification measures.
Generally, budget expenditure seemed to have been handled efficiently and all disbursements were
as planned. The project workplan was followed fairly accurately, taking into account the
modifications to the Project in the early stages and the budget constraints previously mentioned.
However, the budget shortfall mentioned above could have represented a serious risk to the project
and is quite probably responsible for the reduced deliver within Component 3 (Diffusion of
Results) and one or two other areas of the project relating to replication and transfer of lessons.
There is a serious lesson to be learned from this. SQA = 70%.
4.2.6 Monitoring and Evaluation
The Project followed standard procedures for reporting and monitoring. Quarterly Reports were
prepared for UNIDO to submit to UNDP in Bratislava (the Implementing Agency's Regional
63
Office). UNDP Annual Project Reports (APR) combined with GEF Project Implementation
Reviews (PIR) were prepared in 2003 and 2004. The Project received some feedback on these
APR/PIRs from UNDP-GEF in New York, but did not receive any feedback from the Regional
Office on the quarterly reports. This lack of feedback on formal project progress reports is a fairly
common concern raised in GEF projects. If a Project Manager/Coordinator has taken time to
prepare such a quarterly update it does seem reasonable to expect the relevant office receiving the
report to at least acknowledge the report even if only to recognise that the project is running on
schedule and efficiently. Of course, lack of response is a more serious concern if such reporting
does actually identify some issues that may be impeding project progress. One example would be
the deficit of co-financing (as discussed above under Workplan and Budget) which should have
received a response from the Implementing Agency) In future it might be advisable to include a
final section within the report format that requires an official response from and signature from the
IA.
All assessments of project objectives and achievements within the 2003 and 2004 APR/PIRs have
been given a Satisfactory or Highly Satisfactory score (which has presumably been accepted by the
EA and IA) and Risks to the project are rated as Low or (at worst) Modest. The Document also
carries an explanation of the changes in procedure and modality for the TEST project approach, and
relates these to the need to develop and promote synergies between the different and
complementary environmental management tools supporting top management decision-making
processes in medium and long-term planning toward environmental compliance and eco-efficiency.
Overall, the approach was improved and redesigned to be more proactive and sympathetic to the
individual needs of each enterprise, whilst still aiming to capture the lessons and best practices
from the overall TEST process for comparative purposes between each enterprise.
The Project Coordinator noted that a set of project indicators is undoubtedly a useful guideline for
project performance but that there needs to be some flexibility to compensate for changes and
improvements in project activities and outcomes. The Evaluator would fully concur with this
observation. It was felt that the original indicators could have been more appropriate. There was
never really an opportunity to re-focus the indicators after the changes were made to the TEST
procedures. Revised indicators were included in the 2004 Annual Project Report which but these
were less quantitative than the original indicators, and were never formally acknowledge or
adopted. One CPC noted that it would have been helpful to have specific indicators to follow
(within the Project Document) that related more accurately to what the project was trying to
achieve.
It seems that this Project was developed and submitted on the cusp of changes to monitoring and
evaluation by GEF. In this respect, it should be taken into account that at the time of the Project
submission the concept of Process, Stress Reduction and Environmental Status indicators were only
just being introduced and this project was submitted prior to their general adoption. Furthermore,
the Evaluator would certainly accept that the original project indicators were no longer appropriate
for the new integrated and modular TEST approach and were probably never appropriate to the
original project, or indeed realistic. With this change in approach, a new set of indicators should
have been discussed and refined at this early implementation stage in the project and adopted by the
Advisory Boards, EA and IA. It would be valuable in future if the format for the APR included a
question on the continued appropriateness of the original indicators and whether they still serve
their purposes for Project M&E, otherwise projects could find themselves being evaluated on the
basis of inappropriate and out-of-date indicators and could find themselves being allocated low
success rates despite good proactive delivery.
64
The Project Advisory Boards in each country were made aware of the project status vis-à-vis the
results from the APR/PIR assessments. There was no disagreements or additional input required as
far as these stakeholders were concerned.
There was no mechanism included in the original project design, nor are there any current plans to
review the status of the demonstration enterprises or the national counterpart institutions following
closure of the Project. This would be a valuable exercise, as it would be with any GEF Project,
although it is rarely included in the design or in the final Project Document workplan. This is not a
fault of the Project as such. At the time that this Project was submitted, post-project evaluation and
assessment was not considered a requirement by GEF. Even today, it is only included if the Project
designer feels it is important, and it is still not an essential part of a GEF Project life-cycle or
workplan. In the case of this project, a `hindsight' recommendation would have been for UNIDO to
cover the post-project assessment and evaluation as part of its co-funding to the project.
Finally, there was no mechanism built into the original project for its evaluation. This is a
considerable oversight in project monitoring and evaluation and created further financial constraints
to the project as funds had to be reallocated for this work. As a consequence there were barely
sufficient funds available to carry out an effective terminal evaluation of the project. However, the
Project managed to react to this oversight proactively and should be given credit for this.
SQA score for this project criteria = 85%
4.3. Overall Project Impact
4.3.1 Objective Achievements
On a general level, the contribution to the long-term objective to clean-up the Danube River basin
is very positive. The Project has demonstrated real actions and processes that can achieve this aim
and has done so very successfully. According to various stakeholders, it has also catalysed some
changes in political thinking within the participating countries and the overall TEST approach is
accepted within most pertinent government circles as being good practice. At the private sector
level there was overall strong support for the process and the demonstration enterprises now
strongly value and support this approach which relates cleaner production to increased savings, and
a more competitive company status in a wider market. As a bonus the companies also gain an
improved image as they enter a different arena of judgement and criticism within the EU.
Although there were differing levels of achievement within the different enterprises, none of the
selected enterprises withdrew from the Project at any stage, and all of them achieved notable results
and a considerable degree of success by moulding the TEST process and modules to suit their
individual needs. The Project Management consider the primary reason for this success to be based
on the confirmation of the overall TEST approach, namely that improving environmental
performance does not have to be achieved at the expense of competitiveness or through non-
retrievable investment. This in itself has provided a `model' demonstration of the basic premise of
TEST. It should, however, be noted that the EU pre-accession environment in Central and Eastern
Europe at the time of the TEST project was very beneficial to project objectives and delivery, and
provided an enormous incentive to the enterprises.
The Project has undoubtedly achieved its overall objective as stated in the Project Document
4.3.2 Constraints
65
One early constraint on project delivery was the problem of explaining to managers the potential
for savings through more cost-effective use of resources and through the introduction of cleaner
production and reduced discharges. A lot of the resistance to these new ideas can be understood
when considering that costs in CEE for utilities and raw materials are relatively low, as is also the
case for waste disposal fees and associated penalties. The Project identified early in the
implementation process that low stakeholder interest in environmental performance along with
limited external motivation to improve environmental performance represented significant
impediments on persuading companies in the initial project stages to participate in TEST. One
project conclusion has been that the usual economic drivers (such as EU pre-accession
requirements) acted as much stronger incentives the environmental ones. A very strong incentive
that was developed through the TEST process was the realisation of a `hidden' non-product factory
producing high levels of wastes, and representing (for example) 60% of their production costs.
The transitional nature of the region, whilst providing strong incentives through the EU pre-
accession landscape, also created some constraints in enterprise selection which had some influence
on the fact that a final set of 17 rather then 20 enterprises were adopted. The economics and politics
of this region are still transitional and mobile. In some instances, companies that had agreed to sign
up to TEST (or actually had signed up) withdrew a few days later with a change in political
thinking or an overnight change in company ownership. However, generally the CPCs felt that
having these contractual agreements helped to keep the companies on track once the TEST process
had started.
Another constraint to the long-term effectiveness of TEST is the weakness of government
ownership of the project. There has been excellent transfer of ownership of the Project to the
national counterparts and their institutes. But there has been very little transfer of ownership and
knowledge to the government agencies (with possible one exception), especially at the policy level.
Understandably such ownership is difficult to build, as this is a relatively small project with little
funding `incentive' to government to raise its profile. One possibility of capturing this government
ownership exists in the potential for follow-up activities in which government funds (such as
environmental trust funds) are accessed to support some of the TEST activities. In this respect, it is
noted that Croatia has recently established a new Environment Fund (supported by, for example,
collecting annual revenues from CO2 emissions from cars and from company emission fees). The
intention is for this funding to be fed back into emissions and industry to improve environmental
performance. However, there is some concern that the funds may be used for domestic wastewater
treatment instead.
At the regional level, ownership is also not as strong as it could be. The Danube Commission has
provided a forum to present some of the TEST activities and results but this is insufficient at the
regional level in view of the potentially large value of TEST to the other Danube countries, and the
potential for improving the overall environmental landscape of the Danube river basin. Additional
effort and planning at the regional level to drive home the lessons and practices from TEST
(throughout all Danube countries) is an essential next step.
In at least one country, the TEST team noted that although they were able to build commitment at
the higher management level, the lower levels were often reluctant to embrace the TEST process as
they saw it as additional work with no real personal pay-back, plus they felt that management didn't
really appreciate the extra effort they had to put in. The team had to work hard to overcome this
attitude initially, which was a product of poor top-down communications.
66
One fairly significant constraint placed upon the project reflects a less than adequate Project
Design. This has been discussed in some detail under the relevant sections. The Evaluator was
concerned to note that there were no references whatsoever in the Project Document to either
Sustainability or Replication of project activities and results. This is further reflected in the fact that
although the project delivery has been superlative, there is no effective mechanism to capture the
lessons and best practices and to actively transfer and replicate them. Without such a mechanism
the GEF investment is questionable. This is addressed (in relation to possible solutions) later under
Recommendations .
5. CONCLUSIONS OF EVALUATION
This Project has provided a highly valuable demonstration to the industrial private sector of how
cleaner production and more environmentally sound technologies can be embraced successfully by
companies without jeopardising their market positions, and without incurring any significant long-
term costs. In fact, the TEST process has demonstrated that CPA and EST can add a significant
market-level advantage and can increase productivity and make companies more cost-effective
while producing less waste materials. These waste materials are not only expensive to dispose of
but also constitute a significant `production' cost to the companies. Therefore the TEST process has
been clearly demonstrated to be a `win-win' exercise for all stakeholders.
One of the major benefits of the CP assessment for the participating enterprises was to provide the
management with a clear definition and identification of the source of their pollution problems, and
closely related to this, the source of much of their loss of production costs. This was then related to
a structured planning process including an analysis of the causes, and identification of a selection of
measures for resolving the problems. Faced with a multitude of potential CP and EST solutions, the
methodology enabled the management to prioritise their actions and select the most effective
solution. One significant output related to the EMA modules is the change in management attitude
away from an intuition-driven investment approach to a proper investment appraisal including
evaluation of environmental impacts in monetary terms.
The demonstration enterprises have made significant improvements at both the type A and B level,
with some type C (high investment, long-term payback) being implemented also. But the
enterprises now find themselves constrained by the need for long-term costly investments and are
keen to identify possibilities for favourable loans and assistance.
Beyond this, the Project has developed capacity and strengthened institutions within the region. The
existing national CPCs have learned new skills and been given new tools with which to advice and
support industry in the growing need for CP and EST improvements linked to the demands and
requirements of the EU and its IPPC and ISO requirements.
The Project has developed a wealth of sound lessons and valuable best available practices which
can be shared and transferred not only throughout the region but also throughout the world. It is
imperative that these lessons and BATs should not be lost.
Now, to be truly effective TEST does need a transfer-and-replication phase that goes beyond just
the admirable but fairly limited national dissemination workshops that were built into the project
design. If there were a shortcoming of the TEST Project then this would be it. Project Management
were aware of this and recognised it as a constraint to project success but, once more, were clear
that financial constraints and initial project design would not allow more than was carried out. At
the very least, the project achievements should be captured and sustained through national
67
programmes promoting TEST, and using the capacity and resources now already in place. Greater
recognition of the value of the TEST process by government departments, coupled with more
widespread awareness within the private sector of the availability of specialist assistance
(Consultants, CPCs, other institutes) would be highly beneficial in this respect. There may also be a
case for enhancing UNIDO's own promotion of the TEST approach within its country-specific
programmes and the through its network of CPCs.
In this respect the Project Management is to be applauded for its persistence and tenacity in
ensuring that the objectives of the Project were not undermined by some of the financial and project
design constraints as well as some of the difficulties encountered in the early stages as a result of
the absence of a preparatory phase. Furthermore, the Project Management has shown a good,
proactive approach to project implementation which has proved to be responsive to the evolving
needs of the clients (the enterprises and the national counterpart institutions) and to the Project
Objectives as a whole. There are several examples of this proactive flexibility that is essential to a
dynamic GEF project. The early modification of the TEST process in collaboration with the
counterparts, the resolution of the problems with the earlier selection of inappropriate counterpart
institutions, the time constraints imposed by the MSP modality along with the funding constraints,
etc. All of these could have proved to be a substantial risk to the success of the project if they had
not been effectively addressed, both quickly and efficiently.
Equally, the national counterparts are to be applauded for their commitment and support both to the
project itself in their endeavours to build their own capacity and experience, and to the enterprises
with respect to the amount of time that the counterparts spent in raising awareness and then
introducing and completing the TEST modules.
The Overall Project Achievement and Impact SQA score and Rating is shown below in Table 12.
The Project has achieved an overall evaluation rating of 4.2 which is equivalent to Excellent
Delivery Overall.
In the Evaluator's opinion, the TEST project demonstrates one of the best investments that GEF has
made in a MSP project within the International Waters portfolio. The important step now will be to
build on this valuable investment and to capture and transfer the lessons, practices and procedures
developed through this very successful project.
68
TABLE 12:
OVERALL SQA SCORES AND RATING FOR PROJECT ACHIEVEMENT AND DELIVERY
PROJECT ACHIEVEMENT
ESTIMATED PERCENTAGE SUCCESS OF COMPONENT OR OBJECTIVE
RATING
5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100
1-5
PROJECT DELIVERY
OUTPUTS AND ACTIVITIES
4.5
THREATS & CAUSES RESOLUTION
4
GLOBAL & NATIONAL BENEFITS
5
STAKEHOLDER PARTICIPATION
4
CAPACITY BUILDING
5
POLICY AND LEGISLATIVE REFORMS
5
REPLICABILITY
3.5
RISKS & SUSTAINABILITY
3.75
PROJECT IMPLEMENTATION
PROJECT DESIGN & PLANNING
3.5
PROJECT MANAGEMENT
5
PROJECT EXECUTION & IMPLEMENTATION
4
COUNTRY OWNERSHIP
3.75
WORKPLAN AND BUDGET
3.5
MONITORING & EVALUATION
4.25
OVERALL ACHIEVEMENT & IMPACT
4.20
69
6. RECOMMENDATIONS
6.1 Recommendations Specific to the TEST project and similar Projects
A. It is now very important that the lessons and best practices evolved through the TEST
project are properly captured and transferred to other relevant projects, initiatives and
agencies. There would be considerable value in the production of a single, consolidated
technical report on the TEST Project that includes the results and achievements from every
demonstration enterprise along with documented lessons and best practices. A valuable
output from the TEST Project as a whole would be a set of Knowledge Products carefully
crafted and aimed at such audiences as the Danube Regional Project, ICPDR, the EU, the
GEF Dnieper River Basin project, and other future GEF projects. Such Knowledge Products
are now a growing requirement of UNDP as part of the evaluation of its projects and it may
be that UNDP could assist in the development of such a package. These could focus on
capacity building and institutional strengthening experiences at the project level, as well as
on BAT and TEST-related lessons at the technical level.
B. The existing national CPC network needs to be sustained and expanded, both
geographically and in substance. The two countries that participated in the TEST project
without a formal CPC need assistance to develop such a centre, and other countries in the
Danube region also need similar assistance. The CPCs should receive all the necessary and
appropriate training to raise them beyond just CP centres to EST centres including training
and experience in the TEST modules. This would help to provide the necessary framework
for more effective transfer and replication of the TEST project results (lessons and Best
Practices). Furthermore, some consideration should be given to marketing of the CPC
expertise within the private sector, not only in the context of their ability to undertake
appropriate TEST modules but also as impartial, independent and non-commercially
orientated advisors to industry on BATs.
C. An effective single CPC networking facility should be maintained at UNIDO based on the
advances made through the TEST project in this area. This should include development of
the Information Management System linked into ICPDR, the Danube Regional Project and
other relevant bodies.
D. Existing demonstration enterprises need some further assistance if possible with identifying
soft-loans and investment opportunities to support their approved type C (and, in some
cases, type B) EST improvements. Such soft-loans could also be made available to new
enterprises wishing to pursue the TEST approach, once they have demonstrated
commitment through the implementation of type A and type B improvements. Furthermore,
companies have identified the need for some financing to help them to develop the technical
aspects of the actual loan applications.
E. There is a need to involve government more closely in the TEST concept and to inform
them of the successes and potential advantages at the national level. This requires a
concerted effort of targeted awareness. Government agencies need to be engaged in the
TEST process as there may well be further concessions and incentives that they can offer to
companies to bring them on board in the first instance (bearing in mind that once companies
have been trained and sensitised in the TEST approach they usually see the advantages for
themselves).
70
F. Some level of post-project assessment would be valuable to see how sustainable the TEST
process has remained within the companies and at the national level. There is a potentially
valuable role for the CPCs to play in this assessment process which could actually go
beyond just assessment and review and involve active encouragement to the demonstration
enterprises to continue their TEST-related activities.
G. In consideration of A-F above, there is a strong case for the development of a follow-up
phase to the TEST MSP. Such a phase could concentrate on capturing the lessons and best
practices, replicating them throughout the Danube region, using the lessons learned to
strengthen the network of CPCs, developing an expanded programme of TEST throughout
further enterprises with more support (both financial and policy-wise) from governments
which includes investment options and soft-loans for the larger EST needs, etc. The
requirement for such a follow-up phase was a consistent priority identified by nearly all
stakeholders during this evaluation and would seem to be a logical consolidation and
expansion of the relatively small but successful investment made so far through the MSP.
6.2 Recommendations Specific to the GEF Project Design and Implementation Process
A. In future, careful consideration should be given to the appropriateness of the MSP modality
for regional projects of this nature. This modality has constraints related to the lack of a
substantial preparatory phase (such as a PDF B) during which project partnerships are
developed, institutional weakness and strengths can be assessed, and financial commitments
vis-à-vis co-financing can be formally confirmed and endorsed. Furthermore, the MSP
modality, by virtue of its fixed funding ceiling, inevitably has constraints related to the
project timescale and to the level of funding available for project management and
administrative support.
B. Implementing Agencies should develop a policy of response to all project quarterly reports
and APRs. Ideally, the APR/PIR format should require a signed response from the IA to
ensure that the Agency is aware of concerns raised in the report and has identified actions
that are be taken to rectify these concerns.
C. Any changes in the Project Document and its Outputs and deliveries should be reflected
(where appropriate) in corresponding changes to the Project's indicators. These should be
documented within the APR/PIR and approved by the EA and IA.
D. GEF projects should strive to develop appropriate measurable indicators with the Mid-Term
and Terminal Evaluation processes in mind. Evaluators depend on these verifiable
indicators to measure the success of a Project and their absence can cause an overly
subjective review. In this context, GEF and IAs should consider modifying MSP formats to
include the Logical Framework approach (or similar logical progression leading to
identification of measurable indicators) as a requirement.
71
7. List of Annexes
Annex I:
ToR
Annex II:
Lists of Persons/Agencies/Bodies interviewed
Annex III:
List of Documents Reviewed
Annex IV:
Questionnaire sent to non-mission country stakeholders
72
ANNEX I:
TERMS OF REFERENCE FINAL EVALUATION
Project number - EG/RER/00/G35
Project Title - Transfer of Environmentally Sound Technology (TEST) in the Danube River
Basin
Background
Enterprises in the countries of the middle and lower Danube River basin are facing numerous challenges as
they go through a radical reshaping in their move towards market economies. At the same time, they are in
the process of responding to the environmental objectives of the Danube River Protection Convention
(DRPC) and the environmental requirements that come with accession to the European Union; the most
significant of these is the EU's Integrated Pollution Prevention and Control (IPPC) Directive. The DRPC
and the EU's IPPC Directive both require enterprises to apply best available techniques (BAT/EST) and best
environmental practices (BEP).
In 1997, the United Nations Development Programme (UNDP), with the financial support of the Global
Environment Facility (GEF), launched the `Pollution Reduction Programme for the Danube River basin',
through which it identified 130 major manufacturing enterprises known as `hot spots' that were significant
sources of pollution to the waters of the Danube River basin. A significant number of these enterprises were
contributing to transboundary nutrient and/or persistent organic pollution.
In April 2001, UNIDO started the implementation of the TEST programme taking on the challenge to
effectively demonstrate to the industries of five Danubian countries (Bulgaria, Croatia, Hungary, Romania
and Slovakia) that it is possible to comply with environmental standards and still maintain, or even enhance
their competitive position.
The TEST programme's primary financial supporter was GEF, with some participation from UNIDO and
other donors (the Hungarian and Czech Governments). The project's national partners (counterparts) were
the National Cleaner Production Centres (NCPCs) of Croatia, Hungary, and Slovakia (members of the
UNIDO/UNEP network of NCPCs), the Institute for Industrial Ecology (ECOIND) in Romania, and the
Technical University of Sofia in Bulgaria.
The objective of the TEST project was to build capacity of the national institutions in the five selected
Danubian countries in integrated environmental management (TEST integrated approach) for the Transfer
of Environmentally Sound Technology. The TEST approach has been introduced at selected industrial hot
spots to demonstrate that it is possible to comply with environmental norms of the Danube River Protection
Convention while at the same time taking into account the needs to remain competitive and to deal with the
social consequences of major technology upgrading. The enhanced institutional capacity would then be
available to assist other enterprises of concern in these countries as well as other Danubian countries.
73
The implementation of the project consisted of three operational components:
· Component I. Institutional Strengthening
· Component II. Entreprise Demonstrations
· Component III. Dissemination of Results
The TEST programme in the Danube River basin targeted 17 hot spots of industrial pollution (the list of
enterprises is provided in table 1), of various industrial sectors. Through the programme, these industrial
polluters have been introduced to the TEST integrated approach having the opportunity to learn how to use
environmentally sound technologies to improve productivity, whilst at the same time complying with
environmental norms and regulations.
Country
Selected Enterprise
Industrial Sector Focus of the project
1
Croatia
Agroproteinka
Meat Rendering
2
Gavrilovic d.o.o.
Integrated meat processing
3
Herbos
Pesticides- Atrazine plant
4
IPK Tvornica Secera Osijek
Sugar production
5
Romania
ASTRA Romana
Petrochemical refinery
6
Rulmentul
Machinery bearing production
7
Chimcomplex
Intermediate Chemicals -Isopropyl -amine
8
SOMES
Pulp and paper
9
Slovakia
AssiDoman Sturov
Pulp and paper
10
Zos Trnava
Machinery - repair railway wagons
11
Hungary
Gunter Tata Kft.
Machinery - repair railway wagons
12
Indukcios es Vedogazos
Steel heat treatment
13
VIDEOTON Audio Company
Electronic products, plastic and wood processing
14
Nitrokemia 2000
Intermediates Chemicals
15
Bulgaria
Yuta JSC
Textile
16
Slavianka JSC
Fish processing
17
Zaharni Zavodi AD
Sugar - Alchool production
Table 1 List of demonstration enterprises participating in the TEST project
TEST project implementation strategy
The project implementation strategy has been adjusted during project implementation, as indicated in the
UNDP/GEF Project Implementation Report (PIR2) June 2003 (section 3) in order to reflect country specific
conditions and to achieve the project objectives in a timely and cost-effective way. By replacing the original
step-by-step approach with the integrated approach, the revised TEST project implementation strategy
promotes synergies between different and complementary environmental management tools supporting top
management decision-making processes in medium and long-term planning toward environmental
compliance and eco-efficiency.
The TEST integrated approach to industrial environmental management developed by UNIDO, has been
designed to assist enterprises in the developing and transitional countries to effectively adopt
Environmentally Sound Technology (EST). The application of the TEST integrated approach and its tools,
leads to continuous improvement of the economic and environmental profiles of companies.
The integrated TEST approach is based on several three basic principles:
.
1. First, it gives priority to the preventive approach of Cleaner Production (systematic preventive
actions based on pollution prevention techniques within the production process) and it moves
intoconsiders the transfer of additional technologies for pollution control (end-of-pipe) only after the
2 For additional details on the revised implementation strategy see the related PIR - June 2003
74
cleaner production solutions have been explored. This leads to an transfer of technologies aimed at
optimizing environmentally and financial ly optimized elementstransfer of technologies: a win-win
solution for both areas.
2. Second, the integrated TEST approach addresses the managerial aspects of environmental
management as well as its technological aspects, by introducing tools like such as Environmental
Management System (EMS) and Environmental Management Accounting (EMA).
3. Third, it puts environmental management within the broader strategy of environmental and social
business responsibilities, by leading companies towards the adoption of sustainable enterprise
strategies (SES).
The scheme below shows the stages of the revised TEST implementation strategy.
Objective and Scope of the Evaluation Mission
The evaluation is an activity in the project cycle, which attempts to determine as systematically and
objectively as possible, the relevance, efficiency, effectiveness, impact and sustainability of the project
against its objectives. It will also identify factors that have facilitated or impeded the achievement of these
objectives. While a thorough review of the past is in itself very important, the in-depth evaluation is
expected to lead to detailed recommendations and lessons learned for the future.
The mission will assess, with respect to:
75
Project Design
-
Appropriateness of the project's concept and design to the overall situation of Danube River Basin
and related situation of the industrial hot spots
-
Relevance of the project design within the framework of GEF programmes and guidelines
Project Implementation
-
General implementation and management of the project in terms of quality and timeliness of inputs
and activities, adherence to workplans and budgets
-
Adequacy of management arrangements as well as monitoring and backstopping support given to
the project by all parties concerned
-
Institutional set-up through the National Advisory Board mechanism and the degree to which it has
encouraged full involvement of the countries
-
Responsiveness of project management to changes in the environment in which the project operates
-
UNIDO execution modality
-
Co-operation among project partners (UNIDO, UNDP/GEF, Commission for the Protection of the
Danube River Basin (ICPDR), National counterparts and National Coordinators)
Project Impact
-
Achievements of the project against the original objectives, outputs and activities as detailed in the
project document
-
Level of awareness of the participating countries regarding project outputs
-
Level of ownership of the project by the participating countries
-
Level of commitment of the countries to support ongoing project and potential future replication
activities
-
Degree of support given by the riparian Governments in integrating the project objectives and goals
into the national development programmes and other related projects, and how well the project fits
into national development policy
-
Impacts on policy and strategy of countries
-
Project impact on enhancing co-operation among industrial sectors and project partners
76
Conclusions and Recommendations
Based on the above points, the evaluation mission should provide conclusions and recommendations,
including:
-
General feedbacks on the implementation of the project
-
The degree to which the project objectives have been satisfied
-
Significant lessons that can be drawn from the experience of the project and its results, particularly
those elements that have worked well and those that have not
-
Recommendations on further actions in the region upon completion of the current project
Methodology
The evaluation will be composed of three activities: studying documents, observing project participatory
processes, and interviewing individuals who are either involved in the project, or who have or might be
expected to have been impacted by the project.
(i)
Studying documents
The team members shall familiarize themselves with the project through a review of relevant documents
prior to travel to the region. These documents include interalia,
-
Project Document
-
TEST project manual
-
National publications
-
GEF Project Implementation Review (PIR) 2003 and 2004
-
UNDP/GEF Quarterly Operational Reports (QOR)
Electronic copies of selected documents will be sent to the Evaluator in advance of the mission.
(ii)
Observing project participatory processes
-
Observe national dissemination seminar in Sofia, Bulgaria (Date to be fixed second half of February
2005)
(iii)
Interviewing project partners
-
Project Coordination Unit at UNIDO
-
Selected national coordinators in the participating countries
-
UNDP Country Office focal points in the participating countries
-
GEF focal points and UNDP-GEF regional coordinator for Central Eastern Europe
-
ICPDR EMIS group - focal points
-
National counterparts/ project coordinators / co-operating institutions
-
Representatives of demonstration enterprises
And in doing so, undertaking field visits to at least two project countries.
77
Composition of the Mission
An international consultant who shall be responsible for the overall review of the project shall conduct the
project evaluation. He should have extensive technical and managerial background at both policy and
institution level of environment and international waters in particular, and in-depth experience of project
evaluation techniques, particularly of those projects, which are funded by GEF. Experience in Central and
Eastern Europe is preferable. Fluency in English is required.
Mission timetable and itinerary
The duration of the consultancy for the Team Leader shall be 15 working days, including travel time, based
on the following itinerary:
2 days
Home-base review of relevant documents
1 day
travel to Vienna
2 days
Interviews with UNIDO project management unit and ICPDR
1 day
Travel to project country one (to be identified)
2 days
National counterparts, Members of national advisory boards and enterprise visits
1 day
Travel to another project country (to be identified)
2 days
National counterparts, Members of national advisory boards and enterprise visits
4 days
Draft and finalize report
Report
In drafting the report, the consultant will be guided by the standard UNDP Guidelines for Evaluators.
The final report should contain the following Annexes:
· Terms of Reference for final evaluation
· Itinerary (actual)
· List of meetings attended
· List of persons interviewed
· Summary of field visits
· List of documents reviewed
· Any other relevant material
As the report is the product of an independent evaluation, it is up to the consultant to make use of the
information provided during the mission. However, the evaluation team is responsible for reflecting any
factual corrections brought to their attention prior to the finalization of the report. Therefore, in order to
ensure that the report considers the view of all parties concerned, is properly understood, and is factually
accurate, it is required that the consultant submit draft report to UNIDO and UNDP/GEF. UNIDO will revert
promptly with collective feedback from project partners in order that the evaluator may finalize his/her
report.
The final version of the evaluation mission report should be submitted in electronic format (MS Word) to
UNIDO and UNDP/GEF no later than one month from the date of start of the evaluation. The report shall
also be submitted in five hard copies to UNIDO.
Contact information for UNIDO and UNDP/GEF:
UNIDO
Mr. Pablo Huidobro
78
Chief International Waters Unit
Energy and Cleaner Production Branch
phuidobro@unido.org
tel.: 0043-1-260263068
Ms. Roberta De Palma
Project manager
Energy and Cleaner Production Branch
rdepalam@unido.org
tel.: 0043-1-260263486
UNDP GEF Regional Office
Mr. Nick Remple
GEF Regional Coordinator for Biodiversity and
International Waters
Europe and CIS SURF
E-mail: nick.remple@undp.org
Tel.: +421-2-59337458
79
ANNEX II:
LIST OF PERSONS/AGENCIES/BODIES INTERVIEWED
UNDP
Andrew Hudson UNDP GEF Principal Technical Advisor on International Waters, UNDP NYC
Juerg Staudenmann Water Governance Advisor, UNDP Bratislava Regional Centre, Slovak
Republic
UNIDO
Roberta De Palma GEF TEST Regional Programme Coordinator UNIDO Vienna
Pablo Huidobro, Chief, Water Management Unit, UNIDO, Vienna
Mihaela Popovici Technical Expert Water Management and Pollution Control - ICPDR
Slovak Republic Stakeholders
Viera Feckova Director - Slovak Cleaner Production Centre (TEST Project NFP)
Jana Balesova National coordinator- Slovak Cleaner Production Centre
Jaroslav Burjaniv Lead Test Consultant TEST to ZOS Trnava enterprise- Slovak Cleaner
Production Centre
Vladimir Hudek Director Slovak Regional Environmental Centre
Zdena Kenarova Senior Advisor Ministry of Environment (ICPDR focal point)
Juraj Sikra Desk Officer for UN Agencies Ministry of Foreign Affairs
Michal Deraj Project Manager - Slovak Regional Environmental Centre
Rudolf Brezovsky Director Zos Trnava Enterprise
Frantisek Ruzicka Deputy-Director - Zos Trnava Enterprise
Jaroslav Striz Market Manager Zos Trnava Enterprise
Frantisek Drgon - Systems Manager Zos Trnava Enterprise
Croatian Stakeholders
Marijan Host Director Croatian Cleaner Production Centre (TEST Project NFP)
Morana Belamaric National Coordinator -Croatian Cleaner Production Centre
Zeljka Medven Project Manager Regional Environmental Centre
Ivana Halle Head of Department, Ministry of Economy, Labour and Entrepreneurship (and
UNIDO Focal Point)
Visnja Jelic Muck State Secretary Environmental Protection Division - Ministry of
Environmental Protection, Physical Planning and Construction
Gordana Ruldic Assistant to State Secretary - Environmental Protection Division - Ministry of
Environmental Protection, Physical Planning and Construction
Mojcu Luksic ICPDR Focal Point Minsitry of Agriculture, Forestry and Water Management
(previously Head of Water Policy Section in EMIS)
Slavko Brebric Technical Director Gavrilovic Enterprise
Milica Opacic Head of Environmental Management - Gavrilovic Enterprise
Dascha (Unknown Surname) Head of Human Resources - Gavrilovic Enterprise
Janez Petek CPC Consultant for IPK Tvornica Secera Osijek Enterprise Demonstration
Nafis Defterdarovic CPC Consultant for Gavrilovic Enterprise Demonstration
Bulgarian Stakeholders
Tzanko Tzanov Associate Professor Technical University of Sofia TEST Project National
coordinator - Bulgaria
Alexandar Kiriy Professor and Dean Faculty of Power Engineering and Power Machines
Technical University of Sofia
80
Milko Lovchev Professor and Expert in Water Chemistry COGen E ngineering Ltd (TEST
Project Team)
Margarita Neznakomova Sector expert- Head of Non-Woven and Technical Textiles Laboratory
Department of Textile Engineering Technical University of Sofia (TEST
Project Team)
Nikola Stankov Energy Efficiency Expert - Technical University of Sofia (TEST Project Team)
Dimitar Kiossev Sector Expert - Technical University of Sofia (TEST Project Team)
Velio Donchev Sector Expert - Technical University of Sofia (TEST Project Team)
Ivona Grozeva EMS Expert for Yuta enterprise (TEST Team- Ministry of Environment
Ilian Ivanov Financial Expert University of National and World Economy
Maria Nonova Expert on Permit Issuance for Waste Treatment and Transportation, Waste Oil and
PCB Waste Management Directorate - Industrial and Hazardous Waste
Management Department Ministry of Environment and Water.
Ivanka Zanivar Technologist and Head of Laboratory and Environmental Concerns - Zaharnia
Zavodi AD Demonstration Enterprise
The Evaluator also received email feedback from the following consultants involved in TEST:
Ivan Zavadsky - Project Manager - UNDP/GEF Danube Regional Project
Vladimir Dobes CP Trainer and consultant (international expert) Assisting in design of the TEST
methodology, providing on-site technical assistance in implementation of the CP
module in Romania and Bulgaria and helping with writing the TEST report
(UNIDO publication).
And received completed Questionnaires (see Annex V for template below) from:
Liliana Chirila - Director Sectoral Operational Programme Management Directorate Ministry of
Environment and Water Management Romania (former GEF focal pint
Romania)
Ágnes Kajdacsy - QMS - EMS Consultant (former environmental manager of Nitrokemia
Enterprise Demonstration Hungary
Mihaela Oprescu Senior Consultant (TEST team) - Head of Environmental Department,
IPROCHIM (Chincomplex) Demonstration Romania
Margareta Nicolau - Managing Director - National R&D Institute for Industrial Ecology (ECOIND)
Bucharest - Romania
Maria Teodorescu National coordinator TEST/Romania -Environmental Senior Consultant
International Programmes, National R&D Institute for Industrial Ecology
(ECOIND) Bucharest Romania
Maria M.Galambos - Chief Senior Counsellor - Ministry of Environment and Water, Hungary
Sándor Kerekes - Dean of Faculty of Business Administration - Corvinus University of Budapest
Hungary (National coordinator TEST/Hungary)
81
ANNEX III: LIST OF DOCUMENTS REVIEWED
1. The Project Document
2. The Project Brief
3. UNDP/GEF Quarterly Operational Reports (QOR)
4. GEF Project Implementation Review (PIR) and Annual Project Review (APR) 2003 and 2004
5. TEST project manuals
6. National publications
7. UNIDO TEST Website
8. ICPDR Website
9. DRP Website
82
ANNEX IV: QUESTIONNAIRE SENT TO NON-MISSION COUNTRY
STAKEHOLDERS
GEF TERMINAL EVALUATION TRANSFER OF ENVIRONMENTALLY SOUND
TECHNOLOGY IN THE DANUBE RIVER BASIN
GUIDANCE: Please answer the questions briefly but feel free to add extra comments to explain
your responses. Not all of the questions may be appropriate to your involvement in the project or
may be outside of your experience in which case please ignore those particular questions. Sections
written in black explain the aim of the question and provide some background. The Sections in
blue are the actual questions
A. OVERALL OBJECTIVE
To build capacity in existing cleaner production institutions to apply the UNIDO Transfer of
Environmentally Sound Technology (TEST) procedure to technology transfer to 17 enterprises that
are contributing to transboundary pollution, and primarily nutrients, in the Danube River Basin and
the Black Sea.
Has the Project achieved its aims in delivering its primary outcomes to:
A.1 To strengthen capacities to introduce Cleaner Technologies for pollution reduction
(with Cleane r Production centres supporting industries to reduce pollution)
A.2 Improve water quality (and reduce the number of Danube industrial hotspots)
B. THREATS AND CAUSES
B.1 Has the project made a significantly contribution by demonstrating workable
solutions to the main problems effecting water quality in the Danube? In particular
has it demonstrated effective and acceptable solutions for industrial installations in
relation to the following:
o Reducing nutrient loading into the Danube and Black Sea
o Reducing other hazardous industrial discharges
o Reducing sewage and other organic inputs from human settlements and activities
o Reducing extraction levels of water for cooling and other industrial processes
B.2 Has this been demonstrated through improvements in water quality and an overall
reduction in harmful discharges if so how has these been measured and quantified?
C. OUTPUT AND ACTIVITY (Indicators provided by MSP Project Document)
Component 1: Establishment of TEST Focal Points
1.1 Set up TEST Focal points in NCPC or PPC.
1.2 Identify appropriate institutions and experts to support NCPC/PCCs to carry out project
requirements
1.3 Introduce Focal points to test Programme, install information management system (linked to
ICPDR), prepare implementation schedule for TEST at selected enterprises
Original Indicators
Ø Networks in 5 countries providing integrated package of services needed to achieve
compliance with environmental norms
Ø NCPC/PPCs have a new service to market to industrial clients
83
1:1 In your opinion, has the project met its Indicators for Success under Component 1? If not what are the likely
reasons?
Component 2: Application of the TEST procedure to 20 enterprises
2.1 Selection of enterprises and securing their commitment to participation in TEST
2.2 Training of TEST teams in analytical techniques (see Figure C.1)
2.3 Application of TEST procedure to 17 enterprises to identify least-cost solution to comply
with environmental norms
Indicators
Ø EST options for nutrient and toxic pollutant reduction under active investment negotiations
in 10 enterprises by end of project.
Ø Measurable nutrient/toxic/organic pollutant reduction (down by 30%) in 8 of the
enterprises by end of project, and some reduction in the remaining enterprises.
Ø EST options for organic pollutant reduction under active investment negotiation in 10
enterprises.
Ø Measurable organic pollutant reduction (down by 40%) in 12? of the enterprises by end of
project.
2:1 In your opinion, has the project met its Indicators for Success under Component 2? If not
what are the likely reasons?
Component 3: Disseminating results to other enterprises and countries
3.1 Preparation of information material for TEST procedure, including the Test Manual and Case
Studies.
3.2 National seminars in each of 5 participating countries.
3.3 One-day assessment at pollution enterprises to demonstrate benefits of applying TEST
procedure.
3.4 Introduction of TEST procedure to the 6 other Danubian Countries through a regional
workshop.
Indicators
Ø TEST Management Toolkit (technical manuals & enterprise reports) for nutrient and toxics
reduction available.
Ø TEST Management Toolkit for organic pollutant reduction available.
Ø At least 20 of 130 enterprises committed to implementing TEST procedure by end of
project
Ø
3:1 In your opinion, has the project met its Indicators for Success under Component 3? If not
what are the likely reasons?
Expected Project Outcomes:
Description of Expected Project Outcomes:
As far as you are aware:
84
C.4. Have all participating enterprises prepared a Sustainable Enterprise Strategy and do
these SESs explain how they would implement best available techniques and best
environmental practices to integrate successfully ESTs into their production processes?
C.5. Have Environmentally sound technology options (a combination of advanced proce ss,
pre-treatment and final pollution control technologies) been successfully identified for 15
enterprises which bring them into compliance with environmental norms of the EU and the
Convention?
C.6. Have there been at least 30 per cent pollutant reductions within 10 participating
enterprises, with some pollutant reductions in the other 10 as a result of implementing
cleaner production (process change) measures?
C.7. Have training courses been undertaken by international experts and have these built
capacity within national institutions to advise the remaining industrial hot spots in the five
participating countries on implementing the TEST procedure?
C.8. Has a TEST management toolkit been developed and made available to national experts
to assist remain ing enterprises to identify and install the appropriate ESTs at least cost.. and
how effective and useful is such a kit?
D. GLOBAL AND NATIONAL BENEFITS
D.1. Operational Programme
D.1.1. Does the project address the aims and objectives of OP8 (waterbo dy-based) and
OP10 (contaminant-based) both in its design and in its implementation?
D.2. Conventions
Statements from Convention
D.2.1 Does the project address the requirements and priorities of the Danube River
Protection Convention and the Danube River Pollution Reduction Programme?
D.3. National Priorities
D.3.1 Were the national priorities and their linkage to GEF's global aims within this
project clearly defined in the Project Document?
D.3.2 Is the project addressing national priorities for develo ping ESTs and for pollution
reduction?
E. PROJECT MANAGEMENT AND IMPLEMENTATION
E.1 Has the Executing Agency fulfilled its responsibilities effectively as per its ToR?
E.2 Has the Implementing Agency fulfilled its role as required?
E.3 Has the PIU/Project Management Unit functioned effectively in its management role?
E.4 Have the channels of communication between the PIU, the EA and the IA worked
e ffectively and has response to requests for assistance etc been forthcoming and
timely?
85
E.5 Has the Project Management networked effectively with other project stakeholders
(e.g. national focal points, hotspot industries, responsible government agencies,
relevant NGOs?
E.6 Are there any improvements which could have been made to Project Management
and Implementation which would be worth capturing for future projects of this
nature?
F. WORKPLAN AND BUDGET
F.1 Has the project followed the workplan (taking into account any amendments made by
the Project National Advisory Boards)?
F.2 Was the Workplan realistic in terms of delivery and timing?
F.3 Has the budget proved to be an accurate assessment of the project's financial needs?
F.4 Have their been any problems in disbursement?
F.5 Based on experiences from this project are there any recommendations regarding
workplan delivery and budget disbursement which would improve efficiency for
further projects
G. MONITORING AND EVALUATION
G.1 Were adequate monitoring and Evaluation procedures (Indicators, criteria for
measuring performance, results, impacts, etc) built into the project design?
G.2 What standard UN-GEF M&E procedures were followed (Quarterly and Annual
reports, site visits, representation at SteerCom?).
G.3 Have these M&E procedures been followed and implemented? (reporting, PIR,
Tripartite Review, MTE?)
G.4 Have any concerns or recommendations arising for the M&E process been acted on to
improve project performance?
G.5 Have all stakeholders been transparently engaged in the M&E process (Advisory
Boards, Project Implementation Review, etc) during project implementation?
G.6 How could M&E have been improved
H. STAKEHOLDER PARTICIPATION
H.1 Were appropriate stakeholders involved in the Project Development?
H.2 Were adequate provisions/arrangements made within the original Project
Document to allow for comprehensive and fair stakeholder involvement and input
to the project?
H.3 Has stakeholder input and involvement been adequate? If not, why not?
H.4 Were any principal stakeholders not included in this process and how would this
have affected the project's delivery and long-term success?
H.5 Have relevant stakeholders (e.g. National/regional experts and specialists, NGOs,
community groups, other Government Departments, etc) been involved directly in
executing specific activities?
H.6 How could stakeholder participation have been improved in the project?
I. CAPACITY BUILDING
I.1 Did the Project Document accurately identify the required capacity building and
institutional strengthening?
I.2 What capacity building and institutional strengthening has been achie ved?
I.3 What training has been achieved?
I.4 Has the project provided any equipment and has this been used effectively?
86
I.5 Has any of the capacity building and institutional strengthening focused on NGOs
or community groups? If yes, how effective has this been?
I.6 Will the capacity building and institutional strengthening that has been provided be
sustainable?
J. POLICY AND LEGISLATIVE REFORMS/IMPROVEMENTS
J.1 Were any policy reforms or legislative amendments identified as project
requirements? If not, should they have been and has their absence affected project
performance?
J.2 If policy reforms and legislative amendments were identified as project
requirements, how effective has the project been at delivering these reforms and
amendments?
K. REPLICABILITY
K.1 How replicable within the project region are the lessons and practices developed
through this project?
K.2 How replicable would they be in other areas throughout the world?
K.3 Does the project have a mechanism for replication and transfer (within project
system boundary, and beyond at global level)
L. SUSTAINABILITY
L.1 Are the activities undertaken so far and their outcomes and deliverables sustainable
in the long-term? Can the enterprises maintain the requirements (e.g. of ISO 14000)
within the ir current financial structures and into the future?
L.2 Are the overall objectives of the project likely to be sustainable beyond the project
lifetime and for the foreseeable future?
L.3 What could need to be done to improve the chances of sustainability?
M. OVERALL IMPACT OF PROJECT
(List any positive and negative impacts)
M.1 Natural/Environment. What improvements or harm has the project had on the
natural environment both within the Danube and beyond?
M.2 Political: Has the project affected political thinking and policy. Is it seen as a good
or a bad thing by local politicians and/or industry?
M.3 Economic: Has the project had a positive or negative economic effect either within
or outside of the project areas?
M.4 Social: Has the project created noticeable and measurable improvements, or notable
problems within associated or affected communities?
N. FEASIBLE IMPROVEMENTS TO PROJECT
N.1 Overall, what improvements could have been made to the project?
O. FURTHER REQUIREMENTS
O.1 Based on achievements and lessons learned from this project, are their any logical
follow-up activities or initiatives which could be considered which would build on
the investment made during this project's lifetime?
87
P. GENERAL QUESTIONS
P.1 Has there been a fair, transparent and equitable involvement and input from all 5
participating countries?
P.2 Has there been effective and transparent involvement of and communication with
the other Danubian countries?
P.3 Has the project had any identifiable (with specific reference) effe ct on general
awareness and understanding of pollution issues within communities along the
Danube and the general regional population?
P.4 Are their any mechanisms in place to monitor the achievements of the project and to
ensure that they improvements made are not lost after project closure (especially in
relation to pollution reduction and water quality)?
Q. DO YOU HAVEANY OTHER COMMENTS OR CONCERNS WHICH HAVE NOT BEEN
ADDRESSED IN PREVIOUS QUESTIONS?
88
LIST OF STAKEHOLDERS TO CONSULT
PROJECT
Project Staff
National Advisory Boards
Other Project Implementation/Execution bodies? (e.g. technical advisory groups)
N.B. The Project Document has no description of Implementation/Execution Arrangements as is a
requirement for GEF Implementing And Executing Agencies.
INTERNATIONAL
UNDP
UNIDO
ICPDR
NATIONAL/LOCAL
TEST Focal Points
Board of Advisors
Programme Coordinators for the DRPC
Enterprise Representatives
National CICs and PPCs
Any relevant NGOs (especially within Danube Environment Forum)
Any relevant community groups
Any Local Community representatives within or associated with enterprises and discharge areas
Representatives of :
Ministry of Environment and Water, Sofia, Bulgaria.
Ministry of Environmental Protection and Physical Planning, Zagreb, Croatia.
Ministry for Environment International Funding Department, Budapest, Hungary.
Ministry of Waters, Forests and Environmental, Bucharest, Romania.
Ministry of the Environment, Bratislava, Slovak Republic.
N.B. Co-Funders? Although the Budget (P.14) lists Other Sources of funding ($1.48 million) , it
does not state where the funding comes from except in the case of UNIDOA or by implication with
respect to National In-kind Co-funding? This makes it difficult to discuss project achievements
with this important body of stakeholders.
89
Figure: C.1 Test
Procedure
Enterprise Selected
No
Yes
Viability Assessment
No
Yes
Cleaner Production Assessment
No
Yes
Industrial Management Assessment
No
Yes
Environmentally Sound Technology
No
Assessment
Yes
Sustainable Enterprise Strategy
No
Yes
Environmental Management System
No
Yes
No
Investment Negotiation
90