BOBLME-2009-REG-5.0-IWS-04
The Bay of Bengal
Large Marine Ecosystem Project
- Monitoring and Evaluation overview
David LaRoche
Monitoring and Evaluation specialist

---

BOBLME Project
Monitoring and Evaluation
· Monitoring and Evaluation ­ DEFINITIONS
· Monitoring and Evaluation ­ PURPOSE
· Monitoring and Evaluation - PHILOSOPHY
· Monitoring and Evaluation - FAO/GEF Projects
· Monitoring and Evaluation - BOBLME M&E
PROCESS and PLAN.

---

Monitoring
Monitoring is the continuous or periodic
process by project personnel to collect and
analyze data to measure the performance
of a program, project, or activity:
· Project Inception Report;
· Quarterly Progress Reports;
· Project Implementation Reviews;
· Project Terminal Report; and
· Technical and Field Reports

---

Evaluation
Evaluations are systematic and
independent quantitative and qualitative
assessments of ongoing or completed
projects or programs, along with their
design, implementation, and results.
· Mid-Term Evaluation;
· Final Evaluation; and
· Periodic Evaluations.

---

BOBLME Monitoring and
Evaluation Purpose
The Purpose of BOBLME M&E should be to:
· Enable "course corrections" as the project
moves through implementation;
· Provide timely and actionable advice to
project personnel;
· Satisfy FAO, GEF, and donor requirements;
and
· Provide a useful base of information for the
conduct of the mid-term and final evaluations.

---

Monitoring and Evaluation
Philosophy
Evaluations are a disciplined,
rigorous, and collaborative process of
gathering and analyzing information
to identify and apply lessons learned
in a way that makes individuals more
effective, and projects and
programmes more successful.

---

Monitoring and Evaluation Philosophy
The approach is disciplined through formulation of
and adherence to a concrete plan for the evaluation.
It is rigorous through application of the methodology
that has been developed and agreed to, and it is
collaborative through involvement of key project
personnel, stakeholders and others at each step of
the evaluation process.

---

Monitoring and Evaluation
Application to BOBLME Project
BOBLME Monitoring and Evaluation will include,
depend upon, or make reference to, among other
things:
· A range of FAO and GEF mandated reporting
requirements;
· Basic project documents: Project preparation
documents, Project Document, logframe;
· Interviews, meeting minutes, workshop reports,
PSC discussions and decisions; and
· Annual National Workplans and Annual Regional
Workplans.

---

Monitoring and Evaluation - FAO/GEF
Reporting Requirements (1)
Project Inception Report (1st Annual RWP)
Quarterly Project Progress Reports (QPPRs)
Project Implementation Reviews (PIRs)
(Annual/GEF mandated)
Periodic Evaluations

---

Monitoring and Evaluation -
FAO/GEF Reporting Requirements
(2)
Technical and Field Reports
(Discretionary)
Mid-term Evaluation (Independent)
Project Terminal Report
Final Evaluation (Independent)

---

Three Major Pillars of BOBLME M&E
1. Logical Framework Analysis (Logframe)
2. Annual National Workplans (ANWPs)
3. Annual Regional Workplans (ARWPs)

---

Logical Framework Analysis (Logframe)
What is it? A tool for planning and managing
development projects through identification of:
Outcomes, Indicators of Success, Baseline
Conditions, Targets, Sources of Verification, Risks
and Assumptions
What are its more specific uses?
Mandatory for Project consideration; Helps Refine
Project Design (Content); Creates an Effective
Way to Track Progress (critical to M&E) and thus
make "Course Corrections" easier; and is an
invaluable tool for Evaluators (M&E Processes)

---

SOURCES OF
RISKS AND
OUTCOME
INDICATOR
BASELINE
UPDATED TARGETS
VERIFICATION
ASSUMPTIONS
Information
National data
Poor cooperation
Targets (Year 1):
Documents related to
Risk that countries will
management and
handling and
among the various
the tools and
not find it possible to
handling tools and
management plans
national level
9 country level, country
procedures developed
commit resources to this
procedures
developed, adopted
ministries responsible
developed D&I plans
by the project
activity given other
developed and
and functioning
for gathering and
2 D&I Working Group
Review of the country pressing economic and
adopted
assessing LME based
meetings (PI)
level plans
social needs
management plans
Copies of SC, PCU,
Some regional focus
2 Reports of D&I
and workgroup
for development of
Working Group Meetings
meetings, workshops
tools previously
(PI)
and reports related to
created by ACEP and
this Activity
WIO-LaB
Short letters of
Interviews with
Agreement describing
project personnel,
data exchange among
sister projects, and
countries (PI)
country and regional
organizations focused
Short letter of Agreement
on development of
among scientists,
national level
countries and the
information and data
ASCLME Project
handling tools and
describing data use and
procedures
publishing (PI)
Review of the D&I
Working Group
reports and minutes
Review of the letter
of agreement among
the scientists,
countries and
ASCLME Project

---

Logframe
Pre-project Logframes ­ Often created
years before implementation begins,
thus can be outdated before the project
is underway, but.....
Can and has been updated during project
(post) Implementation

---

Pre-project Logframe
BOBLME Project
Examples of Limitations when using Best
Available Information at time of preparation:
Post-tsunami assessment;
TDA refinement; and, in general....
Inevitable changes and shifts given the 5
years since logframe development.
Pre-project logframes are in many ways,
and necessarily, a "best guess" exercise.

---

Logframe Update:
Post Implementation Update Example
(BCLME)
Project Goal, Outcomes and Baseline
remained largely unchanged; but
Targets (deadlines), Sources of Verification,
and Risks and Assumptions were refined and
expanded.
Update benefits included: Improved activity
definition; improved budgeting; improved
logframe and M&E interface; improved co-
finance; more synergistic partnerships;
overall improved focus and efficiency.

---

Logframe: "Rolling" Approach (1)
As was true with the BCLME post
implementation Logframe update, a
BOBLME update would be one where
the.......
Project Global Development Objective,
Outcomes, and Baseline would remain
largely unchanged, however......
Targets, Sources of Verification and Risks
and Assumptions will be added and could
become increasingly refined as time
horizons become ever tighter.

---

Logframe:
Rolling Approach (2)
·
The Project would adopt a "rolling" approach
through yearly logframe updates, meaning it
would be initially, and continuously updated
consistent with:
·
TDA ­ Gap analysis results and further work;
·
Assessments of changes/events of past 5
years;
·
Yearly updated progress report in meeting
defined targets; all through...
·
Ongoing consultation with RCU, PSC approval,
and selected stakeholders.

---

PROJECT
STRATEGY
OBJECTIVELY VERIFIABLE INDICATORS AND CONDITIONAL FACTORS
GOAL/Global
To ensure the long-term sustainability of the living resources of the Agulhas and Somali LMEs
Development
(ASCLMEs) through an ecosystem based approach to management of the ASCLMEs.
Objective:
SOURCES OF
RISKS AND
INDICATOR
BASELINE
YEARLY TARGET
VERIFICATION
ASSUMPTIONS
Undertake an
environmental
baseline
assessment of the
Agulhas and
Somali Current
Large Marine
Ecosystems to fil
information gaps
needed to improve
management
decision-making;
and to ascertain
the role of external
forcing functions
(such as the
Mascarene
Plateau and the
Southern
Equatorial
Current). This
information wil be
used to develop a
TDA and SAP for
the Agulhas
Current LME, and
a TDA for the
southern portion of
the Somali Current
LME

---

PROJECT
STRATEGY
OBJECTIVELY VERIFIABLE INDICATORS AND CONDITIONAL FACTORS
GOAL/Objective
To ensure the long-term sustainability of the living resources of the Agulhas and Somali LMEs (ASCLMEs) through an
ecosystem based approach to management of the ASCLMEs.
SOURCES OF
RISKS AND
INDICATOR
BASELINE
YEARLY TARGET
VERIFICATION
ASSUMPTIONS
Objective of the
An effective regional
Project: Undertake
and national capacity
an environmental
established and
baseline assessment
sustainable that will
of the Agulhas and
lead to cooperative,
Somali Current
transboundary
Large Marine
management of the
Ecosystems to fill
LMEs
information gaps
needed to improve
Application of GEF
management
Process, Stress
decision-making;
Reduction, and
and to ascertain the
Ecosystem Status
role of external
Indicators.
forcing functions
(such as the
Mascarene Plateau
and the Southern
Equatorial Current).
This information
will be used to
develop a TDA and
SAP for the Agulhas
Current LME, and a
TDA for the
southern portion of
the Somali Current
LME

---

PROJECT
STRATEGY
OBJECTIVELY VERIFIABLE INDICATORS AND CONDITIONAL FACTORS
GOAL/Objective
To ensure the long-term sustainability of the living resources of the Agulhas and Somali LMEs (ASCLMEs) through an
ecosystem based approach to management of the ASCLMEs.
SOURCES OF
RISKS AND
INDICATOR
BASELINE
YEARLY TARGET
VERIFICATION
ASSUMPTIONS
Objective of the
An effective regional
The ASCLMEs are
Project: Undertake
and national capacity
perhaps the most
an environmental
established and
poorly understood
baseline assessment
sustainable that will
LMEs in the world.
of the Agulhas and
lead to cooperative,
Somali Current
transboundary
Transboundary issues
Large Marine
management of the
have yet to be
Ecosystems to fill
LMEs
identified by the
information gaps
participating countries,
needed to improve
Application of GEF
and countries have nit
management
Process, Stress
engaged in a joint
decision-making;
Reduction, and
SAP development
and to ascertain the
Ecosystem Status
exercise.
role of external
Indicators.
forcing functions
Formal structures and
(such as the
binding agreements
Mascarene Plateau
have yet to be
and the Southern
established and
Equatorial Current).
adopted. National
This information
realignment in policy,
will be used to
legislation and
develop a TDA and
management practices
SAP for the Agulhas
are essential in order
Current LME, and a
to embrace a truly
TDA for the
transboundary
southern portion of
ecosystem approach.
the Somali Current
LME

---

Logframe Update Flowchart
ANWPs
Gap
INCEPTION
First Updated
ARWP
Analysis/
MEETING
Logframe
TDA
Updates

---

Annual National Work Plans (1)
(ANWPs)
Prepared by each country and submitted to RCU
as tool for preparation of the ARWP. It provides:
A review of the past year's activities;
Plans for the coming year;
Discussion of technical activities;
A provisional financial report; and
Reports on communications/dissemination,
monitoring and IT.

---

Annual National Work Plans (2)
As the ANWPs are critical to forming the ARWPs,
and both are essential to the M&E process:
The Monitoring and Evaluation Specialist will assist
countries in preparation of a format for preparation
of the ANWPs as necessary, as well as assisting in
development of country monitoring and evaluation
plans.

---

Annual Regional Work Plans (1)
(ARWPs)
Prepared by the RCU and submitted to the PSC
for their endorsement within 45 days of
January 1: As with the ANWP they provide:
A review of the past year's activities;
Plans for the coming year;
Discussion of technical activities;
A provisional financial report; and
Reports on communications/dissemination,
monitoring and IT.

---

Annual Regional Work Plans (2)
(ARWPs)
The ARWPs are a central ingredient needed to
measure project progress. They will:
Overall, be a key resource that allows the PSC to
track progress and inform its decision-making;
Help form, and be informed by, the ANWPs;
Be a key resource for Project Progress Reports;
Periodic Evaluations; the Mid-Term Evaluation;
QPPRs; and the Final Evaluation.

---

Quarterly Project Progress Reports
(QPPRs)
The Regional Coordinator must prepare
Quarterly Progress Reports (QPPRs) which
contain, among other things:
· An account of actual implementation of
project activities compared to the ARWP;
· Identification of problems and constraints;
· Clear recommendations for corrective
actions;
· Lessons learned; and
· A detailed workplan for the next reporting
period.

---

Intensive Periodic Evaluations (1)
Two intensive, independent Periodic Evaluations
could be added to the M&E process, and they
would include:
Interviews with key personnel responsible for
implementation of activities related to
achievement of project outputs.
Interviews with policy level country
representatives in the participating countries
of the project.
Interviews with other selected stakeholders.

---

Intensive Periodic Evaluations (2)
A description of the overall progress observed
towards successful project implementation,
noting the extent to which project specific
milestones have been achieved and which have
not.
Where progress has been demonstrated, identify
best practices and describe lessons learned;
Where progress has lagged, describe the relevant
causes, both internal and external to project
implementation activities; and
Provide timely recommendations to the Project
for revisions that may be necessary to achieve
greater success.

---

Project Periodic Evaluations (3)
The first periodic evaluation would begin at the
time of the second Steering Committee meeting
in 2011, and would help inform the Project Mid-
Term Review.
The first periodic evaluation would utilize the first
ARWP, the initial ANWPs, project documents and
reports, and interviews with key project
personnel and selected stakeholders.
The second periodic evaluation would utilize
subsequent ARWPs and ANWPs and also project
documents and field-based interviews.

---

Annual Project Implementation Reviews
(PIRs/GEF Mandated)
The GEF, through PIRs for its IW projects, uses three
categories of Indicators of Project progress:
1. Process Indicators;
2. Stress Reduction Indicators; and
3. Environmental Status Indicators

---

Process Indicators
GEF Process Indicators are actual, on-the-
ground evidence of institutional and political
progress. Examples include the creation and
functioning of:
· The Project Steering Committee
· National Project Advisory Groups
· Interministerial Committees
· National and Regional level TDA/SAP
working groups

---

Stress Reduction Indicators
Stress Reduction indicators refer to on the ground or
on/in the water measures implemented by the
participating countries at national or regional level.
Examples include:
Point and non-point source pollution control measures.
Amount of underwater or wetland areas placed into
protection in the form of no-take zones or marine
protected areas.
Percentage of fishing capacity decreased and replaced
by alternative livelihoods.

---

Environmental Status Indicators
Environmental Status indicators are measures
of actual performance or success in restoring
and protecting the targeted water body.
Examples include:
· Improved recruitment classes of targeted
fish species, diversity, or keystone species.
· Demonstrable reduction of persistent organic
pollutants in the food chain.
· Quantified extent of coral reef restoration as
a result of enforced effluent standards.

---

SOURCES OF
RISKS AND
OUTCOME
INDICATOR
BASELINE
TARGET
VERIFICATION
ASSUMPTIONS
Information
National data
Poor to non-existent
Targets (Year 1):
Documents related to
Risk that countries will
management and
handling and
tools and procedures
the tools and
not find it possible to
handling tools and
management plans
at both national and
9 country level, country
procedures developed
commit resources to this
procedures
developed, adopted
regional levels
developed D&I plans
by the project
activity given other
developed and
and functioning
2 D&I Working Group
Review of the country pressing economic and
adopted
Poor cooperation
meetings (PI)
level plans
social needs
among the various
Copies of SC, PCU,
national level
2 Reports of D&I
and workgroup
ministries responsible
Working Group Meetings
meetings, workshops
for gathering and
(PI)
and reports related to
assessing LME based
this Activity
management plans
Short letters of
Interviews with
Some regional focus
Agreement describing
project personnel,
for development of
data exchange among
sister projects, and
tools previously
countries (PI)
country and regional
created by ACEP and
organizations focused
WIO-LaB
Short letter of Agreement
on development of
among scientists,
national level
countries and the
information and data
ASCLME Project
handling tools and
describing data use and
procedures
publishing (PI)
Review of the D&I
Working Group
reports and minutes
Review of the letter
of agreement among
the scientists,
countries and
ASCLME Project

---

PROJECT
STRATEGY
OBJECTIVELY VERIFIABLE INDICATORS AND CONDITIONAL FACTORS
GOAL/Obhective
To formulate an agreed upon Strategic Action Programme (SAP) whose implementation over time will lead to an
environmentally healthy Bay of Bengal Large Marine Ecosystem.
SOURCES OF
RISKS AND
INDICATOR
BASELINE
TARGET
VERIFICATION
ASSUMPTIONS
Objective of the
An effective regional
Transboundary issues
Key data generated,
Minutes and other
Risk that pressing
Project:
and national capacity
have yet to be fully
analyzed and
documentation of SC/PCU
domestic economic social
To support a series
established and
identified by the
information gaps filled
meetings, work groups,
issues will prevent senior
of strategic
sustainable that will
participating countries, (PI)
and other entities related
national political figures to
interventions that
lead to cooperative,
and countries have not
to project implementation.
grasp the long-term
will provide critical
transboundary
engaged in a joint
Artisanal fisheries
importance of the need to
inputs into the SAP
management of the
SAP development
assessments and socio-
Cooperative, collaborative sustainably manage the
whose
BOBLME through
exercise.
economic analyses,
and ongoing monitoring
living marine resources
implementation
adoption and
using existing
and assessment to advise
within the LME. A lack of
will lead to
implementation of a
Preliminary
information, filled and
policy and governance
political will.
enhanced food
SAP.
institutional analysis
used to inform the
decisions.
security and
not yet conducted and
TDA process (PI)
Risk that national level
reduced poverty
Application of GEF
formal structures and
Formal partnerships at
political leaders will not
for coastal
Process, Stress
binding agreements
Completion of TDA
regional level established
see the benefits and thus
communities.
Reduction, and
have yet to be
and successful
in support of training and
conclude importance of
Ecosystem Status
established and
negotiation of a
capacity building.
regional coordination of
Indicators.
adopted.
regional level SAP
efforts to sustainably
with specific targeted
Effective, documented
manage the LME.
National realignment
measures identified
stakeholder participation
in policy, legislation
and endorsed at
in ongoing SAP
Overall assumption that
and management
multiple Ministerial
implementation activities.
the resulting regional and
practices, essential in
levels (PI)
national structures can be
order to embrace a
made politically and
truly transboundary
Formal mechanisms in
economically sustainable.
ecosystem approach,
place, including M&E
not yet undertaken.
mechanisms, to ensure
sustainability of all
processes beyond the
life of the Project
(SRI)

---

Logframe Update Flowchart
ANWPs
Gap
INCEPTION
First Updated
ARWP
Analysis
MEETING
TDA
Logframe
Updates

---

M&E: Inception Meeting to Mid-Term Evaluation
ANWPs
QPIRs
Final TDA
INCEPTION
Updated
Updated
ARWP
GEF
ARWP
MIDTERM
MEETING
TDA
Logframe
PRs
PIRs
Evaluation
Post
Tsunami
ANWPs
Eval

---

M&E: Mid-Term to Final Evaluation
QPPRs
GEF
PiR
Final
Evaluation
Mid-Term
ARWPs
GEF
Final
TDA/SAP
Final
Evaluation
PRs
PiR
Process
SAP
PR
SAP
Implementation
ANWPs
QPiR

---

Bay of Bengal Large Marine Ecosystem Project
Key Monitoring and Evaluation Activities and Timelines
Year 1
Year 2
Year 3
Year 4
Year 5
Inception Workshop/WS Report



Monitoring and Evaluation Workplan


Initial M&E Country Consultations


Updated Logical Framework Analysis





Project Steering Committee Meetings




Quarterly Project Progress Reports



















Annual National Workplans








Annual Regional Workplans




Periodic Evaluations




Project Implementation Reviews








Mid-Term Evaluation



Project Terminal Report


Project Final Evaluation

---

Let the voyage begin

---