Professional Documents
Culture Documents
August 2008
The author appreciate input and review comments from Peter Ngatia
Country project Coordinator, Job Ong’ombe (Faith Based Liaison Officer
ADRA ABY- Kenya), Barrack Bosire (Behaviour Change Specialist officer
ADRA ABY- Kenya), Rosemary Nyamagere Biseko (Behaviour Change
Specialist officer ADRA ABY- Tanzania), Dickson Humphey (Behaviour
Change Specialist officer ADRA ABY- Tanzania), Faraja Mafuru (Behaviour
Change Specialist officer ADRA ABY- Tanzania) and Grace Mahumbuga
(Behaviour Change Specialist officer ADRA ABY- Tanzania)
To you all I say thank you and May God bless you abundantly as you work
in his vine yard.
The objectives of the Abstinence and Behavior Change for the youth Project
Monitoring and Evaluation Plan (henceforth, the ABY M&E Plan) are to
provide:
The effect level changes will be captured through several indicators. The
application of this M&E plan will enable the timely generation, processing
and analysis of both qualitative and quantitative variable information for
both magnitude and direction of change of effect level indicators.
This plan will serve as a guide for both ABY implementing partners and the
project staff and as a capacity building tool for the partners.
NOTE
Since the ABY was approved in September 2004, some changes have been
made in the indicators and M&E activities. This was done with full
consultation between ADRA ABY and USAID. As the ABY project continues
to implement its activities, new lessons will be learned and changes in the
operating environment may necessitate future adjustments. ADRA ABY and
USAID will together continue to review and exchange information when
such changes occur and make mutually agreed adjustments, if necessary.
To reduce HIV and AIDS infections in the countries of Kenya and Tanzania
through a variety of HIV and AIDS prevention strategies, which will reach
over one million youth, aged 10-24, with Abstinence and Be Faithful (AB)
messages by September 2009.
To achieve the Project goal, the project focuses on the following three
Strategic Objectives (SO):
ABY project has three main components and as such each component tries
to contribute in different ways to Strategic Objectives mentioned above.
Within each Strategic Objective are multiple intermediate results which
contribute to the strategic objectives. The three main components are:
1.1.1 Training
The ABY project will achieve its Strategic Objectives through a series of
Intermediate Results (IRs). Strategic Objective #1 will lead to expansion of
skills-based HIV training for both in and out of school youth. This will be
accomplished directly through trainings and outreaches aimed at providing
AB information to the target group. These will in turn lead to reduction in
numbers of teen pregnancies among the target group as well as increased
knowledge in HIV and AIDS and directly contributing to both PEPFAR
USAID’s indicator 2.1A and 2.2.
The ABY Strategic Objective #3. The training, Community outreaches and
Information Education and Communication components contribute directly
to these strategic objective. These aims at creating an enabling
environment for parents to share with their children by giving them
communication skills and equipping them with information on HIV and AIDS,
hence leading to increased community discussions on HIV and AIDS right
from the family level which will translate to general community awareness
on preventive health practices. This contributes directly to PEPFAR USAID’s
indicator 2.1, 2.1A and 2.2
The ABY strategic objectives are not isolated from one another. Once the
activities to reach these objectives are completed and the objectives met,
the overall result will be reduced HIV and AIDS infections in the countries of
Kenya and Tanzania
SO2: Broad Community Discourse on Healthy Norms and Risky Behaviors Stimulated
Percent of young people aged 15–24 who both correctly identify ways of
preventing the sexual transmission of HIV and who reject major
misconceptions about HIV transmission
The proportion of youth between ages 10-24 years whom their Sero-status is
negative after HIV and AIDS testing as per the countries indicator survey.
The proportion of youth between ages 10-24 years who can correctly identify
ways of preventing themselves from contracting HIV and AIDS
This is the recommended direct and proxy outcome measurement for the
overall goal of the project. It is an accepted primary indicator for long-term
impact of HIV and AIDS reduction. The changes in this indicator are
cumulative, so the age interval of the youth needs to be limited (best if
measured at the end of the project through the national AIDS indicator
survey). Only needs to be measured at the baseline and final evaluations.
Best if a comparison group can be measured at baseline and final
evaluation stages as well.
Estimated Sample Size: The recommended sample size for this indicator
(using simple random sampling) is 225-250 youth per strata.
Definition:
Double counting is the instance where an individual is erroneously counted
and reported as having been reached with an (“A”) or Be faithful (“B”)
message or training for two or more times while in reality the individual
received the message or training once.
The ADRA ABY project has categorized the problem of double counting into
two major essential types and defines it as follows;
This a situation where one partner at one site provides the same messages
at multiple times to the same individual within one reporting period and
counts the individual as having received the messages multiple times within
the same reporting period.
ADRA Response
ADRA ABY project through its definition of reach has ensured that an
individual is counted as reached once a year if he/ she receive at least two
types of either A or AB messages as prescribed in the USAID ABC guideline.
This is ensured through giving each participant a unique identifier, which
the project uses to identify those who have been reached to avoid the
issues of double counting.
ADRA Response
ADRA through the ministries of Education, Culture and social services and
youth and sports at the districts level, did a mapping exercise within its
catchment’s areas to get to know USG partners operating within the
districts and the type of services they were offering. This was solely done to
enable the project avoid catchments areas where other USG partners where
operating so as to avoid double counting
Objectives
ABY management studies will have multiple objectives due to the diversity
of interventions strategies being carried out. In general, all management
survey studies will involve key stakeholders in defining study objectives,
carrying out surveys, analyzing information, and reflecting on results.
Typical ABY management survey objectives will include the following:
Methods
Sample Size
n = sample size
z = standard score corresponding to a given
confidence level (z = 1.96 for the 95%
confidence level)
p = expected proportion with the characteristic
(1-p) = expected proportion without the characteristic
Sample size is easy to calculate for a simple random sample (using the
formula above), but it gets more complex when the study design has
multiple stages (as most do, certainly in ABY type of management and
evaluation studies). You can reasonably estimate the required
decrease or increase in sample size by using the following table. If you
do this while planning your sampling strategy it will help you to make
decisions
n = 255 x 1.10
n = 281
Sampling error
n = sample size
SE = 4.7
The survey had initially calculated a sample size and error based on
the "worst case scenario". Now that the results are known, the error
estimation for current KPC is of ± 4.7. This means that the ABY can
be 95% sure that KPC lies between 12.3 and 21.7 percent of
respondents
Analysis
The MSSs will be analyzed in a variety of ways. First, total score for
each institution will be computed. These score will then be averaged
for each type of institution. An overall average representing the
number of institutions scoring a total of 75% or higher of the maximum
number of points possible will then be computed. This percentage
represents an estimate of the total number (when adjusted to the
population of institutions represented by the sample) of institutions
that satisfy the criteria for capacity-building.
Descriptive data for each of the questions on the score sheets will also
be generated, revealing which areas of capacity building are furthest
along in terms of average score. This will allow staff to adjust their
strategy to focus more on capacity-building topics that are scoring low.
The case study should be conduct by the project staff (from both the
field and headquarters). The comparative advantage of inviting an
external expertise should be considered. The objective of a case study
should always be clearly defined. The following should be included;
type of intervention being studied, study justification, a detailed
description of the study, methodology including indicators to be used,
detailed documentation of evidence, strengths, weakness,
opportunities, cost-benefit analysis, potential indicators identified,
alternative perspectives, recommendations and detailed references.
See the tool attached in annex 6 & 7.
Objectives
ADRA ABY project will undertake a final evaluation during the last
quarter of fiscal year 5. The purpose of the evaluation is to measure
the impact of ABY and also identify challenges and opportunities
resulting from the project. The result of the evaluation will serve as a
baseline for the preparation of a new ABY in FY 2010. The evaluation
will utilize available project data from evaluation activities to assess
overall program performance and provide ADRA ABY project with an
opportunity to examine its current program strategies, program quality
and impact.
The sampling strategy for the final surveys will utilize the same
methodology as the baseline surveys for each of the three
components. When possible a non-participant group will be used so
that post-hoc comparisons can be made and some level of attribution
(or contribution) can be made toward the findings. Sample sizes
should be adequate to allow 6-7% error in the findings. The final
evaluation will also utilize qualitative techniques – key informant
interviews, group discussions, mini case studies, etc. – to enrichen the
findings.
Project staff will carry out routine monitoring activities with technical
assistance from the M&E-Unit. The stakeholders (beneficiaries,
government counterparts, and partner NGOs) will be involved at all
levels of the progress monitoring and evaluation activities.
The main tools for qualitative assessment that will be used are focus
group discussions (FGD), perception agreement scale, case studies
(including quantitative methods), ranking, participatory monitoring,
and in-depth interviews. The aim of qualitative assessment is to:
Descriptive reports will be written for each survey after its completion.
Qualitative findings will be incorporated where necessary to add depth
and clarity to quantitative information. Separate qualitative reports will
also be produced. Table 3 illustrates the monitoring and evaluation
reports that will be prepared during the life of the ABY.
Training
Feedback
Monthly/Quarterly/biannually/annually reporting
Quality of data Supervision
Decision
Monitoring
Monthly/Quarterly/biannually/annually reporting
Feedback Supervision
Decision
Monitoring
Feedback Quarterly/
biannually/annually
Technical Backstopping
Finance Backstopping
JPMC
Feedback
Supervision Biannually/annually
USAID
The survey reports will be posted on the ABY web site for access by a
wider audience and as a documentation of lessons learned and best
practices that may be adopted in other areas or regions. The program
management in collaboration with the projects and M&E Unit will be
responsible for ensuring information dissemination.
Organizati
ons
1. Aims/objectives 1. Share planning and mobilization of
2. Targets members
3. Resource allocation 2. Integrate project activities into group’s
Partner
4. Partner contribution activities
CBOs
5. Reporting formats 3. Realistic partner planning and budgeting
Partner
6. Participatory monitoring and 4. Achieve meaningful partner contribution
FBOs
evaluation of project 5. Achieve timely and effective reporting
activities 6. Achieve objective selection of TOTs based
7. Selection criteria of TOTs on merit
1. Aims/objectives 1. For shared planning and collaboration
2. Targets and Activities efforts
Partner 3. Resources 2. To learn from project activities and
NGOs 4. Project location /area of experiences
coverage 3. Avoid duplication of efforts and resources
5. Project partners
Governme
nt
1. Annual plans 1. Planning, co-ordination
Ministry of 2. Status reports 2. Allocation of time, personnel, resources
Health 3. Evaluation reports 3. What to expect from project; what is
expected by project
1. Peer Education Training 1.
reports
Ministry of
2. Distribution of IECs reports
Education
3. Joint-Selection of schools
4.
Constituenc 1.Quarterly reports 1.Planning and coordination
y AIDS 2.Semi-annual reports
Control 3.Annual reports
Committee
Donor
1. Coverage, financial 1. Budget
information 2. Accountability
Donor
2. Status reports 3. Planning
3. Evaluation reports
Also see the relationship of the strategic objective, intermediate results and the project overall goal in a
diagrammatic form
DEVIATION
DEVIATION
FY05 TARGETS
FY06 TARGETS
FY09 TARGETS
LOA TARGETS
INDICATOR
FY07 TARGETS
FY08 TARGETS
LOA ACHIVEMENTS
DEVIATIONS
DEVIATION
DEVIATION
DEVIATION
ACHIVEMENTSFY08
ACHIVEMENTSFY07
ACHIVEMENTSFY09
FY05 ACHIVEMENT
FY06 ACHIVEMENT
Outcome 1 50000 20000 30000 40000 50000 500,0
# of youth 0 0 0 0 00
who 10%
received 40% 60% 80% 100% 100%
ABY and
HIV/AIDS
messages
Outcome 2 96 96 96 96 96 96
# of TOT
Master 100% 100% 100% 100% 100% 100%
Guides
trained in
use of ABY,
life skills
and
HIV/AIDS
curricula.
Outcome 3 250
# of
Pathfinder 100%
leaders
trained by
TOTs
Outcome 4 5,000 40,00 90,00 150,0 210,0 210,0
# of 0 0 00 00 00
pathfinders 2.4%
who 19% 43% 71% 100% 100%
Activities What has been done Reports from the project Identify SWOT
What has not been done but was Records of activity Prioritize actions
planned undertaken Identify training and
What problems were encountered Workshop reports other needs
How were the problems resolved Minutes of meetings
Other relevant information
What is the current status (date)
Progress toward Progress towards achieving goals Survey reports Lessons learned
objectives Are intended beneficiaries being Observations Indicators validation
reached? Workshop reports Modify strategies
Are intended beneficiaries Modify/Add
benefiting? interventions
What is the current status (date)
Other relevant information
1
Community Media
Annex 54
1
Recommendations:
Annex 55
Annex 5: ADRA staff reporting format
ADRA KENYA & ADRA TANZANIA ABY
MONTHLY REPORT FOR STAFF
Reporting Period: From ……../…../…….. To ………/…../…….. Date: ……../…../……..
Staff Name: ______________________________
Date Partn Distri Target Activity Target Achieved Targets Output Comme
dd/m er ct audie Planned Indicator nts
m nce
Plann Underta S I 10-25+ Male Female Progr Natio
ed ken O R am nal
M F T 1 1 25 T 1 1 25 T T
otal 0- 6- + otal 0- 6- + otal otal
1 2 1 2 M&
5 4 5 4 F
Training Activity
- - -
Total
- - - - - - - - - - - -
Community Outreach out of school youth
- - -
- - -
Total
- - - - - - - - - - - -
Community Outreach-In-School-Youth
- - -
- - -
Total
- - - - - - - - - - - -
Community Media Activity:
- - -
Annex 56
- - -
Total
- - - - - - - - - - - -
Annex 57
Annex 6: Intervention Case Study Outline
Type of intervention Indicate what the intervention is (Training,
being studied community outreach etc.)
Study justification Describe what stimulated carrying out the study and
the hypothesis to be tested.
A detailed description of Here describe study location (maps etc.), the current
the study observations, past experiences in similar situations
elsewhere, sources of secondary information
Alternative perspectives Observations that are not covered in the above but
are or could be useful for the intervention.
Annex 58
Include what future direction or activities would need
to be taken based on the study findings.
Adjustments/modifications should also be included
here.
Annex 59
Annex 7 Training Reports
Front Page
1. title and Partner
2. Date
3. Venue
4. Picture (optional)
5. Names of facilitators
Annex
• Time table
• List of participants (include contacts)
• Pre – Post Test (reports)
• Training Evaluation
• Work plan template
• Training Needs Assessment
Annex 60
Annex 8 Trip Reports
When to write a trip report- is a journey which has an objective and has a
budget attached to it; incase a group of staff attend one occasion the
person taking lead or who took the imprest should do the trip report
• Normal daily activities to be reported monthly
• Other activities with a budget e.g. meetings should have trip reports
NB: The entire trip report to be submitted to the Accounts office; Accounts
office to get original receipts
Annex 61
Dates
:
Visit
to:
Visit
by:
Key objectives:
1.
2.
3.
Principal contacts
Name Position Contact (e-mail,
telephone )
Annex 62
Conclusion :
Signature:
Annex 63
Annex 9 Peer Education tracking register
Organization Name: ………………….……………………. Period (MM/YY):....………………. Participants: New
Old
Program Type: ___________________________ School/ Camp Name ____________________Class/Grade: __________
Ward/Shehia ___________________
Village/Street: ____________________________ Facilitators Name(S):
_______________________________________________________________________
PRIMARY PREVENTON MESSAGE: Abstinence Only (A) Abstinence & Faithfulness (A&B)
AGE Session
% Attendance
Total Sessions
SEX 1 2 3 4 5 6 7 8
( Tick ) #
N A M E
NO.
10-15
16-24
> 25
M, F TOPIC
CODE
Annex 64
TOTAL
TOPIC CODES: 01= Values Self Confidence 02 =My Future 03= Human Development 04 = Teen Pregnancy & Sexual Abuse 05 = STIs,
HIV & AIDS Facts and Myths 06= Sexual Relationship 07= Making active decision 08= My Personal Abstinence
i 09 = Personal Abstinence Strategy Part II 10 =Being Faithful/Putting HIV into Perspective Stigma 11= Putting it together 12 = Other
Specify ______________________________
Annex 65
Please complete this section at the end of your
sessions
Type of Materials Title( if Quantity
applicabl
e)
Distribut Request Variance
ed ed
T - shirts
Fliers/Brochures/Leaflet
s
Booklets ( by type)
Magazines
Posters
Other (specify)
Annex 66
4. Do you have any specific recommendation for future
sessions like this?
Signature: _____________________
Date: ____________________
Why a register?
3) Topic code: Record the title of topic discussed during the session
under topic code row of the register below session No. The topic codes
and topic titles are listed at the bottom of the register.
Annex 67
If more than one topic were discussed in one session – include their
topic codes below session number
5) After the last outreach session tally the Total attendance for each
participant (count the number of sessions the participant attended)
and record the attendance rate.
Client Information
• Name: Record the name of each participant.
• Sex: Record the gender [male (M) or female (F)] of the
participant.
• Age: Record the age of the participant
Annex 68