Reflections about PM&E Based on Stories of Change

К оглавлению
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 
85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 
102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 
119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 
136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 
153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 
170 171 172 173 174 175 176 177 178 

The PM&E process of regularly reviewing progress on local initiatives, discussing

lessons learned, making new plans, and then reflecting together on accomplishments

and change can be a deeply profound experience for a community. These

PM&E events provide a practical opportunity for groups to become self-conscious

about their latent power. This was certainly the case in Ovsiste; one can easily

imagine their excitement as they rehearsed and reveled in their accomplishment.

Such discussions enable groups to build a larger story for themselves about who

they are as a group and what they can accomplish through their own concerted

efforts. These are all based on strong experiential messages that have the potential

to alter images dramatically. Ultimately, this is the key to long-term changes in

behavior. Therefore, this PM&E approach, with its emphasis on stories of change,

is transformative.

The system uses a highly inductive approach in which indicative changes become

the basis for drawing conclusions about results. This is valuable when objectives

of community development include ideas like increased participation,

self-confidence, local responsibility, capacity for problem-solving, and transparency.

Such benefits are extremely difficult to evaluate. Rather than being confined

to a narrow range of predetermined indicators, this approach is flexible and

adaptive to changing circumstances. Of course, it is in sharp contrast to conventional

monitoring approaches that are deductive in orientation, begin with a theoretical

idea about intended results identified by experts far from the locus of

activity, and then attempt to identify indicators of its occurrence (Davies, 1998).

It gives those closest to the activity being monitored, in this case community

residents, the opportunity to guide the process by making their choices and interpretations

at the beginning of the process rather than at the end. External officials

and senior program managers are put into the position of responding to explanations

generated from below. Consistent with the aims of community development,

the basic monitoring agenda is thereby established from the bottom rather than

the top. However, its participatory nature is not limited to the communities; all

stakeholders are effectively tied into a single unifying PM&E conversation.

The retelling of stories about selected changes has a modeling effect within

communities. Such changes tend to be stories about success, focusing on positive

results rather than negative shortcomings. Through feedback mechanisms, best

Facilitating Participatory Evaluation Through Stories of Change 439

practices are highlighted and become the primary subject of reflection and appraisal

by all stakeholders. The monitoring work thereby becomes a direct contributor

to institutional learning at every level of the project.

Possible Uses of Stories of Change in Other Settings

Although this PM&E system has been created and used primarily in international

development, it should be easily adaptable to any situation where subgroups are

working within a large setting, be they different geographical locations of a company

or different departments or sections of a public agency housed in a single

building.

Stories of change are also applicable for PM&E of facilitated processes themselves,

especially when there is an interest, or a demand, to provide some data on

the ultimate impact of such processes. The initial brainstorm about the changes

participants have experienced need not be general. In a systematic way, specific

topics might be stipulated. Instead of asking a group to come up with four selected

changes in an open-ended manner, groups can be asked to make a selection within

particular areas—for example, “What are the most important changes that have

occurred in the following areas? As a group, choose one change each in the areas

of team effectiveness, group initiative and responsibility, capacity for problem solving,

transparency in decision making, creativity, and self-confidence?”

By systematically collecting data in these areas, along with stated reasons about

why choices were made, a wealth of information is available for drawing conclusions.

A mechanism for involving stakeholders from other levels in a review of the

stories and providing feedback on their thoughts enables various stakeholders to

become involved in a common dialogue. Collective knowledge about the significance

of facilitated processes is thereby generated and is shared throughout the organization.

PM&E based on stories of change can provide a practical means for

enabling institutional learning on any initiative or program.

APPENDIX 24A: PM&E IN THE COMMUNITY EMPOWERMENT

PROGRAMME, ETHIOPIA

The Community Empowerment Programme (CEP) took place in five districts, or

woredas, in central Ethiopia. The only support that CEP offered was the facilitation

of community planning workshops (CPW) where members of local kires, or

villages, met to discuss issues and plan local projects based exclusively on local resources.

The tables are quantitative summaries of work in 309 villages. The story

that follows is one of 288 that was documented during one PM&E reporting

period:

In Jebukie kire in Legehida . . . prior to the CPW the work on clearing

springs was done only when clearing was very desperately needed; even

then only a minimum amount of work was done. Additionally, no

fences were ever built nor had any repair been done. After the CPW,

however, we tried to work harder. The work on the springs involved

both the construction of fences and continuous follow-up. An example

of follow-up by the community was when it was noticed that some

pieces of wood had been stolen from the fences around the spring we

had built and reported this to the kire leader. The kire leader then called

an afersata [judicial meeting] to investigate the crime and the criminal

was found. Then we were able to return back the pieces of wood

to the kire as a result of the afersata, and the spring fence was repaired.

Why selected by the woreda officials: From this we can see that the

people understand that they are responsible for their own development.

It indicates that they really feel a sense of ownership in what they

planned and accomplished [Bergdall and Powell, 1996, p. 68].

Facilitating Participatory Evaluation Through Stories of Change 441

Community Empowerment Programme Workshop Numbers

and Attendance Figures

Final Database Debra Legambo Saint Woreilu Totals

Figures, Sina and Kelala

15 November 1996

CPW Number of 76 81 81 71 309

workshops

Attendance 9,493 7,531 10,211 5,099 32,334

F/U-1 Number of 62 70 68 62 263

workshops

Attendance 6,003 3,971 7,438 3,581 20,993

Final Database Debra Legambo Saint Woreilu Totals

Figures, Sina and Kelala

15 November 1996

F/U-2 Number of 47 55 56 46 204

workshops

Attendance 4,226 2,730 5,843 2,810 15,609

F/U-3 Number of 42 44 47 30 163

workshops

Attendance 3,380 2,054 4,653 1,670 11,757

Totals Number of 227 250 252 209 938

workshops

Attendancea 23,102 16,286 28,145 13,160 80,693

Source: Bergdall and Powell 1996)

aWomen 24,543 (30 percent); youth 18,678 (23 percent); men 37,472 (46 percent).

COMMUNITY EMPOWERMENT PROGRAMME

QUANTITATIVE ACCOMPLISHMENTS, FINAL DATABASE

FIGURES THROUGH 15 NOVEMBER 1996

ACTIVITIES Debra Sina Legambo Saint Woreilu Totals

(76 kires) (81 kires) (81 kires) and Kelala

(71 kires)

Tree planting 315,150 1,020,300 419,222 605,500 2,360,172

(number)

Terracing 196 67.90 200 11 475

(kilometers)

Spring clearing/ 428 528 698 228 1,882

protection (number)

Spring development 43 10 9 9 71

(number)

Facilitating Participatory Evaluation Through Stories of Change 443

ACTIVITIES Debra Sina Legambo Saint Woreilu Totals

(76 kires) (81 kires) (81 kires) and Kelala

(71 kires)

Footpath construction 85 7 10 1 103

(kilometers)

Footpath maintenance/ 69 35 109 26 240

repair (kilometers)

Checkdam construction 68 6 15 1 90

(kilometers)

Unabled house 28 18 56 14 116

construction (number)

Unabled house repair 14 7 44 12 77

(number)

Agricultural assistance 1 12 54 59 126

for unabled (hectares)

Diversion canal 49 34 32 9 125

construction (kilometers)

Diversion canal 16 26 21 17 79

maintenance (kilometers)

New irrigation schemes 72 194 97 49 412

(hectares)

New income-generating 3 108 102 33 246

activities (number)

Raising local seedlings 253,500 801,110 941,002 884,000 2,879,612

(number)

Church/mosque 17 11 73 10 111

construction/repair

(number)

New pit latrines 0 6 0 33 39

(number)

New local savings 0 3 3 0 6

schemes (number)

Note: Kires refer to the community groupings in villages that served as the basic operational units around

which the Community Empowerment Programme was organized.Woredas is the word used in Ethiopia

for districts.

Source: Bergdall and Powell (1996).

APPENDIX 24B: AN EXAMPLE OF STORIES OF CHANGE

FROM THE PHILIPPINES

Following are the four stories of change that a community in the Philippines identified

during a Barangay Development Council meeting in October 1997 (Bergdall,

1997). Barangays are the lowest level of government administration in the Philippines,

like a precinct, and are similar to villages. The PM&E exercise was within the

Governance and Local Democracy Project (GOLD) funded by USAID. One of the primary

aims of GOLD was to conduct communitywide planning workshops so that local

communities could collectively prioritize and plan local projects. Instead of a small

grants program, GOLD enabled communities to budget small capitalization funds,

which were redistributed to local communities from the central government’s tax revenues.

The following stories of change are from the Nagbitin Barangay of the Villa

Verde municipality in the Nueva Vizcaya province.

The community now has a “structured guide” to follow for its planning. In the past,

planning was only done by a few leaders and it was often haphazardly based on

personal favoritism.Most people in the community did not understand how projects

were planned and were not consulted about their views. Now, since the “planning

and budgeting workshop,” people know about the projects, understand the

priorities, and are aware about progress in implementation—or reasons for delay.

Consultation has been widespread and everyone now stands behind the community

plans. The extraordinary efforts to successfully raise money from community

members to complete the multi-purpose slab (that is, a paved area in the center of

the community for basketball, which is the national sport in the Philippines, and

other community activities) in time for the November fiesta is an example of what

has happened because of the new approach to planning: people understood why

there were limited community funds and acted to solve the problem.

Reason selected by the community: People have come to realize that everyone can

and should be involved in community planning.

Community leaders are serving the community better. Before, community leaders

tended to have a narrow political view of their role and often made decisions based

more on personal connections than on community-wide development needs. This

also often resulted in the chairman being the primary person in charge: he was the

one who made major decisions and assigned people to do particular work. If things

went wrong, the chairman was always blamed. Now, development priorities

planned by the community are the focus of work and many people are assuming

responsibility for agreed upon projects—everyone shares in their success or failure.

Outside visitors can see this change by reviewing the budget plan and talking

to community members about their knowledge of the plans.

Reason selected by the community: Leaders have a better understanding about

their proper functions in participatory development planning and are making sacrifices

to serve.

Greater transparency in managing development funds has minimized perceptions of

corruption. Before, very few people knew details about development money used

in the community. They didn’t know how much money was available or how it

was used. Now, because of the planning, all projects are known as well as the

amounts and sources committed for each. Expenditures are reported to the community

in ways they can easily understand and appreciate. Evidence of this change

can be seen by viewing the written reports and financial records which are now kept

in an orderly way at the community office (written records were not kept before).

Reason selected by the community: People can see practical results from the use

of development funds.

Improved access to health services. Before, people could not depend upon local health

services because they didn’t know where or when to go since no regular schedule

was maintained.Mothers would have to waste a lot of time and money going to the

clinic in town. Construction of the community health center was made a high priority

during the “planning and budgeting workshop.”Work has since completed

and the center was opened in the past year. Now a regular schedule is maintained

by health workers: community members know when and where they can go for immunizations

and other assistance. Still there are problems with shortages of medicines,

but people no longer have to waste time and money to travel long distances

for basic health care. This change can be seen by visiting the health center.

Reason selected by the community: Maintaining good health for the family is one

of the most important responsibilities a parent has which means the health center

has been a high priority for the community.

APPENDIX 24C: FACILITATING GROUPS AND DOCUMENTING

STORIES OF CHANGE

The following procedures provide guidelines for facilitating PM&E Review Meetings

with communities participating in the Topola Rural Development Program (Opto 2002).

Facilitating Participatory Evaluation Through Stories of Change 445

At three intervals—project commencement, project completion, and six months

after completion—facilitators will meet with local project management groups to

review progress of local initiatives planned by the community and to spell out next

steps. The second and third of these meetings will also provide an opportunity for

community residents to reflect on changes that have occurred since becoming involved

with the Topola Rural Development Program. Four basic steps are involved

in this process: 1) a group brainstorm on changes, 2) selecting the four most important

changes, 3) deeper “probing” to more fully understand the selected

changes, and 4) documenting four selected changes for monitoring and evaluation

(M&E) efforts of the project with other stakeholders.

Step 1: A Brainstorm of Changes by the “Project Management Group”

Members of the group are asked to brainstorm “significant changes” which have

occurred since the commencement of the project (or since the last time they did

a group reflection on “significant change”). Phrasing the question so that it can be

easily understood in Serbian language is crucial. By asking this question, we are attempting

to enable participants to think about things that have happened which

they think are important. The question needs to be phrased in order to broaden

the group’s thinking. Variations in ways to phrase the question might include the

following:

• What changes have occurred?

• What have been key accomplishments?

• How have things improved because of the project?

• What difference has the project made in the quality of life for local residents?

In generating the brainstorm, all of these questions can be asked. List these

“changes” on a piece of flip chart paper displayed at the front of the room (i.e., a

“template”).

Step 2: Selecting the Four Most Important Changes

After a list has been created with several ideas, the facilitator then asks the group

to select four.However, instead of making general selections, the facilitator provides

categories for making choices.What has been the most important change in regards

to:

• Decision-making in the community?

• Practical benefits resulting from completion of the particular project?

• Trust and co-operation among people benefiting from the project?

• Any other change (an undesignated category)?

The facilitator lists the four categories on a piece of flip chart paper and explains

them all before the group makes its selections. After making this introduction, the

group is asked to make it selections (one per category).

Step 3: Deeper “Probing” to More Fully Understand

the Selected Changes

After making their four selections, the facilitators need to ask some additional questions

to more fully understand the nature of the changes selected and to assist in

the documentation. The following questions need to be asked for each of the selected

changes:

• What was the situation like before this change (or accomplishment)?

• How is the situation different now?

• What is an example or illustration about this change which you would show (or

tell) a visitor as evidence of this change?

• Why is this change important? What is the reason for your selection?

A simple template can be created to enable this discussion. It might look

like the table on the next page (with the four selected changes placed in the four

left-hand boxes).

Step 4: Documenting the Four Selected Changes

At the end of the workshop, the facilitator responsible for documenting the proceedings

needs to write paragraphs for each of these four changes. A title phrase

or sentence is written to broadly describe the change. A sentence or two then

Facilitating Participatory Evaluation Through Stories of Change 447

Type of Change Before Now Example Reason

Selected

Selected

change about

decisionmaking

Selected

change about

the benefit of

local projects

Selected

change about

trust and

co-operation

Selection

about another

important

change

describes the situation “before,” a sentence or two describes the situation “now,”

and a sentence or two describes the “example” of the change. Finally, the reason

the community selected this change is added as a separate explanation after the

paragraph on “before, now, and an example” of the change.

In writing the paragraph, the facilitator tries to remain as faithful to the actual

words used by the community members while also being descriptive enough that

other readers outside of the project can understand as much as possible about the

perceived importance of the selected change. Once a draft has been prepared, these

are reviewed by community residents; modifications are made as needed to accurately

reflect the community’s views.