First Stage - Define the Aim

Stage 1

The single most important element is the definition of the aim of the programme or project. If an evaluation’s function is to measure the programme’s success, it is necessary to know what the programme aims to do. It is East Oxford Action's experience that this is where a lot of problems arise, both in terms of power - with different groups vying to define the aim, and in terms of lack of clarity – with different groups having different interpretations of the aim and thus different views on whether the project has been a success.

The function of the evaluator in this process is to mediate between those with differing views on the aim of the project, and to enable them to resolve their differences and to come if possible to a consensus on the aim. At the end of the stage the group should have negotiated and agreed one aim. Where the evaluation has not started at the beginning of the programme the differences over the aim between those involved can often explain the problems the project is encountering and part of this stage of the workshop much time may need to be devoted to exploring this. Workshops undertaken at the middle and end of the programme repeat the question. This reveals whether the aim has changed or perhaps even needs to be. In cases where there has been disagreement over the aims in the first evaluation it is useful to see whether these differences have been resolved.

It should start at the beginning but...

The first point of note is that while ideally these sort of evaluations would start at the beginning of the programme, East Oxford Action has yet to experience one that has done so. The reality of funding programme rounds with the short notice between the funding being announced and the project or programme being submitted virtually prohibits this. Another inhibitor is the need to fit the programme to the funder's specification, to balance the participants' objectives and those of the funder – power dynamics in play before the project/programme even starts. However where participatory and empowerment evaluations are employed albeit after the start of the programme, it is possible to use them as the starting point for the next programme, hence the end-of-programme evaluation session is also the opening session of the next programme.

Introduction to Participatory and Empowerment Evaluation

“Participatory monitoring and evaluation is not just a matter of using participatory techniques within a conventional monitoring and evaluation setting. It is about radically rethinking who initiates and undertakes the process, and who learns or benefits from the findings.” Institute of Development Studies, 1998

Participatory and empowerment evaluation places all stakeholders at its heart, especially those who will live with the results of the programme. In such an approach the evaluation is not seen as a final judgement on whether the project or programme has been a success or not, but as a means to help people help themselves and improve their programmes. In this approach the evaluator is not an external judge but a critical friend, holding up a mirror to those involved in delivering the project, and facilitating the generation of solutions to problems and a more accurate self-image.

Participatory approaches to evaluation are used extensively by organisations working in the Third World, such as OXFAM and the United Nations Development Programme. The characteristics of participatory evaluation as opposed to more traditional evaluations are:

• the participation if a broad range of stakeholders rather than a limited number in a traditional evaluation

• the focus on participants: residents, project staff and stakeholders, rather than funders and programme managers

• the evaluator as facilitator and critical friend, rather than as expert

• a focus on learning through the process rather than on accountability/judgement

• the involvement of the participants in collecting the data, with the support of the evaluator, rather than the outside experts doing so.

There are many ways of delivering participatory evaluation and many tools for the job, especially rapid appraisal techniques, which will be familiar to the world of community development and regeneration.

Extract from paper by Zoe Brooks CEO East Oxford Action, 2007