New Regeneration Blog

I have just started blogging about community regeneration generally on http://transformativeplanning.blogspot.com/

commenting on current issues and reflecting on over twenty years' experience. I hope you enjoy it.

Happy 2009.

Zoe

EOA - evaluation services offered

EOA has a team of trained evaluators, led by myself, including specialists in housing, ethnic minority involvement and monitoring, managements systems, mediation, regeneration, social enterprise, enterprise development, social impact assessment, charitable law and culture. For most evaluations two evaluators, chosen to best suit the client’s needs, work together.

Evaluations start at £5,000 plus VAT and expenses. This includes preparation, a stakeholder workshop, some follow-up interviews and a written report. Longitudinal evaluations with a series of workshops and interviews spread over the lifetime of a programme are also available.

If you are interested, please contact me on zoe at eastoxford.com (replacing the "at" with an @ and deleting the spaces) or ring 01865 203367

Stage 5 - Developing an action plan

This stage of an empwerment evaluation often will happen as a separate workshop at a later date or alternatively as a second half to the rest of the workshop, depending on the needs and make-up of the project. Using the actions and scoring developed so far as the basis the fifth stage aims to identify actions to address the issues coming out of the workshop and what actions are needed to a) to evaluate performance in future and b) plan for future actions. By using the scoring from the previous stages a continuity is created into the future. For each of the criteria identified further goals are identified, actions and strategies to deliver the goals and evaluation evidence to verify success. These then form the basis of either work within the project or programme or for future programmes of action. In the case of the latter this approach is particularly important in securing sustainability for programmes, creating a basis and argument – in terms of project and partnership development – for future fundraising bids or mainstreaming.

Stage 4 - In-depth discussion

The evaluator then facilitates a discussion based on the scoring table. The function of the discussion is to get beneath the numbers to people's issues and views. The discussion is an open one and is based on a process which has been transparent and logical from the beginning. If the voting process was designed to ensure the people scored without regard to the views of others, this stage makes those involved listen to others and to value their opinions. It is East Oxford Action's experience that the democratic structure of the workshop means that there is less chance of programme staff either feeling threatened by the process or feeling the need to play power games. This is aided by the breaking down of the scoring into component parts for different criteria – as staff and others can see that scoring badly in one criterion does not mean a condemnation in others. Where tensions do arise these need to mediated by the evaluator, but they should not be ignored. The evaluator will often find themselves assisted in this by the other group members and where this happens the evaluator should encourage it – it is a sign that the group is beginning to take responsibility for the evaluation process.

Stage 3 - Scoring performance

Having prioritised the criteria/actions the participants individually are asked to score the project's current performance against the criteria. Scoring is out of ten, with ten being perfect. They do this on their own on a piece of paper and then either they read out their votes and the evaluator marks them on to a scoring grid (see fig1) or they write them on the grid themselves. This approach helps stop people from being influenced by what the others are scoring. In this and in the writing down of criteria (stage 2) the evaluator encourages the participants not to think too hard, but to write down what comes easily to mind. Whilst the workshop members are having a tea break the evaluator totals the scores both by criteria and participant.

This element of the workshop can be seen as producing a quantifiable scoring of performance that over a longitudinal evaluation can be used to measure progress. However caution should be exercised in doing this, as it should with all similar surveys and as is made clear in Stage 4 of the evaluation. It is the experience of East Oxford Action that programmes which include capacity and social capital building often do not score as might be expected – i.e. approval ratings do not rise as problems are tackled. The reason for this is that in disadvantaged communities there are often low expectations. If through the programme one raises expectations then the scoring of performance may be affected adversely. For example in East Oxford there was at the beginning of the regeneration programme an important piece of open space which was used as a dumping ground for rubbish, and as an area for drug abuse. The regeneration programme transformed the open space into a pleasant park. However, with their expectations raised, local people would still complain about the occasional dropping of litter.

Stage 2 - Prioritising criteria for success

In empowerment evaluation this is done through a voting system; equal weight is given to all participants. Each participant is given a set number of sticky dots (number dependent on the number of criteria to vote on, but usually 5 – 10) and each participant is asked to stick their dots next to their priority criteria – they can distribute the dots across the criteria or if they so feel put all the dots against just one. The participants come up to the flip chart and vote together, this again allows participants to think about the views of others on the subject. The evaluator then counts the dots and orders the criteria by numbers of vote. There is often a discussion at this point.

It is interesting in a longitudinal evaluation to see the variation between the results on stage 2 . This can show that concerns that featured highly at the beginning of the project have now been addressed and are no longer a consideration. For example in a training project evaluation that East Oxford Action undertook, for the first evaluation “recruitment of people onto the training course” ranked very highly, this had totally disappeared by the end evaluation workshop. Through the evaluation process however, the importance of recruitment at the beginning had been captured and so could be learnt from for future project planning.

Stage 2 - Criteria for success

In this stage the workshop focuses on the criteria for success in the evaluation. This is couched in terms of a question: “What needs to happen in order to achieve the agreed aim?” It is important to use accessible language and questions, hence “what needs to happen” instead of “what are the criteria of success”. It is important also for the evaluator to ensure that the language in the discussions is inclusive. This usually means asking for explanations and even getting the group to agree the banning of acronyms. Each group member is given five post-it notes and the group is asked individually to choose five things that need to happen and write one on each of the post-it notes. The participants then place their post-its on a flipchart. They are encouraged to bunch together notes which have the same criteria. The process allows the participants to see what others have written and to think. By writing the notes individually everyone's views are given equal weight. The evaluator then talks through the notes with the participants, seeking clarification as appropriate and the evaluator facilitates the group to agree the top things (Fetterman suggests ten to twenty) which need to happen. These are written up on a flipchart.