Reflecting on the innovative impact assessment project
RedR UK and the University of Sussex are designing and testing innovative methods to better understand participants’ learning and changes in behaviour following humanitarian capacity building interventions.
Building on recommendations from our earlier work, this project set out to identify ways in which RedR can design programmes for increased learning and behaviour change of participants, and for increased ability to capture and report on these changes.
Three methodologies were tested.
Of these three, two proved to be largely successful in the user feedback, where evidence of learning and behaviour change was found:
Video capture and view-back – Within this methodology, 3 prototypes were tested, using different technology ranging from high end cameras, to frugal technology, to participants’ own camera phones. From the 3 courses we learnt that view-back can be used for different purposes depending on the aim of the session or activity. These can be as conversation starters, which worked especially well for group activities that depend on body language, or for reflective practice, which worked better for individual training activities. In order to achieve deeper self-reflection, tailored feedback sheets with specific and targeted questions were developed, piloted and refined, to guide participants’ learning.
Reflective practices – Across all the reflective practices considered by this methodology (coaching, mentoring, learning essays, learning journals), evidence of learning as a result of the method was highlighted by learners and trainers. These methodologies also promoted behaviour change at both individual and organisational levels. User feedback, gathered from learners and trainers in all of the 4 reflective practices, confirmed a number of reasons why this methodology was so successful. These include: it is flexible, tailored and specific to the individual; the timeline for interaction is longer (than a face to face course), which allows more time for reflection, and to achieve and observe change and impact; it is empowering; participants tend to be self-selecting.
One methodology was less successful:
Ongoing engagement – Tailored scenarios and discussion questions were developed for two flagship RedR courses and shared with participants bi-weekly over a period of 3 months after the face to face learning event. Across the two pilot courses, no participants responded to the email scenarios. 3 months after the course, 25% and 30% of participants responded to the planned Survey Monkey follow-up, compared to an average response rate during the project period of 44%. When user feedback was gathered from participants, all those interviewed stated that the scenarios were relevant and valuable, but they did not actively engage due to other priorities, or because the completion of the course/issuing of the certificate did not require it.
We acknowledge that there is room for improvement in the management of the use of the methodologies, for example in terms of ensuring sufficient time for video view-back during a course, and better incorporation into course design. RedR learning events are comprised of detailed session plans and time is fully allocated to existing activities and content. It was therefore sometimes challenging to fully incorporate the prototype methodologies without compromising the existing course requirements. This needs to be carefully considered and sufficient time for adaptation and incorporation must be allocated for future use of the methodologies.
The key learning for RedR has been the value of reflective practices, and the importance of these being individualised and offered over a prolonged time period. This learning will inform the design of new programmes, and the review and updating of existing programmes in the coming months. For example, we are currently designing a new preparedness training programme which will combine face to face workshops with ongoing, remote support to individuals, with the aim of enabling them to effect change in their organisations.
RedR will continue to use reflective practices, and will work to incorporate more of this into existing programmes and new project development. The use of video capture and view-back, particularly using participants’ own camera phones, will be considered as an option in the design of all Training of Trainers programmes, and in the development of any new sessions with objectives around communication / body language.
In addition, we will be considering integrating the methodologies, for example by using video capture during face to face training, and then providing individualised feedback on this linked to coaching for a period following the face to face event.