First Pilot of Video Capture Completed Successfully

This is the third blog post for the Innovative Impact Assessment project, describing the first pilot undertaken in late November to test our prototype of using video recording of in-class exercises to better assess training impact. For this pilot, the University of Sussex researchers travelled to RedR’s offices in London to conduct human-centred design research during a weeklong Training of Trainers (ToT) teaching course.

Based on our preparatory work, discussed in the previous blog, and extensive consultations with the course trainer, we put together a prototype of a technology kit with different variations of phones/cameras, lighting and microphones. There were six participants from a wide variety of geographical and sector backgrounds who took the course, with very little training experience between them. This small number and diversity was ideal for our first pilot, as it allowed for close observation of class room dynamics and participants’ and trainers’ reactions to the technology, as well as informal conversations with everybody.

On Monday (the first day of training), after we introduced ourselves and the context of the project to the group, we obtained permission to record everybody. This voluntary opt-out is important to maintain the training room as a safe space where participants feel comfortable. We then recorded each participant delivering a 10-minute presentation they had pre-prepared on a topic that they knew well and wanted to share with the rest of the group. This meant that we learned about a wide range of things, from making tiramisu, to first aid delivery, to the basics of Danish living.

During this first recording session, we kept the equipment at the back of the training room, to get participants used to being recorded. The recordings were then downloaded onto six laptops and at the end of the day, each participant was able to watch their presentation on their own, filling out a feedback sheet to capture their reflections. These sheets showed the positive reactions to watching the recording, which participants found helpful to identify things, often related to bodily mannerisms, they wanted to improve. Further conversations revealed that some participants had been recorded before, but had not been able to watch the footage, so making this possible in the training room emerged as an important take-away of the pilot.

We returned to London on Thursday to film a 30-minute training session that the participants, who had been split into two groups, had prepared over the last two days, putting the learning from the course so far into practice. This hands-on learning is an effective pedagogy and it was impressive to see the activities participants had put together, focusing on getting prepared for a first humanitarian deployment and working with vulnerable children, in such a short period of time. The 30-minute recording was then edited to a 15-minute clip, which the groups watched on their own. That participants said that they would have been okay watching the recordings in a large group was one indication that they had become more confident about being recorded and could also see the value of peer feedback, in addition to feedback from the trainer and their own self-reflections.

The questions on the feedback sheet focused on the improvements participants could see from Monday, which were also discussed in a group reflection session at the end of the day. One insight we obtained was that many observations focused, again, on bodily mannerisms (‘I’m moving my arms too much,’ ‘I’m still doing the tap dance’), so that more efforts need to be made to shift reflection and discussion towards more specific areas of training delivery such as clarity of instructions, transitions between activities and interactions with the rest of the group, for example.

We are now evaluating the footage from this first pilot and summarising our findings into a report that will help us prepare a more specific technology kit and set of activities for the next pilot, which will take place early in the new year in an international location.

Authors: Anke Schwittay and Paul Braund, University of Sussex