First pilot of Nguvu: What did we learn?

In preparation for a more structured evaluation of the implementation of the intervention and research activities, we felt it was necessary to conduct a first trial run of the research and intervention activities that comprise the Nguvu project. To do so, we sampled 60 women from one of the seven Congolese zones in Nyarugusu camp following procedures developed for the upcoming randomized controlled trial. Eligible participants were adult female Congolese refugees living in Nyarugusu with a past-year history of IPV and experiencing moderate to severe psychological distress in the past month. We recruited women from local women’s groups in the camp, which focus on group activities and skills including cooking and weaving. If a woman met the aforementioned eligibility criteria she underwent an interview, which consisted of all measures we were considering for inclusion in the randomized controlled trial. She was then assigned to a pair of facilitators, who proceeded to administer the intervention to her and 11 other women assigned to the group over the course of eight weeks (one session per week). Upon completion of all intervention groups in February 2017, we randomly selected 10 women who had attended most sessions and 10 women who had attended very few sessions to interview them about their experience with the Nguvu project.

Pilot Evaluation of Measurement Methods: Results

We assessed the psychometric performance of the measurement tools we tested during the pilot study and reviewed the qualitative exit interviews and other structured assessments of intervention fidelity and monitoring. Preliminary results from the measurement performance evaluation revealed that most of the measures we included were reliable over time (i.e., test-retest reliability) and across research assistants (i.e., inter-rater reliability). We found that within a measure, the items seemed to be highly correlated, which suggests that they are measuring a unidimensional construct or similar constructs that are highly correlated (i.e., internal consistency). There were a few measures that did not perform as well and thus required further adaptation prior to the randomized controlled trial. For example, the 22-item functional impairment measure, which assesses difficulty in performing daily activities had very low internal consistency. Further examination of the functional impairment items revealed that several types of activities including farming, trading and making money, and fetching firewood were irrelevant for many women in this setting. Furthermore, socializing and taking part in community events displayed low test-retest reliability suggesting that these behaviors were not stable over time. Items displaying poor performance on these dimensions were removed, which resulted in an improved 12-item scale that displayed adequate test-retest reliability, inter-rater reliability and internal consistency.

Pilot Evaluation of Intervention Implementation: Results

Direct supervision revealed that the facilitators were capable of implementing the Nguvu intervention. We had several very strong members of our facilitator team that served to mentor members with less experience and general skills in counseling and the delivery of the intervention. This peer support and direct supervision was capable of improving their performance to a level where they would sufficiently be able to deliver the manualized Nguvu intervention in a structured evaluation. Results of the exit interviews suggested that participants that attended the majority of the intervention sessions were very pleased with the intervention and found it useful in their lives. The participants that attended few or no intervention sessions, on the other hand, cited several logistical barriers and lack of incentives as their reasons for not attending more sessions.

A somewhat unexpected finding of the pilot study was the overwhelming commitment and support for the potential benefits of the intervention by the facilitators. Several of the intervention facilitators have experienced violence and psychological distress, similar to the target population of this intervention. During refresher trainings and team debrief meetings they would often share of the personal benefit they have derived from the Nguvu intervention in their lives. This same message was not communicated by the research team, as would be expected given that they did not have any (or minimal) exposure to the intervention and instead only studied the measurement tools and general objective of the program. Instead, the research team expressed a more concerning response. Several members of the research team reported feeling distressed by their personal situation of violence and psychological distress. To address this concern, we began to provide structured psychological support services to members of our research team and will continue to do so for the research assistants throughout the duration of the Nguvu project.

Next Steps

We are currently implementing the randomized controlled trial, which incorporated the lessons learned in the pilot into the content and delivery of the intervention and research evaluation components of the Nguvu project.In preparation for a more structured evaluation of the implementation of the intervention and research activities, we felt it was necessary to conduct a first trial run of the research and intervention activities that comprise the Nguvu project. To do so, we sampled 60 women from one of the seven Congolese zones in Nyarugusu camp following procedures developed for the upcoming randomized controlled trial. Eligible participants were adult female Congolese refugees living in Nyarugusu with a past-year history of IPV and experiencing moderate to severe psychological distress in the past month. We recruited women from local women’s groups in the camp, which focus on group activities and skills including cooking and weaving. If a woman met the aforementioned eligibility criteria she underwent an interview, which consisted of all measures we were considering for inclusion in the randomized controlled trial. She was then assigned to a pair of facilitators, who proceeded to administer the intervention to her and 11 other women assigned to the group over the course of eight weeks (one session per week). Upon completion of all intervention groups in February 2017, we randomly selected 10 women who had attended most sessions and 10 women who had attended very few sessions to interview them about their experience with the Nguvu project.

Pilot Evaluation of Measurement Methods: Results

We assessed the psychometric performance of the measurement tools we tested during the pilot study and reviewed the qualitative exit interviews and other structured assessments of intervention fidelity and monitoring. Preliminary results from the measurement performance evaluation revealed that most of the measures we included were reliable over time (i.e., test-retest reliability) and across research assistants (i.e., inter-rater reliability). We found that within a measure, the items seemed to be highly correlated, which suggests that they are measuring a unidimensional construct or similar constructs that are highly correlated (i.e., internal consistency). There were a few measures that did not perform as well and thus required further adaptation prior to the randomized controlled trial. For example, the 22-item functional impairment measure, which assesses difficulty in performing daily activities had very low internal consistency. Further examination of the functional impairment items revealed that several types of activities including farming, trading and making money, and fetching firewood were irrelevant for many women in this setting. Furthermore, socializing and taking part in community events displayed low test-retest reliability suggesting that these behaviors were not stable over time. Items displaying poor performance on these dimensions were removed, which resulted in an improved 12-item scale that displayed adequate test-retest reliability, inter-rater reliability and internal consistency.

Pilot Evaluation of Intervention Implementation: Results

Direct supervision revealed that the facilitators were capable of implementing the Nguvu intervention. We had several very strong members of our facilitator team that served to mentor members with less experience and general skills in counseling and the delivery of the intervention. This peer support and direct supervision was capable of improving their performance to a level where they would sufficiently be able to deliver the manualized Nguvu intervention in a structured evaluation. Results of the exit interviews suggested that participants that attended the majority of the intervention sessions were very pleased with the intervention and found it useful in their lives. The participants that attended few or no intervention sessions, on the other hand, cited several logistical barriers and lack of incentives as their reasons for not attending more sessions.

A somewhat unexpected finding of the pilot study was the overwhelming commitment and support for the potential benefits of the intervention by the facilitators. Several of the intervention facilitators have experienced violence and psychological distress, similar to the target population of this intervention. During refresher trainings and team debrief meetings they would often share of the personal benefit they have derived from the Nguvu intervention in their lives. This same message was not communicated by the research team, as would be expected given that they did not have any (or minimal) exposure to the intervention and instead only studied the measurement tools and general objective of the program. Instead, the research team expressed a more concerning response. Several members of the research team reported feeling distressed by their personal situation of violence and psychological distress. To address this concern, we began to provide structured psychological support services to members of our research team and will continue to do so for the research assistants throughout the duration of the Nguvu project.

Next Steps

We are currently implementing the randomized controlled trial, which incorporated the lessons learned in the pilot into the content and delivery of the intervention and research evaluation components of the Nguvu project.