Challenges of implementing a rigorous, innovative research methodology in a humanitarian setting: national partner perspectives

It is one thing to design a research project in a series of meetings between collaborators; it is quite another to implement the resulting program of work in the field. Over the last year we have run an ambitious, rigorous, innovative research project assessing the impact of the No Lost Generation program with Syrian and Jordanian youth in four sites across Jordan. This has involved conducting face-to-face interviews with youth and, critically, collecting blood, saliva and hair samples to measure biological stress, as well as assessments of cognitive function using specially-designed iPad apps. To date we have undertaken a pilot test and two waves of program evaluation, learning from the first wave to increase the rigor of the second.

We have faced many challenges over recent months. But there are three major ones we want to reflect on, given that our experiences may be useful to others:

The first challenge was the logistical coordination of two groups working in parallel at four field sites: the program implementation team and the research evaluation team. We initially faced difficulties in coordinating the registration of youth for program activities with enrolment in the research study. One big challenge was working through long registration lists for the numerous after-school programs on offer, continuously updating data on our research participant list, and reconciling occasional differences in information between lists. Another hurdle was scheduling fixed times for family visits to program sites so the research team could explain the study, obtain consent, and conduct assessments – early on, we often had many youth and parents arriving at the same time, which meant long waits. We also agreed that siblings should be kept together, to save families making multiple trips to the program centers. Eventually, we managed to synchronize our procedures, and later phases of the research were much smoother. Lessons learnt in our pilot work and Wave 1 were critically important for Wave 2.

The second challenge concerned the acceptability and understanding of our physiological data collection effort. From the outset, we were aware that requesting consent to take blood pressure and saliva, hair and blood samples would raise questions for many participants and their parents, and we were committed to providing full details about what these procedures involved, and why they were of interest to us. To address one concern, we hired a hairdresser to offer free haircuts to youth following our collection of a small hair sample close to the scalp, to be used for lab analysis of cortisol hormone levels before and following the intervention.

We thoroughly explained the procedure for the finger-prick blood sample, used for analysis of immune function, and for cheek swabs, which would allow in-depth analysis of stress biomarkers. Only a few parents declined consent for a component of the physiological assessment, and a small number of youth declined participation in the blood collection. The vast majority agreed to participate in the full complement of measures; the key here was to have any unfamiliar procedures thoroughly well-explained by locally-trusted research team members.

The third challenge was a product of the decision to move to a randomized controlled trial design in Wave 2. We agreed to randomize youth into two time periods – ‘now’ (the upcoming program) or ‘later’ (the next available sessions, twelve weeks later). We wanted families to be fully on board with this process, and also streamline registration. We developed a procedure where lollipops of two different colors were drawn from a ‘candy bag’ in a way that made the allocation to ‘now’ or ‘later’ explicitly down to chance, rather than the decision of the program team. However, there were some difficulties in carrying this out. For example, there was some disappointment if some friends were allocated to ‘now’ and others to ‘later’. Also, by the time the ‘later’ wave of the programming came around, some youth had lost their enthusiasm to engage. Overall, however, given that Mercy Corps did not have the capacity to deliver the program to all interested youth at the same time, this randomization procedure made the allocation of participants to available places transparent.

We have learned so much over the last year working together on this project and dealing with new people every day. We have enjoyed our role in explaining the purpose of the study to the parents and participants. In the course of this, we have heard many stories that made us more passionate about the purpose of this study: ensuring we develop programmes that bring real benefit to youth and their families in contexts of adversity.

Rana Dajani, Founder and Director of Taghyeer, and Noura Shahed, No Lost Generation Project Coordinator with Mercy Corps Jordan, are key collaborating partners responsible for coordinating local data collection across research project sites in Irbid, Jarash, Zarqa and Mafraq, Jordan.