1-800-SAMPLING : one phone call away from a survey

This month we examine the results of our baseline food security survey in Galkayo, Somalia. Also some thoughts about how our calls will be implemented.

Insights from Galkayo

After analysis of the face to face survey carried out in Galkayo in September, we now know much more about prospective mVAM respondents. We’ve also determined the extent to which they differ from the general population – not much, it turns out.

How did we select mVAM respondents? : a note on sampling

A two stage cluster sampling approach was taken for the face to face survey. Four major Internally Displaced Person (IDP) settlements were selected: Galkayo North, Galkayo South, Guriel and Abduwaq. Each settlement was considered as a separate reporting domain, and applying the rule of thumb for household food security surveys, the minimum target number of households to be interviewed was determined at 150 per settlement. Within each settlement, households were selected following systematic sampling. In total, 656 households were interviewed out of which 495 households owned a phone and some 400 households expressed interest in participating in the mVAM surveys. This sub sample is the initial pool of mVAM respondents.

The mVAM project will focus on collecting ‘food security outcome’ indicators, such as the food consumption score and the coping strategies index. It is important to establish whether the respondents are good proxies for the general population – that an mVAM approach would produce accurate results. In order to do that, we compared the food consumption score of people who agreed to participate in mVAM surveys and the general IDP population.

Results for the food consumption score (FCS) for the general IDP population and mVAM respondents appear in graph 1. For both groups, one household out of five had ‘poor’ food consumption, while the same proportion had ‘borderline’ food consumption. Our t-test show that, in statistical terms, the FCS for both groups is identical (p=0.71). This indicates that the mVAM respondents offer a good proxy for the general population for this indicator. We obtained similar responses for coping. Whereas 94% of households reported having implemented coping strategies, the proportion was nearly identical for mVAM respondents.

Graph 1. Food consumption groups, general IDP population and mVAM respondents

Overall, the analysis shows that mVAM respondents are generally very asset-poor and very few households own productive assets. About one-third of mVAM respondents own small livestock, a proportion similar to the general population. mVAM respondents’ consumption and asset profile matches that of the general population due to the fact that over two thirds of households own mobile phones, and that a large majority of mobile phone owning households are willing to participate.

The literature on mobile phone surveys indicates that to limit bias it is good practice to distribute mobile phones to respondents who do not own phones; we planned on doing so in both DR Congo and Somalia. However, in humanitarian contexts, distributing phones could put respondents at risk. In Somalia we were not at ease with the prospect of distributing mobile phones. The fact that the population of interest has a high level of mobile phone access and that prospective mVAM respondents are a good proxy group implies that we do not need to distribute mobile phones at the start of the project in order to limit bias in our Somalia sample.

Of course, we understand that the challenge will be to ensure that a high percentage of prospective respondents actually answer our phone calls. Thanks to the baseline data we’ve collected, we would also be able to deal with respondent attrition in future rounds by reweighing the sample we obtain in order to ensure that households with specified observable characteristics are adequately represented – for instance, women headed households or very asset-poor households.

Internal vs external service provision: an issue of scale?

In both Somalia and DR Congo we’ll be placing our calls from the nearest WFP sub-office, by dedicated staff that will also work in project monitoring and evaluation. It’s worth discussing why we’ve come to this decision, after exploring partnerships with private companies.

During the scoping phase, we reached out to companies with call centres, with the expertise in telecoms and data management that mVAM requires. However, after several rounds of discussions we have realised that the mVAM pilot is simply too small to get the big commercial guys involved. Once our calling volume is large enough – in the thousands of calls per month, rather than the hundreds that mVAM will implement in its current phase – the large phone companies might take a second look at offering this type of service.

Of course, there are advantages to in-house implementation as well as WFP will be able to institutionalise learning by doing, and that we will have the capability to replicate the set up ourselves in another country in the future.