Can we use SMS for food security surveys in a Congolese IDP camp?

Almost one year into data collection, we are now fairly confident that live voice calls, placed by operators, are a good way to stay in touch with people in the extremely vulnerable communities we work with. Since January 2014, we have been able to conduct monthly rounds of phone surveys typically reaching between half and two-thirds of selected respondents, while collecting data of good quality. However, it’s not yet clear if either IVR or SMS offer the same advantages in our pilot contexts.

Response rates to voice calls, DR Congo

Source: WFP

SMS: cool tool, wrong setting?

This month, we attempted to understand whether SMS surveys would work in an IDP camp. Using SMS is attractive, because it is low-cost and easy to automate using free software. While we have had good results with SMS(link is external)when running simple national or province-level food security surveys, we have yet to evaluate the tool’s suitability in a high-vulnerability refugee camp setting. In November, two Rome-based mVAM team members, Marie and Lucia, travelled to Goma to attempt to do just that. They helped the team in Goma organize a simple food security survey involving face-to-face interviews, live voice calls and SMS. The data collected from this exercise will allow us to understand the strengths and weaknesses of these different survey tools.

Residents in a Congolese refugee camp responding to text messages

In order to run the SMS survey, we used Pollit(link is external) a free, open-source tool. It is easily accessible with an internet connection and requires only minimal hardware to function—a computer, a mobile phone and an internet connection. The tool was developed by In(link is external)STEDD, the same company that developed Verboice(link is external), the programme we are using for IVR calls. In the future, Pollit may allow us to periodically run short SMS surveys in-house, bringing a lot of flexibility to our field teams. During the Goma test, Pollit proved to be a simple and flexible tool. It was easy to set up and worked smoothly during the six days of data collection.

However, response rates to SMS surveys turned out to be low, particularly compared to voice calls and face-to-face surveys. Our enumerators reported that people in the camp are not used to using the SMS function on their phones. They typically communicate using voice calls, due to low literacy and habit. In some cases, the phones people owned were broken or had dirty screens, making it difficult to read and reply to the messages we were sending. These issues, however, do not prevent us from using voice calls, which seem to be the preferred modality amongst respondents in DR Congo. This seems to suggest that we should stick to live calls for Mugunga 3 camp, and use SMS questionnaires in other settings. We are now analyzing the data we collected in Goma in order to answer other questions we have, which includes comparing data quality for the different survey modes. We’ll be sure to share those insights later.