Comparing RAM-OP with SMART

RAM-OP: Comparing RAM-OP with SMART from the second field test, December 2014

HelpAge International, Valid International and Brixton Health are developing an innovative method of assessing the needs of older people in emergencies, including their nutritional needs. The Rapid Assessment Method for Older People (RAM-OP) is meant to offer a simple, quick, reliable, robust and cost effective method for assessing the nutritional status and vulnerabilities of older people in emergencies.

Two field tests were carried out, each comparing RAM with the SMART method. The first field test was undertaken in Addis Ababa (Ethiopia) in February/March 2014. The second one was done in a rural district of Tanzania, Kibaha District Council in September 2014. This blog describes key results of the second field trial.

For this second field test, we tried a different RAM prototype: we used a smaller sample (192 older people instead of 240), and for the first stage sampling we sampled 12 clusters of 16 individuals (instead of 20 clusters of 12 individuals).

The questionnaire was also modified between the first and second test, by using more standardised, already tested indicators, rather than simple questions (for example, the question: “are you having problems with your eyesight?” was replaced by a rapid test of visual acuity, using the tumble E at two meters; or the question “is the person confused?” was replaced by six questions forming a rapid dementia screening). This was decided after comparing RAM and SMART results in the first field test, and discovering that for some questions, the answers were very different from one survey to the other, meaning these questions were not thorough enough to result in a precise and accurate answer.

We compared the results from the SMART and RAM-OP surveys using a statistical test. We also compared the 95% confidence intervals on the difference between indicator estimated from the SMART and RAM-OP surveys.

A first conclusion was that the two survey methods produced broadly similar results for all indicators. No statistically significant differences in estimates of indicator levels were found between the two methods. This probably reflects the efforts made in standardising the questionnaire, as well as translating it in the enumerators’ language (Swahili), and in improving the enumerators training.

Secondly, all the estimates of the indicators from the RAM-OP data were produced with useful precision. However, the precision achieved by the RAM-OP survey was generally worse than the precision achieved by the SMART survey.

The cost of the RAM-OP survey was about 60% the cost of the SMART survey.

In conclusion, the second field test has shown that the RAM-OP survey provided results comparable to the SMART survey at about two-thirds of the cost. The loss in precision indicates that we should recommend using the same sample size (192 individuals) while increasing the number of clusters in the first stage sampling, i.e. 16 clusters of 12 individuals.

Our next steps are to simplify the questionnaire, finalise the data entry tool, and start writing the user’s manual.