Creating reliable and valid measures for GBV survivors

As part of delivering quality services for GBV survivors, practitioners hope to reduce survivors’ felt stigma and improve their psychosocial functioning. BUT - what do stigma and psychosocial functioning really mean and how would you measure them? Complex psychosocial constructs such as these can be difficult to measure – and most of the time, they can be more difficult to properly assess than straight-forward outcomes like blood pressure or mid-upper arm circumference for malnutrition.

So how do we know if we are measuring these concepts well? We have to look at their reliability and validity in the tools that we are using.

Reliability

Within the Raising the Bar project, we are testing the reliability of felt stigma and psychosocial functioning across two different refugee populations in Jordan and Kenya. While there are different ways to examine reliability, we are going to test how our measures are performing by examining their internal consistency. In non-jargon language, this essentially means we are looking to understand how all of the different items or questions that we think might increase our understanding of felt stigma or psychosocial functioning, correlate with each other. This will also tell us which items in a scale are not that valuable to understanding psychosocial functioning and can then be dropped from the scale to make a shorter M&;E tool that might be more easily integrated into GBV response services.

We will be able to assess this by asking survivors the questions in our adapted scales and running statistical analyses. More information about how to measure felt stigma and psychosocial well-being across cultures can be found here.

Validity

Let’s imagine we now have a scale for psychosocial functioning that we think is very reliable across Jordan and Kenya. Does that mean it is actually measuring psychosocial functioning? Well - not exactly. Even though a scale can be reliable, it does not always mean it is a valid measure of what you are trying to achieve. And not surprisingly, there are many ways and manners in which statisticians can look at validity.

For our purposes within the Raising the Bar project, we are primarily going to look at validity through construct validity. To tell us whether our new measures have this type of validity, we have added previously tested scales of emotional wellbeing and enacted stigma, among other scales, to the surveys we are administering to survivors in case management or psychosocial support services. We have added these scales in because we think that a survivor’s score on felt stigma, for example, would be correlated with their emotional wellbeing or actual enacted experiences of stigma. If the correlation is high, this provides evidence that our new measures are valid and are actually achieving what they are meant to.

Of course, there are a number of other ways to examine different types of reliability and validity of measures, but these are just a few ways in which we will be doing it within Raising the Bar. While it requires a number of steps, such as involving survivors in fairly lengthy surveys and carrying out statistical analyses, it will hopefully yield a valid and reliable M&;E tool that is easily used and ultimately going to improve services and outcomes for GBV survivors.