In this guide
Hide menuSpecific instruments to measure social norms
Show sectionsSpecific instruments to measure social norms
Survey questions are the main quantitative method used to measure the presence and strength of norms, with respondents asked about their own views, about those whose opinions influence their views (i.e. their reference groups) and their perspectives of their communities. Surveys may also contain vignettes, which elicit responses to hypothetical scenarios. Respondent reactions to vignettes – and their perceptions of community reactions – can also shed light on the influence of norms on specific outcomes.
Survey-based instruments
Survey-based instruments may be administered more than once, e.g., at baseline and endline, to monitor and evaluate the impact of interventions to change norms. This can be seen in the evaluation by Dhar et al. (2018) of a multi-year school-based intervention in Haryana, India, which engaged adolescents in classroom discussions about gender equality. The intervention aimed to shift norms around women’s work outside the home and female education, with a survey drawing out individual attitudes as well as normative expectations on what people think others are thinking. The researchers administered survey questions as follows:
Example of evaluation
Set 1:
E1. Do you think that women should be allowed to work outside home? [Individual belief]
E2. Do you think that people in your village/community think that women should be allowed to work outside the home? [Normative expectation]
E3. Do you think the community will oppose you since [if] you disagree with them? [Perceived sanction]
E4. If the community did not oppose you, would you encourage your sister/cousin sister to work outside home after marriage? [Readiness to act on beliefs]
Set 2:
E1. Do you think that girls should be allowed to study in college even if it is far away?
E2. Do you think that people in your village/community think that girls should be allowed to study in college even if it is far away?
E3. Do you think the community will oppose you since [if] you disagree with them?
E4. If the community did not oppose you, would you encourage your sister/cousin sister to study in college even if it is far away?
Source: Dhar et al. (2018) with my additions in brackets. Note that in the study, students were randomly assigned questions from Set 1 or Set 2.
Using this data, researchers constructed an index of gender attitudes, by assigning a value of ‘1’ to agreement with gender-progressive statements and ‘0’ otherwise, then computing an average score. Using a regression analysis, they then analyse whether attending a school that was randomly selected to participate in the intervention increased the likelihood that a respondent held progressive views. The results showed that the classroom discussions led to more progressive attitudes toward employment and education. They also bolstered student perceptions that others in their community held a progressive view and that the community would not hinder their acting on progressive beliefs.
Many other approaches have been used to uncover norms in surveys (see box below). In some cases, researchers have been able to simplify questionnaires after initial testing by reducing the number of components of a norm that they ask about and/or by reducing multi-item responses to binary measures of approval or disapproval. Indeed, Heise and Cislaghi (2016) argue that many existing methods of measuring social norms are ‘unnecessarily complex’ and that ‘quick and simple ways’ may produce equally valid results.
Survey-based questions designed to uncover norms
Options for wording questions
I. A common approach to structuring norms questions is to use Likert scales to assess the degree to which individuals agree with key summary statements about their setting or reference group. For example:
1. To what extent do you agree with the following statements:
a. Most people in my community would not talk about being beaten by their husband to people outside of the family (Agree, somewhat agree, somewhat disagree, disagree.)
b. Most people in my community would think poorly of a woman who discussed being beaten by her husband with people outside of her family. (Agree, somewhat agree, somewhat disagree, disagree.)
II. Other researchers instead ask people about the frequency with which people engage in the behaviour of interest, or the number of people who do so. For instance:
1. Number:
a. In your village, how many young girls get married before the age of 18? (all, most, some, few, nobody)
b. Among people in your family, how many would approve of you getting married before the age of 18 (all, most, some, few, nobody)
2. Frequency:
a. How often do your friends drink alcohol when socialising? (very often, often, sometimes, never)
b. How often do others [your friends] disapprove if they see you drinking alcohol at a party? (very often, often, sometimes, never?)
III. A third strategy is to simply ask people to report on what they observe about behaviour and attitudes of others in a specific situation:
1. In your experience, when congregating on the street, do most boys around here:
a. Tease young girls when they pass by
b. Let girls pass by without comment
c. Neither
2. In your opinion, when young boys tease girls as they pass by, do “most people around here”:
a. Approve of the teasing
b. Disapprove but tolerate the teasing
c. Disapprove of the teasing
d. Have no strong opinion
IV. Finally, some researchers focus explicitly on the possibility of positive or negative sanctions arising from conforming to or violating a norm. For example:
1. If a young girl was not married by the time she was 18, this would reflect badly on her family [Agree, agree somewhat, disagree somewhat, disagree]
2. If a married woman left her husband and returned to her family after being beaten, neighbours would gossip about her [Agree, agree somewhat, disagree somewhat, disagree]
3. In your experience, if a married woman is beaten by her husband, what percentage of families in your village would accept her back home [All, most, some, few, none]
Source: Cislaghi and Heise (2017)
One ‘quick and simple’ way highlighted by Heise and Cislaghi (2016) draws on the experiences of CARE. Instead of asking respondents about normative expectations directly, researchers asked simpler questions about the sanctions they perceived would apply to community members who did not comply with prevailing norms (see table below).
Example of CARE’s simpler approach to asking about norms
| CARE's measures of social norms | |
|---|---|
| Empirical expectations | Normative expectation/Sanctions |
|
Please tell me how much the following activities are prevalent in your neighbourhood. Do you think such practices/activities and incidents are very prevalent, can be seen sometimes or rarely? “Practices/activities and incidents” were:
[Very prevalent | Sometimes observable | Rarely observable | Do not know] |
I am going read out some attitudes prevalent in our society towards men and women. Could you please tell me to what extent such attitudes exist among the people in your neighbourhood? “Attitudes” were:
[Great extent | To some extent | Does not exist | Do not know] |
Source: Heise and Cislaghi (2016): 18
Vignettes
Vignettes are another way to investigate the presence of social norms and their strength, and are useful tools for monitoring and evaluating whether interventions are helping to shift those norms. A vignette relates a hypothetical story that places an invented character in a specific context, with guiding questions that enable structured responses. Typically, a vignette shows this character taking a particular action, and respondents are asked for their perceptions of that behaviour and/or how it would be perceived within their community (or another reference group).
Vignettes are often used in qualitative research to provoke open-ended discussion around social expectations, sanctions and potential responses. However, they can also be integrated in surveys that present respondents with a pre-established set of responses and ask them to select the extent to which they agree that a character is behaving in conformity with social expectations.
Vignettes have several potential advantages. First, they establish a common frame of reference among respondents by inviting them to reflect on the same scenario. Second, asking respondents to reflect on the behaviour of a hypothetical character may provoke more honest responses on sensitive issues, because it does not demand personal revelations. Third, vignettes could mitigate ‘social desirability bias’, in which respondents adjust their answers based on what they think the questioner wants to hear. Finally, surveys that contain vignettes that vary in a systematic way can enable researchers to determine the causal influence of different potential influences on norms, as shown by Horne et al. (2013) and Liebe et al. (2017). If participants are assigned different vignettes at random, and the vignettes show that different circumstances or actions are associated with distinct outcomes, then this suggests a causal relationship.
One example is the survey-based vignette used by Horne et al. (2013) to investigate norms surrounding the payment of bridewealth in rural Ghana, which sets out different scenarios for a couple who have been living together for three years (without the payment of bridewealth, with partial payment or in a marriage, with payment made in full). It presents aspects of the woman’s behaviour, before asking how her behaviour would be viewed by the community and the likely reaction. The scenarios were set out as follows:
Example of a survey-based vignette
For three years a man and the woman have been:
a. living together. The man has paid no bridewealth
b. living together. The man has paid some of the bridewealth
c. married with full bridewealth paid.
They have no children. The man works in a Governmental institution. The woman has a big store selling cloth in the market that she started with her own money. One day the man found out that the woman had:
a. been using contraception
b. giving most of her earnings from her shop to an old female friend from high school in the next village without telling him.
The questioner then asks the respondent how specific groups within the community would evaluate the rightness/wrongness of the woman’s behaviour, using a 10-point scale.
“When the man found out that the woman had been [using contraception/giving most of her earnings from her shop to an old female friend from high school in the next village] he was very angry. When he came home he beat her.”
Again, the respondent is then asked how specific groups in the community would react to this behaviour.
By comparing how average participant approval of a woman’s contraceptive use varied depending on whether no brideswealth, partial bridewealth or full bridewealth was paid in the vignette, the study shows that the payment of bridewealth strengthened normative obligations that reduced women’s reproductive autonomy. The findings relating to economic autonomy were less clear cut.
Vignettes have also been used to design and measure the effectiveness of initiatives to drive normative change.
- WHO uses vignette-based measures related to norms in its Global Early Adolescent Study, to enhance understanding of the risks to sexual health and the effective promotion of healthy sexuality. Analysis by Blum et al. (2018) suggests that the piloting of this method could capture the perspectives of young adolescents (age 10-14), and confirmed its value in quantifying differences between boys and girls in communication approaches, social inclusion, interpersonal styles and acceptance of gender atypical peers.
- Oxfam WE-Care’s Household Care Survey presented respondents across five countries with three vignettes, each involving a different distribution of paid and unpaid care work between a woman and her spouse. Respondents were asked their opinion of each vignette, how they thought other members of their community would respond and how they would compare the situation with that in their own household. Analysis of the impact by Karimli et al. (2016) found that respondents who had participated in a project to change norms around the gendered division of labour on care issues were more likely to disapprove of the vignette describing a gendered distribution of labour, and the strength of the effect was greater in households in which both the man and woman participated.
Respondents to vignettes in surveys are often asked to indicate their approval or disapproval of a given scenario – for example, the woman’s decision to use contraception in the vignette on bridewealth in Ghana. There is debate over whether the responses should be treated as cardinal (which assumes the distance between them is the same) or ordinal (which treats the ranking of responses as important but makes no assumption about the distance between them). If the responses are treated as cardinal, an average level of approval is often calculated for the population as a whole and/or for distinct groups (e.g. men and women).
Where the data are recognized as ordinal, then statistics such as the median (or ‘typical’) response are common. These indicators can be compared across vignettes, between groups and/or over time, and may be used as an input into multivariate analysis – i.e. to gauge the impact that individual and/or community attitudes have on a given outcome, after controlling for other relevant factors.
Limitations of instruments focused solely on the measurement of social norms
While it is important to develop instruments that yield measures of social norms and their strength, there are important limitations, as noted by Stefanik and Hwang (2017). Measuring social norms can be time- and resource-intensive, and data collection tools can be difficult to explain to enumerators and facilitators, confirming the need for tools that are more efficient and simpler. Respondents to surveys may find it challenging to identify and rank reference groups, and those responding to vignettes may struggle to answer hypothetical questions about how members of their reference groups might react to a given behaviour.