Heads up – this post is about content rather than data viz
1. Describe your sample – they are not the population
Describe who answered your survey – as well as how they may be different from who didn’t answer it. For example, if a survey about hot lunches is answered by 100 of 400 parents at a school. Report both these numbers and note that parents who buy lunches are more likely to have answered the survey. NOT “Our hot lunch survey shows that 70% of the parents at our school are satisfied with the lunches being served.”
2. Report frequencies by response category rather than number counted
USE:
Oranges 4
Apples 3
Berries 2
Grapes 2
Bananas 1
Plums 1
NOT:
Plums, Bananas 1
Grapes, Berries 2
Apples 3
Oranges 4
3. Always let your reader know how many people your % is based on – and who they are
In text – In this region, % of 112 participants used…
Under chart/table – Base = 112 participants
In methodology para – All percentages calculated on responses from 112 participants.
Do not use percents when the number of responses is less than 20. Report numbers rather than percents. For example, “four participants responded they liked oranges.”
4. For questions your reader may want to compare - use consistent denominators - for example, calculate percents for each question on a) all responses or b) a sub-group of responses filtered on the same question
5. Report on all those who were asked the question, not just those who answered it
Non-responses are “responses” as well. They let you know how relevant the question is to those answering your survey. They also let you know when you’re not getting all the information you want with the response categories you’ve included. If a large proportion of your respondents didn’t answer a question:
1. Describe your sample – they are not the population
Describe who answered your survey – as well as how they may be different from who didn’t answer it. For example, if a survey about hot lunches is answered by 100 of 400 parents at a school. Report both these numbers and note that parents who buy lunches are more likely to have answered the survey. NOT “Our hot lunch survey shows that 70% of the parents at our school are satisfied with the lunches being served.”
2. Report frequencies by response category rather than number counted
USE:
Oranges 4
Apples 3
Berries 2
Grapes 2
Bananas 1
Plums 1
NOT:
Plums, Bananas 1
Grapes, Berries 2
Apples 3
Oranges 4
3. Always let your reader know how many people your % is based on – and who they are
In text – In this region, % of 112 participants used…
Under chart/table – Base = 112 participants
In methodology para – All percentages calculated on responses from 112 participants.
Do not use percents when the number of responses is less than 20. Report numbers rather than percents. For example, “four participants responded they liked oranges.”
4. For questions your reader may want to compare - use consistent denominators - for example, calculate percents for each question on a) all responses or b) a sub-group of responses filtered on the same question
5. Report on all those who were asked the question, not just those who answered it
Non-responses are “responses” as well. They let you know how relevant the question is to those answering your survey. They also let you know when you’re not getting all the information you want with the response categories you’ve included. If a large proportion of your respondents didn’t answer a question:
- report the total percentage and some of the possible reasons why so many did not answer it – was it perhaps not applicable to them, did their likely response not fit with the categories listed (based on the themes in responses to other questions). For example, “65% were satisfied with this service, though 20% didn’t rate it – possibly because they didn’t access it.”
- report the number who answered the question, as well as recalculated percentages and some of the possible reasons why others didn’t answer it. For example, “80 clients rated this service, likely because they accessed it. Of these 80 clients, 81% were satisfied with the service.”
RSS Feed