Can we trust the results?

No-one can guarantee that every answer given by every respondent in a survey is completely accurate and honest.  However, we can go a long way to improving our confidence in the figures by taking care over each aspect of the process:

  • Questionnaire design

We strive to ask questions in a clear and unbiased way, concentrating on what has actually happened in pupils' and students' lives.  So, we ask questions like 'what did you eat or drink before lessons this morning?', and not 'what do you usually eat for breakfast?' (which, in just seven words, makes three different assumptions that might not be correct!).  Questions are scrutinised before use, not just by SHEU staff, but also by expert groups from local authorities, such as teachers, advisers and public health professionals.

  • Piloting

Questions are tested with smaller groups before use across our services.  Once we have evidence that the answers from small-scale work are meaningful and accurate, we can support more wide-scale use.

  • Post-questionnaire interviews

We have regularly conducted interviews with pupils and students following completion of a pilot questionnaire.  These interviews can let us know that pupils and students are willing and able to find and choose an answer that properly reflects their situation.

  • Introduction to the pupils and students

Pupils and students must feel that the exercise has some value if they are to cooperate.  We produce extensive guidance for schools explaining ways of introducing the purpose and nature of the survey to pupils and students. 

  • Feedback from schools about data collection

We ask for feedback from every single class that completes one of our surveys.  Staff who supervise data collection are alert to problems that arise and provide important feedback on questions that could be worded better. 

  • Response checking

Staff at SHEU inspect every set of responses coming into the Unit.  Our experienced data preparation staff are sensitive to any odd-looking responses and we will respond, perhaps by excluding an individual's questionnaire (if they are not taking it seriously) or even by excluding everyone's answers if the responses from one question are not consistent with other trusted information.

  • Internal reliability

Questions in one part of the questionnaire can be related to another; for example, a question on recent spending might report spending on sweets, which can be compared with a question many pages away about eating habits.  When we do these comparisons, the match is always very good.

  • External reliability

Figures from one survey can be compared with results from another survey.  For example, we know a lot about patterns of cigarette smoking among young people over the last 30 years from Government surveys.  The questions they use have been double-checked, not just using the approaches described above, but also by saliva tests that show if someone is smoking.  SHEU figures show very much the same levels and also the same rises and falls over the years as are seen in the Government reports.

  • Feedback from schools and authorities

Once results are circulated, we invite feedback from schools and authorities by asking directly if they have confidence that the figures accurately reflect their young people.  They may have other supporting sources of information that we aren't aware of, or a sense that the picture they see is about right.

 

For more information, see the SHEU publications from the Young People into… series, or contact Dr.David Regis, the Research Manager at SHEU.

Topics: 

Comments about SHEU

"Our use of the Health-Related Behaviour Questionnaire was commended as part of our accreditation for the National Healthy Schools Scheme." Headteacher

"Every school involved in the National Healthy School programme should start with an HRBQ survey." Health Education Co-ordinator

Health Education Co-ordinator

"Just to say a huge thank you for all your efforts in helping us with the Health survey amongst pupils. It has provided us with significant data which will be used across the school to help us improve. It helped us to obtain a healthy schools standard as well. I hope we can make this an annual feature as we can track the changing health of our pupils." Headteacher

Headteacher

"Over the last twenty years you have achieved much. The surveys and subsequent reports have painted the clearest picture we have of what young people are doing and what they think." OFSTED 1998

OFSTED

...our analyst here in Public Health- is beside himself with excitement about all the juicy data pouring in...he can't wait to get his hands on it!!!!
He is happier than I have seen him for years.

Public Health Principal

"The Schools Health Education Unit is the jewel in the crown of the Health Education Authority." Major General Sir John Acland 1928 - 2006

Major General Sir John Acland

"Within the curriculum, we are part of the Healthy Schools programme - and the local, Director of Public Health Award. We cover many facets of health from emotional intelligence to safety education and our very strong, Anti-Bullying and Child Protection programmes. You can imagine our delight when the Local Authority and our school nurse made the following comments after we took part in the regional Schools Health Education Unit Survey: " Head Teacher.
“This was an amazing set of outcomes and really good evidence that (your school) is doing a wonderful job in prioritizing the health and well-being of its pupils … Well done to staff, governors and parents for all your work on this through the Director of Public Health award and other strategies. It is very clear that pupils feel happy, safe and involved at the school and your caring ethos shines through this data.”
Healthy Schools Coorduinator.

 

Headteacher & Healthy Schools Coordinator

Many thanks to you and your team for turning this around so quickly.
We really appreciate it, and also your extra support with the administration this time.

PSHE Lead Advisor

"I have never looked at myself in this way before." Pupil

Pupil

"The data from the 2018/19 survey is still in heavy use here, the physical activity related findings were pivotal in changing the relevant strategy recently to target less active groups like girls towards the end of secondary school, and I’ve three fairly hefty jobs on the to-do list that will use the data with other sources to identify target schools for mental health and physical activity projects, and another looking at community safety. I call it the gift that keeps giving and that certainly seems to be the case!"

Senior Public Health Specialist (Intelligence)