Can we trust the results?

No-one can guarantee that every answer given by every respondent in a survey is completely accurate and honest.  However, we can go a long way to improving our confidence in the figures by taking care over each aspect of the process:

  • Questionnaire design

We strive to ask questions in a clear and unbiased way, concentrating on what has actually happened in pupils' and students' lives.  So, we ask questions like 'what did you eat or drink before lessons this morning?', and not 'what do you usually eat for breakfast?' (which, in just seven words, makes three different assumptions that might not be correct!).  Questions are scrutinised before use, not just by SHEU staff, but also by expert groups from local authorities, such as teachers, advisers and public health professionals.

  • Piloting

Questions are tested with smaller groups before use across our services.  Once we have evidence that the answers from small-scale work are meaningful and accurate, we can support more wide-scale use.

  • Post-questionnaire interviews

We have regularly conducted interviews with pupils and students following completion of a pilot questionnaire.  These interviews can let us know that pupils and students are willing and able to find and choose an answer that properly reflects their situation.

  • Introduction to the pupils and students

Pupils and students must feel that the exercise has some value if they are to cooperate.  We produce extensive guidance for schools explaining ways of introducing the purpose and nature of the survey to pupils and students. 

  • Feedback from schools about data collection

We ask for feedback from every single class that completes one of our surveys.  Staff who supervise data collection are alert to problems that arise and provide important feedback on questions that could be worded better. 

  • Response checking

Staff at SHEU inspect every set of responses coming into the Unit.  Our experienced data preparation staff are sensitive to any odd-looking responses and we will respond, perhaps by excluding an individual's questionnaire (if they are not taking it seriously) or even by excluding everyone's answers if the responses from one question are not consistent with other trusted information.

  • Internal reliability

Questions in one part of the questionnaire can be related to another; for example, a question on recent spending might report spending on sweets, which can be compared with a question many pages away about eating habits.  When we do these comparisons, the match is always very good.

  • External reliability

Figures from one survey can be compared with results from another survey.  For example, we know a lot about patterns of cigarette smoking among young people over the last 30 years from Government surveys.  The questions they use have been double-checked, not just using the approaches described above, but also by saliva tests that show if someone is smoking.  SHEU figures show very much the same levels and also the same rises and falls over the years as are seen in the Government reports.

  • Feedback from schools and authorities

Once results are circulated, we invite feedback from schools and authorities by asking directly if they have confidence that the figures accurately reflect their young people.  They may have other supporting sources of information that we aren't aware of, or a sense that the picture they see is about right.

 

For more information, see the SHEU publications from the Young People into… series, or contact Dr.David Regis, the Research Manager at SHEU.

Topics: 

Comments about SHEU

I think the HRBQ is an absolutely wonderful, informative tool and I am keen to really encourage schools/other relevant partners (as appropriate) to utilise this invaluable data to encourage voice of the child, and subsequently contribute to shaping priorities and services.

Senior Health Improvement Practitioner (Children and Young People)

"It's good to talk with you again - we used so much of those data and did so much with it in schools - we really got the place buzzing!"

Consultant in Public Health

"Many thanks to SHEU for your excellent professional support over the years."

PSHE teacher

"The Unit is to be congratulated in preparing ... material of the highest standard and worthy of wide dissemination." National Association for Environmental Education

National Association for Environmental Education

"The data for (us) are very useful ... This is especially important when evaluating the impact of interventions regarding alcohol or other areas, as the survey data are likely to provide an earlier indication than routine data sources."

Specialist Registrar in Public Health

"The service you provide is of national significance." Health Development Agency

Health Development Agency

At the time, the results were very useful and the feedback report very useful and insightful. Significant changes will occur in our schools health and wellbeing provision next year and conducting another survey will certainly help me to ensure I am planning effectively for the needs of our pupils.

Head of Health and Wellbeing

"Just to say a huge thank you for all your efforts in helping us with the Health survey amongst pupils. It has provided us with significant data which will be used across the school to help us improve. It helped us to obtain a healthy schools standard as well. I hope we can make this an annual feature as we can track the changing health of our pupils." Headteacher

Headteacher

"I would like to say that this survey was very useful and made me realise things about PE and health that I had never realised before......Food at school is groovy, especially if your school does Jamie Olivers School Dinners. Viva apples and thanks for the survey." Female pupil, 13 yrs old

Female pupil, 13yrs

"The credit goes to you for the fabulous information the survey yields!"

Assistant Director Schools and SEN