Can we trust the results?

No-one can guarantee that every answer given by every respondent in a survey is completely accurate and honest.  However, we can go a long way to improving our confidence in the figures by taking care over each aspect of the process:

  • Questionnaire design

We strive to ask questions in a clear and unbiased way, concentrating on what has actually happened in pupils' and students' lives.  So, we ask questions like 'what did you eat or drink before lessons this morning?', and not 'what do you usually eat for breakfast?' (which, in just seven words, makes three different assumptions that might not be correct!).  Questions are scrutinised before use, not just by SHEU staff, but also by expert groups from local authorities, such as teachers, advisers and public health professionals.

  • Piloting

Questions are tested with smaller groups before use across our services.  Once we have evidence that the answers from small-scale work are meaningful and accurate, we can support more wide-scale use.

  • Post-questionnaire interviews

We have regularly conducted interviews with pupils and students following completion of a pilot questionnaire.  These interviews can let us know that pupils and students are willing and able to find and choose an answer that properly reflects their situation.

  • Introduction to the pupils and students

Pupils and students must feel that the exercise has some value if they are to cooperate.  We produce extensive guidance for schools explaining ways of introducing the purpose and nature of the survey to pupils and students. 

  • Feedback from schools about data collection

We ask for feedback from every single class that completes one of our surveys.  Staff who supervise data collection are alert to problems that arise and provide important feedback on questions that could be worded better. 

  • Response checking

Staff at SHEU inspect every set of responses coming into the Unit.  Our experienced data preparation staff are sensitive to any odd-looking responses and we will respond, perhaps by excluding an individual's questionnaire (if they are not taking it seriously) or even by excluding everyone's answers if the responses from one question are not consistent with other trusted information.

  • Internal reliability

Questions in one part of the questionnaire can be related to another; for example, a question on recent spending might report spending on sweets, which can be compared with a question many pages away about eating habits.  When we do these comparisons, the match is always very good.

  • External reliability

Figures from one survey can be compared with results from another survey.  For example, we know a lot about patterns of cigarette smoking among young people over the last 30 years from Government surveys.  The questions they use have been double-checked, not just using the approaches described above, but also by saliva tests that show if someone is smoking.  SHEU figures show very much the same levels and also the same rises and falls over the years as are seen in the Government reports.

  • Feedback from schools and authorities

Once results are circulated, we invite feedback from schools and authorities by asking directly if they have confidence that the figures accurately reflect their young people.  They may have other supporting sources of information that we aren't aware of, or a sense that the picture they see is about right.

 

For more information, see the SHEU publications from the Young People into… series, or contact Dr.David Regis, the Research Manager at SHEU.

Topics: 

Comments about SHEU

Any comments on specific survey questions that may have caused difficulty? No problems. My children were fully briefed before the survey and they understood that they could miss questions if they did not want to answer them. We did not have any children with any concerns regarding the survey at all.

Any comments on the use of the web site? No it was easy to use.

Any general comments on the exercise? The children really enjoyed completing the survey, It has lead to many positive discussions about our health and how we all have slightly different experiences and home lives.

Class teacher

...Many thanks for all your work with this year’s survey. It’s already proving to be invaluable especially against the severe cuts in services that are looming.

Health Promotion Manager

"Many thanks to SHEU for your excellent professional support over the years."

PSHE teacher

"This week I have been working on a major strategy for service design -- it is easy to get solely focussed on hospitals, performance targets, and work force planning -- all very important; but at the same the needs of young people and how we need to work across agencies to address the health needs of today and tomorrow must be recognised. SHEU is founded to do just this."

PCT Performance Manager

I think the HRBQ is an absolutely wonderful, informative tool and I am keen to really encourage schools/other relevant partners (as appropriate) to utilise this invaluable data to encourage voice of the child, and subsequently contribute to shaping priorities and services.

Senior Health Improvement Practitioner (Children and Young People)

"I would like to take this opportunity to thank you for your work regarding writing and compiling the sex education survey. The survey was well executed and the schools have found their individual reports very helpful. The results of the survey have enabled the Local Campaign Group to justify the need for young men's campaigns and given us invaluable insight as to the thoughts and experiences of this target group."

Teenage Pregnancy Strategy Manager

(Our) Senior team were very enthused with the rich source of data provided within the reports (and thought that the analyses including within the appendices section of the main reports were really interesting).

Health Improvement Specialist (Children, Schools and Families)

"Thank you very much, David, for another excellent survey.  We look forward to receiving our reports."

Healthy Schools Co-ordinator

"The credit goes to you for the fabulous information the survey yields!"

Assistant Director Schools and SEN

Many thanks to you and your team for turning this around so quickly.
We really appreciate it, and also your extra support with the administration this time.

PSHE Lead Advisor