SHEU paper and online questionnaires for young people

 

Paper and online surveys completed by pupils in schools.
Do the same questions perform differently?

 

School and College paper-based surveys  

The Schools Health Education Unit [SHEU] is probably the oldest survey service in England dedicated to collecting data about young people’s healthy lifestyles. Millions of questionnaires have been completed by pupils.

     
SHEU school and college online surveys  

SHEU started school-based paper surveys in 1977 and launched its online survey service in 2004. At that time we had the idea that nearly all the survey work would be online within a few years.

While the proportion of work online has increased, some schools and colleges find that paper surveys are still more convenient for them. We have found that our surveys give robust data via online or paper-based questionnaires. Often the final decision about which survey method to use can depend on the availability and ease of access of computers in schools and colleges

Do the same questions perform differently?

The following is from a small study we carried out some time ago. We had just completed the second wave of survey work in an Authority. The idea was originally that the survey be conducted on our online survey website but in the end about half of the schools preferred to complete the exercise on paper.  This arrangement of 17 schools working on paper and 16 online allowed us to perform a ‘natural experiment’ to compare whether the same questions performed differently in the different media. We generated a whole set of tables that showed the paper and online results side by side.  The figures showed good agreement, we thought: then, we decided, very good... actually, these are spookily similar!

 

Paper

Online

All

No

72%

71%

72%

Don't know

3%

4%

4%

Yes

24%

25%

25%

Valid Responses

%

100%

100%

100%

Count

1720

1779

3499

Total Sample (Count)

1760

1867

3627
 
Not all the tables are that close but you get the idea. This gives us a lot of confidence that the questionnaire is robust across the different methods of delivery.  Whatever headaches it gives our programming colleagues in explaining to the different sets of computers how to combine the information, the pupils seem to take the questionnaire in their stride.
In 2014 Dr David Regis wrote, “Ten years ago, I thought we would be doing everything online by now! But I fairly often get told something like:
"There's 2 year groups to get through, 6 classes each, so we need to book the IT suite for 12 slots, and it's already booked up, and the classes are 30 but the IT suite has just 25 computers, 24 of which work... Just send us a parcel of questionnaire booklets, and we can do it all in one morning!"
“Also, the best-laid plans of a school can oft gang agley because of power failure, sulky networks, flooding, winter vomiting, or ...”
“SHEU will continue to offer both paper questionnaires and our online survey service, and in case of events will be able to switch to the other approach for a school at short notice.”

One example of research about school-based health questionnaires:

Web-based questionnaires and paper-based questionnaires, used as part of a school-based study of young people's health-related behavior, are compared in terms of (a) completion rates and (b) data contents. It is concluded that, on the basis of the quantitative data from this survey, there is little evidence of a mode effect linked to web-based questionnaires.
Web-Based Questionnaires and the Mode Effect: An Evaluation Based on Completion Rates and Data Contents of Near-Identical Questionnaires Delivered in Different Modes. Social Science Computer Review Summer 2006 vol. 24 no. 2 246-254
http://jwolf-ra.wdfiles.com/local--files/mode-effects/WebBasesQuestandModeEffect2006.pdf
 
 

Topics: 

Comments about SHEU

Any comments on specific survey questions that may have caused difficulty?
Pupils at our primary school found the questionnaire very easy to understand and most of them completed the questions in less than 45min.

Teacher

"Thank you very much, David, for another excellent survey.  We look forward to receiving our reports."

Healthy Schools Co-ordinator

"Just to say a huge thank you for all your efforts in helping us with the Health survey amongst pupils. It has provided us with significant data which will be used across the school to help us improve. It helped us to obtain a healthy schools standard as well. I hope we can make this an annual feature as we can track the changing health of our pupils." Headteacher

Headteacher

"The Unit is to be congratulated in preparing ... material of the highest standard and worthy of wide dissemination." National Association for Environmental Education

National Association for Environmental Education

"Our use of the Health-Related Behaviour Questionnaire was commended as part of our accreditation for the National Healthy Schools Scheme." Headteacher

"The survey reports have been used to inform commissioning at specific commissioning groups. They are also being used within our Extended Schools Clusters and to inform The Annual Public Health and the Joint Strategic Needs Assessment." Programme Manager - Young People

Thank you for following up with schools the comments that caused concern. It is very sad to hear that some of our children have these thoughts and feelings and I am so grateful that you were able to make the schools aware of this so that they can attempt to offer some support.

This is another reason why I am so pleased that we have been able to work with you to offer this survey.

Health and Wellbeing Adviser

...our analyst here in Public Health- is beside himself with excitement about all the juicy data pouring in...he can't wait to get his hands on it!!!!
He is happier than I have seen him for years.

Public Health Principal

"Many thanks for all of the fantastic information that you have sent to me over the years, it has really helped me to plan relevant courses for my students to follow and to help me to focus on the needs of the students I teach."

PSHE teacher

"The children found the survey very interesting and enjoyed doing it." Class Teacher

Class teacher