You’ve identified your survey goals, written clear and concise questions, designed a lovely layout, and found a rich pool of potential respondents.
It looks like you’re all set to hit the almighty “send” button and start collecting valuable data, but have you skipped one of the most crucial steps of survey design?
Have you tested your survey?
Testing a survey involves more than a simple proofread; it’s a detailed process that can make or break your project’s results.
Follows these seven handy survey testing tips and you’ll be collecting better data from happier respondents.
Tip One: Testing Surveys With and Without Survey Logic
Logic is a great way to simplify the survey-taking process by showing people only the questions that are relevant specifically to them. You can skip questions, pages, or disqualify respondents completely based on their answers with just a few simple steps.
While it’s certainly a great way to shorten the length of your survey and decrease abandonment, logic creates a higher level of complexity and a greater chance of error. That means you’ll most likely need to test your survey with and without your logic in place.
Ideally you’ll do your initial testing prior to adding any logic. It sounds like an extraneous step, but if you find errors after your logic is in place you may have to rebuild it anyway. It’s much easier to make dependent text changes and move questions and pages around prior to adding logic.
Once your survey passes the test with no logic in place, you can implement all the logic that you need and then recreate your tests.
Already have all your logic in place? You can get around at least some of it using our “Ignore Page Logic” feature during testing. This allows you to get a more intuitive look at your survey by disabling page logic during testing.
But the best route is always to test before and after the application of logic.
Tip Two: Have Stakeholders Take your Survey
Hopefully by the time your online survey is complete your stakeholders have been along for most of the ride, helping you develop your survey goal and objectives and brainstorming the questions that you will be asking.
Now that you have designed and built your survey, it’s time to re-engage your stakeholders to make sure that the final result lines up with the survey that you all had in mind when you started the process.
Stakeholders may have forgotten some of your early planning meetings, so before sending them the survey to take as a test run, make sure to remind them of your survey goal and the actionable results that you are hoping to derive from this survey.
Caution: This is often a time when stakeholders will request to have more questions added to your survey. Be strong. Only add questions that are inline with your survey goal and will lead to actionable results!
If there are lots of questions that your stakeholders are hoping to add to your existing survey, it may be a sign that you need a follow-up project as well.
Tip Three: Have Someone Outside the Project Take your Survey
Up to this point, you and your stakeholders have been the only people that have had exposure to your survey.
Chances are you’re really close to the project, and you can probably recite the survey questions and options in your sleep. It’s time for somebody else to take a look.
There are sometimes limitations around the people with whom you can share your surveys due to security and privacy issues, but if possible try to find someone who is not familiar with your survey to review it.
It’s best to find some test respondents that have no exposure to your project at all. This will give a fresh perspective on your survey and give you a feeling for how your “real” respondents will view the survey.
Feedback from these unbiased test users can be especially helpful in deciphering confusing language or identifying questions without applicable answer option for all respondents.
Tip Four: Read Your Survey Out Loud
Most online survey tools provide you with an estimate of how long it will take to complete your survey, but keep in mind that it’s just an estimate.
If you’re going to promote that length of time that it will take to complete a survey, you should be confident in the time that you’re promising to respondents.
It takes about twice as long to read text aloud as it does to read it silently, so if you can read your entire survey out loud in seven minutes then you can be confident in telling people that they’ll be done in less than seven minutes.
Tip Five: Test in Multiple Browsers
If you know your target audience well, you may be able to predict which browser they will be using, but in reality you can’t be completely sure. It is best to test in multiple browsers to guarantee that respondents get the best survey experience and are able to navigate properly through your survey.
As of October, 2015, the most common web browsers were:
- Chrome: 66.5%
- Firefox: 20.0%
- Internet Explorer: 6.9%
- Safari: 3.8%
- Opera: 1.4%
There has also been a dramatic increase in surveys being taken in a mobile environment. So make sure to test your survey on mobile devices like smart phones and tablets.
Most modern survey themes should be mobile compatible, but even if your design fits on mobile devices you’ll need to follow some established best practices to give your respondents a great survey taking experience:
- Avoid awkward question types. These include sliders and slider lists, matrix or table-based questions, essay or long answers, a list of text boxes, custom groups, and custom Tables
- Choose fonts carefully. Minimize eye-strain by making the text as easy to read as possible.
- Use images only if necessary. These can be difficult to see on a smaller screen, and they’re very likely to obscure your question. If you can do without images, leave them off.
- Provide appropriate instructions. If your survey is easier to take in landscape mode, let people know on your welcome page. The same goes for any additional instructions that might make your survey more enjoyable for mobile respondents.
Tip Six: Test Your Reports
So, now that your survey is ready to go there is still another item to test: reports.
Sure, you know that questions that you want to ask, but you also want to know how the data will look in summary reports and exports. It is important that the data will be usable and easy to analyze.
Run some test data and see how the results will look in reports and exports.
You can tweak your questions, add validation and set up reports charts and graphs so that your data comes to you in a useable manner.
Tip Seven: Set Aside Time for Testing, Fixing and Re-testing
All this testing might sound like it will take quite a bit of time. Well, it does take time, but be prepared to do it! It is important to set aside time in your market research plan for all types of testing.
When we scope a project we often leave time for building our survey, and having our stakeholders review it, but we forget to build in time for testing.
Often, when a test is conducted, there will be changes to make in your project. Make those changes, but be sure to test the project again, even if it was a small change.
Small changes to an online survey can make for cascading problems across your project. This may seem obvious if you are making changes that might affect logic and page order, but you should re-test even if you are changing wording to a questions.
Even one word on one question can create confusion or inconsistencies in your survey.
Bonus Tip: Send to a Sample of Real Respondents and Review
You’re ready to go, but all of your testing was done in a testing environment.
If you still have time in your market research project schedule, get some real, live responses and make sure that there are no snags in your survey. Double check to make sure that the reports look the way that you expect them to.
To do this, you can send your survey to a sub-set of your sample; send to approximately 10 percent of your total sample.
If there are no issues with your survey, you can use the responses from this sub-set. If there are issues, you still may be able to use some of the partial data that you collected.
Test, Test, and Then Test Some More
So, if we were not clear enough, we recommend that you test your surveys, test again and then test some more.
Spending the time and the effort to test your survey will save you the anguish of having to deal with the issues that come while your survey is in the field.
It will also leave you the time to rejoice in the praise, appreciation and gratitude of a job well done!
