One of the most commonly used question types among survey designers is the matrix question, also known as a table question. This question type allows the respondent to pick one attribute from a list of attributes that are rated using the same Likert scale.
Here’s how to use this question type to collect survey data, and, more importantly, how to report on the data once you’ve got it.
Example of a Table Survey Question
Please indicate your level of satisfaction with each of the following aspects of our service.
This type of question is popular for surveys because it is easy to design and easy for the respondent to complete, both good reasons to use it. In this article, I look at the best way to report the data obtained from this question type.
Alternatives
- Table with Counts and Percentages:(plus)Alchemer adds “Average %” at the bottom of the table.
- Individual Bar Charts: This requires five charts, one for each attribute. I have shown an example of what one would look like above.
- Stacked Bar Charts:This option essentially puts all five bar charts on the same chart. This chart was created in Microsoft Excel using the percentages Alchemer gives in the original table. It is one of the chart type options Microsoft products offer and is undoubtedly a part of other charting software as well.
- Average* Attribute Ratings Bar Chart: This chart was created by first assigning numbers to the Likert scale levels; 5=Very satisfied, 4=Satisfied, 3=Neutral, 2=Dissatisfied, 1=Very unsatisfied and then averaging all the ratings for each attribute. The five averages are then plotted using a horizontal bar chart not unlike the vertical bar chart in alternative 2.
Advantages and Disadvantages of Tables
Alternatives 1, 2 and 3 all have a key advantage over alternative 4.
From the averages in number 4, one can’t recreate the raw data as you can with the other three alternatives. In addition, averages can be difficult to interpret. The percentages in the first three alternatives have very specific meanings.
If we look at just alternatives 1, 2 and 3, alternative 3, the stacked bar charts, has some distinct advantages. It is less cumbersome than the five charts required for alternative 2 and it is easier to read than the 61 numbers in the alternative 1 table.
With alternative 3, one can make quick comparisons between the different attributes. One measure I like to use when comparing attributes is the percentage of respondents that were either “Very satisfied” or “Satisfied” (the first two sections in the stacked bar chart).
So reporting matrix data can be pretty straight forward, but the problem gets more complicated when you want to look at the same data over time.
This leads to the goal of “one number reporting”.
By “one number reporting” I mean developing a single number that best depicts, for example, your client satisfaction.
One number reporting is always desirable (for simplicity and tracking purposes) and always challenging with the tradeoffs that are required.
