Gareth Hodgson, Director, had some interesting food for thought this morning scrolling hisTwitter feed.
Moneysavingexpert.com’s Martin Lewis had posed a question to his followers regarding how strict they are when using a five-point scale…
I don’t follow him on Twitter, but the reason it came to my attention was that some people I follow do, and they were responding to this.
One said…
“If it’s for a service provided by a company or worker, I will generally just give full marks as that stuff affects their job.”
Another replied to him saying…
“This”
And another replied saying…
“I am the same. I think for service jobs I’ll tend to give full marks unless they have been really bad. If I know it affects someone’s work, I don’t want to see someone getting done for it.”
I resisted the urge to jump into a 280 character debate and instead reflected on what I was reading. Martin Lewis was essentially just asking how good something needs to be in order to warrant a “5 star” rating. The comments however took it in another interesting direction and it immediately got me thinking. Is this something we think enough about as researchers? The broader consciousness of the mind-set of the respondent, and the influences this has on the rating they ultimately give? If a 5 star rating is “it was fine, but I don’t want anybody to get in trouble or lose their job” then how much stock should we be placing in that 5 star anyway? How prevalent is this mind-set? It’s certainly something we should be considering and factoring in from an insight perspective.
The actual response to the poll was interesting as well. Not only for the 6,000+ respondents in half an hour (Note to self… need to work on getting Mustard more Twitters followers…), but for the breakdown of responses.
A quarter of people indicate that they go “full 5” in instances where there are ‘”no problems”. So, for one in four people visiting your store, theme park, restaurant… no need to be exceptional, just get the basics right, don’t cock anything up and the “customer” will give you the maximum score available. If you happen to “be nice” as well, then two thirds will hit that top box.
A quarter are more discerning, and will only give the top rating infrequently when the service has been exceptional.
Notably however LESS THAN 10% will give a perfect score when they have experienced perfection.
So this means that the highest possible rating on a scale is 8 TIMES more likely to be given for “fine/acceptable” than it is for “perfect”. Wow. Maybe we shouldn’t be using numerical scales after all? Without the understanding of the mind-set behind the rating, then surely the number is meaningless (or at the very least, carries less meaning). And maybe businesses shouldn’t be spending millions going above and beyond?
Questionnaires are often designed with a definition at the top and bottom end of the scale but the numbers in between are left to interpretation.
There’s a strong argument that we should just be using definitions to enable respondents to more effectively articulate their experience instead (and help us understand the mind-set more)… “It was absolutely perfect”… “It was absolutely fine and I didn’t need or want any more than this”… however if people are just going to say that it was perfect because they don’t want anybody to get into trouble or impact their jobs, would this even work anyway?
And don’t get me started on how different people interpret the word “satisfied”.
Right, that’s enough entering the matrix for one day, I’ll get back to charting this tracker…
