It seems everyone is “familiar” with NPS.
Many utilize this metric to continuously measure customer loyalty. Next, analytics is incorporated to correlate those independent drivers that move the needle to better understand the ups and the downs of loyalty. Then, internal changes are implemented to further enhance the customer experience…which increases customer loyalty.
Makes sense, right???
Well…There is a current beneath the surface of this mode of data collection. These days – most data collection is online…and there are many great reasons for this (don’t get me started). That said, this does create some bias and doesn’t always tell the whole story.
Our founder, Dave Sparks, always said – individuals who complete mail surveys (not entirely the same as online, but these days it kind of is) are older…and madder.
As we looked at four years of actual real consumer monthly satisfaction and loyalty tracking survey data, we dove deeper to see if this held true by looking at various loyalty, satisfaction, and ease questions by survey methodology: Telephone vs. Online.
Well, guess what we uncovered…
NPS, Ease of Working with the brand and Overall Satisfaction of the brand Top Two Box (9’s and 10’s on a 10-point scale) scores are significantly higher for telephone when compared to online….to the tune of roughly 12% to 14% higher at the aggregate level. THAT is a lot and VERY important.
We also found…
Significantly higher proportions of respondents 55 years old and older complete online surveys when compared to those under 55 years old.
So, why is that? Before we answer– I’ll bet you have received an online survey for a car rental, or a hotel stay and I’m sure you received that survey in the form of an email asking for your opinions. If all was “fine” with your recent interaction…did you complete the survey??? I’m guessing probably not, because…well it is work per se.
Now imagine if you received this email survey request when your hotel room had no hot water or smelled like something very unfriendly. This experience could leave you feeling angry/grumpy because I know it would do that to me. So now you get a chance to give them this feedback…angry feedback.
“The consideration of how you collect your data is just as important as the metrics that you are measuring and analyzing.”
As researchers we find conducting online surveys fairly easy, fast, and inexpensive. But for respondents – completing a survey is work and can be time consuming. Telephone surveys on the other hand almost fall in your lap. You don’t have to do anything – you listen and answer questions.
Side Note: Telephone surveys will ALWAYS capture richer verbatim feedback. Typing a long response online about why you rated your satisfaction as a 7 usually results in a “It was good” type of response.
Now this is where we believe the mode of data collection is so important because, again, we have seen it happen with actual real data. If you don’t include telephone surveying as part of your data collection methodology, you aren’t receiving a true representation of customer feedback. Now, let that sink in for a moment.
If you are tracking customer experience metrics with online surveys only, I’d invite you to test to see if this holds true. Perhaps blend online and telephone surveys (what we call Mixed-Mode) for a month or two and analyze the results. You may be very happy you did. Remember, what we found is from several years’ worth of data!
The consideration of how you collect your data is just as important as the metrics that you are measuring and analyzing. If given an opportunity to propose a customer experience program for any client, I’m always going to strongly recommend a mixed-mode (telephone AND online) survey methodology if it is feasible. Don’t forget – based on our findings, your customer sentiment may actually be higher than you think! Now, let that sink in for [more] than a moment.
