NPS Blog
Survey sabotage: Is your customer data misleading you?
This article originally appeared on Forbes.com.
In this age of Yelp, Facebook and Twitter, consumer opinions have never been so powerful. While it’s easier than ever to find reviews and ratings, companies still grapple with how to best gauge their relationships with their clients.
Many top companies are using or have adopted a Net Promoter® system to learn more about the quality of their relationships. A closed-loop feedback process can help businesses identify key clients, product features customers like or dislike, service opportunities and problems that need immediate attention. But these efforts can only go so far if the survey data is flawed by biased answers.
Like a virus that yields no obvious symptoms, bias can creep into survey data all sorts of ways. Here are some of the most common factors that can contaminate your data, and how to avoid them:
Risk: Fear and embarrassment. Consumers may avoid giving negative scores out of embarrassment, particularly when surveyed face to face or on the phone. In business-to-business (B2B) situations, smaller customers may avoid negative reviews when surveyed by a large or critically important vendor, for fear of being cut off.
Prevention: To coax out candid feedback, leading NPS practitioners find ways to offer each customer appropriate levels of confidentiality. In B2B settings, for instance, companies can maintain transparency by reporting average scores for an account while keeping individual ratings confidential.
Risk: Employees who try to game the system. Sales and service representatives at auto dealerships are notorious for begging customers for top survey scores. Employees of other companies may engage in shenanigans, such as altering the phone numbers of unsatisfied customers so surveyors can’t reach them.
Prevention: Companies with robust Net Promoter systems pay close attention to potential gaming of this sort. At the car rental firm Enterprise, which uses a customer loyalty system called ESQi, gaming the system is called “speeding” and is grounds for dismissal.
Risk: Responder bias. This occurs when the individuals who respond to a survey offer a different view than the overall population being surveyed. Imagine that a company administers a survey with a 20% response rate. It indicates that it’s Net Promoter score is 50% (60% promoters minus 10% detractors). Now imagine that the firm studies the behavior of nonresponders by looking at whether these customers made repeat purchases or spent more over time. The additional data determines that the mix of that group is 10% promoter, 40% passive and 50% detractor. That means 80% of customers who ignored the survey had an NPS of negative 40%. So the true NPS for the company—the weighted average of the two groups—is negative 22%, not 50%. The company’s relationship with its customers isn’t as rosy as it seemed.
Prevention: If you can’t study nonresponders, consider scoring all of them as detractors (probably not too far off in business-to-business settings) or as a 50-50 mix of passives and detractors (a reasonable estimate for many consumer businesses). Naturally, a high response rate reduces the risk of responder bias.
There is only one sure way to check whether your system has effectively defused the land mines of low response rates and bias: You must regularly validate the link between individual customers’ scores and those customers’ behavior over time.
Does your data match what you see at your business? If not, bias may to be blame.
-
Recent Posts
- Podcast: The Customer Experience-Based Organization, with USAA’s Julio Estevez-Breton
- Podcast: The Business Lessons of Rejection, with FirstService’s Charlie Chase
- Podcast: Building Temporary Relationships That Last, with Elwood Staffing’s Fernando Cadena
- Podcast: Fostering a Feedback Culture, with Year Up’s Garrett Warfield and Jess Britt
- Podcast: How to Lead with Grit, with Angela Duckworth
Search the Loyalty Blog
Receive new posts by email
Contributors
Categories
Archives
- July 2018
- June 2018
- May 2018
- April 2018
- March 2018
- February 2018
- January 2018
- December 2017
- November 2017
- October 2017
- September 2017
- August 2017
- July 2017
- June 2017
- May 2017
- April 2017
- March 2017
- February 2017
- January 2017
- December 2016
- November 2016
- October 2016
- September 2016
- August 2016
- July 2016
- June 2016
- May 2016
- April 2016
- December 2015
- November 2015
- October 2015
- September 2015
- August 2015
- July 2015
- June 2015
- May 2015
- April 2015
- March 2015
- February 2015
- January 2015
- December 2014
- November 2014
- October 2014
- September 2014
- August 2014
- July 2014
- June 2014
- May 2014
- April 2014
- March 2014
- February 2014
- January 2014
- December 2013
- November 2013
- October 2013
- September 2013
- August 2013
- July 2013
- June 2013
- April 2013
- March 2013
- February 2013
- January 2013
- December 2012
- November 2012
- October 2012
- September 2012
- July 2012
- June 2012
- March 2012
- February 2012
- January 2012
- November 2011
- July 2011
