Is your survey a customer listening vehicle? Or could it be improved? On this episode of the ‘Amas Talks’ podcast, Amas and Martha agreed that if you’re considering how to improve surveys, a survey audit is the first step toward excellent survey design. The audit ensures your questions are sufficiently powered to achieve robust insights—and without a doubt, you need smart insights to steer your company!
How to Improve Surveys, Summarized:
Martha and Amas discussed the problems with how companies use surveys, particularly Net Promoter Score (NPS), and offered advice on how to improve surveys to gather actionable customer feedback.
Key takeaways:
- NPS has become a gamed system: Companies often prioritize the score itself over genuine customer feedback, leading to data that’s not meaningful.
- Gather real customer insights: Improve surveys by using a combination of well-crafted surveys, in-depth interviews, and careful analysis of customer behavior.
- Make sure your surveys are giving you meaningful data: Audit your existing surveys for any biases, tailor questions to specific interactions, and use a mix of quantitative and qualitative data.
- Poor feedback leads to bad decisions: Companies risk making poor decisions based on flawed survey methods.
- Go beyond NPS: Gather intelligence, embrace the scientific method, and value expertise over opinions.
Martha encourages companies to move away from simply chasing scores and instead prioritize truly listening to and understanding their customers. But here’s some of the pros and cons of NPS that Martha and Amas discussed.
How to Improve Surveys: The Full Conversation
Here’s an AI-based transcript from Martha and Amas’s discussion about how to improve surveys.
Amas: You’ve been around this space forever. You lead one of those firms that I really admire because of some of the research you do. But let’s jump right in. I have a burning topic I want to chat with you about.
Um, so I stayed in Seattle a few weeks ago, and before I got in, I got the text from the hotel that says, hey, we see you’re coming. Is there anything you need? I’m like, oh, how nice this is. This is wonderful. I responded, I’m fine. I get in, it’s an uneventful stay. There’s a few things here and there, but I quickly realized what this text was all about.
A day before I left, the text was, hey, you’re checking out tomorrow? We want five stars. Is there any reason why you will not give us five stars? If so, speak now so we can fix it. Because by gosh, we are going to give you five stars. I didn’t respond the day of my checkout. Hey, I’m just checking in, making sure you’re going to give us. I’m like, what?
So I guess my first question out of the gate is should we bother as consumers with surveys because it’s getting so obvious what they’re doing here?
Martha: Okay, I think in most cases, as a consumer you shouldn’t because I think it’s just a completely gamed system. Yeah, I think when it’s B2B and there are some exceptions to that where consumer kind of overlaps B2B, but it’s I think the criteria set is are they truly a partner to you.
That’s first like are they truly I don’t mean enterprise rental car. I don’t think Hilton Hotel or I don’t think they’re you consider them a partner. You consider when you get the lowest rate. That’s the one I’m going with. Right. Right, right. Okay. So if they’re truly a partner where it’s a collaboration where you feel like you’re affecting their decisions in the same way they’re affecting your decisions okay. That’s criteria one. Criterion two would be that you feel like they’re truly listening.
So the question isn’t like, when will you give us five stars? Right? Right. It’s not that. It’s we’re here to listen. We want to know what you really think, right? And whether bad or good, all thoughts and opinions and perceptions matter. Right? So they have to, you know, really communicate that they’re truly thinking.
And three which kind of overlaps with the really thinking that they that they’re valuing you. Like, I just wrote something on LinkedIn about, you know, truly people consider incentives. You know, I just got paid $50 to do a survey, and I was I was happy to do it. You know, I was like, true, they were they were truly listening with me and they were signifying, we’ve we value you, you know, $50.
You know, it’s fine. Like I scan it in. I have $50 of buys in Portland that’s, you know, two glasses of wine. You know, it’s like, okay, that’s something it says. So I think and with many of our clients, we do Starbucks gift cards, you know, a latte on us. You know, something, you know, just to say, hey, we appreciate you as a customer. So and in B2B, come on, these relationships can be worth hundreds of thousands, if not millions of dollars. So come on, a latte or two on us like that should not be what breaks the bank.
So I think that would be that would be the set that they’re caring, they’re listening. And it’s a true partnership. And if all of that criteria set is met I think absolutely you should you should feedback or if at least two of those um, those that criteria set is met, you know, consider it. But you know, your enterprise your your Hilton I think it’s just American Airlines. Yeah. Oh, don’t get me started on them. Um, but really, I think it’s it’s BS.
Amas: Yeah. So. Well, let me ask you personally, do you fill out surveys?
Martha: Mhm. I did for American Airlines because I just couldn’t get them to respond. And so I wanted I like consumer affairs. Consumer affairs you know please. Like you delayed me for four days. Come on. Please respond. Give me some kind of compensation for this. That’s you know, I had a rental car and hotels and, you know, the whole thing. A delay of four days adds up. I mean, for me, it was just shy of two, $2,000. Um, and so it’s like, please. And then new, new, uh, tickets and Alaska Airlines just to get finally home in any event. Um, yeah.
They sent me their survey twice. Both times in that Net Promoter score zero with. Here’s why I’m giving you a zero. It still didn’t affect anything. I still didn’t get a response. But yeah, I in desperate times I’ve tried it. I’ve never actually seen it work, but it is sort of like, okay, is there any way you’ll listen to me? Anybody? Is there, is there anybody home?
Amas: Like it’s crazy. I think we just on in the last 30s came to the conclusion that surveys is just another channel. It’s an escalation channel. Right?
Martha: Right, yeah.
Amas: You’re not answering me and all of these other places. Maybe if I put it in survey, I would.
Martha: Maybe.
Amas: Maybe. Yeah. You got an answer? So not to pick on our friends at American Airlines. But I, too, have an American Airlines story, and I want to. So they canceled my flight. I now live in, you know, flyover state in Oklahoma City. So they take me to Dallas a lot. Right. So cancel my flight in Dallas. I’ve made this drive dozens of times. It’s about 3.5 hours. I get a rental because I’m just trying to get home.
So I drive home the next day. Get the survey. Amas, how did you enjoy the flight? You know, Mrs. Lincoln, how did you enjoy the flight? Right. One of those kind of. And then it goes on to ask me, how was the meal? Uh, how was the flight? It did all of that. And so I am struck by the fact that there was very little on that survey that they didn’t already know or should know. Right. It got me thinking when I was planning on talking to you today to go all right, what was the point of surveys when we started doing this thing at scale? What was the original intent?
Martha: Yeah. Yeah. Well, I mean, originally. So, you know, back in the 90s, I would say I think there really was an interest in listening and feedback. Mhm. Um, I think that it’s just become a very gamed system. I mean, just to kind of reflect on your point.
Um, companies should never bother you with information they already know. Like, never. Like time is precious. Like if their time is precious, guess what? Their customer’s time is precious, right? Especially business travelers. Time is precious. So don’t bother me with, you know, like, which flight were you on? They know, um, you know, which, you know, class were you in or, like, petty stuff about the meal? Like, they kind of. They should have a better way to QA the quality of first class meals, right?
Then, like, then asking you. Right. Right. So there are things that they should be able to QA, um, that have and things that they should just know in their CRM system that there’s no reason to bother you with. Um, but I, you know, originally pre 2003. Net promoter score taking off. I think there really was like an interest in market research, customer feedback. Um, it’s just gotten, you know, kind of bastardized is what I would say.
Amas: So so I want to I want to get into this whole NPS business. You’ve mentioned it a couple of times, 2003. So. 2003. Happens the famous HBR article. Right. You know, all of this deal and then we get NPS. I think Bain or one of them kind of made it into a thing. Tell me, and I’ll also, as one who lived through it, this was also the internet now had a foothold in society, right? Emails were now like mainstream. So paint the picture for me. 2003. Um, how did that moment that leads up, up to now change the way companies thought about service?
Martha: Right. So I think there’s a confluence of factors. First, that article in HBR by Fred Reichheld, it made a lot of sense. Like it was good. When I read it, I was like, ding ding, ding. That’s good. Because instead of hitting the nail on the head with, you know, rate this company or how satisfied were you? It’s like. Huh? Coming in on the side. Yeah. Would you recommend how likely are you to recommend? I was like, huh? At least an interesting kind of an interesting new question, right? Um, so I think there was a, like, an excitement about, like an interesting new question then, like you said, emails. Right. They become really cheap. Yeah. So it’s like, okay, asking this question by email, right. Costs us pennies on the survey. Right? Like not even penny like fractions of pennies on the survey.
Basically free. Yeah. Okay. So we have internet email. It’s all like exploding and free and memory is basically free. It’s all it’s free. And the question seems kind of interesting and new. And there is this kind of talk about, um, you know, we’re post Ralph Nader. Right? So there is this kind of like, huh? Could we learn from customers? Like, is there a possibility we could learn from customers that customer feedback is valuable? Because Ralph Nader had been, you know, hammering away on that through, I don’t know, the auto industry, the 70s, 80s, you know, and really for beginners.
From the beginning of this society of consumer affairs professionals, you know. So I think it’s like those factors where it’s like, huh, maybe we can learn from our customers. And now, you know, interesting people like Steve Jobs would kind of push back on that, right? He’d say, no, actually, we set the bar for the customer. Right. Which I think, you know, there’s some idea, good ideas there. I think Hertz set the bar. It’s not that. It’s they examined customer behavior. I don’t think this is a survey question and said, what? What are people like you and me care about. When we get to the rental car station. You know, we like the Martha Brooks car is in slot 91.
You know, we like that little personalization, and we want to get out quickly. Oh, the keys are right there in the car. I go, this is Martha Brooks, you know, slot 91. I’m gold. I’m acknowledged, I’m personalized. And I get out of the airport quickly. Like that’s what I care about. And that’s not something you surveyed about. That’s like examining customer behavior and saying, oh, let’s craft a customer experience around what the customer cares about.
I mean, you know, going back to the iPods, it seems like so long ago, but, you know, pre iPad, you know, it was like, well, what a what a customers care about. Like a cool interface where they can carry 25,000 songs with them anywhere. You know, it wasn’t like a customer who came up with that idea. So this is I get that I’m kind of going here and there with an answer to this question, but I think there is, um, you know, coming back to NPS and the pros and cons of it, there is this like, well, we could learn from customers. And yet there’s also a, you know, a path of but what exactly do we want to learn from our customers that’s worth thinking about.
Amas: Yeah. And I think I want to come back there, but I want to go to this point of purpose. Mhm. So purpose is I want research, I want insights. Um NPS came and I feel like the purpose is now changed. I, I feel like not, not just I feel like it’s clear to me now the purpose of NPS is NPS. Am I, am I off base on that? Like it seems like the score of it has become the idol now.
Martha: I think that’s like 98% of consumer companies. I mean, think about podcasts you listen to. It’s like, if you can give us five stars, give us, you know, by all means, rate us five stars. But it’s always prefaced like that. Just like with your hotel experience, when you’re ready to give us five stars, you know we’re ready to interact. But until that time, we’re we’re really not interested.
I mean, car servicing companies are famous for this, you know, just like, give us five stars, give us five, you know, pleading, begging. You know, it’s really. And you know what? It’s just annoying. Yeah, it’s just annoying. It’s not like we can smell like that’s not real. Like that’s not real. That’s not any kind of. And that’s not market research or customer listening. That’s just, you know, worried about employees feeling like, if I don’t get five stars, I’m going to get fired, which is a really that’s a bad system, right? That’s not fair.
Amas: I mean, is it is it Warren Buffett who said, like, show me the incentives and I’ll show you like, what the company’s goals are, right? I mean, I’m speaking as someone who, you know, when I ran, I was in charge of CX for, you know, big consumer goods company, the wonderful company. Um, I got a bonus from NPS, so guess what? You know, there were entire company organizations. So the the C-suite, they were entire programs.
Yeah, we wanted to improve the experience. But if the experience improved and NPS didn’t improve, there would be hell to pay. So very, very quickly it became about the NPS score and if tangentially, the experience improves. I mean, we hired outside parties and everyone does it. Like how do we improve, um, customer experience. So would you what would you say about this incentives of where we’ve gotten ourselves into not just about NPS, but as long as you have a culture that’s about the score in and of itself is a good financial thing. What do you tell companies who are contemplating this issue?
Martha: Yeah, fire the culture. You need a new culture. Yeah. Like that’s not that’s not good. Like I think there’s this and there’s this. Um, let’s just offload this to the customer. Well, what about Xiang? What about watching? What about listening to how the customer interacts with the customer and making decisions based on your brand promise and what you allege your company is about? After all, your customer doesn’t know.
Like maybe, maybe your brand promises about speed. So are you answering these questions quickly and efficiently? Maybe your company is about education. We educate the customer. Well, forget this then you should be the. The customer isn’t studying your brand promise. I promise you that right now. Um, so then then you’re listening for. And that can be through a I, but also through, you know, real research both, um, listening for like, are they informing the customer and proactively and giving ideas, you know, if that’s what your culture, if that’s what your brand promise is, is like, oh, we’re all about education and blah, blah, blah. Well, then listen for that. Um, so, uh, yeah, I don’t know if that answered your question head on or indirectly it does.
Amas: Um, I want to I want to go back for a second and go back to what you and your team does. How did you end up in this space of, um, you know, looking deeply into how did you how did you become this sort of expert around this topic?
Martha: Sure. So I work for two coms, you know, back in the day, both.
Amas: I can’t believe .com is back in the day now. But go on.
Martha: So food.com and then Lucy.com. Yeah. Um, so, uh, and then Lucy transitioned into work for Nike and Adidas, and it was actually I worked for a guy named Sam Braddock at Nike who was really cool. And he said, Martha, I really care about evaluating our customer service, our email, our chat, our calls. And honestly, I don’t care what our score is, as long as I have a hockey stick that I can prove to, our C-suite has shown that we’ve moved up. Right. Okay.
So it was kind of like, yes, measurement of aspects of the customer experience. In this case, it was customer service. And of course, customer experience is a lot more than customer service. But customer experience is everything. It’s the invoicing. It’s the pricing. It’s the supply chain. It’s everything. But this was specifically about customer service. And, you know, being able to have real metrics on that. And so that became and then Nike was a very early client that became metrics, Interaction Metrics.
So measuring the quality of interactions with real metrics. Um, and in the very beginning of the company, we were just about customer service evaluation. And then our clients pushed us into like, we’re really interested in beyond customer service, customer satisfaction.
And so we got into interviews. Interviews are still like a fantastic research tool that we love to use. They’re quite expensive, right? And so that transition transitioned of course into surveys. And so now those really are our methods, our workshops, customer service evaluations. And then, you know, genuine customer listening interviews and mostly surveys because they’re less expensive. And but you know, the way we do them is none of this BS about like give us a five stars and we’ll, you know, blah blah blah. You know, it’s.
Amas: Like we’ll bribe you for it.
Martha: Yeah, yeah.
Amas: I am fascinated to see this journey we’ve been on where we started at one place, and now we’re bribing people into giving us five stars. Um, but to get back to NPS Net Promoter Score, for anyone listening who doesn’t know what NPS is, this is the old famous questions, what have you. But I’ll. I won’t butcher it. I’ll let you. Um, what are your thoughts on the question to begin with? The one that says so the again, it is how likely are you to recommend XYZ to your friends and family? I believe some version of that, or.
Martha: Friends and colleagues sometimes if it’s a business.
Amas: Yeah, exactly. Yeah. What are your thoughts on it?
Martha: Well, like I say, back 20 years ago, I was like, oh, interesting. Um, and then it became clear, you know what? It’s a leading question.
Amas: What do you mean by that?
Martha: How likely are you to recommend assumes you’re somewhat likely to recommend. Right. So it’s like huh? Now, what I will say about NPS is if you want to benchmark. And so we and many of our, if not most of our clients want some version of NPS, right? Um, you know, the pro about it is that you can benchmark against other companies in your sector, right? Right. So I like that about it. Um, sometimes I think it requires some tweaks, right. Um, you know, the empirical question is, have you recommended yes or no.
Amas: How likely I never yeah. I hadn’t thought about that until you mentioned it. But but I also want to point out for me what rubbed me wrong about NPS. The question is I think the bar is way too high, especially because they now put these questions on service interactions. So you’re saying I am going to recommend your company to my family because I spoke with, uh, Joe on the one 800 number.
Martha: No, like, that’s just not like. It’s just not how humans work, right? Like, you didn’t have a phone call or you didn’t receive a package that was packaged correctly, and now you’re like, hey, Bob, gotta buy from this company. You know, like, no, it’s like it’s really off kilter and you want good listening. Has is in concert with how humans actually, you know, conduct themselves. Right. So it’s. Yeah. But you know, I don’t think Reichheld originally intended it. It was like at the end of your interactions, right about the entire company. Like, how likely are you to recommend ABC to a colleague? So it was at the end, it just became again butchered and bastardized. So it’s like, you know, you got a package at your doorstep. How are you going to recommend, you know, call center or how likely are you to recommend every, you know, every five minutes?
Amas: Right. Like every five minutes I’m recommending. I’m recommending. Um, here’s here’s a thought that I want to get your, your kind of perspective on. I think because I was kind of living it in the moment and that. 0507i think when NPS first came, I ran most of my background is in service. We didn’t even get to ask the question. We still had our old school customer satisfaction, what have you.
But what would happen would be there was this sort of usually lived in marketing or what have you consumer analysis that was doing NPS and we’d say our score is X, Y and Z. Well, inevitably they would say, well, amass you’re the problem. Well, you’re the VP of whatever. So customer service needs to. And then I remember we went from that to well, let’s just ask the same question across the board because they were struggling with connecting my. Csat scores. That was going up and down. There was just no correlation whatsoever.
So the thinking then became, well, let’s bring that in. And from day one, I just want to tell you, as someone who led contact centers, it was always war because I would go to that representative and my supervisors and managers will sit next to this poor person and say, hey, listen, um, your NPS score was this. And they would go, no, I had a great interaction. The QA scores were fantastic. This customer has no issue with me. They were not recommending the company.
Because of that. And so it never got any buy in because to your point, I think the real thing that sunk NPS. And I want to get to what you think we do about MPs going forward. Here is it became too popular for its own good, right? Every department, it became just bastardized. So I guess I will ask you a question of as you work with companies who are trying to just understand the genuine ones who are like, I need to connect. I want the MPs to benchmark, but I need to be able to measure these interactions along the way. What do you say to those guys?
Martha: Sure. I think as you know, as long as it’s the end of interactions and you’re very clear that this is at the end of the journey, and now it’s like, how likely are you to recommend I’m good with that.
Now, along the way, you want to know how did that touch point go? Yes. Um, please rate your experience with James Brown. Yeah. Martha Brooke, please rate your experience with, based on his or her politeness. Based on her his or her knowledge. Like, there are some very specific questions simply about that interaction and being very clear.
Like, was Martha polite and courteous and courteous? Did Martha know what she was talking about? And keep it on that so that you can actually train to that data? Does that make sense? Like train to that because maybe Martha was very courteous, but she didn’t have any answers. Right? Well, okay. Great. Now we need to train. Martha’s good on the polite, but let’s do an, you know, a good training class to give her some more, you know, knowledge about the product or service. Um, and so, you know, the takeaway here is tailor your questions to the interaction and don’t be sloppy and just have one question, does it? This is crucial to how to improve surveys and ensure you’re getting useful data.
Amas: I agree you can’t, you can’t MPs everything and we’re going to move on from MPs. But I do have one final question for you around this topic. The long knives are out for MPs and I, I, I am I am wielding one of the knives. Um, where do you stand on. We now have this then it’s ubiquitous. It’s the standard. It has flaws. You just pointed out a new one. It’s like I hadn’t heard your take on it before. The person, the people who came up with the score are writing mea culpa letters on. Hey, we screwed up, what have you. What are you saying to folks who just genuinely ask you what to do with NPS?
Martha: Yeah. Um, I would say, you know, make sure you’re benchmarking it so that there’s a true point. Yeah. Right. That you’re not just assaulting your customers with something that they’ve heard over and over again. Um, and really study the journey and make sure you’re asking NPS at the right time, in the right place, which again is at the end of a journey, and consider how you might want to tweak it. Like so often, it’s just how long do you recommend friends and family?
Like, um, say you’re a B2B company working with distributors? Well, a distributor shouldn’t be recommending you to others, right? Because it’s their it’s their job to sell you. Right? So of course they’re going to recommend you. Do you know what I mean? It just doesn’t make sense. So in you know, in that case you’re going to need to tweak the question. So just be very selective about your audience as, as much as you’re selective about, you know, the touch point itself. Does that make sense?
Amas: So it does. I think that nugget about at the end of the journey, not every touch point, I think I think these are way to reform it because to your point, can’t throw the baby away with the bathwater. We need to come up with a better, um.
Martha: That is a pretty ugly.
Amas: Like, who was doing this, by the way? Was that a thing in the 1600s? Um, but, you know, I think you’re giving folks some really important tips. Um, so I’ve got Martha Brook on the podcast. I’ve been looking forward to having this conversation.
I’m going to ask you a bit of a fun question here, uh, to get people to get to know you. So, um, I’m going to ask you what was a purchase under $50 that has impacted your life the most? And while you’re thinking about it, I’ll give you mine. Mine is the neti pot. I you know, I, I can have bad allergies. And when things get really, really bad, I mean, I’m not graceful with the thing. I end up like I’m waterboarding myself, but my gosh, does that thing work? It works like it clears the whole thing, and I can I get some relief. What would be. What would be yours?
Martha: Yeah. One thing I just want to kind of reflect on the question and say, you know, sometimes it’s the small splurges in life that really make a difference. Like, of course we love the big fancy resort vacations. You know, that’s terrific. But sometimes it’s just little stuff.
So I took up tennis a year ago, a little over a year ago, and recently bought slow balls. Not the slowest balls, but slower than the green balls. And it’s just a really it makes it really in terms of getting into the technique, you know, because it’s all about technique, right? Um, just it helps really slowing down the ball. They’re called orange balls. It really helps you get into the right form. Right. So I like that one. Um, kind of in the neti pot world of things, I have this thing I think it’s called like a back arch. Huh? Anyway, you lay on it and it like, like goes into your entire spine and kind of curves it. Maybe it’s called a back curve or. I don’t know, it’s really it kind of opens up the chest. It relaxes the spine. I don’t know, maybe it was $12 on Amazon or something. It’s like, oh, I kind of like this, you know, for a minute. I don’t want to spend my life on this thing. Um, but it’s it’s kind of neat. So those are, those are some a couple things.
Amas: Yeah. That’s cool. Um, so I do want to get back to we now live in a world. And again, correct me if I am exaggerating here. I think that many organizations, particularly consumer facing organizations, are just not gathering feedback the right way, feedback specifically from consumers. What are the real world ramifications of us continuing? Most companies continue to live their lives without gathering real good consumer feedback.
Martha: Oh, that I love that question. That they lead their company astray. So you think that everything’s great because you’ve begged and pleaded and coerced or downright coerced for those five stars? And so you’re like, hey, we’re good. Things are awesome. And yet, meanwhile, you’re, you know, you could be losing market share. Your revenue could be crumbling like any number of things. And so you’re sitting in this self-satisfied seat and meanwhile, things are going.
I mean, coming back to our American Airlines, you know, it’s like, well, you think things are good, but maybe they’re really not. And if you listened to those conversations or read the emails pleading for please respond to me, you’d you might say, you know what, we need to change this, that and the other thing so that we don’t lose market share so that we protect our revenue over the long haul. You know, beyond the quarter. So I think that that, you know, the old expression in the 70s is garbage in, garbage out. So you bring in garbage data. You’re not going to make good, smart, strategic business decisions. And hey, guess what? You and I are both in that space of helping companies improve strategically, right?
Amas: I couldn’t agree with you more. I think from the fall in survey rates, the sample size is already problematic. The way you’re asking the questions is leading. As we learned today. There’s all of this. And honestly I’ve when you said about doing other kinds of things like live interviews and what have you, at the end of the day, if it’s information, you want data intelligence to make a decision and you’re not a slave to the score. I think it’s very, very much worth it to at least find some other ways to get the pulse of your true customers. Um, given that the whole survey complex has been a bit bastardized.
Martha: Yeah, yeah, the military industrial survey complex. Right. Um, you know, you you just used a great word, and that’s intelligence. Yeah. Are you gathering intelligence or are you chasing a score? That’s right. And if you’re chasing a score, you know, and maybe you need to know. Right. Come on. We work with a lot of clients. Maybe you need to chase the score. That’s fine. Right. Don’t confuse it with intelligence. Like, have your team that chases a score. Have at it. But let’s also consider, you know, another aspect of your company which is gathering really good intelligence so that you can, you can grow so that you can steer your company correctly. Right.
Amas: What is surprised you the most? I have just a few a few more questions for you. What surprised you the most in your research lately. Let me let me kind of give you a little hint. I’m there’s lots of breaks in the data. Mhm. Um, like a CSI American customer satisfaction index was on the uptrend. And then we’ve done this reversal and we’re back to where we were in 2003. It’s more it’s like something is breaking in in my world of service and overall in the world of customer experience. And I can’t put my hands on it. So as someone who is looking at data all the time, if I were to ask you, what have you seen where you’re like, this is a dramatic or significant change over the last few years, what would you pinpoint?
Martha: Yeah, I would look at our culture, which is not valuing science and expertise. And I mean, like, I don’t want a car salesman to do my heart. God forbid, God forbid, I need heart surgery. But, you know, like, there’s you want the excellent cardiac surgeon to do your your heart surgery and you should be valuing their opinion. And maybe that seems like way out of line, but I just feel like with social media sometimes there’s this devaluation of the experts and everybody can weigh in on whether, you know, vaccines are good or bad or actually there are medical professionals who have a more informed opinion than I do about vaccinations. And I think that this kind of devaluing of expertise and science, both.
And they’re a little bit of a different lens, but both of that devaluing has kind of devalued how businesses operate. Yeah. And that’s not a good thing. And so coming back to like what are good you know, what are good values that you want to teach your children. Uh, what are good values for companies to have. And I think it’s like going back to the ABCs of, of not everybody’s opinion is equal. It should be equally weighted, like experts should get more value with regard to their expertise and science.
The scientific method. After all, it goes back to the Renaissance. It goes back hundreds of years. And back then, you know, it was considered a really excellent, like way to conduct oneself in the world. Right? And it’s worked for hundreds of years. And we should continue to embrace the scientific method where you come up with a hypothesis, but then you’re willing to deny that you ran good experiments that mean you measured the results and you learned, you know what, that wasn’t such a good hypothesis or that was a great hypothesis.
So, um, so anyway, getting beyond, you know, the medieval world was all about beliefs, right? And gods and all of that. And then Francis Bacon came along, you know, and Galileo and there’s this whole, you know, complex around building out a new way of thinking, the scientific method. And it’s a really good method that really I don’t think can be beaten.
Amas: I agree. We’ve tried we’ve tried other things. And so, you know, this happy go lucky, let me use my emotions to get you to give me five stars. Let me bribe you. Let me do all of this. That’s not you’re now. It’s witchcraft again. You’re just sort of. You’re no longer. You have to be willing to have bad answers if you want the truth, right. Or unpleasant answers. So you cannot live in a world where a customer I do not feel comfortable being.
I don’t hardly ever feel surveys, but I find it hard to be honest. I mean, I think you should comment about that a little bit. This notion of we are all kind of I feel a little I don’t want to penalize that, you know, poor representative. Sure. Because I don’t trust what you’re going to do with the data. How is this all of us all being fake, nice and pressured? How has that impacted this whole profession?
Martha: Right. So if I had one question, I was just, you know, every day I could have a different question. But the question that came to mind as you were talking is, if I just had one question, I’d say, what could we improve? What could we what assumes some now. Now it is leading. Yes. Right. It assumes that something could be improved. But I think it’s a better leading than how likely, you know, but just like, hey, what could we improve. Yeah, that would be I think that’s a that’s a good question. I love that because it’s leading, you know, in a place of humility. It assumes that something’s broken. We’re not perfect. And it could be small. It could be something small.
Amas: So you release that. That makes sense. My last question for you would be sure, um, in your work with leaders who are interested in doing this correctly, I assume? Sure. If you’re anything like me, most people are checking the box or doing a project or what have you. But then you meet these leaders who are like, I really want to do this thing correctly. What do you tell those guys? What do you tell that person who shows up and goes, I want to really fix this and do this whole gathering feedback and intelligence. Well.
Martha: Sure. Um, I would say let’s start with and we do it for free audit. Let’s audit your survey. Let’s audit the deliverables out of that survey. And honestly we’ll tell you if you’re in a good place meaning you don’t have to be perfect. But at the point of diminishing returns we’ll tell you and like go on with what you’re doing. And if we find there’s some, some gaps in how you’re asking your questions or how you’re bundling up those results, we’ll tell you that too and tell you how to get that fixed.
So I would start with audit. Audit what you’re doing. Does that make sense. Like really like have that intellectual curiosity I would start with that. Like that’s a good place to begin is I’m just intellectually curious. Could are we asking a survey that’s maybe we again maybe we need to do this like gaming, blah, blah, blah. That’s one side of our company. But maybe there’s also, ah, maybe there’s space to listen to our customers as partners and tweak some things, or maybe make radical shifts in things. Or maybe we need to. Maybe everything’s great. Our price point is good, our supply chain is good, but our communication around those isn’t great.
So, um, and, you know, honestly, we could talk about this for hours, like I do a workshop on how to improve the survey invite so it’s less leading, right? How to improve surveys themselves, how to bundle up and run analysis like correlation analysis, is almost always fascinating finding out what variables are moving together, right? Because those can become if it’s a really high correlation drivers. This is a key aspect of how to improve surveys and extract meaningful insights.
Well that’s really interesting. Um so and then text analysis you know getting I we love it. I mean we use a lot of AI. In fact, I think we’re posting something about AI on my LinkedIn today. But you have to use it selectively. It’s not like this one tool that does it all, because it’s not. Understanding the nuances of AI is becoming increasingly important to how to improve surveys and ensure accurate data interpretation. It will be at the point of general intelligence, but right now it’s large language models. And so you really need to kind of work your research arm alongside AI. Um, so, you know, there’s examination of how you’re using AI for customer experience and are you getting truly accurate information because we’ve worked with companies where even like with their unstructured data. So that’s the, you know, open ended text or the conversations they’re running AI, they’re like, we’re great. We’re in a good place. It’s like, okay, well, we’ll audit some of that for free. Like, you know how you’re doing because we’ve found cases where even the sentiment was wrong.
Which is the easiest thing to measure, like the center. But it was wrong because, you know, customers would say something like, I really like this about company XYZ. Um, however, when they do, um, ABC, I’m really kind of dismayed. And what the what the I got was the very first part and not the second. Yeah. And not the second part. And so if a human is listening to that or reading that, it’s like it’s a mixed reaction. It’s not positive, it’s not negative, it’s mixed, which is a very different kind of a very different kind of sentiment. But that’s the easiest part of any kind of, you know, coding, decoding of unstructured data. Then there’s the whole, you know, around content, um, you know, what are the themes anyway? We truly I mean, this is I love talking about good data and how to get it.
Amas: We have to do a part two to this. I just I just got to say, um, I didn’t know this field existed. Most companies, they want to do surveys or what have you. They just copy what everyone else does. People bring all of their practices from. I used to work here. We set the thing up, we start firing it off. And even just the audit you just talked about, like I used to scream at the quality of our data around surveys and who was getting surveys and the triggering. And I just thought, this is not science at all. And I but I was just another guy screaming in the void.
I am so happy to learn that, um, people like you and your team exist who are who can really help people do this thing the right way. It is science, ladies and gentlemen. That’s what we’ve forgotten because we’re asking a question about feelings. How would you feel about it? Is it is science, right?
Martha: Right, right. And you touch it. I think this would be a great part to sampling techniques, you know, like that’s really interesting. That’s part of it. It’s like there’s the questions you’re asking to capture data, but then it’s like, which audiences are you going out to and how are you going out to those audiences to ensure that you’re getting representative response? Because if it’s not representative, that’s another kind of like, well, that data is it’s gained in its own way. So that that would be a really interesting topic to deep dive into.
Amas: I can’t wait to have you on. Martha. This was fantastic. Thank you so much for coming. Uh, your info will be on. So folks, if you want to engage with Martha even more and her team, um, contact her on LinkedIn. But thank you so much for doing this.
Martha: Yeah, I appreciate it. Love it. Thank you.
============================================
Would you like to learn more about how to improve surveys to help you steer your company? Get in touch!
============================================
