Use CX to Give Customers a Seat at the Table

The other thing, I know that we talked about this quite well for the first session. Please make sure that you are submitting questions through the mobile app. I have my phone up here because we’re a digital company, so I’m going to check throughout it to make sure we understand how many questions we have and we’ll try to get to as many as we can at the end.

All right. So we are going to talk about some CX today. We have people kind of in a broad range of different positions, gathering feedback, implementing feedback, so we’re going to spend some time talking about that. So let’s get started with the basics. How do you gather customer feedback today?

Katie: I’ll start. Am I on?

Christy: Yeah.

Katie: Can you hear me? Okay, sorry. So I think the important thing to think about with customer feedback is all the different types of customer feedback and inputs you take in. So it’s App Store reviews, it’s direct customer feedback through a customer feedback link.

But we also use customer…what we call customer councils. So we have certain customers that have been with us either a long time or have a certain profile that we look to help test products or just ask questions of.

And we also leverage feedback from our care agents. They are directly on the lines with our customers every day. So we get a ton of valuable feedback that way. And then there’s NPS, there’s a lot of different mechanisms to get feedback.

Ashley: So another one that we use, which is a little bit different based on our space, is that we do ride alongs with our technicians as well as part of the installation process. So we talk directly to the customers as they experience the installation of the security system and then take out a lot of that information back into the product organization and adjust feature sets accordingly based on that firsthand data.

Dan: So at Dunkin’, we get a lot from our call center, a lot from our app reviews, but also we get a lot from the franchisees who have that, you know, one to one customer interaction with the guest. So they provide their feedback all the time, whether it’s directly to us on the digital side, although I’ll present to our senior leadership team. And then we also look at all our social channels and then any kind of emails that we receive from our guests too.

Christopher: At Premera Blue Cross, we like to use different channels to gather qualitative and quantitative feedback. In terms of quantitative feedback, we have our analytics from our website as well as mobile app. Qualitative feedback, we use App Store reviews.

Product managers will also often sit with customer service representatives to listen to the calls. We also send out emails after a member has used our member portal on the web or in the mobile app. And also, when they call customer service, we send them an email as well to follow up with them.

Christy: So Christopher talked a bit about kind of quantitative feedback, but a lot of the things, drive alongs, talking to franchisees, those can be really qualitative. How do you ensure that when you’re getting that qualitative feedback you have a representative customer base?

Christopher: Yeah, in the healthcare space is kind of complicated because, you know, we’re highly regulated and the regulation really is challenging because we have to be very careful and mindful about how we use member data, so patient health information, PHI or PPI, as we call it. So whenever we do our user studies, some of the complications include we can’t recruit Premera members and we also can’t use actual member data.

So we have to recruit externally and we also have to use dummy data which can be challenging because just sort of a loss of context, right, when you do user studies with that type of demographic and with the dummy data. But we do work around that challenge by recruiting participants for those user studies that closely mirror our member demographic. So we’re very specific about who we’re looking for.

And the other thing that we do is that we also have to do a lot of upfront storytelling in order to create a very strong narrative before we ask a participant to complete a task on our mobile app during the user study. And I think that’s effective way to sort of understand sort of some of the challenges and pain points that our users are facing, our actual Premera members.

And we also try to parse out some of that feedback that we get, you know. The member feedback can be very complex. A lot of it, you know, is related…the member pain points is related to the actual participant frustration with their own plan as opposed to Premera, or frustration with…the healthcare industry in general, as we all know, is very challenging and difficult space to work in.

Christy: Well, and Dan, I know you had some comments on kind of if there’s a franchisee who knows leadership, how they might, you know, think that they know best. How do you ensure that you are being representative?

Dan: Yes, we got a… So franchisees love to chime in and give us feedback on the app. So just because one franchisee says to us, you know, “Hey, you need to fix our enrollment app or the enrollment for our DD Perks program,” we’ll actually pull up Flurry Analytics, does that jive with what this franchisee is saying? So do we have a huge drop off rate once a guest gets to the credit card portion of enrolling in DD Perks?

And then they’re like, “I don’t feel secure with a credit card.” Or if they’re doing On-the-Go Mobile Ordering and there’s a huge drop-off once they place the order, they hit Submit, but they haven’t checked in.

So we start to look at Flurry Analytics, where are the drop-off rates, but then we also chime in with our support center, our call center to find out, are they seeing the same trends, are we seeing the same trends in social? And if that lines up, then we’ll go ahead and, you know, put that into one of our sprint releases that are upcoming.

Katie: Yeah, I guess I would make sure one thing, just kind of as a call to action, make sure you understand what data you need. I think in the small business space, it’s not enough to say, “I have 5 or less employees or 3 or less employees or 10 or less.” Like what matters to us a lot is the vertical type. Am I a restaurant? Am I a retail? Am I brick and mortar? Am I online only?

So there’s a lot of different kinds of feedback we’ll get based on those segments and it’s really important to us to actually make sure to tackle those audiences. So we’ll do that either through segmenting the data after, if we happen to know, you know, their shopper ID and we know a little bit of information about them because they were on the phone with an agent, or we will actually leverage things like customer surveys to very specifically target a audience that we need information from.

Christy: So that leads really nicely into kind of where we want to take this. What listening strategies have you found most effective? We just listed all of the different ways you get feedback, which ones are most effective?

Katie: Oh, I have the mic. I’m just going to talk again. So I think about this in a couple of different ways because I think what’s most effective depends on what you want to do with the data. So, you know, app reviews can be very effective because you might mine specific bugs out of that feedback or feature requests. Other types of feedback will give you what you need to look out for your product roadmap. So that might either be research or it might be surveys that you’re doing.

And then one thing that I think has really been impactful for our customer experience is making sure the employees have a tie into that listening and a direct one to one relationship with our customers because then the empathy you start to see and the change of the mind frame of your employees that you start to see because they’re more connected to that customer feedback just actually changes the way you make product decisions and the way you develop your software or your service.

So we do what we call buddies. You know, a lot of our employees across the company who are formalizing the program now actually have a small business buddy that they will talk to. I have three just because I know a lot of small business people in my life. I’m trying to limit it to three because it’s a lot of time, but I’m experiencing their pain, their questions, I’m helping them connect with care to get answers, I’m walking them through, you know, their bug or their feature requests.

So those kinds of programs. Every week, people have a dial in to listen for an hour to care calls so that they are hearing firsthand what our agents are hearing from our customers. So I think that’s just really, really powerful when your employees have that experience.

Ashley: And I would ditto that comment. We have a group at ADT called our Technical Assistance Group, and it’s our tag organization that all the customer issues get escalated to them to answer. And on a weekly basis, we meet with them to see what their top issues are, what’s driving their customer calls, and then wrapping those issues into our product roadmap as part of our backlog to ensure that what we’re doing isn’t driving additional calls into the call center and we’re actually tackling the things that customers are worried about most.

We also go out to these call centers and sit next to those care agents and with our sales team to listen firsthand as well to those customers as they call in. That really brings everything together in a nice succinct manner.

Dan: Yeah, we are… So my journey has… I was in the call center. I managed our mobile app and website called Center for Dunkin’. And so I know how important it was to take that and transition it over to this current role to listen to what the guests are saying.

So we pulled…we listened to our support center, our call center regularly. We’ll meet with them once a month to get what’s the newest trends and have them feel that they have a say of what’s going to go on into the mobile app, what’s going on at dd.com.

We kind of rely on them a lot too. So we say, “Hey, you know, what changes do we need to make? What are we missing? What are the guests saying?” And they’ll provide it to us and they love…when we make that update, they love seeing that because they know that they had some sort of impact.

Christy: So as a customer success person, I love that we get to work with people who listen to their customers. Hopefully, it’s apparent to our customers that we try to spend as much time between product and customer success as we can.

My followup question to that is how do you manage expectations? So if the best way to get feedback is talking directly to customers or to customer care teams or things like that, you can’t do everything and you have…as product people, product marketing, product managers, you all have better insight into kind of the roadmap and where you want to take things.

And if you were to go and do everything that every customer said, you wouldn’t get any of your goals done. How do you manage those expectations back to those teams to ensure they know they’re being heard, but you’re not spending all of your time just chasing all the feature requests?

Dan: I’ll go. So we meet with the support center once a month. So when I first met with them the first time in the new role, I, you know, set the expectation, “Hey, this is what our sprint schedule looks like. This is the timing. This is when… You know, this is our priority list.” And get them to kind of level set. This is the gate, this is our war room, this is what we got going on.

So when the customer service agents are saying, “Hey, we want to see the X, Y, and Z to get into the app,” we’ll sit there and say, “Okay, that’s kind of a high priority. We’ll throw that in there.” But we also have competing priorities and this is how many, you know, developers we have available for us to get that done.

If we can’t get into the next, you know, sprint, we’ll give them the timing, “Hey, it might be one sprint, two sprints down the road.” In that way, we’ve level set those customer service agents to say, “Yeah, we understand we’ve got it. Yes, you’re right. That is a priority.” Or, “Yeah, that might wait two or three sprints. You might see that in November,” or something like that. We try to manage that expectation and so those monthly cadences that we have with them, just everybody’s on the same page, they know what’s coming and what’s not.

Christopher: Yeah, I think one of the challenges of working in product development is that oftentimes you have to make decisions based off of maybe 70% information. Right? And that’s one of the biggest challenges. But I think an effective way to work around that is to clearly communicate to your different teams and to bring them into their product roadmap and development process, marketing, sales and operations, your executives, your engineers, your designers, and user researchers.

I think once you include them into the discussion and there’s a lot of transparency, there’s more understanding coming from those teams when the product features don’t roll out when you expect them to and also when priorities start to shift because that often happens, right? So based off of member data, your number one feature release for the next sprint cycle could suddenly become number two or three, and that happens often.

Christy: Perfect. So shifting gears just slightly, we talked about how you’re getting feedback, how you’re implementing that feedback. All of you work in industries where it’s not just an app that is representing your company. There’s either brick and mortar or, you know, small businesses that are actually representing you as a company and none of you are just mobile. All of you have…none of you are just app. All of you have a mobile experience, have an online experience. How do you ensure that there is customer experience parity across the different channels?

Katie: So for GoDaddy, I think it’s been a little bit of a challenge at times. And when I think of channels, for us, that would be merchandising. So you could think of marketing and advertising, you could think of when you land on our front of site, and you’re looking at merchandising pages to decide what to buy. It’s our commerce experience, so what is my cart experience like, what are the emails that I get, the receipts, what happens if my credit card payment fails, that whole commerce experience.

And then there’s actually like, “Okay, now I have the product. I need to activate it.” And then there’s ongoing engagement, and that can happen in our app, that could happen on our sites, and with our care agents. It’s yet another channel. So when we look at that across the board, that’s a lot of touch points across a lot of different teams in our organization.

For us, as a company, last year, we reorganized to create a customer experience team that’s specifically focused on tracking major customer feedback points, our NPS performance, and what we can do to respond to some of that feedback and drive that score up and just looking at our flows end to end, looking at the whole experience end to end across all the different organizations and coordinate what that ideal experience is going to look like.

So we have, you know…we’ve chosen to take a very firm organizational approach to tackling that and it’s… You know, we’re in the first, not quite, year of that and it’s… You know, you test…just like with product, you test and you change, and so we’ve been fine-tuning it, but I think it takes something that was hit on earlier, which I think is just kind of a theme across the board when you’re doing this in your org, which is communication and transparency and you’re willing to experiment and change your approach.

Ashley: And we’re set up a little bit differently. You know, we have a call center, we have field agents, we have care agents, we have, you know, over 300 SSL locations nationwide. So for us to ensure that the entire company is talking the same story and telling the same product, feature sets is really collaborative effort across the entire organization.

So we meet with them on a weekly basis, we go through communications, we go through what’s on the roadmap, what’s coming, what’s in the next release. We look at the go to market strategy to make sure that everybody’s in sync as we roll out a new feature and that we’re all talking the same story.

If we’re not, then it’s a big loss for the customer because then the customer has no idea that we just launched this great new feature because no one told them about it through the whole process or they got different messages from different vehicles from our marketing campaign. So it’s really that collaborative effort across the entire organization that keeps us in sync as best we can.

Christy: So to be super clear on this question, Katie mentioned, “Hey, we struggle with this a little bit.” Everyone struggles with this and that is why… I mean, so this group up here has awesome ideas of it, but if you’re feeling a little bit like, “My team is not good at creating parity,” it’s okay. Everyone else is feeling the same way right now.

For Dan and Christopher, as kind of followups, you have a little bit different model where you have some field agents and things like that. Dan’s dealing with franchisees that each represent the brand in a different way and Christopher’s dealing with service providers who are not part of the company that are representing the brand.

For you to… Kind of the same question, how do you create parity outside of your core business unit? Not just in different channels, but in people who may have kind of different goals and different things they’re trying to do.

Christopher: Yeah, I’ll answer that. You know, Premera Blue Cross has the luck of being part of a larger Blue Cross Blue Shield network, which is a very large national healthcare network. But to answer your question, you know, we work really closely with our internal teams as well as external partners, which includes our national account managers and directors, as well as producers, in order to effectively communicate on a yearly basis to healthcare providers that we onboard onto the networks about the initiatives that we’re working on and some of the changes in the healthcare plans and the business.

Christy: And I think you said that when the different Blue Cross teams that might not be part of the same business meet together, you’re sharing a lot of the data and you’re learning from each other, so one team doesn’t have to do all the research. Is that…

Christopher: Yeah. You know, we do share some…you know, across the Blue Cross Blue Shield network, we do meet on a regular basis each year in order to share some of the, you know, experiences that we’ve had in the past year as well as some of the products and strategies that we’re working on and some of the innovation that we’re working on as well.

Dan: So for us, it’s…we bring in all sorts of areas of Dunkin’, loss prevention, if we’re dealing with some security issues, or we got security enhancements that are going on, communications in case we got to get some sort of email released.

But the biggest one is we are always getting our operations team involved because the operations team, if we don’t communicate clearly to the operations team to say, “Hey, we’re making X, Y, and Z changes in the mobile app,” then they’re not able to communicate it to the franchisee, the franchisee is not able to communicate it to the crew member, store employee at the Dunkin’ restaurants, and then that crew member is left out to dry with the guest and they don’t know how to handle it. So if we miss that…if we have an opportunity and we miss it, we set the crew member…in the end, we set the crew member up to fail.

Christy: So we have a few questions. Obviously, the four panelists up here work in very different fields and we have some specific things we thought it’d be nice for people to hear a little bit about specific issues they’re facing in their different verticals.

So, Dan, you led really nicely into this question. Thank you. You recently rolled out the On-the-Go Mobile Ordering, and that is a super complex experience because you are dealing with a mobile app, you’re dealing with in-person, you’re dealing with in-store, and getting all of those things in sync and blending together can be really difficult.

Can you share a little bit about the strategy for customer experience with regard to the On-the-Go Ordering, and most importantly, how did you incorporate customer feedback into the process both before and after you launched?

Dan: Yeah, with the On-the-Go Mobile Ordering, with Dunkin’, it’s all about quick and convenient. We want to get them in, get them out. We didn’t want them to be sitting in long lines anymore waiting to just get a coffee. So when we developed the On-the-Go Mobile Ordering, it was about, you know, convenience and utility and let’s get…you know, order your coffee, walk in, pick it up.

The drive-through is a little clunky because you still got to wait in the line to pick up your order, but mostly when you’re in the city, across the country, people just want to walk in, grab the coffee, and walk to work, or grab a train. So that’s where the whole strategy came about. It’s like speed is a… When you talk to Duncan employees, it’s about speed of service and accuracy. So if we got your order accurate and we’re going to get you in and out, that was the whole purpose of the On-the-Go Mobile Ordering.

Since we’ve launched that, we’ve gotten a lot of feedback, both, you know, on functionality, but also on menu items. So we didn’t think…we thought that functionality would be one of the top items and it ended up being menu items, or things that we’re missing when you are placing On-the-Go Mobile Ordering.

So when we first launched it, almond milk was the number one trend. We didn’t have it. So, of course, we scrambled and got through that into a sprint and got that in. And then we said, “All right, so we’ve solved the almond milk issue, let’s work on functionality.” And it turned out to be ice, right? So how complex could ice be? People want to sit there and say, “I want more ice. I want less ice in my cup.” So now we’ve thrown that in.

Now we feel like we’ve solved that. People now want more swirls, you know, different types of swirls, caramel mocha. They want to change their swirls. And we’re like, “All right, what about the functionality?” And so we just launched a redesign in June, which is more about utility and changing skins and just making it a little bit sleeker.

I was really excited about reading the reviews and the first review that we got was, “So excited that Dunkin’…” I think this is verbatim. “So excited that Dunkin’ released a new mobile app. Couldn’t wait to use On-the-Go Mobile Ordering, same crappy experience. So disappointed.” So finally, we got some functionality, because it takes 12 screens to order 1 coffee. So that is one of the pieces that now we’ve taken away from the guests and said, “All right, we’ve got to clean that up and that’s, you know, one of our top priorities.”

Christy: Last little followup. How are you…what kinds of things are you doing to gather that feedback at scale? Are you using surveys? Are you like in person asking people? How are you getting that?

Dan: Yeah, it’s app reviews, it’s the call center. But it’s also we do consumer insights. So we have a consumer insights team that will go in and survey guests and they’ll ask them a series of questions and try to get the feel for what the guest is saying. So as we go through a new change in our on-the-go experience, we’ll most likely sit down 20, 25 guests and say, “Hey, what do you think of this,” and get their feedback before we haul off and launch.

Christy: Cool. All right. Christopher, so working in healthcare, there must be a lot of privacy restrictions on what you can and can’t do. You mentioned, you know, when you’re going and testing, when you’re getting customer feedback, you can’t use real data and things like that. Can you talk about how you ensure that you give customers a great experience without necessarily being able to talk to the exact people you’re going to be working for?

Christopher: Yeah. Like I said before, you know, we try to recruit user study participants that closely mirror our own member demographic profiles. And you know, we do these studies on a frequent basis and we try to gather as much information as possible. And I think these user studies are particularly useful because you really get a deep insight into some of the challenges, wishes, and desires of users. So I find that’s particularly, you know, helpful.

Also, we do…we send out surveys. You know, we recently did a baseline study, a remote baseline study, measuring basically the ease of use of our mobile app, and, you know, all that work culminated into really outstanding results. And a result of that baseline study indicated that our iOS app got over 95% score and our Android app got 90%, and the median for that is 60%. So that’s really excellent.

So I really, you know, recommend using all the tools and channels that you have to get the qualitative and the quantitative data in order to understand some of those challenges and desires that your users have.

Christy: Perfect. All right. Ashley, how have you used consumer insights to create your product marketing strategy at ADT?

Ashley: So we use it in a couple of different ways from when we start seeing the surveys come through. Like I was telling you this morning, we had a new release of our mobile app just recently and we started seeing the surveys and the comments come through saying, “What did you do with our Touch ID? It’s gone.” And we said, “No, it’s actually not gone. You just have to set it up again.” So we were able to send out an email communication in the next e-newsletter to our customers and say, “This is how you set this up.”

So the surveys, the customer care surveys that we have, the reviews, all of those things get rolled into our communication plan into what videos we produce from our how-to, what FAQs we need to update on our help center, what additional training do we need to do from the field perspective, from our technicians into our sales team to say, “These are the things that you need to talk about and make sure customers, you know, how to manage their security system and how to use it most efficiently.”

We also take those surveys, things that come up and drive them into the backlog priority for future releases. So when we see that passwords are a big issue for our customers, how can we get the functionality to keep you logged into your security system? It sounds very, very, very like a small task, but for us, very much like you, Christopher, where security is really key, there’s a lot of things that go into just checking the box that says “Keep me logged in.”

So it’s a feature that we know is driving calls, it’s driving a lot of feedback from a survey perspective and just getting that priority in the roadmap.

Christy: Perfect. All right. Katie, when building your product roadmap, how do you make sure customers’ voices are taken into consideration?

Katie: So the product roadmap is an interesting timeline because I think your product roadmap could be a quarter, it could be a year, it could be three years. So you kind of have that longer-term vision, you have the immediate, you know, quarter’s worth of sprints that you’re looking at, and you kind of have to think about what inputs are going where and how long certain efforts are going to take. And I think like the main thing we do is take the direct customer feedback on the immediate needs.

So if there’s something that is, you know, just really causing a lot of customer pain or is holding us back from a key metric, like, people get a bunch of O365 credits with their GoCentral Website Builder, but they don’t activate it. So they have these free email credit sitting around that they never use. So recently, we have a backlog of other features we’d like to look at, but we were like, “Hey, this is actually something people are asking for. They don’t realize they have it. They’re not figuring out how to activate it.”

So rather than building something completely new, we’ll actually have a little effort around activating products, which is something you wouldn’t normally think about on your product roadmap. But that’s coming in from both customer feedback and care agents and then just the quantitative data we’re seeing in just usage.

So that’s…one thing is just to look at all the different inputs and then figure out, “How do I not just have a roadmap of what I want to build, but how do I prioritize it and how do I put resources against it?” Because you might choose to put resources in something in your roadmap against, you know, improving your NPS or DSAT from your customers, and then you might be looking forward thinking for things people haven’t thought about yet that’s an industry trend that’s coming.

So it’s an interesting mix of making sure you’re looking at all the data and all the inputs, but I think we do that via the channels we’ve kind of all talked about in addition to some of the research channels, whether that’s a survey or an actual usability study where we’re sourcing from existing customers to have them try out, you know, new things that we’re working on.

Christy: Anybody else want to add to that particularly? Okay. All right. We’re going to shift gears a bit. I just want to say it’s really great to work with companies who prioritize actually listening to customers. Again, as somebody in customer success, to feel heard is a huge deal and to feel like you can actually impact the roadmap is a big deal. So thank you all for doing that.

All right. So for the last few minutes, we’re going to talk about kind of what’s coming next. So what do you see on the horizon that companies will need to add to their CX toolkit in the next year?

Katie: We were just talking about that.

Christopher: Yeah. Machine learning, AI. I know that’s the big topic these days, but there’s some truth to that in terms of how it’ll help you improve your customer experience. I don’t think that it will replace, you know, having human customer care agents by any means, but it’s definitely a good way to sort of handle the volume that you’re getting, you know, from customers.

And I think that’s a useful tool that can be integrated in different digital touch points such as the mobile app, web portals, and anything else that you can think of. So I think that’s the biggest trend probably.

Katie: Yeah. We were talking about this a little bit earlier and I think machine learning, in particular, is going to start helping deal with volumes of verbatim feedback. So as you scale and you get larger, the ability to see the pattern, see the trends quickly enough to respond to them gets harder and harder and I think that machine learning can help a ton with that.

And we were talking a little bit about the chatbots and how maybe that’s a nice way to connect the customer to maybe solve a quick issue, but we agreed, you know, like hey, we think there’s our human on the end if you get to a certain point and there’s a nice handoff there. But there’s a lot of opportunity to like ease your customer.

I think the mental or emotional state of your customer is super important and you have to figure out like what emotional state are they in when they come to you, what emotional state were they in when they wrote feedback, and how does that emotional state play out as you solve their issues? And so some of these technologies can actually bridge some of that.

Christy: All right. Anything else on that? Okay. We are like laid on time. Cool. So I’m just going to go through some of the questions so that we have some time. There’s one more that I really want to ask, but I want to make sure we have some time for questions.

So let’s see, we haven’t… Does anybody have a specific example of where CX data drove changes within your organization? Dan kind of shared this with like the ice and things like that, but does anybody else have a specific example of the data driving changes?

Katie: Yeah, so we actually…we have these meetings that overlook revenue and how we’re performing and how that maps to our merchandising, and we recently started having one of those on the customer experience side. So we’ll take leaders and product owners from across the company and every week, we go over, “Hey, here’s what’s trending, here’s what we’re seeing, and here are some top issues that we want to actually get action on immediately.”

So every individual team is doing that, but then as part of this meeting and kind of effort, we can collectively look at what’s eroding, you know. Like let’s say, just, for example, we have something in the cart, you know, there’s a bug in the cart or something is not totaling right, that’s going to impact everybody’s experience for that customer, not just…it’s not just the carts customer that’s GoDaddy’s customer.

And so anytime there’s an issue that can kind of flow through all of the different pieces of your product experience, that’s something that we’re all going to be tracking and looking at and resolving and kind of prioritizing so that we can better utilize resources to tackle some of these big issues as they come up.

Christy: All right. How do you prioritize customer feedback in terms of what to go after first? We’ve talked about a few of the tactics, but a wrap-up would be awesome.

Dan: So we get a lot of one-off requests, though it depends on if we see anything through Flurry Analytics or anything else, but we’ll take whatever is the hot-button issue. So, you know, back in August of 2017, we changed ways to change your location when you do an On-the-Go Mobile Order.

We went from having it very cold out and simplified because it’s based on the last restaurant you went to, and what we did is we, for some reason, hid it on the top of one of the screens so guests didn’t see it. So what was happening is they placed their order. They might’ve been in Boston yesterday, placed an order there, but now they’re in Plymouth. And they place the order, it goes to Boston, they show up to Plymouth, and their order is not there. And they got charged for it too. So now they’re out the money.

So we ended up…we saw that volume just ramped right through the roof everywhere, social, call center, you know, franchisee feedback. So we immediately put a quick change in there, put together a call to action right in the On-the-Go Mobile Ordering floor. Just simplified it. I mean, we made a big box, says, “Change location,” and simplified it, and then we just watched that volume dropped right off. So stuff like that is what we use.

Christy: Two more questions. There are a few in here we’re not going to get a chance to answer. There’s one specifically for Dan. Fun fact, we’re going to do networking after this. At some point, there’s going to be a vertical networking, so just go see him directly. All right, two more. How do you know you’re getting enough feedback?

Christopher: Yeah, I think that’s, you know, like I said, oftentimes you try to make a decision based off of a limited amount of information. And I think, you know, whenever you feel uncomfortable about making a decision about whether or not that’s the next feature that you need to be building or not, take some extra time and go and gather more feedback.

You know, use your channels, do those user studies, look at those analytics, talk to your teams from customer service to marketing to anybody that you have on the ground in order to really get a bigger picture so that you feel comfortable making that decision for the future releases.

Katie: Yeah, I’d say it depends on what you’re looking for. I mean, if you’re looking for feedback on a bug fix, that’s going to be different than feedback for a new feature. So I think you need to know exactly what do I want feedback on, and the most important thing about feedback is what are you going to do with it?

So you can solicit a bunch of feedback, but if you don’t know how you’re going to take action or use it in decision making, it’s, you know…you’re kind of running at a deficit. So you have to think about what am I going to do with the feedback, what segments of data do I need for…and what particular issue am I trying to solve? Because then you can kind of gauge, do I have enough feedback or not?

I think one thing people struggle with is beta feedback. So you could get feedback forever if you just leave a beta period open, but at a certain stage, you’re going to look for, you know, trends of volume and spikes and, you know, usually, in the first week, you’re going to have most of the actionable feedback you’re going to take, for instance, on a mobile app release. So I think, again, I would just really think about what data do I need for what scale and what am I going to do with it?

Christy: All right. Last… Oh, did you want to…

Ashley: For us, we do a lot of friends and family, and when we go to much bigger product launches, friends and family can be 2 weeks if it’s a minor change, it can be 30 to 45 days if it’s a really significant change. And we survey those participants throughout the whole process and if it looks the trends are headed in a positive direction, then we’ll go ahead and take the leap to launch that whatever particular feature or product that may be.

We gather feedback afterwards as well. Like we were talking for a small change in the app, we thought, “Oh, it’s great from a UI perspective. We’re going to remove this button.” And we immediately got feedback that said, “Where’d the button go?” So we rolled that back into the next release. So as you constantly iterate through the whole process, you know, for us, we shortened the release schedule as well so that we could react more quickly to the feedback that we get right after launch.

Christy: Perfect. All right, last question, how do you help leadership to understand how important customer feedback is in driving decisions? Or another way of putting it, how do you prove your LTV or your ROI?

Ashley: So for us, we take the number of calls that we get. That’s a real high priority for us at ADT and I’m very fortunate to work for a company that does listen to customers. How long do customer calls take? How many are we getting? What are they talking about? Have we listened? Do we talk to customers on a regular basis?

So for us, we are in a little bit more of an advantage based on our leadership right now. That is their number one priority. But a lot of the financials of is it driving calls, are we losing revenue, are we losing customers, help build that business case to make those changes.

Katie: Yeah, I’d agree with that. I’d say GoDaddy has a great leadership in support of understanding and listening to your customers, but at the end of the day, that’s part of it and the business case is going to help a ton because you’re going to have to figure out how to prioritize your resources. So happy customers equals customers that stay and buy more of your stuff and so that means retention.

So, you know, one metric, you can look at acquisition all you want, but if you’re not retaining, it doesn’t matter. You’re bleeding out the funnel. So retention metrics are really important and I think understanding kind of your whole funnel metrics in general because you need to know where you’re bleeding out customers and where they’re most unhappy. So I think, again, just to echo, I think the business case is an important piece of that.

Dan: Our leadership is really strong as far…and opened-minded to any kind of changes. So they might toss out a few ideas, but we look at, you know, is it going to drive revenue, is it going to drive guests, you know, transactions, but also, is it going to drive the repeat guests? So if it’s not going to drive a repeat guest, we’re not going to really look at it, if it’s a onetime thing.

So we present trends and, you know, what’s the potential revenue on this enhancement as part of our revenue stream? So we look at all that stuff. They’re very open-minded to all our changes, which is really good.