Rethinking the Mobile Surveys vs. Web Surveys Debate

April 15, 2015

Alchemer Mobile (formerly Apptentive) mobile surveys

When marketers evaluate their options for conducting market research and gauging their Net Promoter Score (NPS) in the digital era, they have historically compared two options: web surveys and mobile surveys.

Over the next few sections, we’ll explore the history and merits of both options, a few trends in survey and web consumption affecting the web surveys vs. mobile surveys debate, and the factors marketers should consider moving forward with their next marketing research or voice of the customer (VOC) initiative.

What are Web Surveys?

Web surveys became popular in the mid 1990s, and have seen a dramatic rise in popularity over the last two decades. Web surveys provided market researchers an automated way to quickly gauge customer sentiment. It’s easy to see why online surveys became so popular so fast, as offered companies the opportunity to slashed costs associated with data collection, organization, and analysis associated with more traditional surveys conducted over the phone, in-person, or by mail.

Often, web surveys are distributed via email or a pop-up form to customers who have completed a specific item (making a purchase, chatting with a support representative, downgrading a plan, etc.) or sporadically to gauge the changing customer sentiment, particularly in light of new updates to a website or product offering.

What are Mobile Surveys?

Mobile surveys became popular with the rise of smartphones in the late 2000s. Such surveys are designed specifically for mobile browsers or built into mobile apps or SMS messaging capabilities. While mobile survey adoption is still low among enterprise organizations, more agile start-ups and SMBs have been quick to incorporate mobile into their marketing and market research strategies – and have greatly increased their collection of customer insights in the process.

Like web surveys, mobile surveys are commonly prompted after key moments of engagement or perceived customer pain points where marketers and developers seek behavioral data points around specific steps of the customer journey.

Web surveys and mobile surveys probably sound like two separate market research instruments – and historically, they have been. The choice of which survey to use has traditionally come down to the individual brand and its customers: Where do my customers most commonly engage with my brand? Which channel has the highest contribution to my bottom line?

By this logic, an online-exclusive e-tailer, for example, has no reason to invest in mobile surveys… Right?

Not anymore.

A Changing Landscape

In 2013, the number of people worldwide accessing the mobile web surpassed the number of desktop internet users. The growth of the mobile web has continued to accelerate over the course of the two years since, while the popularity of desktops has begun to plateau. And in fact, adoption of the mobile web is growing 8x faster than web adoption did at its peak in the 1990s and early 2000s. (Click to Tweet)

Mobile web consumption vs. desktop web consumption
Data source: Morgan Stanley, Image source: ramanalokanathan.com

While retailers undoubtedly can – and do – continue to thrive without a mobile-first mentality, these trends carry a few pretty big implications that all companies should note:

  1. Regardless of how they ultimately engage with your brand, your customers are increasingly mobile. According to an October 2014 by the Pew Research Center:
    1. 90% of American adults own a cell phone. (This percentage is even higher in several emerging nations including China, Russia, Chile, and South Africa.)
    2. 63% of adult cell phone owners use their phones to browse the web.
    3. 34% spend more time on the web using a cell phone than a desktop or tablet device.
    4. 7% are smart-phone dependent when it comes to internet access, meaning they do not have home broadband service and have few, if any, other means of connecting to the internet.
  2. The process of discovery and customer acquisition is increasingly multi-channel, meaning that mobile plays a huge role in your customer’s journey regardless of whether or not you have a mobile-optimized website.
    1. 90% of people move between devices to accomplish a task, according to a Google study on the Multi-Screen World.
    2. 25% of searches are initialized from a mobile device, according to a report by The Search Agency.
  3. Customers increasingly prefer – no, expect – brand communication to be personalized, contextual, and in their preferred channel.
    1. The average consumer is exposed to as many as 5,000 marketing messages per day. Brand communication needs to be relevant to even attempt to break through this clutter.
    2. 47% of brands believe they deliver relevant communications and 44% believe they have correctly identified the preferred channels for customer communication, according to an IBM Commerce study.
    3. In reality, only 21% of customers reported that brand communications are “usually relevant” to them and 72% of consumers expect mobile-friendly sites.

What do these three trends have to do with the web surveys vs. mobile surveys debate?

Simply put – there is no longer such a thing as a web survey.

All online surveys are now mobile surveys. (Click to Tweet)

Your customers’ lifestyles are changing with regard to web consumption and their preferred method of brand engagement. And this means your marketing needs to adapt to the changing needs of your customers.

But don’t take our word for it – the proof is in the data. These two charts on search interest over time serve as a good proxy for changing preferences around web and mobile surveys.

In the first, we see that search frequency for both “web survey” and “online survey” has continually decreased since 2004, with the exception of a brief spike in 2011:

Search interest over time for online surveys and web surveys

Looking at search interest for the “mobile survey” and “app survey” search terms shows a very different story, with an upward trajectory closely correlated with the rise of smartphones:

Search interest over time for mobile surveys and app surveys

If you’re still on the fence on the issue of mobile surveys vs. web surveys, ask yourself the following questions:

1. What percent of my surveys are completed from mobile devices?

If you’re on par with the average brand, you can expect approximately 23% of your survey responses to come from mobile devices (a number which has grown from 15% 12 months ago). Optimized or not optimized, your surveys are seeing increasing mobile engagement.

2. How does not optimizing for mobile skew my data?

If you’re targeting only your web customers, you’re imposing a number of selection biases on your data – requiring you to take any newly discovered customer insights with a grain of salt. Preferred communication medium varies by demographic while web surveys see a considerably higher exit rate among mobile respondents – both of which can considerably skew your data.

3. How many respondents am I writing off by not engaging them on their terms?

The #1 most popular – and least intrusive – means of distributing a customer satisfaction survey continued to be email. Yet, a whopping 66% of emails are first opened on mobile devices. Failing to meet their customers where they’re already at is a leading reason why so many brands see low engagement and response rates for their messaging. If you’re distributing a web survey via email, you’re catering to only one-third of your email list.



Mobile Customer Decision Process


So how should we look at surveys?

Let’s remove “web survey” and “mobile survey” from our vocabulary. All online surveys, regardless of their design intent, are now mobile.

Instead, I propose re-classifying the aforementioned survey types as mobile-unfriendly, mobile-optimized, and mobile-friendly.

What is a mobile-unfriendly survey?

An example of a mobile-unfriendly survey

A mobile-unfriendly survey is any survey not designed with the mobile web in mind. Mobile-unfriendly surveys take the same methodologies and instrumentation techniques used in the antiquated web survey in the hopes that they’ll be just as effective on a smaller device.

As such, mobile-unfriendly surveys are often characterized by:

  • Question types that are hard to answer on a small screen, including matrices (pictured right) that require extensive work and scrolling to answer.
  • Free-form text-input fields, designed around the ease of a physical keyboard.
  • Long pages requiring horizontal and vertical scrolling.
  • More questions, or more labor-intensive questions resulting in a high LOI (length of interview) not conducive to consumer behavior on a mobile device.

Mobile-unfriendly surveys are a sure way to alienate your increasingly mobile customers. While they may do the job for respondents on a desktop device, there’s simply no way to ensure that the survey will only be distributed to users of the desktop web – meaning you always need to consider how your surveys – and all of your online content, for that matter – will look on other devices.

What is a mobile-optimized survey?

An example of a mobile optimized survey or a responsive web survey

Also known as responsive surveys, mobile-optimized surveys make a few crucial adjustments based on the screen size and device type of the respondent. When viewed on a desktop computer, the optimized survey will appear exactly as intended. When viewed on a mobile device, however, select question types and formatting will be automatically changed or abbreviated to be more conducive to the mobile user.

Mobile-optimized surveys are often characterized by:

  • Responsive scaling of questions to eliminate the need for horizontal scrolling and reduce the need for vertical scrolling.
  • Conversion of question types and response options to less time-consuming formats (converting matrix rating scales into dropdowns, as pictured to the right).
  • Introduction of auto-completion for free-response or dropdown fields.
  • Page load times under one second. (73% of mobile internet users have exited a web page that took too long to load.)

While it may seem like a trivial task, checking these boxes can have a huge effect on your survey open, completion, and abandonment rates. And in fact, smartphone users are about half as likely to abandon a mobile-optimized survey (11% abandonment rate) than they are a mobile-unfriendly survey (21% abandonment rate), according to a study by Survey Sampling International. (Click to Tweet)

Mobile surveys automatically scale to the size of the device
Mobile-optimized surveys automatically scale to the size of the device (Image credit)

What is a mobile-friendly survey?

Alchemer Mobile (formerly Apptentive) in-app mobile surveys

And finally, mobile-friendly surveys are those designed, first and foremost, for mobile performance. Distributed either in a web browser or within a mobile app, mobile-friendly surveys are the most conducive to an on-the-go consumer and designed specifically to maximize completion rates and minimize exit rates.

Designing a mobile-friendly survey is just as much a matter of writing the right questions as it is about the technology powering the survey. Characteristics of mobile-friendly surveys include:

  • Low requirements on respondent’s time. Mobile-friendly surveys should be able to be completed in under two minutes and pose no more than 10 questions.
  • Asking only the need-to-know questions. Use the information you already know about your mobile customers (e.g., device type) from your mobile analytics tool to avoid asking unnecessary questions. Save your limited number of questions for asking only those questions you care most about.
  • Contextually relevant and narrowly focused. Mobile-friendly surveys are prompted at key moments of engagement within your mobile website or app, and questions should be about those specific events. Your customers should know exactly what they’re being asked and why their feedback is important. (Read more on contextual mobile marketing.)

In terms of performance, mobile-friendly surveys offer, by far, the highest response rates. When you design a contextually relevant, well-timed, and well-placed survey, your customers will give you their thanks in the form of taking the time to respond to your survey.

Just remember that surveys aren’t a zero-sum game. The best survey is one that provides value to both the researcher and the respondent. (Click to Tweet)

For best practices and tips on designing mobile surveys, check out these related posts on the Alchemer Mobile (formerly Apptentive) blog or sign up for a free trial of Alchemer Mobile (formerly Apptentive) for mobile surveys made easy:

Final Thoughts

We’ve focused a lot on the technology and the device behind the survey, but surveys are about neither of these. Surveys are about communication, about listening and engaging.

As you consider which type of survey makes the most sense for your business, consider first who your customer is and how he or she likes to be engaged. Mobile might be the right channel, or it might not. It all comes down to the customer.

Know of any great tips for creating mobile surveys (or web, for that matter) that we may have missed? We’d love to hear them in the comments below!

  • Get Your Free Demo Today
    Get Demo
  • See How Easy Alchemer Is to Use
    See Help Docs
  • Start making smarter decisions

    Start a free trial

    Request a Demo

    By accessing and using this page, you agree to the Terms of Use . Your information will never be shared.