Survey: What’s in a Word?

As those of us in the survey research field are aware, survey response rates in the United States and other countries have been in decline over the last couple decades.  The Pew Research Center sums up the concerning* state of affairs with a pretty eye-popping table showing response rates to their telephone surveys from 1997 (around 36%) to 2012 (around 9%).  Others have noted, and studied the same phenomenon.

So what’s really going on here?  There are plenty of explanations, including over-surveying**, controlled access, and a disinterested public.  But what else has changed about sampled survey respondents or their views towards surveys in recent years that might contribute to such a drop?  As a survey methodologist, my first instinct is to carry out a survey to find the answer.  But conducting a survey to ask people why they won’t do a survey can be like going fishing in a swimming pool.

One place many people*** are talking these days is on social media.  In the past decade, the percentage of Americans using social media has increased from 0 to “most.”  I was curious to see how the terms survey and surveys were being portrayed on online and social media.  Do those who use (or are exposed) these terms have the same things in mind as we “noble” researchers?  When we ask someone to take a survey, what thoughts might pop into his or her mind?  Social media is by no means the only place to look****, but there is a wealth of data out there and you can learn some interesting things pretty quickly.

Using Crimson Hexagon’s ForSight platform, I pulled social media posts that included the word survey or surveys from 2008 (the earliest data available) to today (January 8, 2014).  First I looked to see just how often the terms showed up by source.  Here’s what I found:

Survey

In sheer volume, Twitter seems to dominate the social media conversation about surveys, which is surprising given that only about 1 in 6 U.S. adults use it. Of course, just because the volume is so high doesn’t mean everyone is seeing these posts.  The surge in volume is quite dramatic late in 2012!  Maybe this had to do with the presidential election?  We’ll see… keep reading!  My next question was what are they saying when it comes to surveys?  I took a closer look at the period before that huge spike in 2012, focusing just on those co-occurring terms that pop up most frequently with survey(s).  I also split it out by Twitter and non-Twitter to see what comes up.

clouds1

We see according, each, and online for Twitter posts and according, found, and new for all other social media.  Hmm, what could this mean?  Drilling down into each term, we can look at individual posts for each term.  I include just one example for each here just to give a flavor of what the data show:

Twitter 5/08-7/12

  • According to one national survey on drug use, each day…
  • D-I-Y Alert! Nor your usual survey job $3 each – Research: This job….
  • We want you 2 do online survey and research for us. Easy…

Other online media 5/08-7/12

  • Nonresidential construction expected to lag in 2010 according to survey…
  • Surprise! Survey found young users protect their privacy online
  • New survey-Oil dips on demand worry, consumer view supports

Among these sample posts, we survey results being disseminated from several kinds of surveys on both Twitter and other online media.  The Twitter posts, though, seem to have more to do with earning money online than other social media.  Next, I looked at August 2012 to today (January 8, 2014):

clouds2

Among the other online media, there’s not much change here from the previous period.  People replaces found among top co-occurring terms, but the focus is still on survey results.  For Twitter, we see a new top 3 terms co-occurring with survey(s): earned, far, and taking.  Here’s what some of the Tweets from the more recent period look like:

Twitter 8/12-1/14

  • Awesomest week ever! I earned $281.24 just doing surveys this week :)
  • Cool! I got paid $112.29 so far from like surveys! Can’t wait for more!
  • What the heck – I got a free pizza for taking a survey!

Now, I know that most of this is pure Twitter spam***** and not every Tweet is read or even seen by the same number of people, but I do think the increasing predominance of ploys to sign up people for paid surveys on networks like Twitter is a sign that term survey is being corrupted in a way that, if it does not contribute to declining response rates, surely doesn’t help matters.  They leave an impression and if these are the messages some of our prospective respondents have in mind when we contact them with a survey request, we are facing an even steeper uphill battle that we might have thought.

So, this leads us back to the classic survey methods question: what should we do?  How do we differentiate the “good” surveys from the “bad” surveys among a public who likely finds the distinction less than salient and won’t bother to read a lead letter, let alone open a message that mentions the word survey? Should we come up with a new term?  Does study get across the task at hand for the respondent?  Would adding scientific before survey help keep our invitations out of trash cans, both physical and virtual?

What are your thoughts on the term survey? Leave a comment here, or discuss on your favorite listserv or social media platform.  If you do, I promise not to send you a survey about it!

*scary=the degree to which lower response rates equate to lower accuracy, which isn’t always the case

**Personally, I sympathize with respondents when I get a survey request on my receipt every time I buy a coffee or sign up for a webinar.  “Enough already with the surveys!  I’ve got surveys to write!”

***not all people, and not all kinds of people, but still many!

****A few years ago, Sara Zuckerbraun and I looked at the portrayal of surveys in a few select print news media.

*****Late 2012 appears to have been a golden age for Twitter spam about paid surveys.

Share
  • Twitter
  • Facebook
  • LinkedIn
  • Digg
  • del.icio.us
  • StumbleUpon
  • email

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>