Should I send a survey? Sure, if…

You may have made it past my first and second attempts to convince you a survey is the wrong tool for you. Because, you know, it’s a valuable tool. I rail against it because it’s too often used to avoid talking to real humans, or as an inadequate substitute for quantitative data.

On my team we frequently use surveys to fill in a specific kind of knowledge gap. Specifically, quick surveys are great when:

  • You know the who. You’ve targeted the type of person you need to research — preferably by behavior or job title/social identity role as opposed to demographics.*
  • You know the what. You’ve already focused in on specific questions. You aren’t in exploratory/digging mode — you know what you need to know.
  • You know the why. Your questions have straightforward answers that don’t require explanations or write-in responses.

For example, suppose you’re seeing dwindling usage from people who’ve downloaded your iPhone app. Many of your App Store reviews complain about your photo-taking feature (which you didn’t realize anyone felt very strongly about).

You could start scheduling interviews — but it’d be faster to start with a survey to see if this photo issue is widespread or just the voice of a vocal minority.

  • You know the who: people who’ve downloaded your iPhone app.
  • You know the what: did people see your app as a tool for taking photos? had they tried the feature? how often do they take photos in other apps?
  • You know the why (or in this case, the ‘why’ might not matter): if people need to take photos and you’ve made the workflow worse, that explains attrition.

You could shoot out a 4-5 question survey to a hundred people and you’d likely have twenty responses within a day. That’s not a huge sample size — but it’s enough to know whether 2 customers care about the photo feature, or 19 of the 20 respondents care.

If no one seems to care about the photo issue, you’d likely want to do some interviews or some in-person usability testing to figure out what exactly the issue is.

If everyone cares about the photo issue, you could probably start with an internal teardown (most likely, your design team could easily identify issues and start mocking up fixes immediately, vs. waiting for ‘proof’ of what was wrong.

* What’s wrong with demographics? They’re almost always a bad proxy for something you can measure directly. Over-65 woman means what, exactly? Is that a lazy shorthand for fears technology, living on a fixed income? Because I know plenty of iPhone-toting grannies with cash to spare. If you want to study people who behave X way, then look for evidence of people behaving X way.