Understand the purpose of the feature or application that you’re about to test.
Why are we building this thing? When it succeeds, how will it benefit the company or product line? What benefit are end users going to get out of it? What are the regulations, expectations, and constraints? The user researcher should be able to answer all of these questions back to the product stakeholder.
What happens too often is that the user researcher doesn’t really know these answers. That hinders them from writing the most effective test scripts and surveys (and usually means that the product manager or other stakeholder takes too much ownership over writing those docs).
When the user researcher is deeply familiar with all of the “why” questions, they can work more independently in creating and conducting the tests. They know which tangents to follow up on and can ask really useful spontaneous questions.
Find out what the questions need to be answered by the testing.
What beyond “can they complete the core tasks” do you want to know? Are there questions you have around how users behave, attitudes they hold, competing technologies they use, their intent? The first user research interview you should be doing is on the stakeholder of the testing.
Task-based usability testing can often feel like it was conceived in a vacuum. If you’re wondering how likely it is that people will buy premium services on top of a product, testing the rote mechanics of going through the checkout flow will probably not answer that question. A combination of surveys and interviews may be a more effective approach.
Design the shortest possible test.
“Here’s a PDF report and links to the 5 hours of videos.” — WHOA. No one is watching those videos, ever.
I’ve gotten tons of value from long customer interviews (some of which I’ve recorded and some where I just took notes), but they’re hard to consume. When people don’t watch the results, they don’t act on the results. Better to do multiple, shorter, tests than long ambitious ones that are overwhelming.
Recommend changes based on test results.
Take a stand. How can you expect other stakeholders to make changes based on research if you aren’t willing to suggest them?
This is why my first point is so critical: without it, the user researcher won’t be equipped to make recommendations that make sense for the user AND the business. “Pure usability” rarely wins in a business context; it’s always a tradeoff.
Stakeholders won’t necessarily follow your results, but a strong stance gives them something to bounce ideas off. In most cases, product managers agree with some of our recommendations and use other bits to form a different — but educated! — opinion.
Tell a (SHORT) story with the results — juicy quotes, numbers, screenshots.
- Recommendations in bullet point form
- Quotes that support your assumptions
- Quotes that totally negate your assumptions
- Really [insert emotion] quotes!
- Screenshot or video still or pie chart (optional — these do take longer)
- X% of participants did X, Y, Z
That is the extent of most peoples’ attention spans. Sure, you can add in that link to the 5 hours of videos at the bottom. But your first job is to grab the first 15 seconds of your stakeholders’ attention and keep it.