Research

Remote observation testing

Instead of watching users interact with apps/sites/prototypes in person, send them a list of tasks and questions, and they'll record their screen and voice as they go through them.


Good for:

  • testing the broad usability of a site
  • when you want to observe simple tasks that can be easily explained in writing
  • speed: testers work more in the evenings and weekends, so you can set tests up overnight/over the weekend and get results the next working day
  • as a test run for other kinds of remote testing, before you launch the tests at scale

Bad for:

  • when you need to test with specific types of people
  • with a prototype that is minimally interactive: it often confuses the testers, even if you warn them
  • follow-up questions: you can't ask any!
  • if you want videos of people's faces – you don't get any

How to use

  • Write a list of instructions, which can be questions to answer or tasks to try doing. These should take no longer than 15-20mins. Be as clear as possible, remembering that these instructions will be the only thing the tester has to refer to, and number each task. Consider including any really key questions in a survey at the end of the test, to be sure you get a clear answer.
  • Put your list of instructions into WhatUsersDo (or alternatives) and select the demographics for the people you'd like to complete it.
  • If you can, run the test with just 1 tester first, to check your instructions are clear enough. Then request more testers.
  • Check back to see if the tests are complete. If they are, you'll be able to view and download videos of people doing the test and speaking aloud, as well as creating clips and downloading just those.
  • If a tester has not followed the instructions you set, you can reject the video and get another for no extra cost. So, if you have a hard deadline, check each video as it comes in to make sure it's ok.

Outputs

  • Videos of people completing the tasks with audio recordings of their voices
  • Survey responses and/or card sorts (if you ask for them)

Software/services

Timeframe

  • About 1 week elapsed time for booking
  • About 2-4 hours to prepare discussion guides and brief the recruiter
  • 20mins to 1 hour per session

Cost

£35/test

What you need

  • A prototype that can be shared via a URL

Advisors

  • Katy
  • Kat

Notes

  • Testers are really just in this for the money, and so will try to go through your instructions quickly. They're also notably less sympathetic than people are when they test something with you in the room – that can actually be pretty good. For instance, if you are concerned that something is too hard to use, remote testing will give you a better idea of whether people will persist long enough to learn how to use it.