n-person user research used to be the only game in town, and as with most industry practices, its procedures were developed, refined, standardized, and then became entrenched in the corporate R&D product development cycle. Practically everything gets tested in a lab, hallway, or conference room nowadays: commercial web sites, professional and consumer software, even video games. But nowadays we’ve got remote usability testing.

Yesterday I posted my 10 tips for ambush ‘guerilla user testing’. Once you’ve got some footage of people using your website, you need to find the best way to present that back to the business. I like to make a summary clip reel of the testing, and here are my 8 tips on putting your clip reel together…

Over the last couple of years I’ve been practicing ‘ambush guerilla user testing’, which is basically the art of pouncing on lone people in cafes and public spaces, and quickly filming them whilst they use a website for a couple of minutes. It isn’t by any means a formal research technique, but you can soon build up a valuable clip library of initial reactions to you and your competitors’ products, and tease out interesting anecdotal evidence about the way that people use and feel about your website.

(While the post assumes a public-facing website that is being tested, there’s absolutely no reason why the same approach couldn’t be taken with staff.)

The motivation for this article is to help UX researchers keep an open mind about online usability testing. There are some researchers who have been using this approach for years and find it useful (in certain situations). Others are new to it, and wanting to learn more about its strength and limitations. Finally, some UX researchers have already formed an opinion about online usability testing, and deemed it not useful for a variety of (unfounded) reasons. I hope by exposing these myths, we (as a UX community) can evaluate this tool based on its actual merits.

As we started experimenting with “card-based classification” on paper, it became clear that, while the technique was simple, it was tedious to create the cards on paper, recruit participants, record the results manually, and enter the data into a spreadsheet for analysis. The steps were easy enough, but they were time eaters.

Until fairly recently, when designers wanted to test an idea or design, they sought out an outside usability agency or, rented a room, some expensive equipment and recruited users to come into an artificial environment to participate in a usability study. In the past few years, technology has become available that has brought the cost and complexity of user testing down to a level where it is available for anyone. In addition, it allows designers to be exposed to users, in their native environment, not a sterile lab across town. You can run a test in only a week for less than a few hundred dollars.

This is not always possible. Time and cost limitations occasionally result in corners being cut with testing often being the victim, because it is thought to take too long, or because the project team believe their experience enables them to create a design without testing.

But without input from the target audience, it’s likely the site will lose some users. Squeezing in some basic testing can make the difference between success and failure.

Why engage users early?

Ideally, users are engaged early in the development to ensure:

• user needs are met rather than what the project team thinks they need
• the time and cost of reworking an already designed site is minimised
• the site is both useful and usable

[CM Briefing 2009-18 by Rebecca Rodgers, read the full article]

Usability.gov is a one-stop source for government web designers to learn how to make websites more usable, useful, and accessible. The site addresses a broad range of factors that go into web design and development. The site will help you to:

• Plan and design usable sites by collecting data on what users need
• Develop prototypes
• Conduct usability tests and write up results
• Measure trends and demographics

The term “usability testing” often gets misconstrued by technical types, project managers and business analysts. It gets turned into a stale, rigid, bureaucratic affair. The old “unit, integration, system” mantra. It’s done as a matter of course, at the end of the gantt chart, to tick a box. That’s pointless.

One of the places we kept noticing this was when we watching people shop online. Asking a shopper to pretend to purchase (“Could you find a pair of shoes you might like to buy and put it in your cart?”) produced extremely different behaviors than when we recruited people who needed the product and gave them the cash to make a real purchase. In the former case, they went through motions and skipped steps that we didn’t see when they were considering and purchasing the product for their own true use.

It starts with any number of scenarios: Design and development have taken too long to produce a prototype, you need to release in three weeks, and you suspect there may be design flaws. You are trying to incorporate usability testing into an Agile development process. Or maybe you simply want to pare down your process to make it shorter and less expensive.