CMb 2006-09

Quantitatively test the effectiveness of your home page

Written by , published June 1st, 2006

Categorised under: articles, intranets, usability & information architecture

Staff should be able to confidently, quickly and accurately step from the home page of the intranet towards the information they require. If staff can’t achieve this without resorting to search, the home page needs to be redesigned.

As discussed in Full site redesign? Start by addressing the home page, many home pages fail because they are exclusively devoted to exposing new and useful content. Both of these have a place on a home page, but they should be kept in proportion with its role as a gateway to all site content.

This article explains a quick and effective technique for assessing whether your home page is an effective gateway to site content.

Define a set of information-seeking tasks

The first step is to define 10-15 common information-seeking tasks. Search engine logs and previous user research are both good resources for discovering what people look for on the site. Don’t use site usage statistics as these will only show what the site effectively enables people to find, which will bias the test.

Examples of information-seeking tasks include:

  • find the sick leave form
  • find the annual report
  • find the process for performance appraisals

Try to avoid tasks where there is already a direct content link on the home page. For example, if in the ‘Useful links’ area of the home page there is a link to ‘sick leave form’ do not include this as a task. This is an evaluation of the effectiveness of the home page as a gateway to the site content, not whether the participant notices a link.

Define right choices

Once a set of tasks is defined, evaluate the home page for each task and make a note of which links could be selected to step the participant towards the information they seek.

For example, the performance appraisal process may be found in the policies and procedures section of the ‘Human resources’ area of the site, thus clicking on either the ‘Human resources’ navigation heading or the ‘Policies and procedures’ link in the ‘Useful links’ section would constitute a correct selection.

Arrange a set of meetings

To test the home page, arrange a number of one-on-one meetings with staff from across the organisation. Each meeting will only take around 10 minutes, so the degree to which the session needs to be formalised will vary greatly between organisations.

The meetings can be conducted either in the working environment of the staff member or in a meeting room. Wherever the test is conducted must have a computer with access to the site.

Ten meetings will provide an indication of the effectiveness of the home page, but 20-30 sessions are needed to provide statistically valid findings.

Conduct the test

At the beginning of each session explain that the organisation is evaluating the effectiveness of the home page. It should be made clear that this is not an evaluation of the participant, but of the home page.

Sit the participant in front of the home page of the site. For each task ask them to point (without clicking) at where they would go as a first step towards finding the information relating to the task.

Make a note of the participant’s selection, but do not indicate whether or not they are correct.

Participants should be directed away from selecting search, site map or A-Z index, as these provide a link to all of the site’s content and thus provide meaningless data.

Evaluate the results

To evaluate the results, identify the percentage of successful selections made by the participants, i.e. the number of selections that matched the right choices divided by the total number of attempts.

If less than 80% of staff select the correct option, it is time to consider redesigning the home page. If the figure is below 60%, the home page needs to be redesigned as a matter of urgency.

(For an overall methodology for developing or redeveloping an intranet, see the
Intranet Roadmap.)

Tags: , , , , ,

Don't miss out!

Every month we publish two new articles, just as good as the one you've just read. Join our monthly newsletter, and we'll notify you when new articles are published, along with other Step Two updates.

(We won't use your details for any other purposes, or provide it to third parties.)



One Comment:

  1. Tracey commented on December 21st, 2011

    I use a visual click tester to perform this kind of testing – particularly useful if you have a web-based service so you can extend your reach past the limits of having a body in a room.

    However – you gain much more insight into how users interact with a page during a face to face session – so if time allows I run a series of one-on-ones and then distribute the same test out to capture a wider audience and regional variation.

    Any testing is (usually) better than none!

2 Trackbacks

  1. […] the quantitative effectiveness test and determine whether your intranet site cuts the muster. Find out how a quick and effective […]

  2. By UX Designer’s Quick Reference | butlerhouse on November 28, 2010 at 12:49 am

    […] Quantitatively Test the Effectiveness of Your Home Page – Iain Barker Up to Page Index […]