Quantitatively test the effectiveness of your home page
Staff should be able to confidently, quickly and accurately step from the home page of the intranet towards the information they require. If staff can’t achieve this without resorting to search, the home page needs to be redesigned.
As discussed in Full site redesign? Start by addressing the home page, many home pages fail because they are exclusively devoted to exposing new and useful content. Both of these have a place on a home page, but they should be kept in proportion with its role as a gateway to all site content.
This article explains a quick and effective technique for assessing whether your home page is an effective gateway to site content.
Define a set of information-seeking tasks
The first step is to define 10-15 common information-seeking tasks. Search engine logs and previous user research are both good resources for discovering what people look for on the site. Don’t use site usage statistics as these will only show what the site effectively enables people to find, which will bias the test.
Examples of information-seeking tasks include:
- find the sick leave form
- find the annual report
- find the process for performance appraisals
Try to avoid tasks where there is already a direct content link on the home page. For example, if in the ‘Useful links’ area of the home page there is a link to ‘sick leave form’ do not include this as a task. This is an evaluation of the effectiveness of the home page as a gateway to the site content, not whether the participant notices a link.
Define right choices
Once a set of tasks is defined, evaluate the home page for each task and make a note of which links could be selected to step the participant towards the information they seek.
For example, the performance appraisal process may be found in the policies and procedures section of the ‘Human resources’ area of the site, thus clicking on either the ‘Human resources’ navigation heading or the ‘Policies and procedures’ link in the ‘Useful links’ section would constitute a correct selection.
Arrange a set of meetings
To test the home page, arrange a number of one-on-one meetings with staff from across the organisation. Each meeting will only take around 10 minutes, so the degree to which the session needs to be formalised will vary greatly between organisations.
The meetings can be conducted either in the working environment of the staff member or in a meeting room. Wherever the test is conducted must have a computer with access to the site.
Ten meetings will provide an indication of the effectiveness of the home page, but 20-30 sessions are needed to provide statistically valid findings.
Conduct the test
At the beginning of each session explain that the organisation is evaluating the effectiveness of the home page. It should be made clear that this is not an evaluation of the participant, but of the home page.
Sit the participant in front of the home page of the site. For each task ask them to point (without clicking) at where they would go as a first step towards finding the information relating to the task.
Make a note of the participant’s selection, but do not indicate whether or not they are correct.
Participants should be directed away from selecting search, site map or A-Z index, as these provide a link to all of the site’s content and thus provide meaningless data.
Evaluate the results
To evaluate the results, identify the percentage of successful selections made by the participants, i.e. the number of selections that matched the right choices divided by the total number of attempts.
If less than 80% of staff select the correct option, it is time to consider redesigning the home page. If the figure is below 60%, the home page needs to be redesigned as a matter of urgency.
(For an overall methodology for developing or redeveloping an intranet, see the