Cards from Shutterstock
At the beginning of any information design exercise, it is normal to be confronted by a very long list of potential subjects to include. The challenge is to organise this information in a way that is useful and meaningful for the users of the system.
While careful investigation and analysis of the information may reveal some clues, it can be virtually impossible to determine which topics should be grouped together.
The difficulty in organising the content stems from a lack of knowledge about how real users make use of this information. Without this, any exercise in information design is a purely theoretical one.
A card sorting session can go a long way towards resolving this problem.
What is card sorting?
In essence, card sorting works as follows:
- Write down each topic on a filing (index) card.
- Give the pile of cards to a number of users and ask them to group the cards into piles.
- Collate the results, and make use of them when completing the information design.
This is a very simple, and often very effective, method of working with users to come up with a usable design.
It should be noted, however, that a card sorting exercise does not produce a finished information design. There are many other inputs into the information design process, such as:
- business requirements
- strategic directions
- technical goals and limitations
- usability guidelines
All of this information must be taken into account when structuring your content. That being said, the card sorting will give you an excellent starting point for this work, as it will inform you of how real users think.
Why use this method?
There are a number of benefits to using card sorting:
- Simple, and well understood.
- Cheap to use (the only real cost is the preparation time).
- Quick to apply, which allows more users to be involved.
- It draws out the natural groupings of information in a way that avoids directly asking users.
- Involves the users in the design process, and helps to demonstrate that the system will be created with the users’ needs in mind.
These last two points are further discussed in the following sections:
Improving on the questionnaire
Surveys are an oft-used method to obtain information on user behaviour and opinions. While surveys are simple to construct, they suffer from the same limitations as the other methods that directly query the users.
That is, you will tend to obtain:
- what the users believe, not what they actually do
- what you want to hear, not the truth
- too much information from those who are coping with their jobs, and too little from those who aren’t.
By replacing direct questioning, card sorting generally avoids these issues (although they are never fully resolved). It is for this reason that the approach is so widely used.
The use of a group format also has considerable benefits. Often, the participants will bring to the session quite different opinions. Through the discussion and eventual resolution of these differences, it becomes possible to identify a workable structure.
The group also provides an energy and enthusiasm that makes light work of an otherwise tedious task.
Involving the users
Card sorting sessions are an important opportunity to involve the actual users in the design process. This makes them feel involved in the project, and emphasises that the end product will be built to meet their needs.
It also provides a forum in which the project can be ‘marketed’. Done successfully, this increases user enthusiasm and reduces any resistance to change when the system is implemented. It also allows realistic and accurate information to be conveyed regarding the timetable and scope of the project.
About this paper
This paper outlines a step-by-step approach to preparing and running card sorting sessions. Guidelines on how to analyse and make use of the results are also provided.
What is written here is based upon our practical experience of using card sorting to design intranets and online documentation. Hopefully these tips will allow you to learn from our mistakes and discoveries.
Preparing the topics
The first step to conducting a card sorting is to determine the list of topics. This list should be drawn from a wide variety of sources:
- existing documents and manuals
- existing online content
- descriptions of business groups and processes
- planned applications and processes
- potential future content.
The net should be thrown as wide as possible. In this way, it becomes possible to create a structure that will not just work for the present content, but for future material. Adding new content should require minimal reworking if the structure is correctly designed.
Creating the list of topics
It is actually quite tricky to come up with a workable list. There are a number of issues that need to be covered:
- The length of the list needs to be manageable. Too few items, and there is little scope for the users to come up with a structure. Too many items, and the task becomes daunting and confusing.
- Existing structures should not be reflected in the description and selection of the topics. It is important not to “pre-judge” the structure by leaving clues that will lead the users back to the current way of arranging the content.
- Words such as “manual” and “guide” should not be used in the topic descriptions. These reflect the current formats, and don’t relate to the actual content.
- The hardest challenge is to find the right level of detail. That is, how much information is encompassed by each item on the list. A topic such as “human resources” is perhaps too broad, while “rate of leave accrual” is probably too specific. As a general guide, each item on the list should be self-contained, and cover one logical group of content.
- Avoid including “grouping terms” (such as “Company News”). The whole purpose of the card sorting exercise is to make the users group the cards.Any headings that you place in the list will tend to encourage the users to place other cards under them, instead of other groupings which may be more natural.
- The terms used in the list must be meaningful to the participants in the session. This should be checked by people knowledgable about the working environment and subject matter.
The general rule when selecting topics for inclusion is to ensure that they are “neutral”, and they don’t direct the users towards a particular structure. After all, there is no point in running a card sorting exercise if all you are going to get back is what you starting with.
It is always a good idea to have several people collaborate on the list. This will help to double-check that the entries are meaningful and workable.
Also, if you are running a number of card sorting sessions with different user groups, make sure that you revise the list between sessions to take into any account lessons learned.
Preparing the survey
The card sorting session provides an ideal opportunity to survey the users about their current usage of information. This will help to guide you when selecting which information to start working on.
We have had success with surveys in the format of:
- A front page introducing the project and the card sorting session.
- A survey to identify how frequently the topics (identified above) are used.
- A final page, asking some general questions about the user as well as requesting their comments and feedback.
The front page
The general style of the survey should be kept light and friendly. This is an opportunity to show the users that you have their interests at heart. The front page introduction can be used very effectively to deliver this message.
Some information should be provided on how to complete the survey. More importantly, however, the introduction can “market” your project. By highlighting the participation of the actual users in the design of the system, you can go a long way towards reducing or eliminating the normal resistance to change. (This should be the first of many times that you involve the users.)
As a side note, if you have access to someone with desktop publishing skills (either within your team or elsewhere), you may wish to consider “prettying up” this document. If a friendly but professional style can be presented, it is likely to be received more warmly by the readers.
In preparing for the session, you have identified a list of potential content. While the aim of the session is to structure these topics, it is also useful to know how frequently the information is being used at present.
This can be obtained via the use of a simple survey which asks the users to rank how frequently they use each item.
When you provide the survey to your participants, it should be emphasised that you are interested in how frequently the information is used, regardless of the source. The participants should therefore consider all ways of obtaining the information, such as:
- paper manuals
- colleagues, supervisors or managers
- personal notes, resources, etc.
- computer systems
Regarding the format of the survey itself, we have found a simple tabular layout works quite well:
|How frequently do you use these topics
(1 = very frequently; 5 = never)
|1.||Internal phone directory||1 . 2 . 3 . 4 . 5|
|2.||Staff leave allocations||1 . 2 . 3 . 4 . 5|
|3.||Events||1 . 2 . 3 . 4 . 5|
|4.||Tender information||1 . 2 . 3 . 4 . 5|
|5.||Sales histories||1 . 2 . 3 . 4 . 5|
|6.||…||1 . 2 . 3 . 4 . 5|
We’ve chosen a very simple scoring system (from 1 to 5), without specifying exactly what is meant by each number. While this may lead to some variation between user’s responses, the survey is only really designed to draw out the 1’s and 5’s.
Note the number against each topic: the use of this will become apparent later, when you document the outcome of the session.
Finally, leave some blank lines at the end of the list for the participants to add any topics they believe have been missed.
A small survey at the end of the list of topics provides the opportunity to obtain further information about the users and their needs. While it is important to keep this section brief, it is nonetheless possible to ask questions such as:
- What is your job role?
- How long have you acted in this role?
- What are the main methods you use to find information?
- What are the most common pieces of information that you use?
- What other information would you like to have online?
- Do you have any other feedback or suggestions?
The first two questions will help you to correlate usage with levels of experience. The second two questions provide a double-check on their answers to the survey, while the last two questions are a common courtesy.
Creating the cards
Each topic on your list should be written on a index card (sometimes called a filing card). Make sure that you also write the topic number in a corner (keep it small, so as not to distract the users from the actual topic titles).
Regarding the right cards to use, use whatever is on hand, as long as it is a reasonable size. If you need to purchase some cards specifically for this exercise, we would recommend 76x127mm cards (or thereabouts). These are large enough to be easy to write on and handle, but small enough so that you can fit a fair few on a table.
It is also possible to do a card sorting exercise with Post-It notes, but is our experience that cards are a bit more durable, which makes them easier to handle.
Note that you will probably need at least a hundred cards for each session, if not more. (This is once you include the additional cards written up during the session.) So buy in bulk, to avoid running out at a vital moment.
As a final note, it should not be necessary to highlight the importance of writing clearly and readability on the card. Please, no last minute scribbling!
Selecting the participants
The attendees at the card sorting session must be the actual end-users of the system you are building. This cannot be emphasised too strongly.
There is no point in doing card sorting with a group of managers: they are not the end users. (Unless of course the system is designed specifically for management use.)
Where there are a number of distinct groups of users, hold one session for each. Each session must only have a single group of users, to ensure that the structure is representative of their specific needs. This allows you to compare the results from the different user groups and identify the differences and similarities in their requirements.
A card sorting session must have at least four participants. Any less than this and there is insufficient discussion and disagreement. Without this healthy debate, it is hard not only to come up with meaningful results, but also to finish the session at all.
More than eight users, and the group becomes more difficult to manage. If nothing else, it is hard to find a table large enough so they can all fit around it. Ideally, all the participants can reach the cards on the table, although this is not always necessary (usually one or more users will “take charge” of placing and moving the cards).
Preparing for the session
You will need to gather a number of items, in preparation for the card sorting session:
- the cards, and enough survey sheets for all the participants
- blank cardsThese will be required for both for adding missed topics and for writing the heading cards (see later for a description of heading cards).
- a thick marker pen (for writing on the heading cards)
- a number of rubber bands (for keeping the cards in order at the end of the session).
- A Session Participation FormThis is filled in by the users at the session (you may not know who they are going to be in advance), giving their names and job roles.
- A notepad and pen, for recording the participant’s verbal feedback.
You should also make sure you understand what is meant by each of the cards (you will almost definitely be asked), or ensure that you have someone sitting in on the session who knows the subject matter well.
Running the session
Introducing the session
Before starting the session itself, it is important to introduce yourself and to provide some information about how the session will work.
Walk the users through the card sorting process, and show to them how each stage works. It is important to highlight to the users that they should organise the cards in a way that works for them. Emphasise that they are to forget everything they know about how the information is currently organised – this is their chance to structure the content to meet their needs.
This is also an ideal opportunity to discuss the project as a whole. Assume that the users have been told nothing about the project (this is commonly the case), and start by discussing the overall goals and aims. Fill the users in on what will be delivered by the project, along with a realistic outline of the project timetable.
The card sorting session is a not-to-be-missed chance to ‘market’ the project. The more enthusiastic the users are, the lower the resistance to change when the system is implemented.
Filling in the survey
After you have introduced the session, hand around the survey to the participants. Explain how to correctly fill in the survey, and then give them enough time to comfortably complete it.
Be prepared to answer questions about the meaning of the various topics that you have identified. If you don’t have the knowledge, ensure that a subject-matter expert is present at the session who can field these questions.
Also ask the users to write down any topics they think have been missed from the list. (Some empty rows should be added to the end of the survey for this purpose.)
Having the users fill in the survey at the beginning of the session may seem like a waste of time. There are several important benefits, however:
- It familiarises the users with the list of topics, which will come in handy when they are sorting the cards.
- It allows the users to ask questions about the survey, which helps to eliminate sources of confusion.
- It ensures that 100% of the surveys are completed and returned to you.
While the users are filling in the survey, take the opportunity to sit back and relax: all the hard work in a card sorting session is done by the users.
Sorting the cards
To summarise your preparation for the session:
- You have identified a list of topics to be organised.
- Each topic has been written on a separate filing card.
- You have gathered together an appropriate group of users.
- All the necessary supporting materials (such as the survey and blank cards) have been brought to the session.
You are now ready to run the card sorting session.
Place the cards in a single pile in the middle of the table, and ask the users to sort the cards into piles. Typically, this means that one user will volunteer to read each card, and to place it according to the consensus of the group.
When a pile is finalised, ask the group to nominate a label for the pile. Write this on a blank card, and place on top of the pile. This is the most important aspect of the session: the labels become the menu items (or subject groupings) in the final system.
Be prepared for a reasonably-lengthy session: the participants are likely to want to rearrange the cards more than once, and some labels will undoubtedly be changed.
Maintaining the energy
The card sorting is likely to start slowly at first, as the participants get the hang of how it all works. In the middle of the session, progress will be relatively rapid, as the ‘easy’ cards are grouped together. Towards the end, only the ‘difficult’ cards will be left, and things will slow down.
During each of these three phases, it is important to ‘facilitate’ the group’s progress. At first, this means encouraging them, and reminding them how the session works.
During the bulk of the session, the challenge is to avoid getting distracted on issues. When working out where to place a card, it is natural for the group to engage in a discussion about the topic written on the card. While this is often useful (see ‘taking notes’ below), if necessary you must bring the group back to the task at hand: sorting the cards.
Towards the end, you may be required to help the users complete the task, as they struggle with the more difficult topics. Some strategies include:
- Allow a ‘to be determined’ pile to be created.
- Where there are two choices that are hard to select between, have the group choose one of them, arbitrarily if necessary. Inform them that ‘not everything has to be in the perfect location’.
- Discard cards altogether, if the meaning of them is not clear to anyone present.
While these methods can be used to ensure completion, it is important during most of the session to ‘force’ users to group things properly. A ‘to be filed’ category should not be allowed at the start of the session.
Keep a notebook handy throughout the sessions. The users will undoubtedly raise some important issues during the session, which should be noted down for future reference.
You might also want to side-track briefly to discuss some of these issues. While this can be very valuable, be careful to avoid overly distracting the session.
Keeping it friendly and fun
One final tip: keep the session light and casual, and avoid using acronyms, jargon or ‘buzzwords’. This is your chance to show the users that the project has a ‘human face’. The card sorting session should also be fun: a good distraction from the day-to-day activities of the user.
Most of all: be honest and open with the users. This will gain you considerable respect, which will be important later in the project.
When all the cards have been sorted, and a general consensus has been reached, wrap up the session. Thank the participants for their involvement, and highlight again the importance of their efforts.
At the end of the session, write down the groupings identified by the participants. This is where the topic numbers come in handy. All you need to do is to write down the list of headings created by the users, and list the topic numbers beside them. You can then expand the full topic titles later, at your leisure.
Carefully bundle up the cards, keeping the groups intact (so you can check your transcription later), and bind them together using a couple of rubber bands.
Analysing the results of the card sorting
You now have the raw results from the session. The challenge is to present this information in a way that is meaningful. There are many different ways to achieve this.
One method is to present the information in a tabular format, listing the headings and the topics underneath them. While this is quick to create, it can be quite hard for a reader to gain a clear picture of what the structure looks like. We would therefore recommend against this approach.
Instead, we would recommend a graphical presentation that displays a simple “mock-up” of what the structure would look like, if implemented. We have used diagrams like the following:
Depending on the number of topics you have included in the session, you may need to run to two pages.
Once the results have been gathered together into an easy-to-read format, you are then ready to make use of them as part of your overall information design process.
While every card sorting session will generate different results, there are a number of common themes:
Similarities vs differences
It is often very revealing to compare the results of card sorting sessions with your different user groups. If a common structure appears across a wide range of users, you can be confident that this is the right way to go.
If there are differences, investigate why. This will probably identify differences in the way the information is used, or the types of activities that the different groups conduct. Either way, these issues will need to be taken into account in an overall information design.
The card sorting sessions serve as input into your information design process. This will generate an overall structure for your information, as well as the major menu items, navigation and more.
The card sorting itself is just another input into this process: it does not generate the final structure. For example, there will undoubtedly be areas where your users disagree about the subject groupings.
The card sorting can, however, identify some important trends, such as:
- Do the users want to see the information grouped by: subject, process, business group, or type of information?
- What are the most important items to put on the main menu?
- How many menu items should there be, and how deep should it go?
- How similar or different are the needs of the users throughout the organisation?
With these sorts of questions answered, you can tackle the information design with much greater confidence.
Analysing the results of the survey
Analysing the survey results is generally just a process of collating all the answers and presenting them in some simple and meaningful fashion.
The usage survey will generate a large number of scores (ranging from 1 to 5). These should be entered into a spreadsheet and analysed there. For example, an average score for each topic is clearly a useful piece of information.
Beyond this, you could explore a number of different graphs to highlight particular aspects of the results, or perform more complex statistical analysis. In general, however, the most useful form of presentation is the one that allows a reader to rapidly identify the most, and least, used topics.
In some form or another, the general feedback filled in by the participants needs to be typed in. If there are modest number of responses, consider just listing these (perhaps as bulleted list).
If there are a large number of responses, it will be necessary to summarise them. Needless to say, care should be taken to ensure that the summary reflects the overall intent of the responses.
Make sure that both positive and negative responses are included. User dissatisfaction is generally more important to recognise than satisfaction, as it has the potential to generate considerable difficulties later in the project.
Once you have done all the preparation, run the session, and analysed the results, you are almost finished. A few final steps neatly completes the exercise:
- Gather together the results, and create a small report. This should include the following information:
- the date of the session
- who participated in the session
- a brief summary of the activities conducted
- a description of the survey questions asked
- the analysis of the results
- any other comments and conclusions drawn
This report should be self-contained, and easy to read. Put some extra effort into this, as this is another opportunity to market your project.
- The report should be sent to a number of key groups:
- your manager (of course)
- the manager of the participants
- the participants themselves
The actual participants are the most important of these, and should be sent the feedback as soon as possible. If necessary, the report can be edited to remove any conclusions or comments that might cause an issue.