Last week in London I had the pleasure of presenting at the Interactions 2011 conference. This was my opening keynote, outlining a vision for the role that intranets should play within organisations.

Key points:

• There are three broad “life stages” for intranets (see Introducing the essential intranet, noting that “new intranets” are now called ad-hoc).
• Intranet teams start asking lots of “what” questions (what is the intranet for?), then “how” questions (how do we make search work?), and eventually “why” questions (why do we have an intranet?).
• Clear business value and impact can only be demonstrated when intranets are essential.
• I then gave a tour of some of the recently-announced 2011 Intranet Innovation Award winners, including Framestore, UK Parliament, QUT and CRS Australia.

The authors used a sample base of around 150 managers to try to establish what the reasons for intranet user acceptance might be, using some classic user acceptance models. The paper is exceptionally well written, and is one of a series of papers that these authors have published. Their main conclusions (as set out in the paper) are that the top management of the port industry must continue to commit to intranet implementation as its use has been proven to significantly affect job performance among the managers in general irregardless of their backgrounds.

Another activity at the intranet strategy workshop we held at a major bank explored this topic, brainstorming all the ways the business case could be created.

This is what the group came up with in just a few minutes:

• Strategy document
• Demonstrate savings
  • $
  • productivity
  • call handling time
• Practical (short-term) roadmap
• What they’re missing out on
  • examples
  • stories
  • scenarios
  • case studies
• SWOT analysis
• Mockups
• Proposed architecture/solution design
• End concept/vision
• Target individual executive motiviations and drivers
• Surveys
• Focus groups
• Video interviews with staff
• Target business drivers
• Usage/search statistics
• Usability testing of current site (with videos)
• “How long could you last without the intranet?”
• Call centre metrics
• Personas
• Emotion
• Evidence
• Risk

There’s a lot that could be said about this, but the clear message is: there are many ways of creating a compelling business case. Go beyond the ordinary, and carefully target the priorities of senior management. Don’t feel constrained to just join-the-dots in a standard business case template and leave it at that.

Some further reading:

• Finding compelling examples to build the intranet business case
• Educating the executive about the intranet
• Planning to demonstrate success
• Target emotions in the business case

What approaches have worked for you?

Alternatively, what are your war stories of what doesn’t work?

Below is a short list of the KPIs that I put together for them on the basis that their home page functions very much as a digital newspaper. These KPIs are intended to provide an indicative view of quality in addition to drawing attention to quick wins and improvement opportunities. Importantly, as an intranet evolves into a more collaborative, digital workspace KPIs such as these would need to evolve quite dramatically.

In a previous article we explained How intranet teams should spend their time. This article outlines some metrics that teams can use to measure their own. These are not the same as the ways to measure the effectiveness of the intranet itself.

Intranet team responsibilities fall into three main areas:

• day-to-day maintenance
• projects and new initiatives
• managing relationships with staff and stakeholders

Day-to-day maintenance

Day-to-day maintenance of the intranet is core business for any intranet team. Most teams will have a variety of valid measures for this, including:

• timeliness of content updates
• number of complete support calls
• number of training sessions run for the publishers
• adherence to service level agreements
• number of content pages reviewed
• up time for the intranet

[CM Briefing 2010-03 written by Catherine Grenfell, read the full article]

What follows is an extract of 2 pages from the report, pages 96 for adoption indicators and page 102 for core business indicators. Note that the number (n =) at the bottom left of each chart shows how many participants out of the 283 organizations are using this type of indicator.

Failure to measure your intranet’s performance above and beyond simple analytics (e.g. HITS and page views) is a failure of responsibility (particularly in this economy where most organizations are looking cut costs wherever they find them).

While measuring return on investment on many enterprise 2.0 initiatives can be difficult, it is also very useful for a variety of reasons. The business case should ensure that the success measures are in line with the enterprise’s overall business strategy. Consideration of the costs and benefits are essential in order to make sure the initiative is successful and delivers what is needed. It can also provide concrete evidence of the value to support further investment.

This leads to the requirement for an intranet business case, ideally with clear measures that quantify the value of the intranet and the benefits it delivers.

In tough economic times, a very bright spotlight is shone on the intranet and the resources it requires. All projects will struggle to proceed without demonstrating how they tangibly improve the bottom line.

This intensifies discussions around intranet measures and metrics. It also produces a new round of intranet business cases being put forward to senior management.

In practice, creating an intranet business case can be hard. In all too many cases, intranet teams put forward their existing plans, and then try to justify them after the fact.

The difficulty is that many of the ‘typical’ intranet improvements are hard to quantify. What is the real dollar value of improving intranet navigation, or cleaning up content?

The arguments than can be put forward to justify these projects tend to be weak, and what is needed is a fresh look at how to plan intranet improvements.

This article will put forward the case that intranet teams can plan from the outset to demonstrate success. By targeting and packaging intranet improvements in the right way, identifying concrete benefits and metrics becomes easy and effective.

[Read the full article]

It’s a popular question from the Chief Financial Officer these days… if we are going to invest ‘x’ amount of dollars in your project, what will the company get in return?

This post identifies ways that intranets can add bottom line value to an organization. People putting together a business case to obtain funding for intranet improvement initiatives will find this post useful.

I think we overemphasize metrics when it comes to usability. They often introduce problems in terms of the validity and reliability of the data. First, ask whether a metric is a valid measure of usability. For example, the amount of time it takes to perform a task is often a usability goal—as in your example Six out of ten novice users shall perform task X in Y minutes. What if all of the participants took more than Y minutes to complete the task, but everyone said, ‘This is the best application for doing this task I have ever experienced. I’m recommending this product to all my friends.’ Does the product fail to meet the requirements regarding that task? What if all participants perform the task in less than Y minutes, but everyone says, ‘That was a horrible experience. I would never buy this product.’ Does the product satisfy requirements?

Let’s say that Organization A has 30,000 employees. Let’s say each employee does a particular task on average 50 times a year. Let’s say it takes 10 minutes longer than it should because it’s badly managed. This is costing the organization 15 million minutes a year in lost productivity. That’s 250,000 hours. Or 33,333 days. Or 150 person years. It’s significant.

Now, I wrote a piece titled 25 reasons why saving time on your intranet is a bad metric, so I’m obviously not a fan. I’ve also commented on Gerry’s piece, in an effort to shift the discussion to more productive topics.

Gerry and I agree on so much, it’s nice to have a good debate online for a change. Keeps the internet interesting.

Where do you stand on this perennial debate?

If you are running a knowledge management programme then you know that you do good work. Whether it is involves communities of practice, learning from projects and operational activities, or documenting organisation knowledge in a myriad of ways (from traditional databases to blogs and wikis), you know what you do is important for your organisation.
However simply knowing this is not enough. At some point, you will have to justify what you do to those above you. If you don’t, you may not get to do it any longer. Often these justifications have to take a financial form.

We’ve been talking about this for a while now. I recently argued that “minutes saved” on the intranet is meaningless, unless it’s backed up by real numbers, and not just theory. 40% of the scoring for the Intranet Innovation Awards comes from business value, with only 30% relating to novelty.

One of the most concrete ways of demonstrating improvements is to show before-and-after figures. Now, we’re reasonably good at measuring things after the project, such as via usability testing of new designs. How often do we measure the intranet before we make changes?

Without the before figures, it is very hard to quantify the concrete benefits. While this is “science 101″, the pressures of the typical workplace makes it difficulty to find time, and to justify, conducting research before the project actually starts.

Maybe if our senior managers don’t expect these kinds of metrics, we can save the time and effort involved in measuring both before and after. I would still argue, however, that we will be super-stars if we really can quote figures that really make an impact.

Once we demonstrate these types of returns once, getting resources will be dramatically easier. With greater management confidence in our performance, perhaps we won’t have to measure to this depth again, at least not for a little while.

So how can we do these types of measurements in practice? I’ve listed a few ideas…

Some lightweight approaches:

• Measure usage levels before the change, putting in a stats package if required.
• Conduct a “user satisfaction survey” before any major change, and then repeat 6 or 12 months after.
• Use the Intranet Review Toolkit to show improvements to core intranet heuristics.
• Collect anecdotes and examples of staff frustrations with the current solution, and then demonstrate increased satisfaction after the improvements.
• Assemble a before-and-after “portfolio”, thereby ensuring that the old designs are not forgotten, and the new designs not taken for granted.
• Record videos of users struggling with the current system, as part of lightweight usability testing.

More robust approaches:

• Measure end-to-end task completion, before and after the improvement (“it took 3 people a week of elapsed time to process a loan application, now it can be done in two hours”).
• Conduct quantitative before-and-after usability testing, focusing on key tasks that operational staff use the intranet for (and not just generic tasks).
• Find ways of saving actual money (although this is often extremely hard).
• Demonstrate improvements to key business metrics that are already being measured (improved product sales, increased customer satisfaction, reduced helpdesk calls, reduced call handling time in a call centre).

Also see our earlier article Metrics for KM and CM for more ideas on intranet metrics and ROI.

Any other lightweight or robust approaches you’d recommend?

I met up with some intranet managers the other day and there was a lot of talk of metrics. How many page views, how many users, who had clicked what and so on.

The conversation swiftly turned to which software each of us was using. However, I am sure that this was missing the point of metrics.

(British Airways won a Gold Award and Commended in this year’s Intranet Innovation Awards, so Allen’s blog is definitely worth following!)