One question that often arises early in the process is whether the same CMS should be used for the website and intranet, or whether two solutions should be deployed.

Historically, this question was often driven by a desire to reuse the public-facing CMS for the intranet. These days, the question is equally likely to relate to reusing the intranet solution (such as SharePoint) for the website.

While there are advantages to having a single solution, this is not a ‘slam dunk’ option as this article will explore.

[May article, read the full article]

I am more cynical about quantitative methods of evaluating vendors. It makes me think of people who buy wine simply based on what The Wine Spectator rating is (without considering the food they might drink it with, or what they actually like vs. what Mr. Wine Critic likes), or my single friends who have a checklist of what they will or won’t accept in a mate. “He must be this tall, have this color eyes, and make at least this level of salary.” Well, the world simply doesn’t allow you to pick partners based on a checklist, perhaps unless you’re Tiger Woods.

We strongly advocate a test-based approach to procurement, based on the value of scenarios; nevertheless, many enterprises want to apply a quantitative, spreadsheet-based assessment approach. In some cases these can become really quite complex. We have advised on dozens of scoring spreadsheets, and like to think we have seen them all.

This is a good post, and I agree completely with Alan’s recommendations. In the work that we do, we encourage our clients to use a super-simple scoring approach. This is enough to put some shape around the evaluation, without falling into the trap of over-complex formulas.

They have an incumbent provider of enterprise solutions, one of the large vendors, who they’d like to evaluate alongside other offerings. (And no, being strictly vendor-neutral, I’m not going to mention the vendor in question, or the client for that matter.)

It’s been a while since I’ve seen this product, so it seemed like an ideal reason to get in touch to obtain a full demo of their current CMS offering. Looking through my records: no, don’t have a specific contact person for this vendor.

Ok, I guess I’ll just have to ring them up. Now the fun really starts.

The call is answered by someone clearly not in Australia, who asks me the nature of my call. “I’d like to talk to someone on the <product x> team.”

“Well, I’m going to have to ask you some questions first.” I then get the 20-questions treatment, asking me the nature of the project, the budget (can’t tell you), the name of the organisation (can’t tell you), when the decision will be made (hopefully early next year). I explain our role in the marketplace, and highlight some of the notable clients I feel should grab his attention (such as Westpac, BHP Billiton, federal government agencies, etc).

20 minutes later, he’s written all this down, and promises to “pass it on”. Two days later, he rings me back to “check on the details”, and we go through the whole lot again.

A week later, I get a phone message from someone at the vendor. I return the call, and go through the 20 questions again, asked in a different way. I’m informed that if I can’t reveal who the client is, it’s “difficult” for them to provide me with an actual contact. They’ll look into it, and will get back to me in a few days. I’m assured that they “aren’t trying to be difficult”, but that isn’t the way it seems to me.

So a week-and-half later, I don’t know anything more about their product, and the clock is ticking. Fail.

It seems this vendor has very carefully put in place processes to prevent potential customers from talking to them, unless they’re deemed “important enough”. I certainly felt like I was the one who had to prove myself, instead of them trying to sell to me.

It’s a funny way of doing business, and it makes me wonder:

Are the big vendors hungry enough for new business?

I get one-hour responses from most vendors within Australia, and regularly receive demos from them. I’ve had vendor CEO’s showing up to walk through products. But not from the really big enterprise vendors, where it’s consistently hard to even find someone to talk to, let alone to get information from.

What have your experiences been?

In one case there was a comprehensive site redesign that included digital strategy, re-branding, and information re-architecture as well as implementing new functionality. In another case, the client was shifting to an outsourced model where a partner was to maintain the full infrastructure and assume all development responsibilities. In situations like these, while the software is important, the biggest risk is choosing the wrong partner to work with.

Regardless of which type of evaluation process is being followed, there is a final short-list of typically three vendors. These vendors are asked to give a structured demonstration, and then the final decision is made.

Too often I’ve seen shortlists with remarkably different products on them, and if the requirements and selection criteria are working properly, this just shouldn’t be possible.

Having worked on dozens of CMS selection projects, this is a good “sanity check”:

The products in the final CMS short-list should be similar, or the selection process is in trouble.

This means:

• All shortlisted products meet core CMS requirements.
• All products should be strong candidates.
• They should be the same “type” of products (and not a mix of web CMS, collaboration tools, IT platforms, portals or document management systems).
• The products should be roughly similar in size and complexity (eg there should not be an “enterprise” CMS alongside a small-scale offering).
• They should all run on the same (preferred) technical platform (IT can easily knock out products because of the platform they run on, leaving just one or two).
• They should all be affordable within the budget, or close thereto.

Now, this is not to say that there can’t be some variation. There’s no problem with having a mix of closed source and open-source options, as long as it’s managed well. There can be both “mid-market” and “upper-market” offerings, as long as this is done deliberately to help understand what paying more would get you.

At the end of the day, you want three strong candidates to pick from so you can be confident that the best solution has been found. If the final short-list fails the criteria listed above, double-check the whole process, and if necessary, get some outside expert input.

Your thoughts and experiences?

It is not unusual for vendors to ask for several hours to do a product demonstration and then only reluctantly show the product towards the very end of the sales meeting. Even when they talk to seasoned industry analysts, vendors tend to find 60 minutes too short. Unfortunately most CMS vendors are quite bad at giving compelling product demonstrations and many buyers waste their time going through dull sales pitches.

To this I would add: if the vendor is reluctant to show you the CMS, or just gives screenshots in a PowerPoint presentation, be suspicious. Hell, be very suspicious. Chances are their product doesn’t work very well, is unusable, very complex, or all three. Buyer beware.

Vendors love when budgets are included in the RFP. This enables them to fit their proposal around the budget, naturally ensuring they spend every penny of it. As a customer you have to be careful: Some might argue that transparency on both essentially make for more open and honest discussions, but most decide not to disclose the exact budget.

I’ve also had this discussion with clients on many occasions, and agree with the pros and cons outlined in this post. Like Peter, I’d tend to lean towards not revealing the budget.

This discussion often comes up during CMS selection projects, sometimes even after the process is well underway. The consequence is a delayed project, with much time wasted on meetings and emails discussing ownership. Too often the online communications team ends up having to argue why a particular system proposed by the IT department is not a good fit; a difficult task.

Testable stories is something we at J. Boye have developed and refined together with several community of practice members and consulting customers since early 2008 whenever we have been involved in the vendor selection process. Testable stories are typically used when few bidders are left in the race, e.g. 2 – 3 vendors. Each bidder gets a copy of the testable stories and is then invited to demonstrate how their solution meets each testable story on-site.

For web content management I don’t believe your industry matters very much. If it did, there would not be 30+ individual Web CMS vendors and open source projects with installations in the U.S. federal government space, and nearly that many among U.K. public-sector agencies. You’ll find a similar breadth of WCM suppliers active within other verticals, like higher education, health care, and financial services.

Of these, the biggest stumbling block is a lack of structure around the selection project: who is doing what, when will it happen, and how are decisions going to be made?

There are three main groups involved in the CMS selection project:

• steering committee or senior sponsor
• stakeholder group
• evaluation team

Of these, the evaluation team is the most critical, and the membership of this group must be defined very tightly from the outset of the project. Failure to do this will slow the pace of the project, as well as generating uncertainty at the key points of decision-making.

[Read the full article]

In this instance, however, I needed to look at a product that has become very popular in the web agency market, produced out of the Czech Republic. Getting a demo in person was clearly not an option. So I made use of their online demo to build an understanding of their product.

First off, it’s great that they offered a fully-featured online demo with a straightforward sign-up process. A few confirmation emails later, I’m working in the CMS that allows everything except customisation of the code. (They also provide completely transparent and public pricing; I wish more vendors managed this!)

I know a lot about CMS, and have probably seen 50+ products in the last 5 years. I know what I want to see, what to look for, and where to probe for weaknesses. Within five minutes I can get a sense of what type of product it is, and therefore what to expect.

What struck me, however, is how important this background knowledge is to assessing the product. Documentation was minimal, and there was no guided tour. If I was a novice, using the online demo to determine what product to buy, I’d be in a lot of trouble.

Most people only select a CMS once (and probably vow never to do it again!). Without a lot of domain knowledge, it would be easy to get lost in these types of online demos. A lot of clicking later, you’d have an idea of how to publish a page, but could easily miss entire sections of the solution (what is the versioning like? how easy is it to setup workflow? what are the online forms capability like?). There are also small but vital elements that need to work (cutting and pasting from Word, table editing, link management).

So: are online CMS demos useful?

In short: yes and no. If you know what you’re doing, they can be valuable. As a way of building an understanding the CMS marketplace, they’re likely to be more baffling than useful. Use with care, and beware of spending time without gaining insight.

What can vendors do to help? Provide a proper step-by-step guide to common tasks in your CMS, given to everyone who signs up for an online demo. Make it easy for people to get a mental map of what the CMS is doing, and what it provides.

What would I recommend? Write up good CMS scenarios, and use these as the basis for a structured vendor demo, held face-to-face. This will give you a lot of information very quickly, as well as providing a framework for product evaluation.

What have your experiences been?

Nothing looks more convincing than a score where one option has more points than another.

But, users don’t necessarily want to use a system just because it has the highest cumulative, weighted score. They want to use a system that helps them efficiently get their jobs done while introducing the fewest number of annoyances.

If the measurement of accuracy is the overall satisfaction with the solution, this method is extremely faulty.

I’m 100% in support of this piece. In our work on CMS selection, we keep scoring to a very simple level, and encourage discussion and debate amongst the evaluation team. A good selection process should help the organisation to learn more about their own requirements, and not just about potential solutions.

Learn from the foremost experts to save money and mitigate technology risks.

Choosing the right web content management system (CMS) is vital. Spend a day with James Robertson, one of the foremost experts on CMS, to obtain the knowledge and techniques you need to maximise outcomes, reduce costs and mitigate risks.

CMS have become vital for both websites and intranets, but it’s a complex marketplace with over 140+ products available in Australia alone (we maintain a list). These products vary greatly in price, performance, design and architecture.

How much: $750 + GST (25% discount for Intranet Leadership Forum members)

[Read the full details and register]