Organisations often make the selection of a CMS much harder than it needs to be. They do this by running into common pitfalls that impact not just on the selection process, but on the overall success of the CMS project.

Over the past ten years, we have worked with many organisations on content management systems, and have seen a huge number of tenders released to the marketplace.

Across these projects, the same issues are seen again and again. These most often relate to how the requirements are documented, or how the overall tender is structured.

They may also arise from a lack of clear scope for the CMS project, or from the limited understanding of content management issues and the CMS marketplace.

With the aim of tackling some of these issues, this article lists the top ten mistakes commonly seen when attempting to select a CMS.

At their heart, these are all simple issues to resolve, primarily by taking a common-sense approach to the selection process.

It is easy to get caught up in the detail of the requirements and the CMS project, losing sight of the overall objectives and process. The starting point is therefore to step back and to evaluate where the project is at, and how it will proceed.

By reviewing the project against the ‘top 10’ in this article, it should be possible to chart a rapid (but careful) path through the selection process, to the final deployment of a CMS that works well for authors, site administrators and the wider organisation.

Organisations make the selection process much harder than it needs to be

1. Not understanding the problem to be solved

All too often, organisations rush into purchasing a new piece of technology before fully understanding the problem to be solved.

For example, CMS selection may unfortunately be done before:

• determining the business needs and overall business goals
• identifying the underlying website design or management issues
• creating an overall website or intranet strategy
• identifying what functionality will be delivered by the site (rather than the CMS)
• choosing which sites will be in scope for the CMS project

Without understanding what will be done to the actual website or intranet, there is no meaningful basis on which to select a CMS solution.

The starting point must therefore be to conduct a needs analysis process within the organisation.

Note that this should not involve asking what people want in terms of CMS functionality, as the business probably has little CMS knowledge on which to base their input.

Instead, this needs analysis should focus on building an understanding of the business needs, which are then mapped by the project team to actual CMS requirements.

So the starting point is therefore to first understand the site, before trying to understand the CMS requirements.

2. Not understanding content management issues

Many organisations are selecting a CMS for the first time. Without any prior experience with content management systems, it can be hard to develop meaningful business requirements.

Even organisations who are replacing an existing system may be limited to their experiences with just the current product.

The real challenge for organisations is therefore to build knowledge and expertise in content management, to ensure that informed decisions are made throughout the selection process.

It must be possible for organisations to answer questions such as:

• which features are widely available in products, and which are not
• what are the key challenges in implementing CMS products
• what are the common problems with the design of CMS products
• what functionality and issues should the CMS project focus on
• what is the best scope for the project
• what are the ‘best practice’ approaches to content management

Much of the success of a CMS project rests on the decisions made by the purchasing organisation, and on the underlying maturity of content management knowledge and expertise within the organisation.

At the end of the day, it’s up to the organisation to be an informed consumer, when approaching the content management marketplace.

Steps should be taken to build this knowledge before writing the first requirement, or seeing the first vendor demo. Practical approaches involve reading and researching existing material, or discussing the issues with other organisations who have already implemented a CMS.

It is vital to be an informed consumer when selecting a CMS

3. Assuming there are only a dozen possible products

The content management marketplace is an extremely complex one at present. It has been estimated that there are over 1,000 products world-wide, with almost every country or region having in excess of 100 products available locally.

The challenge is to find and understand these products, with few effective channels for customer research or vendor marketing.

Out of this myriad of solutions, a few vendors have spent sufficient marketing money to gain some visibility. These are often the international players, focused on gaining wins in the high-end of the market.

The trap often made by organisations is to assume that there are only a dozen ‘real’ content management systems in the marketplace, and to evaluate solely these solutions.

Without having done the necessary research to uncover the broader set of products, organisations often get a very skewed perception of the CMS marketplace.

The problem is that products are hugely variable in their capabilities, perhaps only 30% similar and 70% remarkably different.

This means that the less products examined, the greater the chance that the most appropriate solution has been missed.

Organisations must therefore spend the time to build an understanding of the local marketplace, and to uncover all the possible solutions.

There are approximately 1,000 CMS products worldwide

4. Bigger is better

Many organisations are naturally risk-adverse in the way they operate, and this extends to the selection of a new CMS.

This may lead the organisation to focus on obtaining a powerful and complex product from a large CMS vendor, with the goal of mitigating (or eliminating) the risk that the vendor will go bust.

To this end, the CMS tender asks for details on the size and financial viability of the vendor, alongside the functional requirements for the product itself.

It can, however, be strongly argued that there is no correlation between price and performance when it comes to content management systems.

Paying more money does not automatically mean that the CMS is better or more capable. It also does not necessarily improve the chances that the overall project will be successful.

This issue also relates to the desire to purchase a CMS ‘for life’, leading to the selection of the product with the greatest number of features within the available budget.

All of this comes down to the fundamental (but flawed) principle of ‘bigger is better’.

The problem is that spending more money naturally increases the risks inherent in the project. Every additional piece of CMS functionality also unavoidably impacts on the usability of the CMS for authors.

At the end of the day, if the CMS is not easy to use, the product is unlikely to prosper. This may lead to failure of the project, and of the website itself.

Perhaps the most practical approach for avoiding this issue is to keep it simple, and to keep it small.

Mitigate risks by spending the least amount of money possible, and maximise usability by getting the simplest CMS that will meet identified business needs.

For more on this, see the earlier articles Will your chosen CMS vendor go bust? and More users = simpler CMS.

Mitigate risks by keeping it simple, and spending less

5. Not distinguishing between requirements and selection criteria

The core trap for those writing content management tenders is failing to distinguish between requirements and selection criteria.

Fundamentally, the purpose of a tender document is to differentiate between products, to knock out unsuitable CMS solutions, and to finally identify a single successful bidder.

All too often, however, tenders end up containing requirements such as:

• “The CMS must provide a web-based authoring environment.”
• “The CMS must support multi-stage workflow rules.”
• “The CMS must integrate with existing systems.”

These exact requirements are seen time and time again, and they are ineffective for a variety of reasons:

• all products may offer the specified functionality
• the stated requirements cannot be evaluated
• insufficient information (or context) is provided to make meaningful requirements

At the end of the day, there must be a clear distinction between requirements and selection criteria.

What is needed are selection criteria that allow the organisation to easily and meaningfully evaluate products against business needs.

This means writing fewer requirements (as discussed below), and writing them in a narrative format that allows the specific needs to be clearly stated.

For more on this, see the earlier article Using narrative in a
CMS tender.

Focus on key selection criteria rather than just requirements

6. Writing too many requirements

The unstated fear when writing tender documents is “if we don’t ask for it, we won’t get it”. This leads to requirements documents that are a hundred pages (or more!), based on the principle that it is better to include more detail rather than less.

Fundamentally, however, writing more requirements actually makes it harder, not easier, to find the right product.

The reality is that writing too many requirements leads to significant problems, including:

• confuses key needs with ‘nice-to-haves’
• makes it hard to identify the real differences between the products (‘losing sight of the forest for the trees’)
• forces the selection of a larger and more expensive product to meet all stated needs
• increases the time and effort required to evaluate responses
• increases the time and cost for vendors to respond
• discourages many vendors from responding entirely

Tender documents should instead focus on the key selection criteria that will drive the evaluation and differentiation of the products.

In general, there are rarely more than a dozen key selection criteria, along with supporting requirements (documented as being of lower importance).

This clarifies the selection process, and ensures that efforts are focused on determining that the products definitely meet ‘life-or-death’ needs.

In all cases, avoid ‘complies’ / ‘does not comply’ responses

7. ‘Complies’ / ‘does not comply’

Driven by the underlying desire to be able to quantify (and therefore compare) vendors responses, a common approach is to ask vendors to indicate whether they ‘comply’, ‘do not comply’ or ‘partially comply’ with the stated requirements.

This is, however, a complete waste of time. Obtaining these responses gives almost no insight into the actual capabilities of the product, and encourages vendors to bend the truth in their submissions.

This is generally compounded by poorly-written requirements, leading to the situation where vendors respond with ‘complies’ to requirements such as “the CMS must integrate with Outlook to deliver email”.

A much better approach is to get vendors to provide descriptive (or narrative) responses, giving screenshots and examples where appropriate.

Better yet, go one step further by explicitly requiring these more extensive responses, indicating clearly that vendors who fail to provide sufficient information will be eliminated from further consideration.

(Note that moving away from ‘complies’ and ‘does not comply’ responses still allows submissions to be evaluated and scored, as long as the requirements are clearly written.)

8. Focusing on the ‘what’ not the ‘how’

“The CMS must provide powerful and flexible workflow capabilities.” This certainly sounds desirable, but how do the workflow features actually work?

The reality is that CMS workflow may range from a single checkbox called “published” through to graphical tools for drawing workflow diagrams with branch points, conditional logic and more.

Furthermore, adding or changing workflow rules may be a point-and-click operation, or it may need several days of development by a CMS specialist.

In almost all of these cases, the vendor can argue that the solution is ‘flexible and powerful’, even though the actual operational details vary widely.

In practice, the success of the CMS project rests on how well the solution fits the day-to-day requirements of the authors and site administrators. This comes back to how the product actually works, in detail.

The selection process must therefore focus on asking ‘how’ questions rather than ‘what’ questions, with the aim of building a clear understanding of the products’ strengths and weaknesses.

In general, this is best done by taking things back to the underlying business need, and asking vendors to outline how they can meet that need.

In the case of workflow, this may involve describing who will be the authors, what roles they will take, and the typical process for publishing content to the site.

Vendors can then demonstrate (in both the written tender response and the vendor demonstration) how the product can be used to manage this publishing model.

Focus on the ‘what’ rather than the ‘how’

9. Confusing the CMS project and the broader website project

When tackling the website or intranet, there is generally more to be done than just selecting a new CMS.

First off, it is almost always the deficiencies of the current site that prompts the CMS project, including dated design, poor site structure, out of date content and ineffective publishing processes.

This generally leads to a redesign of the site itself, and there may be the temptation to bundle this in with the CMS project, potentially with the goal of finding one provider that can deliver both a new design and a new CMS.

This is a dangerous approach, however, and one that is driven more by convenience than any strategic considerations.

In practice, it is generally better to separate the design and the CMS, with the redesign of the site being completed first, before the selection of a new technology solution.

(See the earlier article Separate design and the CMS for more on this.)

Distinguish between the CMS and wider website projects

Equally significantly, a number of different technologies may end up being bundled into the CMS project, including:

• search
• portals
• collaboration tools
• discussion groups
• mailing lists

While these may all be needed to deliver the site, the question is whether they should be delivered by the CMS.

While some vendors will include a number of additional modules to address these needs, an equal number do not (concentrating instead on core CMS functionality).

In practice, it is important to distinguish between the CMS and the wider website project. If the vendor is not expected to provide these additional tools, then they should not be included in the tender.

A more pragmatic approach is to instead seek the best solutions in each of these areas, separately from the CMS project, recognising that there is often little (or no) integration or connection to the CMS.

The CMS should be selected by the site owners, not IT

10. Running the selection as a technology project

Finally, a common cause of project failure is to treat the CMS selection as a technology project, run by an IT team.

This can easily create a requirements document that is poorly understood by the organisation (including the site owners), and the acquisition of a product that may be technically suitable, but ineffective on a day-to-day basis for authors and site administrators.

When it comes down to it, the technology issues are the smallest element of the selection process. IT must provide clear guidance on technology platform and infrastructure requirements, but will generally have little input on the business requirements.

The selection process should therefore be run by the site owners, who are best placed to document the requirements and to evaluate the solutions.

With the right set of requirements, the selection process need not be complex or overwhelming, and should be well within the capabilities of most web and intranet teams.

Driving projects in this way, from the business rather than IT perspectives, has been demonstrated to deliver the best outcomes.

Summary

Common sense underpins the successful selection of a content management system. While there are common pitfalls that many organisations fall into, these are fairly easy to avoid once recognised.

Start by building a good understanding of content management issues, and the CMS marketplace. A clear set of business needs can then be used to generate effective business requirements.

Good requirements will then avoid most of the common problems, particularly when the CMS project is simplified down to focus on the core tasks of selecting a new product, with the other needs handled separately (in parallel).

In short, take steps to mitigate or avoid the ‘top 10’ mistakes made when selecting a CMS:

1. Not understanding the problem to be solved
2. Not understanding content management issues
3. Assuming there are only a dozen possible products
4. Bigger is better
5. Not distinguishing between requirements and selection criteria
6. Writing too many requirements
7. ‘Complies’ / ‘does not comply’
8. Focusing on the ‘what’ not the ‘how’
9. Confusing the CMS project and the broader website project
10. Running the selection as a technology project

Further reading

Previous articles have addressed many different aspects of the CMS selection process, including:

• CMS vendors are evaluating us
• Sources of CMS uncertainty
• What is the purpose of a CMS tender?
• Specifying technology in a CMS tender
• The importance of CMS usability
• Requirements-focused CMS selection