How many AI bots do we have, and are they the right ones?

If the promises of the AI revolution are to be fulfilled, organisations must pursue rapid and widespread adoption of AI capabilities. As outlined in the AI framework for the digital workplace, AI can take many forms and address a wide range of use cases.

In an enterprise context, various forms of “agents” and “chatbots/bots” are seen as fertile ground for AI. A core function of these is to draw upon a body of knowledge to allow users to use natural language queries to obtain accurate and relevant answers.

Major vendors including Microsoft, Google, OpenAI and Anthropic all provide straightforward ways for business users to create agents/bots (as well as more powerful tools for developer use). A multitude of other vendors provide smaller or industry-specific solutions (such as Harvey and Legora for law firms).

No vendor can deliver yet a single agent/bot that can effectively encompass an entire organisation’s collection of documents and online content. Even when it becomes possible to create a single AI entry point that passes the user’s query to the relevant agent/bot, these smaller solutions will be as or more important (assuming that this “AI orchestration” proves to be feasible).

What’s needed is an approach that enables organisations to make mindful decisions about where and when to deploy AI agents/bots. This article introduces the “knowledge domains for AI” framework developed by Step Two. Keep reading, or jump straight to the full Knowledge domains for AI whitepaper (free download).

Challenges at scale

The proliferation of agents/bots created by business areas poses many risks and challenges. In our client work, we’ve seen common issues emerge, including:

• Poor quality source material, such as badly written, out of date or conflicting content, is poisoning AI responses.
• Business areas are spinning up local solutions built on enterprise sources such as intranets, without any visibility of the broader platform owners.
• The challenges of dealing with complex bodies of knowledge are often underestimated by local teams.
• Insufficient focus on the responses provided by AI is exposing organisations to significant business risks.
• A lack of clear objectives means that business benefits are poorly understood and frequently failed to be met.
• Business areas aren’t getting the support they need to succeed with AI.
• Many AI solutions are going straight from pilot to production, without the underlying management to ensure long-term sustainability and viability.
• When dozens of AI solutions grows to hundreds (or thousands), confusion inevitably reigns.

Strategy then governance

A clear AI strategy underpins the successful deployment of AI solutions at scale. This whitepaper assumes that businesses have identified strategic imperatives to adopt AI widely, deeply and at speed. A meaningful strategy should then identify priorities in terms of business units, business processes or organisational outcomes. This determines where enterprise resources and budget will be allocated.

As outlined in the previous section, however, once the wider organisation becomes engaged with AI solutions such as agents/bots, significant issues inevitably arise.

To support the overarching strategy, this whitepaper introduces a new “knowledge domains for AI” framework, offering a practical governance and management approach.

Knowledge domains for AI are bodies of enterprise information selected as the foundation for well-managed AI agents/chatbots that give accurate and useful responses, while delivering clear business benefits

The key principle underpinning the concept of “knowledge domains” is that organisations will get the most from AI when they are judicious in where to apply it. Rather than a free-for-all, AI agents and chatbots/bots should be built upon carefully chosen bodies of content that enable the solutions to give accurate and meaningful answers.

In this way, a top-down strategic emphasis on AI can be shaped into a series of deployments that the business can be (reasonably) confident will deliver clear business benefits. The knowledge domains for AI framework also offers a ‘stage gate’ for AI pilots to go through before being deployed into production.

Three elements of the framework

The objectives of the framework is to bring key stakeholders together within an organisation to take a common and aligned approach and to establish systematic approaches for creating and managing agents/bots.

The framework consists of three key elements:

A set of criteria that underpin the successful deployment of effective knowledge domains, while ensuring that they are sustainable and manageable over time.

An approach for mapping knowledge domains, taking into consideration their potential business impact, complexity and degree of risk.

Processes that can be followed when putting the knowledge domains for AI methodology into practice, both at the enterprise level and at the local level within individual business units.

Download the free Knowledge domains for AI whitepaper, that digs into each of these components. Start on the journey to establish an enterprise-wide landscape of AI agents/bots that deliver clear benefits while mitigating key risks.