The corporate conversation about artificial intelligence has moved beyond whether an organisation should experiment with the technology. For many boards, the more demanding question is how promising demonstrations become dependable services without weakening accountability, security or customer confidence. An assistant that drafts a marketing email is not the same proposition as a system that summarises a loan application, recommends an insurance response or influences staff decisions. The technical components may look similar, but the consequences of an error are different.
That distinction matters to executives allocating technology budgets. Success cannot be established merely by counting pilots, measuring how often employees use a chatbot or comparing the cost of generating a page of text. The stronger test is whether a particular process delivers measurable value while maintaining control over information, decisions and failures. The US National Institute of Standards and Technology’s AI Risk Management Framework provides a useful starting point: AI risk management is a continuing organisational discipline, not a one-off product approval.
Start with the decision, not the model
Technology programmes often begin with a model demonstration and search for a business case afterwards. A more disciplined approach starts with a defined operational problem. Is the organisation attempting to reduce time spent classifying invoices, improve the completeness of customer-service responses, or help a compliance analyst locate relevant policy documents? Each objective requires different evidence and a different tolerance for mistakes.
A financial services business may find that an AI tool can summarise long documents efficiently while struggling with calculations, ambiguous contractual clauses or unusual exceptions. The appropriate use could therefore be preparing material for a qualified employee to review, rather than making a binding determination. A retailer might accept a lower level of precision when categorising internal product descriptions than a bank would when checking account access or communicating regulated information.
The implementation team should write down the decision being supported, the information needed, the people affected and the consequences of a wrong answer. That record is useful when managers later ask whether the system is performing as intended. Without a clear purpose, pilot results risk becoming demonstrations of technical possibility rather than evidence of operational improvement.
Treat data quality as a governance issue
An AI application inherits weaknesses from the information and workflows around it. Inconsistent customer records, outdated policy documents and poorly defined ownership can create misleading outputs even when the underlying model performs well on a benchmark. Connecting more documents to a system does not automatically improve their reliability.
Companies should establish which repositories are approved, who maintains each source, how frequently information changes and when outdated material is removed. Access controls matter equally: an AI assistant should not expose a document merely because it can retrieve it. In organisations with complex permissions, a successful deployment may require more work on identity, classification and document management than on prompts.
This is particularly important when suppliers host models or applications. Procurement teams need to understand what data is transmitted, what is retained, whether information may be used for model improvement and which subcontractors are involved. Contractual safeguards should be supported by technical testing and clearly assigned internal responsibilities. A supplier's general security assurances do not replace a customer's own assessment of the particular deployment.
Measure outcomes and failure modes together
A credible business case should contain both operational and control measures. Time saved is useful, but only when paired with the rate of corrections, escalations, customer complaints and exceptions. If a system saves an analyst five minutes per case while requiring additional review of most outputs, headline efficiency estimates may overstate the benefit.
Testing should include representative routine cases, difficult edge cases and deliberately challenging inputs. Teams should also monitor whether performance changes as customers, products or internal rules evolve. A pilot tested against a static set of documents may deteriorate when the underlying knowledge base becomes outdated or when staff begin using it for tasks outside the original scope.
The NIST Generative AI Profile identifies distinctive concerns such as unreliable information, privacy issues, security and the way generative systems can produce plausible but incorrect content. These risks do not mean companies should avoid the technology; they mean evaluation must examine the kind of failure the organisation can reasonably anticipate.
Where the technology assists a high-impact decision, the company should be able to reconstruct what information was available, what the system produced and what a human decided. Logs and traceability should be designed with privacy and retention obligations in mind, rather than collecting everything indefinitely.
Human oversight needs an operating model
It is easy to write that a human will remain in control. It is harder to define what that means at 4 p.m. on a busy Friday when hundreds of suggestions are waiting for approval. Human oversight is ineffective if employees are expected to approve every output rapidly without context or authority to challenge the system.
Practical oversight starts by identifying which decisions can be automated, which require sampling and which must be individually reviewed. Employees need clear escalation routes when information conflicts, an output seems fabricated or the system behaves differently from established policy. Management should track whether reviewers actually detect errors, not just whether a review button was clicked.
Training is another control. A staff member using an AI assistant should understand the permitted tasks, the need to protect confidential information and the limits of generated explanations. Training should be tailored: an employee drafting routine communications faces different risks from a developer integrating an AI service into an authentication workflow.
Regulatory planning without assuming one rule applies everywhere
Organisations operating across jurisdictions face a fragmented legal environment. The European Union's AI Act framework applies obligations progressively, according to the role of the organisation and the characteristics of the relevant AI system. The rules for certain higher-risk systems have later application dates than many general transparency and governance provisions. The European Commission’s implementation guidance provides the relevant timetable; companies should avoid assuming that every AI application is legally classified as high risk or that every obligation started on the same date.
For global businesses, this creates a need for an inventory that records system purpose, deployment location, supplier, affected users and applicable obligations. The inventory should not become a paperwork exercise with no operational owner. Its purpose is to help teams distinguish ordinary productivity software from applications requiring stronger legal, technical or managerial review.
Financial institutions may also need to assess existing obligations relating to outsourcing, privacy, operational resilience, consumer protection and recordkeeping. An AI application does not erase those responsibilities. The controls should reflect the actual activity and jurisdictions, with appropriate professional advice where necessary.
Build procurement around evidence
Vendor selection often focuses on model features and commercial pricing. Those considerations matter, but purchasers also need evidence about implementation support, service availability, data handling, change management and exit arrangements. A company dependent on a proprietary platform should ask how it will retrieve records, replace the service or maintain continuity after a major product change.
Contracts should identify responsibilities for incident reporting, access management, testing and material changes to the service. If an AI product can take actions in other enterprise systems, permissions should be limited to what the workflow requires. An assistant that prepares a payment file poses different risks from an agent that can authorise its release.
Independent testing, supplier documentation and customer references may all help, but none is a universal guarantee. A successful procurement process records assumptions and tests them in the customer's own environment. Those findings should inform contract terms and deployment boundaries rather than being filed away after the purchase order is signed.
Choose a portfolio of controlled uses
Not every AI opportunity deserves the same investment or oversight. One sensible approach is to group initiatives by potential value and consequence of failure. Low-impact internal drafting tools may be suitable for rapid, supervised experimentation. Systems touching regulated communications, sensitive records or material decisions may need stronger validation and approval before launch.
The OECD’s 2025 survey of SME digitalisation highlights the importance of understanding how companies use digital technologies and the obstacles they face in adapting business processes. The implication for executives is straightforward: purchasing technology is only one part of transformation. Roles, procedures, incentives and employee capabilities must change with it.
Leaders should periodically retire experiments that do not solve a meaningful problem. Keeping every pilot alive creates hidden costs in licensing, maintenance, monitoring and staff attention. Portfolio discipline also protects the capacity needed to scale the few uses that demonstrate clear value.
The board's question is whether the system remains governable
As AI becomes embedded in ordinary operations, the boundary between a technology initiative and a business-control issue becomes harder to maintain. Boards should ask management which processes depend on AI, what failures have occurred, how those failures were handled and whether benefits continue to justify exposure.
Reporting need not rely on elaborate technical language. Useful measures include the number of material use cases, the share with named owners, results against agreed service standards, unresolved control issues and the quality of incident response. The objective is not to promise flawless automation. It is to create a system in which problems can be detected, understood and corrected before they become larger organisational failures.
The businesses likely to gain durable value from AI are not necessarily those announcing the most pilots. They are those that connect technology to specific decisions, invest in the underlying information, give staff meaningful oversight and preserve the ability to change course. Responsible adoption is therefore not the opposite of commercial ambition. It is the operating discipline that can turn experimentation into a reliable business capability.
References
NIST — AI Risk Management Framework — https://www.nist.gov/itl/ai-risk-management-framework
NIST — Generative AI Profile — https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence
European Commission — AI Act regulatory framework — https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai
European Commission — AI Act enforcement timetable — https://ai-act-service-desk.ec.europa.eu/en/ai-act/faq/when-does-enforcement-start
OECD — SME digitalisation for competitiveness (2025) — https://www.oecd.org/en/publications/sme-digitalisation-for-competitiveness_197e3077-en.html