A practical look at ownership, testing and oversight as organisations incorporate generative AI into everyday work.
The governance gap
Artificial intelligence can begin as an experiment and become an operating dependency with surprising speed. A team may first use a generative tool to summarise documents, draft internal correspondence or locate information. Once colleagues rely on those outputs in routine work, questions emerge about accountability, data quality, confidentiality and error handling. The question is no longer simply whether a model produces a plausible response. It is whether the organisation can explain how the result was produced, checked and used. [1]
The US National Institute of Standards and Technology (NIST) describes its AI Risk Management Framework as a voluntary resource for identifying and managing risks associated with AI systems. The framework was released in January 2023, and NIST has since published a companion profile addressing the distinctive risks of generative AI. Neither publication is a blanket assurance of safety or a substitute for local legal requirements. [1][2]
Start with a named business owner
An AI tool used in recruitment, customer support, procurement or internal reporting may affect several departments at once. Effective oversight begins with a clearly designated owner for each use case: someone responsible for its purpose, acceptable inputs, output review and escalation when the technology is unsuitable. Ownership should also extend to the data on which the process depends and the people authorised to change a workflow. [1]
NIST organises AI risk management around four functions: Govern, Map, Measure and Manage. Governance cuts across the other functions, while mapping focuses on context, measurement on evaluating risks, and management on responses and ongoing controls. This structure is more useful as a continuing management discipline than as a one-time sign-off. [3]
Design controls for the actual task
Different tasks warrant different levels of human review. A draft agenda for an internal meeting generally presents a different risk profile from an automated customer notice, compliance summary or decision-support report. Organisations can document the intended use, identify information that must not be entered into a tool, and specify when a human must verify figures, quotations or cited authorities before a result is acted upon. [2]
A practical control record might show the model or service in use, the type of information processed, who approved the use case, the test cases selected and the procedure for correcting errors. These measures do not eliminate model limitations, but they give managers a clearer basis for judging whether performance remains acceptable when conditions change. [1][2]
Measure more than apparent productivity
Time saved is only one part of an implementation assessment. Teams can also examine how often outputs require correction, whether sources can be traced, whether sensitive data is exposed, and how frequently a human reviewer overturns the draft conclusion. A quicker process is not necessarily a better process if errors are difficult to detect or expensive to resolve. [2]
Testing should reflect the material an organisation actually handles. A generic benchmark may not reveal how a system performs on unusual customer language, fragmented documents, different jurisdictions or incomplete source records. The NIST framework emphasises risk management throughout the AI lifecycle rather than treating assessment as a single installation exercise. [1][3]
Keeping the operating model adaptable
A governance process also needs a way to retire or modify AI use cases. Vendors update products, internal documents change and new tasks are assigned to systems built for a narrower purpose. Periodic review can establish whether the tool still fits its stated role and whether staff have begun relying on outputs beyond approved boundaries. [4]
For business leaders, the central lesson is operational rather than promotional: adopting AI responsibly involves processes, people and evidence in addition to software. Organisations that can identify where AI is used, who owns each decision and what happens when results are wrong are better placed to evaluate the technology on its actual contribution to work. [1][2]
References
1. NIST, AI Risk Management Framework (AI RMF 1.0)
2. NIST, Generative AI Profile (NIST AI 600-1), July 2024