Category: Uncategorized

  • The New Digital Trust Question: What Should We Allow AI to Decide?

    The New Digital Trust Question: What Should We Allow AI to Decide?

    As artificial intelligence moves from generating answers to taking action, a new question is moving rapidly up the leadership agenda: what should we actually allow AI to decide?

    The challenge is no longer simply whether organisations should adopt AI. It is how they determine where machines may act autonomously, where human judgement must remain decisive, and who remains accountable when those boundaries become blurred.

    From Assistance to Agency

    For much of the recent AI era, organisations have treated artificial intelligence primarily as an assistant: generating content, analysing information, identifying patterns and supporting human decisions.

    That boundary is shifting. Emerging AI systems can increasingly initiate actions, interact with other systems and pursue objectives with less continuous human direction. The governance question therefore changes from “Can AI help us?” to “What authority are we prepared to give it?”

    Governance Is Moving Towards the Same Question

    Policymakers and standards bodies are beginning to confront this shift directly. Singapore’s Model AI Governance Framework for Agentic AI recommends that organisations bound an agent’s autonomy and access to tools and data, while establishing meaningful points at which human approval is required. The principle is significant: autonomy should be designed, limited and governed — not simply switched on.

    In the United States, NIST is approaching the challenge from another critical direction: identity and authority. Its work on software and AI agents asks how autonomous systems should be identified, authenticated and authorised, and how their actions can be audited. Once an AI system can act, the question of who or what has authority to do what becomes a digital trust issue, not merely an AI performance issue.

    Human Oversight Is Not the Same as Human Accountability

    It is tempting to treat “human oversight” as the answer. But oversight only matters when the human involved has the authority, information and time to intervene meaningfully. A person who merely approves an AI-generated recommendation without understanding how it was reached may provide the appearance of control without its substance.

    The more consequential the decision, the more clearly an organisation should be able to answer three questions: Who has authority to approve it? Who can stop or reverse it? And who is accountable for the outcome? AI may perform the action, but accountability cannot simply disappear into the system.

    A Practical Test for AI Decision Authority

    Before granting an AI system greater decision-making authority, organisations should test the proposed use against four dimensions: consequence, reversibility, accountability and control.

    1. Consequence

    What happens if the AI gets the decision wrong? Low-impact, routine and easily corrected decisions may tolerate greater autonomy. Decisions affecting safety, rights, employment, finance, security or access to essential services demand a much higher threshold for human involvement.

    2. Reversibility

    Can the decision be undone quickly and without significant harm? An AI system scheduling a meeting is fundamentally different from one terminating access, transferring funds or making a consequential decision about a person. The harder an action is to reverse, the stronger the case for human approval before execution.

    3. Accountability

    Is there an identifiable person or function that remains responsible for the decision? Delegating an action to AI should never create an accountability vacuum. Organisations need clear ownership of the system, its permissions, its decisions and the consequences that follow.

    4. Control

    Can the organisation see what the AI is doing, restrict what it can access, intervene when necessary and reconstruct what happened afterwards? Effective control requires appropriate permissions, monitoring, audit trails and the ability to stop or override the system.

    Applied Cybernexa Scenario™: When the AI Wants to Act

    A financial institution deploys an AI agent to strengthen its response to suspected account compromise. The agent can analyse unusual login behaviour, compare activity across systems and identify accounts that may have been breached.

    Initially, the agent recommends actions to the security team. The organisation now proposes allowing it to automatically suspend user access when its risk threshold is reached.

    Should the AI be allowed to decide?

    Applying the four-part test changes the discussion. The consequence of a mistaken suspension may be significant; the action is reversible, but disruption could still affect customers or critical staff; accountability must remain with an identifiable security function; and the organisation needs controls that limit the agent’s permissions, record its actions and allow rapid human intervention.

    The appropriate answer may therefore not be simply yes or no. The organisation might permit autonomous suspension in narrowly defined, high-confidence circumstances while requiring human approval for ambiguous cases or actions with greater consequences.

    The Leadership Question Is About Authority

    The most important AI governance decisions may therefore be less about the sophistication of the technology than about the authority organisations choose to delegate to it.

    Boards and executive teams should know where AI systems are operating with meaningful autonomy, what data and systems they can access, what actions they can initiate, and where human approval remains mandatory. Those boundaries should reflect the potential consequence of the decision rather than enthusiasm for the technology.

    This also changes the nature of digital trust. Trust cannot depend solely on whether an AI system is accurate. It depends on whether the organisation can demonstrate that the system operates within defined authority, under effective controls and with clear human accountability.

    The Question Every Organisation Should Be Asking

    The arrival of increasingly autonomous AI does not mean organisations must choose between innovation and control. It means they need to become much more deliberate about where authority sits.

    Some decisions can safely be delegated. Others should remain firmly human. Between those two points lies a wide range of circumstances in which autonomy can be bounded by thresholds, permissions, escalation rules and meaningful human intervention.

    The organisations most likely to earn digital trust will not necessarily be those that automate the most. They will be those that can explain what their AI is allowed to decide, why it is allowed to decide it, and who remains accountable when it does.

    That is becoming one of the defining governance questions of the AI era.

    Sources & Further Reading

    Infocomm Media Development Authority (Singapore), Model AI Governance Framework for Agentic AI, updated 20 May 2026.

    National Institute of Standards and Technology (United States), Accelerating the Adoption of Software and Artificial Intelligence Agent Identity and Authorization, 5 February 2026.