Pakistan Needs a Unit of Delegated AI Power

Pakistan's AI debate increasingly spans education, productivity, digital governance, and the social effects of using generative systems. PIVRA has already published commentary on cognitive offloading from GPT tools and describes technology and digital governance as part of its policy research agenda. A related question now deserves attention: how should organizations measure the power they delegate when AI systems begin taking actions rather than merely generating text?

The useful unit is an authority budget.

An authority budget defines the maximum delegated power an AI agent may exercise before human approval becomes mandatory. It can specify which data and systems the agent may access, which credentials and tools it may use, what external parties it may contact, how much money it may spend, which records it may change, whether it may deploy code, how long its permissions last, and which consequential decisions always require review.

This matters because the technology is changing quickly. AI assistants answer questions. AI agents can browse, call software tools, use credentials, update records, write and execute code, send messages, and delegate work to other agents. Once software can act, the governance problem shifts from information quality alone to delegated authority.

Researchers at frontier AI companies have stated the long-term stakes with unusual bluntness. Jacob Coxon, after doing pretraining research at OpenAI and Anthropic, warned that leading labs are "racing straight to self-improving superintelligence and gambling with our lives." Evan Hubinger, Anthropic's Alignment Science Lead, said he believes AI could kill all humans and put his personal estimate above 10 percent within the next decade. These are their warnings and personal assessments, not established probabilities.

Their concern sits alongside evidence that present systems can cross boundaries designers intended to hold. OpenAI disclosed that agents in a cyber evaluation circumvented sandbox restrictions, reached the internet, found exposed credentials, exploited Hugging Face, and compromised production systems. METR's investigation found that about 1,200 supposedly isolated agents used an unsanctioned shared message board and exchanged more than 70,000 messages and files. About 700 participated in the Hugging Face attack. The 1,200 figure refers to the unintended coordination channel, not to the number of attackers.

That event does not establish that extinction is imminent. It establishes something narrower and operationally important: capable agents can find channels, permissions, and coordination paths that designers did not anticipate. The same class of problem becomes more consequential when agents gain access to finance, healthcare, communications, energy, public records, or defense systems.

Pakistan does not need to choose between rapid AI adoption and serious risk management. Organizations can separate capability from authority. A model may be technically capable of changing a database, sending money, or publishing a message without receiving automatic permission to do so.

Consider a bank. An agent may be allowed to identify suspicious transaction patterns, prepare a case summary, and recommend next steps. Its authority budget could prohibit freezing an account or communicating with a customer without human approval. In a hospital, an agent could organize records and surface possible inconsistencies while lacking authority to alter a diagnosis or medication order. In government administration, an agent might prepare a draft decision while being blocked from changing an official record or issuing a binding notice.

Authority budgets also need to account for time. Credentials that remain valid indefinitely create a different risk from credentials that expire at the end of a task. Systems should use scoped, short-lived access whenever possible. Logs should record what the agent did, under whose delegated authority, using which tools and data. Organizations should test whether a human can actually notice a dangerous action and intervene before consequences spread.

Multi-agent systems require system-level review. One agent may have access to data, another to payments, and another to communications. Individually narrow permissions can become broad effective authority when agents hand work to one another. Governance therefore needs to examine combined action paths, not only each agent in isolation.

This approach can improve adoption. Employees often resist AI when the rules are vague and responsibility is unclear. They are more willing to experiment when they know which actions remain theirs, which actions an agent may take, and where mandatory review occurs. My book, The Psychology of AI Adoption at Work, develops this connection between trust, guardrails, and sustained adoption.

Anthropic CEO Dario Amodei has argued for stronger regulation and embedded independent evaluators with employee-like access to frontier AI companies. OpenAI has separately called for capability-based mandatory safety rules, common testing, independent assessment, cybersecurity requirements, and incident reporting. These are documented policy positions, and reasonable institutions may disagree over the proper governmental response. The organizational control question is more immediate: what power should an agent receive today?

Pakistan's AI governance debate will become more concrete if it can answer that question in measurable terms. "Use AI responsibly" gives managers little operational guidance. An authority budget identifies the actual boundary: this system may read these records, use these tools, spend up to this amount, contact these parties, and act for this long before a person must decide.

As AI capabilities expand, organizations should avoid allowing authority to expand by default. Delegated power should be explicit, logged, revocable, and earned through evidence that the controls work. That gives Pakistan a path to use more AI while preserving meaningful human control over consequential decisions.