Chains of Command: Power, Accountability, and Agentic AI

This interactive roundtable brings together voices from international organizations, government, industry, and civil society to map emerging risks of agentic AI and co-create effective mitigation measures. Through facilitated discussion, participants will map emerging human rights impacts; explore potential safeguards—including human oversight, impact assessments, evaluations, grievance mechanisms, and multilingual access; debate where to draw boundaries on AI autonomy and who decides; and explore public procurement requirements from a rights-based approach.

Agentic AI systems go beyond predictions to autonomously decompose objectives, retain information, connect to external tools, and adapt based on new data—essentially making decisions with reduced human oversight. Already deployed in customer service, code generation, content moderation, and enterprise workflows, these systems can automate repetitive work, offload certain decision-making structures, simplify navigation of complex institutions, and coordinate large-scale information processing that would be impossible manually.

Yet the same features that make agentic AI powerful create novel human rights risks and exacerbate existing ones. When an AI system operates autonomously, the distance between a human decision and what happens in practice grows, making it harder to provide interpretability and assign responsibility. When systems retain detailed information about individuals over weeks or months, they accumulate data, which, without proper oversight, can be re-used, mis-used, or lead to profiling. And when agentic systems connect to real-world tools, such as personal calendars, bank systems, social media platforms, or government databases, probabilistic outputs can translate into consequential actions. The scale and stakes are even higher when multiple AI agents work together, amplifying errors and biases at volume and at pace through new coordination pathways.

The technology is moving fast, largely shaped by profit-driven priorities and values, and being deployed globally without meaningful consultation with marginalized communities and the Global Majority, who are disproportionately impacted. These systems threaten to embed structural discrimination into automated decisions across sectors; amplify misinformation at scale, especially in non-dominant languages; automate and supercharge state surveillance and repression; and undermine human autonomy and democratic oversight by concentrating power into technologies held in state and corporate hands.

We cannot manage these risks by simply checking whether the AI produced the right output at the end. We need to examine the entire process: What data can the system access and retain? How does it translate a user’s request into sub-goals? What external systems and data can it access and/or manipulate? How much authority does it have to act without human approval? Does its operation change over time? Can affected people understand why a decision was made, contest it, and reverse it if it was wrong? And who is accountable at each stage of the process?

Recommendations and takeaways from the discussions will be shared publicly and used to directly inform ECNL and Humane Intelligence’s research on impact assessments and evaluations for agentic AI. Participants will be invited into a growing community of practice, with follow-up workshops, ensuring the expertise shared here shapes advocacy and practical standards beyond UNGA and builds momentum toward governance models that redistribute power.

This is an invite-only event. To request an invite, email [email protected] with a short bio and work in agentic AI.