Beyond the Chat BoxEssay 04 / 10
Agentic interaction
ESSAY 04

The Agent Should Show the Boundary of Its Agency

Trustworthy agents make their operating boundary visible when it matters.

An AI agent may compare options, prepare a document, schedule a meeting, or carry a workflow across tools. A system that can act needs a different interface from one that can only suggest. The central question becomes: where does the agent's authority begin and end? A progress indicator is not enough. People need to understand what was delegated, what the agent may change, what it will do next, and which decisions remain theirs.

Delegation as a contract

Someone asking an assistant to organise a research meeting may approve finding times and drafting invitations, while keeping final sending authority. Another person may allow a calendar hold but not an invitation to an external participant. Boundaries should be explicit: goal, allowed actions, prohibited actions, limits, approval checkpoints, and an undo or recovery path.

A ladder of agency

A useful interface can distinguish observing, recommending, preparing, executing with approval, and executing within a standing rule. The current level should be visible and adjustable. An assistant should not quietly move from preparing to executing because it predicts that the user would agree. A draft should look different from a completed action.

Reversibility and consequence

Undo is not always possible. A sent message can be corrected but not unseen; a payment or booking may carry a fee. Actions should be classified by consequence and reversibility. High-impact actions deserve a specific confirmation that identifies the target and consequence, not a generic “Are you sure?”

Where this can fail

An agent may exceed task scope, reuse old permissions in a new context, or hide a failed action behind reassuring progress text. Users may also misunderstand whether an action was prepared or executed. Logs should show what happened, when, under which permission, and with what result.

Questions for research

  1. Do users understand the difference between a draft and an executed action?
  2. Which permission controls can be understood without technical knowledge?
  3. How does visible agency affect trust and willingness to delegate?
  4. Can users revoke authority mid-task and recover safely?

Design hypothesis

A helpful agent is not simply capable; it is governable. Agency should be a visible permission contract, not a vague personality trait.

Research starting points

  1. Amershi et al. (2019), Guidelines for Human-AI Interaction, CHI. https://doi.org/10.1145/3290605.3300233
  2. Shneiderman (2020), Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy, International Journal of Human–Computer Interaction.
  3. NIST AI Risk Management Framework. https://www.nist.gov/itl/ai-risk-management-framework
← Back to all ten essays