Which permissions should an AI agent receive?

Separate reading, proposing and executing

List the operations the application needs. “Use the CRM” might become looking up customer details, producing a summary, preparing a draft note and saving a note. Define permitted records and conditions for each verb. Employees can then receive useful suggestions without giving the model broad rights to change data directly or choose its own access boundary.

Distinguish internal effects from external consequences. An internal draft task is generally easier to correct than a message to a customer or a payment instruction. The appropriate boundary depends on the process and agreements, rather than only financial value. Even a free address change can affect delivery, access or confidentiality. Discuss those consequences with the people responsible for the underlying work.

Base access on dependable identity

The application needs to establish who is making the request and which organisation they represent. An email address mentioned in a conversation is not a reliable account check. Use the existing sign-in process and have the application determine the current role. Then restrict data and functions to what that user needs for the defined task.

A shared technical account with broad rights makes this harder. Investigate minimum scopes and separate credentials where the connected service supports them. If an external API provides only extensive access, record the limitation and consider a controlled intermediary. Such a layer can narrow the functions exposed to the agent, but does not remove the need to assess underlying access and data flows.

Validate permission outside model output

A model can propose a function call containing a customer reference or a requested change. The application should then check completeness, user access and whether the operation fits the agreed task. OpenAI’s function-calling documentation describes a collaboration in which application code actually processes the requested function, rather than the model directly performing the business operation.

Refresh sensitive state where required. A suggestion may have become outdated because another employee changed the record. Check the current case status before committing an action, for example. Bind approval to the specific operation and relevant record version. General consent at the beginning of a conversation may not remain appropriate for a materially changed suggestion several steps later.

Alongside technical permissions, define waiting, review ownership and execution after approval: Human approval in workflows: make authorisation meaningful.

OpenAI: Function calling ↗

Make approval understandable

An employee needs to understand what they are approving. Show the intended recipient, proposed change, relevant source record and any unresolved uncertainty. Avoid a single “continue” button if the consequences are hidden elsewhere. Where there are several operations, explain whether approval covers one action or an entire sequence and make the scope visible.

Define how long a proposal remains valid. An operation that seemed appropriate yesterday may be wrong today. Expired or changed suggestions should be reviewed again. Rejection also needs a practical outcome: employees can correct the proposal, request more information or return to the ordinary route. If every departure requires a technical support ticket, people may approve automatically just to finish their work.

Treat retrieved content as untrusted input

Customer emails, documents and websites can contain text intended to redirect an agent. This remains possible when a file comes from an ordinary business system. OWASP describes prompt injection as a risk in which such instructions influence behaviour. A retrieved passage is information to assess; it does not grant permission to open other systems or transmit data.

Reduce potential consequences through several controls. Expose only required tools, validate their arguments and keep execution behind ordinary access checks. Include suspicious or unusual requests in testing. Filters and model instructions can help, but do not provide absolute protection. The design should limit what unwanted model output can cause even when the model follows an inappropriate instruction.

OWASP: Prompt Injection ↗

Illustrative permissions matrix for a service assistant

A fictional maintenance company starts with three capabilities: searching public manuals, reading a signed-in customer’s own appointment and drafting a rescheduling request. A person still changes the schedule. The assistant gathers information without deciding which technician or customer receives priority. Each draft clearly identifies the appointment to which it relates, with no implied confirmation of a new time.

A later release allows an employee to approve one proposed change. The application rechecks the customer, availability and appointment version. Requests for another customer, missing time windows and expired proposals are rejected. If a failure occurs after approval, the system checks the actual schedule before trying another write. The matrix therefore describes behaviour under uncertainty as well as the normal successful path.

Test boundaries and revocation

Test a different organisation, a revoked role, a changed record and a request concealing several operations. Check that refusals do not reveal additional records or confidential information. Exercise the legitimate route too: employees still need to complete work when a request properly falls within their authority. A system that only demonstrates blocking is not ready for useful operation.

Practise removing access. When an employee leaves or a supplier changes, identify the sessions, credentials and integrations that must end. Verify that the actual task stops afterwards. A record marked “access revoked” is not sufficient evidence if an existing session can continue changing data. Record the exercise and any limitations so that operational ownership remains clear.

Assign ownership and support investigation

Appoint an owner for the permissions matrix and review changes before exposing new functions. Retain enough evidence to investigate an action: time, record, requested operation, approval and actual outcome. Limit logs to what this purpose needs, and agree who can inspect them and how long they are retained. Broad access to complete conversations is not automatically necessary.

Give employees a straightforward way to pause execution. They should know which work continues, where outstanding suggestions remain and how the ordinary process stays available. A capability is manageable only when it can be stopped, investigated and deliberately re-enabled. That turns the permissions matrix into an operating tool rather than a document that becomes obsolete immediately after launch.

Turn the explanation into a project decision

Record your assumptions, unknowns and acceptance checks in the matching tool. Use the service guides to discuss delivery and scope.

• AI agent development ↗
• AI chatbot development ↗

Questions about this topic

Continue with a related question