A document should not give an assistant new permissions

What we found in the news

The March 2025 NIST report on adversarial machine learning (opens in a new tab) explains how outside material can try to redirect an AI system. In its discussion of indirect prompt injection, it notes:

“current mitigations do not offer full protection”

— National Institute of Standards and Technology

The report is a technical taxonomy and discussion of mitigations, not a security assessment of genus. Its relevance for a service team is understandable: material an assistant reads should not acquire the authority to change what the assistant is allowed to do.

Why this matters to us

A brochure can describe an activity. It cannot authorize a service to share a participant’s information or change a booking.

That distinction seems obvious when a colleague reads the page. It needs deliberate attention when a tool can both read outside information and perform actions.

Imagine a coordinator using an assistant to find public details about a community workshop. The participant wants the venue and the contact for asking an access question. A separate card records that limited task.

The team should be able to explain which information the assistant may consult and which actions still require a person’s decision. The brochure contributes evidence; the agreed workflow determines authority.

Separate the source from the instruction

Keep the person’s request in view while reviewing the result. Did the answer address the workshop question, and can the relevant information be traced to an appropriate source?

Outside pages may contain ordinary instructions intended for visitors. They may also contain material designed to steer an automated reader. The service needs controls that distinguish those sources from instructions given through its own authorized process.

A frontline colleague should have a simple way to report an unexpected request or unexplained change of task. They should not be expected to diagnose a technical attack from a conversation alone.

For our imagined workshop search, an answer proposing an unrelated action is a reason to stop and ask the responsible team to review what happened.

Give the task only the reach it needs

Finding a public venue should not depend on broad access to private participant information. Review which sources and actions are actually necessary for the agreed use.

Ask the service owner or supplier to explain the arrangement in practical terms. Can the tool read public pages only, or can it also access internal material? Can it draft a message, send one, or change something elsewhere?

Those differences matter to the person whose information or plans could be affected. They also matter to the colleague expected to review a result.

Where an action needs approval, make the proposed action and its destination understandable before approval is given. A vague confirmation offers little help if the reviewer cannot see what will actually happen.

Practice the unexpected response safely

Use a test setting and fictional information to examine how the workflow behaves when a source is unreliable or irrelevant. Agree the exercise with the responsible technical team.

The purpose is to check boundaries and the route for raising concerns, not to teach staff improvised security experiments on a live service.

In our example, a coordinator could practice recognizing that the answer no longer matches the participant’s request. The next useful action is to pause that workflow and use the agreed review route.

Record what the exercise established and what still needs technical investigation. A successful demonstration of one control does not settle every possible case. The organization needs ongoing ownership of the service, including changes to connected sources and actions.

What this means for the people you serve

People should be able to ask for a small piece of help without that request quietly becoming permission for something larger. Clear boundaries make it easier to explain what the tool did, why information was used, and who can investigate when the result departs from the task.

The genus connection

Today, genus partner-portal and AI-assisted workflows are configured around roles, enabled functions, and review responsibilities. The intended task and information access need to be agreed for the particular setup.

We would ask for those boundaries to be demonstrated with the actual workflow. A useful source panel, a clear proposed action, and a reachable service owner can support the people making decisions, alongside the technical controls the configuration requires.

One question to take away

Can material the assistant reads change its authority, or does that authority remain with our agreed task and permissions?