11. AI assistance, with people responsible
The most useful intelligence helps people bring their knowledge forward.
Industry perspective
“Center AI development on people’s needs, preferences, and goals throughout its lifecycle.”*
National Academy of Medicine · “Engaged” principle
Health Care Artificial Intelligence Code of Conduct · Accessed 2026-10-10
A principle in NAM’s AI Code of Conduct framework, presented here as a guide for thinking about responsible design.
Begin with a task worth helping
A frontline coordinator notices that participants keep asking the same question before a first visit. She has an idea for a short preparation tool, but no time to write a specification. She can describe the difficulty, the people affected, and what would improve their experience. An assistant can help turn that description into a draft the team can discuss.
That is a useful place to begin with AI: a real person, a recognizable task, and an opportunity to reduce the effort between noticing a need and exploring a response. The draft does not have to be perfect to make the conversation more concrete. It does need a clear path to review.
genus combines configurable AI models with partner knowledge, purpose-built tools, and application workflows. Depending on the use, this can include frontier models and more specialized systems. The model is one part of the arrangement. The instructions, information, available actions, and people responsible for review matter just as much.
Connection map adapted from carePhysics 3.7, chapter 9. The original fictional names are generalized here to keep attention on the distinct sources and responsible review.
- A person’s account, a supporter’s account, and a staff observation can differ; preserve their source and context.
- Use clinical guidance only within the appropriate professional remit and configured task.
- AI can help prepare a draft. An accountable person reviews it and decides what is appropriate to use.
Knowledge and purpose change the assistance
An assistant with a general understanding of community care is different from one configured with a partner’s approved resources, service boundaries, terminology, and priorities. Local knowledge can help the experience become more relevant to the organization and its audience.
That knowledge needs maintenance. A confident answer based on an outdated service description is still a problem. Partners should identify who owns the source material, which resources may be used, and how changes reach the experience. They should also test whether the assistant recognizes questions the material cannot answer.
Purpose determines which tools are appropriate. A public information assistant may help someone explore resources. A staff assistant may help draft a pathway or organize an innovation proposal. Permission to prepare a draft does not imply permission to publish, change records, or act for a person.
Layers of responsibility
The platform includes server-side checks around tool permissions, input structure, and approval requirements. Some workflows deliberately create proposals for human review rather than carrying out the proposed change. Those mechanisms help define what an assistant can do in a particular context.
A responsible arrangement brings several layers together: appropriate information access, clear instructions, bounded tools, review of consequential actions, and a way to inspect and improve behavior. The exact controls depend on the enabled capability. A partner should review that configuration instead of relying on the broad phrase “AI safety.”
Testing should include ordinary questions and difficult ones. Try missing information, conflicting instructions, an out-of-scope request, and a request involving someone else’s information. Check whether the response is useful, whether boundaries hold, and whether a person can reach help when the assistant cannot complete the task.
The carePhysics connection
Care AI — Bring Human Wisdom Within Reach places assistance within a wider understanding of care. The aim is to make useful knowledge easier to reach while preserving listening, choice, and human responsibility.
An answer can be technically fluent and still miss what someone needs. Communication and engagement models encourage a team to ask whether the response acknowledges the question, fits the situation, and offers a manageable next step. The person should be able to disagree, ask for clarification, or choose another route.
From a useful draft to a reviewed tool
The innovation workflow can help assemble a structured submission from an idea. The practical value is to make the challenge, audience, proposed benefit, and open questions easier for a team to assess. That can shorten the distance between a frontline observation and a shared working draft.
Moving the idea into use still requires decisions about scope, testing, privacy, accessibility, integration, and support. Some focused changes can move quickly. Others need substantial review. The goal is to keep useful ideas moving at the pace the work responsibly allows, with people able to see and improve what is being proposed.
Bring your knowledge
Choose one task where your team spends time translating its knowledge into repeatable work. Identify the approved information the assistant would need and the actions it should be allowed to take. Then define what a reviewer needs to see before accepting the result.
Test the arrangement with the people who will use it. Ask whether it saves effort, preserves their meaning, and makes uncertainty visible. Keep a route for reporting a poor answer or unsuitable suggestion. Feedback should lead to changes in content, configuration, or workflow, rather than simply asking people to be more careful.
Explore further — Questions for an AI configuration review
Ask what information the assistant can access, which model or service is involved, what data-handling terms apply, and which tools are enabled. Identify which actions require review and who provides it. Agree how failures are reported, how changes are tested, and what happens if the assistant is unavailable. These questions turn an abstract promise into an arrangement a partner can understand and operate.