Whose experience is missing from the demonstration?

What we found in the news

An October 2023 study in npj Digital Medicine (opens in a new tab) tested how four language-model systems answered selected questions about race and medicine. Researchers found unsupported race-based content in some responses and noted:

“The concerning outputs do not always appear”

— Jesutofunmi A. Omiye and colleagues

The study concerned particular medical questions and model versions. It cannot establish how a community tool performs. It does show why one convincing response is a limited basis for confidence, especially when a system may behave differently across attempts.

Why this matters to us

Imagine a demonstration of an assistant that helps people find a community activity. The presenter asks a clear question in familiar language and receives a useful answer. Everyone can see the appeal.

A participant later asks the same question using the name their neighborhood uses for the venue. Another asks by voice in a busy room. Someone else needs the answer in a format they can read with their usual support.

The demonstration did not tell the team what would happen in those situations. At genus, we want a useful first impression to lead to better questions about who can use the experience.

Ask what the demonstration actually covered

Write down the task, the version of the tool, and the conditions shown. Was the presenter using prepared examples, live responses, or a prototype? Each can be useful when its role is clear.

Ask how the team chose its examples. A demonstration may show a straightforward route while leaving ambiguity, missing information, and unusual wording unexplored.

Avoid treating a single failed attempt as the entire verdict either. It may reveal a wording problem that can be corrected, a support need, or a limitation that changes where the tool belongs.

The useful question is what the example establishes and what still needs examination before people rely on it.

Invite people to try their own version of the task

In our imagined activity finder, participants could bring the questions they would naturally ask. Staff who welcome newcomers might contribute unfamiliar place names or common points of confusion.

Offer the communication formats and assistance people choose. A test should help the team learn about the experience, without making participants feel that their ability is being graded.

Pay attention to whose contribution is absent. Someone unable to attend a group demonstration may still offer useful feedback through a brief individual session or another accessible route.

Use fictional details where a task would otherwise involve private information. The team can examine how the tool responds without collecting more personal context than the exercise needs.

Compare the difficult moments before expanding use

Record what happened in each case and which help was required. When appropriate, repeat a task to see whether its behavior is consistent enough for the intended use.

Look at the substance of the response. Did it preserve the person’s question, invent a service detail, or quietly steer them toward an option that did not fit? A friendly tone does not answer those questions.

Bring the findings back to the people shaping the trial. Explain which changes are proposed and which limits remain. A small test can reveal a useful next adjustment without claiming that every user or situation has been covered.

Decide what evidence would justify widening the trial and what would call for keeping a task with staff. Make that decision before enthusiasm for the demonstration becomes a service promise.

What this means for the people you serve

People’s ways of asking, reading, and participating should help shape the evaluation. A person who encounters a difficulty is contributing evidence about the design. They should be able to report it and understand whether there is another useful route while the team investigates.

The genus connection

Today, genus care-app and care-wall experiences are configured around audiences, content, access, and enabled features. Supported AI guidance has a defined role; its available functions and information access need confirmation for each setup.

We would want a proposed experience tried with the people expected to use it, using the actual task and configuration. The platform’s presence does not prove fairness or accessibility for every person. What matters is what the review establishes, what remains uncertain, and how the team responds.

One question to take away

Whose way of using this tool have we not yet seen—and how will their experience influence the next decision?