Who can afford to participate?

For most of my career I said participants were giving us their time. It's a generous phrase, and it hides a great deal.

Participants are working. We scheduled the session, set the agenda, recorded it and took the output back to a company that will make money from it. All of that describes a job, except that nobody gets paid.

People with flexible time say yes when nobody is paid, and they aren't a random sample of anybody.

They're retired people, people between jobs and the self-employed. They're also advocates who already work close to our field and have decided to spend their energy this way.

These are thoughtful participants and I've learned an enormous amount from them.

But as a group they aren't the people our products are hardest on. The hourly worker on a split shift is missing, and so is the parent whose evenings are already allocated.

So is the person whose energy on any given day is finite and already spoken for. That describes a great many disabled people, and it's the condition our interfaces handle worst.

So an unpaid study recruits for spare capacity. Then we write "we tested with disabled users" in the report, as though the category were the sample.

What comes back is a friendly sample and a flattering result.

There's a second effect that gets even less attention. The same few people say yes to everybody, and over a few years they become semi-expert.

They're fluent in our vocabulary, patient with our prototypes and skilled at describing a problem in terms a product team can act on.

That fluency is useful, but it isn't the same as being representative. A participant who can tell you the heading level is wrong here has already done half of our job for us.

The person we most need to hear from is the one who can't name what went wrong and leaves. That person is very rarely in a study, because nothing in how we recruit reaches them.

We both believe our field does not intend to study only the reachable. It happens when nobody owns the recruitment line and the payment line is the first to be cut.

We also ask for more than the hour in the calendar invitation, and the extra isn't shared out evenly.

There's getting there, or getting a room and a connection sorted. There's setting up assistive technology on a machine that isn't theirs, or arguing with ours.

I've sat in sessions that ran long because our equipment wouldn't cooperate with a participant's, and every minute of that overrun was charged to them.

There's the energy the session takes and the energy it takes afterwards, which for a lot of people is the larger number.

And there's a cost we almost never count, explaining your own disability to a stranger, again, as the price of admission to a conversation about a product.

Then there's a complication I didn't know about for most of my career (nothing in my professional training mentioned it).

In the United States, payment can interact with disability benefits in ways that matter a great deal to the person receiving them.

So "We will pay you" is not automatically a kindness. It can be a problem someone now has to manage, and they may not want to explain why on a recruitment call.

I don't take that as an argument against paying people, only as a reason to ask, early and privately, and to have more than one answer ready.

Decent practice costs far less than a grant

At the Rhonda Weiss Center we ran more than thirty studies with disabled participants around a stakeholder group of about eighty people.

A federal grant made the logistics fundable, but most of what I took from it costs far less than that.

• Pay at the same rate you pay any other participant. Don't pay less because the session was remote, and don't give a gift instead because the person seemed glad to help.
• Offer a choice of how. Cash, a card, a donation to somewhere they choose, or declining the payment without losing the slot. Ask in writing, before scheduling, and don't ask them to justify the answer.
• Ask about access needs before you schedule, not after. A session booked first and adapted second has already told the participant where they came in the ordering.
• Budget the logistics as their own line. That's recruitment, scheduling, setup time and coordination. This work decides whether study two happens, and it's invisible in every research plan I have ever seen.

And pay stakeholders too, not only usability participants. Advisory time is often more work, spread over months, and it's the category most likely to be treated as goodwill.

Open the budget for your next round of research and find the line for participant payment. If it isn't there, that is already a finding.

If it exists but has never been increased, ask who has been saying yes at that rate, and who has been priced out of telling you what is wrong.