What does a model owe you when it is not sure?
Every tool I have built can be fluent about something it got wrong, and I had never thought of that as a design choice.
In a machine shop there is a tool called a go/no-go gauge. It has two ends. One end should slide over the part and the other should not. It cannot tell you the part is elegant, and it cannot be persuasive about a part that is out of tolerance. It fits or it does not.
Every accessibility tool I have built generates text. Which means every one of them can be fluent about a part that is out of tolerance.
What did we pay for string generation?
The failure I keep writing about on this blog has one shape. A model reaches for
a fact it does not have, and what comes back is a string: an aria-label from a
guess, a contrast ratio that is plausible and wrong, a spacing token that never
existed in the system.
The danger is not that the answer is wrong. Plenty of things are wrong. The danger is that a string carries no signal about how sure the thing that wrote it was. "4.6:1" looks the same whether it was retrieved or invented. There is no surface on it to grip.
So it ships, and the person who discovers the difference is the person who cannot check: someone with low vision reading helper text that was never graded, or a screen reader user told that a control is something it is not.
We built our review practices around output that can be read for doubt. Uneven work, a hedge, a comment that trails off. Generated prose is uniformly confident by construction, and we kept reading it as though confidence still meant something.
What changes when a model cannot write?
On 15 September, TypeSafe AI came out of stealth with Jev (opens in a new tab), which they call a System One model. The part that interests me is a constraint rather than a capability: Jev gives up string generation entirely.
You hand it some state and a typed question. It returns one of the options you supplied, with a probability distribution and a confidence score. There are three primitives (opens in a new tab). Choice picks from a list of up to 255 options. Score rates the state against a rubric. Noul answers whether a statement is true, as a number between zero and one.
It cannot return something outside the set you gave it, and it cannot return something malformed. The probabilities are calibrated, which is meant to mean that a low number is honest about being low.
I want to be exact about what that does. It does not make the model right. It makes the model's uncertainty into a value my own code can read, and those are different properties. For our work the second one is worth more.
Why does that matter for accessibility in particular?
Because our failures are the quiet ones, and a number is loud.
A wrong contrast value does not throw. A missing focus style does not fail a test. An invented accessible name does not look different from a researched one. Nothing in a normal pipeline stops on any of them, which is why they reach people instead of reviewers.
Now consider the same questions asked as typed ones. Which success criterion does this failure fall under, as a Choice across the 87 in WCAG 2.2, comfortably inside the 255 the model allows. Does this accessible name describe what the control actually does, as a Noul. Does this alt text narrate or does it merely label, as a Score against a rubric.
Every one of those comes back with a number attached. A 0.51 is a flag on a specific line. Under the arrangement we have now, that same uncertainty arrives as a fluent sentence with nothing on it at all.
A check that can say it is not sure is worth more than a check that is usually right, because the first one tells you where to look.
Nobody decided that our tools would answer confidently regardless of how much they knew. It came with the format, and we inherited it without examining it, which is the shape almost every default on this blog turns out to have.
What it does not fix, and what I have not measured
Jev cannot write the alt text. Writing the sentence is string generation, and string generation is the thing it gave up. So the generator that invents is still in the pipeline. What is new is that something downstream can grade what it produced and hand back a number instead of an opinion.
The claims are also young, and TypeSafe are more careful about this than the coverage has been. They describe the hallucination rate as zero and say plainly that the figure is theoretical rather than empirical. They note that the tested workflows were built by their own capabilities team, and that some bias could exist. They have not published benchmarks.
I have not run it against anything on this site. I have measured nothing here, and this is a reading of a design decision rather than a report on a run. Stars and launches are facts. Whether this makes any interface more usable is not, and I would rather say so than let the two blur.
What I am confident about is the shape, not the vendor. A model class whose failures arrive as numbers suits this work better than one whose failures arrive as prose, whoever ends up shipping it.
What I would point a typed question at
- Anything you currently trust because it sounds right. Accessible names, alt text, link text. These are the answers with the least surface for doubt and the most consequence when wrong.
- Anything with a number in it. Ratios, sizes, durations. A plausible number is the easiest thing in the world to produce and among the hardest to spot in review.
- Ask one thing at a time. The documentation is explicit that these models work best on a single well-scoped question, and that combining factors belongs in your code. That is also just better test design.
- Route on the confidence, not only the answer. A low score that goes to a person is a working system. A low score nobody reads is theater.
- Write down what the options are. You cannot ask a typed question without first enumerating the acceptable answers, and doing that will show you which of your standards were never written down.
- Keep measuring the rendered result. A model that grades a declared color is grading the wrong side of the problem, however calibrated it is.
The conventions here are a week old
A new model class arrives with its defaults unexamined, the way they all do. What gets asked, what happens to a low confidence score, whether anybody routes it to a human. None of that is settled. It is being decided right now, mostly by people building real-time control loops for games and drones, who have not been asked who gets excluded when a check quietly passes.
That is the cheapest this will ever be to influence.
Take the check in your pipeline you trust the most because it reads well. Work out what it would look like as a question with a fixed set of answers and a number on the end. Whatever you cannot enumerate is a standard your team has never actually written down, and finding that out costs you an afternoon.