UKG: asking 184 customers what they were trying to get done

A roadmap is a list of things somebody believes. This was an attempt to make it a list of things we could show.

Asking about the job, not the software

Outcome-Driven Innovation — Anthony Ulwick's application of Jobs To Be Done. The premise is that people do not want a product. They want a job finished, and they hire a product to finish it.

So the interviews ask about the job, not the software. Sixteen contextual interviews with recruiting admins produced 201 outcome statements: small, flat sentences about something a person is trying to get done, each one written so it can be rated rather than agreed with.

Then the statements go to a much larger group, who rate every one of them twice. How important is this to you. How satisfied are you with it today.

Why every statement was rated twice

Neither number means much alone. Important-and-satisfied is a part of the product that already works. Unimportant-and-unsatisfied is a part nobody misses.

The distance between them is what ranks. A job everyone needs and nobody can do well rises to the top of the list. A job already done well sinks, however important it sounds in a meeting. That property is what stops a roadmap rebuilding the part that already works, and it is why the method survives contact with people who disagree with it.

The scores band: above fifteen is an extreme opportunity, twelve to fifteen strong, ten to twelve solid, under ten not worth the money.

Who answered, and how many

683 customer companies invited. 184 people answered, from 148 companies, and left 784 written comments. Rebecca Tremaglio, Casey Wright and May Bamaung ran the study. The enterprise cut is Rebecca's and mine — the same instrument read against the accounts paying for most of the product.

I commissioned it because the same method had been run for Learning the year before and come back useful, which is a low bar and the only evidence anyone had.

Who we did not ask

The people rating the statements administered the system. Recruiting admins, coordinators, HR IT. Not one of them was a candidate applying through it.

That is a real hole, and naming it is part of the method rather than an apology for it. A survey of administrators tells you what administrators need. We asked the people who bought the software, not the people who had no choice about using it, and any claim I make from this study stops at that line.

What I would take to the next study

The finding was unflattering, and it set what my team worked on for two years. I have written about that in what the thirtieth study tells you.

The methodological lesson is smaller and more useful. A research instrument is worth commissioning when it can return an answer you did not want, in a form that is hard to argue with. Importance minus satisfaction is arithmetic. It does not care which team owns the feature, and that is the entire point of it.

The findings themselves are UKG's and are not published here.