AI / Governance and risk

Using AI Isn't the Same as Getting Results from AI

You don't want people who refuse AI, and you don't want people who burn tokens for show. You want results. Here's how executives start measuring the difference.

Colin Dowling

Someone who implements AI for companies put it cleanly on my podcast: you don't want the person on your team who uses no AI, and you don't want someone who is AI-maxing just to run the tokens up. You want people who use those tools to maximize effectiveness and drive results. And we don't know how to measure that yet.

That last line is the part most board decks skip.

Activity is easy. Results are the job.

It's been my experience that the first AI reports look impressive. Logins up. Prompts up. Tokens up. Then the CFO asks what moved in the P&L, the warehouse or the project schedule — and the room gets quiet.

Using the tool is not the same as getting something useful out of it. One is attendance. The other is work.

The tension: adoption vs. effectiveness

On the one hand, leaders need people to try the tools or nothing changes. On the other, celebrating raw usage creates the wrong heroes: the person who pastes everything into a chat window and calls it transformation, and the person who refuses to touch it and calls it principle.

I kind of land in the middle. Require a fair try on named workflows. Reward the outcomes those workflows produce. Don't score people on how loud their usage graph is.

What "driving results" looks like

An example

Say a manufacturing firm rolls out AI for change-order summaries and vendor email triage. Team A logs heavy use and still ships change packages late. Team B uses the tool less often, but average turnaround drops from two days to four hours and rework tickets fall. If you only watch usage, you celebrate Team A. If you watch results, you study Team B and copy what they did.

Signs you're measuring the wrong thing

What to do this week

Pick one workflow where AI is already in play — claims drafting, bid research, exception triage, meeting notes into action lists. Write the outcome you care about in one line (hours to close, first-pass acceptance, customer response time). For the next ten working days, track that outcome next to usage. Ignore the vanity graph until you can see whether the outcome moved.

Common mistakes

Who should own this

The function leader owns the outcome metric for each AI-assisted workflow in their area. HR and finance should not invent a universal "AI score" that ignores the work. The CEO asks one question in reviews: what result moved, and how do we know?

Measure the work, not the wattage.

Start by measuring one outcome on one workflow for ten days. When you can tell a results driver from a token-maxer on that path, you're ready to talk about the rest of the org.

Questions

Should we require everyone to use AI?

Require a fair try on the workflows you've paved. Don't celebrate raw usage. Celebrate outcomes.

What if we can't measure AI value yet?

You're not alone — someone who does this work for companies said we still don't know how to measure it cleanly. Start with one before-and-after metric on one workflow anyway.

How do I spot token-maxing?

High activity, low change in cycle time, quality or rework. Peers spend time cleaning up the output.