AI can give you an answer in seconds. That is the easy part.
The hard part is everything that comes after. Is it the right answer? Is it the right answer to the right question? And what are you going to do with it once you have it?
We got into all of this on the latest episode of the KatAnu Connect Podcast, and the three of us did not agree on every point, which is exactly what made it one of our favorite conversations in a while. We went back and forth on what the human in the loop really means, why so many AI transformations are failing, and where the line sits between what AI should do and what only a human can do.
Here is the honest version of that conversation.
AI Sees the Data. We See the Story.
Everyone talks about the AI algorithm, the thing that chews through information and produces a set of outputs. We spend far less time talking about the human algorithm, and that is the one that matters most.
The AI algorithm produces outputs. The human algorithm turns those outputs into outcomes. AI sees the data, which is brilliant, because we miss things. But the human sees the story behind the data, and the story is where the decision really lives.
Three things stay with the human, and only the human.
- Context, because AI optimizes for speed while we have to ask whether the fast answer is the right one.
- Ethics, because just because you can does not mean you should.
- Judgment, because AI gives us probable answers and options, and someone still has to choose what makes the most sense for this team, this organization, at this moment.
| “AI sees the data. We, the human, we see the story behind the data.” ANU SMALLEY |
You Have to Be the Driver of the Train
Think of AI as a thought partner. An intern. A coworker who can move faster than you on the grunt work. A good one on a good day, a frustrating one on a bad day, and you cannot ever fully take your eyes off it.
Ask it for ten user stories and you might get five you would have written yourself, which just saved you an hour. You get two you would never have thought of, and they are genuinely interesting. You get two that are terrible. And you get one where you cannot even tell what it was thinking, because it went off the rails. That is roughly where the tools sit right now. They save you time, and they miss the point, in the same breath.
There is also a quality to the human experience that AI will never have. It is disembodied. It does not live in the world. It can tell you how many stars are in the sky, but it does not know what it feels like to sit under them. So whatever it hands back, you are still the one who has to drive the train.
| “It can chew through data faster than you, but it can’t do with the data what you and I can do. You have to be the driver of that train.” RYAN SMITH |
We are already watching what happens when people forget that. Work gets outsourced to AI, handed to an expert, and the expert says none of this is true, you made this up. We are seeing it in court filings built on cases that do not exist. That is not an AI problem. That is the human error of not checking. You cannot hand the whole thing over, not yet.
Where the Human Stays in the Loop
So where do we stay in the loop, and where can AI run on its own? Draw the line for every role on the team. For each one, ask where the AI job ends and the human job begins. The principle underneath it all is human-led AI, never AI-led humans, because AI-led humans leave you with a pile of outputs and no idea whether any of them are right for you.
Five workflow principles make that real:
- Define the question first. Know exactly what you want out of it before you write a single prompt.
- Review outputs as hypotheses. Treat every output as a starting point to test, not a verdict to accept.
- Name a human decision owner for every recommendation. Someone says nope, not that one, that one makes more sense for us.
- Put ethical checkpoints in. Build them into whatever you are deciding, every time.
- Protect the retro. Keep AI out of the room when the team is reflecting on the team. The moment someone says let me check what AI thinks of my retro, they have stopped bringing themselves to it.
Here is why context wins. A leader panics because velocity has dropped 20%. Ask AI what to do and it offers five sensible options, none of which apply. The real reasons were that two people had quietly checked out, the stakeholders were setting sprint goals without talking to a customer or the team in months, and the company had cancelled retros because they were not in the backlog. Morale was the cause. AI could never have seen that. The human in the loop is the only one who can.
Why So Many AI Transformations Are Failing
The numbers are sobering. Forbes reports that around 95% of enterprise generative AI initiatives fail. RAND puts the failure rate of AI projects at more than 80%. Those are enormous numbers, and the reason behind them is not the technology.
Organizations are failing because they do not understand why they are using AI. They saw a shiny squirrel, called it a silver bullet, and jumped on the bandwagon. We have lived this movie before. Twenty five years ago it was Agile. Companies bought Jira, ran a daily scrum twice a week, declared themselves Agile, and wondered why nothing changed. Show us where the Manifesto promises to make you faster. It does not. It is about the people. AI is no different.
The fix is not a new tool. It is a better question. Do not start with we are going to use AI and then go hunting for a job for it to do. Start with the problem you are trying to solve, then ask how AI can help you solve it. If you do not know where you are going, any road will take you there.
AI-Enabled Is Not AI-Native
One company handed out 1,200 ChatGPT licenses across the enterprise, and nothing happened. Then they paused. They took a small group, asked what they were doing, why they were doing it, and where they were struggling, and worked out how AI could genuinely help. From there the whole company shifted.
That is the difference between AI-enabled and AI-native. AI-enabled means you have the tools, you have a login to Claude or GPT or Gemini. AI-native means AI is woven into the ethos and culture of the organization, so it understands who you are, what you do, and why, and then works alongside you to offer options that you choose from, because you know you best.
It is the same reason a coach does not tell a client what to do. With thirty years of experience, why hold back? Because you are not the expert on them. AI is not the expert on you either. You are.
Use the Right Tool for the Job
Not all frameworks are created equal, and neither are AI agents. We teach leaders when to use Scrum versus Kanban versus XP. The same judgment applies here. Use one model for one kind of work and a different one for another, because you understand the power of each tool, not because one of them is winning the headlines this month.
And please, do not reach for a frontier model to find a meatloaf recipe. That is an expensive search engine. Knowing which tool fits the situation is the skill, and it is one we have to teach leaders, starting at the top.
The Ethical Guardian
This is the heart of it. The human in the loop is not only there to catch what AI gets wrong. The harder job is catching what AI gets right for the wrong reasons. A model can be 95% accurate as a prediction and still be 100% wrong for your organization.
AI is a baby right now. It will hand you hallucinations, because it has not learned enough yet, and because at its core it is a predictive model guessing what comes next in the sequence. It does not know the answer. It predicts one. The human is the only one in the room who can catch that, apply context and ethics, and decide whether the confident answer is the right answer for you.
| “We’re not using AI to replace thinking. We’re creating more space for higher quality thinking for the human in the loop.” KATE MEGAW |
What to Take Home
If you take five things from this episode, take these:
- Start with the problem, not the tool. Lead with the business problem, then ask how AI helps. The tool is never the strategy.
- Treat every output as a hypothesis. Test what it gives you. Do not accept it.
- Keep a human decision owner on every call. AI recommends. A named human decides.
- Protect the human spaces. Some conversations belong to people. Keep AI out of the retro.
- Learn when to trust it, when to challenge it, and when to override it. That is the skill the next chapter rewards.
One Last Thing
The future belongs to the people who know when to trust AI, when to challenge it, and when to override it. We are not trying to take humans out of the process. We are trying to take humans out of the repetitive work, so we have more room for the meaningful work only we can do.
Just because AI can do something does not mean it should. That is where you come in.
We will see you in the loop.