Home About The Work Field Notes
TDCI Scorer Connect
The Root Cause  |  Issue 06  |  May 2026

Demo is the Diagnosis

Agentwashing is what vendors do. Agent-shaped is what they ship. Three questions for the demo pod floor, and the one that comes before them.

Read time
10 min
Home / Field Notes / Issue 06

The demo always works. Until it doesn't. That moment is the diagnostic.

There are two problems with the agentic AI being sold to enterprises in 2026. A significant portion of it is not agentic AI. The portion that is, cannot be evaluated through the demos vendors run.

Before you can run the diagnostic, you have to know which problem you are looking at.

Is the agent you were just shown actually an agent? Most enterprise buyers do not know to ask. The ones who do usually accept the wrong answer.

Gartner has a name for the practice that makes this confusion possible. Agentwashing. Labeling AI assistants as agents when they do not operate independently. On the demo pod floor it shows up differently. Agentwashing is what vendors do. Agent-shaped is what they ship.

When a demo breaks on stage, you learn more in fifteen seconds than the rehearsed flow gives you in an hour. The SE either knows the system well enough to recover, or the system reveals itself. Either way, the room gets honest.

The governance gap is the diagnostic

Every major vendor has a governance story now. Some are selling cures. Some are selling fear. The campaigns are getting sharper, the visuals more elegant, the language more confident. The buyer's questions are not.

Seven in ten enterprise executives say AI governance is slowing their transformation. That number is fresh out of IBM Think 2026. The same demos that are not showing governance are the ones being purchased.

That gap is not an oversight. It is the diagnostic.

To be fair, the demos do show governance. They show pieces of it. Approval workflows. Audit logs. A user clicking approve in a clean UI. That is governance theatre, not governance architecture. The pieces that matter for production safety are the pieces demos almost never include. Model-layer refusal under ambiguity. Tool-scope enforcement when the agent reaches beyond its boundaries. Recovery when the agent gets something wrong. Those are not on the pod floor because most vendors have not built them.

Two demos on every pod floor

The first is the one the SE wants to show you. It runs fast, it looks impressive, the agent always wins. The second is the one that reveals what is actually built. That demo is not on the schedule. You have to ask for it.

Buyers have always asked some version of this question. Show me the workflow. Show me the data model. Show me how it actually works. The good ones ask. The SE either shows them, or blames the demo data, or promises to come back later with the customer's data set up properly.

Watch for the other tells. When the SE switches from the live demo to slides midway through, the system is breaking. When they say "let me show you in the other environment," the system has already broken. When they restart the demo with a fresh dataset, the foundational layer revealed something they did not want you to see. Practitioners learn to read these. Buyers should too.

Those answers worked when software was deterministic. They do not work for an agent. An agent that cannot handle data it has not seen is not reasoning. It is running paths.

The questions need to evolve as the system does.

Agent-shaped, not agent-functional

I have watched engineers stand in front of customers and call something an agent that is a hardcoded if-this-then-that workflow with an LLM bolted on for cosmetic effect. The thing was agent-shaped. It looked like an agent. It sat in the place an agent would sit. It was not an agent. The engineers knew it. The customers did not.

The difference between agent-shaped and agent-functional is whether the thing reasons.

There is one test for this, and it is the cleanest test you will run all day.

Ask the SE what the agent will do in a scenario they have not tested. If they can tell you with certainty, the system is not reasoning. It is executing. If they say I am not sure, let us try it, you might be looking at an agent.

The test takes thirty seconds. It cuts through more marketing than anything else you can ask.

If the system passes the test, you can run the diagnostic.

The three diagnostic questions

The diagnostic is three questions. Each tests one of the three things agents do differently from deterministic software. Reasoning in ambiguity. Acting with consequence. Failing in production.

The first question is about reasoning.

Show me the agent encountering a request you did not anticipate, and walk me through how it decides what to do.

Agents are non-deterministic principals. Traditional software demos show you what the system does for expected inputs. That was appropriate when the system was deterministic. Agents are interesting and dangerous precisely because they handle unexpected inputs in their own way. A vendor who can only show you the expected-input behavior is showing you choreographed software pretending to be an agent. If the SE cannot show you reasoning under pressure, the agent does not reason. It just executes.

The second question is about consequence.

Show me the agent attempting to reach beyond its scope, and show me what stops it.

The failure mode of agents in production is almost always tool-related, not reasoning-related. The agent makes a decision. Then it takes action through tools. If the tools are not scoped properly, the agent's worst decisions become production damage. Cursor and Railway and PocketOS was a tool-layer failure. The agent had access to production credentials and used them. The question tests whether the vendor has thought about constraint at the action surface. Most have not. If the SE cannot show this, the agent is one bad decision away from production damage every time it runs.

The third question is about failure.

Show me a scenario where the agent gets something wrong, and walk me through how it is caught and reversed.

Agents will get things wrong. The only question is whether failures are recoverable. Most demos show the happy path because the vendor does not know what the recovery path looks like, or because there is not one. The question tests for forensic capability and reversibility. Auditors care about this. Regulators care about this. Engineering teams often have not gotten there yet. If the SE cannot show this, the agent is shippable to engineering and unshippable to compliance.

Three questions. Three layers. The agent that passes all three is doing something real. The agent that fails any one is not ready for the floor.

A note about the keynote stage

Walk any mainstage in the past twelve months. Cloud Next. Frontier. Think. Ignite. Dreamforce. Knowledge. The pattern is universal. When a vendor labels a capability visionary on stage, they are telling you the truth in a way they did not intend. Visionary means it is not on the demo pod floor. It is not in production. It is not yet built. Even the most thoughtful design leaders at the largest enterprise software companies sometimes have to use that word. When they do, they are confirming the gap between vision and floor.

That gap is the diagnostic gap.

Twelve years on the other side

For twelve years at ServiceNow I sat on the other side of these evaluations. The Gartner Magic Quadrants. The Forrester Waves. The IDC scoring sessions. Those demos had to be live, and live demos sometimes broke.

One I remember in particular. Enterprise Service Management. A true end-to-end demo across IT and HR, breaking the silos that had defined those departments for decades. An onboarding and manager workload sequence. The pitch was the platform as single source of truth, IT and HR data flowing together across the employee lifecycle. Mid-demo, the IT data did not display. Then a HR paternity leave field did not render correctly into the scheduler automations downstream. A bug, a config, or both. The choreographed sequence stopped working in front of the room.

The only path forward was the foundational layer. The data model. The configuration. The actual mechanism underneath. I had to show that the data was there, that it could be configured, that the platform really was the source of truth even when the rehearsed flow did not show it that way. The score recovered.

The pattern across those years was simple. When the SE or PM could go to the foundational layer and show how the thing actually worked, the score recovered. When they could not, the score fell.

The break was never the problem. The recovery was the diagnosis.

The three questions in this Field Note are the starting kit. They cover the three things that make an agent an agent. The full diagnostic adds two more layers. Integration, which covers audit and observability. Workflow, which covers human gates for irreversible actions. Five layers total. Scoring criteria for each. Examples of strong vendor answers. Examples of evasive ones.

The full diagnostic ships in the coming weeks as a landmark page on thinkroot.io. If you want it when it drops, subscribe to the Field Notes.

The next time you stand on a demo pod floor, run the test. Run the diagnostic. Watch what the SE shows you. Watch what they do not.

If a vendor passes, share what you saw. If a vendor fails, name what was missing. The methodology spreads through use.

The demo is the diagnosis. Analysts have been reading it for twenty years. Buyers are about to.

Think from the root.

Chad thinkroot.io
A note on the diagnostic series

Issue 04 named the framing problem. The horseless carriage we keep reaching for when something new arrives. Issue 05 named the layer where governance is failing open by design, with the heuristic that started this diagnostic conversation. Show me the agent attempting something it should not be able to do, and show me where it gets refused. Issue 06 turns that heuristic into a usable diagnostic for any agentic demo. The full five-layer methodology ships next as a landmark page. Carriage. Engine. Diagnosis. Methodology.

Read the full series →

Issue 05: The Carriage and the Engine All Field Notes

Get each issue
when it drops.

No cadence promises. No filler content. Published when there is something worth saying to the people who need to hear it.

Subscribe via Beehiiv →Free. No schedule. Just signal.