How can we design for judgement, not just information?
Our tools are getting smarter. What about our decisions? My take on what's missing
As we’ve all embraced AI to help with our knowledge work in the last three years, something is starting to bug me—and it’s not the em dash or the overly confident, declarative writing style. Quite the opposite, actually.
The problem is that the output is getting very good. With the right context, I’m getting clear writing, accurate data, and well-structured reasoning from my AI assistants. That’s all great. But while the data and the information is often right, the interpretation is sometimes off. Not wildly off. Just off in a way that would lead to a bad decision if I didn’t notice it.
For example, when I ask an AI to synthesize a regulatory landscape for me, it pulls the right signals, cites the right stakeholders, maps the right risks. But when I read the brief, something is lacking. The logic holds and the facts check out, but the framing points somewhere that doesn’t match what I know about the client, the political dynamics, or the way this particular issue actually moves.
When I dig in, the answer is almost always the same. The AI framed the situation the way most people would frame it, by defaulting to the average interpretation, the one that usually emerges when you synthesize a large body of information and land on the most common lens. But as we know in this business, looking at the average rarely helps you make sense of your options.
I’ve been struggling to come up with a term for this, but I’ve landed on the “Mean Frame.” Actually, who am I bullshitting…I asked AI to help me come up with that name. And it works well enough.
The concept is this: The better and more polished AI gets, with more accurate, well-sourced outputs, the harder it is to spot the deficiency in the underlying frame it used to interpret the information.
This got me thinking about a broader question: we spend real money on information quality in this industry. Monitoring platforms, media tracking, stakeholder databases, sentiment tools, polling. But what do we spend on judgment quality? Sure, we spend money on smart people— I’ll come back to that. I mean the actual systems and structured processes that ensure the lens we use to interpret all that information is the right one.
I think the honest answer, for most of us, is nothing. And I think AI is making that gap more dangerous. Because when the brief looks this good, when the data is accurate and the writing is sharp and everything is well-sourced, we risk failing to question the lens. Why would we when the picture looks so clear?
I’ve spent the last few months writing about how institutional decisions break down:
How we commit too early when we fail to pressure-test.
How we self-censor because naming something outside the consensus window feels like career suicide.
I wrote those as separate failure modes and built a visual framework showing how they connect.
However, these challenges are symptoms of the same thing. Good people, with access to good data, lacking a structured way of challenging the lens through which all that data got interpreted. We rely on human judgement for that interpretation. Get the information right and the right decisions will follow, right? Wrong.
“That’s what our people are for. That’s why we hire experienced professionals.”
Well, consider this story I recently found in Damon Centola’s Change book:
Barack Obama is giving a lecture at MIT about how leaders make good decisions under uncertainty. He describes sitting at the table with his cabinet, with the staffers lining the edges. If you’ve ever sat in a political briefing, you know what this looks like. The data analysts, policy wonks, and the people with binders who actually do the work, sit behind their bosses.
Obama points out that those sitting at the table didn’t have time to look at the data. They skimmed summaries from senior staff, then — he said, only partially joking — explained them probably inaccurately. The real knowledge was at the edges of the room, with people who had been told by their bosses not to speak.
His fix was simple. He made a habit of calling on those peripheral staffers directly. They were terrified, but the president asked, so they answered — and they brought insight the polished summaries had compressed out of existence.
The President of the United States, backed by the most sophisticated intelligence apparatus in history — still had to contend with a judgment gap. The frame-challenging insight was right there in the room; it just had no way into the conversation unless Obama personally pulled it in. Every single time. That’s not judgment infrastructure. That’s one leader’s personal habit holding everything together.
We’ve all seen versions of this. I wrote about a client whose experienced team locked in a frame before the environment finished forming. I wrote about exceptional analysts who self-censored because the professional cost of being wrong outweighed the institutional benefit of being imaginative. Good people, real judgment, no system for surfacing it when it counts.
And the context keeps getting harder. The conversation where the frame gets set increasingly isn’t happening in a room — it’s a Slack thread at 9pm or a WhatsApp exchange between meetings, where a default interpretation slides through unchallenged before anyone’s had coffee the next morning.
If the most powerful office in the world needed a deliberate workaround to hear what was already in the room, what are the odds we’re nailing this by accident?
The Predecision — the technique I wrote about a few weeks ago — was my attempt at addressing this gap. When a recommendation feels ready, you force a collision; make the best case against your own position and keep the frame elastic long enough to genuinely test it. It works. But it works late.
By the time we’re in Predecision territory, the frame has already shaped everything: Which data got prioritized? Which scenarios made it into the deck? Which options the team even considered? If the frame was wrong from the start, the Predecision is just pressure-testing a conclusion built on the wrong foundation.
What’s actually missing is a discipline that operates upstream. Before the analysis runs, before the brief gets drafted, before anyone starts building toward a recommendation. A structured challenge to the frame itself — not the conclusions that flow from it.
The encouraging part? This discipline already exists. Just not in our world.
When a finance team produces a report, someone challenges the assumptions underneath the numbers — not whether the math adds up, but whether the model makes sense. Legal pressure-tests reasoning. Engineering runs QA against intent, not just against whether the code compiles. These functions figured out a long time ago that accuracy without the right frame is a well-organized path to the wrong answer.
I’m not sure we have anything like this in public affairs. The frame that shapes a campaign or a regulatory approach gets selected informally — usually by whoever speaks first with a coherent view, or by the most senior person in the conversation. Once it takes hold, everything downstream gets evaluated inside it. The data gets sharper and the analysis gets tighter, but the frame itself never gets questioned.
The fix is to take the same review discipline that already exists in those other functions and point it at the frame shaping our work, before that frame starts shaping the analysis.
In practice, that means asking a specific set of questions before the analysis starts, not after: What frame are we using, and who selected it? What alternative frames did we consider and discard? What would have to be true for a completely different lens to be the right one? What does this look like through the frame our opponents are using?
These questions should be asked explicitly, documented, and revisited when the environment shifts, the same way an auditor revisits assumptions when conditions change.
My theory is that when this discipline exists, the downstream difference show up fast. We walk into a briefing — or an async update — aligned on why we see things the way they do, not just what the data says. When the CEO pushes back, the response isn’t scrambling or folding. It’s explaining the frame, showing it survived challenge, and moving directly to what action to take.
That shift — from “is this analysis right?” to “what do we do about it?” — is where I’ve watched many clients burn weeks. The data was fine and the analysis was sound. Nobody could move because nobody had surfaced the frame underneath it, let alone tested whether it was the right one.
I’ve been building what I’m calling a capabilities index for teams that need to make better decisions under conditions that won’t wait for certainty. The previous essays each examined a specific failure mode — paralysis, premature commitment, self-censorship. This essay is about the structural absence that makes all three possible.
The capability is Judgment Under Uncertainty: the organizational discipline of challenging the frame before it shapes the analysis, not just the conclusion after.
We’ve built extraordinary systems for knowing more, and almost nothing for seeing clearly. Every dollar we invest in sharper AI-powered analysis without a corresponding investment in how we decide what it all means just widens the gap between how good our briefings look and how sound our decisions actually are.
The tools will keep getting better. That’s the easy part. Building the judgment to use them well?
Photo by Derick McKinney on Unsplash