Sufficient Beats Exhaustive

Sufficient Beats Exhaustive

Early in my career, I wanted people to see my work.

I mean really see my work.

The analysis. The methodology. The assumptions. The statistical tests. The alternatives I'd considered. Why I'd chosen one approach over another. The diagnostics. The caveats.

All of it.

Part of that was the academic in me. I'd spent years learning how to do rigorous analytical work, and somewhere along the way I'd absorbed the idea that rigor needed to be visible.

If I'd done the work, why wouldn't I show it?

I learned pretty quickly that most people (especially people running businesses) didn't particularly care to see everything I knew.

They cared that I knew what I was doing.

They cared that the analysis was sound.

They cared about what we'd learned, what it meant, what we should do about it, and what could go wrong.

And, perhaps most importantly, they had absolutely no desire to relive the statistics courses they'd struggled through - or happily forgotten.

It took me a while to get that distinction.

Good analysis needs to be rigorous.
Good communication needs to be relevant.

Those aren't the same thing.

“So what?”

I had a boss who was particularly good at making this point.

When analysts (and it felt like me in particular) would begin walking through everything we'd done, he’d interrupt and say “I need you to boil all of this down to just three things:

So what?

Who cares?

What difference does it make?”

Those questions could feel a little brutal when you'd spent three weeks buried in the analysis.

But they were 100% the right questions.

Another boss put it even more succinctly:

“Woody – I don't need to know everything you've learned since second grade.”

Noted…

Both were teaching me the same lesson.

The purpose of analysis isn't to demonstrate how much work went into producing it.

The purpose is to help someone understand something well enough to make a better decision.

And that changes what “complete” means.

Complete for whom?

We often talk about an analysis, presentation, or recommendation being “complete” as though completeness were an objective property.

I don’t think it is.

Completeness means different things to different people.

A CEO deciding whether to approve a significant capital initiative may need to know the conclusion, expected benefit, principal risks, and perhaps enough about the approach to trust the recommendation.

The executive responsible for implementing it needs more.

The analyst reviewing the work needs considerably more.

And someone attempting to reproduce the analysis may need nearly everything.

Same decision.

Same underlying evidence.

Very different definitions of “completeness” or “sufficiency.” 

The mistake is assuming that because all of the information exists, all of it belongs in every conversation.

It doesn't.

Sufficient beats exhaustive

My old logic courses introduced me to the distinction between necessary and sufficient conditions.

Decades later, I've found myself thinking about that distinction with a different wrinkle.

When communicating evidence, I increasingly prefer:

Sufficient beats exhaustive.

That doesn't mean incomplete.

It doesn't mean hiding inconvenient evidence.

And it certainly doesn't mean oversimplifying something until the uncertainty disappears.

It means giving someone enough of the right information to understand the decision at the level appropriate to their role.

That's harder than dumping “everything you’ve learned since second grade” on a slide.

Because exhaustive communication is often about:

What all do I know?

But sufficient (relevant) communication is about asking, answering, and delivering:

What does this person need to know?

This distinction is important.

Start with the decision

Over the years, I've developed a simple way of thinking about the first layer of analytical conversations:

CABR.

Conclusion. Approach. Benefits. Risks.

What did we conclude?

How did we get there?

What happens if we're right?

What should we be worried about?

Not every decision fits neatly into four boxes, of course.

That's not really the point.

The point is that the first conversation should usually begin with the information most relevant to the decision-maker  - and not with the chronology of the analysis.

If someone wants to understand the approach more deeply, show them.

If they want to examine the assumptions, show them.

If they want to see the model(s), show them.

If they want to challenge the evidence, assumptions, methodology, alternatives, diagnostics, or raw data...

Fair enough…

Here it is.

That's where progressive transparency matters.

Transparency doesn't mean showing everything at once

I believe strongly in analytical transparency.

But I've come to believe that transparency is sometimes confused with information volume.

Heft is not transparency – it’s overwhelming -  and more times than not, interferes with the message.

Transparency means the reasoning and evidence can be examined.

It doesn't mean everyone needs to examine all of it before a decision can be made.

Think of transparency as layers.

At the first layer, someone may need the conclusion, benefits, risks, and enough of the approach to understand why the conclusion deserves consideration.

Go one layer deeper and the assumptions become visible.

Deeper still, the evidence, alternatives, methodology, diagnostics, and limitations are available for scrutiny.

Eventually, someone should be able to trace the conclusion all the way back to the evidence that supports it.

Nothing important is hidden.

But not everything is forced upon everyone.

That's progressive transparency.

Expertise creates its own problem

There's another reason this matters.

The longer we work in a field, the harder it becomes to remember what it was like not to know what we know.

Concepts that once required enormous concentration eventually become obvious.

Terminology becomes shorthand.

Connections become automatic.

We forget how many intermediate steps disappeared as our expertise developed.

And that creates an interesting communication risk.

We can give people far too much because we want them to appreciate the complexity.

Or far too little because we've forgotten which parts aren't obvious.

Neither is particularly helpful.

Expertise isn't demonstrated by making something complicated sound complicated.

Sometimes the greater expertise is knowing which complexity matters right now… and which can wait.

The evidence should always be there

None of this is an argument for “trust me.”

Quite the opposite.

The executive may never want to inspect the model.

But the model should survive inspection.

The board may never ask to see the assumptions.

But the assumptions should be explicit and defensible.

The client may never ask why one analytical approach was chosen over another.

But there should be an answer.

That's the difference between simplifying the communication and simplifying the thinking.

The communication can be concise.
The reasoning underneath it cannot be careless.

That's the standard.

Do the exhaustive thinking.

Preserve the evidence.

Make the reasoning traceable.

Then communicate what is most appropriate for the person and decision in front of you.

And always leave the door open for scrutiny.

Because the measure of good analytical communication isn't how much of our knowledge we managed to jam into a deck.

It's whether the people in that room had what they needed to think clearly and decide well.

Reflection

The next time you're preparing to explain an important analysis, recommendation, or decision, ask yourself:

Am I showing them everything because they need to know it...

...or because I want them to know that I know it?