Avoidable Ignorance

Avoidable Ignorance - Reflection #028

I was looking at some promotional results recently, and something wasn't making sense.

I'd been asked to evaluate a short-term, localized marketing initiative. The objective was straightforward – determine whether the promotion was generating incremental business.

I had layers of data, control markets, and a conventional analytical approach.

I also knew about several broader promotions during the pre-period that could distort the results. I'd accounted for those. Check.

But the initial results didn't make sense.

My first thought was that I'd screwed something up in my data pulls or analysis templates.

Nope.

My next thought was that maybe the promotion hadn't been executed as planned.

This created some organizational swirl… but nope again.

What the heck?

What hadn't occurred to me was that another part of the organization had recently run an entirely separate marketing promotion in one of the regions I was evaluating.

For thirty days… during my pre-period.

Nobody was hiding anything. The activity had been approved and executed.

Some in the organization knew.

I didn't.

And now I wasn't certain I could fully account for its effects in my analysis.

The information existed.

It just hadn't found its way to the people whose decisions it might have changed.

I've encountered variations of this problem throughout my career… more times than I care to recall.

A regional manager runs a contest.

A local market sponsors an event.

Someone introduces an incentive, distributes coupons, changes operating procedures, or adjusts staffing.

Each activity almost always makes perfect sense to the people responsible for it.

Sometimes these activities are broadly communicated.

Sometimes they're documented somewhere nobody thinks to look.

And sometimes they aren't documented at all – just living in a handful of heads.

Then someone sits down to evaluate something or make a consequential business decision.

They have data.

They've got analysis.

They think they understand what happened… but…

The organization actually knows more than the decision-makers do.

We talk a lot about making data-driven decisions.

We invest enormous amounts of money in systems that collect, process, and analyze information.

We have dashboards for almost everything. Usually way more than we need.

We can produce “sophisticated” analyses faster than ever before.

And yet we still make decisions without information our organizations already possess.

That bothers me.

I'm not talking about things we genuinely couldn't have known. There will always be unexpected events and circumstances we couldn't reasonably anticipate.

Some of Rumsfeld's famous “unknown unknowns.”

Those are legitimate gaps in knowledge.

But there are also things we don't know that someone else in our organization does.

When that knowledge might affect a decision, but never reaches the people making it, we're dealing with something different.

Avoidable ignorance.

A decision can be reasonable given the information available and still be avoidably uninformed.

The analysis might be empirically buttoned-up.

The recommendation might be consistent with the evidence available.

And the conclusion might change substantially if someone had simply known about a contest, a promotion, or an operational change.

Sometimes we discover that missing information before making the decision.

If we're lucky, afterward.

And unfortunately, sometimes we never discover it at all.

But the decision got made…

For years, I've found myself repeatedly grappling with this same frustration.

There has to be a better way.

Not another dashboard.

Not another statistical model.

Not another system producing increasingly precise answers without improving our understanding of the conditions that produced the data.

Something that helps preserve what an organization knows.

What happened. When it happened. Where it happened. What we believed. What we assumed. What we measured. And what the evidence established.

Something that makes consequential information available when someone needs it – even if nobody recognized its importance when it was recorded.

That's a large part of why I decided to build LiftConductor.

A system for creating and preserving decision integrity over time.

The ambition isn't simply to help organizations perform better analyses.

It's to help them make better-informed decisions, understand the evidence behind those decisions, and preserve that understanding for the next decision.

Because knowledge shouldn't disappear when a project ends, someone changes roles, or a conversation is forgotten.

And the answer to “What did we know when we made that decision?” shouldn't depend on finding the right person to ask.

Of course, I know this won't eradicate the problem.

People are still going to do things.

They're going to run contests, launch promotions, sponsor events, change operations, and respond to circumstances.

Some think to document those activities.

Some don't.

And some won't recognize that what they're doing might matter to an entirely different decision six months later.

Systems can't preserve information they don't know about.

And no system can guarantee that everything it receives will be recognized as relevant.

But I have a hard time accepting that the alternative is to keep discovering consequential information after the fact.

We may never eliminate avoidable ignorance.

That doesn't mean we shouldn't try to reduce it.

For lack of a conclusion…

We can't know everything.

But we ought to do a better job of preserving what we already know.

I believe there has to be a better way.

I'm working on one. Stay tuned…