The Data Driven Falacy

Posted on September 6, 2023  •  4 minutes  • 714 words

“Show me the data.”

I stopped to think about what they were telling me. They were demanding irrefutable evidence that what I was saying was true.

But I never said it was true. I said it was my intuition based on my experiences being close to the problem and speaking with other people who were living with the problem daily.

But my experience didn’t align with their data so they decided what I said was wrong unless I could prove it. Data is what they required to consider my ideas. They weren’t going to hear any more until I produced it.

It wasn’t that I didn’t think I could gather the required data. I knew it still wouldn’t be good enough. The person I was dealing with was “data driven.”

They weaponized data to align with their opinions and goals. Worse yet, they didn’t have an opinion unless they discovered it in “the data.”

When we become blinded by data we can’t answer to anything else. We lose the ability to think for ourselves and have intuition about a problem, and we can no longer be proactive about solutions.

Being driven by data is one of the worse things that can happen to your company. You will lose sight of the road because you take your hands off the steering wheel. You become disconnected from your customers. And you lie to yourself and your customers because “that’s what the data says.”

If you think you’re headed down that road you need to ask yourself these questions:

1. What data am I not collecting?
2. Who has access to the data?

What data am I not collecting?

There’s no simpler way to confirm our own biases than to collect the data we want. When you dig through data you’re looking for supporting evidence and dismissing the rest.

If you’re collecting data you’ll always look at the sources you know about. You can’t collect data from places you don’t know exist.

And yet those places are often exactly where our intuition lead us. We struggle with something long enough and then emerge wondering how we got here.

You may not know the solution, but you have a new appreciation for the problem.

Who has access to the data?

My very first meeting at Amazon I thought I was in the wrong meeting. I must have logged into the wrong bridge because the document I was looking at had revenue numbers for the service I was working on. I was certain these numbers were intended for the executive team only.

Before I joined Amazon I never knew how important knowing the numbers could be to my job as an independent contributor (IC). My years at Disney I was never allowed access to any data I didn’t directly collect.

I never knew how much this impacted my decisions until I was trusted with access to the data. No longer was I questioning the impact of the work I was doing. I didn’t have to have prolonged debates about the most important work to do.

I had the data to help me make decisions faster, and it was shared with the entire team. Everyone had access to the some data and because we took the time to measure and present it on a weekly basis it became a driver. It turns out “you are what you measure” really is true.

How to succeed with data

You should be using data to prioritize work. If you’re using it to find solutions you need to spend more time with the problem.

Front line workers should have access to more data. You can’t bog them down with all the data, but they should have access to the most important data and more data provided if needed. Sharing data that drives the incentives you want is much easier and effective than withholding data and telling them what they should do.

Understand that no amount of data will be enough and it will always be biased. If you are waiting for the data to exist to act then you’re too late. Data—especially reliable data—is the last thing to come from emerging markets.

Data is what separates the early adopters from the majority. You can’t drive from the back seat. So stop letting data take the wheel.