When interacting with humans, seek reliable information

We interact with humans every day but is what we are getting back reliable?

For example, if you ask someone about the prevalence of tornadoes and asthma, tornadoes are seen as more frequent killers than asthma, although the latter cause 20 times more deaths. Daniel Kahneman explains in Thinking Fast and Slow,

• “People tend to assess the relative importance of issues by the ease with which they are retrieved from memory — and this is largely determined by the extent of coverage in the media.”

If you ask someone how many times they plan to work out this week, they will give you an answer that represents their ideal self, not their real self. If you want a more reliable answer for how many times they will work out and a better representation of their real self, you should ask them how many times they worked out last week. James Clear explains in Atomic Habits,

• Your actions reveal how badly you want something. If you keep saying something is a priority but you never act on it, then you don’t really want it. It’s time to have an honest conversation with yourself. Your actions reveal your true motivations.

And, if you want to know why people use your product or service, observe them in a more natural environment. Don’t invite them to your office and ask them point-blank.

The intent is to get reliable information with our human interactions but our tendency is to dive in as we always have. If you are an adult and reading this, this is ingrained into you. Oren Jay Sofer explains in Say What You Mean,

• “we are stuck in habitual, conditioned views, and our attention becomes narrowly focused on a way of seeing that is rooted in experiences from our past. Our whole nervous system enters a familiar pattern, based on our ingrained views and corresponding intentions.”

Thus, if you desire reliable information with your human interactions, it requires you to break from these patterns.

In a business setting, we call our interactions with humans “qualitative research” or “customer development” or “usability testing” or “customer research”. For a lot of people, these methods carry a lot of stigma and skepticism.

It’s easy to understand why. Pick up the phone and start talking to your customers. Ask them how much they’d pay for your new product, or if they like your website, or if they have ideas to recommend.

This will feel good. You will think you are making progress. But, you will never see results.

In the quantitative research world, it’s easier to see what’s happening, because it has a name:

• False-positive — a result that incorrectly indicates that a particular condition is present.

In the qualitative world, we don’t have a name for it but it still exists. Thus, as false positives accrue, qualitative research methods begin to lose their credibility.

If you want to see this widespread loss of credibility, search twitter for this alleged Henry Ford quote.

Steve Jobs is another famous reference point for this lack of credibility.

But, before becoming enamored with quotes that glorify creative geniuses, you should know that even creative geniuses have a method to their madness.

Because, as with most things in life, the more complex something is, the more care it requires.

This care comes in our method. Our method needs to be grounded in principles that give us reliable information.

We seek reliability by identifying trends across many people, avoiding cognitive biases, not asking leading or speculative questions, and valuing what people do over what they say.

Teresa Torres explains that we should elicit storytelling,

• “We rarely want to ask people our research questions directly. Instead, we want to translate them into interview questions. Interview questions are what we ask to elicit stories. We want people talking about what they care about, not what we care about, because what matters most is how your customers frame their own problems.”

We set standards in our quantitative research. So, start setting stronger standards in your qualitative research.

Require 95% statistical significance. Require replication and peer review. Require consistency across populations.

Require reliable information.