{"id":11100,"date":"2020-11-09T10:55:33","date_gmt":"2020-11-09T10:55:33","guid":{"rendered":"https:\/\/www.experfy.com\/blog\/?p=11100"},"modified":"2023-10-09T10:27:52","modified_gmt":"2023-10-09T10:27:52","slug":"your-dataset-is-giant-inkblot-test","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/ai-ml\/your-dataset-is-giant-inkblot-test\/","title":{"rendered":"Your Dataset Is A Giant Inkblot Test"},"content":{"rendered":"\n

The danger of apophenia in analytics and what you can do about it<\/em><\/p>\n\n\n\n

There\u2019s a fine line between telling stories with data<\/a> and telling lies. Before I tell you how to spot a top-notch data analyst<\/a> and boost your analytical excellence, let me scare you a little.<\/p>\n\n\n\n

The psychological trap in data analytics<\/h2>\n\n\n\n

Human brains are pattern-finding powerhouses\u2026 but those patterns don\u2019t always have much to do with reality. We are the sort of species that finds rabbits in clouds and Elvis\u2019s face in a potato chip<\/a>.<\/p>\n\n\n\n

\"Your
Do these look like a rabbit and a portrait of Elvis to you? Image: SOURCE<\/a>.<\/figcaption><\/figure><\/div>\n\n\n\n

Take a moment to consider the Rorschach test<\/a> \u2014 the one where people are shown random inkblots and asked what they see \u2014 and you\u2019ll appreciate just how eagerly the mind injects spurious<\/a> interpretations into randomness.<\/p>\n\n\n\n

\"Your
Bat? Butterfly? Or just an ink blot? This is the first of the ten cards in the Rorschach test<\/a>, created in 1921.<\/figcaption><\/figure><\/div>\n\n\n\n

Psychologists have a pretty name for this tendency to conjure false meaning out of nothing: apophenia<\/a>. Give humans a vague stimulus and we\u2019ll find faces, butterflies, and a reason to allocate budget to our favorite project or launch an AI system<\/a>.<\/p>\n\n\n\n

Uh-oh.<\/em><\/p>\n\n\n\n

There\u2019s plenty of random noise in most datasets, so what are the chances there\u2019s no apophenia<\/a> going on with your analytics<\/a>? Can you really trust your interpretation of the data?<\/p>\n\n\n\n

What the mind does with inkblots it also does with data.<\/p><\/blockquote>\n\n\n\n

To make matters worse, the more ways there are to slice-and-dice those datasets and the more complex they are, the more vague they are as stimuli. That means they\u2019re practically begging you to see false nonsense in them.<\/p>\n\n\n\n

Complex datasets practically beg you to find false meaning in them.<\/p><\/blockquote>\n\n\n\n

Are you sure your latest data epiphany isn\u2019t an apophany<\/a> in disguise?<\/p>\n\n\n\n

\"Your
Another great word is pareidolia<\/a>, which is a kind of apophenia (finding familiar things in vague sensory stimuli). In Japan, they even have a museum<\/a> of rocks that look like faces. It\u2019s a beautiful world.<\/figcaption><\/figure><\/div>\n\n\n\n

Lies, damned lies, and analytics<\/h2>\n\n\n\n

If that sounds dismal, I\u2019m not done yet. Taking data analysis courses can pour fuel on that psychological fire. Students are conditioned to expect that looking at\u00a0data\u00a0yields real meaning because every homework exploratory analysis exercise has buried treasure in it. Very few professors have the heart to send you on wild goose chases (for your own good!) and it\u2019s hard to grade open-ended assignments, so you usually don\u2019t get enough exposure to them as a student.<\/p>\n\n\n\n

Students grow up believing that every dataset is ready to cough up a nugget of solid truth.<\/p><\/blockquote>\n\n\n\n

Data storytelling is just a hop, skip, and jump away from outright lying with data. Setting aside the issue of whether the patterns are real, let\u2019s talk about multiple interpretations. Just because you see a bat shape in that inkblot doesn\u2019t mean that there isn\u2019t also a butterfly, a pelvis, or a pair of foxes in it. If I hadn\u2019t mentioned the foxes, would you have seen them? Probably not. Psychological mechanisms related to motivation and attention have stacked the deck against you. It takes a special sort of skill to release the bat interpretation and force yourself to see a superposition of meanings.<\/p>\n\n\n\n

Once people glom on to their favorite \u201cinsight\u201d, they\u2019ll struggle to unsee it.<\/p><\/blockquote>\n\n\n\n

The trouble is that once people glom on to their favorite \u201cinsight\u201d, they\u2019ll struggle to unsee it in favor of others. People tend to believe most strongly in whichever interpretation captured their attention first and each additional meaning reduces their motivation to keep searching. Juggling multiple potential stories without overweighting your favorite is a mental muscle that takes hard work to build. Alas, not every analyst has the discipline for it. In fact, many are incentivized to \u201cprove\u201d one side of a story through data exploration. Why grow skills that only get in the way of engorging your data science paycheck?<\/p>\n\n\n\n

What color is your lightsaber?<\/h2>\n\n\n\n

There are ways to prove things with data (honestly and rigorously)\u2014 my data-splitting article<\/a> will tell you more \u2014 but exploratory data analysis (EDA) is not one of them. Open-ended data exploration is always a fishing expedition. What determines the color of your lightsaber is what you\u2019re fishing for.<\/p>\n\n\n\n

\"Your<\/figure><\/div>\n\n\n\n

If you join the dark side, you\u2019re fishing for evidence to support a theory you already \u201cknow\u201d to be true (so you can sell it to some naive victim). You might not even realize that your lightsaber is red if you genuinely believe in data objectivity and your own unbiasedness<\/a>.<\/p>\n\n\n\n

Open-ended data exploration is always a fishing expedition.<\/p><\/blockquote>\n\n\n\n

With a sufficiently complex (vague) dataset, you\u2019ll find a pattern you can spin as support for your favorite story. That\u2019s the beauty of the Rorschach test<\/a>, after all. Unfortunately, it\u2019s worse with data<\/a> than with inkblots because the more mathemagical <\/a>your method (p-hacking<\/a>, anyone?), the more legitimate and convincing you\u2019ll sound to those who don\u2019t know any better.<\/p>\n\n\n\n

\"Your
Satellite photo of the \u201cFace on Mars<\/a>\u201d which many people took as evidence of extraterrestrial habitation.<\/figcaption><\/figure><\/div>\n\n\n\n

Those who reject the dark side also go fishing, but they\u2019re after something else: inspiration. They\u2019re looking for patterns that might be interesting or compelling, but they know better than to take them as evidence<\/em>. Instead, they practice a sort of open-minded analytics zen with the discipline to be mindful of as many interpretations as possible.<\/p>\n\n\n\n

The best analysts challenge themselves to find as many interpretations as possible.<\/mark><\/p><\/blockquote>\n\n\n\n

This takes a sharp eye and a humble, unsticky mind. Rather than tricking their stakeholders into seeing only one side of a story, they challenge themselves to do the creative thinking required to digest the same data into as many stories as possible. They present their findings in a way that inspires rigorous follow-up without causing their leadership team to run overconfidently off a cliff.<\/p>\n\n\n\n

Open-mindedness gives data analysis a chance to be worthwhile.<\/p><\/blockquote>\n\n\n\n

As an added bonus, the discipline to look for multiple interpretations is an analyst\u2019s secret weapon for not snoozing past the real treasures buried in the data. If you\u2019re distracted by a falsehood you believe in, confirmation bias<\/a> makes it hard to notice evidence that points in the opposite direction. Why bother analyzing anything if your conclusions are determined in advance? Open-mindedness gives the whole endeavor a chance to be worthwhile.<\/p>\n\n\n\n

\"Your
This grilled cheese sandwich fetched $28,000 in auction<\/a> because it features the Virgin Mary. Alternative interpretations of what we\u2019re seeing, anyone?<\/figcaption><\/figure><\/div>\n\n\n\n

Hiring a great analyst<\/h2>\n\n\n\n

If you liked my other articles about analytics, here are the traits you\u2019re already looking for in a great analyst:<\/p>\n\n\n\n