{"id":10049,"date":"2020-10-05T12:08:51","date_gmt":"2020-10-05T12:08:51","guid":{"rendered":"https:\/\/www.experfy.com\/blog\/?p=10049"},"modified":"2023-10-25T09:46:14","modified_gmt":"2023-10-25T09:46:14","slug":"data-driven-think-again","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/bigdata-cloud\/data-driven-think-again\/","title":{"rendered":"Data-Driven? Think again"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

The psychological habit most people lack and why you can\u2019t hope to use data to guide your actions effectively without\u00a0it<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\n

Businesses are hiring data scientists in droves to make rigorous, scientific, unbiased, data-driven decisions.<\/p>\n\n\n\n

And now, the bad news: those decisions usually aren\u2019t.<\/p>\n\n\n\n

For a decision to be data-driven, it has to be the data\u200a\u2014\u200aas opposed to something else entirely\u200a\u2014\u200athat drive it. Seems so straightforward, and yet it\u2019s so rare in practice because decision-makers lack a key psychological habit.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

Data-drivenness destroyed<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\n

Imagine that you are considering buying something online instead of making a pilgrimage to the other side of town to fetch it. You\u2019ve boiled your decision down to whether or not you trust the online seller. A quick search yields some relevant data<\/a>: you see that the seller has an average rating of\u00a04.2<\/a>\u00a0out of 5.<\/p>\n\n\n\n

\n

Without decision-making fundamentals, your decision will be at best inspired by data, but not driven by\u00a0it.<\/em><\/p>\n<\/blockquote>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\n

Now you can\u2019t use that 4.2 to drive your decision. Game over! Once we\u2019ve seen the answer, we\u2019re free to pick the most convenient question. If the first thing we do is poke around in our data, our decision will be, at best, something I like to call\u00a0data-inspired<\/em><\/strong>.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

Data-inspired<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tThat\u2019s where we, like whales encountering plankton, swim around in some numbers, and then reach an emotional tipping point and\u2026 decide. There are numbers near our decision somewhere, but those numbers don\u2019t drive it. The decision comes from somewhere else entirely.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\n

The decision-maker\u2019s mind was made up before the data, so the decision was there all along. Turns out humans interact with data selectively to confirm choices we\u2019ve already made in our heart of hearts. We find the most convenient light in which to see evidence, and we don\u2019t always know we\u2019re doing it. Psychologists have a lovely name for this:\u00a0confirmation bias<\/strong><\/a>.<\/p>\n\n\n\n

\n

Many people only use data to feel better about decisions they\u2019ve already\u00a0made.<\/p>\n<\/blockquote>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

Fitting the question to the\u00a0answer<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\n

Is 4.2\/5 a good number? Depends on your unconscious biases. A decision-maker who really wants to make the online purchase will squint at that 4.2 and sing a happy song about how that\u2019s a high number. \u201cIt\u2019s more than 4.0!\u201d They can even show a rigorous analysis about how it is statistically significantly higher than 4.0. (With certainty! It\u2019s the\u00a0p-value<\/a>\u00a0you\u2019ve always wanted.) In the meantime, someone who really doesn\u2019t want to use that seller will find another way to frame the question in response to the data: \u201cWhy would I settle for a seller with less than 4.5 stars?\u201d Or perhaps \u201cBut look at those 1-star reviews. I don\u2019t like how many there are.\u201d Sound familiar?<\/p>\n\n\n\n

\n

The more ways there are to slice the data, the more your analysis is a breeding ground for confirmation bias.<\/em><\/p>\n<\/blockquote>\n\n\n\n

Mathematical complexity doesn\u2019t provide the antidote, it merely makes it harder to see the problem. As a result, what\u2019s obvious in the trivial example we just saw becomes hidden in a jumble of gorgeous\u00a0Gaussians<\/a>. Don\u2019t assume your friendly neighborhood data scientist sees it either. The more ways there are to slice the data, the more your analysis is a breeding ground for confirmation bias.<\/p>\n\n\n\n

The result? Decision-makers end up using data to feel better about doing what they were going to do anyway.<\/p>\n\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

An expensive hobby<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\n

When the analysis is complex or the data are hard to process, a pinch of tragedy finds its way into our comedy. Sometimes boiling everything down to arrive at that 4.2 number takes months of toil by a horde of data scientists and engineers. At the end of a grueling journey, the data science team triumphantly presents the result: it\u2019s 4.2 out of 5! The math was done meticulously. The team worked nights and weekends to get it in on time.<\/p>\n\n\n\n

What do the stakeholders do with it? Yup, same as our previous 4.2: look at it through their confirmation bias goggles, with no effect on real-world actions. It doesn\u2019t even matter that it\u2019s accurate\u2014nothing would be different if all those poor data scientists just made some numbers up.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\n

Using data like that to feel better about actions we\u2019re going to take anyway is an expensive (and wasteful) hobby. Data scientist friends, if your organization suffers from this kind of decision-maker, then I suggest sticking to the most lightweight and simple analyses to save time and money. Until the decision-makers are better trained, your showy mathematical jiu jitsu is producing nothing but dissipated heat.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

Antidote to confirmation bias<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\n

Problem:<\/strong>\u00a0you\u2019re free to move the goalposts\u00a0after<\/em>\u00a0you find out where the data landed. (Of course you score a goal every time. You\u2019re just\u00a0that<\/a>\u00a0good.)<\/p>\n\n\n\n

Solution:<\/strong>\u00a0set the goalposts in advance and resist temptation to move them later.<\/p>\n\n\n\n

In other words, the decision-maker has some homework to do\u00a0before<\/em>\u00a0anyone analyzes the data.<\/p>\n\n\n\n

\n

Until decision-makers are better trained, showy mathematical jiu jitsu only produces dissipated heat.<\/em><\/p>\n<\/blockquote>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\n

Framing the decision and setting decision criteria is a science of its own (we\u2019ll dive into it in future posts, as the problem we examine here is just the tip of the iceberg), but in the meantime a quick fix that goes a long way is to come up with your decision boundary up front in your data science project.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

Practice makes\u00a0perfect<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\n

I recently went clothes shopping in Brooklyn with my friend\u00a0Emma<\/a>. Showing off a pretty dress, she tugs at the pricetag on the back. \u201cHey, what does this say?\u201d she asks me. \u201cIf it\u2019s less than 80 bucks, I\u2019ll buy it.\u201d<\/p>\n\n\n\n

Now that\u2019s some decision intelligence! Instead of first seeing the price and then talking herself into a decision she\u2019s already made, she uses the data to drive it. With a well-practiced reflex, she weighs how much she likes the dress and her budget, then sets the decision boundary, and only allows herself to see the data (price) once that\u2019s done. She\u2019s in the habit of using data in the right order and that\u2019s a muscle you can exercise too.<\/p>\n\n\n\n

People don\u2019t always need to be data-driven and Emma knows that. She doesn\u2019t have to make unimportant decisions that way, but she also knows that practice makes perfect. It\u2019s much easier to build the habit on trivial decisions than to struggle when the important ones come around.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

Lessons from negotiation class<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\n

This idea is not new. Many different courses teach it, though one that\u2019s almost guaranteed to cover it on day 1 is negotiation. If you haven\u2019t put a value on your\u00a0BATNA<\/a>\u00a0(~ a walk-away point) before entering a negotiation, you may as well paint \u201cno idea what I\u2019m doing\u201d on your forehead. It\u2019s the same thing by a different name: figuring out your decision boundary between your\u00a0default action<\/a>\u00a0and the alternative.<\/p>\n\n\n\n

\n

The antidote is setting your decision criteria in\u00a0advance.<\/em><\/p>\n<\/blockquote>\n\n\n\n

In fact, standard advice for negotiators is to think through the entire range of potential offer combinations and plan your reactions to them in advance, otherwise it\u2019s very easy for an experienced opponent to take advantage of you. Even without all the persuasion tactics at your counterpart\u2019s disposal, irrelevant short-term factors like your blood sugar levels, your mood, how much the other party is smiling, and whether the sun is shining can have a disproportionate effect on the deal. Again, the same goes for data analysis\u200a\u2014\u200athink of the data as negotiating with you to\u00a0change your mind<\/a>. The antidote there is planning your response in advance. Next time you\u2019re negotiating a salary, for example, make sure you\u2019ve thought about your number\u00a0before<\/em>\u00a0you hear theirs.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

It\u2019s easy when you get the hang of\u00a0it<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\n

Whether you think about what a number means to you before or after you see it, you still have to think about it. Doing it beforehand helps you counter some of the bugs in your human programming, with large payoffs in decision quality and negotiation performance. Improving the order of operations here is a valuable habit to cultivate and crucial if you\u2019d like to be involved in data-driven decision-making. And here\u2019s some bonus good news: with practice it\u2019ll feel automatic.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"

For a decision to be data-driven, it has to be the data\u200a\u2014\u200aas opposed to something else entirely\u200a\u2014\u200athat drive it. Seems so straightforward, and yet it\u2019s so rare in practice because decision-makers lack a key psychological habit.<\/p>\n","protected":false},"author":335,"featured_media":10050,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[187],"tags":[688,693,694],"ppma_author":[2050],"class_list":["post-10049","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-bigdata-cloud","tag-data-scientists","tag-data-driven-decisions","tag-unbiased"],"authors":[{"term_id":2050,"user_id":335,"is_guest":0,"slug":"cassie-kozyrkov","display_name":"Cassie Kozyrkov","avatar_url":"https:\/\/www.experfy.com\/blog\/wp-content\/uploads\/2020\/04\/medium_df35f80d-2bff-4fe3-b741-a94d51320e00-150x150.jpg","user_url":"https:\/\/careers.google.com\/?src=Online\/LinkedIn\/linkedin_profilepage&utm_source","last_name":"Kozyrkov","first_name":"Cassie","job_title":"","description":"Cassie Kozyrkov is Chief Decision Scientist at Google, Inc. With a unique combination of deep technical expertise, and world-class public-speaking skills, she has provided guidance on more than 100 projects and designed Google's analytics program, personally training over 15000 Googlers in statistics, decision-making, and machine learning.\u00a0"}],"_links":{"self":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/10049","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/users\/335"}],"replies":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/comments?post=10049"}],"version-history":[{"count":5,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/10049\/revisions"}],"predecessor-version":[{"id":33719,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/10049\/revisions\/33719"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media\/10050"}],"wp:attachment":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media?parent=10049"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/categories?post=10049"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/tags?post=10049"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=10049"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}