{"id":2049,"date":"2019-11-05T01:55:41","date_gmt":"2019-11-05T01:55:41","guid":{"rendered":"http:\/\/kusuaks7\/?p=1654"},"modified":"2024-03-05T06:21:09","modified_gmt":"2024-03-05T06:21:09","slug":"what-is-data-science","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/bigdata-cloud\/what-is-data-science\/","title":{"rendered":"What is Data Science?"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

Myths, dreams, and reality of this beautiful job<\/h3><\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tData Science is considered as one of the most\u00a0modern\u00a0and\u00a0fascinating\u00a0jobs of our time. It can be funny and can give you\u00a0satisfaction, but is it really as it\u2019s described?\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tIn this article, I\u2019ll show you the\u00a0reality\u00a0of a Data Scientist\u2019s life.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

What you think it is<\/h1><\/h1>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tAt the beginning of their career, Data Scientists think that Data Science is a wonderful,\u00a0magical world\u00a0full of algorithms,\u00a0Python\u00a0functions that performs every possible spell with\u00a0a line of code\u00a0and statistical models able to detect the most useful\u00a0correlations\u00a0among data that could make you an invincible\u00a0superhero\u00a0in your company. You start dreaming about your CEO congratulating with you and shaking your hand, you begin to see\u00a0decision trees\u00a0and\u00a0clusters\u00a0everywhere and, of course, the most terrifying\u00a0neural network\u00a0architectures your mind can dream.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tBut since the very first day of your first Data Science project, you\u00a0start to realize\u00a0what reality is.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

What it really is<\/h1><\/h1>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

Expectation for results<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tManagers often think that Data Science is the\u00a0Holy Grail\u00a0of information technology. They have\u00a0huge expectations\u00a0about it and they want them to be satisfied\u00a0here and now.\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tIn reality, results are very\u00a0difficult\u00a0to achieve and need much time. Sometimes a result\u00a0can\u2019t be reached. Think about\u00a0clustering, for example. You can spend an entire life searching for a clustering pattern that simply doesn\u2019t exist in your data. Most managers don\u2019t understand this fact and it can be\u00a0very stressful\u00a0for you and your team.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

Explaining<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tThe only thing better than a good algorithm is an\u00a0explainable\u00a0algorithm. Never forget this. No sane manager in the world would follow an unknown algorithm for managing their company\u2019s money only because its AUROC is greater than 95%. Managers\u00a0need to understand\u00a0algorithms, figure out how they\u00a0think\u00a0about data and this is often a great task for a Data Scientist. Explaining algorithms to somebody with no scientific background can be quite difficult, but it\u2019s\u00a0very common\u00a0in large companies and you must face this fact.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tMost of your time you\u2019ll find yourself trying to erase that awful question mark on your boss\u2019 face, simplifying as much as possible to\u00a0make them understand\u00a0your results. Remember: if you can\u2019t explain your results, managers will start to ask themselves whether you are useful or not in your company.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

Business understanding<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tYou\u2019ll spend a lot of time interviewing\u00a0product owners\u00a0and\u00a0ITC professionals\u00a0to understand the information hidden inside business data they know or produce. There\u2019s\u00a0no way\u00a0you can make it without their help.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Many times data comes from complex and\u00a0heterogeneous systems\u00a0<\/strong>and this often implies lines of log files that you need to understand. Data isn\u2019t everything;\u00a0information is everything<\/strong>. Never forget this. Information is buried inside data and you\u2019ll need somebody telling you where you should dig.<\/p>\n

The larger the company, the\u00a0more difficult\u00a0<\/strong>it is to find the right people to interview and when you finally make it, their answers will generate\u00a0more questions\u00a0<\/strong>and these people may not have enough time for you and your \u201cnerdy stuff\u201d.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

Data visualization<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

You\u2019ll find yourself using data visualization more often than you would have ever imagined.\u00a0Charts<\/strong>, slides and other graphical tools will be like silver bullets in your shotgun. Maybe you have magic formulas in your mind, graphs and so on. Forget about them. Data Science is told by\u00a0graphical representations\u00a0<\/strong>and it\u2019s often difficult to find the proper visualization technique suitable for your audience.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

Deadlines<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

There they are. We are slaves in a world of deadlines and expectations. When you were a software engineer you had\u00a0milestones\u00a0<\/strong>in your plan and you weren\u2019t allowed to delay a second. In Data Science, things aren\u2019t easier.<\/p>\n

There are deadlines and milestones even in Data Science, and there is a\u00a0great difficulty\u00a0<\/strong>inside them: Data Science is something very close to\u00a0academic research<\/strong>, so it doesn\u2019t fit well in the classical,\u00a0waterfall\u00a0<\/strong>ITC project management style. Instead, some\u00a0Agile framework\u00a0<\/strong>(e.g. Scrum or Kanban) should work well, due to its physiological ability to quickly\u00a0adapt to changes<\/strong>. But Agile is difficult to teach to managers. It can give them the false idea that there\u2019s\u00a0no clear delivery date\u00a0<\/strong>and this is very difficult to accept by companies.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

Algorithms and programming<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

And finally, the fun part. Python, R, Knime, reading scientific papers, optimization algorithms, cross-validation and so on. The technical and\u00a0nerdy real fun\u00a0<\/strong>is a very small part of the work and it takes very little time in the whole project lifetime. Maybe you have already\u00a0lost enthusiasm\u00a0<\/strong>in the previous phases before writing your first line of code and things no longer seem as funny as you thought at the beginning.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

\n

What\u2019s the best way to do Data Science?<\/h1><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

According to my experience, I can answer with a single word:\u00a0Agile<\/strong>. There\u2019s no need to perform all the business understanding part before writing your first Python code line. Start with a\u00a0simple business understanding\u00a0<\/strong>of a small piece of data, explore it,\u00a0visualize it\u00a0<\/strong>and begin with a\u00a0simple model<\/strong>. Create the first, quantifiable results\u00a0week by week\u00a0<\/strong>keeping your\u00a0customers constantly engaged\u00a0<\/strong>in the process. Deliver\u00a0small results\u00a0<\/strong>with a\u00a0constant delivery rate\u00a0<\/strong>and, please, don\u2019t fall into the waterfall trap.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Simplicity is the key<\/strong>. Never forget it. Start with the simplest things possible and add a small piece of complexity only if needed.<\/p>\n

There\u2019s a psychological sense of\u00a0relief\u00a0<\/strong>in constant, small results and this is another weapon you have to use if you want to survive in the jungle of companies\u2019 deadlines and business processes. In this way, every colleague of yours who is committed to your project will\u00a0feel your difficulties\u00a0<\/strong>and start to understand how difficult Data Science is.<\/p>\n

Remember, companies still think about Data Science as an\u00a0ITC branch<\/strong>; they are not completely wrong, but they shouldn\u2019t expect you to follow the waterfall approach. So, you have to suffer the struggle to guide your company toward an\u00a0Agile way of thinking<\/strong>.<\/p>\n

Concerning the\u00a0explanation\u00a0<\/strong>part of the job, I prefer to start with the\u00a0simplest machine learning model\u00a0<\/strong>possible:\u00a0k-nearest neighbors<\/strong>. It\u2019s very easy to understand. You only need paper, a pencil and a Cartesian plane with some points drawn on it. That\u2019s it. If it produces very nice results, everybody will finally see you like the great business partner you think you are.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

If KNN doesn\u2019t work, then you can use\u00a0regressions\u00a0<\/strong>and\u00a0decision trees\u00a0<\/strong>(random forests, gradient boosted tree classifiers and so on), which are very easy to explain, or\u00a0Bayesian networks<\/strong>, which have a very useful graphical representation.<\/p>\n

Finally, visualize.\u00a0Visualize everything<\/strong>. Ask your boss to buy you a\u00a0course in data visualization<\/strong>, learn as much as possible about the best visualization techniques and, please, remember to\u00a0avoid pie charts<\/strong>. They are pretty useless and misleading. If you provide a simple\u00a0scatter\u00a0<\/strong>or\u00a0bar plot<\/strong>, people will catch all the relevant information.<\/p>\n

Simple results are the best ones. Some days ago, my team and I presented some results about a time series analysis using only\u00a0three slides<\/strong>: high-level\u00a0KPIs<\/strong>\u00a0describing the business phenomenon, a\u00a0confusion matrix<\/strong>\u00a0and some\u00a0performance metrics<\/strong>. Our audience was enthusiastic since the first slide, only because we started with clear numbers explaining the business in a simple way. In many situations, a\u00a0small building block<\/strong>\u00a0can really save your life.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

Conclusions<\/h1><\/h1>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Data Science is an exciting job, but it can be very difficult to perform if you speak to a non-technical audience. Data and business are intimately related to each other and you must remember this point when you work with business-oriented people. The only way to survive is to\u00a0find a middle point\u00a0<\/strong>between a\u00a0data-driven<\/strong>\u00a0bottom-up approach and a\u00a0business-driven<\/strong>\u00a0top-down approach.<\/p>\n

Finally, as Data Science is hard and\u00a0time-consuming<\/strong>, delivering small results with a constant delivery rate is the only way you can keep your customers engaged.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"

Data Science is an exciting job, but it can be very difficult to perform if you speak to a non-technical audience. Data and business are intimately related to each other and you must remember this point when you work with business-oriented people. The only way to survive is to find a middle point between a data-driven bottom-up approach and a business-driven top-down approach.  Finally, as Data Science is hard and time-consuming, delivering small results with a constant delivery rate is the only way you can keep your customers engaged.<\/p>\n","protected":false},"author":618,"featured_media":2634,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[187],"tags":[94],"ppma_author":[3328],"class_list":["post-2049","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-bigdata-cloud","tag-data-science"],"authors":[{"term_id":3328,"user_id":618,"is_guest":0,"slug":"gianluca-malato","display_name":"Gianluca Malato","avatar_url":"https:\/\/www.experfy.com\/blog\/wp-content\/uploads\/2020\/04\/medium_918623b2-8f36-4110-8343-6fc9228595dd-150x150.jpg","user_url":"http:\/\/www.gianlucamalato.it\/","last_name":"Malato","first_name":"Gianluca","job_title":"","description":"Gianluca Malato is Data Scientist at Poste Italiane SPA.\u00a0 He is also a fiction author and software developer, Editor of\u00a0Data Science Journal<\/a>,\u00a0The Trading Scientist<\/a>, and\u00a0The Writer\u2019s Notebook<\/a>. His books are available on Amazon<\/a>."}],"_links":{"self":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/2049","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/users\/618"}],"replies":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/comments?post=2049"}],"version-history":[{"count":5,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/2049\/revisions"}],"predecessor-version":[{"id":36230,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/2049\/revisions\/36230"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media\/2634"}],"wp:attachment":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media?parent=2049"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/categories?post=2049"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/tags?post=2049"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=2049"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}