{"id":1944,"date":"2019-09-11T03:17:54","date_gmt":"2019-09-11T03:17:54","guid":{"rendered":"http:\/\/kusuaks7\/?p=1549"},"modified":"2024-04-16T12:19:25","modified_gmt":"2024-04-16T12:19:25","slug":"more-data-science-methodology-options-has-much-changed-part-2","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/bigdata-cloud\/more-data-science-methodology-options-has-much-changed-part-2\/","title":{"rendered":"More Data Science methodology options \u2013 has much changed? – Part 2"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tLet\u2019s continue our focus on\u00a0Data Science<\/a>\u00a0methodologies. The reason for this focus is the need for more methodical delivery by many\u00a0Data Science teams<\/a>.\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tIn the\u00a0first post of this series<\/a>, I made the case for having a Data Science methodology and shared 3 popular options. I hope you found those useful, but I\u2019m also conscious that they are all old methodologies.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tIn that first post, I reviewed\u00a0CRISP-DM<\/a>,\u00a0KDD<\/a>\u00a0&\u00a0SEMMA<\/a>\u00a0methodologies. All of which were created during the heyday of Data Mining. Before the \u201cAI winter<\/em>\u201d when exciting things were happening, but largely through using large stats packages or bespoke applications.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

Drivers of a fresh approach to Data Science methodologies<\/h2>\n<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tMost of the\u00a0data science teams<\/a>\u00a0I know use an in-house bespoke methodology. Why have the not just used one of those I shared in my last post? As I\u2019ve chatted with different\u00a0Data Science leaders<\/a>\u00a0a few themes have emerged in their answers.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tFirstly, the way their teams work is different. Many have fully or partially transitioned to\u00a0agile working<\/a>. Although this does not preclude more rapid versions of iterative methodologies (like CRISP-DM), it changes the steps. So, a Data Science methodology using language more familiar to those using an\u00a0Agile Development<\/a>\u00a0approach is needed.\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tSecondly, the tools being used by these teams have changed. Many are less reliant on\u00a0IT<\/a>\u00a0departments than they were 20 years ago & the majority are coding in\u00a0R<\/a>\u00a0or\u00a0Python<\/a>. This causes them to break free of methods more focussed on traditional statistical analysis packages, like SEMMA.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\nFinally, the type of data they work with is much more diverse.\u00a0Data Scientists<\/a>\u00a0are often\u00a0wrangling<\/a>\u00a0both structured and\u00a0unstructured<\/a>, internal and\u00a0external data<\/a>. Some are working with more complex relationship data, for instance from social media or other digital sources.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tThe way this data is accessed has also become more diverse. Data Science teams may routinely need to draw data from a variety of database structures (including column and even graph databases). They may also need to use their wider data access to load data into\u00a0Data Lakes<\/a>\u00a0and spend longer on sourcing data than previous generations.\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tFor all\u00a0those reasons, it is not surprising to see an explosion of more varied options. It is nigh on impossible for me to be comprehensive in this post, but let me share some exemplars that I think typify different approaches to this challenge.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

If in doubt, turn to Wikipedia<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

https:\/\/en.wikipedia.org\/wiki\/Data_analysis<\/span><\/p>\nAnother generational difference is the amount of material available online. So, as every\u00a0student<\/a>\u00a0knows, if in doubt first check out the Wikipedia entry.\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tOn the topic of Data Science methodologies, this is a useful exercise. Under an entry for \u201cData Analysis<\/em>\u201c, it shares this popular simple overview of a Data Science process. As you can see it shares some similarities with the simple steps and feedback loops of KDD but has evolved.\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tTo reflect modern\u00a0Data Science<\/a>\u00a0practice, it also recognises the creation of either \u201cdata products<\/a>\u201d or \u201cvisualisation<\/a>\u201d that drives a business decision. It is encouraging to also see the Exploratory Data Analysis stage (emphasised by SEMMA method) has been emphasised.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tA little Googling reveals the source of this Wikipedia entry is\u00a0Springboard<\/a>. In this helpful blog post on KD Nuggets, they explain how it is meant to work:\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

Technology providers continue to shape Data Science methodologies<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tI hope you can see some benefits to the clean simplicity of the above method,\u00a0however\u00a0it is also quite high-level. It risks being\u00a0simplisitic\u00a0and failing to highlight all the steps that should be considered.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tIn my\u00a0last post<\/a>, I recalled how\u00a0SAS Software\u00a0<\/a>used to dominate the world of statistical analytics or Data Mining. One of the benefits of their focus was the creation of the SEMMA methodology.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tFor today\u2019s\u00a0Data Scientists<\/a>, the technology behemoths are surely Amazon, Google & Microsoft. Those who own not only useful toolkits but whole environments (or ecosystems) including the data storage and deployment. So, it\u2019s not surprising to see that once again they are taking the lead in shaping methodologies for practitioners.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tAs just one example of these useful contributions, I\u2019ve shared above a visual summary of\u00a0Microsoft\u2019s<\/a>\u00a0Team Data Science Process Lifecycle. Despite feeling like too long a name, I like the emphasis on both teamwork & a lifecycle for use of data.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tBeyond that, the diagram above shows that this methodology could be accused of being too flexible. But it does reflect the need for steps including pipelines for data, feature engineering and\u00a0intelligent<\/a>\u00a0apps. A clear evolution with regards to steps needed today.\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tIf you value the flexibility and more comprehensive nature of this method, then much more detail is available within Microsoft\u2019s Azure support documentation:\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

Management Consultants don\u2019t want to miss out on this party<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tGiven both the complexity of\u00a0Data Science<\/a>\u00a0options and the speed of change for organisations, it\u2019s not surprising that many are unsure. As ever, management consultants are on hand to step into this gap. It seems almost daily that a new report is available on how to implement Data Science or\u00a0AI<\/a>\u00a0in your business.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\nHowever, beyond just the\u00a0marketing<\/a>\u00a0opportunity, some consulting firms are taking this field seriously and partnering with academics. This can be a really helpful step forward in educating today\u2019s leaders and identifying where changes are needed.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tOne such example is the work of\u00a0Booz Allen Hamilton<\/a>. They have partnered with a number of leading Data Scientists, including\u00a0Kirk Borne<\/a>\u00a0to produce the handy \u201cData Science Field Guid<\/em>\u201c. Not only is this a useful tool for educating executives about Data Science it also includes a methodology.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tBecause of this way this is visualised, it is hard to do it justice in the graphic above. I recommend downloading the field guide and flicking through the pages to appreciate the visual prompts they provide. It suggests to me that there is further\u00a0data visualisation<\/a>\u00a0work to be done here. Could\u00a0Data Viz<\/a>\u00a0produce a purely visual Data Science methodology?\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

So, which Data Science methodologies are being used?<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tHaving shared all that about how Data Science methodologies have developed, there appears to be little research in this area. While searching for research or surveys into which methodologies are more used today, the most recent I could find was a survey updated by KD Nuggets in 2014.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tHowever, this slightly older research is still informative. They compare results with a comparable survey which they ran in 2007. Not much has changed and the surprise for me is that despite the examples shared in this post, most respondents are still using CRISP-DM.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tApart from\u00a0individuals\u2019s\u00a0or organisations own methods, KDD or SEMMA are still popular. So, perhaps the\u00a0Data Science world<\/a>\u00a0hasn\u2019t changed that much since my first post recollecting the 1990s?\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tHere are the full results. Please, do let me know if you discover a more recent source of research on Data Science methodology usage:\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

The end of our brief foray into Data Science methods?<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tI hope you have found these last two posts useful. I would be very interested in hearing which Data Science method you use.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tIf you use a\u00a0Data Science<\/a>\u00a0methodology that I have not covered and are convinced it is worth recommending to others, please share.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tI\u2019m happy to add a third in this series with recommended methods from readers.\u00a0Data Science methodologies<\/a>\u00a0is definitely still an evolving field.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"

Let\u2019s continue our focus on\u00a0Data Science\u00a0methodologies. The reason for this focus is the need for more methodical delivery by many\u00a0Data Science teams. In the\u00a0first post of this series, I made the case for having a Data Science methodology and shared 3 popular options. I hope you found those useful, but I\u2019m also conscious that they<\/p>\n","protected":false},"author":639,"featured_media":3905,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[187],"tags":[94],"ppma_author":[3364],"class_list":["post-1944","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-bigdata-cloud","tag-data-science"],"authors":[{"term_id":3364,"user_id":639,"is_guest":0,"slug":"paul-laughlin","display_name":"Paul Laughlin","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/?s=96&d=mm&r=g","user_url":"","last_name":"Laughlin","first_name":"Paul","job_title":"","description":"Paul Laughlin is Founder and Managing Director at Laughlin Consultancy Ltd<\/a> that helps companies generate sustainable value from their customer insight, His speaking focuses on topics including Customer Insight, Leadership, Data, Analytics, Data Science, and Research & Database Marketing. Follow Customer Insight blog<\/a>."}],"_links":{"self":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/1944","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/users\/639"}],"replies":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/comments?post=1944"}],"version-history":[{"count":6,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/1944\/revisions"}],"predecessor-version":[{"id":36620,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/1944\/revisions\/36620"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media\/3905"}],"wp:attachment":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media?parent=1944"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/categories?post=1944"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/tags?post=1944"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=1944"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}