{"id":22472,"date":"2020-11-27T11:26:31","date_gmt":"2020-11-27T11:26:31","guid":{"rendered":"https:\/\/www.experfy.com\/blog\/citizen-data-scientist-digital-transformation-debts-post-covid19\/"},"modified":"2023-10-03T09:23:33","modified_gmt":"2023-10-03T09:23:33","slug":"citizen-data-scientist-digital-transformation-debts-post-covid19","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/future-of-work\/citizen-data-scientist-digital-transformation-debts-post-covid19\/","title":{"rendered":"5. Citizen Data Scientist: Digital Transformation Debts post-Covid-19"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

Citizen Data Scientist: Digital Transformation Debts post-Covid-19<\/h1><\/h4>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

This article is the fifth part of a ten-part series on Digital Transformation Debt, post-Covid-19.\u00a0\u00a0<\/a>Part 1<\/a>focused on Culture.\u00a0 Part 2 <\/a>\u00a0delved deeper into Operational Excellence and inter-enterprise Value Streams As A Service.\u00a0Part 3<\/a>\u00a0explained the spectrum of Automation and the shifts post-Covid-19..\u00a0Part 4<\/a> demonstrated how a new harvest of Low Code\/No Code platforms is empowering Citizen Developers.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Due to Covid-19, organizations need to be agile and responsive. They need to understand trends and predict actions leveraging enterprise, sensor, customer, and partner Data<\/em>. They also need to be in motion and autonomic <\/a>. This article focuses on another critical dimension for alleviating Digital Transformation Debt: the emergence of the Citizen <\/em> Data Scientist. Mining patterns from increasingly exploding data lakes and then acting upon those in real-time is critical for survival post-Covid-19.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

Data-Centric Organization<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

By any estimate, the digital era is facing an unprecedented explosion of information. Digital technologies, solutions, and content generate\u00a0 2.5 quintillion bytes of data each day <\/a>! However, like the IT application development bottleneck, a more severe\u00a0 data scientist challenge <\/em> is the shortage of Data Scientist. Organizations are hoarding data – but often mining and benefiting from the heterogeneous data lakes is a challenge. A new harvest of productivity, self-service, drag-and-drop data tools is\u00a0emerging and allowing\u00a0 citizens \/em>\u00a0to discover and deploy analytical models \u2013 predictive, machine learning, or even deep learning. Nothing short of\u00a0Artificial Intelligence platforms for the masses. We are witnessing the emergence of easy to use Citizen\u00a0AI tools for customer engagement<\/a>, with proven results.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

In the Covid-19, Data is becoming even more critical. The application of the models mined from the Covid-19 infection databases is obvious. Equally important are the supply chain, societal interaction, and overall economic trends amid shifts and transformation. The Covid-19 era is also accelerating the “Process +Data<\/a>” narrative, where organizations need to complement and balance data-centricity combined with digitization and Automation of value streams. Bottom line \u2013 pre or post-Covid-19 – it is not just about the data. The insights <\/em>need to be mined, discovered, or harvested from the vast, often messy lakes of data. Raw data to insights should be the mantra. Once insights are discovered, they need to be acted <\/em>upon.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

Database Management Systems<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

DBMSs that separated the management of the data from the application started to appear in the 1970s with navigational\u00a0hierarchical <\/em> and\u00a0 network <\/em> models. In the 1980s, we saw a significant evolution to\u00a0 relational databases <\/a>\u00a0that became quite popular, especially with SQL’s emergence as the de-facto query language for databases! The evolution of databases from relational included\u00a0 Object-Oriented Databases <\/a>\u00a0that combined Object-Oriented and Database capabilities for persistent storage of objects and\u00a0 Object-Relational Databases<\/a>\u00a0that combine the characteristics of both relational and object-oriented databases.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

More recently \u2013 especially for handling large\u00a0 unstructured <\/em> multi-media data in new digital applications – we saw the emergence of\u00a0 NoSQL<\/a>\u00a0to handle the demands of\u00a0 Big Data <\/a>: with large volume, variety, velocity, and veracity. This new generation of database focuses on dealing with the explosion of heterogeneous data and the storage and management of this Data for innovative Internet applications (especially IoT). Still, by and large, most\u00a0transactional\u00a0data for mission-critical systems of record (which require transactional integrity) remains relational. All these trends are culminating in\u00a0intelligent\u00a0<\/em> DBMSs<\/a>.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

Data Lakes<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Recently we have also seen the emergence of “Data Lakes.” Here is how\u00a0 AWS<\/a>\u00a0explains “Data Lakes:”<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Faced with massive volumes and heterogeneous types of data, organizations are \ufb01nding that in order to deliver insights in a timely manner, they need a data storage and analytics solution that offers more agility and \ufb02exibility than traditional data management systems… Data Lake allows an organization to store all their data, structured and unstructured, in one, centralized repository.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

The following illustrates the key components and capabilities of a Data Lake.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

The emergence of many heterogeneous data sources is at the core of the Data Lake. According to\u00a0 Aberdeen<\/a> there is a clear distinction in business execution between Data Lake leaders and followers (aka lagers). Strategic Data Lake investments and maturity characterize the leaders.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

The Data Scientist<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

The sections above illustrate the complexity of\u00a0 Data <\/em>\u00a0in enterprises \u2013 too many databases, repositories, sources, and strategies. The Data Scientist role is a relatively new one. Many assumptions that we had taken for granted in the management of databases, including integrity or logic pertaining to the independence of the data from the application, are now being challenged. The past couple of decades have created powerful gatekeepers of the enterprise data (the\u00a0 Database Administrators (DBA) <\/em> <\/a>) who sometimes block agility and the speed of change needed to sustain business requirements. The world \u2013 or I should say the\u00a0 digital <\/em>\u00a0world \u2013 is changing. The introduction of\u00a0 NoSQL databases<\/a>, especially for Big Data<\/a>, has introduced additional complexity for managing and maintaining heterogeneous DBMSs consistency. This transformational change emanates from the need to engage customers directly. It also results from the explosion of information on the Internet, especially with the\u00a0 Internet of Things.<\/a>\u00a0But more importantly, the mining of business value through analysis and machine learning techniques has given rise to this new \u2013 and sometimes DBA evolved \u2013 role in the enterprise, namely the “Data Scientist.”<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Data Science is complicated. Data Science is multi-disciplinary. Here is a definition of the role of a Data Scientist from a business perspective<\/a>:<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

A data scientist identifies important questions, collects relevant data from various sources, stores and organizes data, decipher useful information, and finally translates it into business solutions and communicate the findings to affect the business positively.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Data Science involves many disciplines. Data Scientists need to have many skills \u2013 from mathematics, statistics, machine learning, to programming, and more. Perhaps more importantly, Data Scientists need to communicate and present their findings in clear terms that the business understands. They also need to be subject matter experts and creative\u2014one role for all this spectrum. No wonder Data Scientists are in great demand! Here is a great illustration <\/a> of Data Science:<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

The Data Scientist’s continuous activities are three fundamental areas: Data Analysis, Programming, and Business Analysis for concrete business results. Unfortunately, poor data quality complicates the Data Scientist’s tasks and objectives. About 70% of their effort is to ingest, prepare, and cleanse the data.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

In my interactions with Data Scientists, they sometimes object to this estimate. It is more. In other words, only 10% – 30% of their time is the discovery of meaningful insights and business value from the often unruly and heterogeneous data sets!<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

The Citizen Data Scientist<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

As indicated above, Data Science involves many disciplines. According to Gartner <\/a>, ” citizen data scientist as a person who creates or generates models that use advanced diagnostic analytics or predictive and prescriptive capabilities, but whose primary job function is outside the field of statistics and analytics .” The 2017 article <\/a> predicts 40% of Data Science tasks will be automated by 2020! Well, we are in 2020 and not even close to that level of Automation. Still, Data Scientists spend 70% or higher their time cleaning and preparing the analysis and discovery data.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Despite many technological advances, methodologies, and techniques, most organizations still suffer from Business-Technical Developers-Operations silos. The trend towards empowered Citizens who can achieve Data Science objectives is not hype. It is also not a panacea. It does have challenges.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

The good news is that some emerging tools and platforms are addressing the requirements of Data Scientists. Intelligence and Automation in all the milestones and phases of the Data Science workflow make it real for Citizen Data Scientists<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Here are some productivity, intelligence, and automation technologies that are targeting the inevitable trends towards a Citizen Data Scientist platform:<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Automation of Data Preparation<\/em>: This is the most crucial category, like cleaning and preparing the data constitutes more than 70% of the Data Scientists’ effort. We are starting to see some tools addressing these needs. Tableau Prep <\/a>, for example, “\u2026 changes the way traditional data prep is performed in an organization. By providing a visual and direct way to combine, shape, and clean data, Tableau Prep makes it easier for analysts and business users to start their analysis, faster.”<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Low Code\/No Code Data Integration<\/em>: Several emerging and robust tools automate data integration and aggregation from different sources. Most structured and unstructured databases have Application Programming Interfaces (APIs). These productivity and automation tools provide easy to use drag and drop capabilities for Data integration. Parabola<\/a> is an example of a Low Code\/No Code platform for automating integration.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Automating Machine Learning (AutoML): Automation in data integration and preparation is a pre-requisite for analysis and machine learning.\u00a0Machine Learning leverages Artificial Intelligence (AI) algorithms to discover patterns in the data. It is critical in the overall Data Science process. Now, when we shift to Citizen\u00a0<\/em> Data Scientists, it becomes critical to automate Machine Learning.\u00a0 Here is one definition of AutoML <\/a>\u00a0\u2013 which is a bit extreme but drives home the objective of AutoML: “Automated machine learning, or AutoML, aims to reduce or eliminate the need for skilled data scientists to build machine learning and deep learning models. Instead, an AutoML system allows you to provide the labeled training data as input and receive an optimized model as output.” Several vendors are positioning their advanced AI automation tools as AutoML \u2013 this includes\u00a0 Google’s Cloud AutoML <\/a>\u00a0and\u00a0 IBM Watson’s AutoAI <\/a>. <\/p>\n