{"id":10737,"date":"2020-10-27T10:20:30","date_gmt":"2020-10-27T10:20:30","guid":{"rendered":"https:\/\/www.experfy.com\/blog\/?p=10737"},"modified":"2023-10-16T17:23:18","modified_gmt":"2023-10-16T17:23:18","slug":"data-management-remains-challenge-data-ai-first-era","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/bigdata-cloud\/data-management-remains-challenge-data-ai-first-era\/","title":{"rendered":"Why Data Management Remains A Challenge In The Data And AI-First Era"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Data is the fuel of the modern organisation. As it\u2019s proliferated across the enterprise, more people are integrating it into their business and operational decisions. This means that having a robust data management strategy and infrastructure is critical for the success of every data-driven business.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Nevertheless, data management remains a fundamental challenge to solve even as we are moving towards data-first and AI-driven organisations. Companies can\u2019t progress towards data innovation and AI deployment if they haven\u2019t taken care of the fundamentals of how they are going to manage their data. In this regard, below we are going to explore what are the most pervasive challenges and how organisations are tackling them.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

Outstanding challenges<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

With some of the persistent hurdles like legacy systems and lack of domain-specific capabilities, organisations are impeded from deploying and scaling their AI initiatives. Working with legacy data and systems is especially a problem in enterprise organisations where data is stored in disparate siloed systems, it\u2019s hard to find and aggregate in a universal data platform in order to accelerate data-driven decisions.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

When it comes to domain specific-capabilities, BARC survey<\/a> reports that companies seriously feel the lack of external knowledge and the skill gap present on the market. Sourcing the rights skills is a general challenge, as companies data management needs cannot be fully met by engaging external resources.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

What\u2019s more, data management can\u2019t be done right without having clearly defined data governance policies and rules that govern that use of data and data operations, something a majority of organisations are still fumbling with. Here, we cannot omit to mention the first and foremost enemy of a data-driven company, which is poor, inconsistent and poor data quality. As the amount of data organisations collect has increased by a great degree, ensuring data quality has become harder because of the diversity of data sources, the various types of data that are difficult to integrate, the sheer volume of data, as well as the rapid pace at which data changes.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

All these challenges, combined with the fact that data and advanced analytics is becoming more vital to support complex business processes, put increased pressure on IT departments to deliver faster, continuous, reliable, secured and higher quality data to support the analytics needs of different functions.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

All the above points to an urgent need for a revamp and overhaul of companies\u2019 data management strategies, platforms and infrastructure, for them to thrive and survive in the data and AI-first era. We can\u2019t talk about deploying AI into production if we don\u2019t have the right foundation in the form of a modern data strategy and platform to support it.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

How to start the process towards successful data management?<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Data management is a complex process that encompasses various topics related to how we collect, manage, store and analyse data such as data security, data quality, MDM, data warehousing, data integration, database management. And each field requires particular attention by the respective stakeholders in the company.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

But here we are going to outline several best practices of how to start creating a good data management plan<\/a>, particularly when considering data migration to a hybrid or cloud environment.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

First and foremost, experts suggest that it\u2019s crucial to establish the goal of the whole initiative. Just like any other transformation project, integrating data management into the processes should be aligned with the data strategy endgame. Part of this step is defining your specific needs which should be clearly communicated to your internal IT department or IT consulting company. Your data management plan should include the types of data you process, your storage need, any restrictions that need to be in place and security measures that are required.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Once your goals and specifications are defined, it\u2019s time to perform an audit of your entire data, which is easily done if it\u2019s broken down by type and source. For a clearer visualisation, it\u2019s recommended that you create a flowchart presenting the paths of data in and out of your business.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Before making a decision on a data migration strategy and solution, you should take the time to define your data and database requirements. The next step is to define how your data is going to be documented and what documentation is needed. Defining metadata\u2014meaning what, where, when, why, and how the data was collected, processed, and interpreted\u2014 enables data and files to be discovered, used, and properly cited.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Your next move should be to establish the process of assessing the quality of the data and amending it in case of errors. Creating data storage and preservation strategy is a critical step in the process because it should take into account everything from cost to security to compliance.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

When defining your data policies, which is the next step, you should make sure they cover all previous steps. Setting out any licensing and sharing agreement, as well as media embargo policies and legal or ethical restrictions, should also be done at this stage.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Your plan should also articulate the roles and responsibilities of all data-related tasks of all employees involved in the data collection, data entry, quality control, metadata creation, backups, data preparation and archive submission.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

And lastly, don\u2019t forget to set up your budget for your migration and ongoing data management activities.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

Data management case studies<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

The above was a short guideline about the most important things to consider when starting with a data management initiative. In order to see how it is done in practice, let\u2019s explore some of the innovative companies that have something to show when it comes to successful data management. We are going to look at two speaker presentations from the Data 2020 Summit that have showcased successful data management journeys and approaches.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Alex Astrogold<\/strong>, H&M Group\u2019s former Data Management Architect, presented the data management journey of the H&M Group while transforming their business with dynamic information models and solutions. Alex gave a step-by-step recount of how their data strategy and vision changed, the new systems they implemented which introduced master data management and transparent data quality principles, and the development of their long-run data governance. H&M Group\u2019s data management initiative resulted in the creation of a bespoke data strategy office that would assist with it and support all data initiatives. Read a detailed summary of H&M\u2019s data management journey here<\/a>.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Another success case study came from\u00a0 Kate Tickner<\/strong>, Senior Director, EMEA Alliances Reltio, during her session at last year\u2019s\u00a0 Data 2020 Summit <\/a>. Kate\u2019s talk focused on <\/a>a modern data management approach\u00a0with a focus on driving increased success in the digital customer experience economy.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

As Kate pointed out, today\u2019s modern customers expect their experience with a brand to be convenient, knowledgeable, efficient and friendly. And this is very difficult to meet if companies don\u2019t have information on their consumers available at their employees\u2019 fingertips. This implies that all the steps we talked about in the previous part should be sorted out for your customer data in order to be able to complete in the customer experience era. However, legacy systems <\/em> that were designed to deliver customer service cannot deliver timely and actionable customer insights \u2014 companies need modern technology to support automated <\/a> actions for their customers. This leads us again to the point of reinventing companies data management strategies, processes and infrastructures to enable their data-driven transformations.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

Set up your data management for the Data and AI-first decade<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

To address all the challenges and needs of creating a data management strategy and platform that is aligned with the enterprise vision and enables an agile, collaborative, governed, future enabled, ethical and customer-driven approach, we have prepared a programme that addresses the most critical pillars when designing your future enabled Data Management strategy and technology stack:\u00a0 Data Governance, Data Quality, Cloud or Multi-Cloud enabled infrastructure and DataOps<\/em>.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

The Data 2030 Summit <\/strong> is a roundtable event that gathers the Data Management community in one platform to discuss ways of enabling faster Data Innovation and AI deployment across the enterprise by setting up a modern Data Management strategy and platform for the new decade.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

This article was originally published at Hyperight Read<\/a>.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"

Companies can\u2019t progress towards data innovation and AI deployment if they haven\u2019t taken care of the fundamentals of how they are going to manage their data. Explore the most pervasive challenges and how organisations are tackling them.<\/p>\n","protected":false},"author":950,"featured_media":10738,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[187],"tags":[857,532,858,592],"ppma_author":[3720],"class_list":["post-10737","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-bigdata-cloud","tag-ai-deployment","tag-challenges","tag-data-driven-business","tag-master-data-management"],"authors":[{"term_id":3720,"user_id":950,"is_guest":0,"slug":"ivana-kotorchevikj","display_name":"Ivana Kotorchevikj","avatar_url":"https:\/\/www.experfy.com\/blog\/wp-content\/uploads\/2020\/10\/Ivana-Kotorchevikj-150x150.jpg","user_url":"http:\/\/www.hyperight.com","last_name":"Kotorchevikj","first_name":"Ivana","job_title":"","description":"Ivana Kotorchevikj is Chief Editor, Hyperight Digital at Hyperight AB, an international event service provider focusing on creating network oriented and crowdsourced business events."}],"_links":{"self":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/10737","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/users\/950"}],"replies":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/comments?post=10737"}],"version-history":[{"count":9,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/10737\/revisions"}],"predecessor-version":[{"id":33479,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/10737\/revisions\/33479"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media\/10738"}],"wp:attachment":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media?parent=10737"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/categories?post=10737"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/tags?post=10737"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=10737"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}