{"id":22965,"date":"2021-04-19T18:57:11","date_gmt":"2021-04-19T18:57:11","guid":{"rendered":"https:\/\/www.experfy.com\/blog\/creating-a-repeatable-data-library-process\/"},"modified":"2023-08-26T06:08:00","modified_gmt":"2023-08-26T06:08:00","slug":"creating-a-repeatable-data-library-process","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/bigdata-cloud\/creating-a-repeatable-data-library-process\/","title":{"rendered":"Creating A Repeatable Data Library Process"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

In this final article in a series on how small analytics teams can build a self-managed data library for effective data management, I will summarize the previous articles and show how to put it all together into a repeatable process.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

A Data Library is Built on a Set of Principles for Data Management, not a Technology Stack<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

A data library does not require a specific technology or skill set. Rather, it is built on principles of good data management that are geared toward helping you to work more effectively.<\/a> These principles are theoretical, but also actionable. The process you build should include whatever steps are necessary to implement your team\u2019s data library principles.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

A Data Library has an Informal Data Architecture, with \u201cPonds\u201d and \u201cReservoirs\u201d<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

It is easy to get caught up in how to name things. Naming conventions, including what I suggest, are less important than a purposeful architecture that supports collecting data in the ponds and preparing that data for analysis and reporting in the reservoirs<\/a>. Your process should include, at a minimum, the work that is required to build automated data ponds, with the construction of the analytics and reporting reservoirs done on an as-needed basis.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

A Data Library Balances Speed, Agility, and Cost, and is Built According to Priority<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Creating a process with many steps can slow things down, but it ensures you do not cut any corners. The data library concept <\/a>should help your team do things quickly in the future, but it takes time to build up the library infrastructure at the beginning. Prioritization is important so that if you are able to catalogue only one data source a quarter, you know you are working on the most important sources first<\/a>.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

Building a Repeatable Process with a Checklist<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

To operationalize your data library<\/a>, you will need to execute many small steps for each data source. Do your best to think of as many possible steps before you get started and then refine that list as you go. At TechSmith, our current list includes 39 steps<\/a>. In a given project, some of these are unnecessary and others will be added. Starting with this list ensures that we will think about all of the important aspects and address each data library principle.<\/p>\n

TechSmith\u2019s checklist includes some final items that I have not mentioned previously. When you finish cataloguing a data source, you should celebrate! It is an accomplishment that will make your team more effective and enable you to better serve your stakeholders. Tell those stakeholders about it, give them examples of what can be done that was not possible before, and allow them to ideate. This process, along with a quarterly check-in with these stakeholders, leads to exciting new ideas.<\/p>\n

Take the time to share knowledge with your team and others as well as doing project clean-up. This may include things such as committing all changes to a GitHub repo or standardizing a folder structure. If you are creating or contributing to open-source development, you should make that available to that community and publicize that it is available.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\"Creating<\/a><\/figure>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"

In this final article in a series on how small analytics teams can build a self-managed data library for effective data management, I will summarize the previous articles and show how to put it all together into a repeatable process. A Data Library is Built on a Set of Principles for Data Management, not a<\/p>\n","protected":false},"author":1135,"featured_media":22967,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[187],"tags":[687,985,977],"ppma_author":[3185],"class_list":["post-22965","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-bigdata-cloud","tag-data-architecture","tag-data-library","tag-data-management"],"authors":[{"term_id":3185,"user_id":1135,"is_guest":0,"slug":"chris-umphlett","display_name":"Chris Umphlett","avatar_url":"https:\/\/www.experfy.com\/blog\/wp-content\/uploads\/2021\/05\/Chris-Umphlett-150x150.jpg","user_url":"","last_name":"Umphlett","first_name":"Chris","job_title":"","description":"Chris Umphlett is the Manager of Data Analysis and Data Privacy at TechSmith, the makers of great software like Snagit and Camtasia. Before that he worked on analytics teams in the consumer packaged goods, life insurance, and utility industries. He lives in East Lansing, Michigan with his wife and young children."}],"_links":{"self":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/22965","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/users\/1135"}],"replies":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/comments?post=22965"}],"version-history":[{"count":8,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/22965\/revisions"}],"predecessor-version":[{"id":31535,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/22965\/revisions\/31535"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media\/22967"}],"wp:attachment":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media?parent=22965"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/categories?post=22965"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/tags?post=22965"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=22965"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}