{"id":22944,"date":"2021-04-19T20:37:20","date_gmt":"2021-04-19T20:37:20","guid":{"rendered":"https:\/\/www.experfy.com\/blog\/prioritizing-data-sources-for-your-data-library\/"},"modified":"2023-08-26T06:04:29","modified_gmt":"2023-08-26T06:04:29","slug":"prioritizing-data-sources-for-your-data-library","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/bigdata-cloud\/prioritizing-data-sources-for-your-data-library\/","title":{"rendered":"Prioritizing Data Sources For Your Data Library"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

In the previous article,<\/a> I prescribe prioritizing data sources inclusion in a data library according to business value, difficulty, and privacy concerns. This can be done utilizing a scoring rubric and interviewing the owners and\/or key stakeholders of each data source. While these things may not be measurable they can be quantified in a relative sense. For example, do you expect that a data source will be more, less, or equally complex to automate as the average data source for your team?<\/p>\n

The scoring rubric we use at TechSmith is shown below. It is designed so that each data source will have a 0 to 100 score along each dimension. While it is theoretically possible to end up with a score outside of that range, in practice we will trim outliers to fit within that range (which we have not needed to do).<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\"Prioritizing<\/figure>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

The items or scoring ranges on your list should be customized to your team and organization but this one may serve as a good starting point. After a half hour meeting with one to three key stakeholders we would discuss the list and come up with a score. For privacy concerns we did not have a rubric. Rather we identified data privacy requirements and then gave a score of low, medium, or high. Once the data source is scored we add it to our PICK chart visualization.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

Creating a PICK Chart in R After Scoring a Data Source<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

According to Wikipedia<\/a>, \u201cPICK charts are a method to prioritize a number of action items or problem solving ideas.\u201d The PICK acronym refers to the recommended action for the four quadrants created on the chart: Possible, Implement, Challenge, or Kill.<\/p>\n

I created my version of the PICK chart with the R package ggplot2<\/em><\/a>, which is part of the tidyverse<\/em> <\/a>group of packages. I also use a package called shadowtext<\/em><\/a> <\/em>to get a certain format on the chart, but this isn\u2019t absolutely necessary. The R script starts then by including these packages:<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
library(tidyverse)<\/pre>\n
library(shadowtext)<\/pre>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
Then I set parameters that are used to make the grid on the chart as well as choosing the background colors. You can change these if you want different colors or shades of these colors:<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
# Don\u2019t change these - help make everything line up well<\/pre>\n
axis_min <- -4<\/pre>\n
axis_max <- 104<\/pre>\n
score_range <- axis_max - axis_min - 8<\/pre>\n
axis_mid <- (axis_max - axis_min) \/ 2 + axis_min<\/pre>\n
# You can change these colors and sizes<\/pre>\n
line_color <- \"grey25\"<\/pre>\n
yellow <- \"#f5f378\"<\/pre>\n
red <- \"#e6706e\"<\/pre>\n
orange <- \"#f2bb63\"<\/pre>\n
green <- \"#60b570\"<\/pre>\n
Q_size <- 8<\/pre>\n
proj_size <- 4<\/pre>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
The plotting is done on a table with four columns. I imported this from my excel spreadsheet shared above. You either should create a table like this or edit the code that makes the chart:<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\"Prioritizing<\/figure>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
The value and difficulty columns are numeric, and the privacy column is an \u201cOrdered factor.\u201d I explicitly set it to know that there is a progression from Low -> Mid -> High.<\/p>\n
The first part of the ggplot object will create the plot, select the data, set values for y and x axes, and create the colored grid:<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
p <- ggplot(plot_df, aes(x = difficulty, y = value)) +<\/pre>\n
\u00a0\u00a0geom_rect(aes(xmin = axis_min, xmax = axis_mid, ymin = axis_min, ymax = axis_mid),              <\/pre>\n
           fill = yellow, color = line_color) + # lower-left<\/pre>\n
\u00a0\u00a0geom_rect(aes(xmin = axis_mid, xmax = axis_max, ymin = axis_min, ymax = axis_mid),\n<\/pre>\n
fill = red, color = line_color) + # lower-right<\/pre>
  geom_rect(aes(xmin = axis_min, xmax = axis_mid, ymin = axis_mid, ymax = axis_max),<\/pre>\n
            fill = green, color = line_color) + # upper-left<\/pre>\n
  geom_rect(aes(xmin = axis_mid, xmax = axis_max, ymin = axis_mid, ymax = axis_max),<\/pre>\n
            fill = orange, color = line_color) # upper-right<\/pre>\n
p<\/pre>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\"Plot<\/figure>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
The next section adds the Q1-Q4 labels to each section of the chart and sets it to a clean \u201ctheme_classic\u201d:<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
p <- p +   <\/pre>\n
  geom_shadowtext(aes(x = axis_mid \/ 2, y = axis_mid \/ 2),<\/pre>\n
                  label = \"Q2\", color = \"white\", size = Q_size, bg.color = \"grey40\") + # lower-left<\/pre>\n
  geom_shadowtext(aes(x = axis_mid \/ 2 + axis_mid, y = axis_mid \/ 2),<\/pre>\n
                  label = \"Q4\", color = \"white\", size = Q_size, bg.color = \"grey40\") + # lower-right<\/pre>\n
  geom_shadowtext(aes(x = axis_mid \/ 2, y = axis_mid \/ 2 + axis_mid),<\/pre>\n
                  label = \"Q1\", color = \"white\", size = Q_size, bg.color = \"grey40\") + # upper-left<\/pre>\n
  geom_shadowtext(aes(x = axis_mid \/ 2 + axis_mid, y = axis_mid \/ 2 + axis_mid),<\/pre>\n
                  label = \"Q3\", color = \"white\", size = Q_size, bg.color = \"grey40\") + # upper-right<\/pre>\n
  theme_classic() <\/pre>\n
p<\/pre>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\"Prioritizing<\/figure>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
Now it is time to plot the systems on the chart. I have included several systems here as examples. Our real chart is full, with applications in every quadrant:<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
p <- p +<\/pre>\n
  geom_label(aes(label = data_source, fill = privacy), size = proj_size, alpha = 0.7, color = \"white\", nudge_y = 2) +<\/pre>\n
  scale_fill_grey(start = 0.05, end = 0.7, name = \"Privacy Risk\")<\/pre>\n
  geom_point(color = \"black\", shape = 21, size = 2, fill = \"white\") +<\/pre>\n
p<\/pre>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\"Difficulty\"\/<\/figure>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
Finally let\u2019s label the chart, remove unnecessary ink in things like axes labels, and get rid of white space:<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
p <- p + labs(x = \"Difficulty\",<\/pre>\n
       y = \"Potential Business Value\",<\/pre>\n
       caption = \"Matrix is an adaptation of \\\"PICK\\\" chart https:\/\/en.wikipedia.org\/wiki\/Pick_chart<\/pre>\n
       Quad # represents priority. While working on 1 & 2, attempt to find ways to move 3 & 4 to the left\",\n<\/pre>
       title = \"Prioritization of Techsmith Data Standardization and Centralization\") +<\/pre>\n
  theme(axis.text = element_blank(),<\/pre>\n
        axis.ticks = element_blank(),<\/pre>\n
        axis.line = element_line(arrow = arrow(type = \"closed\", angle = 20, length = unit(0.1, \"inches\")),<\/pre>\n
                                 color = \"grey40\", linetype = \"dashed\"),<\/pre>\n
        legend.position = \"top\") +<\/pre>\n
  scale_x_continuous(limits = c(axis_min , axis_max), expand = c(.02, .02)) +<\/pre>\n
  scale_y_continuous(limits = c(axis_min , axis_max), expand = c(.02, .02))<\/pre>\n
p<\/pre>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\"Prioritizing<\/figure>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
In the final article I will describe how to develop a consistent, manageable process to add sources to a data library once you have prioritized the order.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"
In the previous article, I prescribe prioritizing data sources inclusion in a data library according to business value, difficulty, and privacy concerns. This can be done utilizing a scoring rubric and interviewing the owners and\/or key stakeholders of each data source. While these things may not be measurable they can be quantified in a relative<\/p>\n","protected":false},"author":1135,"featured_media":23735,"comment_status":"open","ping_status":"open","sticky":false,"template":"single-post-2.php","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[187],"tags":[985,977,722],"ppma_author":[3185],"class_list":["post-22944","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-bigdata-cloud","tag-data-library","tag-data-management","tag-data-sources"],"authors":[{"term_id":3185,"user_id":1135,"is_guest":0,"slug":"chris-umphlett","display_name":"Chris Umphlett","avatar_url":"https:\/\/www.experfy.com\/blog\/wp-content\/uploads\/2021\/05\/Chris-Umphlett-150x150.jpg","user_url":"","last_name":"Umphlett","first_name":"Chris","job_title":"","description":"Chris Umphlett is the Manager of Data Analysis and Data Privacy at TechSmith, the makers of great software like Snagit and Camtasia. Before that he worked on analytics teams in the consumer packaged goods, life insurance, and utility industries. He lives in East Lansing, Michigan with his wife and young children."}],"_links":{"self":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/22944","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/users\/1135"}],"replies":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/comments?post=22944"}],"version-history":[{"count":14,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/22944\/revisions"}],"predecessor-version":[{"id":31531,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/22944\/revisions\/31531"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media\/23735"}],"wp:attachment":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media?parent=22944"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/categories?post=22944"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/tags?post=22944"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=22944"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}