{"id":2259,"date":"2020-02-14T02:50:19","date_gmt":"2020-02-13T23:50:19","guid":{"rendered":"http:\/\/kusuaks7\/?p=1864"},"modified":"2024-01-10T15:47:37","modified_gmt":"2024-01-10T15:47:37","slug":"five-more-tools-and-techniques-for-better-plotting","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/bigdata-cloud\/five-more-tools-and-techniques-for-better-plotting\/","title":{"rendered":"Five more tools and techniques for better plotting"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

And getting the most out of your data<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

In real-life data science, plotting does matter.\u00a0In my day-to-day life, I spend more time plotting and analysing those charts, than doing anything else. Let me explain myself, I work at Ravelin Technology. Our business is data, and specifically, analyzing and predicting fraud for online merchants. The main product of the company uses a combination of machine learning, network analysis, rules and human insights for predicting if a transaction might or might not be fraud. We have an ad-hoc machine learning model for each one of our clients, but the building of that model is something that happens at the beginning of the relationship with them and then it mostly requires maintenance.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Maintenance how? Sometimes it\u2019s for introducing new features or because of behavioural changes in customers. However, it can also be the case that something changes in the data we receive. Or perhaps there\u2019s just something we were originally missing for not having enough data when we built the model for the first time. It can also happen that either the client or us spot some dodgy performance in our predictions in, for example, one specific country. Whatever is the case, there\u2019s usually an extensive investigation to find out what\u2019s the problem and\/or what could we do better. And just to give a bit more of context, analyzing the performance of a model for us usually implies dataset with millions of rows and thousands of columns. This can only be addressed by plotting. It\u2019s almost impossible to find patterns or insights just by looking at the data. Plotting allows us to compare features\u2019 performance, see the evolution through time, distribution of values, differences in mean and median values, etc., etc., etc., etc.<\/p>\n

As I said in my previous story, in our field we must equally weight the importance of explainability and interpretability. Real-life Data Science never finds you working alone on a project and your workmates and\/or clients usually won\u2019t know much about the data you\u2019ll be using. Being able to explain your thinking process is a key part of any data-related job. That\u2019s why copying and pasting are not enough and charts personalization becomes key.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Today we\u2019ll go through 5 techniques to make better charts that I\u2019ve found useful in the past. Some of them are day-to-day tools, while others you\u2019ll use them every now and then. But having this story at hand, hopefully, will come in handy when the moment arrives. The libraries we\u2019ll be using are:<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\n
import matplotlib.pyplot as pltimport seaborn as sns<\/div>\n

With the following style and configurations:<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
plt.style.use(\u2018fivethirtyeight\u2019)%config InlineBackend.figure_format = \u2018retina\u2019%matplotlib inline<\/div>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

1. Change range and steps in axis<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

The default configuration of matplotlib or seaborn for setting up the range and steps it\u2019s usually good enough for visualizing out data, but sometimes we\u2019ll want to see all the steps in our axis explicitly shown. Or perhaps, something I\u2019ve found useful is drawing all the data but including the axis labels just for a specific range of the y or x-axis.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

For example, let\u2019s say we\u2019re plotting the distribution of our model\u2019s predictions and we want to concentrate ourself in the values in between 30 and 50, with a step every two units, and without losing sight of the rest of the values. Our original seabon\u2019s \u2018distplot\u2019 would be like:<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

We have now two options for accomplishing the idea above:<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
ax.set_xticks(range(30, 51, 2))ax.xaxis.set_ticks(np.arange(30, 51, 2))<\/div>\n

In both cases, we need to specify the starting point, ending point and step. Mind how the ending point follows a \u2018less than\u2019 kind of logic instead of \u2018equal to or less than\u2019. The result would be the following chart:<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Also, mind how I\u2019m calling both options from the \u2018ax\u2019 object, given that\u2019s the default. When we create any kind of chart the axis (\u2018ax\u2019) and a figure (\u2018fig\u2019) are automatically created. We can also do the same the following way:<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\n
myplot = sns.distplot(mydata)myplot.set_xticks(range(30,51,2))<\/div>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

2. Rotate ticks<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

This is an easy one but very very useful tip if, for example, we\u2019re dealing with text labels instead of numbers. We can do this just by using the \u2018rotation\u2019 hyperparameter in \u2018set_xticklabels\u2019:<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
ax.set_xticklabels(labels=my_labels, rotation=90)<\/div>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Note how I\u2019m also passing \u2018my_labels\u2019 to the \u2018labels\u2019 hyperparameter since that\u2019s mandatory when using \u2018xticklabels\u2019. However, if you\u2019re drawing a \u2018distplot\u2019, you can simply pass the range of values to be shown, while for any other chart, you can pass exactly the same array you specified for the x-axis. Also, you can combine this with the first technique like this:<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
range_step = np.arange(30, 51, 2)ax.xaxis.set_ticks(range_step)ax.set_xticklabels(labels=range_step, rotation=90);<\/div>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Obtaining the following result:<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

3. Change the space in between plots<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

More often than not, we\u2019ll want to plot several charts at once to compare their results, visualize them all together, or perhaps just to save time and\/or space. In any case, we can do that by using \u2018subplots\u2019 in a very simple way:<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
fig, ax = plt.subplots(figsize=(18,10), nrows=2)<\/div>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

We specified two rows, and therefore we\u2019ll be plotting two charts:<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
sns.distplot(mydata, ax=ax[0])sns.lineplot(x=mydata[\u2018xaxis\u2019], y=mydata[\u2018yaxis\u2019], ax=ax[1])<\/div>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Now, sometimes, instead of having only two charts, we might have more. And perhaps we need to include titles for all of them. We may also have some charts with text labels that will need to rotate them for better readability. In cases like this, we could end up with some overlap between plots and increasing the space between charts could help us to visualize them better.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
plt.subplots_adjust(hspace = 0.8)<\/div>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Mind how the hight of the figure remains the same (10 in this case), but the space in between charts increases. If you want to maintain your charts\u2019 size, you\u2019d have to increase your figure size through the \u2018figsize\u2019 hyperparameter.<\/p>\n

\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Also, the hyperparameter \u2018hspace\u2019 follows the horizontal space. If you were drawing multiple columns instead of rows, you could accomplish the same by using the hyperparameter \u2018vspace\u2019.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

By the way, if you want to found out how to set titles for your charts, you can find that tip and some others for better plotting in my\u00a0previous story<\/a>.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\n

NOTE<\/strong>: just like I specified two rows to be drawn above, you could also specify a fixed number of columns. In that case, the indexing of the charts would follow a two indexes logic like ax=[0,1].<\/p>\n<\/blockquote>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

4. Customize your confusion matrix<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Unfortunately, this is not the space for explaining in depth how the confusion matrix works or what it is useful for. Nonetheless, if you fancy learning more about it, I always recommend\u00a0this story<\/a>\u00a0from M. Sunasra.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Now, if you\u2019re already familiar with the concept, you might have encountered in the past that sometimes the default heatmap created by \u2018plot_confusion_matrix\u2019 from \u2018sklearn.metrics\u2019 library, comes out with the upper and lower squares cut off, like the following picture:<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

source:\u00a0https:\/\/gis.stackexchange.com\/<\/a><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

We can solve this by plotting our own confusion matrix from scratch using just a bunch of lines. For example:<\/p>\n

fig = plt.figure(figsize=(12,10))<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

cm = skplt.metrics.confusion_matrix(real_y, pred_y)<\/span><\/p>\n

labels=[0,1,2,3,4]<\/span><\/p>\n

ax = sns.heatmap(cm, annot=True,annot_kws={\u201csize\u201d:12}, fmt=\u2019g\u2019, cmap=\u201dBlues\u201d, xticklabels=labels, yticklabels=labels)<\/span><\/p>\n

bottom, top = ax.get_ylim()<\/span><\/p>\n

ax.set_ylim(bottom + 0.5, top \u2014 0.5)<\/span><\/p>\n

ax.set(ylabel=\u2019True label\u2019)<\/span><\/p>\n

ax.set(xlabel=\u2019Predicted label\u2019)<\/span><\/p>\n

plt.show()<\/span><\/p>\n

What we\u2019re doing here is:<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
    \n \t
  1. We create an empty figure. Wider than taller since we\u2019ll have the annotations to the right of the heatmap<\/li>\n \t
  2. We get the values of our confusion matrix through \u2018skplt.metrics.confusion_matrix\u2019<\/li>\n \t
  3. We specified the \u2018labels\u2019 according to the number of categories we have<\/li>\n \t
  4. We create a heatmap using the values from point 2 and specifying: i) \u2018annotations\u2019 equal True, ii) \u2018annot_kws\u2019 for specifying the font size of the annotations (12 in this case), iii) \u2018fmt\u2019 for the passing the string formatting code, iv) \u2018cmap\u2019 for the colour pattern, v) And finally we specify the labels for both axis (in a confusion matrix both of them are the same)<\/li>\n \t
  5. We get the y-axis view limits and we set the again +- 0.5<\/li>\n \t
  6. Last step: set the y and \u2018xlabels\u2019 to true and predicted label respectively<\/li>\n<\/ol>\n

    The result should be something like this:<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

    \n\t\t\t\t\t\t
    \n\t\t\t\t\t
    \n\t\t\t
    \n\t\t\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t

    5. Plot accumulative distributions<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t

    Surely I don\u2019t need to size how useful can be plotting accumulative distributions, either for better understanding the percentage of elements up to certain value or for comparing two different groups within our data.<\/p>\n

    You can easily get this kind of charts through Seaborn\u2019s \u2018distplot\u2019 chart itself just by setting the following:<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t
    sns.distplot(my_data, label=\u2019my label\u2019, color=\u2019red\u2019, hist_kws=dict(cumulative=True))\n<\/pre>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t
    We can make the chart look better by setting the limits for the x-axis:<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t
    sns.distplot(my_data, label=\u2019my label\u2019, color=\u2019red\u2019, hist_kws=dict(cumulative=True)).set(xlim=(0, my_data.max()))<\/div>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t
    As I said at the beginning of the story, some of these tools or tips I use them all the time, while some others only every now and then. But hopefully, knowing these quick fixes and techniques will help you to make better plots and to better understand your data itself.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"
    Real-life Data Science never finds you working alone on a project and your workmates or clients usually won’t know much about the data you’ll be using. Being able to explain your thinking process is a key part of any data-related job. That’s why copying and pasting are not enough and charts personalization becomes key. This blog goes through 5 techniques to make better charts that are useful. Some of them are day-to-day tools, while others you’ll use them every now and then.<\/p>\n","protected":false},"author":726,"featured_media":3684,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[187],"tags":[94],"ppma_author":[3568],"class_list":["post-2259","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-bigdata-cloud","tag-data-science"],"authors":[{"term_id":3568,"user_id":726,"is_guest":0,"slug":"gonzalo-ferreiro-volpi","display_name":"Gonzalo Volpi","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/?s=96&d=mm&r=g","user_url":"","last_name":"Volpi","first_name":"Gonzalo","job_title":"","description":"Gonzalo Ferreiro Volpi is Data Science Fellow for Investigations at Ravelin Technology,<\/a> award-winning fraud detection and prevention platform for online merchants and the payments industry in eCommerce. "}],"_links":{"self":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/2259","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/users\/726"}],"replies":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/comments?post=2259"}],"version-history":[{"count":5,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/2259\/revisions"}],"predecessor-version":[{"id":35448,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/2259\/revisions\/35448"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media\/3684"}],"wp:attachment":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media?parent=2259"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/categories?post=2259"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/tags?post=2259"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=2259"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}