{"id":2253,"date":"2020-02-12T04:24:21","date_gmt":"2020-02-12T04:24:21","guid":{"rendered":"http:\/\/kusuaks7\/?p=1858"},"modified":"2024-01-11T07:06:56","modified_gmt":"2024-01-11T07:06:56","slug":"the-bootstrap-the-swiss-army-knife-of-any-data-scientist","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/bigdata-cloud\/the-bootstrap-the-swiss-army-knife-of-any-data-scientist\/","title":{"rendered":"The bootstrap. The Swiss army knife of any data scientist"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\n

Every measure must be followed by an\u00a0error estimate<\/strong>. There\u2019s no chance to avoid this. If I tell you \u201cI\u2019m 1,93 metres tall\u201d, I\u2019m not giving you any information about the\u00a0precision\u00a0<\/strong>of this measure. You could think that my precision is on the second decimal digit, but you can\u2019t be sure.<\/p>\n

So, what we really need is some way to assess the precision of our measure starting from the data sample we have.<\/p>\n

If our observable is the mean value calculated over a sample, a simple precision estimate is given by the\u00a0standard error<\/strong>. But what can we do if we are measuring something that is not the mean value? That\u2019s the point at which bootstrap comes in help.<\/p>\n\n<\/section>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

Bootstrap in a nutshell<\/h1><\/h1>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Bootstrap<\/strong>\u00a0is a technique made in order to measure confidence intervals and\/or\u00a0standard error\u00a0<\/strong>of an observable that can be calculated on a sample.<\/p>\n

It relies on the concept of\u00a0resampling<\/strong>, which is a procedure that, starting from a data sample,\u00a0simulates a new sample\u00a0<\/strong>of the same size, considering every original value with\u00a0replacement<\/strong>. Each value is taken at the same probability of the others (which is\u00a01\/N<\/em>).<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

So, for example, if our sample is made by the four values {1,2,3,4}, some possible resampling can be {1,4,1,3}, {1,2,4,4}, {1,4,3,2} and so on. As you can see, the values are the same, they\u00a0can be repeated<\/strong>\u00a0and the samples have the\u00a0same size<\/strong>\u00a0as the original one.<\/p>\n

Now, let\u2019s say we have a sample of\u00a0N<\/em>\u00a0values and want to calculate some observable\u00a0O<\/em>, that is a function that takes in input all the\u00a0N<\/em>\u00a0values and gives in output one real number. An example can be the average value, the standard deviation or even more complex functions as the quantiles, the Sharpe ratio and so on.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Our goal is to calculate the expected value of this observable and, for example, its\u00a0standard error<\/strong>.<\/p>\n

To accomplish this goal, we can run the resampling procedure\u00a0<\/span><\/mark>M<\/span><\/em><\/mark>\u00a0times and, for each sample, we can calculate our observable. At the end of the process, we have\u00a0<\/span><\/mark>M\u00a0<\/span><\/em><\/mark>different values of\u00a0<\/span><\/mark>O<\/span><\/em><\/mark>, that can be used to calculate the expected value, the standard error, the confidence intervals and so on.<\/span><\/mark><\/span><\/p>\n

This simple procedure is the\u00a0bootstrap process<\/strong>.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

\n

An example in R<\/h1><\/h1>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

R has a wonderful library called\u00a0bootstrap\u00a0<\/em>that performs all the calculations for us.<\/p>\n

Let\u2019s say we want to measure the\u00a0skewness\u00a0<\/strong>of the\u00a0sepal length<\/strong>\u00a0in the famous\u00a0iris dataset<\/strong>\u00a0and let\u2019s say we want to know its expected value, its standard error and the 5th and 9th percentile.<\/p>\n

Here follows the R code that makes this possible:<\/p>\n

The mean value is 0.29, the standard deviation is 0.13, the 5th percentile is 0.10 and the 95th percentile is 0.49.<\/p>\n

These numbers allow us to say that the real skewness value of the dataset is 0.29 +\/- 0.13 and it is between 0.10 and 0.49 with a 90% confidence.<\/p>\n

Surprisingly, isn\u2019t it?<\/p>\n

This simple procedure is universal and can be used with\u00a0any kind of observable<\/strong>.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

The larger the dataset, the higher the precision<\/h1><\/h1>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Somebody could think that the larger our dataset, the smaller the standard error we take due to the\u00a0law of large number<\/strong>.<\/p>\n

It\u2019s definitely true and we can simulate this case.<\/p>\n

Let\u2019s simulate\u00a0N<\/em>\u00a0random uniformly distributed numbers and let\u2019s calculate the skewness over them. What we expect is that the\u00a0standard deviation<\/strong>\u00a0of the bootstrapped skewness\u00a0decreases<\/strong>\u00a0as long as\u00a0N<\/em>\u00a0increases.<\/p>\n

The following R code performs this calculation and, (not so) surprisingly, the bootstrap standard deviation decreases as a\u00a0power law<\/strong>\u00a0with the exponent equal to -1\/2.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

The slope of the double-log linear regression is the\u00a0exponent<\/strong>\u00a0of the power law and it\u2019s equal to\u00a0-0.5<\/strong>.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

Conclusions<\/h1><\/h1>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Bootstrap is a method to extract as much\u00a0information\u00a0<\/strong>as possible from a finite size dataset and it makes us calculate the expected value of any observable and its precision (i.e. its standard deviation or confidence intervals).<\/p>\n

It\u2019s really useful when we need to calculate an error estimate for some scientific measure and it can easily generalized for\u00a0multivariate observables<\/strong>.<\/p>\n

Any data scientist should not forget to use this powerful tool, which has shown several useful application even in\u00a0machine learning<\/strong>\u00a0(e.g in the Random Forest classification\/regression models).<\/p>\n\n<\/section>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"

Bootstrap is a method to extract as much information as possible from a finite size dataset and it makes us calculate the expected value of any observable and it’s precision. It’s really useful when we need to calculate an error estimate for some scientific measure and it can easily be generalized for multivariate observables. Any data scientist should not forget to use this powerful tool, which has shown several useful applications even in machine learning. <\/p>\n","protected":false},"author":618,"featured_media":3655,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[187],"tags":[94],"ppma_author":[3328],"class_list":["post-2253","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-bigdata-cloud","tag-data-science"],"authors":[{"term_id":3328,"user_id":618,"is_guest":0,"slug":"gianluca-malato","display_name":"Gianluca Malato","avatar_url":"https:\/\/www.experfy.com\/blog\/wp-content\/uploads\/2020\/04\/medium_918623b2-8f36-4110-8343-6fc9228595dd-150x150.jpg","user_url":"http:\/\/www.gianlucamalato.it\/","last_name":"Malato","first_name":"Gianluca","job_title":"","description":"Gianluca Malato is Data Scientist at Poste Italiane SPA.\u00a0 He is also a fiction author and software developer, Editor of\u00a0Data Science Journal<\/a>,\u00a0The Trading Scientist<\/a>, and\u00a0The Writer\u2019s Notebook<\/a>. His books are available on Amazon<\/a>."}],"_links":{"self":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/2253","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/users\/618"}],"replies":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/comments?post=2253"}],"version-history":[{"count":6,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/2253\/revisions"}],"predecessor-version":[{"id":35470,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/2253\/revisions\/35470"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media\/3655"}],"wp:attachment":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media?parent=2253"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/categories?post=2253"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/tags?post=2253"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=2253"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}