{"id":22619,"date":"2021-02-12T10:30:24","date_gmt":"2021-02-12T10:30:24","guid":{"rendered":"https:\/\/www.experfy.com\/blog\/mathematical-laws-know-data-scientist\/"},"modified":"2023-09-05T06:35:10","modified_gmt":"2023-09-05T06:35:10","slug":"mathematical-laws-know-data-scientist","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/ai-ml\/mathematical-laws-know-data-scientist\/","title":{"rendered":"3 Mathematical Laws To Know As A Data Scientist"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Some interesting laws that help you as a Data Scientist<\/em><\/p>\n

While Data Scientist was working with Data for their main activity, it doesn’t mean that Mathematical knowledge is something we do not need. Data scientists need to learn and understand the mathematical theory behind machine learning to efficiently solving business problems.<\/p>\n

The mathematical behind machine learning is not just a random notation thrown here and there, but it consisted of many theories and thoughts. This thought creates a lot of mathematical laws that contributed to the machine learning we able to use right now. Although you could use the mathematical in any way you want to solve the problem, mathematical laws are not limited to machine learning after all.<\/p>\n

In this article, I want to outline some of the interesting mathematical laws that could help you as a Data Scientist. Let\u2019s get into it.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

Benford\u2019s Law<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Benford\u2019s law<\/strong> also called the Newcomb\u2013Benford law<\/strong>, the law of anomalous numbers<\/strong>, or the first-digit law, <\/strong>is a mathematical law about the leading digit number in a real-world dataset.<\/p>\n

When we think about the first digit of the numbers, it should be distributed uniformly when we randomly took a number. Intuitively, the random number leading digit 1 should have the same probability as leading digit 9, which is ~11.1%. Surprisingly, this is not what happens.<\/p>\n

Benford\u2019s law states that the leading digit is likely to be small in many naturally occurring collections of numbers<\/strong>. Leading digit 1 happens more often than 2, leading digit 2 occurs more often than 3, and so on.<\/p>\n

Let\u2019s try using a real-world dataset to see how this law is applicable. For this article, I would use the data from\u00a0Kaggle\u00a0regarding Spotify Track song from 1921\u20132020. From the data, I would take the leading digit of the song durations.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\"3\t\t\t\t\t\t\t\t\t\t\t
Image created by Author<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

From the image above, we can see that the leading digit 1 occurs the most, then it is decreasing following the higher number. This is what Benford\u2019s Law state above.<\/p>\n

If we talk about the proper definition, Benford law state that a set of numbers is said to satisfy Benford\u2019s law if the leading digit d ( \u22081,\u2026,9) occurs with the equation.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\"3\t\t\t\t\t\t\t\t\t\t\t
Image created by Author<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

From this equation, we acquired the leading digit with the following distribution.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\"3\t\t\t\t\t\t\t\t\t\t\t
Image created by Author<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

With this distribution, we can predict that 1 as the leading digit is 30% likely to occurs more than the other leading digit.<\/p>\n

Many applications for this law, for example, fraud detection on tax forms, election results, economic numbers, and accounting figures.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

Law of Large Numbers (LLN)<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

The Law of Large Number stated that <\/mark>as the number of trials of a random process increases, the results’ average would get closer to the expected values or theoretical values.<\/strong><\/mark><\/p>\n

For example, when rolling the dice. The possibility of 6-side dice is 1,2,3,4,5 and 6. The mean for the 6-side dice would be 3.5. As we are rolling the dice, the number we get would be random from 1 to 6, but as we keep rolling the dice, the result’s average would get closer to the expected value, which is 3.5. This is what the Law of Large Numbers denote.<\/p>\n

While it is useful, the tricky part here is that you need many experiments or occurrences. However, a large number required means that it is good to predict long-term stability.<\/p>\n

The Law of Large Numbers is different than the Law of Average, where it was used to express a belief that outcomes of a random event will \u201ceven out\u201d within a small sample. This is what we called \u201cGambler\u2019s Fallacy,\u201d <\/strong>where we expect the expected value would occur in a smaller sample.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

Zipf\u2019s Law<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Zipf\u2019s law was created for quantitative linguistic, which states that given some natural language dataset corpus, any word’s frequency is inversely proportional to its frequency table rank<\/strong>. Thus the most frequent word will occur approximately twice as often as the second most frequent word, three times as often as the third most frequent word.<\/p>\n

For example, in the previous Spotify dataset, I would try to split all the words and punctuation to count them. Below is the top 12 of the most common words and their frequency.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\"3\t\t\t\t\t\t\t\t\t\t\t
Image Created by Author<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

When I sum all the word that exists in the Spotify corpus, the total is 759389. We could see if Zipf\u2019s law applies to this dataset by counting the probability when they occur. The first most occurring word or punctuation is \u2018-\u2019 with 32258, which has the probability of ~4% then followed by \u2018the,\u2019 which has the probability of ~2%.<\/p>\n

Faithful to the law, the probability would keep going down in some of the words. Of course, there is a little deviation, but the probability would go down most of the time following the frequency rank increase.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

Conclusion<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

These are some interesting mathematical laws to know as a Data Scientist and definitely would help you in your Data Science work. The laws are:<\/p>\n

  • Benford\u2019s Law<\/li>
  • Law of Large Number<\/li>
  • Zipf\u2019s Law<\/li><\/ul>\n

    I hope it helps!<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"

    Some interesting laws that help you as a Data Scientist While Data Scientist was working with Data for their main activity, it doesn’t mean that Mathematical knowledge is something we do not need. Data scientists need to learn and understand the mathematical theory behind machine learning to efficiently solving business problems. The mathematical behind machine<\/p>\n","protected":false},"author":1052,"featured_media":18704,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[183],"tags":[394,92,1331],"ppma_author":[3873],"class_list":["post-22619","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-ml","tag-data-scientist","tag-machine-learning","tag-mathematical-laws"],"authors":[{"term_id":3873,"user_id":1052,"is_guest":0,"slug":"cornellius-yudha-wijaya","display_name":"Cornellius Yudha Wijaya","avatar_url":"https:\/\/www.experfy.com\/blog\/wp-content\/uploads\/2021\/05\/Cornellius-Yudha-Wijaya-150x150.jpeg","user_url":"https:\/\/careers.allianz.com","last_name":"Yudha Wijaya","first_name":"Cornellius","job_title":"","description":"Cornellius Yudha Wijaya is Data Scientist at Allianz Life, a global leader in insurance and financial services."}],"_links":{"self":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/22619","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/users\/1052"}],"replies":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/comments?post=22619"}],"version-history":[{"count":5,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/22619\/revisions"}],"predecessor-version":[{"id":32229,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/22619\/revisions\/32229"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media\/18704"}],"wp:attachment":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media?parent=22619"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/categories?post=22619"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/tags?post=22619"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=22619"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}