{"id":10777,"date":"2020-10-29T09:59:37","date_gmt":"2020-10-29T09:59:37","guid":{"rendered":"https:\/\/www.experfy.com\/blog\/?p=10777"},"modified":"2023-10-16T11:49:56","modified_gmt":"2023-10-16T11:49:56","slug":"next-frontier-machine-learning-anyone-master","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/ai-ml\/next-frontier-machine-learning-anyone-master\/","title":{"rendered":"The Next Frontier In Machine Learning Is Something Anyone Can Master"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

It is frustrating trying to learn about machine learning. Do I use YOLO, Keras, Tensorflow, PyTorch or all of them together somehow? And even if you figure out the PhD stuff, you still have to then master about three other disciplines to get it to work in production; devops, programming, and counting disciplines.<\/mark><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Well I am here for you brothers and sisters in computers, I have good news for your peace of mind.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

The most important thing in getting machine learning to work properly is training data.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

And even more importantly, the skills required to make a good training set have nothing to do with math, computers, or engineering.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

What is training data?<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

The tools to take that data and turn it into a machine learning model<\/a> that works exist today<\/a> and are easy to use. You don\u2019t have to go Berkeley to use them, you just have to be a little familiar with computers.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Is it obvious where the magic is yet?<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

The Magic.<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Where did the 2000 examples e-mails come from? YOU! That\u2019s right, anyone can create a dataset with just a bit of elbow grease, some focus, and the ability to follow a few simple conventions that I\u2019ll outline below after I finish making this awesome point.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Sure, you could go online and find an existing dataset of spam, but if you\u2019re actually try to solve a problem at your company, you ought to create your own dataset that comes from the problem you are trying to solve.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

For example, perhaps your company gets inundated with e-mails that you\u2019d consider spam but don\u2019t fall under the traditional classification of such that perhaps GMAIL uses today.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Well, all you need to do is go through your companies e-mails and put examples of spam into one folder, and examples of not spam into another.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

I agree, it is a bit tedious, but it is worth it in the end because what you are doing is GOLD! YOU ARE BASICALLY MAKING GOLD!!<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

It is practically a miracle what having a human-curated, problem-specific data set will do for your machine learning model\u2019s accuracy. Companies would kill* for such a dataset.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

To summarize, the reasons for this are two-fold;

<\/p>\n



  1. Machine learning models perform best when they\u2019re trained for a specific problem, and aren\u2019t being too generalized. It is like trying to use a hammer on a screw instead of a screwdriver. A human thinking about the problem they\u2019re trying to solve, and building a curated training set around that is the best way to have success with your model.<\/li>
  2. There is less and less room for generalized machine learning models to make a difference for businesses, and so they need to build models based on their own datasets, which some companies might not have or don\u2019t know how to obtain.<\/mark><\/li>
    <\/ol>
    <\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
    \n\t\t\t\t\t\t
    \n\t\t\t\t\t
    \n\t\t\t
    \n\t\t\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t

    What about visual data?<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
    \n\t\t\t\t\t\t
    \n\t\t\t\t\t
    \n\t\t\t
    \n\t\t\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t

    It is the same thing. There is an explosion in data labeling startups<\/a> for visual imagery because of all of what I excellently mentioned above. It really requires the same work as finding spam e-mails; go through your images and label them.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t

    Now I get it \u2014 it sounds incredibly tedious, and it is tedious, which is why you can hire third-party companies to do the labeling for you, and I know plenty of good ones<\/a> out there that I\u2019ve worked with in the past.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t

    But on smaller scales, you can do this yourself, and become a machine learning master! Here are some simple guidelines to get started;<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

    \n\t\t\t\t\t\t
    \n\t\t\t\t\t
    \n\t\t\t
    \n\t\t\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t

    Guidelines<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
    \n\t\t\t\t\t\t
    \n\t\t\t\t\t
    \n\t\t\t
    \n\t\t\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t

    <\/p>\n

      <\/p>\n

    1. For classification, you must have a balanced dataset. That means, if you have 1000 spam e-mails, you also need 1000 not-spam e-mails. The tolerance for this is minimum to none, meaning don\u2019t try and get away with 1300 spam and 700 not-spam \u2014 I\u2019ll know<\/li><\/p>\n

    2. You must have a very clean dataset. That means, don\u2019t have spam e-mails labeled as not-spam and vice versa. Incorrect labels will throw your model off by a surprising amount. I\u2019ve seen accuracy increase by 20% or more after a thorough dataset cleaning.<\/li><\/p>\n

    3. Be random. If you can, select a random selection of spam and regular e-mails. Don\u2019t just start at the beginning of your e-mail history and work your way backward. If you do this, you could be training a model on what spam USED to look like, not what it looks like today.<\/li><\/p>\n

    4. Context matters. If you\u2019re training a model to detect people in your office lobby from your security camera, use example images from the same camera, some with people and others without. You must train the model on what is NOT<\/strong> people as much as you are training it on what is people. Does that make sense? If not\u2026 well\u2026 I dunno this is a blog post so I can\u2019t really answer your questions directly. Moving on\u2026<\/li><\/p>\n

    5. Consistency matters. When I was building a fake news dataset<\/a>, I kept having to start over because I kept changing my mind about what constitutes fake news; satire? blog posts? opinion pieces? Draw the line in the sand somewhere and stick to it throughout the whole process.<\/li><\/p>\n

    6. Validation. Set aside 20% of your training dataset and don\u2019t train your model with it. Use the 80% to train the model, and the 20% to then test your model to see how well it is performing. And make sure you choose that 20% randomly.<\/li><\/p>\n

      <\/ol><\/p>\n

      <\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

      \n\t\t\t\t\t\t
      \n\t\t\t\t\t
      \n\t\t\t
      \n\t\t\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t\t

      Conclusions<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
      \n\t\t\t\t\t\t
      \n\t\t\t\t\t
      \n\t\t\t
      \n\t\t\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t\t\t\t\t\t

      Good datasets matter. Practice making them, play around with the tools<\/a> to see what kind of accuracy you get, or just keep this all in mind when you call a data labeling company to label your data for you.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"

      The most important thing in getting machine learning to work properly is training data. And even more importantly, the skills required to make a good training set have nothing to do with math, computers, or engineering.<\/p>\n","protected":false},"author":227,"featured_media":10778,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[183],"tags":[205,94,92,852],"ppma_author":[1909],"class_list":["post-10777","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-ml","tag-data","tag-data-science","tag-machine-learning","tag-training"],"authors":[{"term_id":1909,"user_id":227,"is_guest":0,"slug":"aaron-edell","display_name":"Aaron Edell","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/?s=96&d=mm&r=g","user_url":"","last_name":"Edell","first_name":"Aaron","job_title":"","description":"Aaron Edell is the co-founder and CEO of Machine Box, Inc., an award-winning startup that builds production-ready machine learning models that anyone can integrate, deploy and scale. A veteran speaker and writer on the topics of machine learning, metadata, and content management, he has published papers on metadata and machine learning and consulted major media and entertainment companies on content management since 2005."}],"_links":{"self":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/10777","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/users\/227"}],"replies":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/comments?post=10777"}],"version-history":[{"count":5,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/10777\/revisions"}],"predecessor-version":[{"id":33450,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/10777\/revisions\/33450"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media\/10778"}],"wp:attachment":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media?parent=10777"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/categories?post=10777"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/tags?post=10777"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=10777"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}