{"id":843,"date":"2018-08-17T03:13:11","date_gmt":"2018-08-17T00:13:11","guid":{"rendered":"http:\/\/kusuaks7\/?p=448"},"modified":"2021-12-15T05:13:41","modified_gmt":"2021-12-15T05:13:41","slug":"newbies-guide-to-deep-learning","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/ai-ml\/newbies-guide-to-deep-learning\/","title":{"rendered":"Newbie\u2019s guide to Deep Learning"},"content":{"rendered":"<p><strong><em>Ready to learn Machine Learning? Browse<\/em><\/strong> <strong><em><a href=\"https:\/\/www.experfy.com\/training\/tracks\/machine-learning-training-certification\">Machine Learning Training and Certification courses<\/a> developed by industry thought leaders and Experfy in Harvard Innovation Lab.<\/em><\/strong><\/p>\n<p id=\"5992\"><a href=\"http:\/\/www.fast.ai\/\" target=\"_blank\" rel=\"noopener noreferrer\" data-href=\"http:\/\/www.fast.ai\/\" data->Fast.ai<\/a>\u00a0offers a free online course on Deep Learning and they offer two parts in their course:<\/p>\n<ul>\n<li id=\"68b1\">Deep Learning Part 1:\u00a0<a href=\"http:\/\/course.fast.ai\/\" target=\"_blank\" rel=\"noopener noreferrer\" data-href=\"http:\/\/course.fast.ai\/\" data->Practical Deep Learning for Coders<\/a><\/li>\n<li id=\"b83e\">Deep Learning Part 2:\u00a0<a href=\"http:\/\/course.fast.ai\/part2.html\" target=\"_blank\" rel=\"noopener noreferrer\" data-href=\"http:\/\/course.fast.ai\/part2.html\" data->Cutting Edge Deep Learning for Coders<\/a><\/li>\n<\/ul>\n<p id=\"e2d1\">After those courses, you may be ready to tackle\u00a0Hinton\u2019s Neural Networks for Machine Learning. Hinton\u2019s course is relatively harder compared to previously mentioned courses since the lectures are quite dry and they contain more Math concepts. If you feel like you cannot tackle the course yet, don\u2019t get discouraged! Leave it for a while and do the Math part (described in next section) and then come back. You will be able to definitely tackle the course this time! Remember, determination, determination and yes, more determination.<\/p>\n<h3 id=\"1d4c\"><strong>Math<\/strong><\/h3>\n<p id=\"65b8\">Deep Learning definitely requires you to have a strong command of Linear Algebra, Differential Calculus and Vector Calculus, just to name a few. If you want to quickly brush up some elementary Linear Algebra and start coding, Andrej Karpathy\u2019s\u00a0<a href=\"http:\/\/karpathy.github.io\/neuralnets\/\" target=\"_blank\" rel=\"noopener noreferrer\" data-href=\"http:\/\/karpathy.github.io\/neuralnets\/\" data->Hacker\u2019s guide to Neural Networks<\/a>\u00a0is highly recommended. I find\u00a0<a href=\"https:\/\/github.com\/hadrienj\/deepLearningBook-Notes\" target=\"_blank\" rel=\"noopener noreferrer\" data-href=\"https:\/\/github.com\/hadrienj\/deepLearningBook-Notes\" data->hadrienj\u2019s notes<\/a>\u00a0on Deep Learning Book extremely useful to actually see how underlying Math concepts work using Python (Numpy). If you like to learn from videos,\u00a0<a href=\"https:\/\/www.youtube.com\/channel\/UCYO_jab_esuFRV4b17AJtAw\" target=\"_blank\" rel=\"noopener noreferrer\" data-href=\"https:\/\/www.youtube.com\/channel\/UCYO_jab_esuFRV4b17AJtAw\" data->3blue1brown<\/a>\u00a0has one of the most intuitive videos for concepts in\u00a0<a href=\"https:\/\/www.youtube.com\/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab\" target=\"_blank\" rel=\"noopener noreferrer\" data-href=\"https:\/\/www.youtube.com\/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab\" data->Linear Algebra<\/a>,\u00a0<a href=\"https:\/\/www.youtube.com\/playlist?list=PLZHQObOWTQDMsr9K-rj53DwVRMYO3t5Yr\" target=\"_blank\" rel=\"noopener noreferrer\" data-href=\"https:\/\/www.youtube.com\/playlist?list=PLZHQObOWTQDMsr9K-rj53DwVRMYO3t5Yr\" data->Calculus<\/a>,\u00a0<a href=\"https:\/\/www.youtube.com\/playlist?list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi\" target=\"_blank\" rel=\"noopener noreferrer\" data-href=\"https:\/\/www.youtube.com\/playlist?list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi\" data->Neural Networks<\/a>\u00a0and other interesting Math topics. Implementing your own CPU-based backpropagation algorithm on a non-convolution based problem is also a good place to start to truly understand how backpropagation works.<\/p>\n<h3 id=\"195e\"><strong>Getting Serious<\/strong><\/h3>\n<p id=\"c37a\">If you want to take a notch up your Machine Learning knowledge and ready to get serious (I mean graduate-level serious), dive into\u00a0<a href=\"http:\/\/www.work.caltech.edu\/telecourse.html\" target=\"_blank\" rel=\"noopener noreferrer\" data-href=\"http:\/\/www.work.caltech.edu\/telecourse.html\" data->Learning From Data<\/a>\u00a0by Caltech Professor Yaser Abu-Mostafa. Be prepared to do all the Math. It can be a bit challenging but it will definitely be rewarding once you have gone through it and did your work. I believe it would be hard for textbooks to capture the current state of Deep Learning since the field is moving at a very fast pace. But the go-to textbook would be\u00a0<a href=\"http:\/\/www.deeplearningbook.org\/\" target=\"_blank\" rel=\"noopener noreferrer\" data-href=\"http:\/\/www.deeplearningbook.org\/\" data->Deep Learning Book<\/a>\u00a0by Goodfellow, Bengio, and Courville. It is freely available online so you might as well download chapter by chapter and tackle the textbook one chapter at a time.<\/p>\n<h4 id=\"eace\"><strong>Papers, papers, papers, oh dang it I can\u2019t catch \u2019em up\u00a0anymore<\/strong><\/h4>\n<p id=\"28f9\">Yeah, the knowledge of deep learning comes primarily from papers and the rate at they are being published is extreme these days. A good starting point is Reddit. Subscribe to\u00a0<a href=\"https:\/\/www.reddit.com\/r\/MachineLearning\/\" target=\"_blank\" rel=\"noopener noreferrer\" data-href=\"https:\/\/www.reddit.com\/r\/MachineLearning\/\" data->\/r\/machinelearning<\/a>\u00a0and\u00a0<a href=\"https:\/\/www.reddit.com\/r\/deeplearning\/\" target=\"_blank\" rel=\"noopener noreferrer\" data-href=\"https:\/\/www.reddit.com\/r\/deeplearning\/\" data->\/r\/deeplearning<\/a>. I find machinelearning subreddit more useful though.\u00a0<a href=\"http:\/\/www.arxiv-sanity.com\/\" target=\"_blank\" rel=\"noopener noreferrer\" data-href=\"http:\/\/www.arxiv-sanity.com\/\" data->ArxivSanity<\/a>\u00a0is a good place to check out papers which are related to the ones you are looking for. One important thing to do when reading papers in Deep Learning is doing a good literature review. Doing a good literature review gives you a good sense of how things evolve. A way to tackle doing literature review is to install\u00a0<a href=\"https:\/\/chrome.google.com\/webstore\/detail\/google-scholar-button\/ldipcbpaocekfooobnbcddclnhejkcpn\" target=\"_blank\" rel=\"noopener noreferrer\" data-href=\"https:\/\/chrome.google.com\/webstore\/detail\/google-scholar-button\/ldipcbpaocekfooobnbcddclnhejkcpn\" data->Google Scholar Chrome Extension<\/a>\u00a0and search for the paper you want to look up. You can follow the \u201cRelated articles\u201d and \u201cCited by\u201d to follow the prior work as well as newer work based on that paper. A good habit to form when reading a paper is to draw a mind map of the concepts in the paper.<\/p>\n<figure id=\"da8f\"><canvas width=\"75\" height=\"60\"><\/canvas><img decoding=\"async\" style=\"width: 700px; height: 573px;\" src=\"https:\/\/cdn-images-1.medium.com\/max\/800\/1*BdhP4VebJ6_o_QIzjnzofQ.png\" data-src=\"https:\/\/cdn-images-1.medium.com\/max\/800\/1*BdhP4VebJ6_o_QIzjnzofQ.png\" \/><\/figure>\n<p id=\"73b5\" style=\"text-align: center;\">I drew this mind map when I read a paper on few-shot learning [1]\u2014 drawn with\u00a0<a href=\"https:\/\/simplemind.eu\/\" target=\"_blank\" rel=\"noopener noreferrer\" data-href=\"https:\/\/simplemind.eu\/\" data->SimpleMind Lite<\/a><\/p>\n<p>The advantage of a mind map is that it is a good way to keep track of the relationships of concepts presented in the paper. I find mind maps very useful to keep track of related literature and how they relate to the paper I am reading. Mind maps give me a clear picture of a paper and also serves as a good summary of the paper after I have read it.<\/p>\n<p id=\"8c6b\">I find Twitter very useful to follow Machine Learning and Deep Learning research. You can start by following well-known individuals in the ML\/DL field and branch out from there. As I usually retweet researches on Adversarial Machine Learning and on self-driving cars, you can also follow\u00a0<a href=\"https:\/\/twitter.com\/ark_aung\" target=\"_blank\" rel=\"noopener noreferrer\" data-href=\"https:\/\/twitter.com\/ark_aung\" data->me on twitter<\/a>\u00a0and treat as your feed. What you can do is check out the people whom I have retweeted, check out their tweets and follow other researchers in their circles. Twitter will also recommend good people to follow once you have provided enough data, i.e. followed enough ML\/DL researchers (ML FTW!).<\/p>\n<h3 id=\"e200\"><strong>Kaggle<\/strong><\/h3>\n<p id=\"ef02\">I cannot stress how useful\u00a0<a href=\"https:\/\/www.kaggle.com\/\" target=\"_blank\" rel=\"noopener noreferrer\" data-href=\"https:\/\/www.kaggle.com\/\" data->Kaggle<\/a>\u00a0is. I highly recommend trying out Kaggle competitions even though you have a slim chance of even getting to top-100. The value of Kaggle competitions is the community. Read the kernels and take good practices from them. Read comments and engage in discussions. That\u2019s where you will learn tremendously. You will learn how people do Exploratory Data Analysis and how they handle various cases of missing data, skewed data, etc. You will also learn how people make decisions on why they chose certain models over the other. There is so much knowledge in Kaggle competitions.<\/p>\n<h3 id=\"880e\"><strong>Inspirations<\/strong><\/h3>\n<p id=\"1212\"><a href=\"http:\/\/www.r2d3.us\/visual-intro-to-machine-learning-part-1\/\" target=\"_blank\" rel=\"noopener noreferrer\" data-href=\"http:\/\/www.r2d3.us\/visual-intro-to-machine-learning-part-1\/\" data->Visual Introduction to Machine Learning<\/a>\u00a0is a good way to visually grasp how statistical learning techniques are used to identify patterns in data.<\/p>\n<p id=\"6530\">Google\u2019s\u00a0<a href=\"http:\/\/tools.google.com\/seedbank\/\" target=\"_blank\" rel=\"noopener noreferrer\" data-href=\"http:\/\/tools.google.com\/seedbank\/\" data->Seedbank<\/a>\u00a0is a great resource to get inspired! Take a look at the examples and follow the literature.<\/p>\n<p id=\"65fd\"><a href=\"https:\/\/distill.pub\/\" target=\"_blank\" rel=\"noopener noreferrer\" data-href=\"https:\/\/distill.pub\/\" data->Distill.pub<\/a>\u00a0is a good place to learn some of the DL concepts interactively. I wish Distill has more articles than it has now.<\/p>\n<h3 id=\"ea4e\"><strong>Tip of the\u00a0Iceberg<\/strong><\/h3>\n<p id=\"2211\">Nothing matters unless you implement what you have learnt yourself. ML and DL sound like magic until you implement the whole pipeline yourself. The whole pipeline includes Data Sourcing, Data Collection, Data Quality Assessment, Data Cleaning, Data Annotation, Data Preprocessing, Building Workflow, Building Models, Tuning Models, Assessing Models, Deploying Models and Reiterating Models. Those steps are just some of the steps in the whole ML\/DL pipeline. Those who have done full-scale DL work knows how important it is to keep the whole development operations as streamlined as possible.\u00a0Not only the whole data sourcing, collection, annotation, cleaning and assessing steps take at least 60% of the whole project, but also they can be one of the most expensive parts of the project (aside from your power-hungry GPUs!).<\/p>\n<p id=\"bc15\">All in all, ML\/DL field is a growing field and you have to keep your ears, eyes, and mind wide open. Don\u2019t just jump onto a shiny new technique just because a paper\/blog\/tutorial\/person\/YouTube video says it performs very well on a particular dataset. I have seen a lot of shiny new techniques come and go quite quickly. Always be aware of the fact that it is important to distinguish signal from noise!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>How to start Machine Learning and Deep Learning? Here, are curated list of some resources.&nbsp;Deep Learning definitely requires you to have a strong command of Linear Algebra, Differential Calculus and Vector Calculus, just to name a few. Be prepared to do all the Math. It can be a bit challenging but it will definitely be rewarding once you have gone through it and did your work. It would be hard for textbooks to capture the current state of Deep Learning since the field is moving at a very fast pace.<\/p>\n","protected":false},"author":324,"featured_media":2668,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[183],"tags":[97],"ppma_author":[2025],"class_list":["post-843","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-ml","tag-artificial-intelligence"],"authors":[{"term_id":2025,"user_id":324,"is_guest":0,"slug":"arkar-min-aung","display_name":"Arkar Aung","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/?s=96&d=mm&r=g","user_url":"","last_name":"Aung","first_name":"Arkar","job_title":"","description":"Arkar Min Aung is Fulbright Scholar, Deep Learning Engineer, and Software developer with interests in telematics and smart driving."}],"_links":{"self":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/843","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/users\/324"}],"replies":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/comments?post=843"}],"version-history":[{"count":2,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/843\/revisions"}],"predecessor-version":[{"id":28395,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/843\/revisions\/28395"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media\/2668"}],"wp:attachment":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media?parent=843"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/categories?post=843"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/tags?post=843"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=843"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}