{"id":1353,"date":"2019-02-15T10:32:05","date_gmt":"2019-02-15T10:32:05","guid":{"rendered":"http:\/\/kusuaks7\/?p=958"},"modified":"2023-09-19T12:07:59","modified_gmt":"2023-09-19T12:07:59","slug":"coding-deep-learning-for-beginners-start","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/ai-ml\/coding-deep-learning-for-beginners-start\/","title":{"rendered":"Coding Deep Learning for Beginners\u200a\u2014\u200aStart!"},"content":{"rendered":"

Ready to learn Machine Learning? Browse courses<\/a>\u00a0like\u00a0Machine Learning Foundations: Supervised Learning<\/a> developed by industry thought leaders and Experfy in Harvard Innovation Lab.<\/em><\/strong><\/p>\n

\n
\n
Intuition based series of articles about Neural Networks dedicated to programmers who want to understand basic math behind the code and non-programmers who want to know how to turn math into\u00a0code.<\/h5>\n<\/blockquote>\n

This is the 1st article of series \u201cCoding Deep Learning for Beginners<\/strong>\u201d.<\/p>\n<\/section>\n

\n

If you read this article I assume you\u00a0want to learn<\/em>\u00a0about one of the most promising technologies \u2014 Deep Learning<\/strong>. Statement\u00a0AI is a new electricity\u00a0<\/strong><\/a>becomes more and more popular lately. Scientists believe that as\u00a0Steam<\/em>Engine<\/em>, later\u00a0Electricity<\/em>\u00a0and finally\u00a0Electronics\u00a0<\/em>have\u00a0<\/em>totally changed the industry later Artificial Intelligence<\/em>\u00a0is next to transform it again. In a few years, basics of Machine Learning will become must-have skills for any developer. Even now, we can observe\u00a0increased popularity of programming languages\u00a0<\/a>that are used mainly in Machine Learning like Python and R.<\/p>\n

Technology that is capable of\u00a0magic<\/strong><\/h4>\n

In the last years, applications of Deep Learning made huge advancements in many domains arousing astonishment in people that didn\u2019t expect the technology and world to change so fast.<\/p>\n

Let\u2019s start from historical match between\u00a0super-computer AlphaGo\u00a0and one of the strongest Go players, 18-time world champion \u2014 Lee Sedol, in March 2016. The AI ended up victorious with the\u00a0result<\/a>\u00a0of 4 to 1. This match had a huge influence on Go community as AlphaGo invented completely new moves which made people try to understand, reproduce them and created totally new perspective on how to play the game. But that\u2019s not over, in 2017\u00a0DeepMind introduced AlphaGo Zero. The newer version of an already unbeatable machine\u00a0was able to learn everything without any starting data or human help. All that with computational power 4 times less\u00a0<\/strong>than it\u2019s predecessor!<\/p>\n

<\/canvas><\/figure>\n

AlphaGo versus Ke Jie in May 2017 (source: The Independent)<\/p>\n

Probably many of you have already heard about self-driving cars project that\u2019s being developed for a few years, by companies like\u00a0Waymo<\/a>\u00a0(Google),\u00a0Tesla<\/a>,\u00a0Toyota<\/a>,\u00a0Volvo<\/a>\u00a0and more. There are also\u00a0self-driving trucks\u00a0that are\u00a0already used on some highways in the US<\/a>. Many countries slowly prepare for the introduction of autonomous cars on their roads, yet their peak is predicted for the next decade.<\/p>\n

But how about autonomous flying car? Just recently Udacity announced their new Nanodegree programme where developers can learn how to become Flying Car Engineers and create autonomous flying cars!<\/p>\n

Flying Cars and Autonomous Flight | Udacity<\/strong>
\nMaster job-ready autonomous flight software engineering skills as you tackle advanced challenges, write real code for\u2026<\/em>www.udacity.com<\/a><\/p>\n

Lately thanks to improvement in AI speech recognition, voice interfaces like Google Home or Google Assistant become totally new development branch.<\/p>\n

\n

Google Advertisement on Google Assistant product.\u00a0https:\/\/youtu.be\/-qCanuYrR0g<\/a><\/p>\n

Future when AI will inform you to leave home earlier because of traffic, buy tickets to cinema, reschedule calendar meetings, control your home and more is closer than you think.<\/p>\n

And of course this list could be longer:\u00a0AI is capable of reproducing human speech with many dialects,\u00a0AI being better at diagnosing cancer than humans<\/a>,\u00a0AI generating new chapter of Harry Potter\u2026<\/p>\n

The key point in mentioning in all of that is making you understand that each of those inventions is using Deep Learning technology. To summarise it Deep Learning is currently excelling in tasks like:<\/p>\n

    \n
  1. Image recognition<\/li>\n
  2. Autonomous Vehicles<\/li>\n
  3. Games like Go,\u00a0Chess<\/a>,\u00a0Poker<\/a>, but lately also\u00a0computer games<\/a><\/li>\n
  4. Language Translation<\/a>\u00a0(but only a few languages)<\/li>\n
  5. Speech recognition<\/a><\/li>\n
  6. Analysis of handwritten texts<\/li>\n<\/ol>\n

    And this is only the beginning because technology gets democratized every day and as more people become capable of using it, the more research is being done and simple ideas tested.<\/p>\n

    So what is Deep Learning?<\/strong><\/h4>\n

    It\u2019s a subset of Machine Learning algorithms, based on learning data representations, called\u00a0Neural Networks<\/strong>. Basic idea is that\u00a0such an algorithm is being shown<\/strong>\u00a0a partial representation of reality in the form of\u00a0numerical data<\/strong>. During this process,\u00a0it\u2019s gaining experience and trying to create it\u2019s own understanding<\/strong>\u00a0of given data. That understanding has\u00a0hierarchical structure<\/strong>\u00a0as algorithm has\u00a0layers<\/strong>. First layer learns the simplest facts and is connected to the next layer that uses experiences from previous one to learn more complicated facts.\u00a0Number of layers is called the depth of the model<\/strong>. The more layers, the more complicated data representations the model can learn.<\/p>\n

    <\/canvas>
    \u00a0<\/figcaption><\/figure>\n

    Neural Network that is used for face detection. It learns hierarchy of representations: corners in first layer, eyes and ears in the second layer, and faces in the third layer (source: strong.io)<\/p>\n

    Is Deep Learning really new technology?<\/strong><\/h4>\n

    Some of you might think that Deep Learning is technology that was developed lately. That\u2019s not entirely true. Deep Learning had very rich history and had various names depending on philosophical viewpoint. People were\u00a0dreaming about intelligent machines over a hundred years ago before first mathematical concepts were built<\/a>. There have been three waves of development.<\/p>\n

    During the first wave, Deep Learning went by name\u00a0Cybernetics<\/a>. First predecessors of modern deep learning were linear models inspired by the study about the nervous system\u2014\u00a0Neuroscience<\/a>. The first\u00a0concept of the neuron (1943)<\/a>, the smallest piece of Neural Network, was proposed by McCulloch-Pitt that tried to implement brain function. A few years later Frank Rosenblatt turned that concept into the first trainable model \u2014Mark 1 Perceptron<\/a><\/p>\n

    <\/p>\n

    Mark 1 Perceptron (source: Wikipedia)<\/p>\n

    But people had problems to describe brain behaviors with theories available at that time. That\u2019s why interest in them decreased for the next 20 years.<\/p>\n

    The second wave started in the 80s and went by name\u00a0Connectionism<\/a>\u00a0but also term Neural Networks started to be used more often. The main idea was that neurons could achieve more intelligent behaviors when grouped together in large number. This concept was introduced by Hinton and is called\u00a0distributed representation (1986)<\/a>. It\u2019s still very central to today\u2019s Deep Learning. Another great accomplishment of a second wave was the invention of\u00a0back-propagation by Yann LeCun (1987)\u2014 core algorithm that is used until today for training Neural Network parameters. Also in year 1982\u00a0John Hopfield<\/a>\u00a0has invented Recurrent Neural Networks, which after additional introduction of\u00a0LSTM in 1997, are used today for language translation<\/a>. Those few years of big hype about Neural Networks has ended due large interest of the various investors which expectations towards implementing AI in products was not fulfilled.<\/p>\n

    <\/canvas><\/figure>\n

    Image of LSTM cell based Recurrent neural Network (source:\u00a0http:\/\/colah.github.io\/posts\/2015-08-Understanding-LSTMs\/<\/a>)<\/p>\n

    The third wave started in 2006. At that time computer became a more common thing that everyone could afford. Thanks to the various groups, e.g. gamers, has grown the market for powerful GPUs. Internet was available to everyone. Companies started paying more attention to analytics \u2014 gathering data in digital form. As a side effect researchers had more data, and computational power to perform experiments and validate theories. Consequently, there was another huge advancement, thanks to\u00a0Geoffrey E. Hinton that managed to train Neural Network with many layers<\/a>. From that moment a lot of different proposals for Neural Network architectures with many layers started to appear. Scientists referred to the number of layers in Neural Network as of \u201cdepth\u201d \u2014 the more layers it had the deeper it was. Very important occurring was usage of Convolutional Neural Network\u00a0AlexNet<\/a>\u00a0in image classification contest ILSVRC-2012. It has revolutionized a lot of industries by providing them with reliable image detection mechanism \u2014 allowing many machines to see e.g. autonomous cars.<\/p>\n

    <\/canvas><\/figure>\n

    Structure of AlexNet CNN (source: Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton, \u201cImageNet Classification with Deep Convolutional Neural Networks\u201d, 2012)<\/p>\n

    In 2014, Ian Goodfellow has introduced new type of Neural Networks called\u00a0Generative Adversarial Networks<\/a>. In this architecture two Neural Networks were competing against each other. First network tried to mimic some distribution of data. The role of a second network was to tell if the data it received is fake or real. The goal of first network is to trick the second network. Competition lead to increase in performance of first network and made generation of any kind of data \u2014 images, music, text, speech possible.<\/p>\n

    <\/canvas><\/figure>\n

    GAN used to transfer style of one image into another (source:\u00a0https:\/\/github.com\/lengstrom\/fast-style-transfer<\/a>)<\/p>\n

    And this is it I guess. Third wave continues until today and it depends on us how far it can go!<\/p>\n

    Why am I creating this series of articles?<\/strong><\/h4>\n

    I am really passionate about Machine Learning and especially Deep Learning. My dream is to become Machine Learning Expert \u2014 person who work with people to solve problems and democratize the knowledge. I am working hard every day to reach that goal and this blog is a part of it. So study with me!<\/p>\n

    In my opinion, the biggest problem with access to this technology is that it was developed at universities and in laboratories by highly qualified Ph. D scientists and still partially stays there. It\u2019s understandable as everything is strongly based on\u00a0Linear Algebra<\/a>,\u00a0Probability<\/a>\u00a0and\u00a0Information Theory,\u00a0<\/a>Numerical Computing<\/a>. But\u00a0in order to become a driver, you don\u2019t need to know the engine<\/strong>\u00a0right? There is still conviction that in order to work in this field you need to be Ph. D but\u00a0it is starting to change in terms of Software Engineering<\/a>.<\/p>\n

    Demand for people with those skills will become so big it will simply become impossible for everyone to have Ph. D title. That\u2019s why in order to make people use it, there must be someone who can translate that to others while skipping complicated proofs, scientific notation and adding more intuition.<\/p>\n

    What I hope to show to\u00a0you<\/strong><\/h4>\n

    My goal is to provide strong understanding of most popular topics related to Deep Learning. I don\u2019t want to be protective when it comes to picking content \u2014 I want to show you even more complicated stuff and at the same time, do my best to provide you with intuition to grasp it. My main priority is to allow you understand to how those algorithms work and teach you how to code them from scratch. Like Mark Daoust (Developer Programs Engineer for TensorFlow) once said to me:<\/p>\n

    Everyone should code Neural Network from scratch once\u2026 but only once\u2026<\/p><\/blockquote>\n

    So there will be a lot of code that I plan to carefully explain. Among the topics, you can expect mini-projects where I will show you how to use what we\u2019ve learned.\u00a0It\u2019s really important for knowledge to be followed by practice<\/strong>.<\/p>\n

    The approach will be bottom-up then:<\/p>\n