{"id":1380,"date":"2019-02-15T10:32:06","date_gmt":"2019-02-15T10:32:06","guid":{"rendered":"http:\/\/kusuaks7\/?p=985"},"modified":"2023-07-28T08:44:42","modified_gmt":"2023-07-28T08:44:42","slug":"the-limits-and-challenges-of-deep-learning","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/ai-ml\/the-limits-and-challenges-of-deep-learning\/","title":{"rendered":"The limits and challenges of deep learning"},"content":{"rendered":"

Ready to learn Machine Learning? Browse courses<\/a>\u00a0like\u00a0Machine Learning Foundations: Supervised Learning<\/a> developed by industry thought leaders and Experfy in Harvard Innovation Lab.<\/em><\/strong><\/p>\n

\"neurons\"<\/p>\n

Deep learning, the spearhead of artificial intelligence, is perhaps one of the most exciting technologies of the decade. It has already made inroads in fields such as recognizing speech or detecting cancer, domains that were previously closed or scarcely available to traditional software models.<\/p>\n

Deep learning is often compared to the mechanisms that underlie the human mind, and some experts believe that it will continue to advance at an accelerating pace and conquer many more domains. In some cases, there\u2019s fear that deep learning might threaten the very social and economic fabrics that hold our societies together, by either driving humans into\u00a0unemployment<\/a>\u00a0or\u00a0slavery<\/a>.<\/p>\n

There\u2019s no doubt that machine learning and deep learning are super-efficient for many tasks. However, they\u2019re not a silver bullet that will solve all problems and override all previous technologies. Beyond the hype surrounding deep learning, in many cases, its distinct limits and challenges prevent it from competing with the mind of a human child.<\/p>\n

Here\u2019s an example. I played Mario Bros on the legendary NES console for the first time when I was six years old. That first experience, which didn\u2019t last more than a couple of hours, introduced me to the concept of platform games. After that, I could quickly apply the same rules to other games such as Prince of Persia, Sonic the Hedgehog, Crash Bandicoot and Donkey Kong Country. I also had no problem importing my previously earned knowledge to the 3D versions of those games when they made their appearance in the mid-90s.<\/p>\n

Meanwhile, I had the inherent power of porting my real-world experiences to the world of gaming. I immediately knew that I had to jump over pits, and that the plants that had sharp teeth would hurt Mario if he ran into them.<\/p>\n

By all standards, I was a person of average intelligence, and not even a very good gamer. This is the process that nearly every kid of my generation went through. However, even this simplest of feats proves to be a \u201cdeep\u201d challenge for deep learning algorithms. The smartest game-playing algorithms have to learn every new game from scratch.<\/p>\n

We humans can learn abstract, broad relationships between different concepts and make decisions with little information. In contrast, deep learning algorithms are narrow in their capabilities and need precise information\u2014lots of it\u2014to do their job.<\/p>\n

In a recent paper called \u201cDeep Learning: A Critical Appraisal<\/a>,\u201d Gary Marcus, the former head of AI at Uber and a professor at New York University, details the limits and challenges that deep learning faces. While I suggest you read the entire paper, here\u2019s a brief summary of the important points he raised.<\/p>\n

How deep learning works<\/h2>\n

Deep learning is a technique for classifying information through layered neural networks, a crude imitation of how the human brain works. Neural networks have a set of input units, where raw data is fed. This can be pictures, or sound samples, or written text. The inputs are then mapped to the output nodes, which determine the category to which the input information belongs. For instance, it can determine that the fed picture contains a cat, or that the small sound sample was the word \u201cHello.\u201d<\/p>\n

Deep neural networks, which power deep learning algorithms, have several hidden layers between the input and output nodes, making them capable of making much more complicated classifications of data.<\/p>\n

\"deep
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 Credit:\u00a0Gary Marcus<\/a><\/figcaption><\/figure>\n

Deep learning algorithms need to be trained with large sets of labelled data. This means that, for instance, you have to give it thousands of pictures of cats before it can start classifying new cat pictures with relative accuracy. The larger the training data set, the better the performance of the algorithm. Big tech companies are\u00a0vying to amass more and more data<\/a>\u00a0and are willing to offer their services for free in exchange for access to user data. The more classified information they have, the better they\u2019ll be able to train their deep learning algorithms. This will in turn make their services more efficient than those of their competitors and bring them more customers (some of whom will pay for their premium services).<\/p>\n

Deep learning is data hungry<\/h2>\n

\u201cIn a world with infinite data, and infinite computational resources, there might be little need for any other technique,\u201d Marcus says in his paper.<\/p>\n

And therein lies the problem, because we don\u2019t live in such a world.<\/p>\n

You can never give every possible labelled sample of a problem space to a deep learning algorithm. Therefore, it will have to generalize or interpolate between its previous samples in order to classify data it has never seen before such as a new image or sound that\u2019s not contained in its dataset.<\/p>\n

\u201cDeep learning currently lacks a mechanism for learning abstractions through explicit, verbal definition, and works best when there are thousands, millions or even billions of training examples,\u201d says Marcus.<\/p>\n

\"Big<\/p>\n

So what happens when deep learning algorithm doesn\u2019t have enough quality training data? It can fail spectacularly, such as mistaking a\u00a0rifle for a helicopter<\/a>, or\u00a0humans for gorillas<\/a>.<\/p>\n

The heavy reliance on precise and abundance of data also makes deep learning algorithms vulnerable to spoofing. \u201cDeep learning systems are quite good at some large fraction of a given domain, yet easily fooled,\u201d Marcus says.<\/p>\n

Testament to the fact are many crazy stories such as deep learning algorithms mistaking\u00a0stop signs for speed limit signs<\/a>\u00a0with a little defacing, or British police software not being able to\u00a0distinguish sand dunes from nudes<\/a>.<\/p>\n

Deep learning is shallow<\/h2>\n

Another problem with deep learning algorithms is that they\u2019re very good at mapping inputs to outputs but not so much at understanding the context of the data they\u2019re handling. In fact, the word \u201cdeep\u201d in deep learning is much more a reference to the architecture of the technology and the number of hidden layers it contains rather than an allusion to its deep understanding of what it does. \u201cThe representations acquired by such networks don\u2019t, for example, naturally apply to abstract concepts like \u2018justice,\u2019 \u2018democracy\u2019 or \u2018meddling,\u2019\u201d Marcus says.<\/p>\n

Returning to the gaming example we visited at the beginning of the article, deep learning algorithms can become very good at playing games, and they can eventually beat the best human players in both video and board games. Just look at the crazy way this AI is playing Mario:<\/p>\n

However, this doesn\u2019t mean that the AI algorithm has the same understanding as humans in the different elements of the game. It has learned through trial and error that making those specific moves will prevent it from losing. For instance, if you look at\u00a02:26<\/a>\u00a0where a baseball is thrown at Mario, the most novice gamer would know that they would have to jump. But the AI keeps on running until it gets hit in the back.<\/p>\n

Moreover, if you give that same algorithm a new game, such as Super Mario 64 or Mega Man, it will have to learn everything anew.<\/p>\n

Marcus\u2019 paper refers to Google DeepMind\u2019s\u00a0mastering of the Atari game Breakout<\/a>\u00a0as an example. According to the designers of the algorithm, the game realized after 240 minutes that the best way to beat the game is to dig a tunnel in the wall. But it neither knows what a tunnel or a wall is. It has just learned through millions of trials and errors that that specific way to play the game will yield the most points in the shortest time possible.<\/p>\n

Deep learning is opaque<\/h2>\n

While decisions made by rule-based software can be traced back to the last if and else, the same can\u2019t be said about machine learning and deep learning algorithms. This lack of transparency in deep learning is what we call\u00a0the \u201cblack box\u201d problem<\/a>. Deep learning algorithms sift through millions of data points to find patterns and correlations that often go unnoticed to human experts. The decision they make based on these findings often confound even the engineers who created them.<\/p>\n

This might not be a problem when deep learning is performing a trivial task where a wrong decision will cause little or no damage. But when it\u2019s deciding the fate of a defendant in court or the medical treatment of a patient, mistakes can have more serious repercussions.<\/p>\n

\u201cThe transparency issue, as yet unsolved, is a potential liability when using deep learning for problem domains like financial trades or medical diagnosis, in which human users might like to understand how a given system made a given decision,\u201d Marcus says in his paper.<\/p>\n

\"fog<\/p>\n

Marcus also points to algorithmic bias as\u00a0one of the problems<\/a>\u00a0stemming from the opacity of deep learning algorithms. Machine learning algorithms often inherit the biases of the training data the ingest, such as preferring to show\u00a0higher paying job ads<\/a>\u00a0to men rather than women, or preferring white skin over dark in\u00a0adjudicating beauty contests<\/a>. These problems are hard to debug in development phase and often result in controversial news headlines when the deep learning\u2013powered software go into production.<\/p>\n

Is deep learning doomed to fail?<\/h2>\n

Certainly not. But it is bound for a reality check. \u201cIn general, deep learning is a perfectly fine way of optimizing a complex system for representing a mapping between inputs and outputs, given a sufficiently large data set,\u201d Marcus observes.<\/p>\n

Deep learning must be acknowledged for what it is, a highly efficient technique for solving classification problems, which will perform well when it has enough training data and a test set that closely resembles the training data set.<\/p>\n

But it\u2019s not a magic wand. If you don\u2019t have enough training data, or when your test data differs greatly from your training data, or when you\u2019re not solving a classification problem, then \u201cdeep learning becomes a square peg slammed into a round hole, a crude approximation when there must be a solution elsewhere,\u201d in Marcus\u2019 words.<\/p>\n

Marcus also suggests in his paper that deep learning has to be combined with other technologies such as plain-old rule-based programming and other AI techniques such as reinforcement learning. Other experts such as Starmind\u2019s Pascal Kaufmann propose neuroscience as the key to creating real AI that will be able to achieve\u00a0human-like problem solving<\/a>.<\/p>\n

\u201cDeep learning is not likely to disappear, nor should it,\u201d Marcus says. \u201cBut five years into the field\u2019s resurgence seems like a good moment for a critical reflection, on what deep learning has and has not been able to achieve.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"

Deep learning, the spearhead of artificial intelligence, is perhaps one of the most exciting technologies of the decade. There’s no doubt that machine learning and deep learning are super-efficient for many tasks. Deep learning is often compared to the mechanisms that underlie the human mind. However, they’re not a silver bullet that will solve all problems and override all previous technologies. Beyond the hype surrounding deep learning, in many cases, its distinct limits and challenges prevent it from competing with the mind of a human child.<\/p>\n","protected":false},"author":109,"featured_media":3196,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[183],"tags":[97],"ppma_author":[1946],"class_list":["post-1380","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-ml","tag-artificial-intelligence"],"authors":[{"term_id":1946,"user_id":109,"is_guest":0,"slug":"ben-dickson","display_name":"Ben Dickson","avatar_url":"https:\/\/www.experfy.com\/blog\/wp-content\/uploads\/2020\/04\/medium_8aaf6bea-c4c1-455f-8156-8007d70910f8-150x150.jpg","user_url":"https:\/\/bdtechtalks.com\/","last_name":"Dickson","first_name":"Ben","job_title":"","description":"Ben Dickson is an experienced software engineer and tech blogger. He contributes regularly to major tech websites such as the Next Web, the Daily Dot, PCMag.com, Cointelegraph, VentureBeat, International Business Times UK, and The Huffington Post."}],"_links":{"self":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/1380","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/users\/109"}],"replies":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/comments?post=1380"}],"version-history":[{"count":3,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/1380\/revisions"}],"predecessor-version":[{"id":29724,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/1380\/revisions\/29724"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media\/3196"}],"wp:attachment":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media?parent=1380"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/categories?post=1380"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/tags?post=1380"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=1380"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}