{"id":22493,"date":"2020-12-11T08:16:17","date_gmt":"2020-12-11T08:16:17","guid":{"rendered":"https:\/\/www.experfy.com\/blog\/shortcut-learning-reason-ml-models-fail-practice\/"},"modified":"2023-09-21T17:57:35","modified_gmt":"2023-09-21T17:57:35","slug":"shortcut-learning-reason-ml-models-fail-practice","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/ai-ml\/shortcut-learning-reason-ml-models-fail-practice\/","title":{"rendered":"Shortcut Learning, The Reason ML Models Often Fail in Practice"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

TLDR, models always take the route of least effort.<\/p>\n\n

Training machine learning models is far from easy. In fact, the unaware data scientist might trip and fall in as many pitfalls as there are living AWS instances. The list is endless but divides itself nicely into two broad categories: underfitting, your model is bad, and overfitting, your model is still bad, but you think it isn\u2019t<\/em>. While overfitting can manifest itself in various ways, shortcut learning is a recurring flavor when dealing with custom datasets and novel problems. It affected me; it might be affecting you.<\/p>\n\n

Informally,\u00a0shortcut learning occurs whenever a model fits a problem on data not expected to be relevant or present, in general<\/em>.<\/strong><\/p>\n\n

A practical example is a dog\/cat classifier that, instead of properly recognizing dog- and cat-features, specializes in detecting leashes. Assuming leashes means dogs will likely do well most of the time, but leashes are not a general descriptor of dogness. That\u2019s lazy work!<\/p>\n\n

In other words, the model took a shortcut to solve the problem. <\/em>It cheated.<\/p>\n\n

Shortcut learning typically arises when there isn\u2019t enough data to force algorithms into learning the task properly<\/em>. In our dog\/cat example, most dog pictures likely included a leash while cat pictures didn\u2019t. Learning to detect leashes is a far simpler task than recognizing pets: they are simple, they contrast with the dog, and they are usually vibrant colored.<\/p>\n\n

The catch is that machine learning algorithms favor the route of least effort.<\/em><\/p>\n\n

Putting it another way, a machine learning algorithm will only learn what you want if that is the easiest thing it can do to maximize its metrics. As long as there are leash-like shortcuts, models will cheat.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\"Shortcut\t\t\t\t\t\t\t\t\t\t\t
Photo by\u00a0Mirko Sajkov<\/a>\u00a0on\u00a0Unsplash<\/a><\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Shortcut learning helps us realize that we don\u2019t give algorithms any real, meaningful instruction on what to learn<\/em>. We can\u2019t blame our tools for slacking \u2014 they just fit data, after all.<\/p>\n\n

From the above, it follows that we don\u2019t have any reliable way to teach algorithms how to tell background from the foreground<\/em>. If all dog pictures were outdoor and all cat ones were indoor, it wouldn\u2019t surprise me to see classifiers becoming experts at detecting sky and grass \u2014 not dogs and cats.<\/p>\n\n

These small observations carry a profound insight into machine learning’s current nature: it is still a highly random process<\/em>.<\/p>\n\n

To this date, the only sure-fire way to reduce the likelihood of shortcuts in your data is to\u2026 add more data!<\/p>\n\n

If we increase our dogs and cats dataset to include outdoor and indoor pictures of each kind and leash-less dogs and leashed cats, we might get a step closer to a true dog and cat classifier. We might get<\/em>. There is no guarantee there aren\u2019t any other shortcuts lingering around \u2014 you never know.<\/p>\n\n

Removing shortcuts is much harder than it seems. It is quite challenging to disentangle concepts by simply adding contrasting data. For instance, most cars are found outdoors; however, a car is a car regardless of its location. If we blindly search for car images, we are bound to get shortcuts such as road = car, asphalt = car, tire = car, etc.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

To better illustrate this, consider this simple custom dataset:<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\"Shortcut\t\t\t\t\t\t\t\t\t\t\t
Dataset provided by the Author<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

This eight-images dataset shows a single book and a single headset. While no one in their right mind would even consider this to be a proper dataset, let\u2019s give it the benefit of the doubt and wonder how a model would fit it.<\/p>\n\n

The book images have many straight lines, lots of white, lots of brown, and text, while the headset images have lots of blue, stripes, black, and curves. A model trained on such data would promptly classify any black book as a headset and any brown thing as a book. A CNN, in particular, would quickly fit local patterns such as text = books and stripes = headset.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

How can we solve this? More data!<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

After some coding, we got ourselves a Google Images crawler and downloaded a thousand book images and a thousand headset images. Let\u2019s inspect what Google has to offer regarding books <\/a>and headsets:<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\"Shortcut\t\t\t\t\t\t\t\t\t\t\t
First two rows of searching \u201cBook\u201d at Google Images<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\"First\t\t\t\t\t\t\t\t\t\t\t
First two rows of searching \u201cHeadset\u201d at Google Images<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

What we got is that most book images are of book stacks or bookshelves, which, by the way, are quite likely to be made of wood, and all headset images come from shopping sites; these are all illustrations, not real images. If we scroll down (a lot), we get pictures of people wearing headsets, opening an avenue for the algorithm to mistake faces for headsets.<\/p>\n\n

In other words,\u00a0blindly pouring data only made our problem worse<\/em>.<\/strong><\/p>\n\n

While all datasets mentioned so far are toy problems, these shortcut issues are more common than we would like to acknowledge. Shortcuts are especially frequent when constructing custom datasets, which is what many people do. Such datasets are typically small and contain data coming from only a handful of sources or were generated by just a couple of hardworking people.<\/p>\n\n

A practical example is AI applied to medicine. Building a medical imaging dataset is quite expensive, as you need everyone\u2019s permission, only doctors can label it, and some conditions are naturally rare. On top of that, images are of high resolution, and their content can be quite complex. It is rather easy for models to fit arbitrary structures instead of learning their intended task. Some images also contain metadata, such as the patient name, hospital logo, and time of day, all of which can become nasty shortcuts.<\/p>\n\n

Custom datasets involving humans, in particular, are quite prone to such issues as well. In many, there is a limited budget for actors and scenes, leading to poor variety. This setup is quite open for algorithms to overfit to specific actors or clothing elements rather than what they are doing.<\/p>\n\n

To better understand this phenomenon, we need a bit of formality\u2026<\/p>\n\n

Classification is a mapping from some data x <\/em>to a set of labels y, <\/em>or x \u2192 y<\/em>. For instance, if we want to detect persons in pictures, x <\/em>are pictures, and y is whether they have a person or not. In this framework, a classifier is a function that, given an x, <\/em>answers its y<\/em>. A good classifier will answer the right label most of the time; a bad classifier won\u2019t.<\/p>\n\n

Without any loss of generality, we might breakdown our x <\/em>elements as a \u201csignal,\u201d which is the real thing we want to track, and a set of \u201cnoises\u201d that tarnish it. Thus, x <\/em>can be rewritten as s + n\u2081 + n\u2082 + n\u2083 + \u2026<\/em><\/p>\n\n

In our original example, the signals are the dogs and cats, the things we really want to track, and the noises are everything else: the leashes, the background elements, the pet color, their pose, etc. By themselves, these noisy elements are harmless. Shortcut learning arises when these are simultaneously easily detectable and correlated with y. <\/em>Thus, detecting the noise becomes the route of least effort \u2014 the shortcut.<\/p>\n\n

This gets worse with powerful models, such as neural networks and support vector machines. These models\u2019 flexibility might backfire into learning non-trivial couplings of noises, such as \u201cleash or outdoor equals dog\u201d and \u201craised tail equals cat.\u201d Never underestimate a model’s ability to overfit.<\/em><\/p>\n\n

It is important to mention that our signal itself is also not a single beacon. Many pet breeds vary in color, and owners might style their pets to their hearts\u2019 content<\/em>. A classifier might fit some of these signals, but not all \u2014 or jointly fit both noise and signal elements.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\"Shortcut\t\t\t\t\t\t\t\t\t\t\t
Photo by\u00a0Ralph (Ravi) Kayden<\/a>\u00a0on\u00a0Unsplash<\/a><\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Apowerful concept in this analysis is the signal-to-noise-ratio<\/em>. As we add more images, some noises cancel out. For instance, as we added unleashed dogs to our data, detecting leashes became increasingly less rewarding. The more data we add, the less noise we get, and, gradually, our signal shall become the only reliable source for classification.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

At this point, we are ready to talk about addressing shortcut learning<\/em>. Without changing the problem, there are a couple of ways in which we can alleviate it:<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
  1. Add lots of data<\/strong>: as discussed, one effective way of raising the signal-to-noise-ratio is pouring more data, evening out the noise. For simple domains, however, blindly adding data might work against you. A safer alternative is to\u2026<\/li>
  2. Add the right data<\/strong>: when it’s expensive to find quality data, it helps to locate current shortcuts and aim for opposing data. In our example, that means searching for leashed cats and unconstrained dogs.<\/li>
  3. Include regularization<\/strong>: techniques such as dropout and weight decay aim at making the learning task artificially harder, forcing the algorithm to rely on robust signals rather than occasional hints. Similarly, you might\u2026<\/li>
  4. Use smaller, dumber models<\/strong>: tiny models cannot focus on as many properties as bigger ones; therefore, they are less prone to rely on spurious correlations and more likely to pay attention to the true signal.<\/li><\/ol>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t

    In this regard, it is imperative<\/em> to highlight that data augmentation is not an effective measure to address shortcut learning<\/strong>. The reason being that most operations are likely to either preserve or augment pre-existing shortcuts. A leash is still a leash even if flipped, rotated, zoomed, sheared, and hue-shifted. Cutout might help to some extent, but I wouldn\u2019t rely on it.<\/p>\n\n

    If you are able to change the problem formulation<\/em>, an effective way of avoiding shortcuts almost entirely<\/em> is recasting classification as detection or segmentation. In these more advanced formulations, you are explicitly telling the algorithm what to look for and requiring it to show its findings. While this approach effectively solves shortcut learning, annotating your existing data might prove to be prohibitive in most cases.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t\t\"The\t\t\t\t\t\t\t\t\t\t\t
    Photo by\u00a0Jeanie de Klerk<\/a>\u00a0on\u00a0Unsplash<\/a><\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t

    One aspect we haven\u2019t discussed so far is how to spot shortcut learning<\/em>. This is a tricky subject, as with detecting any form of overfitting; however, we can take some specific actions to uncover shortcuts or at least find evidence that something is wrong.<\/p>\n\n

    The direct approach is to run\u00a0interpretability models<\/a>. These algorithms try to explain which features of your data contribute the most to the models\u2019 outputs. Many of these techniques are model agnostic, meaning that they can be applied regardless of the model used, such as the\u00a0SHAP\u00a0<\/a>technique.<\/p>\n\n

    In the case of vision models, most interpretation tools highlight the specific areas that contributed the most to a specific answer. It takes some time to inspect all images, but it certainly pays off. You should look for the obvious: the area that contributes to \u201cdog\u201d should be the dog itself or some of its dog-features, such as its face, ears, and feet.<\/p>\n\n

    Occasional exceptions are OK, but you should move them to a separate folder for a second look. Inspecting all odd detections together might reveal interesting patterns that you wouldn\u2019t otherwise spot. I also recommend sorting your data by loss. You are bound to find odd entries and badly labeled data at both the low-loss and high-loss ends.<\/p>\n\n

    If you don\u2019t want to go through all this trouble, there are some automated ways of checking if something is odd. These methods won\u2019t generally tell you what is wrong but are sure to raise some eye browns if something is awry<\/em>.<\/p>\n\n

    A quick assessment is swapping your training\/test split. If you train-on-test and test-on-train, not much should change, apart from somewhat inferior results. If results are significantly lower (or higher!) or the training progresses at a particularly different pace, something might be off.<\/p>\n\n

    A more robust assessment is\u00a0cross-validation<\/a>. The general idea is to create several training-test splits out of the same dataset and train a model for each. In normal circumstances, all models should reach similar performance levels. If that\u2019s not the case, your model is not truly generalizing to unseen data.<\/p>\n\n

    Inthis article, we overviewed what shortcut learning is, why it is so hard to get rid of it, a couple of measures we can take to handle it, and, finally, how we can search for them directly and indirectly. If you are working with custom, in-house, datasets, I highly encourage you to go through some of the outlined steps and see if your models are really fitting to the intended signals and, if they aren\u2019t, uncover what they are really fitting to.<\/p>\n\n

    Throughout this piece, I focused on image classification. However, this is not strictly a vision issue. There can be shortcuts on text, graphs, and audio as well. Similar issues also arise in regression and reinforcement scenarios.<\/p>\n\n

    I hope this has been a pleasant read for you as it was for me to write. If you would like to read more on the topic, I highly recommend the recent work of Geirhos\u00a0et al.\u00a0<\/em>You can find it here.<\/a><\/p>\n\n

    Thanks for reading <\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"

    Training machine learning models is far from easy. Shortcut learning typically arises when there isn\u2019t enough data to force algorithms into learning the task properly.<\/p>\n","protected":false},"author":996,"featured_media":18148,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[183],"tags":[563,92,1110,1111],"ppma_author":[3859],"class_list":["post-22493","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-ml","tag-algorithms","tag-machine-learning","tag-ml-models","tag-shortcut-learning"],"authors":[{"term_id":3859,"user_id":996,"is_guest":0,"slug":"serpa","display_name":"Ygor Rebou\u00e7as Serpa","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/10c04385a4cc7e1aba3fc8d1561f4778f1cd025cd5a15a2fc39f55e0da7d24da?s=96&d=mm&r=g","user_url":"https:\/\/medium.com\/@ygorrebouasserpa%20","last_name":"Rebou\u00e7as Serpa","first_name":"Ygor","job_title":"","description":"Ygor Rebou\u00e7as Serpa is a Data Scientist and Game Developer. He is currently working on explainable AI models applied to healthcare."}],"_links":{"self":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/22493","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/users\/996"}],"replies":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/comments?post=22493"}],"version-history":[{"count":4,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/22493\/revisions"}],"predecessor-version":[{"id":33125,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/22493\/revisions\/33125"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media\/18148"}],"wp:attachment":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media?parent=22493"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/categories?post=22493"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/tags?post=22493"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=22493"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}