{"id":22545,"date":"2021-01-06T10:36:42","date_gmt":"2021-01-06T10:36:42","guid":{"rendered":"https:\/\/www.experfy.com\/blog\/underspecification-dangerously-problem-facing-machine-learning\/"},"modified":"2023-09-13T11:45:20","modified_gmt":"2023-09-13T11:45:20","slug":"underspecification-dangerously-problem-facing-machine-learning","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/ai-ml\/underspecification-dangerously-problem-facing-machine-learning\/","title":{"rendered":"Underspecification: The Dangerously Underdiscussed Problem Facing Machine Learning"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

good model loss \u2260 good learning<\/p>\n\n

All machine learning models originate in the computer lab. They\u2019re initialized, trained, tested, redesigned, trained again, fine-tuned, tested yet again before they are deployed.<\/p>\n\n

Afterwards, they fulfill their duty in epidemiological modelling, stock trading, shopping item recommendation, and cyber attack detection, among many other purposes. Unfortunately, success in the lab may not always mean success in the real world \u2014 even if the model does well on the test data.<\/p>\n\n

This big problem \u2014 that the machine learning models<\/a> being developed on the computer to serve a purpose in the real world can often crash \u2014 has had little research or discussion. Usually, the solution is to assume that the data given to the model was not representative, or something similar.<\/p>\n\n

A valid solution as that may be, recently in November of 2020 a group of several researchers proposed a more broad explanation for this phenomenon: underspecification. While underspecification certainly has been explored deeply, the researchers write, \u201cits implications for the gap between iid (model training \u2018in the lab\u2019) and application-specific generalization are neglected.\u201d<\/p>\n\n

Take this example \u2014 we\u2019re trying to use machine learning to find the future maximum number of birds there will be in an exhibit at the zoo.<\/p>\n\n

The first step is to create a model modelling the number of birds were on the zoo x<\/em> years since the exhibit opened. The model our algorithm derives is a quartic one, with three extrema. The number of birds in the exhibit steadily rises through reproduction, then suddenly plummets (perhaps a disease).<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\"Underspecification:\t\t\t\t\t\t\t\t\t\t\t
Created by author.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

The next step is to find the maximum value of this model we\u2019ve created. We\u2019ll create an optimizer, which initializes at a random point and uses derivatives to find in which direction it should move.<\/p>\n\n

For instance, in this case the initialized optimizer can move left or right: if it moves left, the number of birds goes up; if it moves right, the number of birds goes down. Since the optimizer\u2019s goal is to find the maximum number of birds, it\u2019ll move to the left.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\"Underspecification:\t\t\t\t\t\t\t\t\t\t\t
This is oversimplifying the actual process of optimization. However, the point is still there: random initialization can determine a lot in optimization. Created by author.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Viola! Our crafty model has found the maximum number of birds there will be in the zoo exhibit will be reached 1.948 years before the exhibit opened.<\/p>\n\n

Obviously, the model has come to a false conclusion. What went wrong?<\/p>\n\n

This is an easy question to answer. We didn\u2019t specify the actual domain of the phenomena we\u2019re trying to model. We should have set some sort of constraint in the code of our model such that the optimizer searches only valid years.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
optimizer.find_maximum(search_range=\"0<=x\")<\/pre>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
Using this would have made the model we created better model the phenomena of bird population growth (and sudden decline). By specifying these limits, we would be able to eliminate answers that are the product of artificial representations, like an object existing before its creation.<\/p>\n\n
In this case, it\u2019s fairly easy to see that the answer the model spits out is patently wrong, and to slap a limitation on it so it behaves. However, this easiness of fixing the model is accompanied, not by coincidence, with the fact that it can\u2019t really do much. The model takes in one input \u2014 the year \u2014 and predicts based on a polynomial the population.<\/p>\n\n
What\u2019s harder \u2014 a lot more harder \u2014 is to do the same thing with massive neural networks, which are the most prone to underspecification. Furthermore, these massive neural networks are often used to model phenomena a lot more complex than birds in a zoo, like the behavior patterns of millions of users or the distinguishing features of thousands of faces.<\/p>\n\n
So, we have a problem (usually, finding the best parameters for a neural network), and multiple<\/em> solutions (combinations of parameters for a neural network). That is, they are parameters that can achieve similarly good loss, both in training and validation.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
Underspecification becomes an issue when the problem has only one solution, but the method through which we find solutions yields multiple.<\/p><\/blockquote>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
There\u2019s one true solution that models the phenomena desired, but we can\u2019t put specifications on the solution-finding process (because it\u2019s too complicated), so we find multiple that all look great by our metric.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\"Underspecification:\t\t\t\t\t\t\t\t\t\t\t
\u201cexists before creation\u201d is used in reference to solutions that technically work in the modelling landscape, but not in the actual phenomena it is intended for use in. Created by author.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
Let\u2019s consider a neural network trained to model the best video to recommend to a user. It takes in inputs about the user\u2019s behavior, and outputs the probability from zero to one that they would enjoy watching a certain video, for tens of thousands of videos. There\u2019s likely only one set of weights that truly captures the thought process of a user as they watch a video, but many weights that can artificially connect input to target quite well.<\/p>\n\n
Think of it as answering 1 + 1 = ?<\/code>. The natural way is to understand that the notation x + y<\/code>means \u201cadd the two numbers\u201d and to answer 2<\/code>. An artificial way to answer is to say that x + y<\/code> means (2x)\/y<\/code>. Indeed, this definition satisfies 1+1=2<\/code>. But it doesn\u2019t capture the phenomena intended, and doesn\u2019t work as well for all<\/em> situations or variants of the phenomena.<\/p>\n\n
(With something as complex as a massive neural network, it\u2019s not infeasible at all to expect it to dream up as solution as contrived as (2x)\/y<\/code>, even with measures like regularization.)<\/p>\n\n
Other solutions may look good in the lab, because they\u2019re artificial solutions, like the maximum number of birds existing 1.948 years before they were even put in the exhibit. However, when they\u2019re deployed in the real world, they won\u2019t perform as well, because they haven\u2019t actually learned<\/em> the mechanics of the phenomena as completely as another solution. They might still be parsing (x+y+2)\/2<\/code> instead of simply adding the two numbers.<\/p>\n\n
Building upon this \u2014 any specification is a condition or fixing of parameters. A model with many fixed parameters will have low underspecification. However, large models with many trainable parameters, like deep neural networks, are especially prone to underspecification.<\/p>\n\n
Further, random initialization plays a problem in underspecification. Because there are so many solutions that look good to the optimizer, and the optimizer can\u2019t tell which ones actually measure the phenomenon or not, we can\u2019t really tell if a trained model really does understand the data. Hence, something as arbitrary as a random seed can be the difference in deployment success.<\/p>\n\n
This is why multiple models can be trained and evaluated on the same data and yield the same loss, while performing very different in deployment.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
Underspecification occurring in ML pipelines can block the reliability of models that behave as expected in deployment.<\/p><\/blockquote>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
Underspecification occurs in many facets of machine learning. The researchers that introduced underspecification in this context ran \u2018stress tests\u2019 \u2014 probing the output based on a carefully designed input to measure the model\u2019s biases \u2014 on a variety of models and contexts.<\/p>\n\n
From genomics to epidemiology, and small neural networks to deep convolutional networks to natural language processing, underspecification applies. This has big implications for how we view learning.<\/p>\n\n
Are natural language processing systems actually<\/em> learning general linguistic principles like grammar, or are they taking artificial shortcuts? Results of stress tests on these systems, along with a large body of research, cast doubt on the idea that our deepest NLP models are actually learning<\/em> the language.<\/p>\n\n
There is an interest in creating models that understand the human language if their purpose is to serve humans. A NLP model that relies on linguistic shortcut and not linguistic understanding to perform a task might work nicely in a lab, but as soon as it is deployed it will have trouble.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\"Underspecification:\t\t\t\t\t\t\t\t\t\t\t
Created by author.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
We\u2019ve always known that machine learning algorithms were lazy, but we haven\u2019t considered them essentially lying to us. Since often phenomena in neural networks is answered with \u201cit\u2019s a black box\u201d, we haven\u2019t explored it to much at all.<\/p>\n\n
Underspecification in this respect is relatively knew, and very important. How to solve it is an important question not only to the practical problem of making model deployment more stable, but how to make models actually learn the mechanics of the data instead of taking shortcuts or generating nonsense artificial solutions.<\/p>\n\n
Going forward, this will become an important idea to investigate and even \u2014 perhaps \u2014 to solve.<\/p>\n\n
Read the paper that introduced underspecification in relation to model deployment\u00a0here<\/a>. If you\u2019re interested in this topic, you may also like some other articles in a similar vein:<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"
Underspecification facing machine learning is relatively new, and very important. How to solve it is an important question not only to the practical problem of making model deployment more stable, but how to make models actually learn the mechanics of the data instead of taking shortcuts or generating nonsense artificial solutions.<\/p>\n","protected":false},"author":884,"featured_media":18368,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[183],"tags":[97,92,1029,1217],"ppma_author":[3782],"class_list":["post-22545","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-ml","tag-artificial-intelligence","tag-machine-learning","tag-model-deployment","tag-underspecification"],"authors":[{"term_id":3782,"user_id":884,"is_guest":0,"slug":"andre-ye","display_name":"Andre Ye","avatar_url":"https:\/\/www.experfy.com\/blog\/wp-content\/uploads\/2020\/08\/Andre-Ye-150x150.jpg","user_url":"https:\/\/www.critiq.tech\/","last_name":"Ye","first_name":"Andre","job_title":"","description":"Andre Ye is Cofounder at Critiq, and Editor and Writer at Medium"}],"_links":{"self":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/22545","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/users\/884"}],"replies":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/comments?post=22545"}],"version-history":[{"count":4,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/22545\/revisions"}],"predecessor-version":[{"id":32844,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/22545\/revisions\/32844"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media\/18368"}],"wp:attachment":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media?parent=22545"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/categories?post=22545"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/tags?post=22545"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=22545"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}