{"id":1036,"date":"2018-12-19T02:12:49","date_gmt":"2018-12-18T23:12:49","guid":{"rendered":"http:\/\/kusuaks7\/?p=641"},"modified":"2021-12-15T02:57:57","modified_gmt":"2021-12-15T02:57:57","slug":"physics-guided-neural-networks-pgnns","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/ai-ml\/physics-guided-neural-networks-pgnns\/","title":{"rendered":"Physics-guided Neural Networks (PGNNs)"},"content":{"rendered":"

Ready to learn Machine Learning? Browse<\/em><\/strong> Machine Learning Training and Certification courses<\/a> developed by industry thought leaders and Experfy in Harvard Innovation Lab.<\/em><\/strong><\/p>\n

\n

Physics-based models are at the heart of today\u2019s technology and science. Over recent years, data-driven models started providing an alternative approach and outperformed physics-driven models in many tasks. Even so, they are data hungry, their inferences could be hard to explain and generalization remains to be a challenge. Combining data and physics could reveal the best of both\u00a0worlds.<\/p>\n

When machine learning algorithms are\u00a0learning<\/em>, they are actually searching for a solution in the hypothesis space you defined by your choice of algorithm, architecture, and configuration. Hypothesis space could be quite large even for a fairly simple algorithm. Data is the only guide we use to look for a solution in this huge space. What if we can use our knowledge of the world \u2014 for example, physics\u2014 together with data to guide this search?<\/p>\n

This is what Karpatne et al. explained in their paper\u00a0Physics-guided Neural Networks (PGNN): An Application in Lake Temperature Modeling<\/a>.\u00a0<\/em>In this post, I will explain why this idea is crucial and I will also describe how they did it by summarizing the paper.<\/p>\n<\/section>\n

\n
\n

Imagine you have sent your alien \ud83d\udc7d friend (optimization algorithm) to a supermarket (hypothesis space) to buy your favorite cheese (solution). The only clue she has is the picture of the cheese (data) you gave her. Since she lacks the preconceptions we have about supermarkets she will have a hard time finding the cheese. She might wander around cosmetics, cleaning supplies before even reaching the food section.<\/p>\n

<\/canvas><\/figure>\n

Finally reached the food section (courtesy of\u00a0Dominic L.\u00a0Garcia<\/strong>).<\/p>\n

This is similar to how machine learning optimization algorithms like gradient descent look for a hypothesis. Data is the only guide. Ideally, this should work just fine but most of the time the data is noisy or not enough. Incorporating our knowledge into the optimization algorithms \u2014 search for the cheese in the food section, don\u2019t even look at the cosmetics \u2014 could alleviate these challenges. Let\u2019s see next, how we can actually do this.<\/p>\n

How to guide an ML algorithm with\u00a0physics<\/h4>\n

Now, let me summarize how the authors utilize physics to guide a machine learning model. They present two approaches for this: (1) using physics theory, they calculate additional features (feature engineering) to feed into the model along with the measurements and (2) they add a physical inconsistency term to the loss function in order to punish physically inconsistent predictions.<\/p>\n

<\/p>\n

\u00a0(1) Feature engineering using a physics-based model<\/p>\n

<\/p>\n

(2) Data + Physics driven loss\u00a0function<\/p>\n

The first approach, feature engineering, is extensively used in machine learning. The second approach, however, is compelling. Very much like adding a regularization term to punish overfitting, they add a physical inconsistency term to the loss function. Hence, with this new term, the optimization algorithm should also take care of minimizing physically inconsistent results.<\/p>\n

In the paper, Karpatne et al. combined these two approaches with a neural network and demonstrated an algorithm they call physics-guided neural network (PGNN). There are two main advantages PGNNs could provide:<\/p>\n