{"id":7877,"date":"2020-05-22T07:48:13","date_gmt":"2020-05-22T07:48:13","guid":{"rendered":"http:\/\/blog.experfy.com\/?p=7877"},"modified":"2023-12-11T15:32:32","modified_gmt":"2023-12-11T15:32:32","slug":"how-deep-learning-can-help-scientific-research","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/ai-ml\/how-deep-learning-can-help-scientific-research\/","title":{"rendered":"How Deep Learning Can Help Scientific Research"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Whether we take it for granted or not, deep learning algorithms have become an inseparable part of our daily lives. Personalized feeds, face and voice recognition, web search, smart speakers, digital assistants, email, and many other applications that we can\u2019t part ways with use\u00a0deep learning algorithms<\/a>\u00a0under the hood.<\/p>\n\n\n\n

But how effective is deep learning in scientific research, where problems are often much more complex than classifying an image and requirements are much more sensitive than recommending what to buy next?<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

To answer this question, former Google CEO Eric Schmidt and Google AI researcher Maithra Raghu have put together a comprehensive guide on the different deep learning techniques and their application to scientific research.<\/p>\n\n\n\n

\u201cThe amount of data collected in a wide array of scientific domains is dramatically increasing in both size and complexity,\u201d the authors write, adding that along with advances in machine learning, this rich corpus of data can provide \u201cmany exciting opportunities for deep learning applications in scientific settings.\u201d<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Titled \u201cA Survey of Deep Learning for Scientific Discovery<\/a>,\u201d their guide provides a very accessible overview of deep learning and\u00a0neural networks<\/a>\u00a0for scientists who aren\u2019t necessarily versed in the complex language of artificial intelligence algorithms.<\/p>\n\n\n\n

I strongly recommend reading the entire 48-page document and visit many of its references. But here are some key takeaways.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

\n

You don\u2019t necessarily need to do deep learning<\/h2>\n<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

With deep learning being all the rage, it\u2019s easy to be tempted to apply it to anything and everything. After all, the basic proposition is very attractive: It\u2019s an end-to-end AI model that takes a bunch of data, develops a mathematical representation, and performs complex classification and prediction tasks.<\/p>\n\n\n\n

Deep neural networks can tackle problems previously solved by other types of machine learning algorithms,<\/a> such as content recommendation or fraud detection. They can also handle problems that were traditionally difficult to handle with other machine learning techniques, including complex\u00a0computer vision<\/a>\u00a0and natural language processing (NLP) tasks.<\/p>\n\n\n\n

However, Schmidt and Raghu warn, when formulating a problem, it is important to consider whether deep learning provides the right set of tools to solve it. \u201cIn many settings, deep learning may not be the best technique to start with or best suited to the problem,\u201d they write.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

For many problems, simpler\u00a0machine learning algorithms<\/a>\u00a0often provide more efficient solutions. For instance, if you want to find the most relevant of a set of chemical characteristics of different substances, you might be better off using \u201cdimensionality reduction,\u201d a technique that can find the features that contribute most to outcomes.<\/p>\n\n\n\n

On the other hand, if you have limited data or if your data has been neatly arranged in a tabular format, you might want to consider trying a regression model before using neural networks. Neural networks usually (but not always) need lots of data. They are also difficult to interpret. In contrast, linear and logistic regression algorithms can provide more accurate results when the data is scarce, especially if the problem is linear in nature. Regression models also provide a clear mathematical equation with coefficients that explain the relevance of each feature in the dataset.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

Deep learning for image-related scientific tasks<\/h2>\n<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t“Covid-net coronavirus detection algorithm” \n
Scientists are using deep learning algorithms to detect signs of COVID-19 infection in chest x-rays of patients (source: COVID-Net<\/a>)\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\n

One area where deep learning algorithms have been very effective is the processing of visual data. The authors describe\u00a0convolutional neural networks<\/a>\u00a0as \u201cthe most well known family of neural networks\u201d and \u201cvery useful in working with any kind of image data.\u201d<\/p>\n\n\n\n

Aside from the\u00a0commercial and industrial applications<\/a>, CNNs have found their way into many scientific domains. One of the best known applications of convolutional neural networks is medical imaging analysis. There are already many deep learning algorithms that examine CT scans and x-rays and help in the diagnosis of diseases such as cancer. Recently, scientists have been using CNNs to\u00a0find symptoms of the novel coronavirus<\/a>\u00a0in chest x-rays.<\/p>\n\n\n\n

Some of the visual applications of deep learning are less known. For instance, neuroscientist are experimenting with pose-detection neural networks to\u00a0track the movements of animals<\/a>\u00a0and analyze their behavior.<\/p>\n\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

NLP technology can expand to other fields<\/h2>\n<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\n

Another area that has benefitted immensely from advances in deep learning algorithms is\u00a0natural language processing<\/a>. Recurrent neural networks, long short-term memory (LSTM) networks, and Transformers have proven to be especially good at performing language-related tasks such as translation and question-answering.<\/p>\n\n\n\n

To be clear, current AI algorithms process language in\u00a0fundamentally different\u2014and inferior\u2014ways\u00a0<\/a>than the human brain. Even the largest neural network will fail at some of the simplest tasks that a human child with a very rudimentary understanding of language can perform.<\/p>\n\n\n\n

This is because like all other types of neural networks, RNNs and Transformers are at their very core pattern-matching machines. They can find recurring patterns in sequences of data, whether it be text or any other kind of information. According to Schmidt and Raghu, these structures can be used in \u201cProblems where the data has a sequential nature (with different sequences of varying length), and prediction problems such as determining the next sequence token, transforming one sequence to another, or determining sequence similarities are important tasks.\u201d<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\n

While this scheme presents\u00a0limits in dealing with the abstract and implied meanings of language<\/a>, it has some very interesting applications in scientific research in areas such genomics and proteomics, where sequential structures play an important role.<\/p>\n\n\n\n

Transformers have proven to be especially efficient in scientific research. In\u00a0one recent project<\/a>, AI researchers used unsupervised learning to train a bidirectional Transformer on 86 billion amino acids across 250 million sequences spanning evolutionary diversity. \u201cThe resulting model maps raw sequences to representations of biological properties without labels or prior domain knowledge,\u201d the researchers write. This is an important step toward understanding protein sequences and extracting general and transferable information about proteins from raw sequences.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

\n

What if you don\u2019t have a lot of data?<\/h2>\n<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\n

One of the\u00a0main criticisms against deep learning<\/a>\u00a0is its need for vast amounts of training data. In many fields of science, there\u2019s not enough labeled data available. In others such as medicine, the data collection is prohibitively expensive and subject to the laws of handling sensitive personal information.<\/p>\n\n\n\n

Deep neural networks also consume\u00a0a lot of compute resources and electricity<\/a>\u00a0during training, requirements that many people and organizations can\u2019t meet.<\/p>\n\n\n\n

But not every deep learning model requires lots of training data. In the past few years, advances in\u00a0transfer learning<\/a>\u00a0have enabled many developers to create deep learning models without the need for a lot of data and computation resources. Transfer learning involves finetuning a pre-trained AI model for a new task. Transfer learning has had remarkable success in computer vision, and there are many freely available AI models that have already been trained on millions of examples.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

As long as the new problem is close enough to the domain of the base model and you have a decent set of examples, you\u2019ll have a reasonable chance of being able to finetune the AI model for the new task.<\/p>\n\n\n\n

\u201cTypically, performing transfer learning is an excellent way to start work on a new problem of interest. There is the benefit of using a well-tested, standard neural network architecture, aside from the knowledge reuse, stability and convergence boosts offered by pretrained weights,\u201d the authors write.<\/p>\n\n\n\n

Meanwhile, they also warn: \u201cNote however that the precise effects of transfer learning are not yet fully understood, and an active research area.\u201d<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\n\n

Another area that is worth watching in the coming months is\u00a0self-supervised learning<\/a>, a branch of artificial intelligence that can learn from raw data without the need for human-labeled examples. Self-supervised learning is still in a very preliminary stage, however, and also an active area of research.<\/p>\n\n\n\n

But an area that has already yielded result is generative models such as\u00a0generative adversarial networks<\/a>\u00a0(GAN). GANs can generate fake data that resembles their real counterparts. Perhaps they\u2019re best known for the natural-but-nonexistent human faces they can create. Artists are now\u00a0using GANs to generate art<\/a>\u00a0that can sell at stellar prices.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\n