{"id":22612,"date":"2021-02-09T10:08:03","date_gmt":"2021-02-09T10:08:03","guid":{"rendered":"https:\/\/www.experfy.com\/blog\/geometric-ml-real-fundamental-sciences\/"},"modified":"2023-09-05T07:48:38","modified_gmt":"2023-09-05T07:48:38","slug":"geometric-ml-real-fundamental-sciences","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/ai-ml\/geometric-ml-real-fundamental-sciences\/","title":{"rendered":"Geometric ML Becomes Real In Fundamental Sciences"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Among many papers on Geometric and Graph ML, applications in biochemistry, drug design, and structural biology shone in 2020. Perhaps for the first time, we are finally starting to see the real impact of these machine learning techniques in fundamental science. In this post, I highlight three papers that excited me the most in the past year (disclaimer: I am a co-author of one of them).<\/p>\n

J. M. Stokes\u00a0et al.<\/em>,\u00a0<\/strong>A deep learning approach to antibiotic discovery<\/strong><\/a>\u00a0(2020)\u00a0Cell<\/em>\u00a0180(4):688\u2013702.<\/strong><\/p>\n

What? <\/strong>A graph neural network-based deep learning pipeline for the discovery of new antibiotic drugs.<\/p>\n

How?\u00a0<\/strong>A graph neural network was trained to predict the growth inhibition of the bacterium\u00a0Escherichia coli<\/em><\/a>\u00a0on a dataset of >2000 molecules (including approved antibiotic drugs as well as natural compounds from the plant and animal kingdoms) with known antibacterial activity. The prediction is based on the molecular graph only and does not rely on any side information such as the drug mechanism of action.<\/p>\n

The\u00a0trained model was then applied to the Drug Repurposing Hub, containing around 6000 molecules of drugs under investigation. The top 100 candidates were selected for lab testing. Surprisingly, an experimental anti-diabetic drug\u00a0halicin<\/a>\u00a0turned out to be a potent antibiotic that showed activity against multiple drug-resistant pathogens in test mice. The graph neural network apparently has good generalisation capabilities, as the halicin molecule does not resemble traditional antibiotics. Yet, it is not completely clear whether its predictive capability boils down to predicting a simple pattern responsible for the antibiotic action hinted in the paper (depolarisation of the cellular membrane).<\/p>\n

Additional experiments were carried out to screen more than 100 million molecular structures in the\u00a0ZINC15 database<\/a>, a curated collection of commercially available chemical compounds prepared especially for virtual screening and routinely used by drug designers. Among the shortlisted compounds, physical tests identified 8 with antibacterial activity, of which two were shown to have potent activity against a broad range of pathogens.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\"Geometric\t\t\t\t\t\t\t\t\t\t\t
One of the big challenges in drug discovery is that the search space is extremely large whereas only a few molecules can be tested in the lab. Graph neural networks applied to molecular graphs are used to predict the properties of molecules, allowing to do a virtual screening of candidate drugs.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Why it matters?<\/strong>\u00a0One of the key challenges in drug discovery is the vast search space, estimated to contain over 10\u2076\u2070 molecules. Since only a tiny fraction of these molecules can be tested in the lab, it is paramount to select promising candidates. Doing this computationally is called \u201cvirtual screening\u201d. While ML methods have been used in the past for virtual screening of molecules and, more broadly, to assist\u00a0different stages of drug development and discovery<\/a>, this is the first time a completely new class of antibiotics has been identified from scratch, without using any previous human assumptions.<\/p>\n

Unlike the majority of in silico<\/em> ML-based drug discovery papers that end with only a computational prediction, the paper of Stokes et al.<\/em> not only identified promising molecules but also extensively validated them in vivo<\/em> on lab animals. While the approach can in principle be applied to discover candidate therapies against other diseases such as cancer, the focus on antibiotics is very timely: antibiotic-resistant microorganisms that can develop as a result of antibiotic abuse are one of the nightmares of world-wide healthcare, and the potential emergence of a highly-contagious bacterial infection that evades the existing arsenal of drugs is not a question of \u201cif\u201d but \u201cwhen\u201d.<\/p>\n

More on this:\u00a0<\/strong>Quanta Magazine\u00a0popular article<\/a>\u00a0and Jim Collins\u2019\u00a0TED Talk<\/a>\u00a02020 (Collins lab was part of this year\u2019s TED\u00a0Audacious Projects<\/a>\u00a0cohort, together with our\u00a0Project CETI<\/a>).<\/p>\n

J. Jumper\u00a0et al.<\/em>, High accuracy protein structure prediction using deep learning (2020) a.k.a.\u00a0<\/strong>AlphaFold 2.0<\/strong><\/a>(paper not yet available)<\/p>\n

What?<\/strong> Very accurate prediction of 3D protein structure from aminoacid sequence, a notoriously hard problem in bioinformatics.<\/p>\n

How?<\/strong>\u00a0AlphaFold 2.0 is an \u201cattention-based neural network\u201d (likely a transformer architecture) trained end-to-end on 170000 protein structures from the Protein Data Bank<\/a>\u00a0as well as protein sequences with unknown structure. Since DeepMind has not yet disclosed the details of the algorithm, we can only hypothesise how it works. From the blog post, proteins are modelled as spatial contact graphs, and the neural network \u201cinterprets the structure of this graph, while reasoning over the implicit graph that it is building.\u201d It sounds like some form of graph neural network with latent graph learning, though there are probably many more additional details and nuances, as the method also uses evolutionarily sequence information, so I would classify it as \u201cgeometric ML\u201d with some caution. The reported computational complexity of training is very high (an equivalent of several years of GPU time) and the prediction of the structure is \u201ca matter of days.\u201d<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\"Geometric\t\t\t\t\t\t\t\t\t\t\t
The \u201csnake toy\u201d is a good metaphor for protein folding, where a one-dimensional sequence of aminoacids folds into a complex 3D shape that gives the protein its functions.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Why it matters?<\/strong>\u00a0Proteins are arguably the most important biomolecules, often called the \u201cmolecules of life\u201d, as we are currently not aware of any life-form that is not protein-based. Proteins are encoded in our DNA and have myriads of functions in our body, including protection against pathogens (antibodies<\/a>), giving structure to our skin (collagen<\/a>), transporting oxygen to cells (haemoglobin<\/a>), catalysing chemical reactions (enzymes<\/a>), and signaling (many hormones are proteins).<\/p>\n

Chemically speaking, a protein is a biopolymer, or a chain of small building blocks called aminoacids that under the influence of electrostatic forces fold into a complex 3D structure. It is this structure that endows the protein with its functions, and hence it is crucial to the understanding of how proteins work and what they do. Since proteins are common targets for drug therapies (typical drugs are small molecules designed to bind to their target), the pharmaceutical industry<\/a> has a keen interest in this field.<\/p>\n

Modern technology allows to sequence proteins (i.e. obtain the string of their constituent aminoacids) cheaply and reliably but getting the 3D structure typically relies on old crystallographic techniques that are capricious, long, and expensive. As a result, there are about 200 million proteins with known sequences and less than 200 thousand with known structures.<\/p>\n

It was long\u00a0hypothesised<\/a>\u00a0that the aminoacid sequence contains enough information to predict the 3D structure of the protein, but the problem turned to be extremely challenging. The\u00a0Critical Assessment of Protein Structure Prediction<\/a>\u00a0(CASP), an ImageNet-like competition running since 1994 in which participants try to predict 3D structures of unknown proteins, has been a classical testbed for bioinformatics research labs and pharmaceutical companies.<\/p>\n

In 2018, DeepMind surprised the research community with a newcomer\u00a0AlphaFold<\/a>\u00a0that came out of the blue as the winner of CASP. The 2020 version of the algorithm, AlphaFold 2.0 seems to work significantly better, achieving a root mean squared distance (RMSD) of 1.6 angstroms, which is considered very accurate by structural biology standards and a far cry from other competitors. This is an \u201cImageNet moment\u201d for protein science.<\/p>\n

While by all means stunning progress on a key problem, the popular press hype with relaxed terminology provides a somewhat distorted picture of what AlphaFold can or cannot do. In particular, for drug design applications, one typically requires sub-angstrom accuracy on the binding site that the method cannot yet deliver.<\/p>\n

More on this:\u00a0<\/strong>Everyone is eagerly waiting for the paper detailing the algorithm to be published. In the meantime, Lex Fridman\u2019s\u00a0YouTube video<\/a>\u00a0provides a good summary, and Mohammed AlQuraishi\u2019s\u00a0blog post<\/a>\u00a0describes the effect AlphaFold has had in 2018.<\/p>\n

P. Gainza\u00a0et al.<\/em>,\u00a0<\/strong>Deciphering interaction fingerprints from protein molecular surfaces using geometric deep learning<\/strong><\/a>\u00a0(2020)\u00a0Nature Methods<\/em>\u00a017(2):184\u2013192.<\/strong><\/p>\n

What? <\/strong>A geometric deep learning pipeline called MaSIF predicting interactions between proteins from their 3D structure.<\/p>\n

How?\u00a0<\/strong>MaSIF models the protein as a molecular surface discretised as a mesh, arguing that this representation is advantageous when dealing with interactions as it allows to abstract the internal fold structure. The architecture is based on\u00a0MoNet<\/a>, a mesh convolutional neural network developed by my PhD student Federico Monti and operating on pre-computed chemical and geometric features in small geodesic patches.<\/p>\n

The network is trained using a few thousand co-crystal protein 3D structures from the Protein Data Bank to address multiple tasks, including interface prediction, ligand classification, and docking, showing state-of-the-art performance. A key difference of MaSIF compared to other methods is that it does not rely on the evolutionary history of proteins. This is of crucial importance in de novo<\/em> protein design, where one tries to create \u201cfrom scratch\u201d new proteins that have never existed before.<\/p>\n

Allowing myself to be more critical as a co-author of the paper, I would highlight the need to precompute the molecular surface mesh and local patches as well as the reliance on hand-crafted features as one of the key drawbacks of MaSIF. Over the course of this year, we have completely\u00a0revamped the architecture<\/a>, which now operates directly on the atomic point cloud as input, computes the molecular surface (represented as a point cloud) on the fly, learns geometric and chemical features, is end-to-end differentiable, and runs orders of magnitude faster (the latter is afforded due to the use of a fast geometric computation library\u00a0KeOps<\/a>, developed by my postdoc Jean Feydy). Moreover, while the\u00a0Nature Methods<\/em>\u00a0paper focused primarily on the computational methodology, the crystallographic structures of several of the new protein binders designs with MaSIF were subsequently obtained by my\u00a0EPFL collaborators<\/a>, coinciding very accurately with the computed one.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\"Geometric\t\t\t\t\t\t\t\t\t\t\t
Predicting the binding site of a protein using MaSIF. The designed protein shown (right) was modified to improve binding to the target from the naturally-occurring \u201cwildtype\u201d (center). MaSIF is able to correctly detect the binding site even though it has a flat structure.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Why it matters? <\/strong>Interactions between proteins and other biomolecules are the basis of protein function in most biological processes. A better understanding of how proteins interact is thus key both for fundamental biology as well as for drug development: many diseases are associated with protein-to-protein interactions (PPI) which are promising drug targets. However, such interactions usually involve flat interfaces deemed \u201cundruggable\u201d, as they are very different from the traditional pocket-shaped structures targeted by small drug molecules.<\/p>\n

The success of MaSIF to identify binders for such targets make it a promising tool for rational protein design and opens multiple exciting applications in biological drug development, such as\u00a0immune checkpoint inhibition<\/a>\u00a0cancer therapy that tries to target the\u00a0PD-1<\/a>\/PD-L1<\/a>\u00a0protein complex responsible for programmed cell death.<\/p>\n

More on this:\u00a0<\/strong>John Pavlus\u2019\u00a0article<\/a>\u00a0in Quanta Magazine (John has\u00a0previously reported<\/a>\u00a0on geometric deep learning), Bharath Ramsundar\u2019s\u00a0blog post<\/a>, and\u00a0my talk<\/a>\u00a0at the Broad Institute earlier this year.<\/p>\n

I am grateful to Sergey Ivanov, Maksym Korablyov, and Jake Taylor-King for comments on an early draft of this post.<\/em><\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"

We are finally starting to see the real impact of Geometric machine learning techniques in fundamental science.<\/p>\n","protected":false},"author":874,"featured_media":18637,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[183],"tags":[1322,1323,92],"ppma_author":[3686],"class_list":["post-22612","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-ml","tag-fundamental-science","tag-geometric-ml","tag-machine-learning"],"authors":[{"term_id":3686,"user_id":874,"is_guest":0,"slug":"michael-bronstein","display_name":"Michael Bronstein","avatar_url":"https:\/\/www.experfy.com\/blog\/wp-content\/uploads\/2020\/08\/Michael-Bronstein-150x150.jpg","user_url":"https:\/\/www.imperial.ac.uk\/people\/m.bronstein","last_name":"Bronstein","first_name":"Michael","job_title":"","description":"Michael Bronstein is Professor, Chair in Machine Learning and Pattern Recognition at Imperial College, London, besides Head of Graph ML at Twitter \/ ML Lead at ProjectCETI\/ ex Founder & Chief Scientist at Fabula_ai\/ ex at Intel #AI #ML #graphs. His main expertise is in theoretical and computational geometric methods for data analysis, and his research encompasses a broad spectrum of applications ranging from machine learning, computer vision, and pattern recognition to geometry processing, computer graphics, and imaging. He has authored over 150 papers, the book Numerical geometry of non-rigid shapes (Springer 2008), and holds over 30 granted patents."}],"_links":{"self":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/22612","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/users\/874"}],"replies":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/comments?post=22612"}],"version-history":[{"count":4,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/22612\/revisions"}],"predecessor-version":[{"id":32259,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/22612\/revisions\/32259"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media\/18637"}],"wp:attachment":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media?parent=22612"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/categories?post=22612"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/tags?post=22612"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=22612"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}