{"id":2180,"date":"2020-01-08T01:30:37","date_gmt":"2020-01-08T01:30:37","guid":{"rendered":"http:\/\/kusuaks7\/?p=1785"},"modified":"2024-01-30T10:19:54","modified_gmt":"2024-01-30T10:19:54","slug":"the-future-of-computation-for-machine-learning-and-data-science","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/ai-ml\/the-future-of-computation-for-machine-learning-and-data-science\/","title":{"rendered":"The Future of Computation for Machine Learning and Data Science"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-post\" data-elementor-id=\"2180\" class=\"elementor elementor-2180\" data-elementor-post-type=\"post\">\n\t\t\t\t\t\t<section class=\"has_eae_slider elementor-section elementor-top-section elementor-element elementor-element-211a4fe2 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"211a4fe2\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"has_eae_slider elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-6aef8846\" data-id=\"6aef8846\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-ebe20f6 elementor-widget elementor-widget-heading\" data-id=\"ebe20f6\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\"><h3 style=\"color: #aaa;font-style: italic\"><em><strong>What are the challenges posing data scientists and how are researchers tackling them?<\/strong><\/em><\/h3><\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-269f45e elementor-widget elementor-widget-text-editor\" data-id=\"269f45e\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<blockquote>\u201cThe human brain uses much less energy than a computer, yet it is infinitely more complex and intricate.\u201d \u2014\u00a0<strong><em>Nayef Al-Rodhan, Global Policy Journal<\/em><\/strong><\/blockquote>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e2a7049 elementor-widget elementor-widget-image\" data-id=\"e2a7049\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/miro.medium.com\/max\/1600\/0*Tqe5T5lIMQ86YTKY.jpg\" alt=\"\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"has_eae_slider elementor-section elementor-top-section elementor-element elementor-element-4704d19 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"4704d19\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"has_eae_slider elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-7c17ad1\" data-id=\"7c17ad1\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-2ba7a04 elementor-widget elementor-widget-text-editor\" data-id=\"2ba7a04\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p style=\"text-align: center;\" data-selectable-paragraph=\"\"><span style=\"font-size: 11px;\"><strong>Source:<\/strong>\u00a0<a href=\"https:\/\/www.nanalyze.com\/2019\/08\/neuromorphic-computing-ai-chip\/\" target=\"_blank\" rel=\"noopener nofollow noreferrer\" class=\"broken_link\">https:\/\/www.nanalyze.com\/2019\/08\/neuromorphic-computing-ai-chip\/<\/a><\/span><\/p>\n<p id=\"a40d\" data-selectable-paragraph=\"\">In this article, I will discuss the current challenges and expected future trends in computational science with a bent towards their impacts on the fields of data science and machine learning. This includes the breakdown of Dennard scaling \u2014 the slowdown of Moore\u2019s law, the burgeoning field of the internet of things (IoT), and the advent of more energy-efficient computing that mimics the human brain, known as neuromorphic computing.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c369efe elementor-widget elementor-widget-heading\" data-id=\"c369efe\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h1 class=\"elementor-heading-title elementor-size-default\"><h1 id=\"a42e\" data-selectable-paragraph=\"\">The Trends<\/h1><\/h1>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e469b01 elementor-widget elementor-widget-heading\" data-id=\"e469b01\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\"><h2 id=\"d701\" data-selectable-paragraph=\"\"><strong>Deep Learning<\/strong><\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-9e0adb7 elementor-widget elementor-widget-text-editor\" data-id=\"9e0adb7\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"b743\" data-selectable-paragraph=\"\">Deep learning has become ubiquitous in the modern world, with wide-ranging applications in nearly every field. As might be expected, people have started to notice, and the hype behind deep learning continues to increase as its widespread adoption by businesses occurs. Deep learning consists of neural networks with multiple hidden layers and has some particularly demanding needs in terms of computational resources:<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-34aa0e9 elementor-widget elementor-widget-text-editor\" data-id=\"34aa0e9\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<ul>\n \t<li id=\"cdbc\" data-selectable-paragraph=\"\">1 billion parameters to train.<\/li>\n \t<li id=\"a8ac\" data-selectable-paragraph=\"\">Computational intensity increases as the network depth grows<\/li>\n<\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ac36620 elementor-widget elementor-widget-text-editor\" data-id=\"ac36620\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"5ff4\" data-selectable-paragraph=\"\">Deep learning has been hugely successful for some important tasks in speech recognition, computer vision, and text understanding.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-595dab8 elementor-widget elementor-widget-image\" data-id=\"595dab8\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/miro.medium.com\/max\/990\/1*2nN6nat5gOTiM7n_lD5dFQ.png\" alt=\"\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-0390e83 elementor-widget elementor-widget-text-editor\" data-id=\"0390e83\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p style=\"text-align: center;\" data-selectable-paragraph=\"\"><span style=\"font-size: 11px;\">The apparent success of deep learning in the field of speech recognition<\/span><\/p>\n<p id=\"2360\" data-selectable-paragraph=\"\">The major downside of deep learning is its computational intensity, requiring high-performance computational resources and long training times. For facial recognition and image reconstructions, this also means working with low-resolution images. How will researchers aim to tackle these big compute problems? And what is the bottleneck of this approach in the first place?<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-b0716f3 elementor-widget elementor-widget-heading\" data-id=\"b0716f3\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\"><h2 id=\"68d0\" data-selectable-paragraph=\"\">Internet of Things (IoT)<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-fd78be9 elementor-widget elementor-widget-text-editor\" data-id=\"fd78be9\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"9d3b\" data-selectable-paragraph=\"\">A total of 20 billion embedded devices are expected to be active in the next several years, stemming from smart fridges, phones, thermostats, air quality sensors, locks, dog collars, the list is impressively long. Most of these devices will be battery powered and severely resource-constrained in memory and computing power. In spite of these limitations, these devices need to perform intelligent tasks and be easy to use.<\/p>\n<p id=\"a577\" data-selectable-paragraph=\"\">The expected prevalence of IoT devices comes with major concerns, particularly regarding the security of these devices (perhaps future criminals will be hackers that can hack my smart locks on my doors). It also portends the advent of data-driven deep learning. Deep learning already requires extensive amounts of data, but the presence of large-scale networked devices or sensors opens new avenues of research \u2014 especially in my own field of environmental science, which is seeing an increasing push towards low-cost and large-scale sensor networks to monitor atmospheric pollutants.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-9517cbb elementor-widget elementor-widget-text-editor\" data-id=\"9517cbb\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"6a51\" data-selectable-paragraph=\"\">This idea also extends to smartphones. Smartphones are becoming increasingly powerful and now rival some old laptops in their computational abilities. Increasingly, machine learning is performed on these devices. Despite their improvements, they are still limited in their computational resources and hence trends towards reducing computational and data overhead are increasingly important on these devices.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c2e6ab5 elementor-widget elementor-widget-text-editor\" data-id=\"c2e6ab5\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"6735\" data-selectable-paragraph=\"\">This idea also extends to self-driving cars, which are beginning to be commercialized after a long period of research and development. Self-driving cars present their own issues but still suffer from the same concerns, the most publicized of which are the security and ethical underpinning of the vehicles.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-b26883c elementor-widget elementor-widget-text-editor\" data-id=\"b26883c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"634f\" data-selectable-paragraph=\"\">More data means more data science (woohoo for the data scientists!), but it also means needing to store and process this data, and to transmit this data through wireless networks (and do it securely, I might add). How are the problems associated with IoT to be tackled? And whose responsibility is it if something goes wrong?<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c6dbdf6 elementor-widget elementor-widget-heading\" data-id=\"c6dbdf6\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\"><h2 id=\"278a\" data-selectable-paragraph=\"\"><strong>Cloud-Based Computing<\/strong><\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-a98829b elementor-widget elementor-widget-text-editor\" data-id=\"a98829b\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"1b07\" data-selectable-paragraph=\"\">There is an increasing push towards the use of cloud computing by companies in order to outsource their computational needs and minimize costs.<\/p>\n<p id=\"7115\" data-selectable-paragraph=\"\">This also aids companies (especially startups) by reducing capital costs associated with purchasing infrastructure and instead moving it to operating costs. With the growing popularity of machine learning and data science, which are computationally intensive and require high-performance systems, as well as the increasing online presence of most corporations, this trend in cloud-based distributed computing is not expected to slow down anytime soon.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-83d611d elementor-widget elementor-widget-heading\" data-id=\"83d611d\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\"><h2 id=\"08b0\" data-selectable-paragraph=\"\"><strong>Scaling Computation Size<\/strong><\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"has_eae_slider elementor-section elementor-top-section elementor-element elementor-element-ee9fab8 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"ee9fab8\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"has_eae_slider elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-ea0270d\" data-id=\"ea0270d\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-ab9dc2c elementor-widget elementor-widget-text-editor\" data-id=\"ab9dc2c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"6d35\" data-selectable-paragraph=\"\">Scaling up computation size with only modestly increased energy consumption is a difficult task. We need such balanced computation scaling for deep neural networks to minimize their energy overhead.<\/p>\n<p id=\"6eb9\" data-selectable-paragraph=\"\">The current solution of this is the parallel use of many small, energy-efficient compute nodes to accommodate large computations.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-bf05a27 elementor-widget elementor-widget-heading\" data-id=\"bf05a27\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\"><h2 id=\"8f3a\" data-selectable-paragraph=\"\"><strong>Reducing Energy Usage<\/strong><\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-4ed96eb elementor-widget elementor-widget-text-editor\" data-id=\"4ed96eb\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"9d2e\" data-selectable-paragraph=\"\">Scaling down energy with only modestly decreased outcome quality (e.g., prediction accuracy) is becoming an increasingly important task. We need such balanced energy scaling for IoT devices and wearables. These considerations are related to a new field called\u00a0<em>approximate computing<\/em>\u00a0which trades off quality with the effort expended.<\/p>\n<p id=\"61de\" data-selectable-paragraph=\"\">The current solution to this is the coordinated use of multiple small, energy-efficient compute nodes to support intelligent computations.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-abbacab elementor-widget elementor-widget-heading\" data-id=\"abbacab\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\"><h2 id=\"4e07\" data-selectable-paragraph=\"\"><strong>More Data<\/strong><\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-1c2ed3f elementor-widget elementor-widget-text-editor\" data-id=\"1c2ed3f\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"fb63\" data-selectable-paragraph=\"\">The data trend is clearly not going to stop any time soon! However, a discussion for this deserves its own section.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-962b935 elementor-widget elementor-widget-heading\" data-id=\"962b935\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h1 class=\"elementor-heading-title elementor-size-default\"><h1 id=\"49af\" data-selectable-paragraph=\"\">Data Overload<\/h1><\/h1>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-b3c015c elementor-widget elementor-widget-text-editor\" data-id=\"b3c015c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"ed70\" data-selectable-paragraph=\"\">The data explosion of the past two decades now means that\u00a0<strong>humanity produces the same amount of data as all of mankind up to 2003, every 2 days<\/strong>. Computing trends show that we are now in the region of exascale data, which refers to the computing of data on the scale of 1 billion gigabytes, as well as exascale computing, which refers to computers capable of performing 10\u00b9\u2078 computations per second.<\/p>\n<p id=\"e93b\" data-selectable-paragraph=\"\">The sum total of data held by all the big online storage and service companies like Google, Amazon, Microsoft and Facebook make up about 4,800 petabytes (4.8 Exabytes).<\/p>\n<p id=\"8d26\" data-selectable-paragraph=\"\">Analyzing large sets of data (even a few gigabytes) requires different approaches to running models on your local laptop \u2014 trying to do so will simply crash your computer.<\/p>\n<p id=\"3060\" data-selectable-paragraph=\"\"><strong>Parallelization<\/strong><\/p>\n<p id=\"add4\" data-selectable-paragraph=\"\">Currently the standard ways of processing highly expensive computations, which are typically done with\u00a0<em>supercomputers<\/em>\u00a0(for\u00a0<strong>big compute<\/strong>\u00a0tasks, requiring lots of computing power), or with\u00a0<em>large distributed systems<\/em>\u00a0(for\u00a0<strong>big data<\/strong>\u00a0tasks, that require lots of computational memory).<\/p>\n<p id=\"1e6e\" data-selectable-paragraph=\"\"><strong>In the near future, if you want to analyze large data sets, you will essentially have no choice but to turn to parallel computing.<\/strong><\/p>\n<p id=\"2af5\" data-selectable-paragraph=\"\">Distributed systems work well up to a point, but quickly break down once we approach\u00a0<em>clusters<\/em>\u00a0(a group of computers working on a single task) with 1,000\u00a0<em>nodes\u00a0<\/em>(individual computers in a cluster). Most of the time in the cluster is spent communicating, which causes overheads that slow down the overall computation of a system. This phenomenon is known as\u00a0<a href=\"https:\/\/en.wikipedia.org\/wiki\/Parallel_slowdown\" target=\"_blank\" rel=\"noopener nofollow noreferrer\"><em>parallel slowdown<\/em><\/a><em>.<\/em><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-a7ed73d elementor-widget elementor-widget-image\" data-id=\"a7ed73d\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/miro.medium.com\/max\/600\/1*JPywLUeUlQy1n-BjVFjm4w.png\" alt=\"\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-d264cd8 elementor-widget elementor-widget-text-editor\" data-id=\"d264cd8\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p style=\"text-align: center;\" data-selectable-paragraph=\"\"><span style=\"font-size: 11px;\">Illustration of parallel slowdown \u2014 initially, we see an improvement in the speedup as the number of processors increases. This relationship breaks down for large numbers of processing elements due to a communications bottleneck (too many people talking, not enough work being done).\u00a0<a href=\"https:\/\/en.wikipedia.org\/wiki\/Parallel_slowdown\" target=\"_blank\" rel=\"noopener nofollow noreferrer\">Source<\/a><\/span><\/p>\n<p id=\"3ec2\" data-selectable-paragraph=\"\">Another compounding issue we run into is the proportion of our computation that is parallelizable, which is described by\u00a0<a href=\"https:\/\/en.wikipedia.org\/wiki\/Amdahl%27s_law\" target=\"_blank\" rel=\"noopener nofollow noreferrer\"><strong>Amdahl\u2019s Law<\/strong><\/a>:<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-55aafe6 elementor-widget elementor-widget-text-editor\" data-id=\"55aafe6\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<blockquote>\n<p id=\"8559\" data-selectable-paragraph=\"\"><strong>Amdahl\u2019s law<\/strong>\u00a0states that in parallelization, if P is the proportion of a system or program that can be made parallel, and 1-P is the proportion that remains serial, then the maximum speedup that can be achieved using N number of processors\u00a0<strong>is<\/strong>\u00a01\/((1-P)+(P\/N)).<\/p>\n<\/blockquote>\n<p id=\"169b\" data-selectable-paragraph=\"\">In layman\u2019s term, our maximum speedup depends on the proportion of the program that must be done sequentially \u2014 the proportion that is not parallelizable. If 50% of our computation is parallelizable, we can get much less speedup than if our program is 95% parallelizable. This phenomenon is illustrated below.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-120de52 elementor-widget elementor-widget-image\" data-id=\"120de52\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/miro.medium.com\/max\/800\/1*-l1HsP-WNkAALureAYUN1Q.png\" alt=\"\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-996cb0f elementor-widget elementor-widget-text-editor\" data-id=\"996cb0f\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p style=\"text-align: center;\" data-selectable-paragraph=\"\"><span style=\"font-size: 11px;\">Amdahl\u2019s law, illustrated. As the number of processors we have available increases, we reach a bottleneck which depends on the serial portion of our algorithm.\u00a0<a href=\"https:\/\/en.wikipedia.org\/wiki\/Amdahl%27s_law\" target=\"_blank\" rel=\"noopener nofollow noreferrer\">Source<\/a><\/span><\/p>\n<p id=\"91e7\" data-selectable-paragraph=\"\">This problem disappears for computations that are\u00a0<a href=\"https:\/\/en.wikipedia.org\/wiki\/Embarrassingly_parallel\" target=\"_blank\" rel=\"noopener nofollow noreferrer\"><em>embarrassingly parallel<\/em><\/a>, which means that the program is 100% parallelizable and essentially separate \u2014 for example, testing of different hyperparameters of a neural network.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-a86a8bc elementor-widget elementor-widget-text-editor\" data-id=\"a86a8bc\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\n<blockquote>\n<p id=\"e332\" data-selectable-paragraph=\"\"><strong><em>Note:\u00a0<\/em><\/strong><em>What is the difference between parallel computing and distributed computing?\u00a0<\/em><strong><em>Parallel computing<\/em><\/strong><em>\u00a0is the simultaneous use of multiple compute nodes over shared memory.\u00a0<\/em><strong><em>Distributed computing\u00a0<\/em><\/strong><em>is the simultaneous use of multiple compute nodes each working on their own memory.<\/em><\/p>\n<\/blockquote>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-289d803 elementor-widget elementor-widget-text-editor\" data-id=\"289d803\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"a4d7\" data-selectable-paragraph=\"\">Distributed computing systems that underpin the online systems of many businesses are commonly seen on the cloud nowadays, with business offloading most of their computational infrastructure to big cloud companies such as Amazon and Microsoft for easier and cheaper maintenance.<\/p>\n<p id=\"7441\" data-selectable-paragraph=\"\">Supercomputers work well but they are extremely expensive to run and maintain, and they are in high demand. Within universities such as Harvard, labs have access to a proportion of the computational resources, which can be shared among the group members. However, rental costs are very expensive. In the field of environmental science, running a global climate model for a month with an hourly time resolution can take several weeks to run, which can be extremely costly.<\/p>\n<p id=\"f408\" data-selectable-paragraph=\"\">Computing power has always been an issue, but nowadays the biggest issue is that the data we have is just getting too big to handle. Even if we assume Moore\u2019s law still to hold (which is discussed in the next section), data production is exponential whereas Moore\u2019s law is linear. The amount of data available to process quickly incapacitates any general-purpose computer, or set of computers configured together in a cluster if the computational intensity is large enough.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-36dbf36 elementor-widget elementor-widget-image\" data-id=\"36dbf36\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/miro.medium.com\/max\/1600\/0*cscvSwrKTCIJcnuO.png\" alt=\"\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-857e066 elementor-widget elementor-widget-text-editor\" data-id=\"857e066\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p style=\"text-align: center;\" data-selectable-paragraph=\"\"><span style=\"font-size: 11px;\">Computing trends in processing power, storage cost, and data growth.\u00a0<a href=\"https:\/\/medium.com\/@rpradeepmenon\/an-executive-primer-to-deep-learning-80c1ece69b34\" target=\"_blank\" rel=\"noopener noreferrer\" class=\"broken_link\">Source<\/a><\/span><\/p>\n<p id=\"08a5\" data-selectable-paragraph=\"\">Cloud computing is not cheap if you have a thousand computers running at once to manage your business\u2019 database servers, but at least it is feasible. How, on the other hand, would you expect to perform business analytics on a dataset that is 100 Petabytes? Disk space is incredibly cheap, but memory is not, and your data must be in memory before it can be readily analyzed.<\/p>\n<p id=\"438c\" data-selectable-paragraph=\"\">Fortunately, we can at least estimate how long a computation (which can also be extended to data) might take by something called the\u00a0<a href=\"https:\/\/en.wikipedia.org\/wiki\/Elapsed_real_time\" target=\"_blank\" rel=\"noopener nofollow noreferrer\"><em>wall clock time<\/em><\/a><em>.\u00a0<\/em>This can be done by first finding the computational complexity of your computation:<\/p>\n<p id=\"5a1f\" data-selectable-paragraph=\"\"><em>CC = CD*N *T<\/em><\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-53adbc4 elementor-widget elementor-widget-text-editor\" data-id=\"53adbc4\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<ul>\n \t<li id=\"a5a9\" data-selectable-paragraph=\"\">CC = computational complexity (# Flops)<\/li>\n \t<li id=\"fd93\" data-selectable-paragraph=\"\">CD = computational density (# Flops\/site\/step)<\/li>\n \t<li id=\"3d74\" data-selectable-paragraph=\"\">N = Domain size or number of data points (# sites)<\/li>\n \t<li id=\"089e\" data-selectable-paragraph=\"\">T = Number of timesteps or epochs (# steps)<\/li>\n<\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-7b83935 elementor-widget elementor-widget-text-editor\" data-id=\"7b83935\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"3322\" data-selectable-paragraph=\"\">To calculate the wall clock time, you need to know the processing power of your laptop (e.g. 1 GFlops\/s). Then:<\/p>\n<p id=\"f17e\" data-selectable-paragraph=\"\"><em>WCT = CC \/ processing power<\/em><\/p>\n<p id=\"5d5c\" data-selectable-paragraph=\"\">For example, if you are performing a computation on a 2D grid with 1000 data points for 1000 timesteps, you will perform 6 computations at each grid point. This has a computational intensity of 6 ExaFlops, which will take you about 6000 seconds (~2 hours) on a 1 GFlops\/s laptop.<\/p>\n<p id=\"e2bd\" data-selectable-paragraph=\"\">For a list of computational complexities for a variety of different machine learning algorithms, see the table below.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e2eeab9 elementor-widget elementor-widget-image\" data-id=\"e2eeab9\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/miro.medium.com\/max\/1320\/1*4vL9aYaMITO_xTMPxaw7MQ.png\" alt=\"\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-982b52e elementor-widget elementor-widget-text-editor\" data-id=\"982b52e\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p style=\"text-align: center;\" data-selectable-paragraph=\"\"><span style=\"font-size: 11px;\">Computational complexities of machine learning algorithms.\u00a0<a href=\"https:\/\/www.thekerneltrip.com\/machine\/learning\/computational-complexity-learning-algorithms\/\" target=\"_blank\" rel=\"noopener nofollow noreferrer\">Source<\/a><\/span><\/p>\n<p id=\"00e3\" data-selectable-paragraph=\"\"><strong>Estimate your wall clock time before you do a large computation!<\/strong><\/p>\n<p id=\"7f54\" data-selectable-paragraph=\"\">As can be seen, the time increases in proportional to the number of computations, which is related to (1) the number of data points, or (2) the size of the computational domain.<\/p>\n<p id=\"bc56\" data-selectable-paragraph=\"\">How important is parallelization within machine learning? Extremely important, as it turns out. There are two ways this can occur:<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f5295ac elementor-widget elementor-widget-text-editor\" data-id=\"f5295ac\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<ul>\n \t<li id=\"3a51\" data-selectable-paragraph=\"\"><strong>Data parallelism \u2014\u00a0<\/strong>distribute data, use the same model.<\/li>\n \t<li id=\"6549\" data-selectable-paragraph=\"\"><strong>Model parallelism<\/strong>\u00a0\u2014 distribute model, use the same data.<\/li>\n<\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-5c563e6 elementor-widget elementor-widget-image\" data-id=\"5c563e6\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/miro.medium.com\/max\/1806\/1*89lq0AzemmIqZDuyeZg9BA.png\" alt=\"\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f9b8fc3 elementor-widget elementor-widget-text-editor\" data-id=\"f9b8fc3\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p style=\"text-align: center;\" data-selectable-paragraph=\"\"><span style=\"font-size: 11px;\">Data parallelism vs model parallelism<\/span><\/p>\n<p id=\"4ae7\" data-selectable-paragraph=\"\">How do people expect to handle the increasingly huge amount of data that must be analyzed? There are a number of options, although this is by means exhaustive:<\/p>\n<p id=\"5711\" data-selectable-paragraph=\"\"><strong>[1]<\/strong>\u00a0Instead of moving the \u2018compute\u2019 to the data, move the data to the \u2018compute\u2019. There is a big push for this in the environmental sciences due to the presence of large-scale distributed networks.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-951faf4 elementor-widget elementor-widget-text-editor\" data-id=\"951faf4\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<ul>\n \t<li id=\"1cfd\" data-selectable-paragraph=\"\"><strong>Edge and fog computing<\/strong>\u00a0\u2014 where computations are done closer to the data sources on edge devices. This will become important in the IoT infrastructure.<\/li>\n<\/ul>\n<p id=\"0aa7\" data-selectable-paragraph=\"\"><strong>[2]\u00a0<\/strong>Hierarchical segmentation of workloads to produce non-interactive workloads that can be distributed (not always possible for interactive workloads).<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f187153 elementor-widget elementor-widget-text-editor\" data-id=\"f187153\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<ul>\n \t<li id=\"d485\" data-selectable-paragraph=\"\"><strong>Grid computing\u00a0<\/strong>\u2014 imagine performing hyperparameter tuning on a neural network and sending one set of hyperparameters to one computational cluster, and another set to a second cluster (an example of model parallelism). The two clusters do not communicate and are effectively independent. This is not (typically) feasible for data parallelism in a neural network, for example.<\/li>\n<\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f3afa92 elementor-widget elementor-widget-text-editor\" data-id=\"f3afa92\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"5ad1\" data-selectable-paragraph=\"\"><strong>[3]<\/strong>\u00a0Improvements in high-speed communication, parallel frameworks, or time management of distributed systems (these are already pretty good though!).<\/p>\n<p id=\"0ac9\" data-selectable-paragraph=\"\"><strong>[4]<\/strong>\u00a0Using specialized hardware such as ASICs (like people have done for bitcoin mining, to optimize hardware for a specific computation), GPUs, and TPUs. One GPU can provide the performance of 8 CPUs on graphical data, and a single TPU can be as effective as 8 GPUs \u2014 naturally, they do not come cheap.<\/p>\n<p id=\"0fb3\" data-selectable-paragraph=\"\"><strong>How can I implement parallelization in my own work?<\/strong><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-157d162 elementor-widget elementor-widget-text-editor\" data-id=\"157d162\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"0655\" data-selectable-paragraph=\"\">Speed is becoming increasingly important for computation due to the need to analyze huge datasets in a tractable timeframe. Python is an excellent language that is the most popular language used by data scientists, but sadly it is not a particularly fast language (for one thing, it is\u00a0<em>interpreted<\/em>\u00a0and not\u00a0<em>compiled<\/em>, which means the code is translated into binary code\u00a0<em>on-the-fly<\/em>, which takes additional time).<\/p>\n<p id=\"55e5\" data-selectable-paragraph=\"\">Parallelization is possible in most languages in order to speed up the code, and this can be in the form of adding extra processors, as we have discussed, or by using additional\u00a0<em>threads \u2014\u00a0<\/em>if you are not familiar, think of this as opening two of the same computer program such that they can share the program load. This is known as\u00a0<em>multithreading<\/em>, and whilst less useful for massive datasets, it can still provide a substantial benefit. One of the most popular implementations of this is OpenMP for use in C++.<\/p>\n<p id=\"b004\" data-selectable-paragraph=\"\">Multithreading is an important issue with Python, due to something called the GIL. We cannot use OpenMP but, fortunately, there is a multithreading library imaginatively called\u00a0<code>multithreading<\/code>\u00a0that can be used.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-34813fa elementor-widget elementor-widget-text-editor\" data-id=\"34813fa\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"0e08\" data-selectable-paragraph=\"\">Parallelization between processors can be done using MPI, and can be extended to graphical processing units (GPUs) and also more specialized equipment (such as Google\u2019s Tensor Processing Unit) using other parallel frameworks such as OpenACC. These are reasonably easy to set up on instances offered by cloud providers (TPUs are only offered by GoogleCloud).<\/p>\n<p id=\"8d76\" data-selectable-paragraph=\"\">If working with images, I\u00a0<strong>highly\u00a0<\/strong>recommend doing computations with at the very least one GPU. This can reduce training times by an order of magnitude \u2014 that is the difference between waiting an hour and an entire day.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-2c13981 elementor-widget elementor-widget-heading\" data-id=\"2c13981\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\"><h1 id=\"761d\" data-selectable-paragraph=\"\">The End of Moore\u2019s Law (as we know it)<\/h1><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-97d6751 elementor-widget elementor-widget-text-editor\" data-id=\"97d6751\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"7249\" data-selectable-paragraph=\"\">In 1965, Gordon Moore, an Intel co-founder, made a prediction that would set the pace for our modern digital revolution. From careful observation of an emerging trend, Moore extrapolated that computing would dramatically increase in power, and decrease in relative cost, at an exponential pace.<\/p>\n<p id=\"1488\" data-selectable-paragraph=\"\">But slower pace is expected in the future. In July 2015 Intel CEO Brian Krzanich said,\u00a0<em>\u201cthe exponential advances in semiconductor manufacturing that enable faster and cheaper computing and storage every two years are now going to come closer to a rate of every two and half years\u201d.<\/em><\/p>\n<p id=\"bd1a\" data-selectable-paragraph=\"\">Rather than continuing to make microprocessors more and more powerful, people now work on new architectures and computing models which leverage parallel processing to provide balanced computation and energy scaling, as noted earlier.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-78446e7 elementor-widget elementor-widget-text-editor\" data-id=\"78446e7\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"eba3\" data-selectable-paragraph=\"\"><a href=\"https:\/\/en.wikipedia.org\/wiki\/Moore%27s_law\" target=\"_blank\" rel=\"noopener nofollow noreferrer\"><strong>Moore\u2019s law<\/strong><\/a>\u00a0\u2014 the phenomenon that the number of transistors on a CPU doubles around every 2 years \u2014 is still observed to some extent, but the assumption that computing power still scales exactly with the area has been seen to have broken down. This assumption of the scaling of power density is known as\u00a0<a href=\"https:\/\/en.wikipedia.org\/wiki\/Dennard_scaling\" target=\"_blank\" rel=\"noopener nofollow noreferrer\"><strong>Dennard scaling<\/strong><\/a><strong>.<\/strong><\/p>\n<p id=\"93c5\" data-selectable-paragraph=\"\">Dennard scaling states, roughly, that as transistors get smaller their power density stays constant, so that the power use stays in proportion with area: both voltage and current scale (downward) with length.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-7e46db8 elementor-widget elementor-widget-image\" data-id=\"7e46db8\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/miro.medium.com\/max\/496\/1*X6ZSDSM8pIo3KKo1dSBBMg.png\" alt=\"\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-19b3e2d elementor-widget elementor-widget-text-editor\" data-id=\"19b3e2d\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p style=\"text-align: center;\" data-selectable-paragraph=\"\"><span style=\"font-size: 11px;\"><a href=\"https:\/\/www.cc.gatech.edu\/~hadi\/doc\/paper\/2011-isca-dark_silicon.pdf\" target=\"_blank\" rel=\"noopener nofollow noreferrer\" class=\"broken_link\">Source<\/a>: Dark Silicon and the End of Multicore Scaling, 2011.<\/span><\/p>\n<p id=\"b2dc\" data-selectable-paragraph=\"\">Since around 2005\u20132007, Dennard scaling appears to have broken down, so even though Moore\u2019s law continued for several years after that, it has not yielded dividends in improved performance. The primary reason cited for the breakdown is that at small sizes, current leakage poses greater challenges, and also causes the chip to heat up.<\/p>\n<p id=\"a184\" data-selectable-paragraph=\"\">There have been diverse approaches to heat management, from pumping\u00a0<a href=\"https:\/\/en.wikipedia.org\/wiki\/Fluorinert\" target=\"_blank\" rel=\"noopener nofollow noreferrer\">Fluorinert<\/a>\u00a0through the system, to a hybrid liquid-air cooling system or air cooling with normal air conditioning temperatures. However, these approaches only work up to a point, and after this the only thing that can be done is to select parts of the chip to power on.<\/p>\n<p id=\"5024\" data-selectable-paragraph=\"\">Because of this, sections of the chip sometimes are not powered-on in order to keep the chip temperature in the safe operating range. These regions are referred to as\u00a0<em>dark silicon<\/em>.<\/p>\n<p id=\"1865\" data-selectable-paragraph=\"\">Designs are needed that can dynamically select parts of the chip to power on \u2014 this is an active area of computational research that will become increasingly important as time passes and Moore\u2019s law continues to deviate from its original predictions.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-9368a44 elementor-widget elementor-widget-heading\" data-id=\"9368a44\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h1 class=\"elementor-heading-title elementor-size-default\"><h1 id=\"3a65\" data-selectable-paragraph=\"\">Neuromorphic Computing<\/h1><\/h1>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f371fcf elementor-widget elementor-widget-text-editor\" data-id=\"f371fcf\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"0a65\" data-selectable-paragraph=\"\">Neuromorphic computing is the development of computing systems that mimic the neurobiological architectures present in the nervous system.<\/p>\n<p id=\"fb2e\" data-selectable-paragraph=\"\">A key aspect of neuromorphic engineering is understanding how the morphology of individual neurons, circuits, applications, and overall architectures creates desirable computations, affects how information is represented, influences robustness to damage, incorporates learning and development, adapts to local change (plasticity), and facilitates evolutionary change.<\/p>\n<p id=\"d59c\" data-selectable-paragraph=\"\"><em>Morphic<\/em>\u00a0is defined as in a specific shape or form.\u00a0<em>Neuromorphic<\/em>\u00a0means having behaviors of neurons.<\/p>\n<p id=\"38e2\" data-selectable-paragraph=\"\">In a conference proceeding to the IEEE in 1990 titled\u00a0<em>Neuromorphic Electronic Systems<\/em>, the concept of neuromorphic computing was first introduced. The proceedings stated that:<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-d1216b6 elementor-widget elementor-widget-text-editor\" data-id=\"d1216b6\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<blockquote>\n<p id=\"6195\" data-selectable-paragraph=\"\">\u201cThe human brain has 10\u00b9\u2076 synapses, with a nerve pulse arriving at each synapse per 100ms. This means 10\u00b9\u2076 complex operations per second. At the power dissipation of 1W (instead of 20W for simplicity here), the brain does each operation only 1\/10\u00b9\u2076 J \u2014 In contrast, the ultimate silicon technology will dissipate 1\/10\u2079 J for each operation.\u201d<\/p>\n<\/blockquote>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-131fb46 elementor-widget elementor-widget-text-editor\" data-id=\"131fb46\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"c70f\" data-selectable-paragraph=\"\">Thus, it is expected that by implementing neuromorphic computing, we may be able to achieve computations that are orders of magnitude more energy-efficient than our current systems.<\/p>\n<p id=\"629c\" data-selectable-paragraph=\"\">The push towards energy-efficient computing comes about for multiple reasons, not just for reducing costs \u2014 although that is a reason. This also frees up computations for hardware, allowing the same number of computations to be done on fewer devices, which have cost benefits as well as potential benefits to parallelization. There are also clear environmental benefits to reducing the total energy consumption of devices.<\/p>\n<p id=\"0b46\" data-selectable-paragraph=\"\">There are many examples of early efforts in neuromorphic computing:<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-64fa3f4 elementor-widget elementor-widget-text-editor\" data-id=\"64fa3f4\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<ul>\n \t<li id=\"2fee\" data-selectable-paragraph=\"\">Neurogrid is a multi-chip system developed by Kwabena Boahen and his group at Stanford, 2005 (6,000 synapses).<\/li>\n \t<li id=\"d28d\" data-selectable-paragraph=\"\">Fast Analog Computing with Emergent Transient States (FACETS), 2005 (50 million synapses).<\/li>\n \t<li id=\"9636\" data-selectable-paragraph=\"\">Torres-Huitzil\u2019s FPGA Model, 2005.<\/li>\n<\/ul>\n<p id=\"68d1\" data-selectable-paragraph=\"\">One of the latest efforts is IBM\u2019s TrueNorth. The TrueNorth consumes about 1\/10\u00b9\u00b3 J per operation, rather than 1\/10\u2079 J in general-purpose computing. Recall that for human brain this number is 1\/10\u00b9\u2076 J. Thus we still have three orders of magnitude to go!<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-a3d335a elementor-widget elementor-widget-heading\" data-id=\"a3d335a\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\"><h1 id=\"9ae6\" data-selectable-paragraph=\"\"><strong>Final Comments<\/strong><\/h1><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-18c11db elementor-widget elementor-widget-text-editor\" data-id=\"18c11db\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"0fba\" data-selectable-paragraph=\"\">In summary, the future of computation looks like it will involve speeding up computations to handle the relentless and exponential increase in data production. However, speeding up individual processors is difficult for various reasons, and Moore\u2019s law cannot last forever \u2014 it is becoming increasingly constrained by the limits of heat transfer and quantum mechanics. A greater push will continue to be seen towards parallel computing, especially with more specialized hardware such as GPUs and TPUs, as well as towards more energy-efficient computing which may become possible as we enter into the realm of neuromorphic computing.<\/p>\n<p id=\"aaad\" data-selectable-paragraph=\"\">The burgeoning area of IoT devices presents different challenges, which are trying to be challenged using more compute-central ideas such as edge and fog computing.<\/p>\n<p id=\"d349\" data-selectable-paragraph=\"\"><strong>Disclaimer<\/strong>: I left out quantum computing out of this article as it is not clear to me that this is likely to be a feasible computational option anytime in the near future.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-7f2dc18 elementor-widget elementor-widget-heading\" data-id=\"7f2dc18\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\"><h1 id=\"fab2\" data-selectable-paragraph=\"\">Further Topics<\/h1><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"has_eae_slider elementor-section elementor-top-section elementor-element elementor-element-295f209 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"295f209\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"has_eae_slider elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-fa721f0\" data-id=\"fa721f0\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-dcdeb62 elementor-widget elementor-widget-text-editor\" data-id=\"dcdeb62\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"1bd2\" data-selectable-paragraph=\"\">There are various other topics that are worthy of note for the interested reader. I recommend looking more into these if they pique your interest, they may become important in the near future:<\/p>\n\n<ul>\n \t<li id=\"2a7e\" data-selectable-paragraph=\"\">Automata processors<\/li>\n \t<li id=\"9e58\" data-selectable-paragraph=\"\">Neuromorphic spike-based computing<\/li>\n \t<li id=\"81cb\" data-selectable-paragraph=\"\">AI Accelerators<\/li>\n \t<li id=\"4dc9\" data-selectable-paragraph=\"\">Heterogeneous computing<\/li>\n<\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ede746b elementor-widget elementor-widget-heading\" data-id=\"ede746b\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h1 class=\"elementor-heading-title elementor-size-default\"><h1 id=\"1f3b\" data-selectable-paragraph=\"\">References<\/h1><\/h1>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e93781b elementor-widget elementor-widget-text-editor\" data-id=\"e93781b\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p id=\"ee49\" data-selectable-paragraph=\"\">Monroe, D. (2014). \u201cNeuromorphic computing gets ready for the (really) big time\u201d.\u00a0<a href=\"https:\/\/en.wikipedia.org\/wiki\/Communications_of_the_ACM\" target=\"_blank\" rel=\"noopener nofollow noreferrer\"><em>Communications of the ACM<\/em><\/a>.\u00a0<strong>57<\/strong>\u00a0(6): 13\u201315.\u00a0<a href=\"https:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" target=\"_blank\" rel=\"noopener nofollow noreferrer\">doi<\/a>:<a href=\"https:\/\/doi.org\/10.1145%2F2601069\" target=\"_blank\" rel=\"noopener nofollow noreferrer\" class=\"broken_link\">10.1145\/2601069<\/a><\/p>\n<p id=\"c7d2\" data-selectable-paragraph=\"\">Modha, Dharmendra (August 2014). \u201cA million spiking-neuron integrated circuit with a scalable communication network and interface\u201d.\u00a0<em>Science<\/em>.\u00a0<strong>345<\/strong>\u00a0(6197): 668\u2013673.<\/p>\n<p id=\"c3c0\" data-selectable-paragraph=\"\">Mead, Carver (1990).\u00a0<a href=\"https:\/\/authors.library.caltech.edu\/53090\/1\/00058356.pdf\" target=\"_blank\" rel=\"noopener nofollow noreferrer\">\u201cNeuromorphic electronic systems\u201d<\/a>\u00a0(PDF).\u00a0<em>Proceedings of the IEEE<\/em>.\u00a0<strong>78<\/strong>\u00a0(10): 1629\u20131636.\u00a0<a href=\"https:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" target=\"_blank\" rel=\"noopener nofollow noreferrer\">doi<\/a>:<a href=\"https:\/\/doi.org\/10.1109%2F5.58356\" target=\"_blank\" rel=\"noopener nofollow noreferrer\" class=\"broken_link\">10.1109\/5.58356<\/a><\/p>\n<p id=\"11bd\" data-selectable-paragraph=\"\">Amdahl, Gene M. (1967).\u00a0<a href=\"http:\/\/www-inst.eecs.berkeley.edu\/~n252\/paper\/Amdahl.pdf\" target=\"_blank\" rel=\"noopener nofollow noreferrer\">\u201cValidity of the Single Processor Approach to Achieving Large-Scale Computing Capabilities\u201d<\/a>\u00a0(PDF).\u00a0<em>AFIPS Conference Proceedings<\/em>\u00a0(30): 483\u2013485.\u00a0<a href=\"https:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" target=\"_blank\" rel=\"noopener nofollow noreferrer\">doi<\/a>:<a href=\"https:\/\/doi.org\/10.1145%2F1465482.1465560\" target=\"_blank\" rel=\"noopener nofollow noreferrer\" class=\"broken_link\">10.1145\/1465482.1465560<\/a>.<\/p>\n<p id=\"db28\" data-selectable-paragraph=\"\">Rodgers, David P. (June 1985).\u00a0<a href=\"https:\/\/dl.acm.org\/citation.cfm?id=327215\" target=\"_blank\" rel=\"noopener nofollow noreferrer\" class=\"broken_link\">\u201cImprovements in multiprocessor system design\u201d<\/a>.\u00a0<em>ACM SIGARCH Computer Architecture News<\/em>. New York, NY, USA:\u00a0<a href=\"https:\/\/en.wikipedia.org\/wiki\/Association_for_Computing_Machinery\" target=\"_blank\" rel=\"noopener nofollow noreferrer\">ACM<\/a>.\u00a0<strong>13<\/strong>\u00a0(3): 225\u2013231.\u00a0<a href=\"https:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" target=\"_blank\" rel=\"noopener nofollow noreferrer\">doi<\/a>:<a href=\"https:\/\/doi.org\/10.1145%2F327070.327215\" target=\"_blank\" rel=\"noopener nofollow noreferrer\" class=\"broken_link\">10.1145\/327070.327215<\/a>.\u00a0<a href=\"https:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" target=\"_blank\" rel=\"noopener nofollow noreferrer\">ISBN<\/a>\u00a0<a href=\"https:\/\/en.wikipedia.org\/wiki\/Special:BookSources\/0-8186-0634-7\" target=\"_blank\" rel=\"noopener nofollow noreferrer\">0\u20138186\u20130634\u20137<\/a>.\u00a0<a href=\"https:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" target=\"_blank\" rel=\"noopener nofollow noreferrer\">ISSN<\/a>\u00a0<a href=\"https:\/\/www.worldcat.org\/issn\/0163-5964\" target=\"_blank\" rel=\"noopener nofollow noreferrer\">0163\u20135964<\/a><\/p>\n<p id=\"10b4\" data-selectable-paragraph=\"\">Dennard, Robert H.; Gaensslen, Fritz; Yu, Hwa-Nien; Rideout, Leo; Bassous, Ernest; LeBlanc, Andre (1974).\u00a0<a href=\"http:\/\/www.ece.ucsb.edu\/courses\/ECE225\/225_W07Banerjee\/reference\/Dennard.pdf\" target=\"_blank\" rel=\"noopener nofollow noreferrer\">\u201cDesign of ion-implanted MOSFET\u2019s with very small physical dimensions\u201d<\/a>\u00a0(PDF).\u00a0<em>IEEE Journal of Solid-State Circuits<\/em>.\u00a0<strong>SC-9<\/strong>\u00a0(5).<\/p>\n<p id=\"98dc\" data-selectable-paragraph=\"\">Greene, Katie (2011).\u00a0\u201cA New and Improved Moore\u2019s Law: Under \u201cKoomey\u2019s law,\u201d it\u2019s efficiency, not power, that doubles every year and a half\u201d.\u00a0<a href=\"https:\/\/en.wikipedia.org\/wiki\/Technology_Review\" target=\"_blank\" rel=\"noopener nofollow noreferrer\">Technology Review<\/a>.<\/p>\n<p id=\"57c8\" data-selectable-paragraph=\"\">Bohr, Mark (2007).\u00a0<a href=\"http:\/\/www.eng.auburn.edu\/~agrawvd\/COURSE\/READING\/LOWP\/Boh07.pdf\" target=\"_blank\" rel=\"noopener nofollow noreferrer\">\u201cA 30 Year Retrospective on Dennard\u2019s MOSFET Scaling Paper\u201d<\/a>\u00a0(PDF). Solid-State Circuits Society. Retrieved January 23, 2014.<\/p>\n<p id=\"d1a7\" data-selectable-paragraph=\"\">Esmaeilzedah, Hadi; Blem, Emily; St. Amant, Renee; Sankaralingam, Kartikeyan; Burger, Doug (2012).\u00a0\u201cDark Silicon and the end of multicore scaling\u201d\u00a0(PDF).<\/p>\n<p id=\"1eb3\" data-selectable-paragraph=\"\">Hruska, Joel (2012).\u00a0<a href=\"http:\/\/www.extremetech.com\/computing\/116561-the-death-of-cpu-scaling-from-one-core-to-many-and-why-were-still-stuck\" target=\"_blank\" rel=\"noopener nofollow noreferrer\" class=\"broken_link\">\u201cThe death of CPU scaling: From one core to many \u2014 and why we\u2019re still stuck\u201d<\/a>.\u00a0<a href=\"https:\/\/en.wikipedia.org\/wiki\/ExtremeTech\" target=\"_blank\" rel=\"noopener nofollow noreferrer\">ExtremeTech<\/a>. Retrieved January 23, 2014.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>The future of computation looks like it will involve speeding up computations to handle the relentless and exponential increase in data production. However, speeding up individual processors is difficult for various reasons, and Moore&rsquo;s law cannot last forever &mdash; it is becoming increasingly constrained by the limits of heat transfer and quantum mechanics. A greater push will continue to be seen towards parallel computing, especially with more specialized hardware such as GPUs and TPUs, as well as towards more energy-efficient computing which may become possible as we enter into the realm of neuromorphic computing.<strong> <\/strong><\/p>\n","protected":false},"author":682,"featured_media":3272,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[183],"tags":[92],"ppma_author":[3471],"class_list":["post-2180","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-ml","tag-machine-learning"],"authors":[{"term_id":3471,"user_id":682,"is_guest":0,"slug":"matthew-stewart","display_name":"Matthew Stewart","avatar_url":"https:\/\/www.experfy.com\/blog\/wp-content\/uploads\/2020\/04\/medium_c57055f3-5301-4262-af65-4cc7d40cbf3d-150x150.jpg","user_url":"https:\/\/criticalfutureglobal.com\/","last_name":"Stewart","first_name":"Matthew","job_title":"","description":"Matthew Stewart is a Machine Learning consultant on AI for\u00a0<a href=\"https:\/\/www.criticalfutureglobal.com\/\" target=\"_blank\" rel=\"noopener\">Critical Future<\/a>, and machine learning engineer at Scalable Magic, an AI-based digital media startup. He is also a Graduate Teaching Assistant and a Ph.D. Candidate at Harvard University."}],"_links":{"self":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/2180","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/users\/682"}],"replies":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/comments?post=2180"}],"version-history":[{"count":6,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/2180\/revisions"}],"predecessor-version":[{"id":35743,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/2180\/revisions\/35743"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media\/3272"}],"wp:attachment":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media?parent=2180"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/categories?post=2180"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/tags?post=2180"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=2180"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}