{"id":2060,"date":"2019-11-11T01:08:09","date_gmt":"2019-11-11T01:08:09","guid":{"rendered":"http:\/\/kusuaks7\/?p=1665"},"modified":"2024-02-29T10:28:01","modified_gmt":"2024-02-29T10:28:01","slug":"what-is-neuromorphic-computing","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/ai-ml\/what-is-neuromorphic-computing\/","title":{"rendered":"What is neuromorphic computing?"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tIn July, a group of artificial intelligence researchers showcased a self-driving bicycle that could navigate around obstacles, follow a person, and respond to voice commands. While the self-driving bike itself was of little use, the AI technology behind it was remarkable. Powering the bicycle was a neuromorphic chip, a special kind of AI computer.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tNeuromorphic computing is not new. In fact, it was first proposed in the 1980s. But recent developments in the artificial intelligence industry have renewed interest in neuromorphic computers.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tThe growing popularity of\u00a0deep learning and neural networks<\/a>\u00a0has spurred a race to develop AI hardware specialized for neural network computations. Among the handful of trends that have emerged in the past few years is neuromorphic computing, which has shown promise because of its similarities to biological and artificial neural networks.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

How deep neural networks work<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tAt the heart of recent advances in artificial intelligence are\u00a0artificial neural networks<\/a>\u00a0(ANN), AI software that roughly follows the structure of the human brain. Neural networks are composed of artificial neurons, tiny computation units that perform simple mathematical functions.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tArtificial neurons aren\u2019t of much use alone. But when you stack them up in layers, they can perform remarkable tasks, such as detecting objects in images and transforming voice audio to text. Deep neural networks can contain hundreds of millions of neurons, spread across dozens of layers.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

The structure of an artificial neuron, the fundamental component of artificial neural networks (source: Wikipedia)<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tWhen training a deep learning algorithm, developers run many examples through the neural network along with the expected result. The AI model adjusts each of the artificial neurons as it reviews more and more data. Gradually it becomes more accurate at the specific tasks it has been designed for, such as detecting cancer in slides or flagging fraudulent bank transactions.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

The challenges of running neural networks on traditional hardware<\/h2>\n<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tTraditional computers rely on one or several central processing units (CPUs). CPUs pack a lot of power and can perform complex operations at fast speeds. Given the distributed nature of neural networks, running them on classic computers is cumbersome. Their CPUs must emulate millions of artificial neurons through registers and memory locations, and calculate each of them in turn.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tGraphics Processing Units (GPUs), the hardware used for games and 3D software, can do a lot of parallel processing and are especially good at performing matrix multiplication, the core operation of neural networks. GPU arrays have proven to be very useful in neural network operations.\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tThe rise in popularity of neural networks and deep learning have been a boon to GPU manufacturers. Graphics hardware company Nvidia has seen its stock price rise in value severalfold in the past few years.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tHowever, GPUs also lack the physical structure of neural networks and must still emulate neurons in software, albeit at a breakneck speed. The dissimilarities between GPUs and neural networks cause a lot of inefficiencies, such as excessive power consumption.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

Neuromorphic chips<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Intel\u2019s neuromorphic chip Loihi (source:\u00a0Intel<\/a>)<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tContrary to general-purpose processors, neuromorphic chips are physically structured like artificial neural networks. Every neuromorphic chip consists of many small computing units that correspond to an artificial neuron. Contrary to CPUs, the computing units in neuromorphic chips can\u2019t perform a lot of different operations. They have just enough power to perform the mathematical function of a single neuron.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tAnother essential characteristic of neuromorphic chips is the physical connections between artificial neurons. These connections make neuromorphic chips more like organic brains, which consist of biological neurons and their connections, called synapses. Creating an array of physically connected artificial neurons is what gives neuromorphic computers their real strength.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tThe structure of neuromorphic computers makes them much more efficient at training and running neural networks. They can run AI models at a faster speed than equivalent CPUs and GPUs while consuming less power. This is important since power consumption is already\u00a0one of AI\u2019s essential challenges<\/a>.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tThe smaller size and low power consumption of neuromorphic computers make them suitable for use cases that require to\u00a0run AI algorithms at the edge<\/a>\u00a0as opposed to the cloud.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tNeuromorphic chips are characterized by the number of neurons they contain. The Tianjic chip, the neuromorphic chip used in the self-driving bike mentioned at the beginning of this article, contained about 40,000 artificial neurons and 10 million synapses in an area of 3.8 square millimeters. Compared to a GPU running an equal number of neurons, Tianjic performed 1.6-100x faster and consume 12-10,000x less power.\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tBut 40,000 is a limited number of neurons,\u00a0as much as the brain of a fish<\/a>. The human brain contains approximately 100 billion neurons.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tAlexNet, a popular image classification network used in many applications, has more than 62 million parameters.\u00a0OpenAI\u2019s GPT-2 language model<\/a>\u00a0contains more than one billion parameters.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tBut the Tianjic chip was more of a proof of concept than a neuromorphic computer purposed for commercial uses. Other companies have already been developing neuromorphic chips ready to be used in different AI applications.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tOne example is Intel\u2019s Loihi chips and Pohoiki Beach computers. Each Loihi chip contains 131,000 neurons and 130 million synapses. The Pohoiki computer, introduced in July, packs 8.3 million neurons. The Pohoiki delivers 1000x better performance and is 10,000x more energy efficient than equivalent GPUs.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

Neuromorphic computing and artificial general intelligence (AGI)<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tIn a\u00a0paper<\/a>\u00a0published in\u00a0Nature<\/em>, the AI researchers who created the Tianjic chip observed that their work could help bring us closer to artificial general intelligence (AGI). AGI is supposed to replicate the capabilities of the human brain.\u00a0Current AI technologies are narrow<\/a>: they can solve specific problems and are bad at generalizing their knowledge.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tFor instance, an\u00a0AI model designed to play a game like StarCraft II<\/a>\u00a0will be helpless when introduced to another game, say Dota 2. That will require a\u00a0totally different AI algorithm<\/a>.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tAccording to Tianjic designers, their AI chip was able to solve multiple problems, including object detection, speech recognition, navigation, and obstacle avoidance, all in a single device.\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tBut while neuromorphic chips might bring us a step closer to emulating the human brain, we still have a long way to go. Artificial general intelligence requires more than bundling several narrow AI models together.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tArtificial neural networks, at their core, are statistical machines, and statistics can\u2019t help to solve problems that require reasoning, understanding, and general problem\u2013solving. Examples include\u00a0natural language understanding and navigating open worlds<\/a>.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tCreating more efficient ANN hardware won\u2019t solve those problems. But perhaps having AI chips that look much more like our brains will open new pathways to understand and create intelligence.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"

Neuromorphic computing is not new. It was first proposed in the 1980s. But recent developments in the artificial intelligence industry have renewed interest in neuromorphic computers. The growing popularity of deep learning and neural networks has spurred a race to develop AI hardware specialized for neural network computations. Among the handful of trends that have emerged in the past few years is neuromorphic computing, which has shown promise because of its similarities to biological and artificial neural networks.<\/p>\n","protected":false},"author":109,"featured_media":2692,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[183],"tags":[97],"ppma_author":[1946],"class_list":["post-2060","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-ml","tag-artificial-intelligence"],"authors":[{"term_id":1946,"user_id":109,"is_guest":0,"slug":"ben-dickson","display_name":"Ben Dickson","avatar_url":"https:\/\/www.experfy.com\/blog\/wp-content\/uploads\/2020\/04\/medium_8aaf6bea-c4c1-455f-8156-8007d70910f8-150x150.jpg","user_url":"https:\/\/bdtechtalks.com\/","last_name":"Dickson","first_name":"Ben","job_title":"","description":"Ben Dickson is an experienced software engineer and tech blogger. He contributes regularly to major tech websites such as the Next Web, the Daily Dot, PCMag.com, Cointelegraph, VentureBeat, International Business Times UK, and The Huffington Post."}],"_links":{"self":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/2060","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/users\/109"}],"replies":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/comments?post=2060"}],"version-history":[{"count":4,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/2060\/revisions"}],"predecessor-version":[{"id":36190,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/2060\/revisions\/36190"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media\/2692"}],"wp:attachment":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media?parent=2060"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/categories?post=2060"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/tags?post=2060"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=2060"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}