{"id":8537,"date":"2020-06-16T07:51:33","date_gmt":"2020-06-16T07:51:33","guid":{"rendered":"https:\/\/www.experfy.com\/blog\/?p=8537"},"modified":"2023-12-05T17:59:41","modified_gmt":"2023-12-05T17:59:41","slug":"what-the-faq-are-ai-anns-ml-dl-and-dnns","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/ai-ml\/what-the-faq-are-ai-anns-ml-dl-and-dnns\/","title":{"rendered":"What the FAQ are AI, ANNs, ML, DL, and DNNs?"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\n

In this column we briefly introduce the concepts of artificial intelligence (AI), artificial neural networks (ANNs), machine learning (ML), deep learning (DL), and deep neural networks (DNNs).<\/strong><\/em><\/p>\n\n\n

\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\n

There are so many new technologies bouncing around these days that it\u2019s difficult to wrap one\u2019s brain around all the terminology and abbreviations, so I thought it might be a good idea to provide some easy-to-read overviews.<\/p>\n\n\n\n

In this column, we are going to briefly introduce the concepts of artificial intelligence (AI), artificial neural networks (ANNs), machine learning (ML), deep learning (DL), and deep neural networks (DNNs). See also:<\/strong> What the FAQ are VR, MR, AR, DR, AV, and HR?<\/a> See also: What the FAQ are the IoT, IIoT, IoHT, and AIoT?<\/a> \u00a0<\/strong><\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

Early Musings on AI<\/h2>\n<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\n

Way back in the middle of the 1800s, The English polymath \u2013 inventor, philosopher, mathematician, and mechanical engineer \u2013 Charles Babbage (1791\u20131871) conceived the idea of a steam-powered mechanical computer that he dubbed the Analytical Steam Engine.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t“What the FAQ are AI, ANNs, ML, DL, and DNNs?” <\/a>\n
Charles Babbage (Click image to see a larger version \u2014 Image source: Max Maxfield & Alvin Brown)\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\n

Babbage worked on his Analytical Engine from around 1830 until he died, but sadly it was never completed. It is often said that Babbage was a hundred years ahead of his time and that the technology of the day was inadequate for the task. However, in his book, Engines of the Mind<\/em><\/a>, Joel Shurkin refuted this, stating:<\/p>\n\n\n\n

\n

One of Babbage\u2019s most serious flaws was his inability to stop tinkering. No sooner would he send a drawing to the machine shop than he would find a better way to perform the task and would order work stopped until he had finished pursuing the new line. By and large this flaw kept Babbage from ever finishing anything.<\/p>\n<\/blockquote>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\n

Working with Babbage was Augusta Ada Lovelace (1815\u20131852), the daughter of the English poet Lord Byron. Ada, who was a splendid mathematician and one of the few people who fully understood Babbage\u2019s vision, created a program for the Analytical Engine. Had the machine ever actually worked, this program would have been able to compute the mathematical sequence known as Bernoulli numbers.<\/p>\n\n\n\n

Based on this work, Ada is now credited as being the first computer programmer and, in 1979, a modern programming language was named ADA in her honor. (In their spare time, Babbage and Ada also attempted to create a system for predicting the winners of horse races, but it is said that they lost a lot of money!)<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Of more interest to us here is the fact that Ada mused on the possibilities of something akin to AI. Babbage thought of his engine only as a means for performing mathematical calculations. He hated the tedium associated with performing calculations by hand and \u2014 as far back as 1812 \u2014 he is reported as saying, \u201cI wish to God these calculations had been performed by steam.\u201d By comparison, Ada wrote notes about the possibilities of computers one day using numbers as symbols to represent other things like musical notes, and she went so far as to speculate of machines \u201chaving the ability to compose elaborate and scientific pieces of music of any degree of complexity or extent.\u201d<\/p>\n\n\n\n

Around 100 years after Ada\u2019s ponderings, the English mathematician, computer scientist, logician, cryptanalyst, philosopher, and theoretical biologist, Alan Mathison Turing (1912\u20131954), was hard at work at Bletchley Park during WWII leading the effort to crack the German Enigma and Lorenz ciphers. After the war, in his seminal paper, Computing Machinery and Intelligence<\/em><\/a> (1950), Turing considered the question, \u201cCan Computers Think?\u201d<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\n

I fear Turing\u2019s cogitations and ruminations didn\u2019t exactly cheer him up, because during a lecture the following year he said:<\/p>\n\n\n\n

\n

\u201cIt seems probable that once the machine thinking method had started, it would not take long to outstrip our feeble powers\u2026 They would be able to converse with each other to sharpen their wits. At some stage therefore, we should have to expect the machines to take control.\u201d<\/p>\n<\/blockquote>\n\n\n\n

As part of his Computing Machinery and Intelligence paper, Turing suggested a game called the \u201cImitation Game\u201d (hence the highly recommended 2014 movie<\/a> of the same name). This game, which eventually became known as the Turing test<\/a>, is a method of inquiry \u2014 based on a computer\u2019s ability to communicate indistinguishably from a human \u2014 for determining whether or not a computer is capable of thinking like a human being.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Unfortunately, in June 2012, at an event marking what would have been Alan Turing\u2019s 100th birthday, a chatbot called Eugene Goostman successfully convinced 29% of its judges that it was human. This prompted Jordan Pearson to pen a column, Forget Turing, the Lovelace Test Has a Better Shot at Spotting AI<\/em><\/a>.<\/p>\n\n\n\n

In this column, we are introduced to the Lovelace test<\/a>, which was designed by a team of computer scientists in the early 2000s, and which is based on Ada\u2019s argument that, \u201cUntil a machine can originate an idea that it wasn\u2019t designed to, it can\u2019t be considered intelligent in the same way humans are.\u201d \u00a0<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

\n

The Dartmouth Workshop and Beyond<\/h2>\n<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\n

The Dartmouth Workshop<\/a> \u2014 more formally, the Dartmouth Summer Research Project on Artificial Intelligence<\/em> \u2014 took place in 1956. This gathering is now considered to be the founding event of the field of artificial intelligence as we know and love it today.<\/p>\n\n\n\n

Having said this, AI stayed largely in the realm of academia until circa the 2010s (pronounced \u201ctwenty-tens\u201d), at which point a combination of algorithmic developments coupled with advances in processing technologies caused AI to explode onto the scene with gusto, abandon, a fanfare of trumpets (possibly trombones), and \u2013 of course \u2014 a cornucopia of abbreviations.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\n

Artificial Intelligence (AI):<\/strong> In computer science, artificial intelligence (AI), sometimes called machine intelligence, is intelligence demonstrated by machines, in contrast to the natural intelligence displayed by humans. AI can refer to any device that perceives its environment and takes actions that maximize its chance of successfully achieving its goals. A more elaborate definition characterizes AI as a system\u2019s ability to correctly interpret external data, to learn from such data, and to use those learnings to achieve specific goals and tasks through flexible adaptation.<\/p>\n\n\n\n

As part of all this, it\u2019s become common to talk about \u201ccognitive systems,\u201d which refers to systems that are capable of learning from their interactions with data and humans and modifying their behavior accordingly \u2014 essentially continuously reprogramming themselves. (It is believed that the majority of embedded systems will have cognitive capabilities in the not-so-distant future; see also What the FAQ is an Embedded System?<\/a>).<\/p>\n\n\n\n

Artificial Neural Networks (ANNs):<\/strong> These are computing systems that are inspired by, but not identical to, the biological neural networks that constitute animal brains. An ANN is based on a collection of connected units or nodes called artificial neurons (ANs), which loosely model the neurons in a biological brain. Each connection, like the synapses in a biological brain, can transmit a signal to other neurons. When an artificial neuron receives one or more signals, it processes them and decides whether or not to signal any neurons connected to it.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\n

A common form of ANN is formed from layers of ANs, with the outputs from each layer feeding the inputs to the next layer. Early ANNs were limited to only a few layers, each containing only a handful of ANs, but today\u2019s ANNs can have hundreds or thousands of layers, each comprising thousands of ANs. Furthermore, early systems supported only linear networks with a single layer per level, while today\u2019s networks can support multiple inputs and\/or multiple outputs along with multiple layers per level.<\/p>\n\n\n\n

Such systems \u201clearn\u201d to perform tasks by considering examples, generally without being programmed with task-specific rules. In the case of image recognition, for example, they might learn to identify images that contain flowers by analyzing example images that have been labeled by humans as \u201cflower\u201d (augmented with the type of flower) or \u201cno flower,\u201d after which they can use the results to identify flowers in other images.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\n

A high-level overview of such a system is as follows. We start by defining the network architecture using a framework like Caffe<\/a> or TensorFlow<\/a>. We then train our network using hundreds of thousands, or millions, of tagged images. At this stage, the network is based on floating-point calculations and weights (coefficients). Next, we use a network converter to translate the network into a smaller, \u201clighter\u201d fixed-point representation in a form suitable for deployment on a physical system involving either a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), a System-on-Chip (SoC), or \u2013 more recently \u2013 an integrated circuit (\u201csilicon chip\u201d) that is custom designed to perform AI applications. Finally, we use a camera to feed new images to the deployed inference engine in the hope that it will correctly identify the contents of these images.<\/p>\n\n\n\n

Deep Neural Networks (DNNs):<\/strong> A deep neural network (DNN) is an artificial neural network (ANN) with a lot of layers of artificial neurons between the input and output layers. A convolutional neural network (CNN) is one form of implementation, and CNNs are currently the method of choice for processing visual and other two-dimensional data.<\/p>\n\n\n\n

DNNs are typically feedforward networks in which data flows from the input layer to the output layer without looping back. By comparison, recurrent neural networks (RNNs), in which data can flow in any direction, are better suited for applications such as language modeling.<\/p>\n\n\n\n

Machine Learning (ML) and Deep Learning (DL):<\/strong> Machine learning (ML) is the scientific study of algorithms and statistical models that computer systems use to perform a specific task without using explicit instructions, relying instead on patterns and inference. Machine learning is regarded as being a subset of artificial intelligence. Machine learning algorithms build a mathematical model based on sample data, known as \u201ctraining data,\u201d in order to make predictions or decisions without being explicitly programmed to perform the task.<\/p>\n\n\n\n

In the same way that machine learning is seen as a subset of artificial intelligence, deep learning (DL) is regarded as a subset of machine learning. Deep learning can be supervised, semi-supervised, or unsupervised. Deep learning architectures such as DNNs, CNNs, and RNNs have been applied to fields including computer vision, speech recognition, natural language processing, audio recognition, social network filtering, machine translation, bioinformatics, drug design, medical image analysis, material inspection, and board game programs, where they have produced results comparable to \u2014 in some cases superior to \u2014 human experts.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"

In this column we briefly introduce the concepts of artificial intelligence (AI), artificial neural networks (ANNs), machine learning (ML), deep learning (DL), and deep neural networks (DNNs).<\/p>\n","protected":false},"author":760,"featured_media":8538,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[183],"tags":[226,319,97,297,206,321,320,322,92,225],"ppma_author":[3615],"class_list":["post-8537","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-ml","tag-ai","tag-ann","tag-artificial-intelligence","tag-artificial-neural-networks","tag-deep-learning","tag-deep-neural-networks","tag-dl","tag-dnn","tag-machine-learning","tag-ml"],"authors":[{"term_id":3615,"user_id":760,"is_guest":0,"slug":"clive-max-maxfield-2-2","display_name":"Clive Maxfield","avatar_url":"https:\/\/www.experfy.com\/blog\/wp-content\/uploads\/2020\/04\/medium_d96bfe3c-1e79-40e7-98cd-26c9c80fb5b5-150x150.jpg","user_url":"https:\/\/www.clivemaxfield.com\/coolbeans\/","last_name":"Maxfield","first_name":"Clive","job_title":"","description":"Clive (Max) Maxfield is Collector and Communicator of Technological Information at Maxfield High-Tech Consulting. He wrote books on Electronics, Computing, FPGAs, Mathematics, and 3D Graphics. In the past, he had acted as Contributing Editor, Editor, and Editor-in-Chief at a variety of publications, including EETimes.com, Embedded.com, and EEWeb.com."}],"_links":{"self":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/8537","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/users\/760"}],"replies":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/comments?post=8537"}],"version-history":[{"count":5,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/8537\/revisions"}],"predecessor-version":[{"id":34693,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/8537\/revisions\/34693"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media\/8538"}],"wp:attachment":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media?parent=8537"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/categories?post=8537"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/tags?post=8537"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=8537"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}