{"id":1575,"date":"2019-03-14T03:32:34","date_gmt":"2019-03-14T03:32:34","guid":{"rendered":"http:\/\/kusuaks7\/?p=1180"},"modified":"2023-08-21T14:09:01","modified_gmt":"2023-08-21T14:09:01","slug":"the-cloud-is-becoming-ais-bottleneck","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/ai-ml\/the-cloud-is-becoming-ais-bottleneck\/","title":{"rendered":"The cloud is becoming AI\u2019s bottleneck"},"content":{"rendered":"
\"Electronic<\/figure>\n

Image credit:\u00a0Depositphotos<\/a><\/span><\/p>\n

In the past decade, artificial intelligence has escaped the confines of research labs and found its way into many of the things we do every day. From online shopping and content recommendation to healthcare and self-driving cars, we are interacting with AI algorithms, in many cases without even knowing it.<\/p>\n

But we\u2019ve barely scratched the surface, many believe, and artificial intelligence has much more to offer. Ironically, one of the things that is preventing AI from realizing its full potential is the cloud, one of the main technologies that helped usher AI into the mainstream.<\/p>\n

\u201cThe reason we still don\u2019t see AI everywhere is not that the algorithms or technology are not there. The main reason is cloud dependency,\u201d says Ali Farhadi, co-founder and CXO at Xnor, a Seattle-based AI hardware startup.<\/p>\n

Edge AI<\/a>, the collective name for hardware and software that enable the performance of AI tasks without a link to the cloud, has gained much steam in the past few years. And as Farhadi and many other experts believe, the edge (or the fog<\/a>, as some like to call it) will unlock many new AI applications that weren\u2019t possible before.<\/p>\n

The costs of cloud-based artificial intelligence<\/h2>\n

AI owes its recent rise in popularity to advances in\u00a0deep learning algorithms and neural networks<\/a>. But one of the main limits of neural networks is their requirements for vast amount of compute resources, which are mostly available in public cloud platforms.<\/p>\n

\u201cAI algorithms are very demanding on compute, memory and power. Those have been very limiting factor at scaling AI use cases,\u201d Farhadi says.<\/p>\n

There are many ways AI\u2019s dependency on the cloud becomes problematic. \u201cMany use cases of AI will happen at the true edge. They need to run 24\/7 and multiple times per second,\u201d Farhadi says, adding that running those many AI inferences in the cloud would be too costly for many use cases and businesses.<\/p>\n

Connectivity to the cloud also imposes costs on the devices that will be running the applications at the edge. Every one of those AI-powered devices will need expensive communication hardware modules as well as a network infrastructure that can support its connection to the cloud. For instance, imagine wanting to deploy an array of smart sensors in a large industrial complex. The individual costs of those devices plus all the switches and networking devices that will connect them to the cloud are truly excessive.<\/p>\n

Even in smaller and well-connected settings such as smart homes, AI-powered devices impose costs that most people can\u2019t bear. An interesting example is smart security cameras, devices that use AI algorithms to detect intruders or safety incidents.<\/p>\n

\u201cThese things cost $200 per device,\u201d Farhadi says. \u201cIn a true smart home, I actually have to have 20-30 cameras. I would go bankrupt if I wanted to install these many devices in my home. And there will also be a constant running cost of AI computation in the cloud.\u201d<\/p>\n

The limits of AI in the cloud<\/h2>\n

\"Cloud<\/p>\n

Costs are not the only problem of cloud-based artificial intelligence. In many settings, a connection to the cloud is either non-present or unstable. This limits some of the very important use cases of AI such as agriculture, where\u00a0computer vision algorithms<\/a>\u00a0and other AI techniques can help in precision farming. But farms are usually located in areas where getting a stable broadband internet connection is a real challenge.<\/p>\n

Another example is automated rescue drones, which need to work in environments where communications infrastructure is weak or has been damaged due to natural disasters. Again, without the AI cloud, the drones won\u2019t be able to function properly.<\/p>\n

Latency is another challenge of the cloud. As AI algorithms find their way into the physical world, they need to perform many of their tasks in real time. An example is\u00a0self-driving cars<\/a>, which use deep learning algorithms to detect cars, objects and pedestrians. In some situations, they need to make split-second decisions to\u00a0avoid fatal collisions<\/a>.<\/p>\n

\u201cI would never get into a car that is going to be driven by an algorithm that runs in the cloud,\u201d Farhadi says, naming several ways that things can go wrong when a car\u2019s AI algorithms are being processed in the cloud.<\/p>\n

Another issue that has turned into a pain point for the AI industry is the\u00a0privacy concerns of sending your data to the cloud<\/a>. Many people are not comfortable with purchasing devices and applications that are constantly streaming audio and video from their home to the cloud to be processed by an AI algorithm.<\/p>\n

\u201cI don\u2019t have any smart security cameras, because I just don\u2019t want a picture of my daughter\u2019s room to be uploaded somewhere in the cloud, even though Google and others will say this is very secure,\u201d Farhadi says. \u201cIt\u2019s just basically a major security concern.\u201d<\/p>\n

AI\u2019s power consumption problem<\/h2>\n

Having a constant connection to the internet and streaming data to the cloud consumes a lot of electricity. This has become a pain point for many use cases of artificial intelligence, especially in fields like wearables and always-sensing devices that don\u2019t have a mains power supply and are running on batteries.<\/p>\n

\u201cIf you have to replace a battery every three or six months, people won\u2019t want to use it,\u201d Farhadi says. This is especially true in settings like smart cities, where the numbers of devices can reach hundreds of millions.<\/p>\n

Power consumption also creates an environmental problem. \u201cIf the future we\u2019re depicting is true, we\u2019ll be surrounded with lots of devices that are going to make our lives much easier and simpler. That means we\u2019re going to have billions of AI-powered devices. And if you want to do it with a cloud-based AI solution, the carbon footprint of doing that many inferences in the cloud would damage the planet significantly,\u201d Farhadi explains. \u201cWhen you think about these problems and scale, power has become one of the biggest issues that we have.\u201d<\/p>\n

The power of edge AI<\/h2>\n

\"artificial<\/p>\n

In the past years, AI hardware has become a growing industry, giving rise to an\u00a0array of startups<\/a>\u00a0that are creating specialized chips for performing AI tasks. Many of the efforts are focused on\u00a0bringing AI closer to the edge<\/a>\u00a0and reducing dependencies on the cloud.<\/p>\n

Xnor is one of several companies developing edge AI devices, but what makes them different is their focus on costs and energy consumption. Two years, ago, the company created an\u00a0object detection AI that could run on a Raspberry Pi Zero<\/a>. The model running on the Raspberry Pi was a 16-layer object detection neural network. In other words, it was a very expensive algorithm running on a cheap and weak platform. They called it the five-dollar deep learning machine.<\/p>\n

More recently, Xnor took a step further and developed a smaller and more efficient device that replaces the $5 Pi Zero with a two-dollar field-programmable gate array (FPGA). Xnor\u2019s new edge AI contraption runs a 30-layered convolutional network, a fairly complicated AI model, and can analyze at 32 framed per second.<\/p>\n

However, what makes the new device interesting is that it\u00a0runs solely on solar power<\/a>\u00a0and needs no other power supply. According to Farhadi, with a coin-cell battery, the device could run round the clock for 32 years.<\/p>\n

\u201cIf you want to scale everywhere, you have to be able to run on commodity platforms,\u201d Farhadi says.<\/p>\n

Low-cost, low-power AI devices that can run independent of the cloud can open the way for many new use cases. They also solve the privacy issue associated with many current AI solutions. Xnor\u2019s smart camera only needs to send a few bits every time it detects an object.<\/p>\n

Since its data transfer requirements for edge AI devices are very low, they can run on communication platforms such as LoRa, which can send data at low rates as far as 20 miles. This again reduces the costs of networking equipment as well as power consumption levels on the device.<\/p>\n

\u201cWe\u2019re going after enabling everything around us to become smarter than it is with literally no cost, on existing compute platforms, maybe no battery or a battery every decade or so,\u201d Farhadi says.<\/p>\n

An area where low-power edge AI can help are smart wearables, where privacy and power consumption are both serious issues.<\/p>\n

Settings such as smart homes and smart cities, where scale is challenge, can also benefit from the cost efficiency of innovations such as Xnor\u2019s low-cost deep learning device. Farhadi says that replacing the FPGA for an application-specific integrated circuit (ASIC) would drop the cost from two dollars to a few cents.<\/p>\n

Also, in environments where you either have to wire thousands of devices or change their batteries every so often, having self-sufficient AI devices that can run on their own power source for decades save a lot of manual maintenance effort.<\/p>\n

\u201cWe want to revolutionize the way people think about AI at the edge, and maybe change the definition of AI at the edge,\u201d Farhadi says.<\/p>\n

Finding the balance between cloud and edge AI<\/h2>\n
\"Low<\/figure>\n

Image credit:\u00a0Depositphotos<\/a><\/span><\/p>\n

Does innovation at the edge mean that cloud AI will disappear? Probably not.<\/p>\n

\u201cCloud was extremely helpful in getting AI out of the labs and into the market. It served AI really well. One of the reasons we saw such a boost in AI was the cloud,\u201d Farhadi says.<\/p>\n

Farhadi believes that some artificial intelligence applications will remain native to the cloud, including applications that need a lot of updates and applications that require a lot of data bandwidth to and from the device.<\/p>\n

Farhadi also acknowledges Edge AI doesn\u2019t come without trade-offs. For one thing, the flexibility of and compute power of cloud can\u2019t be replicated at the edge. \u201cYou can\u2019t analyze video at hundreds of frames per second on a two-dollar device,\u201d he says.<\/p>\n

But for most applications, the functionality offered by resource-constrained, edge AI devices is more than enough.<\/p>\n

\u201cWe\u2019re not advocating that everything has to go to the edge. But there are a lot of AI applications that don\u2019t really need to run in the cloud. That\u2019s what we\u2019re after,\u201d Farhadi says.<\/p>\n

In the long run, artificial intelligence will be divided into cloud-native, edge-native and hybrid applications. The point, Farhadi believes, is to understand the opportunities and power that lies in edge AI and to choose solutions that best serve each use case. \u201cCloud is not the only solution. There\u2019s a spectrum,\u201d he says.<\/p>\n","protected":false},"excerpt":{"rendered":"

We are interacting with AI algorithms, in many cases without even knowing it.Many believe artificial intelligence has much more to offer. Ironically, one of the things that is preventing AI from realizing its full potential is the cloud, one of the main technologies that helped usher AI into the mainstream. The reason we still don’t see AI everywhere is not that the algorithms or technology are not there. The main reason is cloud dependency.<\/p>\n","protected":false},"author":109,"featured_media":4143,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[183],"tags":[97],"ppma_author":[1946],"class_list":["post-1575","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-ml","tag-artificial-intelligence"],"authors":[{"term_id":1946,"user_id":109,"is_guest":0,"slug":"ben-dickson","display_name":"Ben Dickson","avatar_url":"https:\/\/www.experfy.com\/blog\/wp-content\/uploads\/2020\/04\/medium_8aaf6bea-c4c1-455f-8156-8007d70910f8-150x150.jpg","user_url":"https:\/\/bdtechtalks.com\/","last_name":"Dickson","first_name":"Ben","job_title":"","description":"Ben Dickson is an experienced software engineer and tech blogger. He contributes regularly to major tech websites such as the Next Web, the Daily Dot, PCMag.com, Cointelegraph, VentureBeat, International Business Times UK, and The Huffington Post."}],"_links":{"self":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/1575","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/users\/109"}],"replies":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/comments?post=1575"}],"version-history":[{"count":2,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/1575\/revisions"}],"predecessor-version":[{"id":29716,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/1575\/revisions\/29716"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media\/4143"}],"wp:attachment":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media?parent=1575"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/categories?post=1575"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/tags?post=1575"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=1575"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}