{"id":673,"date":"2018-05-09T03:48:58","date_gmt":"2018-05-09T00:48:58","guid":{"rendered":"http:\/\/kusuaks7\/?p=278"},"modified":"2025-06-19T10:28:15","modified_gmt":"2025-06-19T10:28:15","slug":"preventing-iot-data-waste-with-the-intelligent-edge","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/iot\/preventing-iot-data-waste-with-the-intelligent-edge\/","title":{"rendered":"Preventing IoT data waste with the intelligent edge"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tReady to learn Internet of Things? Browse courses<\/a>\u00a0like\u00a0Internet of Things (IoT) Training<\/a> developed by industry thought leaders and Experfy in Harvard Innovation Lab.<\/em><\/strong>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tCompanies are doing a lot of interesting things in the IoT and industrial IoT space, but it often seems to me that while there are a lot of exciting and flashy features taking place at the edge, not a lot is being done with the collected data afterwards.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\n
For example, we are being inundated with\u00a0smart retail<\/a>, smart manufacturing,\u00a0self-driving cars<\/a>, robotics and other amazing use cases. These things are incredible, but we don\u2019t see a lot of trend analysis afterwards or real-time reaction to collected data. We should be seeing a lot more insights from the edge. We see basic things, but the advanced analysis we have come to expect from machine learning and predictive analysis is missing. Furthermore, we are not seeing real-time data analysis and reaction that is possible.\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t<\/header><\/section>This, however, was just my general consensus for the industry based on many conversations I\u2019ve had with developers, customers, and partners; however, this article \u2014 \u201cInternet of things data proving to be a hard thing to use<\/a>\u201d by\u00a0Sharon Shea<\/a>\u00a0\u2014 takes a much deeper look and confirms my suspicion. It states, based on research by Forrester, that only 1% of IoT data collected is ever used. That is staggering, but should not be surprising.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tWhy are companies collecting so much data, yet using so little? I believe the answer to be very straightforward \u2014 traditionally this has been a very hard problem to solve. This is compounded by the fact that most of the people solving these problems, while incredibly smart, are first and foremost electrical engineers and do not have strong backgrounds in big data.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tMost engineers implementing IoT projects, with good reason, are heavily focused on hardware challenges like battery life, device size and selecting the right manufacturer. They are also obviously focused on solving their primary use cases and getting to market quickly, and not as concerned with\u00a0value-added analytics<\/a>, which normally comes later.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tThey do not have time to design and build the traditional big data architectures required to consume IoT data and transform that data into valuable insights, identify patterns and make it actionable. They don\u2019t have the time, skill set or incentive to fully optimize their data value chain.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tHow could they? The technologies available on the market either offer very basic functionality, are very complex or very expensive. Often a data value chain optimized for trend analysis and analytics is comprised of five to seven different products. These might include a caching mechanism on the device, an\u00a0IoT gateway<\/a>, a NoSQL database, an SQL database, a map-reduce technology, and middleware to tie it all together. This requires a large investment in licenses, infrastructure and technical resources in addition to already expended capital on IoT hardware.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tAs a result, they often turn to technologies that provide the most basic of functions and allow them to move forward with the least resistance with the plan that, after reaching the market, they will circle back toward optimizing their data value chain to monetize the other 99% of the data they are collecting. As we know, this rarely to never happens.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tAs I mentioned above, solving big data problems is really complicated. Database vendors have taken the path that offloads that complexity onto the developer instead of internalizing this complexity and presenting simple solutions to complex problems. Furthermore, the people innovating within the technology landscape are a lot different than they were 15 years ago, yet the most popular database technologies, like MongoDB and SQLite, are 11 and 18 years old respectively.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tFolks driving innovation today are electrical engineers,\u00a0data scientists<\/a>, artificial intelligence researchers and quantum physicists. These are all incredibly smart folks, but they are not first and foremost computer programmers. Furthermore, most new developers do not have four-year degrees in CS but rather graduated from code school. As a result, the data management industry needs to focus on providing solutions to massive big data problems that empower these folks to focus on building incredible innovations, and not worry about overly complex and expensive big data architectures that leave 99% of data unused.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\nAs programming becomes a more cross-disciplinary skill, it is unreasonable to have an expectation that database end users will have 10, 15 and 20 years of experience implementing data management systems. These technologies need to be easy to install, easy to maintain and easy to use. This is especially important in IoT. In other verticals and workloads, the solution to the above has recently become\u00a0DBaaS<\/a>. That does not work in IoT, where your data value chain begins far outside the cloud.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tAdoption of IoT, in turn, is going to continue to drive an adoption of hybrid cloud. As a result, the solution to these problems cannot simply be providing a managed service for an incredibly complex product. The product itself needs to be simple and easy to use, yet highly scalable, so that it can be deployed and managed directly on the edge and in the cloud with the same level of effort and skill set. In order to continue to drive and support innovation the data management industry will need to transform to support a new generation of innovators.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"

Adoption of IoT is going to continue to drive an adoption of hybrid cloud. As a result, the solution to these problems cannot simply be providing a managed service for an incredibly complex product. The product itself needs to be simple and easy to use, yet highly scalable, so that it can be deployed and managed directly on the edge and in the cloud with the same level of effort and skill set. In order to continue to drive and support innovation, the data management industry needs to design and build the traditional big data architectures required to consume IoT data and transform that data into valuable insights, identify patterns and make it actionable.<\/p>\n","protected":false},"author":292,"featured_media":3617,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[195],"tags":[93],"ppma_author":[1835],"class_list":["post-673","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-iot","tag-internet-of-things"],"authors":[{"term_id":1835,"user_id":292,"is_guest":0,"slug":"stephen-goldberg","display_name":"Stephen Goldberg","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/?s=96&d=mm&r=g","user_url":"","last_name":"Goldberg","first_name":"Stephen","job_title":"","description":"Stephen Goldberg is the CEO of HarperDB<\/a>. He has previously founded two startups, and most recently was CTO at Phizzle, Inc. managing product, engineering, product marketing, and support. He has worked at companies large and small including Red Hat, Inc. where he led Infrastructure for their Global Support Services division. Stephen has 4 pending patents and has been a speaker at both Salesforce.com’s  Dreamforce as well as SAP’s Sapphire."}],"_links":{"self":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/673","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/users\/292"}],"replies":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/comments?post=673"}],"version-history":[{"count":4,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/673\/revisions"}],"predecessor-version":[{"id":37884,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/673\/revisions\/37884"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media\/3617"}],"wp:attachment":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media?parent=673"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/categories?post=673"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/tags?post=673"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=673"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}