{"id":1745,"date":"2019-06-07T03:35:19","date_gmt":"2019-06-07T03:35:19","guid":{"rendered":"http:\/\/kusuaks7\/?p=1350"},"modified":"2023-07-06T11:28:42","modified_gmt":"2023-07-06T11:28:42","slug":"a-short-history-of-big-data","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/bigdata-cloud\/a-short-history-of-big-data\/","title":{"rendered":"A Short History Of Big Data"},"content":{"rendered":"

90% of the available data has been created in the last two years and the term\u00a0Big Data<\/a>\u00a0has been around 2005 when it was launched by O\u2019Reilly Media in 2005.\u00a0 However, the usage of\u00a0Big <\/a>Data and\u00a0the need to understand all available data has been around much longer.<\/p>\n

 <\/p>\n

In fact,\u00a0the earliest records<\/a>\u00a0of using data to track and control businesses date back from 7.000 years ago when accounting was introduced in Mesopotamia in order to record the growth of crops and herds. Accounting principles continued to improve, and in 1663,\u00a0John Graunt<\/a>\u00a0recorded and examined all information about mortality roles in London. He wanted to gain an understanding and build a warning system for the ongoing bubonic plague. In the first recorded record of statistical data analysis, he gathered his findings in the book Natural and Political Observations Made upon the Bills of Mortality, which provides great insights into the causes of death in the seventeenth century. Because of his work, Graunt can be considered the father of statistics. From there on, the accounting principles improved but nothing spectacular happened. Until in the 20<sup>th<\/sup> century the information age started. The earliest remembrance of modern data is from 1887 when\u00a0Herman Hollerith<\/a>\u00a0invented a computing machine that could read holes punched into paper cards in order to organize census data.<\/p>\n

The 20th Century<\/h2>\n

The first major data project is created in 1937 and was ordered by the Franklin D. Roosevelt\u2019s administration in the USA. After the Social\u00a0Security<\/a>\u00a0Act became law in 1937, the government had to keep track of contribution from 26 million Americans and more than 3 million\u00a0employers<\/a>.\u00a0IBM got the contract<\/a>\u00a0to develop punch card-reading machine for this massive bookkeeping project.<\/p>\n

The first data-processing machine appeared in 1943 and was developed by the British to decipher Nazi codes during World War II. This device, named\u00a0Colossus<\/a>, searched for patterns in intercepted messages at a rate of 5.000 characters per second. Thereby reducing the task from weeks to merely hours.<\/p>\n

In 1952 the National\u00a0Security<\/a>\u00a0Agency (NSA) is created and within 10 years contract more than\u00a012.000 cryptologists<\/a>. They are confronted with information overload during the Cold War as they start collecting and processing intelligence signals automatically.<\/p>\n

In 1965 the United States Government\u00a0decided to build the first data centre<\/a>\u00a0to store over 742 million tax returns and 175 million sets of fingerprints by transferring all those records onto magnetic computer tape that had to be stored in a single location. The project was later dropped out of fear for \u2018Big Brother\u2019, but it is generally accepted that it was the beginning of the electronic data storage era.<\/p>\n

In 1989 British computer scientist\u00a0Tim Berners-Lee<\/a>\u00a0invented eventually the World Wide Web. He wanted to facilitate the sharing of information via a \u2018hypertext\u2019 system. Little could he know at the moment the impact of his invention.<\/p>\n

As of the \u201890s, the creation of data is spurred as more and more devices are connected to the internet.\u00a0In 1995 the first super-computer is built<\/a>, which was able to do as much work in a second than a calculator operated by a single person can do in 30.000 years.<\/p>\n

The 21st Century<\/h2>\n

In 2005\u00a0Roger Mougalas<\/a>\u00a0from O\u2019Reilly Media coined the term\u00a0Big Data<\/a>\u00a0for the first time, only a year after they created the term Web 2.0. It refers to a large set of data that is almost impossible to manage and process using traditional business intelligence tools.<\/p>\n

2005 is also the year that\u00a0Hadoop was created<\/a>\u00a0by Yahoo! built on top of Google\u2019s\u00a0MapReduce<\/a>. Its goal was to index the entire World Wide Web and nowadays the open-source Hadoop is used by a lot of organizations to crunch through huge amounts of data.<\/p>\n

As more and more social networks start appearing and the Web 2.0 takes flight, more and more data is created on a daily basis. Innovative startups slowly start to dig into this massive amount of data and also governments start working on Big Data projects. In 2009 the Indian government decides to take an iris scan, fingerprint, and photograph of all of its 1.2 billion inhabitants. All this data is stored in the largest biometric database in the world.<\/p>\n

In 2010\u00a0Eric Schmidt speaks<\/a>\u00a0at the Techonomy conference in Lake Tahoe in California and he states that “there were 5 exabytes of information created by the entire world between the dawn of civilization and 2003. Now that same amount is created every two days.”<\/p>\n

In 2011 the\u00a0McKinsey report<\/a>\u00a0on Big Data:\u00a0The next frontier for innovation, competition, and productivity,\u00a0<\/i>states that in 2018 the USA alone will face a shortage of 140.000 \u2013 190.000 data scientist as well as 1.5 million data managers.<\/p>\n

In the past few years, there has been a massive increase in Big Data startups, all trying to deal with Big Data and helping organizations to understand Big Data and more and more companies are slowly adopting and moving towards Big Data. However, while it looks like Big Data is around for a long time already, in fact, Big Data is\u00a0as far as the internet was in 1993<\/a>. The large Big Data revolution is still ahead of us so a lot will change in the coming years. Let the Big Data era begin!<\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

The term Big Data has been around 2005, when it was launched by O’Reilly Media in 2005.  However, the usage of Big Data and the need to understand all available data has been around much longer. While it looks like Big Data is around for a long time already, in fact Big Data is as far as the internet was in 1993. The large Big Data revolution is still ahead of us so a lot will change in the coming years. Let the Big Data era begin!<\/p>\n","protected":false},"author":550,"featured_media":2969,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[187],"tags":[95],"ppma_author":[3218],"class_list":["post-1745","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-bigdata-cloud","tag-big-data-amp-technology"],"authors":[{"term_id":3218,"user_id":550,"is_guest":0,"slug":"mark-van-rijmenam","display_name":"Mark Rijmenam","avatar_url":"https:\/\/www.experfy.com\/blog\/wp-content\/uploads\/2020\/04\/medium_6e38fd10-e332-41e3-98b2-8874278fc272-150x150.jpg","user_url":"https:\/\/datafloq.com\/","last_name":"Rijmenam","first_name":"Mark","job_title":"","description":"Dr Mark van Rijmenam is the founder of\u00a0Datafloq<\/a>\u00a0and\u00a0Imagjn<\/a>.\u00a0He is a globally recognised speaker on big data, blockchain and AI, a strategist, influencer and author of\u00a03 management books<\/a>. His last book,\u00a0The Organisation of Tomorrow<\/a>, discusses how AI, blockchain and analytics turn every business into a data organisation. He has been named a global thought leader on\u00a0big data<\/a>,\u00a0blockchain<\/a>\u00a0and\u00a0artificial intelligence<\/a>. He holds a PhD in Management from the University of Technology, Sydney. He is a strategic advisor to several blockchain startups and publisher of\u00a0the \u2018f(x) = ex\u2018 newsletter<\/a>."}],"_links":{"self":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/1745","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/users\/550"}],"replies":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/comments?post=1745"}],"version-history":[{"count":2,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/1745\/revisions"}],"predecessor-version":[{"id":29035,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/1745\/revisions\/29035"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media\/2969"}],"wp:attachment":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media?parent=1745"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/categories?post=1745"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/tags?post=1745"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=1745"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}