{"id":984,"date":"2018-11-15T04:31:30","date_gmt":"2018-11-15T01:31:30","guid":{"rendered":"http:\/\/kusuaks7\/?p=589"},"modified":"2021-05-11T14:01:29","modified_gmt":"2021-05-11T14:01:29","slug":"divide-and-conquer-algorithm","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/bigdata-cloud\/divide-and-conquer-algorithm\/","title":{"rendered":"Divide-and-Conquer Algorithm"},"content":{"rendered":"<p><strong><em>Ready to learn Data Science? Browse&nbsp;<a href=\"https:\/\/www.experfy.com\/training\/tracks\/data-science-training-certification\">Data Science Training and Certification<\/a> courses developed by industry thought leaders and Experfy in Harvard Innovation Lab.<\/em><\/strong><\/p>\n<p>A very popular algorithmic paradigm, a&nbsp;typical Divide and Conquer algorithm solves a problem using following three steps:<\/p>\n<ul data-rte-list=\"default\">\n<li>\n<p><em>Divide:<\/em> Break the given problem into subproblems of same type.<\/p>\n<\/li>\n<li>\n<p><em>Conquer:<\/em> Recursively solve these subproblems<\/p>\n<\/li>\n<li>\n<p><em>Combine:<\/em>&nbsp;Appropriately combine the answers<\/p>\n<\/li>\n<\/ul>\n<p>Following are some standard algorithms that are Divide and Conquer algorithms:<\/p>\n<p>1 &#8211;&nbsp;<strong>Binary Search<\/strong> is a searching algorithm. In each step, the algorithm compares the input element x with the value of the middle element in array. If the values match, return the index of middle. Otherwise, if x is less than the middle element, then the algorithm recurs for left side of middle element, else recurs for right side of middle element.<\/p>\n<p>2 &#8211;&nbsp;<strong>Quicksort<\/strong> is a sorting algorithm. The algorithm picks a pivot element, rearranges the array elements in such a way that all elements smaller than the picked pivot element move to left side of pivot, and all greater elements move to right side. Finally, the algorithm recursively sorts the subarrays on left and right of pivot element.<\/p>\n<p>3 &#8211;&nbsp;<strong>Merge Sort<\/strong> is also a sorting algorithm. The algorithm divides the array in two halves, recursively sorts them and finally merges the two sorted halves.<\/p>\n<p>4 &#8211;&nbsp;<strong>Closest Pair of Points<\/strong>: The problem is to find the closest pair of points in a set of points in x-y plane. The problem can be solved in O(n#k8SjZc9Dxk2) time by calculating distances of every pair of points and comparing the distances to find the minimum. The Divide and Conquer algorithm solves the problem in O(nLogn) time.<\/p>\n<p>5 &#8211;&nbsp;<strong>Strassen&rsquo;s Algorithm<\/strong> is an efficient algorithm to multiply two matrices. A simple method to multiply two matrices need 3 nested loops and is O(n#k8SjZc9Dxk3). Strassen&rsquo;s algorithm multiplies two matrices in O(n#k8SjZc9Dxk2.8974) time.<\/p>\n<p>6 &#8211;&nbsp;<strong>Cooley&ndash;Tukey Fast Fourier Transform (FFT)<\/strong> algorithm is the most common algorithm for FFT. It is a divide and conquer algorithm which works in O(nlogn) time.<\/p>\n<p>7 &#8211;&nbsp;<strong>Karatsuba algorithm for fast multiplication<\/strong>:&nbsp;It does multiplication of two <em>n<\/em>-digit numbers in at most 3n#k8SjZc9Dxk(log3)&nbsp;single-digit multiplications in general (and exactly n#k8SjZc9Dxk(log3) when <em>n<\/em> is a power of 2). It is therefore faster than the classical algorithm, which requires <em>n#k8SjZc9Dxk<\/em>2 single-digit products.<\/p>\n<h3><strong>COUNTING INVERSIONS PROBLEM<\/strong><\/h3>\n<p>We will consider a problem that arises in the analysis of rankings, which&nbsp;are becoming important to a number of current applications. For example, a number of sites on the Web make use of a technique known as collaborative filtering, in which they try to match your preferences (for books, movies, restaurants) with those of other people out on the Internet. Once the Web site has identified people with &ldquo;similar&rdquo; tastes to yours&mdash;based on a comparison&nbsp;of how you and they rate various things&mdash;it can recommend new things that&nbsp;these other people have liked. Another application arises in recta-search tools on the Web, which execute the same query on many different search engines and then try to synthesize the results by looking for similarities and differences among the various rankings that the search engines return.<\/p>\n<p>A core issue in applications like this is the problem of comparing two&nbsp;rankings. You rank a set of rt movies, and then a collaborative filtering system consults its database to look for other people who had &ldquo;similar&rdquo; rankings. But what&rsquo;s a good way to measure, numerically, how similar two people&rsquo;s rankings are? Clearly an identical ranking is very similar, and a completely reversed ranking is very different; we want something that interpolates through the middle region.<\/p>\n<p>Let&rsquo;s consider comparing your ranking and a stranger&rsquo;s ranking of the&nbsp;same set of n movies. A natural method would be to label the movies from 1 to n according to your ranking, then order these labels according to the stranger&rsquo;s ranking, and see how many pairs are &ldquo;out of order.&rdquo; More concretely, we will consider the following problem. We are given a sequence of n numbers a1, &hellip;, an; we will assume that all the numbers are distinct. We want to define a measure that tells us how far this list is from being in ascending order; the value of the measure should be 0 if a1 &lt; a2 &lt; &hellip; &lt; an, and should increase as the numbers become more scrambled.<\/p>\n<p>A natural way to quantify this notion is by counting the number of&nbsp;inversions. We say that two indices i &lt; j form an inversion if ai &gt; aj, that is, if the two elements ai and aj are &ldquo;out of order.&rdquo; We will seek to determine the number of inversions in the sequence a1, &hellip;, an.<\/p>\n<p style=\"text-align: center;\"><img decoding=\"async\" alt=\"inversions.jpg\" data-image=\"https:\/\/static1.squarespace.com\/static\/59d9b2749f8dce3ebe4e676d\/t\/5bb283b6652dea2694fc80b3\/1538425786929\/inversions.jpg\" data-image-dimensions=\"1920x1080\" data-image-focal-point=\"0.5,0.5\" data-image-id=\"5bb283b6652dea2694fc80b3\" data-image-resolution=\"750w\" data-load=\"false\" data-position-mode=\"standard\" data-src=\"https:\/\/static1.squarespace.com\/static\/59d9b2749f8dce3ebe4e676d\/t\/5bb283b6652dea2694fc80b3\/1538425786929\/inversions.jpg\" data-type=\"image\" src=\"https:\/\/static1.squarespace.com\/static\/59d9b2749f8dce3ebe4e676d\/t\/5bb283b6652dea2694fc80b3\/1538425786929\/inversions.jpg?format=750w\" style=\"width: 700px; height: 394px;\" \/><\/p>\n<p>Here&rsquo;s a pseudocode for a <strong>O(n log n)<\/strong> running time algorithm:<\/p>\n<div><span style=\"font-family:courier new,courier,monospace;\">Merge-and-Count(A,B)<br \/>\n1 &#8211; Maintain a<\/span><span style=\"font-family:courier new,courier,monospace;\">Current&nbsp; pointer<\/span><span style=\"font-family:courier new,courier,monospace;\"> into each list, initialized to point to the front elements.<br \/>\n2 &#8211; Maintain a variable<\/span><span style=\"font-family:courier new,courier,monospace;\">Count&nbsp; for<\/span><span style=\"font-family:courier new,courier,monospace;\"> the number of inversions, initialized to 0.<br \/>\n3 &#8211; While both lists are nonempty:<br \/>\n4 &#8211;&nbsp;&nbsp;&nbsp;&nbsp; Let a_{i} and b_{j} be the elements pointed to by the Current<\/span><span style=\"font-family:courier new,courier,monospace;\">pointer<\/span><span style=\"font-family:courier new,courier,monospace;\">.<br \/>\n5 &#8211;&nbsp;&nbsp;&nbsp;&nbsp; Append the smaller of these two to the output list.<br \/>\n6 &#8211;&nbsp;&nbsp;&nbsp;&nbsp; If b_{j} is the smaller element then:<br \/>\n7 &#8211;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Increment Count by the number of elements remaining in A.<br \/>\n8 &#8211;&nbsp;&nbsp;&nbsp;&nbsp; Endif:<br \/>\n9 &#8211;&nbsp;&nbsp;&nbsp;&nbsp; Advance the&nbsp;<\/span><span style=\"font-family:courier new,courier,monospace;\">Current&nbsp;pointer<\/span><span style=\"font-family:courier new,courier,monospace;\"> in the list from which the smaller element was selected.<br \/>\n10 &#8211; EndWhile<br \/>\n11 &#8211; Once one list is empty, append the remainder of the other list to the output.<br \/>\n12 &#8211; Return Count and the merged list.<\/span><\/div>\n<p>&nbsp;<\/p>\n<p>The running time of <strong>Merge-and-Count<\/strong> can be bounded by the analogue&nbsp;of the argument we used for the original merging algorithm at the heart of <strong>Mergesort<\/strong>: each iteration of the <strong>While<\/strong> loop takes constant time, and in each iteration we add some element to the output that will never be seen again. Thus the number of iterations can be at most the sum of the initial lengths of A and B, and so the total running time is <strong>O(n)<\/strong>.<\/p>\n<p>We use this <strong>Merge-and-Count<\/strong> routine in a recursive procedure that&nbsp;simultaneously sorts and counts the number of inversions in a list L.<\/p>\n<div><span style=\"font-family:courier new,courier,monospace;\">Sort-and-Count (L)<br \/>\n1 &#8211; If the list has one element, then there are no inversions.<br \/>\n2 &#8211; Else:<br \/>\n3 &#8211;&nbsp;&nbsp;&nbsp;&nbsp; Divide the list into two halves:<br \/>\n4 &#8211;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; A contains the first [n\/2] elements.<br \/>\n5 &#8211;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; B contains the remaining [n\/2] elements.<br \/>\n6 &#8211;&nbsp;&nbsp;&nbsp;&nbsp; (rA, A)&nbsp; = Sort-and-Count (A).<br \/>\n7 &#8211;&nbsp;&nbsp;&nbsp;&nbsp; (rB, B) = Sort-and-Count (B).<br \/>\n8 &#8211;&nbsp;&nbsp;&nbsp;&nbsp; (r, L) = Merge-and-Count (A, B).<br \/>\n9 &#8211; Endif<br \/>\n10 &#8211; Return r = rA + rB + r,&nbsp; and the sorted list L.<\/span><\/div>\n<p>&nbsp;<\/p>\n<p>Since Merge-and-Count takes O(n) time, the running time of <strong>Sort-and-Count<\/strong> is <strong>O(n log n)<\/strong> for a list with n elements.<\/p>\n<h3><strong>MULTIPLICATION OF LONG NUMBERS PROBLEM<\/strong><\/h3>\n<p>The problem we consider is an extremely basic one: the multiplication of two&nbsp;integers. In a sense, this problem is so basic that one may not initially think of it even as an algorithmic question. But, in fact, elementary schoolers are taught a concrete (and quite efficient) algorithm to multiply two n-digit numbers x and y. You first compute a &ldquo;partial product&rdquo; by multiplying each digit of y separately by x, and then you add up all the partial products. Counting a single operation on a pair of bits as one primitive step in this computation, it takes O(n) time to compute each partial product, and O(n) time to combine it in with the running sum of all partial products so far. Since there are n partial products, this is a total running time of O(n#k8SjZc9Dxk2).<\/p>\n<p>If you haven&rsquo;t thought about this much since elementary school, there&rsquo;s&nbsp;something initially striking about the prospect of improving on this algorithm. Aren&rsquo;t all those partial products &quot;necessary&quot; in some way? But, in fact, it is possible to improve on O(n#k8SjZc9Dxk2) time using a different, recursive way of performing the multiplication.<\/p>\n<p style=\"text-align: center;\"><img decoding=\"async\" alt=\"karatsuba.jpg\" data-image=\"https:\/\/static1.squarespace.com\/static\/59d9b2749f8dce3ebe4e676d\/t\/5bb284301905f4c421ca3c0b\/1538425908133\/karatsuba.jpg\" data-image-dimensions=\"1280x720\" data-image-focal-point=\"0.5,0.5\" data-image-id=\"5bb284301905f4c421ca3c0b\" data-image-resolution=\"750w\" data-load=\"false\" data-position-mode=\"standard\" data-src=\"https:\/\/static1.squarespace.com\/static\/59d9b2749f8dce3ebe4e676d\/t\/5bb284301905f4c421ca3c0b\/1538425908133\/karatsuba.jpg\" data-type=\"image\" src=\"https:\/\/static1.squarespace.com\/static\/59d9b2749f8dce3ebe4e676d\/t\/5bb284301905f4c421ca3c0b\/1538425908133\/karatsuba.jpg?format=750w\" style=\"width: 700px; height: 394px;\" \/><\/p>\n<div><span style=\"font-family:courier new,courier,monospace;\">Recursive-Multiply (x, y)<br \/>\n1 &#8211; Write x = x_{1} * 2#k8SjZc9Dxk(n\/2) + x_{0} and y = y_{1} * 2#k8SjZc9Dxk(n\/2) + y_{0}<br \/>\n2 &#8211; Compute x_{1} + x_{0} and y_{1} + y_{0}<br \/>\n3 &#8211; p = Recursive-Multiply(x_{1} + x_{0}, y_{1} + y_{0})<br \/>\n4 &#8211; x_{1} * y_{1} = Recursive-Multiply(x_{1}, y_{1})<br \/>\n5 &#8211; x_{0} * y_{0} = Recursive-Multiply(x_{0}, y_{0})<br \/>\n6 &#8211; Return x_{1} * y_{1} * 2#k8SjZc9Dxkn + (p &mdash; x_{1} * y_{1} &mdash; x_{0} * y_{0}) * 2#k8SjZc9Dxk(n\/2) +&nbsp; x_{0} * y_{0}<\/span><\/div>\n<p>&nbsp;<\/p>\n<p>We can determine the running time of this algorithm as follows. Given two n-bit&nbsp;numbers, it performs a constant number of additions on O(n)-bit numbers, in addition to the three recursive calls. Ignoring for now the issue that x_{1} + x_{0} and y_{1} + y_{0} may have n \/ 2 + 1 bits (rather than just n\/2), which turns out not to affect the asymptotic results, each of these recursive calls is on an instance of size n\/2. Thus, in place of our four-way branching recursion, we now have a three-way branching one, with a running time that satisfies <strong>T(n) &lt;_ 3T(n\/2) + cn&nbsp;<\/strong>for a constant c. This is case 3 of the Master Theorem describes below, which gives the running time of <strong>O(n#k8SjZc9Dxk1.59)<\/strong>.<\/p>\n<h3><strong>THE MASTER THEOREM<\/strong><\/h3>\n<p>The master theorem provides a solution to recurrence relations of the form:<\/p>\n<p><img decoding=\"async\" alt=\"master-theorem.png\" data-image=\"https:\/\/static1.squarespace.com\/static\/59d9b2749f8dce3ebe4e676d\/t\/5bb2845571c10b91471ced94\/1538425946009\/master-theorem.png\" data-image-dimensions=\"294x90\" data-image-focal-point=\"0.5,0.5\" data-image-id=\"5bb2845571c10b91471ced94\" data-image-resolution=\"750w\" data-load=\"false\" data-position-mode=\"standard\" data-src=\"https:\/\/static1.squarespace.com\/static\/59d9b2749f8dce3ebe4e676d\/t\/5bb2845571c10b91471ced94\/1538425946009\/master-theorem.png\" data-type=\"image\" src=\"https:\/\/static1.squarespace.com\/static\/59d9b2749f8dce3ebe4e676d\/t\/5bb2845571c10b91471ced94\/1538425946009\/master-theorem.png?format=750w\" \/><\/p>\n<p>for constants a &gt;= 1&nbsp;and b &gt; 1&nbsp;with f&nbsp;asymptotically positive. Such recurrences occur frequently in the runtime analysis of many commonly encountered algorithms. The following statements are true:<\/p>\n<p style=\"text-align: center;\"><img decoding=\"async\" alt=\"masterTheorem.png\" data-image=\"https:\/\/static1.squarespace.com\/static\/59d9b2749f8dce3ebe4e676d\/t\/5bb28469652dea2694fc8a74\/1538425965188\/masterTheorem.png\" data-image-dimensions=\"1272x204\" data-image-focal-point=\"0.5,0.5\" data-image-id=\"5bb28469652dea2694fc8a74\" data-image-resolution=\"750w\" data-load=\"false\" data-position-mode=\"standard\" data-src=\"https:\/\/static1.squarespace.com\/static\/59d9b2749f8dce3ebe4e676d\/t\/5bb28469652dea2694fc8a74\/1538425965188\/masterTheorem.png\" data-type=\"image\" src=\"https:\/\/static1.squarespace.com\/static\/59d9b2749f8dce3ebe4e676d\/t\/5bb28469652dea2694fc8a74\/1538425965188\/masterTheorem.png?format=750w\" style=\"width: 700px; height: 112px;\" \/><\/p>\n<p><strong><em>Sources:<\/em><\/strong><\/p>\n<ul data-rte-list=\"default\">\n<li>\n<div><a href=\"https:\/\/www.geeksforgeeks.org\/divide-and-conquer-algorithm-introduction\/\" rel=\"noopener\"><em>Divide and Conquer Algorithm &#8211; Introduction<\/em><\/a><em>&nbsp;(GeeksforGeeks)<\/em><\/div>\n<\/li>\n<li>\n<div><a href=\"https:\/\/brilliant.org\/wiki\/master-theorem\/\" rel=\"noopener\"><em>Master Theorem<\/em><\/a><em>&nbsp;(Brilliant)<\/em><\/div>\n<\/li>\n<li>\n<div><a href=\"https:\/\/www.cp.eng.chula.ac.th\/~piak\/teaching\/algo\/algo2008\/count-inv.htm\" rel=\"noopener\"><em>Counting Inversion<\/em><\/a> <em>(Chula Engineering)<\/em><\/div>\n<\/li>\n<li>\n<div><a href=\"https:\/\/brilliant.org\/wiki\/karatsuba-algorithm\/\" rel=\"noopener\"><em>Karatsuba Algorithm<\/em><\/a> <em>(Brilliant)<\/em><\/div>\n<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>In computer science, divide and conquer is an algorithm design paradigm based on multi-branched&nbsp;recursion. A divide and conquer algorithm works by recursively breaking down a problem into two or more&nbsp;sub-problems&nbsp;of the same or related type, until these become simple enough to be solved directly. A&nbsp;typical Divide and Conquer algorithm solves a problem using following three steps: Divide: Break the given problem into sub-problems of same type. Conquer: Recursively solve these sub-problems. Combine:&nbsp;Appropriately combine the answers.<\/p>\n","protected":false},"author":86,"featured_media":3486,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[187],"tags":[94],"ppma_author":[1842],"class_list":["post-984","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-bigdata-cloud","tag-data-science"],"authors":[{"term_id":1842,"user_id":86,"is_guest":0,"slug":"james-le","display_name":"James Le","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/?s=96&d=mm&r=g","user_url":"","last_name":"Le","first_name":"James","job_title":"","description":"James Le is a Software Developer with experiences in Product Management and Data Analytics. He played a pivotal role in the operation of a start-up organization at Denison University."}],"_links":{"self":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/984","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/users\/86"}],"replies":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/comments?post=984"}],"version-history":[{"count":1,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/984\/revisions"}],"predecessor-version":[{"id":6244,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/984\/revisions\/6244"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media\/3486"}],"wp:attachment":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media?parent=984"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/categories?post=984"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/tags?post=984"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=984"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}