CS6045: Advanced Algorithms Sorting Algorithms Heap Data Structure • A heap (nearly complete binary tree) can be stored as an array A – – – – – Root of tree is A[1] Parent of Left child of A[i] = A[2i] Right child of A[i] = A[2i + 1] Computing is fast with binary representation implementation Heaps • Max-heap property: • Min-heap property: Maintain the Heap Property • Running Time? • O(log n) Build a Heap Correctness • Loop invariant: At start of every iteration of for loop, each node i+1, i+2, …, n is the root of a max-heap • Analysis running time? • O(n log n) • Tighter bound? O(n) Heapsort Analysis • • • • • BUILD-MAX-HEAP: O(n) for loop: n – 1 times Exchange elements: O(1) MAX-HEAPIFY: O(log n) Time: – O(n log n) Min-Heap Operations 2 2 4 6 8 10 11 13 7 4 9 6 8 3 12 2 10 3 13 12 7 6 3 9 8 11 10 7 4 13 12 9 11 Insert(S, x): O(height) = O(log n) 2 12 6 8 10 13 4 11 12 5 6 9 8 10 13 4 4 11 5 6 9 8 10 13 4 12 11 5 6 9 8 5 11 10 13 Extract-min(S): return head, replace head key with the last, float down, O(log n) 12 9 Priority Queues • Priority Queue – Maintains a dynamic set S of elements – Each element has a key --- an associated value • Applications – job scheduling on shared computer – Dijkstra’s finding shortest paths in graphs – Prim’s algorithm for minimum spanning tree Priority Queues • Operations supported by priority queue – Insert(S, x) - inserts element with the pointer x – Minimum/Maximum(S) - returns element with the minimum key – Extract-Min/Max(S) - removes and returns minimum key – Increase/Decrease-Key(S,x,k) – increases/decreases the value of element x’s key to the new value k Comparison Sorting • The only operation that may be used to gain order information about a sequence is comparison of pairs of elements • Insertion sort, merge sort, quicksort, heapsort • Lower bound for comparison sorting? Decision Tree Model • Abstraction of any comparison sort • Counting only comparisons • Abstract everything else: such as control and data movement • How many leaves on the decision tree? – >= n! • What is the length of the longest path from root to leaf? – Depend on the algorithm Lower Bound for Comparison Sorting • A lower bound on the heights of decision trees in the lower bound on the running time of any comparison sort algorithm • (n log n) – n! <= l <= 2h 2h >= n! h >= lg(n!) >= lg (n/e)n //Stirlling’s approximation = nlg(n/e) = nlgn – nlge = (n log n) Non-comparison Sorts • Counting Sort Analysis • O(n + k) • How big a k is practical? Radix Sort Correctness • Induction on number of passes (i in pseudocode). • Assume digits 1, 2, ……, i -1 are sorted. • Show that a stable sort on digit i leaves digits 1, 2, ……, i sorted: – If 2 digits in position i are different, ordering by position i is correct, and positions 1, 2, …… , i - 1 are irrelevant. – If 2 digits in position i are equal, numbers are already in the right order (by inductive hypothesis). The stable sort on digit i leaves them in the right order. Analysis • O(n + k) per iteration • d iterations • O(d(n + k)) total Bucket Sort • Idea: – – – – Divide [0,1) into n equal-sized buckets Distribute the n input values into the buckets Sort each bucket Go through buckets in order, listing elements in each on