Notes on the merge sort algorithm

  • Merge sort is one of few algorithms that has O(nlog(n)) running time

    • O(nlog(n)) is also the theorectical low bound on running time of sorting algorithms

  • Some problems with the pure Merge Sort algorithm:

    1. There is a very high overhead to execute a recursive algorithm for tiny sublists

    2. Merge sort will merge even when the 2 halves are sorted

    3. Merge sort is not an in-place algorithm

      • Merge sort requires another array to perform the mereg operation

    The first 2 problems can be fixed...

Why is recursion with tiny sublists a problem in Merge Sort

  • Suppose the input array has 1,000,000 elements: 

                     mergeSort(A, 0, 1000000, H)

     

Why is recursion with tiny sublists a problem in Merge Sort

  • The base case (= stop criteria) is n = 1: 

                     mergeSort(A, 0, 1000000, H)

















    mergeSort(A,0,0,H) ...       mergeSort(A,k,k,H) .....

    There will be 1,000,000 mergeSort(A,k,k,H) invocations !!!

Why is recursion with tiny sublists a problem in Merge Sort

  • Furthermore, the base case will simple return: 

                     mergeSort(A, 0, 1000000, H)





       if ( n <= 1 )
          return;










    mergeSort(A,0,0,H) ...       mergeSort(A,k,k,H) .....

    All 1,000,000 method invocations do no work but cost time to execute !!!

The hybrid merge sort algorithm

Review:   merge sort algorithm 

 // Merge sorts the array elements A[s] ... A[e-1] using helper array H

 public static <T extends Comparable<T>> void sort(T[] A, int s, int e, T[] H)
 {
     if ( e - s <= 1 )      // A[s]..A[e] has 0 or 1 element
     {


         return;            // No need to sort an array of 1 element...
     }


     int m = (e+s)/2;       // m = middle of s and e
   
     MergeSort.sort(A, s, m, H);

     MergeSort.sort(A, m, e, H);

     // Merge both sorted arrays

     merge(A, s, m, e, H); // We have discussed merge() previously !
 }

The hybrid merge sort algorithm

The hybrid merge sort algorithm uses a different sort algorithm when input size is below some threshold: 

 // Merge sorts the array elements A[s] ... A[e-1] using helper array H

 public static <T extends Comparable<T>> void sort(T[] A, int s, int e, T[] H)
 {
     if ( e - s <= CUTOFF ) // Use a different sort alg for small inputs
     {
         SelectionSort.sort(A, s, e);

         return;            
     }


     int m = (e+s)/2;       // m = middle of s and e
   
     MergeSort.sort(A, s, m, H);

     MergeSort.sort(A, m, e, H);

     // Merge both sorted arrays

     merge(A, s, m, e, H); // We have discussed merge() previously !
 }

The hybrid merge sort algorithm is called TimSort

The hybrid merge sort algorithm

Another improvement:   check if the 2 (sorted) array halves "overlap" before invocating merge( ) 

 // Merge sorts the array elements A[s] ... A[e-1] using helper array H

 public static <T extends Comparable<T>> void sort(T[] A, int s, int e, T[] H)
 {
     if ( e - s <= CUTOFF ) // Use a different sort alg for small inputs
     {
         SelectionSort.sort(A, s, e);

         return;            
     }


     int m = (e+s)/2;       // m = middle of s and e
   
     MergeSort.sort(A, s, m, H);

     MergeSort.sort(A, m, e, H);

     // Merge both sorted arrays only when values overlap
     if ( A[m-1] < A[m] )  return;
     merge(A, s, m, e, H); // We have discussed merge() previously !
 }

What do I meant by overlap

  • When you have 2 sorted arrays that are adjacent to each other, you do not need to merge them if the values do not overlap: 

          1    2   4    5    6    7   8   9 
      <------------->    <------------>
         array 1            array 2

  • You need to perform a merge operation when the values overlap: 

          1    2   4    7    5    6   8   9 
      <------------->    <------------>
         array 1            array 2

    The 7 in the left array "extends" into the right array

Non-recursive implementation of Merge Sort

  • Previously, we have used the MergeSort.merge( ) method to sort an array of 4 elements as follows: 

        public static void main(String[] args)
    {
        Integer[] list = {7, 3, 8, 5};

        Integer[] help = new Integer[list.length];      // Helper array

        printArray(list);

        // Merge arrays of 1 elements
        MergeSort.merge(list, 0, 1, 2, help);
        MergeSort.merge(list, 2, 3, 4, help);

        // Merge arrays of 2 elements
        MergeSort.merge(list, 0, 2, 4, help);

        printArray(list);
    }

  • (0) An array of 1 element is sorted
    (2) Merge a pair of (adjacent) array of 1 element into a sorted array of 2 elements
    (2) Merge a pair of (adjacent) array of 2 element into a sorted array of 4 elements
     

DEMO: 14-sort/10-merge-sort/Demo3.java

Non-recursive implementation of Merge Sort

  • We can use the MergeSort.merge( ) method to sort an array of 8 elements as follows: 

        public static void main(String[] args)
    {
        Integer[] list = {7, 3, 2, 1, 8, 5, 4, 6};

        Integer[] help = new Integer[list.length];      // Helper array

        printArray(list);

        // Merge arrays of 1 elements
        MergeSort.merge(list, 0, 1, 2, help);
        MergeSort.merge(list, 2, 3, 4, help);
        MergeSort.merge(list, 4, 5, 6, help);
        MergeSort.merge(list, 6, 7, 8, help);

        // Merge arrays of 2 elements
        MergeSort.merge(list, 0, 2, 4, help);
        MergeSort.merge(list, 4, 6, 8, help);

        // Merge arrays of 4 elements
        MergeSort.merge(list, 0, 4, 8, help);

        printArray(list);
    }

  • And so on !!!

Non-recursive implementation of Merge Sort

  • The Merge Sort can be implemented without using recursion as follows: 

        public static <T extends Comparable<T>> void sort(T[] A, T[] H)
    {
        int mergeSize = 1;

        while ( mergeSize < A.length )
        {
            int s, m, e;

            for (s = 0; s < A.length; s += 2*mergeSize)
            {
                m = s + mergeSize;
                e = Math.min(A.length, s + 2*mergeSize);

                    merge( A, s, m, e, H);
            }

            mergeSize = 2*mergeSize;
        }
    }

     

Non-recursive implementation of Merge Sort

  • Start merging arrays of mergeSize = 1 and double the mergeSize after each iteration: 

        public static <T extends Comparable<T>> void sort(T[] A, T[] H)
    {
        int mergeSize = 1;

        while ( mergeSize < A.length )
        {
            int s, m, e;

            for (s = 0; s < A.length; s += 2*mergeSize)
            {
                m = s + mergeSize;
                e = Math.min(A.length, s + 2*mergeSize);

                    merge( A, s, m, e, H);
            }

            mergeSize = 2*mergeSize;
        }
    }

     

Non-recursive implementation of Merge Sort

  • We only need to perform merge( ) when there are > 1 piece of array half: 

        public static <T extends Comparable<T>> void sort(T[] A, T[] H)
    {
        int mergeSize = 1;

        while ( mergeSize < A.length )
        {
            int s, m, e;

            for (s = 0; s < A.length; s += 2*mergeSize)
            {
                m = s + mergeSize;
                e = Math.min(A.length, s + 2*mergeSize);

                    merge( A, s, m, e, H);
            }

            mergeSize = 2*mergeSize;
        }
    }

     

Non-recursive implementation of Merge Sort

  • The start index s of the first array are: 0, 2*mergeSize, 4*mergeSize, ...: 

        public static <T extends Comparable<T>> void sort(T[] A, T[] H)
    {
        int mergeSize = 1;

        while ( mergeSize < A.length )
        {
            int s, m, e;

            for (s = 0; s < A.length; s += 2*mergeSize)
            {
                m = s + mergeSize;
                e = Math.min(A.length, s + 2*mergeSize);

                    merge( A, s, m, e, H);
            }

            mergeSize = 2*mergeSize;
        }
    }

     

Non-recursive implementation of Merge Sort

  • The end index m of the first array are: mergeSize, 3*mergeSize, 5*mergeSize, ... : 

        public static <T extends Comparable<T>> void sort(T[] A, T[] H)
    {
        int mergeSize = 1;

        while ( mergeSize < A.length )
        {
            int s, m, e;

            for (s = 0; s < A.length; s += 2*mergeSize)
            {
                m = s + mergeSize;
                e = Math.min(A.length, s + 2*mergeSize);

                    merge( A, s, m, e, H);
            }

            mergeSize = 2*mergeSize;
        }
    }

    Note:   m is also the start index of the 2nd array

Non-recursive implementation of Merge Sort

  • The end index s of the 2nd array are: 2*mergeSize, 4*mergeSize, ... unless... 

        public static <T extends Comparable<T>> void sort(T[] A, T[] H)
    {
        int mergeSize = 1;

        while ( mergeSize < A.length )
        {
            int s, m, e;

            for (s = 0; s < A.length; s += 2*mergeSize)
            {
                m = s + mergeSize;
                e =                    s + 2*mergeSize ;

                    merge( A, s, m, e, H);
            }

            mergeSize = 2*mergeSize;
        }
    }

    Unless:   we are at the end of the input array !

Non-recursive implementation of Merge Sort

  • The end index s of the 2nd array cannot be > A.length: 

        public static <T extends Comparable<T>> void sort(T[] A, T[] H)
    {
        int mergeSize = 1;

        while ( mergeSize < A.length )
        {
            int s, m, e;

            for (s = 0; s < A.length; s += 2*mergeSize)
            {
                m = s + mergeSize;
                e = Math.min(A.length, s + 2*mergeSize);

                    merge( A, s, m, e, H);
            }

            mergeSize = 2*mergeSize;
        }
    }

     

Non-recursive implementation of Merge Sort

  • Merge the 2 arrays: 

        public static <T extends Comparable<T>> void sort(T[] A, T[] H)
    {
        int mergeSize = 1;

        while ( mergeSize < A.length )
        {
            int s, m, e;

            for (s = 0; s < A.length; s += 2*mergeSize)
            {
                m = s + mergeSize;
                e = Math.min(A.length, s + 2*mergeSize);

                merge(A, s, m, e, H);
            }

            mergeSize = 2*mergeSize;
        }
    }

     

DEMO: demo/14-sort/12b-non-recursive-merge-sort/MergeSort.java + Demo.java