Parallel sorting
 

  • So far, the kind of coorperation between threads that we have seen is:

      • Thread synchronization: a "faster" thread must wait for a "slower" thread to "catch up" to the synchronization point (before proceeding with further computation).

  • Parallel sorting algorithm is especially challenging for multi-processing systems because:

      • To perform the sort operation, information must be exchanged between threads

    We did not have to need to exchange information between threads so far....

Parallel sorting algorithms
 

  • There have been many parallel sorting algorithm developed for multi-processors:

      • Odd-even (transposition) sort algorithm
      • Parallel merge sort algorithm
      • Hyper-cube quick sort algorithm
      • ...

  • Some of the parallel sort algorithm are not suitable for GPU

  • In this brief into to CUDA programming, I like to show you the odd-even (transposition) sort algorithm that can be implemented in CUDA C

The odd-even sort algorithm

  • The Odd-Even (Transposition) Sort algorithm is based on the Bubble Sort algorithm

  • A sorting phase in the Odd-Even sort algorithm consists of 2 phases:

      • The even phase that compares and sorts the array elements: 

             A[0] <--> A[1]   (first index is even)    
   A[2] <--> A[3]
    ...

      • The odd phase that compares and sorts the array elements: 

             A[1] <--> A[2]    (first index is odd)  
   A[3] <--> A[4]
    ...

Example of the odd-even sort algorithm

Sample input to the odd-even sort algorithm: 

       A[0]  A[1]  A[2]  A[3]  A[4]  A[5]  A[6]  A[7]  A[8] 
         8     5     9     1     4     2     3     6     7

Example of the odd-even sort algorithm

Phase 1's even phase will compare the following pairs: 

       A[0]  A[1]  A[2]  A[3]  A[4]  A[5]  A[6]  A[7]  A[8] 
         8     5     9     1     4     2     3     6     7

Ph1:
 even:

Example of the odd-even sort algorithm

Result after Phase 1's even phase: 

       A[0]  A[1]  A[2]  A[3]  A[4]  A[5]  A[6]  A[7]  A[8] 
         8     5     9     1     4     2     3     6     7

Ph1:
 even:   5     8     1     9     2     4     3     6     7

Example of the odd-even sort algorithm

Phase 1's odd phase will compare the following pairs: 

       A[0]  A[1]  A[2]  A[3]  A[4]  A[5]  A[6]  A[7]  A[8] 
         8     5     9     1     4     2     3     6     7

Ph1:
 even:   5     8     1     9     2     4     3     6     7
 odd:

Example of the odd-even sort algorithm

Result after Phase 1's odd phase: 

       A[0]  A[1]  A[2]  A[3]  A[4]  A[5]  A[6]  A[7]  A[8] 
         8     5     9     1     4     2     3     6     7

Ph1:
 even:   5     8     1     9     2     4     3     6     7
 odd:    5     1     8     2     9     3     4     6     7

Example of the odd-even sort algorithm

Phase 2's even phase will compare the following pairs: 

       A[0]  A[1]  A[2]  A[3]  A[4]  A[5]  A[6]  A[7]  A[8] 
         8     5     9     1     4     2     3     6     7

Ph1:
 even:   5     8     1     9     2     4     3     6     7
 odd:    5     1     8     2     9     3     4     6     7

Ph2:
 even:

Example of the odd-even sort algorithm

Result after Phase 2's even phase: 

       A[0]  A[1]  A[2]  A[3]  A[4]  A[5]  A[6]  A[7]  A[8] 
         8     5     9     1     4     2     3     6     7

Ph1:
 even:   5     8     1     9     2     4     3     6     7
 odd:    5     1     8     2     9     3     4     6     7

Ph2:
 even:   1     5     2     8     3     9     4     6     7

Example of the odd-even sort algorithm

Phase 2's odd phase will compare the following pairs: 

       A[0]  A[1]  A[2]  A[3]  A[4]  A[5]  A[6]  A[7]  A[8] 
         8     5     9     1     4     2     3     6     7

Ph1:
 even:   5     8     1     9     2     4     3     6     7
 odd:    5     1     8     2     9     3     4     6     7

Ph2:
 even:   1     5     2     8     3     9     4     6     7
 odd:

Example of the odd-even sort algorithm

Result after Phase 2's odd phase: 

       A[0]  A[1]  A[2]  A[3]  A[4]  A[5]  A[6]  A[7]  A[8] 
         8     5     9     1     4     2     3     6     7

Ph1:
 even:   5     8     1     9     2     4     3     6     7
 odd:    5     1     8     2     9     3     4     6     7

Ph2:
 even:   1     5     2     8     3     9     4     6     7
 odd:    1     2     5     3     8     4     9     6     7

Example of the odd-even sort algorithm

Phase 3's even phase will compare the following pairs: 

       A[0]  A[1]  A[2]  A[3]  A[4]  A[5]  A[6]  A[7]  A[8] 
         8     5     9     1     4     2     3     6     7

Ph1:
 even:   5     8     1     9     2     4     3     6     7
 odd:    5     1     8     2     9     3     4     6     7

Ph2:
 even:   1     5     2     8     3     9     4     6     7
 odd:    1     2     5     3     8     4     9     6     7
 
Ph3:
 even:

Example of the odd-even sort algorithm

Result after Phase 3's even phase: 

       A[0]  A[1]  A[2]  A[3]  A[4]  A[5]  A[6]  A[7]  A[8] 
         8     5     9     1     4     2     3     6     7

Ph1:
 even:   5     8     1     9     2     4     3     6     7
 odd:    5     1     8     2     9     3     4     6     7

Ph2:
 even:   1     5     2     8     3     9     4     6     7
 odd:    1     2     5     3     8     4     9     6     7
 
Ph3:
 even:   1     2     3     5     4     8     6     9     7

Example of the odd-even sort algorithm

Phase 3's odd phase will compare the following pairs: 

       A[0]  A[1]  A[2]  A[3]  A[4]  A[5]  A[6]  A[7]  A[8] 
         8     5     9     1     4     2     3     6     7

Ph1:
 even:   5     8     1     9     2     4     3     6     7
 odd:    5     1     8     2     9     3     4     6     7

Ph2:
 even:   1     5     2     8     3     9     4     6     7
 odd:    1     2     5     3     8     4     9     6     7
 
Ph3:
 even:   1     2     3     5     4     8     6     9     7
  odd:

Example of the odd-even sort algorithm

Result after Phase 3's odd phase: 

       A[0]  A[1]  A[2]  A[3]  A[4]  A[5]  A[6]  A[7]  A[8] 
         8     5     9     1     4     2     3     6     7

Ph1:
 even:   5     8     1     9     2     4     3     6     7
 odd:    5     1     8     2     9     3     4     6     7

Ph2:
 even:   1     5     2     8     3     9     4     6     7
 odd:    1     2     5     3     8     4     9     6     7
 
Ph3:
 even:   1     2     3     5     4     8     6     9     7
  odd:   1     2     3     4     5     6     8     7     9

Example of the odd-even sort algorithm

Phase 4's even phase will compare the following pairs: 

       A[0]  A[1]  A[2]  A[3]  A[4]  A[5]  A[6]  A[7]  A[8] 
         8     5     9     1     4     2     3     6     7

Ph1:
 even:   5     8     1     9     2     4     3     6     7
 odd:    5     1     8     2     9     3     4     6     7

Ph2:
 even:   1     5     2     8     3     9     4     6     7
 odd:    1     2     5     3     8     4     9     6     7
 
Ph3:
 even:   1     2     3     5     4     8     6     9     7
  odd:   1     2     3     4     5     6     8     7     9

Ph4:
 even:

Example of the odd-even sort algorithm

Result after Phase 4's even phase: 

       A[0]  A[1]  A[2]  A[3]  A[4]  A[5]  A[6]  A[7]  A[8] 
         8     5     9     1     4     2     3     6     7

Ph1:
 even:   5     8     1     9     2     4     3     6     7
 odd:    5     1     8     2     9     3     4     6     7

Ph2:
 even:   1     5     2     8     3     9     4     6     7
 odd:    1     2     5     3     8     4     9     6     7
 
Ph3:
 even:   1     2     3     5     4     8     6     9     7
  odd:   1     2     3     4     5     6     8     7     9

Ph4:
 even:   1     2     3     4     5     6     7     8     9

Example of the odd-even sort algorithm

Phase 4's odd phase will compare the following pairs: 

       A[0]  A[1]  A[2]  A[3]  A[4]  A[5]  A[6]  A[7]  A[8] 
         8     5     9     1     4     2     3     6     7

Ph1:
 even:   5     8     1     9     2     4     3     6     7
 odd:    5     1     8     2     9     3     4     6     7

Ph2:
 even:   1     5     2     8     3     9     4     6     7
 odd:    1     2     5     3     8     4     9     6     7
 
Ph3:
 even:   1     2     3     5     4     8     6     9     7
  odd:   1     2     3     4     5     6     8     7     9

Ph4:
 even:   1     2     3     4     5     6     7     8     9
  odd:

Example of the odd-even sort algorithm

Result after Phase 4's odd phase: 

       A[0]  A[1]  A[2]  A[3]  A[4]  A[5]  A[6]  A[7]  A[8] 
         8     5     9     1     4     2     3     6     7

Ph1:
 even:   5     8     1     9     2     4     3     6     7
 odd:    5     1     8     2     9     3     4     6     7

Ph2:
 even:   1     5     2     8     3     9     4     6     7
 odd:    1     2     5     3     8     4     9     6     7
 
Ph3:
 even:   1     2     3     5     4     8     6     9     7
  odd:   1     2     3     4     5     6     8     7     9

Ph4:
 even:   1     2     3     4     5     6     7     8     9
  odd:   1     2     3     4     5     6     7     8     9

The CPU version of the odd-even sort algorithm 

void oddEvenSort(int a[], int n)
{
    int  isSorted = 0; // Initially array is unsorted
    int  i;

    while ( isSorted == 0 )
    {
       isSorted = 1;    // Assume the array is sorted...

        // Even phase
        for (i = 0; i < n-1; i = i + 2)
        {
           if (a[i] > a[i+1])
           {
              SWAP(a[i], a[i+1]);
              isSorted = 0;     // Detects unsorted
           }
        }

        // Odd phase
        for (i = 1; i < n-1; i = i + 2)
        {
           if (a[i] > a[i+1])
           {
              SWAP(a[i], a[i+1]);
              isSorted = 0;     // Detects unsorted
           }
        }
    }
}

DEMO: /home/cs355001/demo/CUDA/7-sort/cpu-odd-even.c