- The follow program fragment depicts a common motif
of parallel programs:
void *worker (void *arg) { .... (You will need to convert arg to the correct type before you can use the value passed) .... } int main(int argc, char *argv[]) { // SEQUENTIAL Section .... Prepare problem (setup shared variables) .... // PARALLEL Section // Start workers for (i = 0; i < NUM_PROCESSORS; i = i + 1) { param[i] = ....; pthread_create(&tid[i], &attr, worker, ¶m[i]) } // Wait for all workers to finish for (i = 0; i < NUM_PROCESSORS; i = i + 1) pthread_join(tid[i], NULL); // SEQUENTIAL Section .... Post process results from workers.... }Note:
- You can pass a parameter
of any type
to the worker() function
- But inside the worker() function, you will have to cast the value to the correct type
- You can pass a parameter
of any type
to the worker() function
- When writing a parallel program, you
must divide the work
into
(preferrably) non-overlapping subtasks
- Usually there are
many different ways
to divide a task into subtasks
(And sometimes, it is not possible to divide the task into non-overlapping tasks and you may have to repeat some steps - necessary evil in parallel programming...
- You may think that dividing the task of
"Finding the minimum value"
in an array is pretty straighforward - and yet, there are
many ways
to do so.
Some ways are better than others.
- Let's do a simple example: split the task of
"Finding the minimum value"
in an array into 2 subtasks (each subtask performed by one thread).
- Solution 1:
- Split the array into 2 (approximate) equal halfs
- Thread 1 finds the minimum in the first half of the array
- Thread 2 finds the minimum in the second half of the array
- Main thread waits for the results and find the actual minimum.
Pictorially:
values handled by values handled by thread 0 thread 1 |<--------------------->|<--------------------->| | | | | V V min[0] min[1] \ / \ / \ / \ / \ / main thread | | V Actual minimumMain Thread:
/* Shared Variables */ double x[1000000]; // Must be SHARED (accessed by worker threads !!) int start[100]; // Contain starting array index of each thread double min[100]; // Contain the minimum found by each thread int num_threads; // ----------------------------------- // Create worker threads.... // ----------------------------------- for (i = 0; i < num_threads; i = i + 1) { start[i] = i; // Pass ID to thread in a private variable if ( pthread_create(&tid[i], NULL, worker, (void *)&start[i]) ) { cout << "Cannot create thread" << endl; exit(1); } } // ----------------------------------- // Wait for worker threads to end.... // ----------------------------------- for (i = 0; i < num_threads; i = i + 1) pthread_join(tid[i], NULL); // ---------------------------------------- // Post processing: Find actual minimum // ---------------------------------------- my_min = min[0]; for (i = 1; i < num_threads; i++) if ( min[i] < my_min ) my_min = min[i];
Worker Thread:
void *worker(void *arg) { int i, s; int n, start, stop; double my_min; n = MAX/num_threads; // number of elements to handle s = * (int *) arg; // Convert arg to an integer start = s * n; // Starting index if ( s != (num_threads-1) ) { stop = start + n; // Ending index } else { stop = MAX; } my_min = x[start]; for (i = start+1; i < stop; i++ ) // Find min in my range { if ( x[i] < my_min ) my_min = x[i]; } min[s] = my_min; // Store min in private slot return(NULL); /* Thread exits (dies) */ }
- Example Program:
(Demo above code)
- Prog file: click here
Compile with: CC -mt min-mt1.C
- Let's do the "Find min" example again, now splitting the task of
"Finding the minimum value"
in an array in a different manner
- Solution 2:
- Split the array into 2 (approximate) equal halfs
- Thread 1 finds the minimum in the
odd-indexed elements
of the array
(I.e.: x[0], x[2], x[4], etc) - Thread 2 finds the minimum in the
even-indexed elements
of the array
(I.e.: x[1], x[3], x[5], etc) - Main thread waits for the results and find the actual minimum.
Pictorially:
values handled by thread 0 | | | | | | | | | | | | | | V V V V V V V V V V V V V V |-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-| ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ | | | | | | | | | | | | | | values handled by thread 1 Thread 0 Thread 1 | | | | V V min[0] min[1] \ / \ / \ / \ / \ / main thread | | V Actual minimumMain Thread: (UNCHANGED)
// ----------------------------------- // Create worker threads.... // ----------------------------------- for (i = 0; i < num_threads; i = i + 1) { start[i] = i; // Pass ID to thread in a private variable if ( pthread_create(&tid[i], NULL, worker, (void *)&start[i]) ) { cout << "Cannot create thread" << endl; exit(1); } } // ----------------------------------- // Wait for worker threads to end.... // ----------------------------------- for (i = 0; i < num_threads; i = i + 1) pthread_join(tid[i], NULL); // ---------------------------------------- // Post processing: Find actual minimum // ---------------------------------------- my_min = min[0]; for (i = 1; i < num_threads; i++) if ( min[i] < my_min ) my_min = min[i];
Worker Thread: (UNCHANGED)
void *worker(void *arg) { int i, s; double my_min; s = * (int *) arg; // Convert arg to an integer // -------------------------------------- // Find min in my range // -------------------------------------- my_min = x[s]; for (i = s+num_threads; i < MAX; i += num_threads) { if ( x[i] < my_min ) my_min = x[i]; }min[s] = my_min; // Store min in private slot return(NULL); /* Thread exits (dies) */ }It's a LOT EASIER to code the worker thread !!!
- Example Program:
(Demo above code)
- Prog file: click here
Compile with: CC -mt min-mt2.C
- Try running the programs using different threads
(the program prints the elapsed time)
- Notice that the first version have drastically improved times
on multi-processors (e.g. on compute
- Can you figure out why the second version is not doing so great ? (hint: paging...)