- Reminder:
- The user will (can)
only
interact with the
CPU portion of the program:
Examples: if you write a C program for the CUDA environment, then:
- A CUDA C program must use
C's scanf( )
(a CPU function) to
read in
user input
- A CUDA C program must use
C's printf( )
(a CPU function) to
write output to
user's terminal
- And so on...
Any user interaction will be performed through CPU C-functions
- A CUDA C program must use
C's scanf( )
(a CPU function) to
read in
user input
- The user will (can)
only
interact with the
CPU portion of the program:
- When a CUDA program is
run by
the computer:
- The CPU portion of the CUDA program is stored in (host) RAM memory
- The GPU portion of the
CUDA program is
stored in the
device memory
Graphically:
- Note that:
- The variables in the CPU program are stored in (host) RAM memory
- The variables in the
GPU program are
stored in
device memory
- The CPU cannot access the variables in device memory !!!
- The GPU cannot access the variables in (host) RAM memory !!!
- Execution of a
CUDA program:
- The user
CUDA program
starts the
program execution at
the main( ) CPU function:
The program execution will remain in the CPU code portion as long as the program does not need to use GPU code
- The CPU program
(that the user can
interact with) can
call (= run) a
GPU function (= program) that
is executed by
the GPU graphic processor:
Note:
- The CPU portion of the CUDA program will be paused until the CUDA function has finished (= return)
- The user
CUDA program
starts the
program execution at
the main( ) CPU function:
- Important note:
- The RAM memory used
by the CPU program and
the device memory used by the
GPU program are
separate memory devices:
Meaning:
- A CPU function (code) can
only access
RAM memory
(and cannot access
device memory)
- A GPU function (code) can only access device memory (and cannot access RAM memory)
- A CPU function (code) can
only access
RAM memory
(and cannot access
device memory)
- The RAM memory used
by the CPU program and
the device memory used by the
GPU program are
separate memory devices:
- $64,000 question:
- How do GPU functions receive (= obtain) their input values and return their output values ????
- Solution:
- CUDA provide
special
transfer functions to
transfer data between
RAM memory and
device memory:
- From RAM memory to
device memory:
- From device memory to
RAM memory:
- From RAM memory to
device memory:
- CUDA provide
special
transfer functions to
transfer data between
RAM memory and
device memory:
- How to
call a
GPU function
(from a CPU function
like main( )):
- Transfer
data that need to be
processed to
the device memory:
- The CPU function
calls (= runs)
the desired
GPU function:
The GPU function can now access a copy of the data in the device memory !!!
- The GPU function
will now execute and
process (= change) the
data:
- When the GPU function
returns, the
CPU function will
continue execution:
- It will transfer the
updated data in
the device memory to
the RAM meory:
Now the CPU program can display the result to the user !!!
-
Initial state:
-
The user data
always resides
initially inside the
RAM
(because the user can only interact with the CPU program code):
- Transfer
data that need to be
processed to
the device memory: