- CUDA:
- CUDA =
acronym for
Compute Unified Device Architecture
CUDA is a (vector) parallel computing platform and application programming interface (API) model created by Nvidia (a graphics card company).
- CUDA =
acronym for
Compute Unified Device Architecture
- I have shown you
a picture of the
CUDA structure before:
- Here is a
cleaner but
more abstract
diagram of the
CUDA architecture:
- A CUDA program consists of
2 parts:
- A CPU part that consists of
CPU instructions
(i.e., "normal" computer instructions
stored in
the RAM memory
that is executed by
the CPU
- A GPU part that
consists of GPU (graphics processor)
instructions stored in
the (GPU) device memory
that
is executed by
the vector processing
GPU (graphics processor):
- A CPU part that consists of
CPU instructions
(i.e., "normal" computer instructions
stored in
the RAM memory
that is executed by
the CPU
- Primary interaction between
the user and
the CUDA environment:
- The user can
only
interact with the
CPU portion of the program:
Examples: if you write a C program for the CUDA environment, then:
- A CUDA C program must use
C's scanf( )
(a CPU function) to
read in
user input
- A CUDA C program must use
C's printf( )
(a CPU function) to
write output to
user's terminal
- And so on...
Any user interaction will be performed through CPU C-functions
- A CUDA C program must use
C's scanf( )
(a CPU function) to
read in
user input
- The user can
only
interact with the
CPU portion of the program:
- Execution of a
CUDA program:
- When a CUDA program is
run by
the computer:
- The CPU portion of the CUDA program is stored in RAM memory
- The GPU portion of the
CUDA program is
stored in the
device memory
- The user
CUDA program
starts the
program execution at
the main( ) CPU function:
The program execution will remain in the CPU code portion as long as the program does not need to use GPU code
- The CPU program
(that the user can
interact with) can
call (= run) a
GPU function (= program) that
is executed by
the GPU graphic processor:
Note:
- The CPU portion of the CUDA program will be paused until the CUDA function has finished (= return)
- When a CUDA program is
run by
the computer:
- Important note:
- The RAM memory used
by the CPU program and
the device memory used by the
GPU program are
separate memory devices:
Meaning:
- A CPU function (code) can
only access
RAM memory
(and cannot access
device memory)
- A GPU function (code) can only access device memory (and cannot access RAM memory)
- A CPU function (code) can
only access
RAM memory
(and cannot access
device memory)
- The RAM memory used
by the CPU program and
the device memory used by the
GPU program are
separate memory devices:
- $64,000 question:
- How do GPU functions receive (= obtain) their input values and return their output values ????
- Solution:
- CUDA provide
special
transfer functions to
transfer data between
RAM memory and
device memory:
- From RAM memory to
device memory:
- From device memory to
RAM memory:
- From RAM memory to
device memory:
- CUDA provide
special
transfer functions to
transfer data between
RAM memory and
device memory:
- How to
call a
GPU function
(from a CPU function
like main( )):
- Transfer
data that need to be
processed to
the device memory:
- The CPU function
calls (= runs)
the desired
GPU function:
The GPU function can now access a copy of the data in the device memory !!!
- The GPU function
will now execute and
process (= change) the
data:
- When the GPU function
returns, the
CPU function will
continue execution:
- It will transfer the
updated data in
the device memory to
the RAM meory:
Now the CPU program can display the result to the user !!!
-
Initial state:
-
The user data
always resides
initially inside the
RAM
(because the user can only interact with the CPU program code):
- Transfer
data that need to be
processed to
the device memory: