Let me explain a simple vector addition operation and specifically how we can do it in a GPU. This example is in the source
From this note, we understood that we can do operations independently and do operations much faster, we can create thousands and millions of threads to operate independent operations
Naive GPU Vector Addition
These are the steps to follow:
- Allocate device memory for vectors like malloc in c
- Transfer inputs host -> Device (Here host means CPU and Device means GPU)
- Launch Kernel and perform additions (Kernels are simply functions that are defined and run on GPUs)
- Copy or transfer back the output from Device -> Host
- Free Device memory like
free()in C language
Note
Normally, we keep data on the GPU as long as possible to asynchronously schedule many kernel launches

This image is taken from this video from GPU Mode channel
Here, we have to vectors and , for each element of these two vectors we launch a thread and a single thread calculates the output for a single element to output vector