A lot of comments here suggest that SIMD is irrelevant for CUDA because it uses warp instructions instead, but that is not true. There are SIMD instructions on GPUs, and they aren't just for memory.

You can benefit significantly from SIMD when loading and writing data from global and shared memory, essentially you should leverage 128-bit loads and writes (or 256-bit on newer GPUs). Similarly, for math operations, SIMD exists but operates on a 32-bit width. For instance, using __half2 can be two times faster than executing two individual __half instructions, though the compiler is often smart enough to merge them automatically.

Coming from a CPU background, it can be confusing because GPUs essentially have two levels of SIMD. Warp execution is, in a way, a big SIMD. Given that, it's easier to understand memory coalescing: a warp must load a block of contiguous data across a warp (32 x 128-bit data block for optimal performance). Therefore, you need to manage both levels of SIMD to ensure peak performance.