Learning both OpenCL and CUDA

Be sure to read Taking on OpenCL where I’ve put my latest insights – also for CUDA.

Should you learn OpenCL and CUDA one after the other, or both at once? The two¹ “camps” each claim you should learn their language first, after which the other is easy. I’m from the OpenCL camp, so I say: learn OpenCL first, with a strong emphasis on understanding the hardware architecture. Had I chosen CUDA, I would have said the opposite. In other words, the order does not matter much. Psychology does play a role, though. You will probably prefer your first language, because that is where you discovered the magic. Also, most people do not like learning a second language that is very similar and adds little. Most programmers just want to get the job done, and both camps know that. Keep this in mind.

NVIDIA is very good at marketing its products. AMD has, to put it mildly, a smaller budget for GPGPU marketing. As a programmer, you should be aware of this difference.

OpenCL offers more possibilities than CUDA, because it supports task-parallel programming and many more architectures. On the other hand, CUDA is far more user-friendly and has a lot of convenience built in.

Why learn OpenCL and CUDA at the same time?

If you learn OpenCL and CUDA at the same time, you get a better overview. With OpenCL, you cannot rely on a convenient, vendor-specific approach. Instead, you have to learn a lot more about the hardware. In my opinion, that is a good investment. If you choose CUDA only, you miss that understanding, as well as support for other architectures.

Learning both is also easier than it sounds. The differences reveal many advantages and disadvantages that the documentation does not mention.

How to learn OpenCL and CUDA: a six-step study plan

Make your own study plan based on the following steps:

1. Learning the basics of GPGPU, like kernels, host, device, memory-transfers, basic commands, etc.
2. Build your first program, and modify the sample programs in the SDKs.
3. Understand the architectures of NVIDIA, AMD, ARM and IBM Cell.
4. Learn how to design larger, multi-kernel programs, and actually write them.
5. Learn to optimise for the architectures you target.
6. Get to know the design patterns for integrating GPGPU into existing software.

Notice that neither CUDA nor OpenCL is mentioned here, nor is any other GPGPU environment.

Steps 1-2: the basics

The first two steps form the base. Here you get a grasp of the technology and learn a few ways to look at it. Current OpenCL and CUDA compilers do not do much for you, unlike the X86 compilers you are used to. Therefore, you need to understand more about the hardware. NVIDIA publishes a lot of information about its architectures, but the same information for other hardware is harder to find.

Step 3: understanding GPU architectures

Understanding the architectures is a very important part. CUDA books give the best description of NVIDIA architectures (concepts like “warps”). Other architectures, however, are rarely described from a GPGPU perspective. Once you understand how GPUs work, you have a better idea of how to optimise.

Two tips. First, Multiscalelab’s SWAN converter helps you convert CUDA code to OpenCL. Second, OpenCL’s embedded profile is available for low-power devices like smartphones, so take it into account when you study the different ARM architectures.

Steps 4-6: bringing GPU power to your software

The last three steps are about upgrading your current software to use GPU power. This requires well-written software, so you might need to study a few design pattern books before you go too far into GPGPU.

Task-parallel programming is supported by OpenCL but not by CUDA. The market currently focuses on data-parallel programming, so GPUs are not really optimised for task-parallel work. You can therefore skip it for 2010. As soon as NVIDIA adds it to CUDA, expect the market to follow.

Final word

I’m in the OpenCL camp for a reason. I can choose between NVIDIA, AMD, IBM, ARM² and upcoming FPGAs. Once one architecture offers a lower price per FLOP for an HPC application, I can switch to it. Still, I learnt NVIDIA’s architecture best by learning CUDA. That is exactly why I recommend you learn OpenCL and CUDA side by side.

---

¹ I did not consider the third camp, Microsoft’s DirectCompute, because in my opinion it is more a rebranded OpenCL than a real alternative. I do like a good discussion, though.

² See my previous blog post, which explains that ARM has no OpenCL yet.