While CUDA has had the advantage of having many more libraries, this is no longer its main advantage if it comes to linear algebra. If one thing changed over the past year, then it is linalg library-support for OpenCL. The choices have been increased at a continuous rate, as you can see the below list.
A general remark when using these libraries. When using them you need to handle your data-transfers and correct data-format, with great care. If you don’t think it through, you won’t get the promised speed-up. If not mentioned, then free.
| Subject | CUDA | OpenCL |
|---|---|---|
| FFT |
 The NVIDIA CUDA Fast Fourier Transform library (cuFFT) provides a simple interface for computing FFTs up to 10x faster. By using hundreds of processor cores inside NVIDIA GPUs, cuFFT delivers the… |
clFFT is a software library containing FFT functions written in OpenCL. In addition to GPU devices, the library also supports running on CPU devices to facilitate debugging and multicore programming. |
| Linear Algebra |
 MAGMA is a collection of next generation, GPU accelerated ,linear algebra libraries. Designed for heterogeneous GPU-based architectures. It supports interfaces to current LAPACK and BLAS standards. |
clMAGMA is an OpenCL port of MAGMA for AMD GPUs. The clMAGMA library dependancies, in particular optimized GPU OpenCL BLAS and CPU optimized BLAS and LAPACK for AMD hardware, can be found in the AMD Accelerated Parallel Processing Math Libraries (APPML). |
| Sparse Linear Algebra |
 CUSP is an open source C++ library of generic parallel algorithms for sparse linear algebra and graph computations on CUDA architecture GPUs. CUSP provides a flexible, high-level interface for manipulating sparse matrices and solving sparse linear systems. |
clBLAS implements the complete set of BLAS level 1, 2 & 3 routines. Please see Netlib BLAS for the list of supported routines. In addition to GPU devices, the library also supports running on CPU devices to facilitate debugging and multicore programming.ViennaCL is a free open-source linear algebra library for computations on many-core architectures (GPUs, MIC) and multi-core CPUs. The library is written in C++ and supports CUDA, OpenCL, and OpenMP. In addition to core functionality and many other features including BLAS level 1-3 support and iterative solvers, the latest release ViennaCL 1.5.0 provides many new convenience functions and support for integer vectors and matrices.VexCL is a vector expression template library for OpenCL/CUDA. It has been created for ease of GPGPU development with C++. VexCL strives to reduce amount of boilerplate code needed to develop GPGPU applications. The library provides convenient and intuitive notation for vector arithmetic, reduction, sparse matrix-vector products, etc. Multi-device and even multi-platform computations are supported. |
| Random number generation |
 The NVIDIA CUDA Random Number Generation library (cuRAND) delivers high performance GPU-accelerated random number generation (RNG). The cuRAND library delivers high quality random numbers 8x… |
The Random123 library is a collection of counter-based random number generators (CBRNGs) for CPUs (C and C++) and GPUs (CUDA and OpenCL). They are intended for use in statistical applications and Monte Carlo simulation and have passed all of the rigorous SmallCrush, Crush and BigCrush tests in the extensive TestU01 suite of statistical tests for random number generators. They are not suitable for use in cryptography or security even though they are constructed using principles drawn from cryptography. |
 The CUDA Math library is an industry proven, highly accurate collection of standard mathematical functions. Available to any CUDA C or CUDA C++ application simply by adding “#include math.h” in… |
Looking into the details of what the CUDA math lib exactly is. | |
| AI |
 A technology preview with CUDA accelerated game tree search of both the pruning and backtracking styles. Games available: 3D Tic-Tac-Toe, Connect-4, Reversi, Sudoku and Go. |
There are many tactics to speed up such algorithms. This CUDA-library can therefore only be used for limited cases, but nevertheless it is a very interesting research-area. Ask us for an OpenCL based backtracking and pruning tree searching, tailored for your problem. |
| Dense Linear Algebra |
 Provides accelerated implementations of the LAPACK and BLAS libraries for dense linear algebra. Contains routines for systems solvers, singular value decompositions, and eigenproblems. Also provides various solvers.
Free (with limitations) and commercial.
|
See ViennaCL, VexCL and clBLAS above. Kudos to the CULA-team, as they were one of the first with a full GPU-accelerated linear algebra product. |
| Fortran |
 The IMSL Fortran Numerical Library is a comprehensive set of mathematical and statistical functions available that offloads CPU work to NVIDIA GPU hardware where the cuBLAS library is utilized.
Free (with limitations) and commercial.
|
OpenCL-FORTRAN is not available yet. Contact us, if you have interest and wish to work with a pre-release once available. |
| Subject |
 Comprehensive GPU function library, including functions for math, signal processing, image processing, statistics, and more. Interfaces for C, C++, Fortran, and Python. Integrates with any CUDA-program. Free (with limitations) and commercial. |
ArrayFire 2.0 is also available for OpenCL. Note that currently fewer functions are supported in the OpenCL-version than are supported in CUDA-ArrayFire, so please check the OpenCL documentation for supported feature list.Free (with limitations) and commercial. |
| Subject |
 The NVIDIA Performance Primitives library (NPP) is a collection of over 1900 image processing primitives and nearly 600 signal processing primitives that deliver 5x to 10x faster performance than… |
Kudos for NVIDIA for bringing it all at one place. OpenCL-devs have to do some googling for specific algorithms. |
So the gap between CUDA and OpenCL is certainly closing. CUDA provides a lot more convenience, so OpenCL-devs still have to keep reading blogs like this one to find what’s out there.
As usual, if you have additions to this list (free and commercial), please let me know in the comments below or by mail. I also have a few more additions to this list myself – depending on your feedback, I might represent the data differently.
Hi,
your comparing sparse and dense linear algebra libraries here. I think that does not make sense. You miss the excellent cuBLAS library which is the Nvidia equivalent of clBLAS.
Cheers,
Sebastian
I know, I know. :/ There is overlap in libraries and I am *not* happy with how I’ve currently represented it. First I’d like to get some more feedback, before I do it differently.
Why isn’t cuBLAS on their libraries-page, by the way?
The wish to say that OpenCL is an alternative to CUDA is so rediculus and the claims here are only proving this.
How can this compare be of any use if there’s no comparison of the performance of each CUDA supported lib versus OpenCL lib? A comparison of the available functionality in both libraries, support for big/small matrixes, support for multi-gpu, multi-queues…. what kind of comparison is that?
For example:
– ViennaCL – can you compare this to a full fledged cuBLAS? they dont support complex types for example. what about other functionality? performace?
– CUDA math library versus a “math.h include in OpenCL” – is that serious? obviously you didnt even look at what the CUDA math library has to offer (just click the link you’ve supplied and compare it to the glorious header file in OpenCL”.
– No mention of cuBLAS
Thank you for your feedback and being critical. A week ago I replied to Sebastian, that I am looking for a better way to compare the two – the current table doesn’t really work out well. I hope you like the next version better.
CuBLAS will be added in the rewrite. Same for math library, as you are right that it is not 100% coverage – I did find functions in other libraries.
Please do share a blog article where you compare CUDA with OpenCL libraries and why the CUDA-libs are more advanced. I dislike comparing all things at the same time. I have done some testing here on FirePro-cards and will think about what I can share.