SPIR-V Logo.

Portable GPU binaries: SPIR-V support for AMD’s ROCm libraries

AMD’s SPIR-V adoption lets GPU software compile once and specialize on the device at runtime. But the ROCm libraries assumed the wavefront size was known at compile time, which broke SPIR-V compilation entirely. We refactored four of AMD’s core libraries to fix this, without any performance regression. In this case study we’ll give some insight on how we made rocPRIM, hipCUB, rocThrust, and rocRAND portable across past and future AMD GPUs.

About the customer

AMD makes the GPUs that compete Nvidia’s, resulting in the two companies leapfrogging in performance and functionality. We see the two brands in laptops, computers, supercomputers and AI-factories.

AMD also builds ROCm, the open GPU computing platform behind much of today’s HPC and AI software. Our collaboration goes back a decade when we helped AMD create rocRAND, their GPU random-number library, back in 2016. This project took that collaboration into a new era of the platform.

The challenge

ROCm has always compiled GPU code ahead of time, once per target architecture. A binary built today cannot run on a GPU architecture that did not exist at build time. Every new ASIC requires a new build, and shipped fat binaries grow with every supported target. AMD’s  SPIR-V adoption changes that model: compile once, specialize on the device at runtime.

Compiler support alone was not enough. The ROCm libraries depended on compile-time-known architecture details, most importantly the wavefront size. AMD GPUs come with wavefront sizes of both 32 and 64. Under AMDGPU-flavored SPIR-V, the wavefront size is no longer known at compile time — the runtime determines it when the SPIR-V assembly lowers to device-specific code. Hard-coded wavefront assumptions broke SPIR-V compilation entirely.

We were asked to make four core libraries (rocPRIM, hipCUB, rocThrust, and rocRAND) work under this new compilation model. Two constraints made the work hard: performance had to stay at the same level, and none of the existing hardware targets could regress.

Our approach

This project did not start as a SPIR-V project. In Q2 2025, AMD asked us to  start deprecating the __AMDGCN_WAVEFRONT_SIZE macro in rocPRIM. This cleanup would become the foundation for SPIR-V support. Because we removed the hard-coded assumptions first, the libraries were ready when the SPIR-V use case landed.

A dynamic dispatch system

Next, we implemented  wavefront size-based dispatch in rocPRIM. The dispatch system handles both cases: runtime-known and compile-time-known wavefront size. Algorithms use the wavefront size as a consteval value where it is known at compile time. Under SPIR-V, where the runtime translation determines the wavefront size, the dispatcher branches on it instead. This design keeps kernel tuning possible: implementations remain specialized per wavefront size, so the best-performing variant is always selected.

The  rocPRIM documentation now describes this pattern as the supported way to build the library with –offload-arch=amdgcnspirv.

Config dispatch: tuned kernels, even for SPIR-V

Dynamic dispatch alone could have cost performance. In Q1 2026, we  reworked rocPRIM’s config dispatch so it generates differently tuned kernels for every architecture, even when targeting SPIR-V as the only offload architecture. The dynamic hardware target is resolved at runtime translation, while compile-time optimization techniques still apply.

A follow-up made this even better. We added  wavefront size forwarding in device algorithms. Now that the config dispatcher knows the wavefront size, kernels compile only the right variant. This reduces kernel size and compilation time without relying on the dynamic dispatch path.

Extending the pattern across libraries

We brought rocRAND to SPIR-V in  Q4 2025, and applied the same architecture in hipCUB and rocThrust. We did the heavy lifting throughout: the design, the coding, the pull requests, and the internal code reviews. AMD’s engineers performed the final code reviews and steered the work from their roadmap side.

The result

All four libraries (rocPRIM, hipCUB, rocThrust, and rocRAND) now support building with target-agnostic SPIR-V. The first public release carrying this work is  rocPRIM 4.6.0, which shipped with ROCm 10.0.0 on 26 August 2026.

The feat here is engineering complexity, not a speed-up. Performance stays exactly where it was, with zero regressions on all existing hardware and compilation targets. One SPIR-V binary now works across wavefront sizes 32 and 64, and on architectures that did not exist when the binary was built.

Key takeaway

Portable binaries no longer mean slow binaries. The dispatch architecture we built for AMD’s ROCm libraries lets one SPIR-V binary adapt to the device it lands on, without giving up the compile-time optimization techniques that made these libraries fast in the first place.

Did you enjoy reading about how we brought SPIR-V support to AMD’s rocPRIM, hipCUB, rocThrust, and rocRAND? You can read about more projects we worked on by checking out our other case studies.