Engineering GPGPU into existing software

At the Thalesian talk about OpenCL I gave in London it was quite hard to find a way to talk about OpenCL for a very diverse public (without falling back to listing code-samples for 50 minutes). Some already knew just about everything about HPC, while others have only heard of CUDA and/or OpenCL. One of the subjects I chose to talk about was how to integrate OpenCL (or GPGPU in common) into existing software. The reason is that we all have built nice, cute little programs that are super-fast… But it’s another story when we want to integrate the nice, cute little programs into some enterprise-level software.

Readiness

The most important step is making your software ready. Software engineering can be very hectic. Managing this in a nice matter (i.e. PRINCE2) just doesn’t fit in a deadline-mined schedule. We all know it costs less time and money when looking at the total picture, but time is just against us.

Let’s exaggerate. New ideas, new updates of algorithms, new tactics and methods all arrive at the wrong moment, Murphy-wise. Your program should’ve been finished yesterday, so we can only test when in production. Programmers just have to understand the cost of delay, but luckily your manager comes to the rescue and says: “It is my responsibility”. Finally, after a year of stress, your software is the best in the company and becomes a “platform”. Which means that your software eventually includes all small little ideas and scripts your colleagues have come up, which are” almost the same as your software”. This will turn the platform into something unmanageable. That is a different kind of software-acceptance!

Needs, Problems & Solutions

We need speed, easy conversion from one domain to the chosen IT-domain and need it for a low price. Speed is limited by the hardware and the software-architecture. The problem is that there is no money for change-management. The logical thing would be to add GPGPU to speed up calculations and that’s it.

As you have read above this should be the last step. First the software need to be ready, by separating the flow and algorithms and use good version-management. Instead of building more and more into one program, all small projects should stay small and provide a service. These small services can be linked together and monitored, while the report-generation gathers it’s information in a flexible way.

Design Patterns

Good design-patterns are strategy and chain of responsibility, depending on taste. The chain of responsibility pattern is a good way to not have to worry about how the data is processed up later down the line. But the strategy pattern can help to support change-management where the beta-report can use the algorithm-implementation-that-had-to-be-finished yesterday and the production-live report uses the latest stable one.

I think you and I should think about engineering the software around accelerated new pieces of library-code. The reason for this, is that GPGPU is new and it just might not work as expected or it cannot deal with that specific kind of data nobody thought about. The kernels should have pre-conditions to check validity of the input-data, but actually that is too late.

Hey, we just want speed!

And you’re right! All these extra layers could make the software less flexible. So that makes it even nicer to find ways of data-transport that is fast enough. But look at your architecture first. For example, loading data from a database-server from another location could be a bigger bottle-neck than the calculation-throughput. If you have measured every step in the process, you are ready for integrating OpenCL and buy those wanna-have-it hardware of NVIDIA, AMD, Intel* or IBM*. As a certified Java-programmer I insist, and I promise you get more of the speed you want.

StreamHPC will release a cross-platform plugin for Eclipse to easily write OpenCL-programs. For who don’t know Eclipse, it is a cross-platform multi-language IDE. If you have any requests, please comment below or send me an e-mail. For pre-orders and questions about i.e. volume licenses, please mail us at sales[at]StreamHPC[dot]eu