OpenCL Basics: Running multiple kernels in OpenCL

Reading Time: 1 minute

This series “Basic concepts” is based on GPGPU-questions we get via email more than once, or when the question is not clearly explained in the books. For one it is obvious, for the other just what they’re missing.

They say that learning a new technique is best done by playing around with working code and then try to combine it. The idea is that when you have Stackoverflowed and Githubed code together, you’ve created so many bugs by design that you’ll learn a lot if you make it work. When applying this to OpenCL, you quickly get to a situation that you want to run one.cl file and then another.cl file. Almost all beginner’s material discuss a single OpenCL-file, so how to do this elegantly?

Continue reading “OpenCL Basics: Running multiple kernels in OpenCL”

Start your GPU-career here

Reading Time: 2 minutes

GPUs have been our mysterious friends and known enemies for years, as they let us run code in expected and unexpected ways. GPUs have solved problems for many of our customers. GPUs have such a high rate of evolvement, that they’ll remain important for the years to come.

Problem is that programming GPUs is not an easy task. Where do you learn to program GPUs? We found these to be the main groups:

  • Universities
  • Research centers
  • GPU vendors (AMD, Nvidia, Intel, Qualcomm, ARM)
  • Self-study

This is far from enough. Add to that, that only a very select group learns the craft at a company. We’d like to change that, and we think now is the time for us to be able to deliver on this.

In January we’ll our internal training program will start with 4 to 8 developers. Focus in on fully understanding recent GPU-architectures, CUDA and OpenCL. It will consist of lectures, workshops, discussions, paper reading and ofcourse coding for one month. The months after that will have guidance, paper presentations, code reviews and time for self-study. The exact form will differ per person.

The hard side

The current measurable requirements are:

  • EU citizen or already having a working permit
  • Great at C/C++
  • High interest in algorithmic optimisations
  • Any performance improvement focus (i.e. Assembly, clean code) is a plus
  • Any GPU experience (i.e. OpenGL, DirectX, self-study) is a plus
  • High interest in performance
  • Willing to move to Amsterdam by January (or earlier)
  • Willing to work for Stream HPC for at least 2 years

The soft side

We’re looking for people that fit our culture and we think we can train. This means that the selection is based for a large part on “the spark”. Therefore the application starts with a speed date, and we’re sorry for not finding a better wording for this. This is a 20 minute discussion about what we like and what we don’t. This can be done via phone, Skype or in person, during the evening, in the weekends or during your lunch break.

The fine print

To avoid people changing their mind after the training and returning to their old job, the training is paid off only after 2 years.

How to apply

Read about our company culture. Look at the jobs we have open. These describe the requirements after the training. Then write us a motivational letter: explain us why this is exactly what you want, why you’re capable and why you’re a cultural fit. If you find it hard to write such letter, then just start with answering the list of requirements. It’s a big bonus to share code (Github, Gitlab, zip-file). Send your email to jobs@streamhpc.com

Other jobs

Feeling more senior? We have other jobs:

    What does it mean to work at Stream HPC?

    Reading Time: 6 minutes

    High performance computing on many-core environments and low-level optimisations are very important concepts in large scientific projects nowadays. Stream HPC is one of the market’s more prominent clubs and is substantially expanding. As we often get asked how it is to work at the company, we’d like to give you a little peak into our kitchen.

    Stream HPC’s DNA

    To understand our DNA, you’d need to know how the company got started. In 2010 the company was born from the deep boredom that was born from within the corporate IT workspace. Stream’s founder Vincent Hindriksen had to maintain a piece of software that was often failing to process the daily reports. After documenting the internals and algorithms of the code by interviewing the key people and some reverse engineering, it was a lot easier to create effective solutions for the bugs within the software. After fixing a handful of bugs, there was simply a lot less to do except reading books and playing online games.

    To avoid becoming a master in Sudoku, he spent the following three weeks in rewriting all the code, using the freshly produced documentation. 2.5 hours needed to process the data was reduced to 19 seconds – yes, the kick for performance optimisation was already there. For some reason it took well over 6 months to port the proof-of-concept, which was simply unbearable as somebody had to make sure the old code was maintained for 40 hours a week.

    The reason to start the company was simple: to make intelligent use of time and provide software that is engineered for performance and maintainability. Lots of exciting projects for fantastic clients followed in the next 8 years that allowed us to broaden our expertise and build up confidence. GPUs were there at just the right time – without GPUs it would have probably been performance engineering on CPUs.

    Continue reading “What does it mean to work at Stream HPC?”

    Meet Vincent in Bay Area between 11 and 16 August

    Reading Time: 1 minute

    Our managing director, Vincent Hindriksen, is in San Francisco’s Bay Area from Saturday 11th up to Thursday 16th of August 2018. He’ll be visiting existing customers, but there is time left.

    Current schedule (excluding several unconfirmed meetings):

    • Saturday: social meetups
    • Monday: full
    • Tuesday: all day good availability,
    • Wednesday: all day good availability
    • Thursday: morning good availability

    Do you want to learn more about GPUs and how we can help you get there? Get in touch via our contact-page, and tell us address and time when you want to meet.

    If you seek a job in GPUs, also get in contact! Stream HPC is growing quickly now, and a good moment to onboard and still make a difference. For job-talks also the evenings are available.

    Help us find our future COO

    Reading Time: 2 minutes

    Is this a motto that goes with your personality? Then we want to talk with you.
    After 8 years the time has come that we have continuous growth for almost 2 years, instead of dealing with the usual peaks and lows of consultancy. I’d like to get your help to find our future COO to help streamline this growth.
    You might have seen that there are hardly any new blog posts – now you know why. By helping us find that special person, there can be put more time of writing new blog posts again.
    If you know the perfect person for this job in Amsterdam, please let them know there is this unique company looking for her or him. Sharing this blog-post would help a lot.
    You can find more information in this job-post:

    We all know that quality comes with attention to detail, but also that with growth the details are the first to be postponed. We seek help in handling daily operations during our growth. The most important tasks are:

    • Customer contact. You make sure the communication is regular and smooth with all our customers, making them more engaged and happy with us.
    • Sales follow up. You take over to discuss the needs of potential customers pre-sales has had contact with.
    • Team support. You help the development-teams to get even better by helping them to solve their daily and long-term problems.

    The job is very broad, but is all around a listening ear and getting things done.

    You have studied business administration or alike, and have a can-do attitude. You know how to work with technical people and are a real team-player. You understand how to develop and engage group dynamics.

    Do you think this is a job written for you, then we would like to hear more from you! Send an email to jobs@streamhpc.com with a motivational letter and listing relevant experience.

    Thanks for helping out!

    If you got sent here, we hope to hear from you!

    How to speed up Excel in 6 steps

    Reading Time: 3 minutes

    After the last post on Excel (“Accelerating an Excel Sheet with OpenCL“), there have been various request and discussions how we do “the miracle”. Short story: we only apply proper engineering tactics. Below I’ll explain how you can also speed up Excel and when you actually have to call us.

    Excel is a special piece of software from a developer’s perspective. An important rule of software engineering is to keep functionality (code) and data separate. Excel mixes these two as no other, which actually goes pretty well in many cases unless the data gets too big or the computations too heavy. In that case you’ve reached Excel’s limits and need to properly solve it.

    Below are the steps to go through, of which most you can do yourself! Continue reading “How to speed up Excel in 6 steps”

    Call for speakers: IEEE eScience Conference in Amsterdam

    Reading Time: 2 minutes

    We’re in the program committee of the 14th IEEE eScience Conference in Amsterdam, organized by the Netherlands eScience Center. It will be held from 29 October to 1 November 2018, and the deadlines for sending the abstracts is Monday 18 June.

    The conference brings together leading international researchers and research software engineers from all disciplines to present and discuss how digital technology impacts scientific practice. eScience promotes innovation in collaborative, computationally- or data-intensive research across all disciplines, throughout the research lifecycle.

    Continue reading “Call for speakers: IEEE eScience Conference in Amsterdam”

    Do you want to join StreamHPC?

    Reading Time: 1 minute

    As of this month Stream exists 8 years. 8 full years of helping our customers with fast software.In Chinese numerology 8 is a very lucky number, and we notice that.

    Over the years we’ve kept focus on quality and that was a good decision. The only problem is that we don’t have enough time to write on the blog, to organise events or even send the “monthly” newsletter. With over 200 drafts for the blog (subjects that really should be shared), we need extra people to help us out.

    Dear developers who are good with C,C++, OpenCL/CUDA and algorithms, please take a look at the following vacancies. I know you are frequenting our blog.

    We’re also seeking an all-rounder that supports in daily operations, that includes management, customer contact, team-support, etc.

    See below for more details.

      We’re looking forward to your application! We accept both remote and Amsterdam-based.

      Selecting Applications Suitable for Porting to the GPU

      Reading Time: 5 minutes

      Assessing software is never comparing apples to apples

      The goal of this writing is to explain which applications are suitable to be ported to OpenCL and run on GPU (or multiple GPUs). It is done by showing the main differences between GPU and CPU, and by listing features and characteristics of problems and algorithms, which can make use of highly parallel architecture of GPU and simply run faster on graphic cards. Additionally, there is a list of issues that can decrease potential speed-up.

      It does not try to be complete, but tries to focus on the most essential parts of assessing if code is a good candidate for porting to the GPU.

      GPU vs CPU

      The biggest difference between a GPU and a CPU is how they process tasks, due to different purposes. A CPU has a few (usually 4 or 8, but up to 32) ”fat” cores optimized for sequential serial processing like running an operating system, Microsoft Word, a web browser etc, while a GPU has a thousands of ”thin” cores designed to be very efficient when running hundreds of thousands of alike tasks simultaneously.

      A CPU is very good at multi-tasking, whereas a GPU is very good at repetitive tasks. GPUs offer much more raw computational power compared to CPUs, but they would completely fail to run an operating system. Compare this to 4 motor cycles (CPU) of 1 truck (GPU) delivering goods – when the goods have to be delivered to customers throughout the city the motor cycles win, when all goods have to be delivered to a few supermarkets the truck wins.

      Most problems need both processors to deliver the best value of system performance, price, and power. The GPU does the heavy lifting (truck brings goods to distribution centers) and the CPU does the flexible part of the job (motor cycles distributing doing deliveries).

      Assessing software for GPU-porting fitness

      Software that does not meet the performance requirement (time taken / time available), is always a potential candidate for being ported to a GPU. Continue reading “Selecting Applications Suitable for Porting to the GPU”

      DOI: Digital attachments for Scientific Papers

      Reading Time: 3 minutes

      Ever saw a claim on a paper you disagreed with or got triggered by, and then wanted to reproduce the experiment? Good luck finding the code and the data used in the experiments.

      When we want to redo experiments of papers, it starts with finding the code and data used. A good start is Github or the homepage of the scientist. Also Gitlab. Bitbucket, SourceForge or the personal homepage of one of the researchers could be a place to look. Emailing the authors is often only an option, if the university homepage mentions such option – we’re not surprised to get no reaction at all. If all that doesn’t work, then implementing the pseudo-code and creating own data might be the only option – not if that will support the claims.

      So what if scientific papers had an easy way to connect to digital objects like code and data?

      Here the DOI comes in.

      Continue reading “DOI: Digital attachments for Scientific Papers”

      Learn about AMD’s PRNG library we developed: rocRAND – includes benchmarks

      Reading Time: 3 minutes

      When CUDA kept having a dominance over OpenCL, AMD introduced HIP – a programming language that closely resembles CUDA. Now it doesn’t take months to port code to AMD hardware, but more and more CUDA-software converts to HIP without problems. The real large and complex code-bases only take a few weeks max, where we found that solved problems also made the CUDA-code run faster.

      The only problem is that CUDA-libraries need to have their HIP-equivalent to be able to port all CUDA-software.

      Here is where we come in. We helped AMD make a high-performance Pseudo Random Generator (PRNG) Library, called rocRAND. Random number generation is important in many fields, from finance (Monte Carlo simulations) to Cryptographics, and from procedural generation in games to providing white noise. For some applications it’s enough to have some data, but for large simulations the PRNG is the limiting factor. Continue reading “Learn about AMD’s PRNG library we developed: rocRAND – includes benchmarks”

      GPU and FPGA challenge for MSc and PhD students

      Reading Time: 3 minutes

      While going through my email, I found out about the third “HiPEAC Student Heterogeneous Programming Challenge”. Unfortunately the deadline was last week, but just got an email: if you register by this weekend (17 September), you can still join.

      EDIT: if you joined, be sure to comment in early November how it was. This would hopefully motivate others to join in next year. Continue reading “GPU and FPGA challenge for MSc and PhD students”

      The single-core, multi-core and many-core CPU

      Reading Time: 3 minutes

      Multi-core CPU from 2011

      CPUs are now split up in 3 types, depending on the number of cores: single (1), multi (2-8) and many (10+).

      I find it more important now to split up into these three types, as the types of problems to be solved by each is very different. Based on the problem-differences I’m even expecting that the number of cores between multi-core CPUs and many-core CPUs will grow.

      Below are the three types of CPUs discussed and a small discussion on many-core processors we see around. Continue reading “The single-core, multi-core and many-core CPU”

      HPC centre EPCC says: “Better software, better science”

      Reading Time: 2 minutes

      The University of Edinburgh houses the HPC centre EPCC. Neelofer Banglawala wrote about a programme which funds the development and improvement of scientific software, and also discussed about the results.

      Many of the 10 most used application codes on ARCHER have been the focus of an eCSE project. Software with more modest user bases have improved user uptake and widened their impact through eCSE-funded work. Furthermore, performance improvements can lead to tens of thousands of pounds of savings in compute time.

      Saving tens of thousands of pounds is certainly worth the investment. This also means more users can work on the same supercomputer, thus reducing waiting times. Continue reading “HPC centre EPCC says: “Better software, better science””

      Demo: cartoonizer on an Altera Arria 10 FPGA

      Reading Time: 2 minutes

      It takes quite some effort to program FPGAs using VHDL or Verilog. Since several years Intel/Altera has OpenCL-drivers, with the goal to reduce this effort. OpenCL-on-FPGAs reduced the required effort to a quarter of the time, while also making it easier to alter the specifications during the project. Exactly the latter was very beneficiary when creating the demo, as the to-be-solved problem was vaguely defined. The goal was to make a video look like a cartoon using image filters. We soon found out that “cartoonized” is a vague description, and it took several iterations to get the right balance between blur, color-reduction and edge-detection. Continue reading “Demo: cartoonizer on an Altera Arria 10 FPGA”

      CPU Code modernisation – our hidden expertise

      Reading Time: 2 minutes

      You’ve seen the speedups possible on GPUs. We secretly know that many of these techniques would also work on modern multi-core CPUs. If after the first optimisations the GPU still gets an 8x speedup, the GPU is the obvious choice. When it’s 2x, would the better choice be a bigger CPU or a bigger GPU? Currently the GPU is chosen more often.

      Now AMD, Intel and AMD have 28+ core CPUs, the answer to that question might now lean towards the CPU. With a CPU that has 32 cores and 256bit vector-computations via AVX2, each clock-cycle 32 double4 can be computed. A 16-core AVX1 CPU could work on 16 double2’s, which is only a fourth of that performance. Actual performance compared to peak-performance is comparable to GPUs here. Continue reading “CPU Code modernisation – our hidden expertise”

      New training dates for OpenCL on CPUs and GPUs!

      Reading Time: 1 minute

      OpenCL remains to be a popular programming language for accelerators, from embedded to HPC. Good examples are consumer software and embedded devices. With Vulkan potentially getting OpenCL-support in the future, the supported devices will only increase.

      For multicore-CPUs and GPUs we now have monthly training dates for the rest of the year:

      Minimum number of participants is two. By request the location and date can be changed.

      The first day of the training is the OpenCL Foundations training, which can be booked separately.

      For more information call us at +31854865760.

      IWOCL 2017 slides and proceedings now available

      Reading Time: 1 minute

      A month ago IWOCL (OpenCL workshop) and DHPCC++ (C++ for GPUs) took place. Meanwhile many slides and posters have been published online. As of today 23 talks are with slides.

      The proceedings are available via the ACM Digital Library. This needs a ACM Digital Library subscription of $198, if your company/university does not have access yet.

      IWOCL 2018 will be in Edinburgh (Scotland, UK), 15-17 May 2018 (provisional).

      Bug fixing the MESA 3D drivers

      Reading Time: 3 minutes

      Most of our projects are around performance optimisation, but we’re cleaning up bugs too. This is because you can only speed up software when certain types of bugs are cleared out. A few months ago, we got a different type of request. If we could solve bugs in MESA 3D that appear in games.

      Yes, we wanted to try that and got a list of bugs to solve. And as you can read, we were successful.

      Below you found a detailed description of one of the 5 bugs we solved by digging deep into the different games and the MESA 3D drivers. At the end of the blog post you’ll find the full list with links to issues in MESA’s bugtracker. Continue reading “Bug fixing the MESA 3D drivers”

      Slow Software Hotline

      Reading Time: 1 minute

      In the perfect world all software is fast, giving us time to do actual work. Unfortunately we live in an unperfect world, and we have to spend extra time controlling our anger as the software keeps us waiting.

      Therefore we have opened the Slow Software Hotline – reachable via both phone and email. It has one goal: make you feel happy again.

      Reporting is easy. Just name the commercial software that needs to be sped up and why. We’ll do the rest. If you need help with initial anger management due to the slow (and/or buggy) software, we’re happy to help with breathing practises.

      Phone: +31 854865760

      Email: hotline@streamhpc.com

      We will not sit still until all software is fast. Speeding up all software out there. One at the time.

      Why did AMD open source ROCm’s OpenCL driver-stack?

      Reading Time: 4 minutes

      AMD open sourced the OpenCL driver stack for ROCm in the beginning of May. With this they kept their promise to open source (almost) everything. The hcc compiler was open sourced earlier, just like the kernel-driver and several other parts.

      Why this is a big thing?
      There are indeed several open source OpenCL implementations, but with one big difference: they’re secondary to the official compiler/driver. So implementations like PortableCL and Intel Beignet play catch-up. AMD’s open source implementations are primary.

      It contains:

      • OpenCL 1.2 compatible language runtime and compiler
      • OpenCL 2.0 compatible kernel language support with OpenCL 1.2 compatible runtime
      • Support for offline compilation right now – in-process/in-memory JIT compilation is to be added.

      For testing the implementation, see Khronos OpenCL CTS framework or Phoronix openbenchmarking.org.

      Why is it open sourced?

      There are several reasons. AMD wants to stand out in HPC and therefore listened carefully to their customers, while taking good note on where HPC was going. Where open source used to be something not for businesses, it is now simply required to be commercially successful. Below are the most important answers to this question.

      Give deeper understanding of how functions are implemented

      It is very useful to understand how functions are implemented. For instance the difference between sin() and native_sin() can tell you a lot more on what’s best to use. It does not tell how the functions are implemented on the GPU, but does tell which GPU-functions are called.

      Learning a new platform has never been so easy. Deep understanding is needed if you want to go beyond “it works”.

      Debug software

      When you are working on a large project and have to work with proprietary libraries, this is a typical delay factor. I think every software engineer has this experience that the library does not perform as was documented and work-arounds had to be created. Depending on the project and the library, it could take weeks of delay – only sarcasm can describe these situations, as the legal documents were often a lot better than the software documents. When the library was open source, the debugger could step in and give the “aha” that was needed to progress.

      When working with drivers it’s about the same. GPU drivers and compilers are extremely complex and ofcourse your project hits that bug which nobody encountered before. Now all is open source, you can now step into the driver with the debugger. Moreover, the driver can be compiled with a fix instead of work-around.

      Get bugs solved quicker

      A trace now now include the driver-stack and the line-numbers. Even a suggestion for a fix can be given. This not only improves reproducibility, but reduces the time to get the fix for all steps. When a fix is suggested AMD only needs to test for regression to accept it. This makes the work for tools like CLsmith a lot easier.

      Have “unimportant” specific improvements done

      Say your software is important and in the spotlight, like Blender or the LuxMark benchmark, then you can expect your software gets attention in optimisations. For the rest of us, we have to hope our special code-constructions are alike one that is targeted. This results in many forums-comments and bug-reports being written, for which the compiler team does not have enough time. This is frustrating for both sides.

      Now everybody can have their improvements submitted, giving it does not slow down the focus software ofcourse.

      Get the feature set extended

      Adding SPIR-V is easy now. The SPIRV-frontend needs to be added to ROCm and the right functions need to be added to the OpenCL driver. Unfortunately there is no support for OpenCL 2.x host-code yet – I understood by lack of demand.

      For such extensions the AMD team needs to be consulted first, because this has implications on the test-suite.

      Get support for complete new things

      It takes a single person to make something completely new – this becomes a whole easier now.

      More often there is opportunity in what is not there yet, and research needs to be done to break the chicken-egg. Optimised 128 bit computing? Easy complex numbers in OpenCL? Native support for Halide as an alternative to OpenCL? All high performance code is there for you.

      Initiate alternative implementations (?)

      Not a goal, but forks are coming for sure. For most forks the goals would be like the ones above, to later be merged with the master branch. There are a few forks that go their own direction – for now hard to predict where those will go.

      Improve and increase university collaborations

      If the software was protected, it was only possible under strict contracts to work on AMD’s compiler infrastructure. In the end it was easier to focus on the open source backends of LLVM than to go through the legal path.

      Universities are very important to find unexpected opportunities, integrate the latest research in, bring potential new employees and do research collaborations. Added bonus for the students is that the GPUs might be allowed to used for games too.

      Timour Paltashev (Senior manager, Radeon Technology Group, GPU architecture and global academic connections) can be reached via timour dot paltashev at amd dot com for more info.

      Get better support in more Linux distributions

      It’s easier to include open source drivers in Linux distributions. These OpenCL drivers do need a binary firmware (which were disassembled and seem to do as advertised), but the discussion is if this is part of the hardware or software to mark it as “libre”.

      There are many obstacles to have ROCm complete stack included as the default, but with the current state it makes much more chance.

      Performance

      Phoronix has done some benchmarks on ROCm 1.4 OpenCL in January on several systems and now ROCm 1.5 OpenCL on a Radeon RX 470. Though the 1.5 benchmarks were more limited, the important conclusion is that the young compiler is now mostly on par with the closed source OpenCL implementation combined with the AMDGPU-drivers. Only Luxmark AMDGPU was (much) better. Same comparison for the old proprietary fgrlx drivers, which was fully optimised and the first goal to get even with. You’ll see that there will be another big step forward with ROCm 1.6 OpenCL.

      Get started

      You can find the build instructions here. Let us know in the comments what you’re going to do with it!