HiPEAC is an academic oriented, 3-day, international conference around HPC, compilers and processors. Last year was in Vienna, this year in Amsterdam – where StreamHPC also is based. That was an extra reason to go for silver sponsorship, besides I find this conference very important.

Compilers have the job to do magic. Last year I had nice feedback on my request to give the developers feedback where in the code the compiler struggles – effectively slapping the guy/gal instead of trying to solve it with more magic. Also learned a lot about compilers in general, listened to GPGPU-talks, discussed about HPC, and most of all: met a lot of very interesting people.

Why you should come too? I give you five reasons:

• Learn about compilers and GPU-techniques, in depth.
• Have great discussions about the latest and greatest research, before it’s news.
• Meet great people who create the compilers you use (or the reverse).
• Visit Amsterdam, Netherlands – I can be your guide. Flights are cheap.
• Only spend €400 for the full 3 day programme and a unique dinner with 500 people – compare that to SC14 and GTC!

If you are seeking for a job in HPC, compilers and GPGPU, you should really come over. We’re there, but several other sponsors are also looking for new employees too.

See the tracks at HiPEAC, which has a lot more GPU-oriented talks than last year. I selected a few from the list in bold.

Monday

• Opening address
• William J. Dally, Challenges for Future Computing Systems
• Euro-TM: Final Workshop of the Euro-TM COST Action
• Session 1. Processor Core Design
• CS²: Cryptography and Security in Computing Systems
• IMPACT: Polyhedral Compilation Techniques
• MCS: Integration of mixed-criticality subsystems on multi-core and manycore processors
• EEHCO: Energy Efficiency with Heterogeneous Computing
• INA-OCMC: Interconnection Network Architecture: On-Chip, Multi-Chip
• WAPCO: Approximate Computing
• SoftErr: Mitigation of soft errors: from adding selective redundancy to changing the abstraction stack
• Session 2. Data Parallelism, GPUs
• James Larus, It’s the End of the World as We Know It (And I Feel Fine)
• ENTRE: EXCESS & NANOSTREAMS
• SiPhotonics: Exploiting Silicon Photonics for energy-efficient high-performance computing
• HetComp: Heterogeneous Computing: Models, Methods, Tools, and Applications
• Session 3. Caching
• Session 4. I/O, SSDs, Flash Memory
• Student poster session / Welcome reception

Tuesday

Don’t forget to meet us at the industrial poster-sessions.

• Rudy Lauwereins, New memory technologies and their impact on computer architectures
• Thank you HiPEAC
• Session 5. Emerging Memory Technologies
• EMC²: Mixed Criticality Applications and Implementation Approaches
• ADEPT: Energy Efficiency in High-Performance and Embedded Computing
• MULTIPROG: Programmability Issues for Heterogeneous Multicores
• WRC: Reconfigurable Computing
• TISU: Transfer to Industry and Start-ups
• HiStencils: High-Performance Stencil Computations
• MILS: Architecture and Assurance for Secure Systems
• Programmability: Programming Models for Large Scale Heterogeneous Systems
• Industrial Poster Session
• INNO2015: Innovation actions in Advanced Computing CFP
• Session 6. Energy, Power, Performance
• DCE: Dynamic Compilation Everywhere
• EUROSERVER: Green Computing Node for European Micro-servers
• PolyComp: Polyhedral Compilation without Polyhedra
• HiPPES4CogApp: High-Performance Predictable Embedded Systems for Cognitive Applications
• Industrial Session
• Session 7. Memory Optimization
• Session 8. Speculation and Transactional Execution
• Canal tour / Museum visit / Banquet

Wednesday

• Burton J. Smith, Resource Management in PACORA
• HiPEAC 2016
• Session 9. Resource Management and Interconnects
• PARMA-DITAM: Parallel Programming and Run-Time Management Techniques for Many-core Architectures + Design Tools and Architectures for Multi Core Embedded Computing Platforms
• ADAPT: Adaptive Self-tuning Computing System
• PEGPUM: Power-Efficient GPU and Many-core Computing
• HiRES: High-performance and Real-time Embedded Systems
• RAPIDO: Rapid Simulation and Performance Evaluation: Methods and Tools
• MemTDAC: Memristor Technology, Design, Automation and Computing
• DataFlow, Computing in Space: DataFlow SuperComputing
• IDEA: Investigating Data Flow modeling for Embedded computing Architectures
• TACLe: Timing Analysis on Code-Level
• EU Projects Poster Session
• Session 10. Compilers
• HIP3ES: High Performance Energy Efficient Embedded Systems
• HPES: High Performance Embedded Systems
• Session 11. Concurrency
• Session 12. Methods (Simulation and Modeling)

Hopefully till then!

With SC14 behind us, there are a few things I’d like to share with you. I’d like to start with the biggest win for OpenCL: AMD leading in the most power-efficient GPU-cluster.

A few months ago I wrote a theoretical article on how to build the cheapest and greenest supercomputer to enter the Top500 and Green500. There I showed that AMD would theoretically win on both GFLOPS/costs and GFLOPS/Watt. Last week I learned a large cluster is actually being built in Germany, which now leads the Green500 (GFLOPS/Watt). It is powered by Intel Ivy Bridge CPUs, an FDR Infiniband network and accelerated by air-cooled(!) AMD FirePro S9150 GPUs, as can be seen on the Green 500 report of November. The score: 5.27 GFLOPS per Watt, mostly because of AMD’s surprise act: extremely efficient SGEMM and DGEMM.

The first NVIDIA Tesla-based system on the list is at #3 with 4.45 GFLOPS per Watt for a liquid cooled system. If the AMD FirePro S9150 would be oil or water cooled, the system could go to over 6 GFLOPS per Watt. I’m expecting such system on the Green500 of June. The PEZY-SC (#2 on the list) is a very interesting, unexpected newcomer to the field – I’ll share more with you later, as I heard it supports OpenCL.

The price metric

The cluster at GSI Helmholtz Center has around 1.65 double precision PetaFlops (theoretical). Let’s do the same calculation as with the 150 GFLOPS system using the latest prices, only taking the accelerator part.

640 x AMD FirePro S9150.

• 2.53 GFLOPS * 640 = 1.62 TFLOPS (I rounded down to 2.0 GFLOPS in the other article)
• US$ 3300. Total price: $2.112M. Price per TFLOPS: $1.304M
  
• 235 Watt * 640 = 150 kWatt (excluding network, CPU, etc)

640 x NVIDIA Tesla K40x

• 1.42 GFLOPS * 640 = 0.91 TFLOPS
• US$ 3160 (got down a lot due to introduction K80!). Total price: $2.022M. Price per TFLOPS: $2.225M
  
• 235 Watt * 640 = 150 kWatt

640 x Intel XeonPhi 7120P

• 1.21 GFLOPS * 640 = 0.65 TFLOPS
• US$ 3450. Total price: 2.208$M. Price per TFLOPS: $3.397M
• 300 Watt * 640 = 192 kWatt

So it’s pretty clear, why GSI chose AMD: $92M or $209M less costs for the same GFLOPS. Also note that more GFLOPS per accelerator is important to lower overhead.

What to expect from June’s Green500

Next year Nvidia probably comes with Maxwell, which probably will do very well in the Green500. Intel has their new XeonPhi, but it’s a very new architecture and no samples have arrived yet – I would be surprised, as they over-promised for too long now. Besides bringing surprises, Intel’s other strengths are its vast collaborations and strong fanbase – the past years I heard the most ridiculous responses on why such underperforming accelerator was chosen instead of FirePro or Tesla, so it’s certainly aiming for a rampage (based on hope). AMD did not enclose any information on a new version of the S9150 (Something like S9200 or S9250).

Then there are the dual GPUs, which have no advantages but lower energy-usage. The K80 just arrived, but the number don’t add up yet – we’ll have to see when the samples arrive. AMD did not say anything about the next version of the S10000, but probably arrives next year – no ETA. Intel did not do dual-chip cards until now. These systems can be built more compact, as 4 GPUs per system is becoming a standard.

Another important change will be the CPUs with embedded CPU being used in the clusters, where now mostly Intel Xeons rule the world. Intel’s Iris Pro line and AMD new Carrizo APU could certainly get more popular, as more complex code can be accelerated very well by such processors. Also 64-bit ARM-processors we’ll see more – hopefully with GPU. This subject I’ll handle in a separate article, as OpenCL could be a big enabler for easy offloading.

Based on the current information I have available, Nvidia aims for Maxwell based Teslas, AMD with S9150 and the dual-GPU variant, Intel with none (aiming for November 2015). It’ll be exciting to see HPC get to 6+ GFLOPS/Watt as a standard – I find that more important than building the biggest cluster.

OpenCL will help select hardware from that year’s winner, not being locked in to that year’s loser. Meanwhile at StreamHPC we will keep building OpenCL-based software, to help our customers pick that winner.

Toolmaker VectorFabrics sent 9 predictions for this year in their newsletter. I’d like to share it with you.

Nine predictions for 2014 that prove the programming landscape is changing

It is not hard to predict that this year will see a lot of activity around multicores and manycores. 2014 will be the year that software has to catch up with highly concurrent hardware. So we expect to see some major changes in how people view multicore programming:

1. Neither Intel, AMD nor Qualcomm releases any new single core processors in 2014. Therefore, it is less and less acceptable to release pure sequentially operating applications.
2. Intel releases a 15-core Xeon. On a typical 4-socket motherboard your OS sees 120 available cores. OpenMP is the preferred programming paradigm on such a platform for data-intensive shared-memory calculations. You have to deal with performance bottlenecks, including Amdahl’s law, cache performances and memory bandwidth issues.
3. Next to ARM big.LITTLE systems, 2014 sees the first true octacore cell phones and tablets. At that point, it becomes painfully clear that applications need changes to benefit from so many cores. Both true octacore and big.LITTLE processors see very little adoption in mobile devices as long as software that can benefit is missing.
4. At least one major mobile phone vendor loses market share because their hardware may be good, but the software (especially the web browser) cannot utilize the hardware to the max.
5. Both the XBox One and PlayStation 4 feature AMD Jaguar octacore processors with GP-GPUs. Very few games will using all the compute power, and customers will wonder why to upgrade as the performance difference to their existing console is not so different.
6. Two great open standards, OpenACC and OpenMP see a nice boost in adoption thanks to upcoming support in the latest open source compilers. Clang 3.5  features OpenMP 4.0 support. In addition GCC 4.9 also receives OpenACC support.
7. In the mobile space, offloading to GP-GPUs is hot as new architectures from Qualcomm Adreno 420, Nvidia Tegra K1 and Imagination PowerVR Series 6 will each allow offloading. Managing and programming the offloading will remain a problem: OpenCL? OpenACC? CUDA? RenderScript?
8. Major desktop applications jump on the compute-offloading bandwagon to win performance, using either OpenCL or OpenACC
9. Programmability of the next-gen Intel Xeon Phi (Knights Landing) will raise some eyebrows. This 2015 chip will have 72 Atom Silvermont cores with local memory, cache and up to 384 GB shared memory. This is outside the comfort zone of most programmers.

You guessed right: their tool has to do with parallel programming.

What are your predictions for 2014?

Every now and then I read stories on Bitcoins (Wikipedia-article), as GPUs are used a lot to “mine” Bitcoins. They have some extensive benchmarks, and also their discussions giving me insights in specific parts of accelerators like GPUs. Also is this group very upwards if it comes to accepting new techniques. Today something changed: they are a bank now. One of the thoughts I had with this, I’d like to share with you.

If you look at various types of currencies, you see they all have various goals (trade, power, resources, energy, properties, etc). The inequality and differences are even more important than the amount. Various currencies are entangled to a certain goal or resource, but there is nothing entangled strongly to technology. Here is where Bitcoins come in…

Bitcoins are entangled with compute-power – a current benchmark for technological progress.

In this article I’d like to share how the tech-economy and Bitcoins are entangled, seen from the perspective of computing. I left out a lot of the “rules of economy” and hope you can put these in – the below text is just to guide you through the thought-process only. Disagreement is only good – as we learn all from it.

Continue reading “The entanglement of Bitcoins and compute-capabilities” →