August 2012: article has been completely rewritten and updated. For driver-specific issues, please refer to this article.

Want to have both your GTX and Radeon working as OpenCL-devices under Linux? The bad news is that attempts to get Radeon as a compute device and the GTX as primary all failed. The good news is that the other way around works pretty easy (with some luck). You need to install both drivers and watch out that libglx.so isn’t overwritten by NVidia’s driver as we won’t use that GPU for graphics – this is also the reason why it is impossible to use the second GPU for OpenGL.

Be aware: these steps can possibly not work for you! Be patient and take your time for these steps, as they will probably not work for you without some magic. Also problems could occur each time there is a kernel-update.

Step 1: Radeon in slot 1

Though AMD’s drivers support headless compute now, I haven’t succeeded to have the Nvidia GTX as my primary GPU. So the first step is to put the AMD in slot 1. As they don’t sell computers with this configuration, I assume you know how to switch GPUs.

Step 2: Download the drivers

First you need to check which version is available in the repository for “nvidia-current”. If you install the ubuntu-x-swat PPA, you might get more recent versions. You need to download the manual installer from the Nvidia driver-site with exact that version.

From AMD you can just choose the latest version.

Read “Failed” below, so you can think of other preparations.

Step 3: Uninstall all graphics drivers

If you have installed the drivers manually, use the –uninstall switches to uninstall them

sudo ./amd-driver-installer-12-6-x86.x86_64.run --uninstall=force

sudo ./NVIDIA-Linux-x86_64-302.17.run --uninstall

Uninstall ‘fglrx’ and ‘nvidia-current’ using your favourite package manager

Step 4: Install nvidia-current

Using your favourite package manager, install ‘nvidia-current’. For me this was the only easy fix to avoid an error “nvidia-current.ko not found” when modprobing nvidia. I haven’t found a better way, but this step will break each time the driver is updated – so make sure you enforce the version in the package manager.

I found out that it is possible to skip this step and install it after step 7 when X is working. Then all works

Step 5: Go out of X

Stopping X equals stopping KDM, GDM or LightDM. First you need to type ‘ctrl-alt-F1’ to get to a screen that doesn’t rely on X. Then you can stop the GUI. For example with Ubuntu 12.04 you use:

sudo service stop lightdm

Replace with gdm or kdm to stop the X you are using. Alternatively you can use:

sudo /etc/init.d/gdm stop

Step 6: Manually install Nvidia-drivers

You have downloaded the drivers already, so go to the download-location and run it. For example:

sudo ./NVIDIA-Linux-x86_64-302.17.run

Don’t let the installer change xorg.conf or install anything OpenGL-like (if asked) – so go for the minimum. Say yes to register with dkms.

If you get a lot of errors, don;t panic and run the installer again.

Step 7: Make sure NVIDIA has no OpenGL-drivers taken over

NVIDIA installs an extra module which overrules the normal xorg-extension. I noticed this since 304. This module needs to be removed:

rm /usr/lib/x86_64-linux-gnu/xorg/extra-modules/libglx.so*

This should fix the problem the computer hangs during boot. check /var/log/Xorg.0.log if no reference to nvidia is made. In case it’s not there, recursively search the file in its subdirecties!

The old trick (pre 304.x) is getting the default GLX-drivers back:

apt-get install --reinstall libgl1-mesa-glx libgl1-mesa-dri \
libxcb-glx0 xserver-xorg-core

Step 8: Manually install AMD-drivers

Just as you did with with the Nvidia-drivers, install the Radeon-drivers. It needs to be done after the Nvidia-drivers are installed, because the OpenGL-drivers need to be overwritten. As the installer sees the OpenGL-drivers, it complaints about a previous installation, so you need to force it to continue. For example:

sudo ./amd-driver-installer-12-6-x86.x86_64.run –force

Just press enter all the time: defaults are ok.

Just as with Nvidia’s installer, running the AMD installer twice has some advantages for some unknown reason.

Now you can try – you should get a graphical interface running using your Radeon.

Failed to get a desktop running?

Each time you fail, just start from step 3. When starting in safe mode, be sure you don’t mount drives read-only. Here is the reason why you needed step 2 before everything else: without GUI one can feel quite helpless.

• If you get a black screen and no X, check if the GLX-drivers are loaded for Nvidia instead of Radeon. Here is where the magic comes: you know why there is an order, so experiment. Check xorg-logs in /var/log to find out.
• If you get ‘FATAL: Error inserting nvidia_current (/lib/modules/2.6.38-13-generic/updates/dkms/nvidia-current.ko): No such device‘ then the nouveau-drivers might be loaded – do a ‘tail /var/log/kern.log‘ to find out and follow-up with removing the conflicting drivers, blacklist it and reboot.
• There are many drivers not updated for CUDA 4.0 and CUDA 4.1, so you might end up with ‘Runtime API error : CUDA driver version is insufficient for CUDA runtime version‘ even with very recent versions. You can check /usr/include/nvidia-current/cuda/cuda.h for CUDA_VERSION.
• Get errors around libOpenCL.1.0 by Nvidia? You can just delete the file and then run the installer again.

Step 9: Enabling compute on Nvidia

When rebooting, the fglrx-driver gets loaded, but the nvidia-driver doesn’t. Below is a script that needs to be run (as root) to get nvidia loaded (check if the directory of lspci is correct when not on Ubuntu). First try if ‘modprobe nvidia‘ actually works for you, and then run clinfo from AMD’s SDK and see if NVidia pops up. If ‘modprobe nvidia‘ doesn’t work, the installation of the driver was not successful.

#!/bin/bash

if [[ $EUID -ne 0 ]]; then
echo "This script must be run as root" 
exit 1
fi

modprobe nvidia

if [ "$?" -eq 0 ]; then

  # Count the number of NVIDIA controllers found.
  N3D=`/usr/bin/lspci | grep -i NVIDIA | grep "3D controller" | wc -l`
  NVGA=`/usr/bin/lspci | grep -i NVIDIA | grep "VGA compatible controller" | wc -l`

  N=`expr $N3D + $NVGA - 1`
  for i in `seq 0 $N`; do
    mknod -m 666 /dev/nvidia$i c 195 $i;
  done

  mknod -m 666 /dev/nvidiactl c 195 255
else
  exit 1
fi

If you want this under init.d, then check out this post (check if the directory of lspci is correct, when not on Red Hat). Above script was taken from this blog post by ‘mfatica’ of NVidia. Above found working with NVidia drivers 280 till 304 and AMD drivers 8.892 till 9.002.

Make sure you have read https://streamhpc.com/blog/2011-06-24/install-opencl-on-debianubuntu-orderly/ too, for installing OpenCL.

Found the magical solution?

I have rewritten (parts of) this post several times. Each time things worked just a little different with new drivers, or I found out new solutions. All was tested with AMD APP 2.6, and CUDA 5.0.

Let me know if it worked for you, or if you needed to do something extra/different.