Sunday, September 30, 2012

Compiling for CARMA

In few days, CARMA will be finally available to the general public. If you are not familiar with the CARMA project, it is the first ARM platform supporting CUDA.
It has a Tegra 3 with 4 cores and 2 GB of memory, ethernet, USB ports and a Quadro 1000M GPU (GF108 with 2 GB of memory, 96 CUDA cores, compute capability 2.1).
It has full OpenGL and CUDA support, but at the moment, no CUDA compiler.

The developer needs to cross-compile from a Linux x86 machine. This blog shows how easy it is to cross-compile once we follow some simple instructions. I strongly suggest that you start with an Ubuntu machine, the cross-compiler are easily available under this platform.

The first thing to do, it is to install the cross-compilers:

sudo apt-get install g++-arm-linux-gnueabi gcc-arm-linux-gnueabi

At this point, we will have the cross-compilers installed under  /usr/bin/arm-linux-gnueabi-gcc and  /usr/bin/arm-linux-gnueabi-g++.

The second step is to install the CUDA Toolkit for ARM on the x86. If you choose the default location,
the installer will create a directory /usr/local/cuda.

If you need to use other libraries for ARM, you will also need to copy the libraries and corresponding header files from CARMA to the x86 machine.  You can place them under /usr/local/arm_lib and /usr/local/arm_include or you can just put them under /usr/local/cuda/lib and /usr/local/cuda/include (my preference will be for the first option to not pollute the CUDA installation).

We are now ready to compile our code, taking care of using the cross compiler and the special nvcc in the CARMA toolkit.  The following makefile will show how to compile a simple c++ code that calls a CUBLAS function and a simple CUDA code.

#  Makefile for cross-compile #
all : dgemm_cublas simple_cuda

NVCC=$(CUDA_HOME)/bin/nvcc -target-cpu-arch ARM --compiler-bindir /usr/bin/arm-linux-gnueabi-gcc-4.5 -m32

# For a standard c++ code, we use CC and the CUDA ARM libraries
dgemm_cublas : gemm_test.cpp
$(CC)   gemm_test.cpp -I$(CUDA_HOME)/include -o dgemm_cublas -L/$(CUDA_HOME)/lib -lcudart -lcublas

# For a standard CUDA code, we just invoke nvcc
$(NVCC) -o simple_cuda

clean :
rm -f *.o dgemm_cublas simple_cuda

Once we generate the executable, since they are for ARM, we will not be able to execute them until we move them on CARMA.


  1. Could you please specify where to get the CUDA Toolkit for ARM?
    standard nvcc doesn't support the -target-cpu-arch ARM option

    many thanks

  2. It will be available from SECO. Let me see if I could find a link.

  3. Just got a reply, SECO will make the toolkit available at

  4. Thanks Massimiliano - I've already got the toolkit and installed it on an x86 machine installed with ubuntu 11.04
    Having already 2 CARMA developing kits I'm trying to test the whole process: installation, cross-compilation of some code and execution on the carma kit
    Currently I'm getting problems in the cross compilation phase related to the linker which cannot find the cuda library that contains the very basic cuda API (like cuInit)- traditionally this is the library installed under /usr/lib while after installing the cuda toolkit for ARM this library is missing!!
    I tried to link against the libcudart as in your post but as expected I'm getting many undefinded references!! (not finding the very basic API like cuInit) since they're not in libcudart
    nm -D /usr/local/cuda/lib/ |grep cuInit find nothing

    Am I missing something? or would it be possibile the cuda toolkit for ARM has something missing

    many thanks

  5. You need to copy the from the CARMA to the x86.
    There should be a script that does that ( I have not seen the final toolkit package, it was present in the prerelease I saw)

  6. More info:
    In the home directory on the CARMA board (/home/ubuntu/), there’s a shell script Copy this script to the toolkit directory and call it with the file system root location as the only argument. This script can be executed from the CARMA board, as well as the development machine, depending on how the two machines have access to each other.

  7. That was very useful
    problem is resolved

    many thanks again

  8. SECO is now hosting forums for CARMA.
    If you have more questions, problems, please post over there ( )

  9. Hi Massimiliano,
    I've just started with CARMA board. I was successfull in compilling the opencv libraries for ARM, but now I'm stuck about how to copy them on the carma board. I don't have the script ho /home/ubuntu directory. Can you send me that by email? alessandro[dot]nuke[at] gmail [dot]com.

  10. I have tried to run opencv with gpu in Carma. However, whenever i call any gpu functions, segmentation errors occurred. Do you have any ideas about this issue. Thanks