I've had some interesting differences in my experience using ROS on the desktop and on embedded systems. Typically these differences stem from the difference in computing resources (disk, speed, memory, etc.). I have decided to document my experiences here.
I had been recently using an Odroid XU3 which includes 4 Arm-Cortex processor cores (armhf – arm7). When compiling ROS packages on the Odroid I started getting “g++ compiler internal error….” messages. These messages are uninformative and they indicate no recourse other than “file a bug report.” Under further investigation I found that, by default, “catkin_make” command uses all cores to compile code in parallel. Actually “catkin_make” was spawning 1.5*4cores = 6 compile processes. This was this pounding on the processor and other compute resources for this embedded system which was causing the errors. I believe out-of-memory or related errors were occurring but it's impossible to know.
I changed my invokation of “catkin_make” to be “catkin_make -j 2” and similarly any “make” call I changed to “make -j 2”. The “-j 2” argument restricts compilation to two threads. I initially tried “-j 1” to restrict compilation to a single thread and this worked to some extent but I could not get ROS to generate needed message header files no matter how I indicated this dependency. By creating 2 threads, even though one thread dies (the one compiling my package without the needed include file for my message) the second thread actually generates the include file successfully. Hence, I was able to circumvent this problem by having 2 parallel compiling threads.
After coping with these issues, my compilation problems went away.