Ovidiu Predescu's Weblog

How to build an Arduino project from the command line

For the projection screen controller I'm building I decided to use an Arduino hooked up to a GuruPlug. This way I offload the communication with the outside world to the GuruPlug, which runs a full Linux server, complete with Ethernet and WiFi connectivity. The Arduino is now used to control the motors and relays, and to receive input from an IR sensor, limit switches and infrared proximity sensors. This solution replaces the relatively cumbersome solution involving an Ethernet shield hooked up to the Arduino.. The software on the Arduino becomes a lot simpler too, without the Arduino TinyWebServer running on the Arduino board itself.

However it does have its own complications. Since the Arduino is now hooked up to a remote computer, I need to be able to build the C++ code for the Arduino on my host development machine (a MacOS X), push it to the GuruPlug running Linux and have the GuruPlug program the Arduino with the new binary.

The build process for the Arduino is handled internally by the Arduino IDE. The logic needed to build a typical Arduino project is hidden inside the Java program, which knows about the Arduino built-in libraries, as well as any additional user-defined libraries. I was hoping I could write a simple makefile to do the actual build, but the library dependencies are determined implicitly from the C header files included by your project's .pde file. It could be done using a combination of shell and awk, and a recursive invocation of make, but it's slightly more complicated. A major problem I hit was with file names containing spaces, which abound at least on MacOS X. Make uses a lot of built-in functions that use space as delimiter between the elements of a list, and it's impossible to use any built-in make functions.

Because of the space character problem, I gave up on make. I was looking for a build system that uses a more programatic approach, and so I found SCons. It's a nice build system built entirely in Python. I was about to write a build file for it when I found arscons, a small build file for Arduino-based projects. The funny thing is that the project was checked in Google Code only two weeks ago. Very timely indeed!

In classic open-source fashion - which I haven't done since I joined Google, back in 2003 - I made few improvements to the code and sent them to the author. I can now build my Arduino project on MacOS X, have it pushed to the GuruPlug and from there uploaded to the Arduino board connected to it.

Prerequisites

For this to work there are a few prerequisites for your development system. Make sure you have Python 2.5 or newer installed. Once you have Python installed, install SCons and the Python Serial module. On a Linux system you can install these by doing:

sudo apt-get install python2.5 python-serial scons

If you're on a Mac, you can get these slightly easier by installing MacPorts. If you do this, make sure you're running the python that comes with MacPorts, and not the default one in the MacOS X distribution. If you decide not to use MacPorts, download and install the packages manually, the process is pretty straightforward.

If you want to push the Arduino binary to a remote machine where the Arduino is connected, you also need to install the Python Fabric package on the host computer. On the remote computer (the GuruPlug in my case), you need to install Python Serial and avrdude. Again if your remote computer is a Linux box like the GuruPlug, you should be able to run apt-get to install the packages.

Once you have these packages installed, download the SConstruct file and put it in your Arduino project's directory. If you plan on remotely pushing the binary, download the pulsedtr.py file and place it in the same directory.

Building your Arduino project

After you install everything, you should be able to build your project using this command:

scons

This will only build the project. To build it and upload it to an Arduino connected to the same computer, run this:

scons upload

If you have a single Arduino board connected to your host computer, scons should be able to identify it automatically. If for some reason it doesn't, run scons like this:

scons ARDUINO_PORT=/dev/<arduino-tty-device> upload

If your Arduino board is not a Duemilanove, you might need to specify the board type using ARDUINO_BOARD. To see a list of possible values for this argument, run scons like this:

scons ARDUINO_BOARD=

Beside the parameters defined above, a few others can be defined in the command line:

• ARDUINO_HOME used to specify the location of your Arduino IDE package
• ARDUINO_VER the version of the Arduino IDE
• EXTRA_LIB if you have additional libraries that you'd like to use for the build, specify that directory using this argument.

For additional parameters that you can specify, look in the SConstruct file.

Deploying to an Arduino board connected to a remote computer

The most complex scenario is when you develop on a machine and push the binary to an Arduino connected to a remote computer (in my case that's a GuruPlug). This usually happens after the development phase, when you deploy your Arduino/GuruPlug project. At this point you don't want to take out the Arduino board from inside your project's box. This option allows you to reprogram the Arduino remotely, directly from your development machine.

To do this, you need to make sure you can login to the remote computer (GuruPlug in my case) using ssh without being asked for a password. You can do this by creating a new RSA key and placing the .pub key in the ~/.ssh/authorized_keys on the remote computer. Let's say your new key is in ~/.ssh/guruplug. You should be able to do this from your host computer, and be logged in to the remote machine automatically (obviously you need to replace user and remote-machine accordingly:

ssh -i ~/.ssh/guruplug user@remote-machine

You can now run scons and have it build, push the Arduino binary to the remote system, and upload the binary to the Arduino board connected to the remote system. Here's how:

scons REMOTE_HOST_STRING=user@remote-machine REMOTE_BOARD=atmega328 \
    REMOTE_TTY=/dev/ttyUSB0 SSH_KEY=~/.ssh/guruplug1 remote

Possible issues

I ran into a strange while trying to push the binary to the remote system using fabric. From the stack trace the error was inside paramiko, the library on which fabric is based. This problem was happening even after I created a new RSA key without a password. Each time I would run scons I got this error:

SSHException: invalid key

The error turned out to be inside ~/.ssh/known_hosts which apparently contained an invalid key, or a key paramiko couldn't parse properly. I removed all the suspect keys from it, and the error disappeared.

Sun Fire X2100 server

I bought a Sun Fire X2100 server to act as a back-up for the current dual AMD server I use for hosting this site.

If you decide to buy such a machine, get the basic configuration and buy the rest of the components yourself. Sun charges a lot of money for upgrades. The slim DVD internal drive is $95, but you can get one for as low as $50 (see this drive here).

The default configuration ships with a 512Mb of RAM. To upgrade from 512Mb RAM to 1Gb Sun charges $270. I used the 512Mb memory DIMM which I took out of my Mac mini after I upgraded its internal memory to 1Gb. I bought two additional DIMMs to further upgrade the memory to 2Gb (2 x Kingston 512MB PC3200 400MHz 184-pin ECC Unbuffered CL3 DDR SDRAM DIMM, ~ $78 each). Upgrading to 2Gb cost me $156 instead of the hefty $590 Sun would have charged me.

As for the hard drives, I got two Western Digital 160Gb SATA drives, which cost $95 each. Sun charges $150 for an 80Gb drive and $300 for a 250Gb disk!

As for the operating system, don't bother getting Solaris or one of the Linux systems they offer. I downloaded and installed the AMD 64 versions of both OpenBSD and Debian, and they both work just fine.

To conclude, from the savings described above, you can buy another X2100 machine! I wonder how many people are doing just this.

(Not so) well done Linux-based consumer device - TomTom Go 700

Yesterday I got a TomTom GO 700 portable GPS system. A friend of mine showed me an older version in action a while ago, and I was impressed by how user-friendly the device was. It's small and easy to carry around, has a touch screen to enter all the data, uses an internal 2.5Gb hard disk to store the map information, and the routing information is easy to read and understand. The device can connect to your cell phone via Bluetooth to download weather and traffic information (not for US though). It also has a built-in microphone and speaker so that calls received on your cell phone can be accepted by simply touching a button on the screen. You can also make phone calls through it, but it's a bit more complicated to scroll through the address book; a voice activated method would be much better.

The biggest surprise was when I connected TomTom to my MacOS X laptop through the provided USB cable. The device showed up as a hard-disk, and I could see a bunch of shell scripts very similar to those in Linux. Sure enough, there is a licenses directory containing the GPL and a bunch of other free software licenses. A readme file points to http://www.tomtom.com/page.php?Page=gpl describing all the pieces of free software they're running.

So Linux can be made very user friendly too, very cool!

Update (October 19, 2005): After using the device for a while, there are few usability issues that drive me nuts:

• You cannot lookup a point of interest by name. You first need to identify its category, only then look it up by name. Trying to find Davies Symphony Hall in San Francisco is not easy: it's neither a Cultural Center, nor a Museum, Tourist Attraction, etc. I contacted customer service about this, and the answer they gave was so-so: [looking up a POI by name] would slow down the unit. BUT, a lot of customers have requested this feature so our engineers and Product Management are working on this so it will be available in future application updates..
• I cannot get live traffic information in the San Francisco Bay Area because the device cannot connect to the Internet using my T-Mobile GPRS service. I have a Motorola V330 GSM phone but apparently this phone is not supported by TomTom. I tried with no luck changing the ppp connect script on the device, it simply doesn't work. If anyone knows how to make this work, I would appreciate knowing it.
• Getting on the right entrance on a highway can sometimes be challenging. The only indication you get is to take the exit that comes up in a specified number of feets/meters, but it doesn't tell you the direction on the highway (north, south etc.). It can be very confusing, and several times now I took the wrong entrance. You can avoid this by looking on the provided map, but that's really inconvenient, especially at highway speeds.
• There are cases when two highways merge, and to continue on the one you already are you need to take an exit. An example is driving south on I280. At the merge with hwy 1, I280 continues if you take the first exit on the right. The continuation of what I280 becomes hwy 1 really. The device doesn't tell you this, and so you can easily miss the exit. This seems to be a maps data problem, rather than the device, but nevertheless is frustrating.

Update (December 9, 2005): I found what the problem was. When asked what phone you have, I chose Motorola V557. For the Internet provide I chose T-Mobile (III), but I think any of the T-Mobile options would work. The crucial setting seems to be whether to allow incoming phone calls while accessing the internet. Choosing the default option (Cannot be accepted while accessing the internet) seems to make things work. If I allow incoming calls, TomTom can no longer connect to the phone. Presumably the connect script is different, making the phone's modem not recognize the options.

More updates: I played some more with the options. Instead of choosing the v557 as described above, I chose v635, and then chose the option to allow incoming phone calls to be made. The internet is active and I can still receive and make phone calls! I'll give this feature a try and post updates on how this works.

I forgot to mention, but all these happened after I upgraded to the latest 5.42 version of their software.

Creating JNI libraries on MacOS X

Building JNI libraries is a bit of black magic, especially when you're dealing with different platforms like Linux and MacOS X. I found the following two references very helpful in understanding of how to build and run a JNI library on MacOS X:

• http://www.mindspring.com/~david.dagon/jni/Native.txt
• http://developer.apple.com/qa/java/java28.html

Now that I figured out the MacOS X part, I hope the Linux part to be very similar.

Playing AAC files on slimserver: faad2 problems

The Seagate Barracuda 200Gb hard-disk of my music system decided to die about a week ago. I'm going to send it out to the manufacturer for an exchange, but in the meantime I went out got a Maxtor 250Gb hard-disk instead.

As if this was not painful enough, reinstalling the slimserver software was not as easy as I had expected. After getting Linux and the slimserver software installed, playing AAC files would produce nothing but a disturbing hiss in the speakers. Searching on google for a fix revealed that there might be a problem with different endianess of the bytes being outputted by faad2, the AAC decoding software.

The proposed solution was not pretty: pipe faad2's wav output to lame, which would correctly re-encode the file as mp3. The problem with this solution is that the machine I'm using to serve the music is not beefy, it has only a meager 600MHz Via processor. So I wrote instead a very simple program to swap the order of bytes from stdin and print them out in the correct order at stdout.

Get the program from here, and compile it like this on your machine:

cc -o swab swab.c

Copy the resulting swab binary in /usr/local/bin, or some other location in your path. If you're on linux where mov123 does not exist, carefully comment out all the lines that refer to it in your slimserver's convert.conf file. Then add the following at the end of the file:

mov wav squeezebox *
    [faad] -w -f 2 $FILE$ | swab

The above did the trick for me, and I'm again a happy squeezebox user!

iTunes music server on Linux

I've been very happy with the squeezebox and small computer setup I put together to store and listen to all my music. I ripped all the CDs I had and all the new ones I buy on my Powerbook, and then rsync them across the network to the computer, which stores them on its large 200Gb hard-disk.

The music is served to the squeezebox using SlimServer, using an HTTP-based protocol especially designed for serving music. While this works fine for squeezebox, another Mac machines in my house are not able to see the music stored on this computer. Ideally, this whole collection should be accessible to iTunes, which has a very nice interface.

Last night I did find a way to make that music collection accessible to iTunes. The program is called daapd, and is really amazing. I now have my dedicated music computer serve the same music collection to my squeezebox and to other iTunes applications running on macs around the house!

Setting up daapd on my computer running Knoppix was a bit more complicated than I had expected. However I got it up and running in three hours, after compiling all the necessary packages from source:

• SDL 1.2.8
• FAAC 1.24
• lame 3.96.1
• libid3tag 0.15.1b
• libmad 0.15.1b
• mpeg4ip
• howl 0.9.10
• Apple's mDNSResponder-58.8
• and finally daapd 0.2.4a

Compiling all the packages, except for Apple's mDNSResponder, was fairly standard. All of these free software packages use autoconf, so getting them to compile is a matter of running ./configure, make, and make install. Apple's package had to be compiled using make os=linux, after cd-ing in the mDNSPosix directory. Most of the time I spent was hunting down the packages and figuring the right order in which to compile and install them (the order above is the one you should follow).

The latest version of daapd was supposed to use howl to implement Apple's Rendezvous. I could not get it working, so I had to use Apple's mDNSResponder to do that piece. It's not a big deal, but I just thought I'd mention it here.

Small computers: mini-ITX

After I got the Squeezebox, I realized I need a machine to serve the music. Unfortunately a regular computer makes a lot of noise, and having it running all the time eats a lot of power. So I started looking around for a smaller system.

A colleague from work suggested going with a low-power alternative, running on a VIA Eden processor. The 533MHz and 600MHz of these processors run without the need for a cooling fan. The 800MHz and 1GHz processors, although still low-power do need a cooling fan on them. But the 533MHz processor has enough CPU capabilities for serving files to the squeezebox, so there's no need to go with a more powerful processor.

I ended up getting this system with the M6000 motherboard and processor. I wanted the ability to add external hard-disks over USB 2.0 or Firewire, otherwise I would have gotten the lower-cost EPIA 533MHz.

The Travla C158 case is very small, nowhere close to the ugly tower cases of today's PC computers. The power supply is external and supports up to 60W. The case does have a small fan at the bottom, which makes a lot of noise. Because the motherboard runs pretty cool, I decided to take out the fan. The only other source of heat is the hard-disk, but it's not too bad.

One thing about the case is that is so small, it needs a memory less than 28mm height to fit inside. Since a regular DDR memory is too big, I had to get a low-profile memory sold by CaseOutlet. I also got a 200Gb Seagate hard-disk from Fry's for $70 after a mail-in rebate, which should be able to lots of music.

At first I tried installing Fedora Core 2, but it failed right after starting the installation. I decided to try Knoppix, a distribution that can run entirely from a CDROM. The system booted just fine, and recognized all the devices inside. After I logged in, I installed the system on hard-disk and rebooted from it. The setup was incredibly smooth, with no major problems or incompatibilities. A really good distribution, highly recommended!

The Knoppix distribution is based on Debian, and this is the first time I have to administer a system based on Debian. The package management is quite different from RPM based system, even though it has similar features.

I copied the music from my powerbook, setup slimserver on it and reconfigured the squeezebox to get the music from the new server.

All of the operations took about 5 hours, including setting up the hardware appropriately, downloading and installing Knoppix, copying the music and setting up the slimserver software. I was quite impressed how smooth everything was.

Adding support for WindowMaker in RedHat 9

I don't like the window managers that come with Gnome or KDE. I typically prefer WindowMaker, a much lighter window manager that I'm using for years now.

To add support for it in RedHat 9, create a /etc/X11/gdm/Sessions/WindowMaker file that reads:

#!/bin/sh
exec /etc/X11/xdm/Xsession wmaker

Create the /usr/share/apps/switchdesk/Xclients.wmaker file, if it's not already there. Make sure it points to your wmaker binary location. Since I compiled from source, mine is in /usr/local/bin/wmaker:

#!/bin/sh
exec /usr/local/bin/wmaker

Make sure these files are executable:

$ chmod a+x /usr/share/apps/switchdesk/Xclients.wmaker
$ chmod a+x /etc/X11/gdm/Sessions/WindowMaker

BTW, on a fresh RH9 installation, if you compiled WindowMaker from source, don't forget to put /usr/local/lib in /etc/ld.so.conf, otherwise the shared libraries installed by WindowMaker will not be found. After you've done this, run ldconfig and make sure wmaker --version works fine.

Linux on IBM Thinkpad T41p

I got an IBM Thinkpad T41p from work yesterday, planning to install Linux on it. The Thinkpad has a really great piece of hardware, well designed and with a robust feel. I really like the trackpoint, it's much faster than using the touchpad.

I prefer using Linux when doing real work, since all the tools I have just work there, as opposed to their equivalents on MacOS X or Windows. My XEmacs environment is heavily customized and I've never been able to make it work to my satisfaction anywhere else.

Anyways, after repartitioning the drive and installing Redhat 9 on the new partition, I started to look into how to configure various things: X-Windows, wireless access and printing on the printers connected to my HP JetDirect 500x print server are the most important ones.

A quick search on Google, revealed these instructions on how to setup Debian on the laptop: horrendous! The basic idea is that I need to recompile a custom 2.6 kernel, install a customized version of the X-Windows server, and use a commercial driver for the wireless card! This is a horribly long set of things one needs to follow to get Linux mostly working on this laptop. I'll give it a shot though. Now I'm downloading the latest updates for RH9 from Redhat, about 300Mb of packages!

BTW, Windows XP works amazingly well on the laptop (I know, this is blasphemy ;). The feature I liked most is that it switches automatically from a wireless network to a wired one as soon as you plug in the Ethernet cable; pretty cool!

Update (16:20pm): After about an hour, I got all the updates installed on my machine. I copied the Linux boot sector on my Windows NTFS partition and rebooted the system.

It turns out that updating the X-Windows server was a lot easier that I expected. I got the ati_drv.o and atimisc_drv.o from here and placed them in /usr/X11R6/lib/modules/drivers. I didn't manage to compile the DRI kernel modules as described here, because of a compilation error related to mismatched arguments to a function. I'll investigate this later, perhaps by getting a new version out of CVS.

I rebooted the machine and ran X in the command line, to generate a configuration file for XFree86:

$ X -configure

I verified the generated config file works fine by running:

$ XFree86 -xf86config ~/XF86Config.new

then copied the newly generated file in /etc/X11XF86Config. This configuration file enables X-Windows to use the maximum 1400x1050 resolution with TrueColor enabled (32bits/pixel)

I configured the printers using the excellent PrintManager from Gnome 2. I created the two printers I have connected to my HP JetDirect 500x print server. For the first printer I specified port 9100, and for the second port 9101. CUPS on RH9 comes configured with knowledge about a lot of printers, including my HP OfficeJet D145 and HP DeskJet 970. Printing worked out-of-the-box, I was really impressed!

In fact it worked better than in Windows XP, where I had to dig for the installation CD of the OfficeJet and had to hunt for options to create a printer attached to a print server. On XP I had to create a new TCP/IP port type, and the n create a local printer using that port. Totally confusing, since the printer is really networked.

Sound is working just fine with the builtin sound card. I ran sndconfig to detect and configure the sound card, and install the necessary module information in /lib/modules.conf.

100,000 threads in less than 2 seconds!

Gerhard Froehlich points out to a new major development in the 2.5 series Linux kernels: the Native POSIX Threading Library, sponsored by Redhat, is able to start up and destroy 100,000 threads on a dual Pentium III Xeon 450MHz in 2.3 seconds, with more than 50 threads running at the same time! The previous 2.5.31 kernel Previously the same test took 15 minutes: wow!

Bash customization for Terminal

If you're using bash as your login shell on MacOS X, you can setup Terminal windows to display the username, the machine and the pathname where of the shell is. Just add the following in your ~/.bashrc file:

case $TERM in
    vt100|xterm|cygwin*)
        TITLEBAR='\[\033]0;\u@\h: \w\007\]'
        ;;
    *)
        TITLEBAR=''
        ;;
esac

export PS1="${TITLEBAR}\u@\h:\w> "

Just make sure you disable all the "Title Displays" in the "Window" section of the preferences in Terminal.app.

This works fine also on Linux and Windows under Cygwin.

SSH behind a firewall

I've finally figured out how to cleanly access hosts outside my company's firewall using OpenSSH.

OpenSSH has a way to specify an external program that performs the TCP/IP connection, instead of doing it directly. This allows you to specify an external program that connects to a Web proxy, and tells it to connect to the external machine you want to connect to, using the CONNECT method.

The following connect.c program does exactly this. Just compile the program on your machine, put the resulting binary in a convenient place (like /usr/local/bin), and add the following line at the top of your ~/.ssh/config file:

ProxyCommand /usr/local/bin/connect -H web-proxy:port %h %p

Each time you run ssh host, OpenSSH will spawn off connect and pass to it in the command line the host and port number you want to connect to. connect will take care of connecting using the supplied Web proxy information.

You may want to configure the ProxyCommand line to be on a host basis, instead of being globally applied to all the hosts you connect to, including those inside the firewall. Read the OpenSSH configuration manual to understand how to do this.