May 06, 2013
BeagleBone Black vs Arduino Uno |
Arduino | BeagleBone | Cool gadgets |
|
I just received my BeagleBone Black from Adafruit. It's a very sweet board and the price is just right. US $45 will get you a 1GHz ARM processor with 512MB RAM and 2GB flash, built-in Ethernet (10/100Mbps), video graphics card with HDMI output port, 65 I/O ports, 4 hardware UARTs. A very capable board for driving the various small projects I have in mind. The first project I will work on is to replace a Guruplug+Arduino-based project that was driving my projection screen. The setup I had was quite nice, but it appears the Guruplug died after running great for the past few years. 
| |
December 08, 2010
Arduino Tiny Web Server - part 3 |
Arduino |
|
Update (September 3rd, 2011): The code is on Github: https://github.com/ovidiucp/TinyWebServer A zip file of the latest version of Arduino TinyWebServer can be found here https://github.com/ovidiucp/TinyWebServer/zipball/master. What is it?Arduino TinyWebServer is a library implementation of a small web server running on an Arduino Duemilanove or Uno using the new Ethernet Shield released at the end of September 2010. The new Ethernet Shield adds a microSD card right on the board, in addition to fixing a number of hardware bugs from the previous iteration. The library is meant to provide a Web interface to your Arduino project. It is not meant to be a full Web server implementation, just the minimum amount to let you get started quickly. To save precious memory space on your Arduino, TinyWebServer encourages you to develop your web applications as AJAX applications. This means that you should delegate the page generation and complex UI interactions to the Web browser, instead of trying to handle it on the Arduino side of your project. The only thing the Arduino should implement is retrieving and displaying the internal state, as well as modifying it using HTTP POST requests. 
Changes from the previous releaseAs you're probably aware, a new Ethernet Shield for Arduino was released at the end September 2010. I bought one and I'm pleased to report that it fixes an annoying hardware bug present in the old shield. In addition it has a new microSD slot that is actually accessible through a dedicated SPI SS pin. This means that you can use an Arduino Duemilanove or Uno with the Ethernet Shield with no modifications whatsoever. I finally had some time to update the Arduino TinyWebServer library to make it work with the new shield. As you can see in the picture above, the setup is much easier than with the old Ethernet Shield and the data logging shield. The new library version drops support for this kludgy setup.
In the new release I've added a new example that shows how you can control an LED from a regular Web browser. The video below demonstrates how it works. DocumentationArduino TinyWebServer is a library and as such requires you to write code customized for your own application. Some almost ready-to-go examples can be found inside the distribution in the An explanation of the library's API can be found in this older post. Some things like the EthernetDHCP code no longer apply, so please look at the new examples for the updated code. Still most of the code is still applicable. | |
September 30, 2010
How to build an Arduino project from the command line |
Arduino | Linux | |
For the projection screen controller I'm building I decided to use an Arduino hooked up to a GuruPlug. This way I offload the communication with the outside world to the GuruPlug, which runs a full Linux server, complete with Ethernet and WiFi connectivity. The Arduino is now used to control the motors and relays, and to receive input from an IR sensor, limit switches and infrared proximity sensors. This solution replaces the relatively cumbersome solution involving an Ethernet shield hooked up to the Arduino.. The software on the Arduino becomes a lot simpler too, without the Arduino TinyWebServer running on the Arduino board itself. However it does have its own complications. Since the Arduino is now hooked up to a remote computer, I need to be able to build the C++ code for the Arduino on my host development machine (a MacOS X), push it to the GuruPlug running Linux and have the GuruPlug program the Arduino with the new binary. The build process for the Arduino is handled internally by the Arduino IDE. The logic needed to build a typical Arduino project is hidden inside the Java program, which knows about the Arduino built-in libraries, as well as any additional user-defined libraries. I was hoping I could write a simple makefile to do the actual build, but the library dependencies are determined implicitly from the C header files included by your project's Because of the space character problem, I gave up on make. I was looking for a build system that uses a more programatic approach, and so I found SCons. It's a nice build system built entirely in Python. I was about to write a build file for it when I found arscons, a small build file for Arduino-based projects. The funny thing is that the project was checked in Google Code only two weeks ago. Very timely indeed! In classic open-source fashion - which I haven't done since I joined Google, back in 2003 - I made few improvements to the code and sent them to the author. I can now build my Arduino project on MacOS X, have it pushed to the GuruPlug and from there uploaded to the Arduino board connected to it. PrerequisitesFor this to work there are a few prerequisites for your development system. Make sure you have Python 2.5 or newer installed. Once you have Python installed, install SCons and the Python Serial module. On a Linux system you can install these by doing: sudo apt-get install python2.5 python-serial scons If you're on a Mac, you can get these slightly easier by installing MacPorts. If you do this, make sure you're running the python that comes with MacPorts, and not the default one in the MacOS X distribution. If you decide not to use MacPorts, download and install the packages manually, the process is pretty straightforward. If you want to push the Arduino binary to a remote machine where the Arduino is connected, you also need to install the Python Fabric package on the host computer. On the remote computer (the GuruPlug in my case), you need to install Python Serial and avrdude. Again if your remote computer is a Linux box like the GuruPlug, you should be able to run Once you have these packages installed, download the SConstruct file and put it in your Arduino project's directory. If you plan on remotely pushing the binary, download the pulsedtr.py file and place it in the same directory. Building your Arduino projectAfter you install everything, you should be able to build your project using this command: scons This will only build the project. To build it and upload it to an Arduino connected to the same computer, run this: scons upload If you have a single Arduino board connected to your host computer, scons ARDUINO_PORT=/dev/<arduino-tty-device> upload If your Arduino board is not a Duemilanove, you might need to specify the board type using scons ARDUINO_BOARD= Beside the parameters defined above, a few others can be defined in the command line:
For additional parameters that you can specify, look in the SConstruct file. Deploying to an Arduino board connected to a remote computerThe most complex scenario is when you develop on a machine and push the binary to an Arduino connected to a remote computer (in my case that's a GuruPlug). This usually happens after the development phase, when you deploy your Arduino/GuruPlug project. At this point you don't want to take out the Arduino board from inside your project's box. This option allows you to reprogram the Arduino remotely, directly from your development machine. To do this, you need to make sure you can login to the remote computer (GuruPlug in my case) using ssh without being asked for a password. You can do this by creating a new RSA key and placing the .pub key in the ssh -i ~/.ssh/guruplug user@remote-machine You can now run scons and have it build, push the Arduino binary to the remote system, and upload the binary to the Arduino board connected to the remote system. Here's how:
scons REMOTE_HOST_STRING=user@remote-machine REMOTE_BOARD=atmega328 \
REMOTE_TTY=/dev/ttyUSB0 SSH_KEY=~/.ssh/guruplug1 remote
Possible issuesI ran into a strange while trying to push the binary to the remote system using SSHException: invalid key The error turned out to be inside | ||
September 22, 2010
Arduino and GuruPlug |
Arduino | Hardware | |
Few months ago I attempted to make an Arduino-based project accessible via a web browser and/or a phone using an Ethernet shield (see here and here). The results were a bit disappointing, as the Ethernet shield tends to behave strange when the files to be transferred are large. Everything seems to be fine if the file to be transferred is just below the Ethernet MTU (1492 bytes by default), with extremely fast transfers all the time. When the file exceeds that threshold, there are a bunch of inexplicable timeouts which make the file transfer to be seconds instead of milliseconds. I then discovered the GuruPlug. For $100 you get a full computer running on a Marvell 1.2GHz processor, with 512MB RAM and 512MB flash as the hard drive, 1 Gigabit Ethernet, 802.11 b/g and 2 USB 2.0 ports! The best part is that the GuruPlug consumes less than 5W, so you can have it on at all times. The company making the device is positioning the GuruPlug as a general purpose computer, but I think that's a mistake. I'm not going to trust such a puny computer to run my file server. However it's powerful enough to run a bunch of dedicated little applications. I've been running my home's internal DNS server for more than a year now on the previous generation SheevaPlug. Since the GuruPlug has 2 USB ports and has both Ethernet and WiFi, you can easily connect it to an Arduino and have the web server run on GuruPlug, while having the actual interface with motors and sensors done by Arduino. The communication between the two can easily happen over USB using a custom made protocol. More on the software side of things in a different post Since I want my project to go inside a single box, and not have lots of wires running around, I disassembled the GuruPlug and removed the main board. Doing so voids the warranty, so do this at your own risk! You need to provide 5V at 1A to the board. I removed the USB connectors on both the GuruPlug and the Arduino board, and ran some wires directly between them. This way I don't need to run a dedicated USB cable between the two boards. On the downside, I lost a USB port which could have been exposed outside of the box. Right now I'm in the process of building a small box for the project. Since the GuguPlug doesn't have an external WiFi antenna, I needed to build the box from plastic instead of metal. Another thing to keep in mind is that the metal part that's on top of the GuruPlug board is a heat sink that runs very hot. I'll most likely put a fan right on top of it, even though the original GuruPlug doesn't have any. More on the building of the actual box in a later post. | ||
June 16, 2010
Arduino Tiny Web Server - part 2 |
Arduino | Hardware | Open Source |
|
Update (December 30, 2010): Latest version of Arduino TinyWebServer: arduino-tinywebserver-20101230.zip. Update (December 8, 2010): The below picture of the required Arduino hardware is obsolete. Look at this newer post for updated information on the new hardware. In part 1 of the Arduino Tiny Web Server I presented some hardware modifications and changes to the Arduino Ethernet shield and the Adafruit Data Logging shield.
In this part I present the Arduino TinyWebServer library or TWS. TinyWebServer allows you to provide a Web interface to your Arduino-based project. You can get very creative with this, and add a full Ajax web interface to your project. This is possible because it's the Web browser doing all the UI work, while your Arduino board interacts with the hardware connected to it. I'm using TWS in a remotely controlled projection screen that I'm currently building to replace an existing system. The end goal is to be able to control the projection screen from an Android phone, and let my kids choose to watch movies either on TV or on the big screen. More on this is a later post, until then read below to see this works. The library has been developed on MacOS X and should most likely work fine on Linux. No guarantees about Windows, but I'd love to hear if it works for you. 
| |
As I mentioned in part 1, there are several hardware modifications, as well as software modifications that need to be made. Make sure you have those modifications done to your hardware before proceeding further.
To make things easy, I've decided to bundle the TWS library with the modifications to those libraries, as well as with two additional libraries that TWS depends on: Georg Kaindl's EthernetDHCP and Mikal Hart's Flash library.
After you download and unzip the package, copy the contents of the directory in the directory where you store your Arduino libraries.
The library comes with few examples, look in TinyWebServer/examples. The simplest one is SimpleWebServer, which shows how to write a basic HTTP server with a GET handlers. The more complex one, FileUpload shows how to implement a PUT handler to implement file uploads and write them on the SD card, and how to serve the files in GET requests.
Basic web server
To make use of the TWS library, you need to include the following your sketch:
#include <Ethernet.h> #include <EthernetDHCP.h> #include <Flash.h> #include <Fat16.h> #include <Fat16util.h> #include <TinyWebServer.h>
EthernetDHCP is optional, but it makes acquiring an IP address a lot easier if you have a DHCP server in your network.
TWS is implemented by the TinyWebServer class. The constructor method takes two arguments. The first one is a list of handlers, functions to be invoked when a particular URL is requested by an HTTP client. The second one is a list of HTTP header names that are needed by the implementation of your handlers. More on these later.
An HTTP handler is a simple function that takes as argument a reference to the TinyWebServer object. When you create the TinyWebServer class, you need to pass in the handlers for the various URLs. Here is a simple example of a web server with a single handler.
static uint8_t mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
boolean index_handler(TinyWebServer& web_server) {
web_server.send_error_code(200);
web_server << F("<html><body><h1>Hello World!</h1></body></html>\n");
return true;
}
TinyWebServer::PathHandler handlers[] = {
// Register the index_handler for GET requests on /
{"/", TinyWebServer::GET, &index_handler },
{NULL}, // The array has to be NULL terminated this way
};
// Create an instance of the web server. No HTTP headers are requested
// by the HTTP request handlers.
TinyWebServer web = TinyWebServer(handlers, NULL);
void setup() {
Serial.begin(115200);
EthernetDHCP.begin(mac);
web.begin();
}
void loop() {
EthernetDHCP.maintain();
web.process();
}
In the loop() function we need the call to the process() to make sure HTTP requests are serviced. If there is no new request, the method returns immediately. Otherwise the process() method blocks until the request is handled.
For a complete working example look in TinyWebServer/example/SimpleWebServer.
Serving files from the SD card
Now that we've seen the basics, let's see how we can extend this web server to serve files stored on the SD card. The idea is to register a handler that serves any URLs. Once the handler is invoked, it interprets the URL path as a file name on the SD card and returns that.
boolean file_handler(TinyWebServer& web_server) {
char* filename = TinyWebServer::get_file_from_path(web_server.get_path());
if (!filename) {
web_server.send_error_code(404);
web_server << "Could not parse URL";
} else {
TinyWebServer::MimeType mime_type
= TinyWebServer::get_mime_type_from_filename(filename);
web_server.send_error_code(mime_type, 200);
if (file.open(filename, O_READ)) {
web_server.send_file(file);
file.close();
} else {
web_server << "Could not find file: " << filename << "\n";
}
free(filename);
}
return true;
}
We can now register this in the handlers array:
TinyWebServer::PathHandler handlers[] = {
{"/" "*", TinyWebServer::GET, &file_handler },
{NULL},
};
Note how the URL for the HTTP request is specified. We want it to be /*, very much like a regular expression. However Arduino's IDE preprocessor has a bug in how it handles /* inside strings. By specifying the string as "/" "*" we avoid the bug, while letting the compiler optimize and concatenate the two strings into a single one.
The * works only at the end of a URL, anywhere else it would be interpreted as part of the URL. If the * is at the end of the URL, the code in TinyWebServer assumes the handler can process requests that match the URL prefix. For example, if the URL string was /html/* then any URL starting with /html/ would be handled by the specified handler. In our case, since we specified /*, any URL starting with / (except for the top level / URL) will invoke the specified handler.
Uploading files to the web server and store them on SD card's file system
Now wouldn't it be nice to update Arduino's Web server files using HTTP? This way we can focus on building the actual interface with the hardware, and provide just enough HTTP handlers to interact with it. After we implement a minimal user interface, we can iterate it without having to remove the SD card from the embedded project, copy the HTML, JavaScript and/or image files on a computer, and plug it back in. We could do this remotely from the computer, using a simple script.
TinyWebServer provides a simple file upload HTTP handler that uses the HTTP 1.0 PUT method. This allows you to implement an Ajax interface using XMLHttpRequest or simply use a tool like curl to implement file uploads.
Here's how you add file uploads to your Arduino web server:
TinyWebServer::PathHandler handlers[] = {
// `put_handler' is defined in TinyWebServer
{"/upload/" "*", TinyWebServer::PUT, &TinyWebPutHandler::put_handler },
{"/" "*", TinyWebServer::GET, &file_handler },
{NULL},
Note that the order in which you declare the handlers is important. The URLs are matched in the order in which they are declared.
This is where the headers array mentioned before comes into picture. The put_handler makes use of the Content-Length. To avoid unnecessary work and minimize precious memory usage, TinyWebServer does not do any header processing unless it's instructed. To do so, you need to declare an array of header names your handlers are interested in. In this case, we need to add Content-Length.
const char* headers[] = {
"Content-Length",
NULL
};
And we now initialize the instance of TinyWebServer like this:
TinyWebServer web = TinyWebServer(handlers, headers);
The put_handler method is really generic, it doesn't actually implement the code to write the file to disk. Instead the method relies on a user provided function that implements the actual logic. This allows you to use a different file system implementation than Fat16 or do something totally different than write the file to disk.
The user provided function take 4 parameters. The first is a reference to the TinyWebServer instance. The second is a PutAction enum which could be either START, WRITE or END. START and END are called exactly once during a PUT handler's execution, while WRITE is called multiple times. Each time the function is called with the WRITE param, the third and fourth parameters are set to a buffer and a number of bytes in this buffer that should be used.
Here is a small example of a user provided function that writes the PUT request's content to a file:
void file_uploader_handler(TinyWebServer& web_server,
TinyWebPutHandler::PutAction action,
char* buffer, int size) {
static uint32_t start_time;
switch (action) {
case TinyWebPutHandler::START:
start_time = millis();
if (!file.isOpen()) {
// File is not opened, create it. First obtain the desired name
// from the request path.
char* fname = web_server.get_file_from_path(web_server.get_path());
if (fname) {
Serial << "Creating " << fname << "\n";
file.open(fname, O_CREAT | O_WRITE | O_TRUNC);
free(fname);
}
}
break;
case TinyWebPutHandler::WRITE:
if (file.isOpen()) {
file.write(buffer, size);
}
break;
case TinyWebPutHandler::END:
file.sync();
Serial << "Wrote " << file.fileSize() << " bytes in "
<< millis() - start_time << " millis\n";
file.close();
}
}
To activate this user provided function, assign its address to put_handler_fn, like this:
void setup() {
// ...
// Assign our function to `upload_handler_fn'.
TinyWebPutHandler::put_handler_fn = file_uploader_handler;
// ...
}
You can now test uploading a file using curl:
curl -0 -T index.htm http://my-arduino-ip-address/upload
For a complete working example of the file upload and serving web server, look in TinyWebServer/examples/FileUpload.
Advanced topic: persistent HTTP connections
Sometimes it's useful to have an HTTP client start a request. For example, I need to be able to enter an IR learning process. This means that I cannot afford TinyWebServer's process() to block while serving my /learn request that initiated the IR learning process. Instead I want the handler of the /learn request to set a variable in the code that indicates that IR learning is active, and then return immediately.
If you noticed the HTTP handlers return a boolean. If the returned value is true, as it was the case in our examples above, the connection to the HTTP client is closed immediately. If the returned value is false the connection is left open. Your handler should save the Client object handling the HTTP connection with the original request. Your code becomes responsible with closing it when it's no longer needed.
To obtain the Client object, use the get_client() method while in the HTTP handler. You can write asynchronously to the client, to update it with the state of the web server.
In my remotely controlled projection screen application, I have another handler on /cancel that closes the /learn client forcibly. Otherwise the /learn's Client connection is closed at the end of the IR learning procedure. Since the Ethernet shield only allows for 4 maximum HTTP clients open at the same time (because of 4 maximum client sockets), in my application I allow only one /learn handler to be active at any given time.
June 15, 2010
Arduino Tiny Web Server - part 1 |
Arduino | Hardware | Open Source |
|
Update (December 8, 2010): The below information of the required Arduino hardware is obsolete and left here for informational purposes. Look at this newer post for updated information on the new hardware. Arduino TinyWebServer is a small and extensible HTTP server implementation designed to run in a limited amount of space on an Arduino Duemilanove. It uses the Ethernet Shield for network connectivity (from Sparkfun or from Adafruit), and the Adafruit Data Logging shield for storage purposes. Web pages, images and other content can be copied manually on the SD card or uploaded through the HTTP server. The latter allows you to push new versions of the web server's content without the need to remove the card, which can be a pain in embedded applications. In the first part I present some changes that have to be made to the hardware used and its accompanying software. Part two presents a small open source software library that implements the Arduino TinyWebServer. 
| |
Hardware modifications: Data Logging shield
The hardware shields need few modifications in order to work together. The cards were designed to work independently and use the default pins allocated on the hardware SPI bus (the CS, MOSI, MISO and SCK lines on the 10, 11, 12, 13 pins on an Arduino Duemilanove). When stacking the boards together they'd end up in a bus conflict and they won't work.
The conflict is solved by having the two boards use different CS pins. They can still share the MOSI, MISO and SCK lines, and if it wasn't for a buggy chip on the Ethernet shield, we'd have ended up using only 5 total digital I/O pins for the whole setup. See below for more info.
To make things easy, I chose to use a different CS pin for the Adafruit Data Logging shield: I use pin 9 as the CS pin. For this to work, first make sure you cut out the original trace that goes to pin 10, as in the picture below.
The Data Logging shield board comes unassembled. After you solder all the components on it, run a wire from the CS pin to pin 9, as shown in the picture below.
Hardware modifications: Ethernet shield
The Ethernet shield uses a Wiznet W5100 chip, which has a buggy hardware SPI implementation. In a post on Adafruit's forum, jaredforshey pointed me to this Arduino playground page which points to an easy way to fix this.
The proposed solution disables the chip' SPI part when not in use. This is done by connecting pin 8 to the lower PAD on the board, as shown below. At the same time, make sure you cut the trace leading from pin 8. This bug ends up costing us another pin, for a total of 6 I/O pins for the whole setup.
Software modifications: Fat16 library
To read and write files on an SD card, we need to be able to access a file system on the SD card. There are two main file systems used on SD cards: FAT16 and FAT32. The main differences between them are the maximum card sizes supported and more importantly, file naming conventions. FAT16 allows only the old 8.3 DOS file format and cards up to 2GB.
Arduino supports both file systems on SD cards using either of these libraries: Fat16 or SdFat. For all its limitations, FAT16' library is smaller that FAT32, so I decided to go with it.
Our Data Logging shield uses pin 9 as the CS pin. The FAT16 library assumes the CS pin used in pin 10, so we need to modify that in the code. For Arduino Duemilanove, the definition of SPI_SS_PIN in SdCard.h needs to change from 10 to 9.
Software modifications: Arduino's Ethernet library
The Ethernet library shipped with the Arduino 018 package has a bug. In the Client class in Client.h, the read() method does not differentiate between an 0xFF byte and the Ethernet hardware not having data available. This is not usually a problem if all you serve through the Web server are text files, including HTML. However for any binary file, including images, zip files etc. this however is a problem.
To fix this problem, I've added two more methods to the Client class:
int read(uint8_t* ch); int read(uint8_t *buf, size_t size);
The first reads a character and puts its value at the address pointed to by ch. The method returns 1 if it succeeded reading a character, 0 otherwise (as when there is no data available). The second method fills in the value of buf with as many characters as it can, up to size. It returns the number of characters it was able to read, or 0 if none were read. Here is how they're implemented:
int16_t Client::read(uint8_t *ch) {
if (!connected() || !available()) {
return 0;
}
return recv(_sock, ch, 1);
}
int16_t Client::read(uint8_t *buf, uint16_t size) {
uint16_t i;
for (i = 0; i < size; i++) {
if (!read(buf + i)) {
break;
}
}
return i;
}
The second change to the Ethernet library is in utility/spi.h, to fix the hardware bug with the Wiznet chip. This change is described on the Arduino playground page.
Arduino TinyWebServer: 
Apple (44)
Arduino (6)
BeagleBone (1)
Cars (5)
Cocoon (26)
Cool gadgets (14)
Emacs (24)
Google (15)
Hardware (9)
Java (31)
JavaScript (2)
Linux (12)
Music (4)
Open Source (22)
People (2)
Photo (23)
Politics (1)
Random (39)
Travel (4)
Web services (3)
Weblogs (30)
Work (1)
XSLT (12)
January 2015
January 2014
May 2013
June 2011
December 2010
September 2010
August 2010
June 2010
June 2009
May 2009
April 2009
February 2008
November 2007
July 2007
June 2007
May 2007
April 2007
October 2006
August 2006
June 2006
April 2006
March 2006
February 2006
January 2006
December 2005
November 2005
October 2005
September 2005
July 2005
June 2005
April 2005
March 2005
February 2005
October 2004
September 2004
August 2004
June 2004
May 2004
April 2004
March 2004
February 2004
January 2004
December 2003
November 2003
October 2003
September 2003
August 2003
July 2003
June 2003
May 2003
April 2003
March 2003
February 2003
January 2003
December 2002
November 2002
October 2002
September 2002
August 2002
July 2002
June 2002
