In this first post, Dave explains how to get an iPhone connected to LilyPad Arduino over a wireless connection.
I’ve been helping out with the communications plumbing of Alyson’s PleaseReadMe book project. My role is to enable an iPhone to receive data from some Arduino sensors hidden inside a book. To make the book feel magical, this communication needs to happen over a hidden wireless connection. The data received by the iPhone will then be processed by Mo Ramezanpoor (the third member of the PleaseReadMe team), who is writing some code to detect the 3D gestures made when people interact with the book.
While writing this plumbing code, I realised it would be handy to have a way to test the project (and particularly its storytelling) without needing the full Arduino setup in place. This post shares what I’ve discovered about iOS wireless options; describes how to get the Arduino and iOS worlds talking; and introduces some useful tricks to speed up development.
Wireless communication options
For the final version of PleaseReadMe, Alyson is hiding an Arduino LilyPad board inside a book. This LilyPad board reads data from sensors (including an accelerometer) embedded in the book, and transmits this data wirelessly to an iPhone. The iPhone then processes the data, and tells an interactive audio story (via bluetooth headphones) based on the book’s movement.
To achieve this, we need to transmit the data wirelessly from Arduino to iPhone. We’ve investigated several options for this, inspired by Alasdair Allan’s excellent iOS Sensor Apps with Arduino book (which is essential reading for this kind of project). It’s fair to say that some approaches have been more successful than others.
The first and most obvious approach we tried was Bluetooth. Bluetooth is great for short-range wireless communication, and there are plenty of options for adding Bluetooth to the Arduino side of the project. However, Apple doesn’t allow iOS developers to access the iPhone’s Bluetooth chip directly – at least, not without jailbreaking the iPhone, which we’re trying to avoid. So unfortunately, Bluetooth isn’t an option if we’re using iOS.
Another option we investigated was Bluetooth LE (where ‘LE’ stands for ‘Low Energy’.) This is a newer variant of Bluetooth, designed specifically for low-power, low-bandwidth communications. It’s ideal for sensors that need to send small amounts of data quickly without using too much power – in other words, it’s exactly what we need.
The good news is that Apple added Bluetooth LE support to its iOS device range starting with the iPhone 4S. Apps on recent Apple devices can use the Apple-provided Core Bluetooth framework to talk directly to the Bluetooth LE chip in the device. Unlike traditional Bluetooth, there are no limitations on how Bluetooth LE can be used on iOS.
The bad news is that there isn’t (currently) a way to add Bluetooth LE support to Arduino. This is, however, about to change. Dr. Michael Kroll is on the verge of shipping a Kickstarter-funded Bluetooth LE shield for the Arduino Uno board, which will enable iOS devices to talk directly to Arduino via Bluetooth LE. (A ‘shield’ is a pre-made electronics board that adds extra functionality to your Arduino setup.)
This shield is too chunky for PleaseReadMe, which needs something small and embeddable; nonetheless, it’s great to see it in development. (I backed the Kickstarter project myself a few months back, and will post some thoughts about the shield when it arrives.)
Dr. Kroll is also working on a smaller BLE shield, called the BLEbee, which is much more suited to PleaseReadMe. This has the potential to remove several of the communication layers described below, but is still in development at the time of writing. Who knows – we may switch to a BLEbee if it becomes available before the project is complete; for now, we need to press on with what’s currently available.
Having discounted Bluetooth LE, we next looked at Wi-Fi, but decided that this wasn’t the way to go. It tends to require more battery power on the Arduino side than we can really supply, and the communications side of things is harder on the iOS side, where we’d need to set up our own web server. It’s not really the right tool for the job, given that all we really want is for two devices to talk directly to each other.
In the end, we have chosen XBee as our wireless technology of choice. XBee is a low-power, mid-range wireless technology, which is popular in the Arduino world when two devices need to talk directly to each other. It’s the simplest way to replace your Arduino’s direct serial cable with a wireless connection instead, and there are loads of XBee modules and boards available for Arduino. Things aren’t quite so easy on the iPhone side – but thanks to Alasdair Allan’s book, we found a solution that works.
Here’s a simplified picture of the approach we’re taking:
Here’s a list of the components in the illustration.
- An iPhone (but it could just as easily be an iPod touch or an iPad)
- A Redpark Serial Cable – this connects the iPhone’s 30-pin dock adapter to a standard RS-232 male serial port (the 9-pin connector you used to see on the back of PC computers), so that you can hook the iPhone up to external gadgets
- An XBee Explorer Serial Board – this connects an XBee module to the RS-232 port on the Redpark Serial Cable, so that the iPhone can talk to the XBee
- An XBee module (we went for the Series 1 XBee 1mW Trace Antenna module, which is good for low-power, direct-communication use) – this manages the iPhone end of the wireless conversation
The XBee module plugs in to the XBee Explorer Serial Board, which is in turn connected to the iPhone via the Redpark Serial Cable. Redpark provide a software framework for iOS that enables us to get the iPhone talking directly to whatever’s plugged in to the cable – in our case, this means getting it to listen to whatever’s coming in via the XBee module.
- A LilyPad Arduino Main Board – this runs the Arduino code to retrieve the accelerometer data, and to package it up ready for wireless transmission
- A LilyPad XBee breakout board – this provides the appropriate connectors to hook up an XBee module to your LilyPad Main Board
- A second XBee module (same type as above), for the Arduino end of the wireless conversation
- A LilyPad Accelerometer – to detect the motion of the book
The XBee module plugs into the LilyPad XBee breakout board. This board, and the LilyPad accelerometer, are then wired up to the LilyPad Main Board, which reads values from the accelerometer, and sends them over the XBee wireless connection to the iPhone.
Downsides to this approach
There are a few notable downsides to this approach. Firstly, the Redpark cable has to be plugged in to the iPhone’s dock connector in order for the iPhone to listen to data from the XBee. This means that you can’t charge the iPhone while the Redpark cable is plugged in. We’ve considered something like the CableJive dockStubz 30-pin Pass Through Adapter, which would sit between the iPhone and the Redpark, but haven’t yet tried this to see if it would solve the problem.
Secondly, you can’t load a new version of your iOS app onto the iPhone while the Redpark cable is plugged in. You have to unplug the Redpark; plug in an iPhone-to-USB lead (one of the Apple-supplied white leads you use to charge your iPhone); load on a new version of your software via Xcode (Apple’s iPhone development software); unplug the lead; and plug the Redpark back in again. This is a pain. (Thankfully, I’ve come up with some neat workarounds – but more on that later.)
Finally, this approach really does need more kit on the iPhone side of things than feels strictly necessary. Once Dr. Kroll’s BLEbee module is available, we should be able to simplify things to look like this:
Sadly, for now we’re stuck with the Redpark approach.
Speedy story development
Because of the downsides mentioned above, I’ve come up with a few ways to speed up development of the PleaseReadMe project. This is particularly important because Alyson is writing a detailed story to respond to people’s interaction with the book. Such a story needs a lot of tweaking and rewriting based on real-world testing; having to do the whole unplug-and-replug dance every time you make a change slows this process down hugely.
There are two main things I’ve set out to achieve, to speed up story development:
- Find a way to use an iPhone’s built-in accelerometer as a stand-in for the Arduino accelerometer. This avoids the need to keep unplugging the Redpark cable in order to get accelerometer data from Arduino hardware via XBee – during story development, we can just get the motion data from the hardware in the iPhone instead.
- Find a way to use the iPhone Simulator, rather than a real iPhone, for processing the accelerometer data. This avoids the need to keep installing the storytelling software on a physical device, thereby speeding up development.
In order to achieve this, we’ve actually ended up with four different development modes. I’ll describe each mode in more detail below.
Mode 1: Arduino Hardware, iPhone Software
This is the original mode described above, which will be use in the final project. Accelerometer input comes from a LilyPad Accelerometer, via XBee, into a physical iPhone. The iPhone processes it, and plays the story in response.
As part of this, I’ve standardised the accelerometer input from the Arduino hardware so that it fits within the same numeric ranges as the iPhone’s built-in accelerometer. This means that Mo can write his gesture detection code in a hardware-agnostic way, and means that our testing is valid regardless of which of the four modes are being used.
Mode 2: Arduino Hardware, Simulator Software
In this mode, we still use the Arduino hardware to give us the accelerometer input, but we run the gesture-processing and storytelling software in the iOS Simulator on a Mac, rather than on an actual iPhone. This is great for testing the story using a real book with all of the hardware hidden inside, as we don’t need to unplug the Redpark cable whenever the story needs tweaking.
To do this, I took the iPhone out of the equation altogether, and connected the listening XBee module straight to a Mac. This was done using an XBee Explorer Dongle, which connects an XBee module to a Mac via USB.
Having got the XBee connected to the Mac, I then adapted Gabe Ghearing’s Arduino Serial Example (from the Arduino & Objective-C page of the Arduino site), to enable the iOS Simulator to read data directly from the XBee Explorer Dongle.
Mode 3: iPhone Hardware, iPhone Software
In this mode, we run everything on a single iPhone. The iPhone’s hardware accelerometer provides the motion input, with the gesture processing and storytelling code also running on the same device.
The advantage to this mode is that we can strap the iPhone to a book, or stick it inside a book-shaped box, and give the whole thing a try without needing the Arduino hardware to be set up.
Mode 4: iPhone Hardware, Simulator Software
This has turned out to be the most useful mode of all. In this mode, we run a special app (called AccSim) on a real iPhone. This special app streams the real iPhone’s accelerometer data over a Wi-Fi network to a Mac, which is running our gesture-recognising and storytelling app in the iPhone Simulator.
The source code download contains an additional class (called AccelerometerSimulation), which can be included in your own apps to make them listen for accelerometer input over the network. I’m using an adapted version of this in our own app.
For our setup, I installed the AccSim app on an iPhone; set the Network setting to ON with a mode of Broadcast on port 10552; and used an Accelerometer mode of Sensor. I then made our app listen for input on the same port using my adapted version of the AccelerometerSimulation class.
The great thing about this approach is that we can do all of our development on a Mac, compiling and building for the Simulator, and just strap an iPhone (or iPod touch) onto a placeholder book during development. In fact, it doesn’t even need the book, meaning that Alyson can develop the story wherever she is, with no more than her Mac and her iPhone to hand.