Sunday, October 24, 2010

Arduino + Ethernet + RESTful JSON


Why use REST and JSON?

REST and JSON have become very popular the last few years and is currently supported by most platforms/development-frameworks (iPhone, Android, Java, .Net). REST with JSON is lightweight compared to protocols like SOAP, and in my opinion, much easier to understand. It is also quite easy to implement a RESTful web service on the Arduino.

Getting started:

Before you start implementing a REST web service in your app, it's a good idea to get familiar with it using a tool like the Poster add-on for Firefox. This add-in lets you interact with the service and use HTTP methods like POST and DELETE. Another great Firefox add-on is JSONView, that lets you view JSON response in the browser similar to how XML documents are shown.

The Arduino X10 RESTful JSON protocol:

I've chosen to use JSON only, XML is not supported. If you go to the base URI of the web service (http://arduino_ip/), you'll get a response containing the info and state of modules seen by the Arduino. "Seen" means all modules that have received messages on their address over the power line. Additionally the protocol supports storing module types (0 = Unknown, 1 = Appliance, 2 = Dimmer and 3 = Sensor) and user defined module names of up to 16 characters. State data, module types and module names are stored in Arduino EEPROM and survives power loss and resets. Here's an example of what the response looks like when I issue a GET request to the base URI on my setup:

{
  "module":
  [
    {
      "house": "A",
      "unit": 1,
      "url": "/A/1/",
      "type": 1,
      "name": "Amplifier",
      "on": true
    },
    {
      "house": "A",
      "unit": 2,
      "url": "/A/2/",
      "type": 2,
      "name": "Dining Table",
      "on": true,
      "brightness": 40
    },
    {
      "house": "A",
      "unit": 4,
      "url": "/A/4/",
      "type": 2,
      "name": "TV Backlight",
      "on": true,
      "brightness": 26
    },
    {
      "house": "A",
      "unit": 8,
      "url": "/A/8/",
      "type": 2,
      "name": "Hall",
      "on": false,
      "brightness": 69
    },
    {
      "house": "A",
      "unit": 9,
      "url": "/A/9/",
      "type": 2,
      "name": "Kitchen",
      "on": false,
      "brightness": 100
    }
  ]
}

If you are using several house codes you can get the JSON response for one specific house by adding the house code to the base URI like this: http://arduino_ip/A/. All my X10 modules use house code A, so in my case the response would be the same.
To get response for one module only you add both house code and unit code to the base URI as follows: http://arduino_ip/A/2/. The output would look like this:

{
  "house": "A",
  "unit": 2,
  "url": "/A/2/",
  "type": 2,
  "name": "Dining Table",
  "on": false,
  "brightness": 40
}

Updating fields with HTTP POST:

All fields except house, unit and url can be updated using post. If you are posting to the base URI of the service you need to include the house and unit code fields in the post data, if you are posting to a base + house URI you only need to include the unit code and when posting to a base + house + unit URI you only need to include the fields you would like to update.

Example 1 (turn of unit 2 on house A):

URI: http://arduino_ip/
METHOD: POST
BODY: house="A"&unit=2&on=false
RESPONSE: Info for all "seen" modules

Note:
When specifying house and unit in POST data the URI is ignored for update, but you still get the response for that URI. You could, for example, update state of module B3 by posting: house="B"&unit=3&on=true to URI: http://arduino_ip/A/1/, but the response will contain data for module A1.

Example 2 (set brightness of unit A4 to 90%):

URI: http://arduino_ip/A/4/
METHOD: POST
BODY: brightness=90
RESPONSE:
{
  "house": "A",
  "unit": 4,
  "url": "/A/4/",
  "type": 2,
  "name": "TV Backlight",
  "on": true,
  "brightness": 90
}

Note:
All POST data fields that result in something being sent over the power line (on, brightness and cmd) are exclusive. As soon as the parser finds one of these fields it will execute the command and stop the parser. Therefore: on, brightness and cmd commands should be the last field in the POST body.

Example 3 (changing type and name of unit A1):

URI: http://arduino_ip/A/1/
METHOD: POST
BODY: type=3&name="Light Sensor 1"
RESPONSE:
{
  "house": "A",
  "unit": 1,
  "url": "/A/1/",
  "type": 3,
  "name": "Light Sensor 1",
  "on": true
}

Deleting module state and info with HTTP DELETE:

Since there is no way for the Arduino to know if a module no longer exists in your setup, the web service lets you delete module data by issuing a HTTP DELETE request.

Example 1 (delete info for module A2):

URI: http://arduino_ip/A/2/
METHOD: DELETE
RESPONSE:
{
  "house": "A",
  "unit": 2,
  "url": "/A/2/"
}

Note:
The response will not be empty when you send the request to specific module URI. But when requesting all modules or all modules on one house code a deleted module will no longer be visible.

Example 2 (delete info for all modules on house A):

URI: http://arduino_ip/A/
METHOD: DELETE
RESPONSE:
{
  "module":
  [

  ]
}

Example 3 (delete info for ALL modules):

URI: http://arduino_ip/
METHOD: DELETE
RESPONSE:
{
  "module":
  [

  ]
}

Warning:
State data, types and names of ALL modules will be deleted permanently.

Using special HTTP POST field "cmd":

A complete RESTful representation of the X10 Extended protocol would be difficult to implement on the Arduino, and the binary size would grow out of proportions. Therefore I have implemented a simple way of using the X10 with Arduino Serial Protocol to send complex messages.

When using the "cmd" field it's possible to send multiple commands at once, simply by concatenating all the messages. This is useful if you want to implement scenarios in your application. The total length of the field value cannot exceed 60 characters (20 standard messages or 6 extended messages), anything longer than this will be ignored. Also keep in mind that the default command buffer size, defined in X10ex.h, is 16. This means that the buffer will run out of space after the 16th command.

Click here for more info on the X10 with Arduino Serial Protocol.

Example 1 (Turn off module A2 and A4):

URI: http://arduino_ip/ (URI is not important)
METHOD: POST
BODY: cmd="A13A33"
RESPONSE: Depends on the URI

Note:
REST service will not wait for command execution to finish when using "cmd" field. Because of this, the response you get back will not represent the state of the modules after the command has been executed. You can get updated state by sending one or more HTTP GET requests.

Example 2 (Turn on module A4 and set brightness of A2):

URI: http://arduino_ip/ (URI is not important)
METHOD: POST
BODY: cmd="A32A17x31x1E"
RESPONSE: Depends on the URI

Example 3 (Turn on modules 3 to 16 in house A):

URI: http://arduino_ip/ (URI is not important)
METHOD: POST
BODY: cmd="A22A32A42A52A62A72A82A92AA2AB2AC2AD2AE2AF2"
RESPONSE: Depends on the URI

4 comments :

  1. Very slick - what about consuming web services from an Arduino - any good ideas on what to use for that?

    ReplyDelete
  2. Hello Thomas,

    Using Arduino X10 with Android programm.
    Great work ,thanks
    Philip, Netherlands

    ReplyDelete
  3. Thomas,
    I cannot thank you enough for this fantastic work and for making it public. I have fully implemented this together with an XTB-IIR from http://jvde.us/ and it is working beautifully. I have one small but significant tissue I can't get over and that is when I send A43, A44, A93 or A94 for units 5 or 10 on/off. The Arduino sends numerous instructions to the X10 controller everytime. I have to avoid those device numbers. The code does this through serial and http. Any ideas?
    Many thanks.

    ReplyDelete
  4. Fiber optic cables are the best internet connection offered today. It guarantees high speeds for shipment of large data volumes. forbrukeretaten, the lines are symmetrical, which means that the upload speed is as high as the download speed. Despite the technical advantages of fiber optic cables, there are some practical disadvantages to choosing this connection type.

    ReplyDelete