X10 with Arduino Serial Protocol

When creating a serial protocol for the Arduino X10 libraries, my goal was to create a protocol that was "human readable" yet lightweight and easy to parse/interface with. In my opinion, making a serial protocol human readable also makes it more understandable and hence easier to integrate with and debug. But there is a tradeoff: making it too literate makes it harder to parse.

Another goal was to use the same protocol when receiving and sending. By this I mean that whatever serial messages that originate from the power line, RF or IR libraries should look the same as the commands you send to the Arduino to control stuff. With the latest version of the protocol, it's possible to copy almost any message you receive and send it back to the Arduino to trigger the same command. This also makes the protocol easier to interface with, since you can use the same logic when reading and creating commands.

Here are some of the things I did to make it more human readable:

• The first version of the serial protocol used ASCII values 0-255. A number of these ASCII values do not map to any displayable character or to characters like line-feed or tab. The obvious improvement here was to use two hex nibbles instead of one byte. Hex is easily parsable in C and is directly supported by the Arduino Serial libraries.
• I've tried to use characters that would make sense to identify different groups of commands. The way the protocol is created all commands are sent in chunks of 3 characters, where the first character identifies the command type. Letters A-P are reserved for house codes, R = Request Module State, S = Scenario Execute and X or x means that the following two characters is a byte written in hex.

Here are some things I did that made it less human readable but easier to parse:

• In the first version of the protocol the messages you received from the Arduino used a lot of words like "On", "Off", "PreSetDim", etc. Although this is easier to understand when you read it, it did not meet my goal of making it possible to use the same messages when receiving and sending commands to the Arduino. Although parsing commands like "A1_ExtendedCode_PreSetDim _50" is possible, it's definitely not easy to do in a few lines of code. So I decided to sacrifice some readability to get a smaller binary size. You can still enable debug mode to get the literate output though.

The Code:

• Arduino X10 libraries – Libraries for power line, RF and IR communication and an Arduino test sketch implementing the X10 with Arduino Serial Protocol. 
• X10.Net library – X10.Net serial communication library with test application. Written in C#.

The protocol:

The libraries and protocol support sending and receiving all standard and extended code X10 messages. Additionally I've added three custom commands: "Scenario Execute", "Request Module State" and "Wipe Module State".

Scenario Execute is a way of triggering sequences of commands and logic using a simple command. By sending message S01 you could, for example, set several X10 modules to different brightness levels and even make the events depend on some variable like the current time, ambient light or the state of another module. The scenario logic must be defined in the Arduino Sketch.

Request Module State let's you query the state of any appliance/dimmer module in your setup. Most X10 modules are one-way, which means that you can't query them for state directly. Because of that I added functionality to the X10ex library that enables it to monitor the power line and keep track of state automatically.
Another feature is the possibility to query state of all known modules on one or all house codes. This also serves the purpose of listing all modules used (that the Arduino has seen) in your setup. With standard X10 commands you would need to know the address of every module in use, or query every possible address, which would take about 4 minutes because of slow X10 power line signaling.
Request Module state has its own return prefix "MS:" and uses X10 extended code to return the on/off state and brightness level. Modules that haven't been "seen" by the Arduino (no state data stored) simply return nothing.

Wipe Module State makes it possible to wipe/clear module state stored in the Arduino EEPROM. State data is stored in EEPROM to make it survive resets and power loss. Currently you can either wipe module state of all modules by sending RW* message, or all modules on one house code by sending RWA (A = House Code A).

Return data Prefixes:

Prefixes are used when messages are sent from the Arduino to identify the message origin.

SD: Acknowledges a Serial Data message sent to the Arduino.
PL: Message originates from data received over the Power Line.
RF: Message originates from data received from RF remotes.
IR: Message originates from data received from Infrared remotes.
MS: Data originates from Memory State stored in Arduino EEPROM.

Error Handling:

"SD:_ExTimOut" Complete 3 or 9 character serial message was not received within one second from start to end.
"SD:_ExSyntax" Syntax error in serial message sent to the Arduino.
"PL:_ExBuffer" Power line message buffer full. Since X10 power line signaling is slow (about .3 to .6 seconds per message) the X10ex library needs to buffer incoming messages. The size of the buffer is defined in X10ex.h and defaults to 16 messages.
 
X10 Standard Messages examples:

A12 (House=A, Unit=2, Command=On)
AB3 (House=A, Unit=12, Command=Off)
A_5 (House=A, Unit=N/A, Command=Bright)
|||
||└- Command 0-F or _  Example: 2=On, 7=ExtCode, _ =No Cmd
|└-- Unit 0-F or _     Example: 0=Unit 1, F=Unit 16, _ =No Unit
└--- House code A-P    Example: A=House A, P=House P :)

X10 Extended Message examples:

A37x31x21 (A4, Cmd=ExtCode, ExtCmd=PRE_SET_DIM, ExtData=33)
B87x01x0D (B9, Cmd=ExtCode, ExtCmd=ShutterOpen, ExtData=13)
    |/ |/
    | └- Extended Data byte in hex      Example: 1F=50% bright.
    └---- Extended Command byte in hex  Example: 31=PreSetDim

Scenario Execute examples:

S03 (Execute scenario 3)
S14 (Execute scenario 20)
||/
|└-- Scenario byte in hex (Hex: 00-FF, Dec: 0-255)
└--- Scenario Execute Character

Request Module State examples:

R** (Request buffered state of all modules)
RG* (Request buffered state of modules using house code G)
RA2 (Request buffered state of module A3)
|||
||└- Unit 0-F or *        Example: 0=Unit 1, A=Unit 10, * =All
|└-- House code A-P or *  Example: A=House A, P=House P, * =All
└--- Request Module State Character

Wipe Module State examples:

RW* (Wipe state data for all modules)
RWB (Wipe state data for all modules using house code B)
|||
||└- House code A-P or *  Example: A=House A, P=House P, * =All
|└-- Wipe Module State Character
└--- Request Module State Character


X10 protocol external resources:

• X10 PLC protocol
  
• X10 Standard and Extended Code Protocol