Saturday, January 17, 2015

SonosUPnP library for Arduino

The UPnP protocol was invented by Microsoft and saw the light of day in the late 1990s. The purpose of UPnP is to allow devices on a network to automatically discover and interface with each other. Microsoft chose to base UPnP on the hottest standards at that time, namely XML and SOAP (both introduced in 1998). XML and SOAP have since been criticized for their verbosity and complexity.

Project goals:

  • Making it possible to both control and read the state of my home speaker system using UPnP.
  • Keeping the binary size and RAM usage low enough for the library to run on Arduino boards using the ATmega328 chip.
  • Learning more about writing parsers, using PROGMEM and the UPnP protocol.

Reverse engineering with Wireshark:

It isn't feasible to implement the entire UPnP protocol on a device like the Arduino, and reading protocol specifications can be tedious. Therefore, for small projects like Sonos speaker integration, it's sometimes faster to use a tool like Wireshark to get an understanding of and replicate some of the messages being sent over the wire.

To get started with Wireshark: download, install, select capture interface and click start. Tip: Use the filter function.

Using PROGMEM (Flash) instead of RAM for storing strings:

The Arduino Uno and Duemilanove boards use the ATmega328 chip with only 2048 bytes of SRAM. Normally when you use strings (char arrays) in your code, all the strings will get copied to the heap on startup and waste your precious RAM.

The SonosUPnP library contains almost 4KB of string data and it would not be possible to run on the ATmega328 without using PROGMEM for strings. More information about using PROGMEM:

Sending UPnP messages seems simple enough, but what about receiving and parsing?

One of my goals when writing the SonosUPnP library was to support reading state in addition to controlling the Sonos speakers. This capability is also what sets my library apart from the other Sonos Arduino libraries I've seen so far.

Again the challenge is the limited amount of RAM: How do you parse and read the values you are interested in from an XML file while only using a few bytes of RAM? To accomplish this I decided to write a library for this particular purpose: MicroXPath

Downloads:

33 comments :

  1. Hi! I really enjoyed your work. Did you publish it somewhere?

    ReplyDelete
    Replies
    1. Oh no, I mean a scientifc paper or something.. By the way, did you have an example using just upnp on arduino or ESP8266? I mean, the code that creates devices.

      Delete
    2. I'm building such source code, see https://github.com/dannybackx/arduino-upnp .
      The code creates a simple device.

      Delete
  2. Utrolig bra jobbet! Om du har hørt om ESP8266, så vet du gjerne om de fordelene/bakdelene den har. Hva med å lage en port av libraryet til å kunne kjøres direkte på en ESP? Har større prosessorkraft og fjernet behovet for å bruke kabel.. Øyvind

    ReplyDelete
    Replies
    1. Jeg har ikke sett på ESP8266, men det bør være relativt enkelt å skrive om koden slik at den fungerer på en annen mikrokontroller. Hvor mye RAM er det på ESP8266? Ta kontakt hvis du ønsker å bidra. Epostadressen min finner du i toppen av alle kildekodefilene som kan lastes ned fra GitHub.

      Delete
  3. Is it possible to build a own sonos speaker with this library?

    ReplyDelete
    Replies
    1. this is a very gernic question.
      Sonos UPNP provides SOAP like messages to controll a speaker. You mihgt get inspired what and how to implement for a speaker. And just joking: you would also need some hardware.

      Delete
  4. Hi Thomas,
    I just started to get your code running on an ESP8266.
    In general your code looks very good structured.
    Where I got stucked: avr/pgmspace.h -> strlcpy_P
    and the ethernet client is strong coupled into the code.
    The setup of a Wifi Client looks a bit different.
    looks like there is a Ethernet.h implementation for the ESP8266.
    But I wonder how good that can work.
    I think I will start to replace the ethernet client.

    gbs

    ReplyDelete
  5. works with ESP8266 Wifi Client. It is funny to hear a norwegian radio station. code needs some clean up. and serial event seems not to work. but that is not part of that lib.

    ReplyDelete
  6. gds, have you published the esp8266 code?

    ReplyDelete
  7. not yet. I don't know a good way to do so. finally these are 10 line diffs to Thomas code.
    The best way would be to modify Thomas code, so it works for avr and esp8266.
    But this might get complicated.
    May be you (someone) can open a esp8266 branch.
    The other approach I copy Thomas code and replace some code to make it more efficient for the ESP8266. But I would have to ask him about it.
    I do not know about the culture for joint open source development.
    Thomas can you respond?

    ReplyDelete
  8. I'm interested for the code snipped also.

    ReplyDelete
  9. can you handle a diff?

    112c112
    < SonosUPnP::SonosUPnP(WiFiClient client, void (*ethernetErrCallback)(void))
    ---
    > SonosUPnP::SonosUPnP(EthernetClient client, void (*ethernetErrCallback)(void))
    175,176c175
    < // "x-sonos-http:" does not work for me etAVTransportURI(speakerIP, SONOS_SOURCE_HTTP_SCHEME, address);
    < setAVTransportURI(speakerIP, "", address);
    ---
    > setAVTransportURI(speakerIP, SONOS_SOURCE_HTTP_SCHEME, address);
    601,605c600
    < if (!ethClient.connect(ip, UPNP_PORT))
    < {
    < Serial.println("we did´nt got a connection");
    < return false;
    < }
    ---
    > if (!ethClient.connect(ip, UPNP_PORT)) return false;
    642c637
    < char buffer[1400];
    ---
    > char buffer[50];
    729c724
    < Serial.println(data);
    ---
    > //Serial.print(data);
    733,734d727
    <
    < //ToDo ESP8266 brings its own write_P, we better use this one
    740,746c733,735
    < {
    < // *((char *)mempcpy(dst, src, n)) = '\0';
    < //https://en.wikibooks.org/wiki/C_Programming/C_Reference/nonstandard/strlcpy
    < //memcpy_P(buffer, data_P + dataPos, bufferSize);
    < strncpy_P(buffer, data_P + dataPos, bufferSize);
    < //strlcpy_P(buffer, data_P + dataPos, bufferSize);
    < Serial.println(buffer);
    ---
    > {
    > strlcpy_P(buffer, data_P + dataPos, bufferSize);
    > //Serial.print(buffer);

    ReplyDelete
  10. diff SonosUPnP.h SonosUPnP_orig.h
    26,27c26
    < //#include "avr/pgmspace.h"
    < #include "pgmspace.h"
    ---
    > #include "avr/pgmspace.h"
    29c28
    < #include "MicroXPath_P.h"
    ---
    > #include "../../MicroXPath/src/MicroXPath_P.h"
    31c30
    < #include
    ---
    > #include "../../Ethernet/src/EthernetClient.h"
    280c279
    < SonosUPnP(WiFiClient client, void (*ethernetErrCallback)(void));
    ---
    > SonosUPnP(EthernetClient client, void (*ethernetErrCallback)(void));
    341c340
    < WiFiClient ethClient;
    ---
    > EthernetClient ethClient;
    370,378d368
    <
    < //#ifdef ESP8266 //has no strlcpy_P
    <
    < /* size_t ICACHE_FLASH_ATTR strlcpy_P(char* dest, const char* src, size_t size) {
    < const char* read = src;
    < char* write = dest;
    <
    < */
    <

    ReplyDelete
  11. and to overcome the not working serialEvent

    void loop()
    {
    if (Serial.available() >=2) handleSerialRead();
    else yield();
    ArduinoOTA.handle();
    server.handleClient();
    }

    void handleSerialRead()
    {
    // Read 2 bytes from serial buffer
    if (Serial.available() >= 2)
    { .....

    ReplyDelete
  12. Gds: thx for your code. I will check and test it in my implementation in the evening.

    To make the code compatible with Arduino and ESP8266 I use:
    #if defined ESP8266
    #else
    #endif

    In order to the missing function strlcpy_P I have done my own:

    char* mystrlcpy_P (char *dest, PGM_P src, size_t siz)
    {
    const char* read = src;
    char* write = dest;
    char ch = '.';
    size_t n = siz;

    /* Copy as many bytes as will fit */
    if (n != 0 && --n != 0) {
    do {
    ch = pgm_read_byte(read++);
    if ((*write++ = ch) == 0)
    break;
    } while (--n != 0);
    }
    /* Not enough room in dst, add NUL and traverse rest of src */
    if (n == 0) {
    if (siz != 0)
    *write = '\0'; /* NUL-terminate dst */
    }
    return dest;
    }

    This works fine at an Arduino Mega but not yet complete on a WeMos D1.

    ReplyDelete
  13. #if defined ESP8266 is a valid approach.
    but what keeps me off:
    I hope for a more generic object oriented solution for different Clients.
    strlcpy seem to be religion, more over ESP8266 WiFiClient brings its own write covering the MTU size 1480 byte limit. (which might fail in some IPSec VPN ...)
    And buffer[50] works but in combination with the loop and strlcpy ..
    you end up in a lot of small wifi packets.
    For a proof of concept it is ok. But for publishing the code I would like to think twice and to rework it.

    ReplyDelete
  14. Hi there, now the code runs on my WeMos D1 an the next step for expanding the code to deal with a cheap IR-Sender can be done. This is my first attempt to code on the ESP8266 platform and I've learnt that it's not a good idea to block the code in the main loop().
    Great example which shows how to deal with the little complex protocol for controlling the sonos devices and gives examples for the most important commands.
    Sure, the code has potential for optimizing and can be beautified. But regardless, thanks to the author for sharing it.

    ReplyDelete
    Replies
    1. Hi mastermind,

      I'm looking to build a sonos controller using a WeMos D1, from your post it sounds like you have some working code. Any chance you could share a copy?

      Thanks
      Paul

      Delete
    2. try this fork
      https://github.com/antonmeyer/sonos

      Delete
    3. Thanks just what I needed. Now I'm 95% there... only 1 small challenge remaining..

      How do I select a playlist? I've read through documenttaion and looked at the examples and I can now select which radio stream i want and volume control plus play/pause is all working I just can't work out how to tell Sonos I want to play a specific play list.

      any help welcome.

      Thanks
      Paul

      Delete
    4. simplest way to figure that out: use the sonos sw-controler and monitor with wireshark the conversation with the box when you select and change the play list. Might be, that you need to construct a new message type in the lib.

      Delete
    5. Does sonos still use HTTP? When I run wireshark and filter for http / http2 I don't see any packets.. First time using wireshark so possibly user error...

      Delete
    6. just start with ip address filter.
      yes it is still http
      I'm not sure where the protocol starts really but I found thinks like
      SOAPACTION: "urn:schemas-upnp-org:service:AVTransport:1#SetAVTransportURI"
      s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/" s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">0x-rincon-queue:RINCON_B8E937B73CC601400#0HTTP/1.1 200 OK

      Delete
  15. Would you mind sharing the code?

    ReplyDelete
  16. Would you mind sharing the code?

    ReplyDelete
  17. This comment has been removed by the author.

    ReplyDelete
  18. Oh, I have a new error.
    Can someone help Please?

    Arduino: 1.6.9 (Windows 7), Board: "Arduino/Genuino Mega or Mega 2560, ATmega2560 (Mega 2560)"

    Sonos_Control_and_State:84: error: 'ethConnectError' was not declared in this scope

    SonosUPnP g_sonos = SonosUPnP(g_ethClient, ethConnectError);

    ^

    exit status 1
    'ethConnectError' was not declared in this scope

    Dieser Bericht wäre detaillierter, wenn die Option
    "Ausführliche Ausgabe während der Kompilierung"
    in Datei -> Voreinstellungen aktiviert wäre.

    ReplyDelete
  19. This comment has been removed by the author.

    ReplyDelete
  20. check where you have declared this function. might be you need a forward declaration in before that line.

    ReplyDelete
  21. Hi, I'm trying to get trackinfo for my spotify tracks but the track URL is empty whenever I play a track from spotify. I've added a SONOS_SOURCE_SPOTIFY_SCHEME "x-sonos-spotify:" and a SONOS_SOURCE_SPOTIFY 6 etc.
    This a TrackURL from spotify, would you expect any problems? x-sonos-spotify:spotify%3atrack%3a2Sd7KgjJDMc7vhDe64nBpi?sid=9&flags=8224&sn=1

    ReplyDelete
  22. Hello!

    I really enjoy your library for Sonos! However, I am missing one part and there is one thing that doesnt work. Maybe because of updated Sonos players?

    1. I would like to get Metadata from the current playing song (so that i can display it)
    2. I cannot get the group and ungroup of players to work. I have tried all sorts of combinations, but nothing happens when i group the players (or ungroup).

    Is it something you can look into?

    Best Regards!

    ReplyDelete