UPnP framework #2115

mikee47 · 2020-10-16T22:29:31Z

The framework contains three libraries:

RapidXML: Parsing, serialising and de-serialising XML data
SSDP: Simple Service Discovery Protocol server
UPnP: Framework to build UPnP compliant device stacks. Written from scratch with reference to the documented standards. The Basic_UPnP sample gives an idea of what can be done. See the library README for further details,

The main purpose for this library was to provide an Alexa-compatible interface. It was important to provide a smooth end-user experience so that device discovery is reliable and painless. As such, device discovery and presentation is essentially complete, tested and working.

Also included is the Basic_Alexa sample with HueEmulator library.

Devices will appear in Windows explorer under 'Network'. You can find more comprehensive test tools at https://www.meshcommander.com/upnptools (Windows only - Device Spy is the most interesting! See #2114 README for notes on testing in Linux.

Both samples will run in Host emulator mode.

Additional changes:

Add isLocal() methods to AccessPoint & Station classes
Add additional toString() functions/methods
Make SMING_ARCH available to code and use it to generate Hue device names
HTTP Headers
Add Accept to standard header fields
Move HTTP header field name handling into separate unit, and add BasicHttpHeaders
Fix HTTP content type field parsing

May contain spaces which must be removed, for example:
```
      CONTENT-TYPE: text/xml ; charset="utf-8"\r\n
```
Also map application/xml (as well as text/xml) to MIME_XML
Fix LimitedMemoryStream, and add getStreamPointer method

Attempting to write data larger than available space discards all of it.
Correct behaviour is to write as much as possible, but still discard the remainder.

Also, seek operation (for reading) is now bi-directional between the start of the buffer and the current write position.

TODO:

Review Feature: Simple Service Discovery Protocol (SSDP) #2114 and merge appropriate content, e.g. Basic_SSDP sample
Handling of service requests and generating responses is not fully complete. Review and re-test, fix as required.
Document any known issues which require further development. Update TODO.rst.
Document BitSet class
Add home page for Basic_UPnp sample showing device stack with links to their individual presentation pages.
Document how to implement service requests (see Wemo device)

slaff

Looks good to me.

Sming/Core/Network/Http/BasicHttpHeaders.h

samples/Basic_UPnP/app/application.cpp

slaff · 2020-10-17T10:37:41Z

Review #2114 and merge appropriate content, e.g. Basic_SSDP sample

If you find something useful feel free to use it in your SSDP component (documentation, sample, code, etc.).
I have started SSDP because I wanted to discover my Smart TV and control it over DIAL. Hope it will be easy to extend your SSDP and create a DIAL component.

Handling of service requests and generating responses is not fully complete. Review and re-test, fix as required.

Do you mean SSDP? When yes take a look at #2114 Ssdp.cpp code.

Thank you for your fantastic work !

mikee47 · 2020-10-17T11:04:43Z

Service requests and discovery are UPnP things. https://smingdev.readthedocs.io/en/feature-upnp/_inc/Sming/Libraries/UPnP/index.html.

mikee47 · 2020-10-17T11:25:16Z

This library at present is focused on supporting devices, whereas your application is to control other devices. The SSDP library does support sending out search requests, and the responses flow into a callback handler. The UPnP DeviceHost class registers this so that all ~~responses~~ requests flow through the device stack.

To initiate a search call DeviceHost::search() you can do this:

		auto ms = new SSDP::MessageSpec(SSDP::MESSAGE_MSEARCH);
		ms->repeat = 2;
		ms->target = SSDP::TARGET_ROOT;
		SSDP::server.messageQueue.add(ms, 1000);

Have a look at DeviceHost.cpp to see how it interacts with SSDP. Using the full UPnP stack incoming SSDP messages get routed to a lambda callback in DeviceHost::begin(). At present only MSEARCH messages are handled. Handling incoming responses is part of UPnP control points which I have not yet dealt with. As you can see, extending the framework to do this should be fairly straightforward.

However, if you only care about SSDP and don't require any of the UPnP framework then you can use it stand-alone.

mikee47 · 2020-10-17T12:17:36Z

You can try discovery out with the Basic_UPnP sample and the above code fragment:

make SMING_ARCH=Host SSDP_DEBUG_VERBOSE=1

slaff · 2020-10-17T13:55:33Z

It would be great to allow Sming Applications to

announce themselves as devices with discoverable services
search for other devices based on search type and once there is response to trigger a callback to handle this response.

@mikee47 I see in your last change how to search for a smart TV, but how to trigger a callback once such a device is found and get the reported headers? I guess this is quite easy and I am just missing to see it.

mikee47 · 2020-10-17T14:19:35Z

@slaff No, you're not missing anything! The messages only appeared in debug output with SSDP_DEBUG_VERBOSE=1 which is nice but not terribly helpful.

I've now added an onNotify method to allow UPnP root devices to inspect incoming SSDP messages. You'll get everything except MSEARCH (i.e. NOTIFY and RESPONSE).

mikee47 · 2020-10-17T14:41:06Z

Found and fixed a bug in header parsing. Here's what my Chromecast reports:

CACHE-CONTROL: max-age=1800
DATE: Sat, 17 Oct 2020 14:33:56 GMT
EXT:
LOCATION: http://192.168.1.66:8008/ssdp/device-desc.xml
OPT: "http://schemas.upnp.org/upnp/1/0/"; ns=01
01-NLS: 2aad33da-1dd2-11b2-ac93-87606d68c49e
SERVER: Linux/3.8.13+, UPnP/1.0, Portable SDK for UPnP devices/1.6.18
X-User-Agent: redsonic
ST: urn:dial-multiscreen-org:service:dial:1
USN: uuid:3e21b953-ec39-083d-4381-5ade69112265::urn:dial-multiscreen-org:service:dial:1
BOOTID.UPNP.ORG: 216
CONFIGID.UPNP.ORG: 3

Fetching the device description file:

<?xml version="1.0"?>
<root xmlns="urn:schemas-upnp-org:device-1-0">
  <specVersion>
    <major>1</major>
    <minor>0</minor>
  </specVersion>
  <URLBase>http://192.168.1.66:8008</URLBase>
  <device>
    <deviceType>urn:dial-multiscreen-org:device:dial:1</deviceType>
    <friendlyName>Chromecast</friendlyName>
    <manufacturer>Google Inc.</manufacturer>
    <modelName>Eureka Dongle</modelName>
    <UDN>uuid:3e21b953-ec39-083d-4381-5ade69112265</UDN>
    <iconList>
      <icon>
        <mimetype>image/png</mimetype>
        <width>98</width>
        <height>55</height>
        <depth>32</depth>
        <url>/setup/icon.png</url>
      </icon>
    </iconList>
    <serviceList>
      <service>
        <serviceType>urn:dial-multiscreen-org:service:dial:1</serviceType>
        <serviceId>urn:dial-multiscreen-org:serviceId:dial</serviceId>
        <controlURL>/ssdp/notfound</controlURL>
        <eventSubURL>/ssdp/notfound</eventSubURL>
        <SCPDURL>/ssdp/notfound</SCPDURL>
      </service>
    </serviceList>
  </device>
</root>

I'd be curious to see what your Smart TV reports (I don't have a telly)...

NB. The HTTP response headers contains Application_URL: http://192.168.1.66:8008/apps/ which matches the DIAL spec. very nicely.

mikee47 · 2020-10-17T14:52:14Z

This is getting interesting. I submit the following request:

http://192.168.1.66:8008/apps/ChromeCast

and get this in response:

<?xml version="1.0" encoding="UTF-8"?>
<service xmlns="urn:dial-multiscreen-org:schemas:dial">
  <name>ChromeCast</name>
  <options allowStop="true"/>
  <state>stopped</state>
</service>

Hopefully you get something similar.

slaff · 2020-10-17T15:21:29Z

I'd be curious to see what your Smart TV reports (I don't have a telly)...

This is what I get. Panasonic Viera TV. It has a running web browser and one can also get things like icons and control it remotely (soon also with Sming :) )

<?xml version="1.0"?>
<root xmlns="urn:schemas-upnp-org:device-1-0" xmlns:vli="urn:schemas-panasonic-com:vli" xmlns:viera="urn:schemas-panasonic-com:viera" xmlns:pxn="urn:schemas-panasonic-com:pxn">
  <specVersion><major>1</major><minor>0</minor></specVersion>
  <device>
    <deviceType>urn:panasonic-com:device:p00RemoteController:1</deviceType>
    <friendlyName>xxxxx_Series</friendlyName>
    <manufacturer>Panasonic</manufacturer>
    <modelName>Panasonic VIErA</modelName>
    <modelNumber>S500</modelNumber>
    <UDN>uuid:4D454930-0200-1000-8001-20C6EBB4B17A</UDN>
    <viera:X_DMSUDN>uuid:4Dxxxxxxxxxxxxxx</viera:X_DMSUDN>
    <viera:X_DMRUDN>uuid:4Dxxxxxxxxx</viera:X_DMRUDN>
    <viera:X_NRCUDN>uuid:4Dxxxxxxxxxxxxxxxxxxx</viera:X_NRCUDN>
    <viera:X_MHC_DEV_ID>4284xxxxxxxx</viera:X_MHC_DEV_ID>
    <viera:X_VERSION>NRC-3.00</viera:X_VERSION>
    <viera:X_DEVICE_TYPE>DTV</viera:X_DEVICE_TYPE>
    <viera:X_KEY_TYPE>PAL-21,PAL-11,PAL-1</viera:X_KEY_TYPE>
    <viera:X_PAD_TYPE></viera:X_PAD_TYPE>
    <viera:X_NRCCAP>VR_DMR,VR_DMS,VR_VECTOR,VR_BROWSER,VR_LAUNCH,VR_MEDIADMS,VR_LVDMS,VR_UPDMS,VR_TUNER1,VR_TVMUTE,VR_MES,VR_UPBROWSER</viera:X_NRCCAP>
    <iconList>
      <icon>
        <mimetype>image/png</mimetype>
        <width>48</width>
        <height>48</height>
        <depth>24</depth>
        <url>/nrc/dlna_icon_48.png</url>
      </icon>
      <icon>
        <mimetype>image/png</mimetype>
        <width>120</width>
        <height>120</height>
        <depth>24</depth>
        <url>/nrc/dlna_icon_120.png</url>
      </icon>
      <icon>
        <mimetype>image/jpeg</mimetype>
        <width>48</width>
        <height>48</height>
        <depth>24</depth>
        <url>/nrc/dlna_icon_48.jpg</url>
      </icon>
      <icon>
        <mimetype>image/jpeg</mimetype>
        <width>120</width>
        <height>120</height>
        <depth>24</depth>
        <url>/nrc/dlna_icon_120.jpg</url>
      </icon>
    </iconList>
    <serviceList>
      <service>
        <serviceType>urn:panasonic-com:service:p00NetworkControl:1</serviceType>
        <serviceId>urn:upnp-org:serviceId:p00NetworkControl</serviceId>
        <SCPDURL>/nrc/sdd_0.xml</SCPDURL>
        <controlURL>/nrc/control_0</controlURL>
        <eventSubURL>/nrc/event_0</eventSubURL>
      </service>
    </serviceList>
  </device>
</root>

mikee47 · 2020-10-17T16:04:26Z

@slaff Have you found details of what commands these apps support? I also haven't found any way to enumerate supported apps on a device... there may not be one!

slaff · 2020-10-18T08:40:56Z

@slaff Have you found details of what commands these apps support?

Look here: https://github.com/jloutsenhizer/CR-Cast/wiki/Chromecast-Implementation-Documentation-WIP#app-list-format

Obtaining Receiver App List
In older versions of the Chromecast firmware, the applist was obtained from:

https://clients3.google.com/cast/chromecast/device/config

However, the applist in current Chromecasts is obtained from here instead:

https://clients3.google.com/cast/chromecast/device/baseconfig

mikee47 · 2020-10-18T09:39:50Z

Those JSON files are large! I'm preparing a PR for a JSON streaming parser which will help.

slaff · 2020-10-18T10:18:30Z

Those JSON files are large! I'm preparing a PR for a JSON streaming parser which will help.

You know about these existing libraries: https://www.arduino.cc/reference/en/libraries/json-streaming-parser-2/ and https://www.arduino.cc/reference/en/libraries/json-streaming-parser/ , right?

mikee47 · 2020-10-18T10:23:41Z

LOL, yep, it's a port of that library... may have made some 'improvements' :-)

samples/Basic_Alexa/app/application.cpp

samples/Basic_UPnP/app/application.cpp

samples/Basic_Alexa/app/application.cpp

May contain spaces which must be removed, for example: CONTENT-TYPE: text/xml ; charset="utf-8"\r\n

…ing. Library header paths changed to <Network/SSDP/...> and <Network/UPnP/...> Standardise on `toString()` functions Library directory structures revised: source code goes into `src` subdirectory to keep things tidy RapidXML assertions changed to runtime error handling - empty string (bodyparser bug) caused it to hang system

Quirk of build system means it's better to use relative paths "" instead of <>

…ample

…ifications

Add NTP client to ensure system clock is correct. Without this the UPnP headers won't contain a DATE field. Revisions to Hue Emulator library to fix device notifications for standard devices.

…ames

…asicHttpHeaders

…rate unique base URL.

…us request handling

…entation pages.

slaff · 2020-10-24T17:57:55Z

@mikee47 Is this PR ready for merging?

mikee47 · 2020-10-24T19:01:37Z

@slaff Yes, all done for now.

mikee47 changed the title ~~UPnP framework~~ UPnP framework [WIP] Oct 17, 2020

slaff reviewed Oct 17, 2020

View reviewed changes

Sming/Core/Network/Http/BasicHttpHeaders.h Outdated Show resolved Hide resolved

samples/Basic_UPnP/app/application.cpp Outdated Show resolved Hide resolved

mikee47 force-pushed the feature/UPnP branch 2 times, most recently from 9a83c96 to 76bb9d3 Compare October 17, 2020 11:52

mikee47 force-pushed the feature/UPnP branch 2 times, most recently from 0a5e9be to 5eecaa0 Compare October 17, 2020 12:53

slaff mentioned this pull request Oct 18, 2020

Feature: Simple Service Discovery Protocol (SSDP) #2114

Closed

slaff added this to the 4.2.0 milestone Oct 18, 2020

slaff changed the title ~~UPnP framework [WIP]~~ [WIP] UPnP framework Oct 18, 2020

slaff reviewed Oct 18, 2020

View reviewed changes

samples/Basic_Alexa/app/application.cpp Outdated Show resolved Hide resolved

samples/Basic_UPnP/app/application.cpp Outdated Show resolved Hide resolved

mikee47 force-pushed the feature/UPnP branch 4 times, most recently from b6a781d to 9a0596d Compare October 18, 2020 23:43

slaff reviewed Oct 19, 2020

View reviewed changes

samples/Basic_Alexa/app/application.cpp Outdated Show resolved Hide resolved

slaff reviewed Oct 19, 2020

View reviewed changes

samples/Basic_Alexa/app/application.cpp Outdated Show resolved Hide resolved

mikee47 added 21 commits October 24, 2020 16:21

Fix HTTP content type field parsing

3268397

May contain spaces which must be removed, for example: CONTENT-TYPE: text/xml ; charset="utf-8"\r\n

Rename BasicHttpHeaders::statusStr() -> statusText()

dc5d262

Update #include paths in library files

aae965a

Quirk of build system means it's better to use relative paths "" instead of <>

Add basic demonstration of searching for DIAL devices to Basic_UPnP s…

b6a5aac

…ample

Added onNotify method for root devices to receive incoming SSDP not…

9653614

…ifications

Bugfix: Empty header values cause problems

9204e85

Fix Basic_Alexa sample application

b26f296

Add NTP client to ensure system clock is correct. Without this the UPnP headers won't contain a DATE field. Revisions to Hue Emulator library to fix device notifications for standard devices.

Make SMING_ARCH available to code and use it to generate Hue device n…

579a298

…ames

Move HTTP header field name handling into separate unit, and extend B…

c2c102a

…asicHttpHeaders

Add Accept to standard header fields

b91a1de

Interpret application/xml as MIME_XML

ddb9d2a

Add DeviceFinder class, based on new UPnP ControlPoint

d753efb

Add a Tea Pot.

9c6f01a

Add an ID for TeaPot, as there may be more than one. Use this to gene…

e098558

…rate unique base URL.

Update Basic Alexa sample with custom device demonstrating asynchrono…

2fae53d

…us request handling

Remove test code from Basic_UPnP sample and simplify. Fix device pres…

d8f67e1

…entation pages.

Fix build warnings

80b6129

Update SSDP to use new UdpConnection methods

d6260a9

Move sample applications into related Library

a193f46

Document BitSet, update Basic_UPnP sample

60d29fd

mikee47 force-pushed the feature/UPnP branch from bf0b3fc to 60d29fd Compare October 24, 2020 15:38

slaff removed the 3 - Review label Oct 24, 2020

slaff merged commit d4e57da into SmingHub:develop Oct 24, 2020

mikee47 deleted the feature/UPnP branch October 24, 2020 19:39

mikee47 added a commit to mikee47/Sming that referenced this pull request Oct 28, 2020

Remove ArduinoFFT - inadvertently included wth SmingHub#2115

1eb5ada

slaff mentioned this pull request Nov 2, 2020

Preparation for Release 4.2.0 #2069

Closed

mikee47 mentioned this pull request Nov 5, 2020

Add ContentType overload functions and strong enums #2140

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

UPnP framework #2115

UPnP framework #2115

mikee47 commented Oct 16, 2020 •

edited

Loading

slaff left a comment

slaff commented Oct 17, 2020

mikee47 commented Oct 17, 2020

mikee47 commented Oct 17, 2020 •

edited

Loading

mikee47 commented Oct 17, 2020 •

edited

Loading

slaff commented Oct 17, 2020

mikee47 commented Oct 17, 2020

mikee47 commented Oct 17, 2020 •

edited

Loading

mikee47 commented Oct 17, 2020

slaff commented Oct 17, 2020 •

edited

Loading

mikee47 commented Oct 17, 2020

slaff commented Oct 18, 2020

mikee47 commented Oct 18, 2020

slaff commented Oct 18, 2020

mikee47 commented Oct 18, 2020

slaff commented Oct 24, 2020

mikee47 commented Oct 24, 2020

UPnP framework #2115

UPnP framework #2115

Conversation

mikee47 commented Oct 16, 2020 • edited Loading

slaff left a comment

Choose a reason for hiding this comment

slaff commented Oct 17, 2020

mikee47 commented Oct 17, 2020

mikee47 commented Oct 17, 2020 • edited Loading

mikee47 commented Oct 17, 2020 • edited Loading

slaff commented Oct 17, 2020

mikee47 commented Oct 17, 2020

mikee47 commented Oct 17, 2020 • edited Loading

mikee47 commented Oct 17, 2020

slaff commented Oct 17, 2020 • edited Loading

mikee47 commented Oct 17, 2020

slaff commented Oct 18, 2020

mikee47 commented Oct 18, 2020

slaff commented Oct 18, 2020

mikee47 commented Oct 18, 2020

slaff commented Oct 24, 2020

mikee47 commented Oct 24, 2020

mikee47 commented Oct 16, 2020 •

edited

Loading

mikee47 commented Oct 17, 2020 •

edited

Loading

mikee47 commented Oct 17, 2020 •

edited

Loading

mikee47 commented Oct 17, 2020 •

edited

Loading

slaff commented Oct 17, 2020 •

edited

Loading