Tuesday, November 29, 2016

Using MQTT with The Things Network

In this episode we will cover how to use MQTT to display the values from the temperature sensing Lora node, built in the previous posts.

1. On the PC using the Chrome extension MQTTlens

I have found Mosquitto cumbersome to install on Windows. As an alternative I found the Chrome extension MQTTlens which can be installed from the Google Chrome webstore into your Chrome browser. The process is very straightforward.
Once the MQTTlens extension is installed open up the MQTTLens application, you should see this window, click on the '+' sign to add a new connection:

This will open  a dialog box to create a new connection. Use the following values:
Connection name: "temperature sensor" Give the connection a meaningful name
Host name: Leave the protocol to TCP, for the host name use: 'staging.thethingsnetwork.org', leave the port to the default 1883.
Client ID use the default (random) ID.
The tricky part is the username and password for the connection.
The username is the application ID and the password is base64 encoded application access key.
Navigate to your application on https://staging.thethingsnetwork.org. On the landing page for the application you will see a link to 'learn how to get data from this app'.

This will open a screen containing the App EUI and the application access keys. Scrolling down the page, click on the 'mqtt' tab. The page will display the 'username' value and the 'password' value required for the MQTT client. Clicking on the eye, will allow you to copy paste the password (access key) from the web page (2 below).

 Click create connection to establish the connection with the MQTT broker from The Things Network.

If the connection is successfully created you will now land on this screen and newly created connection will show up in green to indicate the connection is working:

As a last step we have to subscribe to topics. Since in this case we have only one device and it is only publishing one value we can take the risk to use the '#' wildcard to subscribe to all topics from all devices and see what we get. Enter '#' in the subscribe field and click on the subscribe button:

Make sure the device is transmitting by observing the messages in device dashboard.

In MQTTLens you should see now the MQTT message appear:

The message is in JSON format, but we see clearly the object 'temperature' has a value of 19.25.
On a side note I change the decode script to return a 'temperature' object. The other objects and their values can be used for other purposes, most of the values make sense by just looking at them. Click on the JSON button on the right to make the information more readable:

The '#' wildcard subscription might overwhelm the viewer when multiple devices and topics are present. To subscribe to a particular devices up message you can enter the application ID and the device ID as follows:

<App EUI in hex>/devices/<full Dev EUI in hex>/up


This will subscribe to the messages send specifically from the temperature node.
in our case this looks like:


For more details I suggest you look at some of tutorials on The Things Network, linked below.

If you got this far, congratulations, you are now able to subscribe with MQTT to a value published by your node. At this point the sky is the limit on what the possibilities are with the data coming from a node. In a coming episode I will show the use of Node Red to subscribe to the MQTT broker and work with the data coming from the node.

2. On Linux using Mosquitto

Using MQTT in Linux from the command line is a lot simpler.

To install Mosquitto client and broker using the latest Linux/Ubuntu, use your favorite package manager and search for mosquitto, or use the following command line:

sudo apt install mosquitto

Once installed run the Mosquitto MQTT client from the command line to subscribe to the temperature node data topic as follows:

mosquitto_sub -h staging.thethingsnetwork.org -u userID -P userPasswd -v -t '#'
-h = hostname
-u = username (obtain from the application dashboard on staging.thehtingsnetwork.org)
-P = userpassword (obtain from the application dashboard on staging.thehtingsnetwork.org)
-v = verbose
-t = topic '#' is all topics

mosquitto_sub -h staging.thethingsnetwork.org -t '#' -u 70B3D57ED0001685  -P 'N2hfjbCNzBGT67hjlbPjDdhRbFFiR5F0r/LClGG/rd8=' -v

(not the real password)

This will return the JSON object for the node data in plain text for processing by other applications:

 70B3D57ED0001685/devices/00000000239810FE/up {"payload":"AACaQQ==","fields":{"temperature":19.25},"port":1,"counter":96,"dev_eui":"00000000239810FE","metadata":[{"frequency":902.299,"datarate":"SF7BW125","codingrate":"4/5","gateway_timestamp":2183734710,"channel":0,"server_time":"2016-11-30T06:08:39.915081058Z","rssi":-44,"lsnr":9,"rfchain":0,"crc":1,"modulation":"LORA","gateway_eui":"5CCF7FFFFF132EEA","altitude":200,"longitude":-117.05,"latitude":32.9879}]}

(emphasis added in bold)

If you want to install the mosquitto client only, you can use this command:

sudo apt install mosquitto-clients

See more details in the link below on how to use MQTT with The Things Network.
My goal here was to simplify the process with some hand holding and wet your appetite.

3. Useful links

using MQTT with The Things Network: https://www.thethingsnetwork.org/docs/current/mqtt/
Installing mosquitto on Windows and Linux: http://www.steves-internet-guide.com/install-mosquitto-broker/
Introduction to MQTT: http://www.steves-internet-guide.com/mqtt/


  1. and if you really want to parse your device data using mosquitto_sub try this:

    TTN_USER_ID=your userID
    TTN_USER_PWD=your user password
    TTN_MY_DEVICE=00000000hhhhhhhh #your TTN device ID
    mosquitto_sub -h staging.thethingsnetwork.org -u $TTN_USER_ID -P $TTN_USER_PWD -C 1 -t '+/devices/$TTN_MY_DEVICE/up' | jq '.payload' | tr -d '"' | base64 --decode | od -t x1

    # gets the JSON date from the TTN application
    # gets just the "payload" part
    # removed the quotes from around the payload
    # decodes it using base64
    # and lastly displays it in hexadecimal format

  2. Very informative blog... MQTT devices will enable one to use MQTT devices with only minimal configuration effort on the side of Home Assistant.