Mapping The Things Network with a LoPy (1) Introduction

Welcome to my pages. If you landed here there must be at least some interest in either The Things Network, mapping its coverage or the LoPy, Pycom’s LoRa enabled controller. If neither is the case you’re still invited to read on and have yourself carried away into the world of Internet of Things…

As a proud owner of a WiPy, Pycom’s first WiFi enabled MicroPython programmable micro controller and of which i briefly blogged a while ago, i could not resist to buy its ESP32 based successor: The LoPy. The LoPy even ads the LoRa protocol and Bluetooth. The device collected quite some dust until i got to know The Things Network: An open Internet of Things wide area network using LoRa technology.

Soon i owned a gateway  (based on a RAK831 and RPi Zero) and was curious what its range would be. This is where TTN Mapper comes in. The idea is simple: Driving around with a LoRa transmitter which sends out a message say, each 15 seconds, your signal might be picked up by one or more gateways. An Android App registered to The Things Network waits for messages being reported. Knowing your location (GPS enabled on the SmartPhone) the App can now report the distance(s) and RSSI value(s) for this particular location and the particular gateway(s).  The principle is shown by it’s developer JP Meijers in a video.

Another possibility is to have the mobile node send it’s GPS location on regular intervals enabling TTN Mapper to process the data without the need of a SmartPhone. This however does not provide the driver with the feedback (and joy!) that his 25mW 868Mhz signal has been picked up by a sometimes 10 km remote gateway. Using this together with the App is no problem. TTN Mapper will be happy if a packet is accompanied by two locations (one from the packet itself and one from the App). Still it would be great if there was some other way of getting feedback.

So… why not use LoRa for this?

Sparked by this discussion (many thanks to  user piair for starting it) i decided to give this idea a try. The result should be that the number of gateways, distance, largest distance and received packets is displayed on the tracker device without the need of a SmartPhone. It will look like this:


The end result should look like this

In short: The LoPy receives it’s locations (LON1, LAT1) from a GPS module over a serial connection. This location is encoded and transmitted over LoRa. When a gateway receives this packet it is forwarded to TTN Mapper. It will also be decoded and send to my Node-Red application at home. As this data also contains the metadata (with the locations of all gateways which received the packet) the largest distance can now be derived. The number of received packets is incremented, the number of unique gateways counted and replied back to the node over LoRaWAN (encoded in a 5 byte string). This data will be scheduled and fed back to the LoPy following the next received uplink packet. The LoPy on its turn can now display these statistics and as a bonus keep track of the current speed and largest LoRa distance as well.

In the next post i will further dive into the specifics of the code and configurations involved and the hurdles taken to get things running.

Posted in LoRa | Tagged , , , | 4 Comments

Using Node-Red to graph scale data

Having used Node-Red briefly about two years ago, i decided to load it again onto the Raspberry Pi which listens for my BS440 scale. Just to dig into Node-Red and see where it got.

Installing Node-Red is a breeze although it takes a coffee and a beer or two before it is installed. But it’s more than worth the wait! A newly added set of nodes is the “dashboard” which can be added through the palette manager once Node-Red is up and running.

My idea was that i could graph the BS440 scale data which is stored into a CSV file (if the CSV plugin is enabled). I was shocked to find out i had made a fully operational webpage showing the data within one hour of coding (or drawing to be more precise)! And that included getting  familiar with Node-Red.

The result is a fancy looking page which shows nicely on my SmartPhone and is lightning fast. It offers a drop down selection for the type of data and two buttons to select the user. Ain’t that cool!

So how is this done? Without giving a lecture on Node-Red (i’ll leave it up to you to explore this piece of magic) this is what the flow looks like:


The core is reading the CSV file. The filename is set by selecting a user. Once the file is read it is converted to a node-js object. Type selects which data we want to graph. The selected type of data is read from the object and fed into the graph node.

The selecting thing might need some explanation as it uses JSONata (just watch the video). It is included in Node-Red and enables you to make a query from a bunch of JSON formatted data in just a single statement. As an example. If you want to craft an array of “vet” values from all data and change the unix timestamp to show milliseconds on the fly the JSONata for that is just:

[ { “key”: “vet”, “values”: msg.payload.[tijd*1000, vet] } ]

and the node “Select vet”  then looks like (note the J: selection)


The rest of the nodes just show the dropdown list and store and show the current user.

The project is made available on my Github

On to the next project. Lets have fun!

Posted in BLE | Tagged , , , , , , , | 4 Comments

Connecting the Medisana BS440 Bluetooth scale (epilogue)

Having used the BS440 bathroom scale script for over a year i got time to update the whole thing. I decided to do a full reinstall and log all the individual steps it takes to get things running. So here is the recipe.


  • Raspberry Pi B+ (note: i have no Pi3 so no experience in using the built in BLE adapter)
  • USB BLE adapter (I use brand ADJ model 100-00006 BT 4.0 adapter which uses driver broadcom BCM20702A0)
  • WiFI adapter (unless you can position a wired RPi within 1 m from the scale) like this one


  1. Download latest Raspbian Jessie Lite build for your RPi from here.
  2. Format and image the SD card. Instructions here
  3. Enable SSH access by placing a file named ‘ssh’, without any extension, onto the boot partition of the SD card.
  4. Use a temporarely wired connection to connect to the RPi using Putty
  5. Change pwd and run sudo raspi-config and do
    expand filesystem
    set timezone
    set hostname like bs440rpi (or any other nice name)
  6. run sudo apt-get update and sudo apt-get dist-upgrade
  7. Enable WiFi using this tutorial.
  8. Follow Tony DiCola tutorial to set up Bluez on the RPi. I selected the latest and greatest Bluez 5.44. Follow this tutorial step by step. No need to enable “experimental”. Building Bluez will take half an hour or so…
  9. Copy btmgmt to the local bin folder with:  sudo cp /home/pi/bluez-5.44/tools/btmgmt  /usr/local/bin so it can be used from anywhere.
  10. Enable BT low enegry with btmgmt le on
  11. run hcitool lescan  and step on the scale with a registered user selected so all your properties are measured and stored. The scale should report with its MAC address and name like F1:37:57:6C:DE:64 [0202B664DE6C5737F1] write down both.
  12. Test connectivity with sudo gatttool -t random -b F1:37:57:6C:DE:64 -I
  13. at the prompt type connect which should respond connected. Error 111 indicates that the adapter is not in LE mode (retry the btmgmt le on).
  14. Avoid having to use super user privileges (sudo) on hcitool by sudo setcap 'cap_net_raw,cap_net_admin+eip' 'which hcitool'
  15. Avoid having to use super user privileges (sudo) on btmgmt by sudo setcap 'cap_net_raw,cap_net_admin+eip' which btmgmt
  16. Install Pygatt 3.0.0 by sudo apt-get install python-pip and sudo pip install "pygatt[GATTTOOL]" This will throw a number of warnings and even errors but should end with Successfully installed pygatt pyserial enum34 pexpect ptyprocess
  17. Get the BS440 files by downloading the zip file from  and put them in a separate folder on the RPi like /home/pi/BS440.
  18. Enable the plugins of your choice by removing the underscore in the plugins folder for the .py and .ini files.
  19. Edit BS440.ini (fill in your scale’s MAC address and name from step 11) and all the enabled BS440<plugin>.ini in the plugins folder
  20. Start with  python Start a second shell and monitor the logfile with tail -f BS440.log
  21. Enjoy
Posted in BLE | 7 Comments

Connecting the Medisana BS440 Bluetooth scale (part 5)

Triggered by a comment on episode 1 of this blog i decided to use Domoticz to visualize the measurements in a graph. I use Domoticz to control my home since july 2013 and it has proven to be very reliable.

To graph the weight first a “virtual sensor” must be created. First find a unique index and paste the url below in your browser:

* did=60 represents the device ID number for this device 
  (could be any unique number). Make a note of it. 
  Upon creation the device will be given an unique index.
* dtype=93 refers to the BWR102 device, a Oregon Scientific connected scale we mimic.
* svalue=75.0 refers to the weight. Every call to this URL will update the value.

After creation of the virtual sensor, push the green button to have the sensor showing up on the “Other devices” tab:


Other Devices tab:


Now the sensors are ready the BS440 program can call the sensor URL with the appropriate Device number and weight (see In the .ini file the Domoticz connection parameters (IP-adres and username/password) can be entered.

Now every time a user steps on the scale the weight will be logged and graphed.


To have consistent results it is advised to do your weighings at the same time of day. In my case this will be between 07:00 and 08:00 in the morning (8:00 – 11:00 in the weekends). I use an IP controlled switch based on a Sonoff ESP8266 based 220V switch flashed with ESP Easy firmware. Now Domoticz can switch on/off the Raspberry through WiFi by calling the ip address of the Sonoff like

http://ip-address/control?cmd=gpio,12,1 (=on) 
http://ip-address/control?cmd=gpio,12,0 (=off)


Before switching off at 8:15 (11:15 in the weekeinds) be sure that a cron job did a shutdown earlier to not corrupt the flash card. Also when powering up, the “python” needs to be started automatically through and a @reboot cron. These are the cron definitions to accomplish this:

@reboot sh /home/pi/ > /home/pi/cronlog 2&1
0 8 * * 1,2,3,4,5 sudo shutdown -h +0
0 11 * * 0,6 sudo shutdown -h +0 

A 1 minute sleep is inserted in to have a stable booted Raspberry before starting the bluetooth communications in background.

cd /
cd home/pi
sleep 1m
sudo python &
cd /
Posted in FF32 | Tagged , , , , , , , | 49 Comments

Connecting the Medisana BS440 Bluetooth scale (part 4)

To start with: i am not a professional programmer, never was, never will be. Started off with some assembly on a 8 bit (yes!) SC/MP processor in my early days and ended with Turbo Pascal on MS-DOS6.2 as highest level of sophistication. Maybe that’s why i enjoy poking around on the credit-card size Raspberry in Python. The wealth of internet resources surely helps. This journey learned me about C-structs, callback routines, running shell commands, logging, parsing ini files, exception handling, pep8 and of course a little bluetooth. The complete code can be found  on my Github pages

The sequence of commands we want to issue through gatttool after being connected to the scale are basically these (when issued from the shell, pygatt will use the interactive mode of gatttool):

gatttool -t random -b F1:37:57:XX:XX:XX --char-write --handle 0x001f -n 0200
gatttool -t random -b F1:37:57:XX:XX:XX --char-write --handle 0x001c -n 0200
gatttool -t random -b F1:37:57:XX:XX:XX --char-write --handle 0x0026 -n 0200
gatttool -t random -b F1:37:57:XX:XX:XX --char-write-req --handle 0x0023 -n 0000000000 --listen

But the problem is we need to connect first and connecting is only possible for about one minute after the person has stepped onto the scale and we don’t know when that will be. The first attempt was just endless keep on trying to connect with 5 second timeouts. This is what it looked like:

adapter = pygatt.backends.GATTToolBackend()
# wait for someone to step on the scale
while True:  
        device = adapter.connect('f1:37:57:xx:xx:xx', 5, 'random')
    except pygatt.exceptions.NotConnectedError:

after which the program would continue with regestering to the indications and reading data… However every connection attempt/timeout would take more and more time every go, eventually slowing down to once a minute. Also CPU utilization would rise to 100% for some obscure reason.

The current approach uses the “hcitool lescan” command to scan for active bluetooth devices (filtered on our device of interest) before attempting to connect. This works better although the bluetooth driver needs a reset after every “hcitool lescan”/timeout by issuing the pygatt “adapter.reset()” command. Probably because “hcitool lescan” is not nicely terminated. There is some discussion about this behaviour of Pygatt at the Pygatt repo of Peplin. For now i just reset the interface after every timeout. Not nice but it works!

Awaiting the code to put data in a database and adding a nice web interface to it, last three measurements will be sent to the user by e-mail. The first step in tracking your weight!


21-03 07:4723-03 07:2323-03 23:30

Gewicht (kg) Vet (%) Spieren (%) Botten (kg) Water (%) Verbruik (kCal)

Feel free to ask for more clarification and meanwhile: enjoy coding!

Posted in BLE | Tagged , , , , , , , | 2 Comments

Connecting the Medisana BS440 Bluetooth scale (part 3)

Time to dive into the (bluetooth) specifics of the BS440 scale. When starting this project i was lucky to bump into this post on StackOverflow where user Edmundo was actually trying to acomplish the same. He snooped the bluetooth traffic on his Android phone and thus discovered the commands to send to the scale to make it send the data we need.

Data is presented to us by use of Indications. Indications are messages conveying the data for certain characteristics the device supports. Characteristics have a long (16 byte globally unique UUID) identifier but also a two-byte shortcut for it: the handle. Writing the command 0200 to a handle tells the device you register to the indications of this characteristic. Writing 0000 will stop it. A data packet of this characteristic will report with a handle 1 less than the handle of the characteristic. The BS440 has 3 kinds of interesting characteristics:

Handle UUID                                  Data
0x26   00008a82-0000-1000-8000-00805f9b34fb  person, gender, age, size, activity
0x1c   00008a21-0000-1000-8000-00805f9b34fb  weight, time, person
0x1f   00008a22-0000-1000-8000-00805f9b34fb  time, person, kcal, fat, tbw, muscle, bone

Some notes: person ranges from 1~8 and corresponds with the user selected before stepping on the scale. Or: being a “smart scale” : if the weight roughly corresponds to a known user weight that user is automatically selected. And when multiple users with the same weight are found it even lets you select the appropriate user.

So what does the data look like?

The last 30 measurements per person will be stored in the scale and upon communication the history for this user will be dumped. So you will receive values like this

handle=0x25, value=0x845302800134b6e0000000000000000000000000
handle=0x1b, value=0x1d8c1e00fe6e0aa056451100ff020900000000
handle=0x1e, value=0x6f6e0aa05602440ab8f07ff26bf11ef0000000
handle=0x1b, value=0x1d961e00fe72fe9f56291100ff020900000000
handle=0x1e, value=0x6f72fe9f5602460ab8f07ff26bf11ef0000000
handle=0x1b, value=0x1d961e00fe2dfe9f569e1000ff020900000000
handle=0x1e, value=0x6f2dfe9f5602460ab8f080f26cf11ef0000000
handle=0x1b, value=0x1d8c1e00feb3fd9f56c21000ff020900000000
handle=0x1e, value=0x6fb3fd9f5602440ab8f080f26cf11ef0000000
handle=0x1b, value=0x1d8c1e00fe39fc9f56641000ff020900000000
handle=0x1e, value=0x6f39fc9f5602440ab8f080f26cf11ef0000000
handle=0x1b, value=0x1d8c1e00fe02fb9f56991000ff020900000000
handle=0x1e, value=0x6f02fb9f5602440ab8f080f26cf11ef0000000
handle=0x1b, value=0x1d8c1e00fe80fa9f561d1000ff020900000000
handle=0x1e, value=0x6f80fa9f5602440ab7f080f26cf11ef0000000
handle=0x1b, value=0x1d8c1e00fef7f89f56021100ff020900000000
handle=0x1e, value=0x6ff7f89f5602440ab8f080f26bf11ef0000000
handle=0x1b, value=0x1d8c1e00fe10f39f566a1000ff020900000000
handle=0x1e, value=0x6f10f39f5602440ab8f080f26cf11ef0000000

Nice, but what does it actually mean?

Now the puzzle starts where to find the data we need, but luckily Edmundo helps us out again:

All multibyte values represented in unsigned int, little endian (multi byte values 
start with least significant byte).
First byte of string is byte 0.
0x1b:                              value                     
fixed: byte: 0                     [0x1d]
weight: byte: 1 & 2                [kg*100]
timestamp: byte 5-8                Unix timestamp
person: byte 13                    [1..8]

0x1e:                              value                    
fixed: byte 0                      [0x6f]
timestamp: byte 1-4                Unix timestamp
person: byte 5                     [1..8]
kcal: byte 6 & 7                   first nibble = 0xf, [kcal]
fat: byte 8 & 9                    first nibble = 0xf, [fat*10]
tbw: byte 10 & 11                  first nibble = 0xf, [tbw*10] 
muscle: byte 12 & 13               first nibble = 0xf, [muscle*10]
bone: byte 14 & 15                 first nibble = 0xf, [bone*10]

0x25:                               value                   
fixed: byte 0                       [0x84]
person: byte 2                      [1..8]
gender: byte 4 (1=male, 2=female)   [1|2] 
age: byte 5                         [year]
size: byte 6                        [cm]
activity: byte 8 (0=normal, 3=high) [0|3]

In Python the datastrings can be interpreted simply using struct.unpack using the pattern ‘BxBxBBBxB’ for the 0x25 data, ‘<BHxxIxxxxB’ for the 0x1b data and ‘<BIBHHHHH’ for the 0x1e data. In my program takes care of the decoding job.

These insights should get us going. Just one more thing. When does the scale start sending data? Data transfer starts after sending 0x02 followed by the unix timestamp to handle 0x23. This is also the way the scale resyncs it’s RTC. Note that this last write must be a write with response (i.e. “write-req” instead of a “write-cmd”).

By now you might start to understand why this project dragged over 3 months to complete. Part 4 will discuss the Python code.


Posted in BLE | Tagged , , , , , , , | 2 Comments

Connecting the Medisana BS440 Bluetooth scale (part 2)

It’s been a while after part one but not because of me abandoning the subject. On the contrary: i must have been stepping on an off that darn scale like a million times trying to debug my way around the BLE mysteries.

Continuing the previous post we now should first install Pygatt. It installs through the pip installer which needs to be installed first.

sudo apt-get install python-pip

after which pygatt and all dependencies may be installed

sudo pip install 'pygatt[GATTTOOL]'

Apart from throwing all kinds of errors and warnings it ends with “Successfully installed…”. To be able to use Pygatt for my purpose i needed to fix another few things.

Fix 1: Support for receiving multibyte indications

Receiving data from the scale is performed by subscribing to an indication. The indications contain 40 hex characters and the installed Pygatt (as of march 2016) does not support that. Ilya Sukhanov fixed that in function “_handle_notification_string” but his changes have not been merged (yet).

But Ilya made some more improvements so just grab his entirely and replace the one found at /usr/local/lib/python2.7/dist-packages/pygatt/backends/gatttool

EDIT: As of 6/9/16 the above changes have been merged into Peplin’s master

Fix 2: Restarting bluetooth

At adapter.start() Pygatt will first try to restart the bluetooth service by “sudo systemctl restart bluetooth” however on debian Jessie this should be “sudo systemctl restart“. This is changed manually at the bottom of by changing “subprocess.Popen([sudo, systemctl, restart, bluetooth]).wait()” into “subprocess.Popen([sudo, systemctl, restart,]).wait()”

Fix 3: Enable le (low energy) mode

Using the gatttool in interactive mode, like (replace address with yours):

gatttool -i hci0 -b 20:C3:8F:E9:56:F1 -I

would respond with the “[20:C3:8F:E9:56:F1][LE]>” prompt. But the connect command would fail:

[20:C3:8F:E9:56:F1][LE]> connect
Attempting to connect to 20:C3:8F:E9:56:F1
Error: connect: Connection refused (111)

In my case this could be solved by

sudo btmgmt le on

So for this to use from Python it is usefull to copy the “btmgmt” command to the /usr/local/bin directory too.

To be frank: Even after fiddling around with the Bluetooth utils it is still not quite clear to me what tool to choose from: hcitool, hciconfig, btmgmt, gatttool…

Anyway: With this installation you now should be able to run the python code i wrote which can be found at my Github pages.

Next post will zoom in the Python code and de BS440 connected scale specifics.


Posted in BLE | Tagged , , , , , , , | 8 Comments