Here at Conserv we are always looking for the best way to get a job done, and trying to find ways we can focus on where we bring value while externalizing those things that aren’t core to our business. Part of that focus for me means finding the right network partners. Our customers rely on our solution, and we rely on our LoRaWAN network providers to help us deliver. I’ve written a lot before about The Things Network (which I love, open source FTW), but as a part of building the best solution for our customers we are always looking at alternatives. Recently, I wanted to give Loriot a try. They have some great analytics built into the platform along with many other features that are attractive to a company building out a managed solution on top of LoRaWAN.
One of the things on their roadmap that I find particularly exciting is the ability to move a device between applications. For us, this means we can configure and test devices on our internal application, and then simply move them to a customer application when we ship the devices out. That’s a much better experience than having to delete and recreate the device, even if we build out our own tools on top of the API to do so. I was also pleasantly surprised by Loriot’s detailed analytics around message counts, radio performance, and all of the other things we need to make sure our solution is reliable for our customers.
Another Loriot feature that got me excited was a better packet forwarder. The legacy forwarder that a lot of gateways use by default is UDP, which means not much in the way of guaranteed delivery. It’s also lacking management features, but from what I understand that forwarder was intended as a starting point, not as the end. Loriot brings a lot to the table here with a more reliable forwarder that also includes some useful management tools. On the downside, the Loriot forwarder does not support the channel plan that we currently use, which means one of two things has to happen. We could reconfigure our devices in the field to use a supported plan (nope), or we can fall back to the Semtech forwarder for now. We’ve been told support for US915 channel plan 2 is on the roadmap for the Loriot forwarder, but as of today it isn’t there. I did test the Loriot forwarder with US915 channel plan 1 and it performed as advertised. Always nice when that happens.
Unfortunately a hiccup occurred when we tried to provision our Tektelic gateways on the Loriot network using the Semtech forwarder. Loriot does not allow a user to set a gateway EUI. Instead, they rely on an IEEE recommended formula that derives the EUI from the MAC address of the gateway. In short, this means taking the MAC address of the gateway and inserting FFFF right in the middle. Easy enough, or so we thought. See the Loriot docs for details.
Tektelic gateways use a very similar pattern for converting the gateway MAC address to an EUI, but instead of inserting FFFF into the address, they insert FFFE. This didn’t seem like a problem at first, since the gateway EUI is exposed in the config.json file used to configure the gateway’s channel plan, key, server address, ports, etc. We changed it in config.json, and restarted the packet forwarder. No good, nothing worked! Remember that Loriot doesn’t allow us to set a gateway EUI, so that leaves the gateway as the only place to change it.
We looked at the logs and discovered the first big thing we learned:
The gateway_ID field in config.json on a Tektelic gateway is NOT the one that is used when the packet forwarder starts.
When the packet forwarder starts, it loads data from two locations: a commissioning DB and config.json. The DB is loaded first. This is a sqlite DB that has a configuration table, containing a single row of data. This row in turn contains fields that can be used to set the EUI, along with other things like a customer provided inventory name / number, and some info that Tektelic populates about the device. When the gateway is running, you can find this db at /tmp/commissioning.db. We hit the DB with sqlite, made the appropriate change to the Customer_Gateway_ID field and restarted the packet forwarder. Still nothing!! What the heck is going on! This led us to the second big thing we learned:
This database is NOT persistent (as its location in /tmp probably indicated). This database is copied to /tmp from a separate flash storage when the packet forwarder starts up. Any changes you make to the db in /tmp will be overwritten.
Here’s the relevant snip from the log file:
Nov 7 17:46:08 kona-micro pkt_forwarder: lddFlashProbe: Could not find commissioning partition in the partition table
Nov 7 17:46:08 kona-micro pkt_forwarder: lddFlashProbe: Proceeding to try raw SPI access method
Nov 7 17:46:08 kona-micro pkt_forwarder: lddFlashProbe: Spansion S25FL127S flash detected: sector size 64 KiB; total 16384 KiB
Nov 7 17:46:08 kona-micro pkt_forwarder: cxLoadDatabase: loaded 6144 byte database to /tmp/commissioning.db
Nov 7 17:46:08 kona-micro pkt_forwarder: CxInit: database integrity verified
OK, so now that we know that, how can we proceed? This may be configurable using Tektelic’s Windows configuration tools, but we can’t really use that. We want to automate the provisioning of all the things! This means we write a lot of Python scripts to grab various bits of data and configure hardware, network services, and our application. How can we set up persistent values in this database, using only the command line on the gateway? Turns out that Tektelic ships a flash write utility with the gateway. Yay!! You can find it here:
This tool lets you write a new commissioning DB to the flash storage. When the packet forwarder restarts, it will pull that new DB from flash, put it in /tmp, and load its settings from there.
Finally, the steps for changing the EUI used by the packet forwarder:
- SSH to the gateway, and make sure the packet forwarder is started. If it isn’t, run /etc/init.d/pkt_fwd start.
- Look in /tmp for commissioning.db
- Make a copy of the commissioning.db file, I put it in /home/root.
- Use the sqlite client to update the configuration table in your copy of the database. Set Customer_Gateway_ID to whatever you need it to be.
- Run /opt/kona-micro-indoor-pkt-forwarder/utils/commissioning_update -f <your new commissioning db file> to write the modified commissioning DB to flash.
- Restart the packet forwarder, and check the logs to make sure your new EUI is picked up.
And that’s it! Once we figured out how to change the gateway EUI, Loriot picked it right up and data started flowing. With that problem solved, we can move on to some real testing.
Note that this might not be the best way to do this. I see some mention in the log files of the forwarder looking for a commissioning partition. It may be the case that this partition is the right place to put a modified database, but I can’t find anything in the docs indicating how that works. If I can figure it out, I’ll amend this article with the new option.