Thread border router technical deep-dive (part 1)

This blog post series documents some deeper technical details on how Thread works, in particular border routers. Most of my insights are from real world findings while working on the OpenThread Border Router add-on for Home Assistant.

“Thread is an IPv6-based, low-power mesh networking technology for Internet of things (IoT) products.” according to the Wikipedia article of the Thread protocol, and that really sums it up nicely. Thread is based on the same physical layer as Zigbee: IEEE 802.15.4. Unlike Zigbee, Thread provides IP connectivity. It does not specify the application level protocol, e.g. how to actually control a light bulb. This is where other protocols come in, like Matter. Similar to Zigbee, some Thread devices act as routers within the mesh. However, there is no single predefined coordinator like in Zigbee networks. There is a single Thread leader in every Thread partition which manages the Thread routers. The Thread leader is selected automatically. And there can be multiple border routers which allow communication with Thread devices. This redundancy avoids a single point of failure and can also improve connectivity with the mesh. In my opinion, the use of IP, and the fact that multiple border routers can provide connectivity to the main network are the biggest advantages of Thread and sets it really apart from existing mesh networks.

This article is mostly focusing on this Thread border routers. To learn more about Thread in general, the excellent Thread Primer on openthread.io is a very good resource and is probably a good read in case you are not that familiar with Thread yet. This first part focuses on describing what type of IPv6 network prefixes are in use in a Thread network.

This post uses ot-ctl commands which can be executed on a OpenThread border router. On Home Assistant this command is only available within the Home Assistant OpenThread Border Router add-on container. The Advanced SSH & Web Terminal community add-on with protection mode disabled or the Home Assistant OS host SSH access on port 22222 can be used to get a shell inside the container. In both cases Docker exec gives a shell inside the add-on container:

docker exec -it addon_core_openthread_border_router /bin/bash

A Thread network is defined by a operational dataset. A operational dataset consists of network name, network key and extended PAN ID and other parts which define the basic parameters required for the Thread network to function. The operational dataset is shared with nodes when they join the network. It becomes the active operational dataset. Operational dataset can be updated: A border router can define a pending operational dataset, which then gets shared among the Thread devices and becomes active after a predefined delay. Operating datasets are often exchanged as a hexadecimal string in a simple TLV format. E.g. the Home Assistant Thread configuration page shows the dataset as a TLV in the information overflow when connected to a local OTBR. This gist contains a rudimentary parser for the TLV format written in Python.

Thread foremost uses IPv6 to address devices within the Thread mesh network. Thread uses parts of the IPv6 address themselves to reflect how the mesh network is connected, in particular the RLOC16 (routing locator) is the last 16-bits of the RLOC address. The IPv6 Addresses chapter of the openthread.io primer has details on how the IPv6 addresses are formed. The OpenThread web interface shows the RLOC16 in its topology view (see the add-on documentation on how to enable the web interface).

When a Thread network is formed, a random ULA (unique local address) network prefix is generated and assigned to the network. This is the so-called “mesh local prefix”. These addresses are not meant to be routed and are used within the Thread network for mesh control traffic only. The mesh local prefix is part of the operational dataset of a Thread network. In my

Show mesh-local prefix currently used, in my case fd15:3b16:79a7:541a::/64.

# ot-ctl prefix meshlocal
fd15:3b16:79a7:541a::/64
Done

Show active operational dataset (containing mesh-local prefix)

# ot-ctl dataset active
Active Timestamp: 1
Channel: 15
Channel Mask: 0x07fff800
Ext PAN ID: 21e7cfcf06499303
Mesh Local Prefix: fd15:3b16:79a7:541a::/64
Network Key: d8f4e92d6be9a38f829fd9729a0528d0
Network Name: ha-thread-4646
PAN ID: 0x4646
PSKc: 39d90ef02a8f09d998e0b9f881a39583
Security Policy: 672 onrc 0
Done

(I am sorry the network key won’t be working, in case you’re a Thread wardriver happen to be nearby 😉)

To talk to Thread devices from outside the Thread network another IPv6 network prefix is used, the “off-mesh routable prefix” (or OMR prefix). A OMR prefix is selected by each Thread border router. Our OTBR use a ULA prefix, a prefix which isn’t routeable in the wider Internet. It seems that Apple border routers currently use a global routable IPv6 prefix if your ISP and router delegates such prefixes. Only one OMR prefix is active at any one time. To see the OMR prefix used and announced by border router the following command can be used:

# ot-ctl br omrprefix
Local: fd56:3488:b35c:1::/64
Favored: fd3c:2447:fa0a:1::/64 prf:low

In this case the border router has a local OMR prefix is not the currently favored/active one. OMR prefixes are distributed in the Thread network through network data. The Thread leader manages and distributes the network data. The network data show the currently active OMR prefix as well:

# ot-ctl netdata show
Prefixes:
fd3c:2447:fa0a:1::/64 paros low a000
Routes:
::/0 s med a000
::/0 s med 2400
Services:
44970 5d fd153b1679a7541a635bdefb4a8a2ab6d12a s a000
44970 5d fd153b1679a7541a734bdd9cef182782d125 s 2400
44970 01 5b000500000e10 s a000
Contexts:
fdf8:76ab:ebd6:1::/64 1 -
fd20:14db:345c:1::/64 2 -
fd3c:2447:fa0a:1::/64 3 c
fd56:3488:b35c:1::/64 4 -
fdcd:650d:f56e:1::/64 5 -
fd0b:3a97:1e87:1::/64 6 -
Commissioning:
27402 - - -
Done

The last 4 digits in the prefix lines is the RLOC16 of the border router which published this OMR prefix. On that particular device the output looks as follow:

# ot-ctl rloc16
a000
Done
# ot-ctl br omrprefix
Local: fd3c:2447:fa0a:1::/64
Favored: fd3c:2447:fa0a:1::/64 prf:low
Done

More details on what else the network data contain will follow in future parts.

Border router announce the OMR prefix on the WiFi/Ethernet main network they are connected to (also called adjacent infrastructure link, AIL). The announcements are sent using the standard IPv6 Neighbor Discovery Protocol (NDP, or often ndisc) via RFC 4191 Route Information Option (RIO). IPv6 NDP packets are ICMPv6 packets. Any device in the WiFi/Ethernet main network which wants to communicate with devices in the Thread mesh network needs to process this IPv6 NDP packets to learn about the prefix and add it to it’s local routing table. Linux by default doesn’t process such packets, presumably for security reasons. If you can’t find the OMR prefix of your Thread network on a particular device, missing Route Information Option support is usually the problem. To enable processing on a particular interface use the following commands:

$ sudo sysctl -w net.ipv6.conf.wlan0.accept_ra=1
$ sudo sysctl -w net.ipv6.conf.wlan0.accept_ra_rt_info_max_plen=64

Note that network managers such as NetworkManager or systemd-networkd have their IPv6 NDP processing logic implemented in user space. YMMV.

Note that the OMR prefix might change over the lifetime of a Thread network. This means devices will change their IPv6 address! This typically happens when the border router which owns a particular OMR prefix goes offline. In that case, the OMR prefix will timeout, and the Thread leader will select a new OMR prefix. Devices will then assign themselfs an address using this new OMR prefix, and border routers will announce RIO on the main network.

Lastly, a Thread border router also generates an IPv6 ULA network prefix for the WiFi/Ethernet main network named “on-link prefix”. The “on-link prefix” is announced using RFC 4861 Prefix Information Option. This is only used when there is no existing IPv6 prefix. If you have no existing IPv6 addressing in your main network devices should get an address with this prefix. These addresses then can be used as a source address when communicating with Thread devices.

# ot-ctl br onlinkprefix
Local: fd2f:7fe0:dc78:8e15::/64
Favored: 2a02:3342:3f21:10::/64
Done

This architecture allows communication with Thread devices with just a Thread border router without any requirements on the router on the main network currently in place. The regular Wi-Fi/Ethernet router which typically routes to the Internet does not need to be configured. Technically, that router doesn’t even need IPv6 support!

However, devices on the Wi-Fi/Ethernet network which want to talk to Thread devices (through the Thread border router) need IPv6 support including, as previously mentioned, support for the IPv6 NDP (specifically RFC 4191 Route Information) to find an IPv6 route to the Thread devices.