There are a million ways to get data from A to B and back again, but doing so reliably isn’t always a cakewalk. IoT devices and applications – also known as “Things” – require a reliable, robust and secure messaging protocol. That’s where the messaging protocol, MQTT comes in.
This article will give you the know-how you need to understand and get started using MQTT and MQTT-SN in an IoT (Internet of Things) environment.
Subjects covered in this article:
What is MQTT?
In the most simple terms, MQTT is a messaging protocol that was designed to create a reliable standard for machine to machine (m2m) communication. IBM’s Andy Stanford Clark and Eurotech’s Arlen Nipper created the protocol in 1999.
MQTT is a publish-and-subscribe protocol, meaning that instead of communicating with a server, client devices and applications publish and subscribe to topics handled by a broker.
MQTT typically uses IP (Internet Protocol) as its transport but can also use other bi-directional transports.
What is MQTT-SN?
MQTT-SN (MQTT for sensor networks) is a variant of MQTT that has been optimised for use in low power environments such as sensor networks, as the name suggests.
On top of the standard feature set, SN adds extra functionality for use cases where lower power is required. These extra features include:
- QoS mode -1: allows for fire-and-forget messaging
- Topic aliases: allow for simplified publishing and reduced data overheads
- Sleep mode (disconnected sessions): allows messages to be queued on the broker while the remote Thing or device is powered off
Get started with Thingstream todayRegister
Why MQTT is perfect for IoT
When you boil it down, the Internet of Things (IoT) has one job: to get data to and from devices on a network. The problem is that these networks could be anywhere in the world and face a million and one conditions that can and will cause them to fail. MQTT, and by extension SN, have myriad features built in to help mitigate some of these issues. Some of the main features are listed below:
1. MQTT is easy
Getting up and running with MQTT is quick and easy. There are millions of ready made client applications and almost as many brokers available. You can get started with MQTT right now by using the broker integral to the Thingstream platform.
2. Reliability of MQTT
Many IoT devices rely on a radio connection to transmit collected data, meaning connectivity isn’t always reliable. MQTT can allow for messages to be stored at the broker until a device is ready to receive it. Thanks to QoS, MQTT has the ability to queue messages, make sure they get where they are going and also ensure that it only gets there once.
3. Bidirectional messaging
It’s perhaps more accurate to say that MQTT is omnidirectional. Any device, Thing or application can publish or subscribe to any topic handled by the broker. This means there is no limit on what can talk or listen to what across the network.
4. Messaging at scale
With MQTT, it is just as easy to broadcast a message to a million devices as it is to send to just a hundred. To be heard by everything on the network, a Thing simply needs to publish to a topic to which all devices are subscribed.
The advantages of MQTT
MQTT client and broker
Don’t think, “client and server”, think, “client and broker” instead. In a traditional client/server relationship, the client and the server connect and the server is treated as a storage and distribution depot for the data. With MQTT, the process is different. The broker is much more passive, acting more like a signpost for where the data should go.
Any Thing (from a microcontroller to a massive server) that runs an MQTT library and connects to a broker over a network can effectively become an MQTT client.
Clients don’t send messages directly to and from each other but instead communicate to topics managed by the broker. These topics work a little like email inboxes. Messages are published by Things to topics; messages are then picked up when a Thing subscribes to those topics.
The broker handles authentication of Things on the network as well as managing connections, sessions and subscriptions. Its main responsibility is to receive all published messages and then send them subscribed clients. The broker also queues messages for subscribed clients, delivering them according to the agreed QoS level.
MQTT messages aren’t delivered directly from Thing to Thing. Instead, they are published to topics. The broker then delivers those messages to any subscribed clients.
Anatomy of an MQTT topic
- Topics are made up of one or more topic levels, separated by a forward slash:
- Topics are case sensitive
- Topics don’t have to be pre-registered at the broker.
MQTT-SN topic features
MQTT SN adds some special topic features to assist in a bandwidth constrained
- If using predefined topics, you can create a topic alias on the broker which the client can then use without the need to register first. This can reduce the number of billable messages.
- Long topic name does not have to be sent over the air in either direction saving on bandwidth, and the need to store long topic name in memory.
- If not using predefined topics, Things use the REGISTER command to register a topic name with the server. The server will respond with a REGACK containing a topic ID consisting of 2 characters. Note that you would need to register the topic each time the client connects.
Why use MQTT topics?
Topics are a great way to organise data as it flows through the network and this becomes more apparent with scale. For example, if you have devices with several sensors deployed across multiple sites, you could put all of the data in one payload and parse it when it gets to its destination or you could do it the MQTT way and use topics to divide up the data, as shown below:
When the data transmitted is divided by topic, Things can then subscribe to the topics they are interested in. If a device is interested in multiple topics, they can be subscribed to individually or wildcards can be used. For example, to get all the data from site1, you would use the wildcard, “site1/#”. You could also get all “temp” data from all sites by using the wildcard, “+/temp”.
Connections are always made between a client and a broker. Clients cannot connect directly to each other.
The connection is established by the client sending a CONNECT message, to which the broker responds with a CONNACK (connection acknowledged). A connection is required in order to publish with a Quality of Service (QoS) level 0, 1 or 2 or to subscribe to any topics. Clients usually connect to a broker using a Client ID (ClientID), username and password. A client may only have one connection with the same broker at any given time.
The “clean session” setting gives you the ability to start fresh with no messages in the queue.
- false: The broker stores all subscriptions for the client and will queue any messages for that subscription that were published with a QoS level 1 or 2
- true: The broker purges all queued messages when the client connects.
The “keep alive” setting defines the longest period of time that the connection can remain in place without the broker or client sending a message. This allows battery-operated to go into a sleeping state during which all messages destined to them are buffered at the server/gateway and delivered later to them when they wake up.
To keep the connection in place, the client sends a PINGREQ before the keep alive window expires, to which the broker responds with a PINGRESP. When the device has not been seen within the keep alive window, the broker can choose to disconnect the session, and will then stop sending messages destined to the device until the next session.
Sleep (MQTT-SN only)
An MQTT-SN client can tell the broker it is going to sleep for a period of time by sending a DISCONNECT with a DURATION greater than 0. The broker will maintain this state for the period specified. The broker will then queue all subscribed messages (Qos 0, QoS 1 or QoS 2) in sleep state. This differs from a being disconnected with “clean session” set to false, which only retains QoS 1 & QoS 2 messages.
Whilst asleep, the client can flush the queue without needing to wake up by issuing a PINGREQ. The broker will then respond with a PUBLISH if there are messages to deliver, and a PINGRESP when the flush is complete, putting the device back to sleep.
MQTT clients don’t connect directly to each other, instead they subscribe to topics to receive messages.
To subscribe to a topic, the client must first send a SUBSCRIBE request to tje broker. The SUBSCRIBE request can include multiple topics. The broker responds to the SUBSCRIBE request with a SUBACK (subscription acknowledged) response. Subscriptions also have a QoS setting which can be used to downgrade the QoS of the published message. In this case, the message is always published at the lower QoS setting.
Subscription wild cards
Subscriptions can use one of two types of wild card shown below.
Single level (+)
A single level wild card replaces one topic level. As shown in the image below.
This wildcard would cover the following topics:
A multi-level wildcard replaces multiple topic levels:
This wildcard would cover the following topics:
MQTT QoS explained
Where the MQTT protocol is concerned, the guarantee of delivery is defined by QoS (Quality of Service).
In this article, you’ll learn how, where and when to use MQTT QoS and which levels are right for your own IoT applications.
MQTT QoS levels
MQTT and MQTT-SN support multiple levels of QoS for guaranteeing message delivery.
QoS -1 – fire and forget
QoS -1 is ideal for low-power non-critical applications where it doesn’t matter if every message gets to where it’s going. By not making a hard connection with the broker and receiving no acknowledgement, considerably less power is used to complete the transaction.
QoS -1 key features:
- Only available for devices using MQTT-SN
- Does not require an MQTT connection to be established
- No acknowledgement from the recipient
- Not retried by the sender
- Analogous to QoS 0 by the time it reaches the broker
When to use QoS -1:
- Ideal for power constrained Things to minimise time on air
- Minimise messaging cost
- OK if message delivery is not critical e.g. data sent frequently
QoS 0 – at most once
QoS 0 is used to ensure that a message reaches its destination no more than once. Unlike QoS -1, this method requires an MQTT connection meaning it is less efficient in terms of power.
QoS 0 key features:
- Best effort message delivery
- No acknowledgement from the recipient
- Not retried by the sender
- Not queued by the broker for disconnected clients with a valid subscription to the topic
When to use QoS 0:
- As QoS-1 except a CONNECT is necessary so not as efficient for power or message consumption
QoS 1 – at least once
QoS 1 is used when message delivery is critical. This is achieved by queueing messages until the subscriber is able to receive it.
QoS 1 key features:
- Guarantees that a message is delivered at least one time to the recipient.
- Sender stores the message until it receives a PUBACK from the recipient
- Messages may be sent or delivered multiple times.
When to use QoS 1:
- Use when you have to receive every message, but make sure you handle duplicates
- Use if you want messages to be queued on the broker for delivery to offline Clients
- Use when the overhead of QoS 2 is too high
QoS 2 – exactly once
QoS 2 is used when the message needs to arrive once and only once. This level is used when delivery essential.
QoS 2 key features:
- QoS 2 is the safest and slowest quality of service level
- Guarantees that each message is received only once by the intended recipients by using at least two request/response flows (a four-part handshake) between the sender and the receiver.
When to use QoS 2:
- Use if message delivery is critical and duplicate data is harmful to subscribers
In cases where there are many devices on the network, different levels of QoS might be needed. To achieve this, MQTT allows downgrading of QoS level at the subscriber node. The result of this is that the QoS for a message that is published does not have to be the same as the QoS for a message that is received.
QoS is defined by the original publisher, but when the broker then delivers the message on to subscribers, the lower of the PUBLISH QoS and the QoS defined during the SUBSCRIBE is used.
Security – is MQTT secure?
Making sure IoT devices are secure is just as important as making sure that they work and every link or node on the network is a potential exploit vector. While MQTT itself does contain some security mechanisms, it’s important to consider factors extraneous to the transport itself.
Network level security
The first place to consider security is the network itself. Ensuring that the network connection is secured by using a VPN tunnel will prevent exposure to network traffic.
Transport level security
As with HTTP traffic, MQTT traffic can be secured at the transport layer with TLS/SSL.
Application level security
Unique client identification and username/password credentials are provided by the protocol itself and should be considered the bare minimum security requirement.
Further security can be added by encrypting the payload itself at the application level.
Thingstream MQTT security
Thingstream adds an extra layer of security on top of those mentioned above by not exposing the IP address of the device. By having no physical connection to the Internet, this makes devices much harder to exploit.
Bonus video: how does MQTT work?
Please excuse the old branding, we made this video a while ago.