MQTT beginner’s guide

Get the know-how you need to understand and get started using the MQTT protocol and MQTT-SN in an IoT (Internet of Things) environment.

There are a million ways to get data from A to B and back again, but doing so reliably isn’t always a cakewalk. IoT devices and applications – also known as “Things” – require a reliable, robust, and secure messaging protocol. That’s where MQTT comes in.

Subjects covered in this article:

• What is MQTT?
• What is MQTT-SN?
• Why MQTT is perfect for IoT
• Advantages of MQTT
• MQTT client and broker
• MQTT devices and applications
• Instant IoT
• MQTT Anywhere
• MQTT topics
• MQTT connections
• MQTT subscriptions
• MQTT QoS
• MQTT security
• MQTT v5.0
• u-blox MQTT IoT services
• Extra resources

What is MQTT? MQTT explained

• MQTT (message queuing telemetry transport) is a messaging protocol that was designed to create a reliable standard for machine-to-machine (m2m) communication.
• MQTT is a publish-and-subscribe protocol, meaning that instead of communicating with a server, client devices and applications publish and subscribe to topics handled by a broker.
• MQTT typically uses TCP/IP (Transmission Control Protocol/Internet Protocol) as its transport but can also use other bi-directional transports.
• MQTT has become the de facto standard for loT communication because of its efficiency and flexibility. u-blox uses this to overlay various radio networks (2G-4G cellular and LoRa) and protocols (USSD, UDP) providing developers with a familiar and simple experience.
• MQTT allows devices and systems (clients) to communicate by sending messages. Messages are not sent directly from client to client but are published by a client to a topic* stored in an MQTT broker.
• Clients receive messages by subscribing to one or more topics and will receive messages from that point forwards.
• Topics are like street addresses - they form a tree which becomes more specific the further down you travel.
• Messages can be published with different Quality of Service levels which define reliability and whether receipts are generated for delivery.
• MQTT was created by Andy Stanford Clark and Arlen Nipper in 1999.

Useful links:

• More information about MQTT from OASIS
• The MQTT Technical Committee

MQTT specification

MQTT is an OASIS standard. The specification is managed by the OASIS MQTT Technical Committee. For more information on the standard and specification, please see the links below:

• MQTT version 3.11 specification (.pdf)
• MQTT version 5.0 specification (.pdf)

 

What is MQTT-SN?

MQTT-SN (MQTT for sensor networks) is a variant of that has been optimized for use in low power environments such as sensor networks, as the name suggests.

On top of the standard feature set, SN adds extra functionality for use cases where lower power is required. These extra features include:

• QoS mode -1: allows for fire-and-forget messaging
• Topic aliases: allows for simplified publishing and reduced data overheads
• Sleep mode (disconnected sessions): allows messages to be queued on the broker while the remote Thing or device is powered off

Why MQTT is perfect for IoT

When you boil it down, the Internet of Things (IoT) has one job: to get data to and from devices on a network. The problem is that these networks could be anywhere in the world and each faces many conditions that can and will cause them to fail. MQTT, and by extension MQTT-SN, has myriad features built in to help mitigate some of these issues. Some of the main features are listed below:

1. It's easy

Getting up and running with MQTT is quick and easy. There are millions of ready-made client applications and almost as many brokers available. You can get started with MQTT right now by using the broker integral to the u-blox Thingstream platform.

2. It's reliable

Many IoT devices rely on a radio connection to transmit and collected data, meaning connectivity isn’t always reliable. MQTT can allow for messages to be stored at the broker until a device is ready to receive it. Thanks to QoS (Quality of Service), MQTT has the ability to queue messages, make sure they get where they are going and if required, ensure that they only get there once.

3. Bidirectional messaging

It is perhaps more accurate to say that MQTT is omnidirectional. Any device, Thing, or application can publish or subscribe to any topic handled by the broker. This means there is no limit on what can talk or listen to what across the network.

4. Messaging at scale

it is just as easy to broadcast a message to a million devices as it is to send to just a hundred. To be heard by everything on the network, a Thing simply needs to publish to a topic to which all devices are subscribed.

What are the advantages of MQTT?

• Simplified communication
  
  Communication is a complex problem. MQTT reduces complexity, allowing a single connection to a message topic. Data is logically structured and can be processed flexibly.
• Eliminate polling
  
  Instantaneous, push-based delivery, eliminates the need for message consumers to periodically check or "poll" for new information. This drastically reduces network traffic.
• Dynamic targeting
  
  MQTT makes discovery of services easier and less error prone. Instead of maintaining a roster of peers that an application can send messages to, a publisher will simply post a message to a topic.
• Decouple and scale
  
  MQTT also makes solutions more flexible and enables scale. It allows changes in communication patterns, adding or changing functionality without sending ripple effects across the system.

MQTT client and broker

Don’t think, “client and server”, think, “client and broker” instead. In a traditional client/server relationship, the client and the server connect, and the server is treated as a storage and distribution depot for the data. With MQTT, the process is different. The broker is much more passive, acting more like a signpost for where the data should go.

MQTT client

Any Thing (from a microcontroller to a massive server) that runs an MQTT library and connects to a broker over a network can effectively become a client. 

Clients don’t send messages directly to and from each other but instead communicate to topics managed by the broker. These topics work a little like email inboxes. Messages are published by Things to topics; messages are then picked up when a Thing subscribes to those topics.

MQTT broker

The broker handles authentication of Things on the network as well as managing connections, sessions, and subscriptions. Its main responsibility is to receive all published messages and then send them subscribed clients. The broker also queues messages for subscribed clients, delivering them according to the agreed QoS level.

 

MQTT devices and applications

In an MQTT network, devices and applications are usually known as “things”. The reason for this is that there is no difference between the two as far as the broker is concerned. To that end, both devices and applications can publish and subscribe to topics managed by the broker.

Devices

There are various types of MQTT-enabled devices in the field today, ranging from simple Arduino-based devices to devices for mission-critical commercial, industrial, and medical applications. Many smart homes and businesses are also built around interconnected MQTT devices.

Instant IoT

For businesses looking to launch digital transformation or IoT trials, u-blox has developed the XPLR-IOT-1.

The XPLR-IOT-1 comes with MQTT Anywhere connectivity baked-in, meaning it can operate in over 190 countries straight out of the box.

Along with access to the u-blox IoT Communication-as-a-Service suite, the XPLR-IOT-1 also contains u-blox GNSS and short range devices, an array of sensors, and easy access to u-blox IoT Location-as-a-Service services.

MQTT Anywhere

 

MQTT topics

Messages aren’t delivered directly from Thing to Thing. Instead, they are published to "topics". The broker then delivers those messages to any subscribed clients.

Anatomy of an MQTT topic

• Topics are made up of one or more topic levels, separated by a forward slash:

 

• Topics are case sensitive
• Topics don’t have to be pre-registered at the broker.

MQTT-SN topic features

MQTT-SN adds some special topic features to assist in a bandwidth-constrained environment:

• If using predefined topics, you can create a topic alias on the broker which the client can then use without the need to register first. This can reduce the number of billable messages.
• The long topic name does not have to be sent over the air in either direction. This saves bandwidth, and there is no need to store the long topic name in memory.
• If not using predefined topics, Things use the REGISTER command to register a topic name with the server. The server will respond with a REGACK containing a topic ID consisting of 2 characters. Note that you would need to register the topic each time the client connects.

Why use topics?

Topics are a great way to organize data as it flows through the network and this becomes more apparent with scale. For example, if you have devices with several sensors deployed across multiple sites, you could put all of the data in one payload and parse it when it gets to its destination or you could do it the MQTT way and use topics to divide up the data, as shown below:

• site1/position
• site1/temp
• site1/vibration
• site2/position
• site2/temp
• site2/vibration

When the data transmitted is divided by topic, Things can then subscribe to the topics they are interested in. If a device is interested in multiple topics, they can be subscribed to individually or wildcards can be used. For example, to get all the data from site1, you would use the wildcard, “site1/#”. You could also get all “temp” data from all sites by using the wildcard, “+/temp”.

Connections

Connections are always made between a client and a broker. Clients cannot connect directly to each other.

The connection is established by the client sending a CONNECT message, to which the broker responds with a CONNACK (connection acknowledged). A connection is required in order to publish with a Quality of Service (QoS) level 0, 1, or 2 or to subscribe to any topics. Clients usually connect to a broker using a Client ID (ClientID), username, and password. A client may only have one connection with the same broker at any given time.

Clean session

The “clean session” setting gives you the ability to start fresh with no messages in the queue.

• false: The broker stores all subscriptions for the client and will queue any messages for that subscription that were published with a QoS level 1 or 2
• true: The broker purges all queued messages when the client connects.

Keep alive

The “keep alive” setting defines the longest period of time that the connection can remain in place without the broker or client sending a message.

Note that while a client is in a connected state, the broker will immediately deliver any messages published to topics that the client is subscribed to. For this reason, especially where connectivity is volatile e.g. in mobile communications, the keep alive should be kept to a minimum to prevent possible data loss.

Sleep (SN only)

An MQTT-SN client can tell the broker it is going to sleep for a period of time by sending a DISCONNECT with a DURATION greater than 0.

While a client is in a sleep state, the broker will queue all messages published to topics that the client is subscribed to regardless of the QoS used for the publish.

From the sleep state, the client can flush the queue by issuing a PINGREQ. The broker will then respond with a PUBLISH if there are messages to deliver, and a PINGRESP when the flush is complete, putting the device back to sleep.

Subscriptions

Clients don’t connect directly to each other, instead, they subscribe to topics to receive messages.

To subscribe to a topic, the client must first send a SUBSCRIBE request to the broker. The  SUBSCRIBE request can include multiple topics. The broker responds to the SUBSCRIBE request with a SUBACK (subscription acknowledged) response. Subscriptions also have a QoS setting which can be used to downgrade the QoS of the published message. In this case, the message is always published at the lower QoS setting.

Subscription wildcards

Subscriptions can use one of two types of wildcard shown below.

Single-level (+)

A single-level wildcard replaces one topic level. As shown in the image below.

 

This wildcard would cover the following topics:

• sensors/soil/out
• sensors/water/out
• sensors/light/out

Multi-level (#)

A multi-level wildcard replaces multiple topic levels:

 

This wildcard would cover the following topics:

• sensors/soil/out
• sensors/soil/in
• sensors/temperature/out

MQTT QoS explained

Guarantee of delivery is defined by QoS (Quality of Service).

Here, you will learn how, where, and when to use QoS, and which levels are right for your own IoT applications.

QoS levels

MQTT and MQTT-SN support multiple levels of QoS for guaranteeing message delivery.

QoS -1 – fire and forget

QoS -1 (minus one) is ideal for low-power non-critical applications where it doesn’t matter if every message gets to where it’s going. By not making a hard connection with the broker and receiving no acknowledgment, considerably less power is used to complete the transaction.

 

QoS -1 key features:

• Only available for devices using MQTT-SN
• Does not require an MQTT connection to be established
• No acknowledgment from the recipient
• Not retried by the sender
• Analogous to QoS 0 by the time it reaches the broker

When to use QoS -1:

• Ideal for power-constrained Things to minimize time on air
• Minimize messaging cost
• OK if message delivery is not critical e.g. data sent frequently

QoS 0 – at most once

QoS 0 (zero) is used to ensure that a message reaches its destination no more than once. Unlike QoS -1, this method requires an MQTT connection meaning it is less efficient in terms of power.

 

QoS 0 key features:

• Best effort message delivery
• No acknowledgment from the recipient
• Not retried by the sender
• Not queued by the broker for disconnected clients with a valid subscription to the topic

When to use QoS 0:

• Good for power-constrained Things to minimize time on air
• Minimize messaging cost
• OK if message delivery is not critical e.g. data sent frequently
• Not as efficient as MQTT-SN QoS -1 due to the requirement of CONNECT

QoS 1 – at least once

QoS 1 is used when message delivery is critical. This is achieved by queueing messages until the subscriber is able to receive it.

 

QoS 1 key features:

• Guarantees that a message is delivered at least one time to the recipient. 
• Sender stores the message until it receives a PUBACK from the recipient
• Messages may be sent or delivered multiple times.

When to use QoS 1:

• Use when you have to receive every message, but make sure you handle duplicates
• Use if you want messages to be queued on the broker for delivery to offline Clients
• Use when the overhead of QoS 2 is too high

QoS 2 – exactly once

QoS 2 is used when the message needs to arrive once and only once. This level is used when delivery is essential.

 

• QoS 2 is the safest and slowest Quality of Service level
• Guarantees that each message is received only once by the intended recipients by using at least two request/response flows (a four-part handshake) between the sender and the receiver.

When to use QoS 2:

• Use if message delivery is critical and duplicate data is harmful to subscribers

QoS downgrade

In cases where there are many devices on the network, different levels of QoS might be needed. To achieve this, MQTT allows downgrading of the QoS level at the subscriber node. The result of this is that the QoS for a message that is published does not have to be the same as the QoS for a message that is received.

 

QoS is defined by the original publisher, but when the broker then delivers the message on to subscribers, the lower of the PUBLISH QoS and the QoS defined during the SUBSCRIBE is used.

Security – is MQTT secure?

Making sure IoT devices are secure is just as important as making sure that they work and every link or node on the network is a potential exploit vector. While the protocol itself does contain some security mechanisms, it’s important to consider factors extraneous to the transport itself.

Network-level security

The first place to consider security is the network itself. Ensuring that the network connection is secured by using a VPN tunnel will prevent exposure to network traffic.

Transport-level security

As with HTTP traffic, MQTT traffic can be secured at the transport layer with TLS/SSL.

Application-level security

Unique client identification and username/password credentials are provided by the protocol itself and should be considered the bare minimum security requirement.

Payload encryption

Further security can be added by encrypting the payload itself at the application level.

U-blox Thingstream MQTT security

The u-blox Thingstream service delivery platform adds an extra layer of security on top of those mentioned above by not exposing the IP address of the device. By having no physical connection to the Internet, this makes devices much harder to exploit.

MQTT v5.0 and future evolution

In 2019, standards body, OASIS released the official MQTT 5.0 standard.

New features added to the Version 5.0 standard:

• Reason codes: Acknowledgements now support return codes, which provide a reason for failure.
• Shared subscriptions: Allow the load to be balanced across clients and thus reduce the risk of load problems
• Message expiry: Messages can include an expiry date and are deleted if they are not delivered within this time period.
• Topic alias: The name of a topic can be replaced with a single number

u-blox MQTT IoT services

• MQTT Anywhere - IoT Communication-as-a-Service SIM-based LPWA
• MQTT Flex - MQTT-SN communication with bring-your-own SIM LTE & NB-IoT connectivity
• MQTT Here - LoRaWAN IoT connectivity solution
• MQTT Now - Cloud-based MQTT integration for IP devices

Extra resources

• u-blox IoT Communication-as-a-Service
• OASIS – standards body responsible for MQTT
• The MQTT Technical Committee