MQTT Introduction

1. Brief Introduction

MQTT (Message Queuing Telemetry Transport) is a "lightweight" communication protocol based on the publish/subscribe mode. This protocol is built on the TCP/IP protocol and was released by IBM in 1999. The biggest advantage of MQTT is that it can provide real-time and reliable messaging services for connected remote devices with minimal code and limited bandwidth. As an instant messaging protocol with low overhead and low bandwidth usage, it is widely used in the Internet of Things, small devices, mobile applications, and other fields.

MQTT is a client-server based message publish/subscribe transport protocol. The MQTT protocol is lightweight, simple, open, and easy to implement, making it applicable in a very wide range of scenarios. In many cases, including constrained environments, such as machine-to-machine (M2M) communication and the Internet of Things (IoT). It has been widely used in sensors communicating via satellite links, medical devices that dial in occasionally, smart homes, and some miniaturized devices.

2. Design Specifications

Since the IoT environment is very special, MQTT follows the following design principles:

(1) Streamlined, without adding non-essential features;
(2) Publish/Subscribe (Pub/Sub) mode, facilitating message transmission between sensors;
(3) Allows users to dynamically create topics, with zero operation and maintenance costs;
(4) Minimize the transmission volume to improve transmission efficiency;
(5) Take into account factors such as low bandwidth, high latency, and unstable networks;
(6) Support continuous session control;
(7) Understand that client computing power may be very low;
(8) Provide service quality management;
(9) Assume data is unknown, do not impose the type and format of transmitted data, and maintain flexibility.

3. Main Features

The MQTT protocol is designed for communication between remote sensors and control devices over low-bandwidth, unreliable networks. It has the following main features:

(1) Uses the publish/subscribe message pattern to provide one-to-many message publishing, decoupling applications.
This is very similar to XMPP, but MQTT has far less information redundancy than XMPP because XMPP uses XML-formatted text to transmit data.
(2) Message transmission that is transparent to the payload content.
(3) Uses TCP/IP to provide network connections.
Mainstream MQTT uses TCP connections for data push, but there are also UDP-based versions, called MQTT-SN. Since these two versions are based on different connection methods, their advantages and disadvantages naturally differ.
(4) There are three message publishing service quality levels:

"At most once": message publishing relies entirely on the underlying TCP/IP network. Message loss or duplication may occur. This level can be used in situations such as environmental sensor data; losing one reading doesn't matter because a second send will occur soon. This method is mainly used for push notifications in ordinary apps. If your smart device is not online when a message is pushed, the push won't be received, and it won't be received when it comes online again.

"At least once": ensures message delivery, but message duplication may occur.

"Exactly once": ensures the message arrives exactly once. This level can be used in some strictly required billing systems. In billing systems, duplicate or lost messages can lead to incorrect results. This highest-quality message publishing service can also be used for push notifications in instant messaging apps, ensuring users receive the message exactly once.
(5) Small transmissions with minimal overhead (the fixed-length header is 2 bytes), minimizing protocol exchanges to reduce network traffic.

This is why the introduction says it is very suitable for "communication between sensors and servers, and information collection in the IoT field." You should know that the computing power and bandwidth of embedded devices are relatively weak, so using this protocol to transmit messages is perfectly suitable.
(6) Uses the Last Will and Testament feature to notify relevant parties of abnormal client disconnection.

Last Will: That is, the will mechanism, used to notify other devices under the same topic that the device that sent the will has disconnected.

Testament: The testament mechanism, whose function is similar to Last Will.

4. MQTT Protocol Principles

4.1 MQTT Protocol Implementation Method

Implementing the MQTT protocol requires communication between the client and server. During communication, there are three roles in the MQTT protocol: Publisher, Broker (server), and Subscriber. Among them, both the publisher and subscriber are clients, the message broker is the server, and a publisher can also be a subscriber at the same time.

Messages transmitted by MQTT are divided into two parts: Topic and Payload.

(1) Topic can be understood as the type of message. After a subscriber subscribes, it will receive the message content (payload) of that topic.
(2) Payload can be understood as the content of the message, referring to the specific content that the subscriber will use.

4.2 Network Transmission and Application Messages

MQTT constructs the underlying network transmission: it establishes a connection from the client to the server, providing an ordered, lossless, byte-stream-based two-way transmission between the two.

When application data is sent through the MQTT network, MQTT associates the related Quality of Service (QoS) and topic name with it.

4.3 MQTT Client

An application or device that uses the MQTT protocol always establishes a network connection to the server. A client can:

(1) Publish information that other clients may subscribe to;
(2) Subscribe to messages published by other clients;
(3) Unsubscribe or delete application messages;
(4) Disconnect from the server.

4.4 MQTT Server

The MQTT server is also called a "message broker" (Broker). It can be an application or a device. It is located between the message publisher and subscribers, and it can:

(1) Accept network connections from clients;
(2) Accept application information published by clients;
(3) Process subscription and unsubscription requests from clients;
(4) Forward application messages to subscribed clients.

4.5 Subscriptions, Topics, and Sessions in the MQTT Protocol

1. Subscription

A subscription contains a Topic Filter and a maximum Quality of Service (QoS). A subscription is associated with a session. A session can contain multiple subscriptions. Each subscription in each session has a different topic filter.

2. Session

After each client establishes a connection with the server, it becomes a session, with state interaction between the client and server. A session exists over a network and may also span multiple consecutive network connections between the client and server.

3. Topic Name

A label attached to an application message that matches the subscriptions on the server. The server sends the message to each client whose subscription matches the label.

4. Topic Filter

A wildcard filter for topic names, used in subscription expressions, representing multiple topics that the subscription matches.

5. Payload

The specific content received by the message subscriber.

4.6 Methods in the MQTT Protocol

The MQTT protocol defines some methods (also called actions) to represent operations on certain resources. This resource can represent pre-existing data or dynamically generated data, depending on the server implementation. Generally, resources refer to files or outputs on the server. The main methods are:

(1) Connect. Wait to establish a connection with the server.
(2) Disconnect. Wait for the MQTT client to complete its work and disconnect the TCP/IP session with the server.
(3) Subscribe. Wait for the subscription to complete.
(4) UnSubscribe. Wait for the server to cancel the client's subscription to one or more topics.
(5) Publish. The MQTT client sends a message request and returns to the application thread after sending is complete.

5. MQTT Protocol Packet Structure

In the MQTT protocol, an MQTT packet consists of three parts: Fixed header, Variable header, and Payload. The MQTT packet structure is as follows:

(1) Fixed header. Present in all MQTT packets, indicating the packet type and the packet's group class identifier.
(2) Variable header. Present in some MQTT packets. The packet type determines whether the variable header exists and its specific content.
(3) Payload. Present in some MQTT packets, representing the specific content received by the client.

5.1 MQTT Fixed Header

The fixed header exists in all MQTT packets, and its structure is as follows:

5.1.1 MQTT Packet Type

Location: bits 7-4 in Byte 1.

It is a 4-bit unsigned value. The types, values, and descriptions are as follows:

5.1.2 Flags

Location: bits 3-0 in Byte 1.

In message types that do not use flags, the flags are treated as reserved bits. If an invalid flag is received, the receiver must close the network connection:

(1) DUP: Duplicate of a published message. Used to ensure reliable transmission of messages. If set to 1, the MessageId is added in the variable header below, and an acknowledgment reply is required to ensure message transmission is completed, but it cannot be used to detect duplicate message sending.

(2) QoS: Quality of Service for published messages, i.e., the number of times message delivery is guaranteed.

Ø00：最多一次，即：<=1

Ø01：至少一次，即：>=1

Ø10：一次，即：=1

Ø11：预留

(3) RETAIN: Publish retain flag. Indicates that the server should retain this pushed information. If a new subscriber appears, push this message to it; if not set, release it after pushing to the current subscriber.

5.1.3 Remaining Length

Location: Byte 2.

The second byte of the fixed header is used to store the total size of the variable header and the payload, but it is not stored directly. This byte is expandable. Its storage mechanism: the first 7 bits are used to store the length, and the last bit is used as an identifier. When the last bit is 1, it means the length is insufficient, and a second byte is needed to continue storing. For example: if the calculated following size is 0

5.2 MQTT Variable Header

An MQTT packet contains a variable header, which resides between the fixed header and the payload. The content of the variable header varies by packet type. A more common application is as a packet identifier:

Many types of packets include a 2-byte packet identifier field. These types of packets include: PUBLISH (QoS > 0), PUBACK, PUBREC, PUBREL, PUBCOMP, SUBSCRIBE, SUBACK, UNSUBSCRIBE, UNSUBACK.

5.3 Payload Message Body

The payload message body is the third part of an MQTT packet, containing messages of four types: CONNECT, SUBSCRIBE, SUBACK, UNSUBSCRIBE:

(1) CONNECT: The message body mainly contains the client's ClientID, subscribed Topic, Message, and username and password.
(2) SUBSCRIBE: The message body contains a series of topics to be subscribed to and the QoS.
(3) SUBACK: The message body is the server's confirmation and reply to the topics and QoS requested by SUBSCRIBE.
(4) UNSUBSCRIBE: The message body contains the topics to be subscribed to.

Category Programming Technology