MQTT Basics: Publish-Subscribe Architecture Explained

MQTT (Message Queuing Telemetry Transport)

1. Basic Definition

MQTT (Message Queuing Telemetry Transport) is a lightweight, publish-subscribe (pub/sub) messaging protocol designed for constrained devices and low-bandwidth, high-latency, or unreliable networks. Developed by Andy Stanford-Clark (IBM) and Arlen Nipper (Arcom) in 1999 for remote oil field telemetry, MQTT is now an OASIS standard and widely used in IoT (Internet of Things), industrial automation, and mobile applications for efficient data exchange between devices and servers.

2. Core Architecture & Pub/Sub Model

MQTT follows a publish-subscribe pattern, which decouples message senders (publishers) from receivers (subscribers) via a central broker:

2.1 Key Entities

Publisher: A device/application that sends messages to a specific topic (e.g., a temperature sensor publishing data to sensors/room1/temp). Publishers do not need to know about subscribers.
Subscriber: A device/application that requests messages from one or more topics (e.g., a dashboard subscribing to sensors/+/temp to receive all temperature data). Subscribers do not need to know about publishers.
Broker: A central server that receives messages from publishers, filters them by topic, and distributes them to relevant subscribers. The broker handles connection management, message routing, and quality of service (QoS) guarantees.

2.2 Topics

Topics are hierarchical strings that categorize messages (similar to file system paths), using / as a separator. Wildcards enable flexible subscription:

Single-level wildcard (+): Matches exactly one level in a topic (e.g., sensors/+/temp matches sensors/room1/temp and sensors/room2/temp).
Multi-level wildcard (#): Matches all subsequent levels (e.g., sensors/room1/# matches sensors/room1/temp, sensors/room1/humidity, etc.).Note: The # wildcard must be the last character in a topic.

3. Key Features & Technical Specifications

3.1 Lightweight Design

Small Packet Overhead: MQTT messages have minimal header size (2 bytes for fixed headers), making them ideal for low-bandwidth networks (e.g., cellular IoT, LoRaWAN).
Low Resource Usage: Requires minimal CPU/memory on devices (works on microcontrollers like Arduino or ESP32).
Binary Protocol: Messages are encoded in binary (not text), reducing data size compared to text-based protocols (e.g., HTTP).

3.2 Quality of Service (QoS) Levels

MQTT defines three QoS levels to ensure message delivery reliability (trade-off between reliability and network overhead):

QoS 0 (At Most Once): “Fire and forget” – messages are delivered once or not at all (no acknowledgment). Used for non-critical data (e.g., real-time sensor readings).
QoS 1 (At Least Once): Messages are guaranteed to be delivered at least once (broker/publisher retries until acknowledgment). May result in duplicate messages (handled by subscribers). Used for critical data (e.g., alarm triggers).
QoS 2 (Exactly Once): Messages are delivered exactly once (no duplicates, no losses) via a four-step handshake. Highest reliability, highest overhead. Used for mission-critical data (e.g., financial transactions, industrial control commands).

3.3 Retained Messages

A publisher can mark a message as “retained,” meaning the broker stores the last message for a topic and sends it to new subscribers when they connect. Useful for providing the latest state (e.g., a sensor’s current temperature to a newly connected dashboard).

3.4 Last Will and Testament (LWT)

A client can specify a “last will” message when connecting to the broker. If the client disconnects unexpectedly (e.g., power loss), the broker automatically publishes the LWT message to a predefined topic. Used for fault detection (e.g., alerting if a sensor goes offline).

3.5 Keep-Alive Mechanism

Clients send periodic “ping” messages to the broker to confirm the connection is active. If the broker does not receive a ping within the keep-alive interval (configurable, e.g., 60 seconds), it assumes the client is disconnected and triggers the LWT (if set).

4. MQTT Packet Structure

MQTT messages are composed of fixed and variable headers, plus a payload (optional):

Fixed Header (2 bytes minimum): Contains the message type (e.g., CONNECT, PUBLISH, SUBSCRIBE), QoS level, retain flag, and remaining length (bytes of variable header + payload).
Variable Header: Message-type-specific data (e.g., topic name for PUBLISH packets, client ID for CONNECT packets).
Payload: The actual message data (e.g., sensor readings, commands) – only present in PUBLISH, CONNECT, and SUBSCRIBE packets.

5. MQTT Versions & Variants

MQTT 3.1.1: The most widely adopted version (OASIS standard), with stable QoS and core features.
MQTT 5.0: Introduced in 2018, adding enhancements like:
- Message expiration (auto-delete stale messages).
- Topic aliases (reduce repeated topic string overhead).
- Reason codes for failures (improved error handling).
- Shared subscriptions (load balancing across multiple subscribers).
MQTT-SN (MQTT for Sensor Networks): Adaptation for low-power, wireless sensor networks (e.g., Zigbee, Bluetooth) that do not support TCP/IP.
MQTT over WebSockets: Enables MQTT communication between web browsers and brokers (used in IoT dashboards).

6. Real-World Applications

6.1 IoT & Smart Devices

Smart Homes: Thermostats (e.g., Nest) publish temperature data to smarthome/thermostat/livingroom; smart lights subscribe to smarthome/lights/+/command for on/off commands.
Industrial IoT (IIoT): Factory sensors publish machine vibration, temperature, and pressure data to a broker; SCADA systems subscribe to monitor equipment health and trigger alerts.
Agriculture: Soil moisture sensors in fields publish data to agri/farm1/field2/moisture; irrigation systems subscribe to activate pumps when moisture is low.

6.2 Telemetry & Remote Monitoring

Oil/Gas Industry: Remote wellhead sensors publish pressure/flow data via satellite (low-bandwidth) to a central broker for real-time monitoring.
Vehicle Telematics: Fleet tracking devices publish GPS location, speed, and fuel level to fleet/truck123/telemetry; dispatch systems subscribe to optimize routes.

6.3 Healthcare

Wearable Devices: Fitness trackers publish heart rate, step count, and sleep data to health/user456/metrics; healthcare providers subscribe to monitor patients remotely.
Medical Equipment: Hospital ventilators publish operational status to hospital/icu/ventilator5/status; nurses’ stations subscribe to alert on malfunctions.

6.4 Mobile & Web Applications

Chat Apps: Lightweight chat applications use MQTT for real-time message delivery (lower battery usage than WebSockets on mobile).
IoT Dashboards: Web-based dashboards (e.g., Node-RED, Grafana) subscribe to MQTT topics to visualize sensor data in real time.

7. Popular MQTT Brokers & Tools

7.1 Brokers

Eclipse Mosquitto: Open-source, lightweight broker (ideal for small-scale deployments and edge devices).
EMQ X (EMQX): Scalable, open-source broker for large-scale IoT deployments (supports MQTT 5.0, clustering, and multi-protocol integration).
HiveMQ: Enterprise-grade broker with security features (SSL/TLS, authentication) and cloud hosting (HiveMQ Cloud).
AWS IoT Core / Azure IoT Hub: Cloud-based MQTT brokers integrated with cloud services (storage, analytics) for IoT applications.

7.2 Client Libraries & Tools

Client Libraries: Paho (Python, Java, C/C++), MQTT.js (JavaScript), Arduino MQTT Library (for microcontrollers).
Testing/Monitoring: MQTT.fx (GUI client for publishing/subscribing), MQTT Explorer (visualizes broker topics and messages), Node-RED (low-code IoT workflow tool with MQTT nodes).