How to inspect corrupted MQTT packets on IoT brokers

Why do MQTT packets get corrupted along the way?

When IoT devices, such as industrial sensors and smart meters, talk to a central server using the MQTT protocol, they rely on a lightweight and fast connection. In practice, this means small packets constantly travel across networks that may suffer from electromagnetic interference or packet drops. If the physical layer fails, the broker (the central server managing messages) receives broken data, creating hard-to-trace bugs in critical systems. To understand how physical infrastructure issues impact delivery, you can check our guide on how to crimp and test CAT6A cables without packet loss on the bench or in the field.

Data corruption doesn't always come from bad cables; it can stem from memory overflows in cheap microcontrollers or voltage fluctuations in the power supply. When hardware suffers from instability, bytes sent over the network arrive at the destination shifted or corrupted, invalidating the payload (the core data cargo inside a message).

Essential tools for real-time traffic capture

To figure out where the data got lost, we need to intercept the traffic before the application processes it. The most famous tool for this task is Wireshark, a network analyzer that shows every packet traversing the network interface. In practice, it works like a digital microscope, opening each packet to reveal the TCP and IP headers along with the exact MQTT bytes.

However, on embedded systems running a minimalist Linux, we often lack a graphical interface. In these scenarios, the tcpdump utility is an engineer's best friend. It runs directly in the terminal and records raw traffic for later analysis. Furthermore, making sure the electrical infrastructure isn't sabotaging data due to dirty power is vital, much like we explain when detailing how to test PoE switch voltage on the bench using a multimeter.

Filtering and inspecting QoS messages in Wireshark

MQTT uses QoS (Quality of Service) levels to ensure messages reach their destination. QoS 0 fires and forgets; QoS 1 guarantees delivery at least once using acknowledgments (PUBACK); and QoS 2 guarantees exactly-once delivery via a four-step handshake. When corruption happens, these confirmation packets usually fail, triggering endless retransmissions.

To isolate these issues in Wireshark, type the mqtt filter into the top search bar. This hides unnecessary network noise and displays only CONNECT, PUBLISH, SUBSCRIBE, and PUBACK connections. If you notice packets marked with invalid flags or payload sizes that don't match the fixed header, there's a strong chance of a buffer overflow (a memory bug where the program overwrites neighboring data) on the sending device.

Step-by-step to capture and analyze corrupted packets in the terminal

If you need to debug an embedded device directly in the field where it is installed, following a standardized command-line capture procedure speeds up diagnosis. Below are the recommended steps to isolate the issue using native Linux tools:

1. Connect to the gateway or IoT device via SSH and identify the active network interface by running the command

   ip link show

2. Start packet capture targeting the standard MQTT port (1883) and save it to a log file using the command

   sudo tcpdump -i eth0 -w mqtt_debug.pcap port 1883

3. Transfer the generated file to your workstation and open it in Wireshark to inspect checksum fields and corrupted payloads.

Conclusion and best practices for MQTT resilience

Investigating MQTT failures in embedded systems requires patience and mastery of network inspection tools like tcpdump and Wireshark. Mapping QoS behavior and monitoring the physical health of the network prevents intermittent glitches from paralyzing industrial or automation operations.

Adopting strict payload size validations in device firmware and keeping active broker monitoring ensures any anomaly is detected immediately, drastically raising the reliability of your IoT ecosystem.