Data Synchronization Between MQTT Brokers and Edge Servers in Intermittent Industrial Networks

The Connectivity Challenge in Industrial Networks

Keeping automation systems connected in manufacturing plants or remote facilities sounds simple on paper, but the physical reality of the factory floor frequently sabotages that stability. Massive metallic barriers, electromagnetic interference from heavy motors, and geographic isolation turn the network into a volatile ecosystem where signal drops are the rule rather than the exception. In practice, this means field devices and sensors continue generating thousands of metrics per second, but hit invisible walls when trying to dispatch those packets to central servers or the corporate cloud.

When network infrastructure fails, traditional architectures based on synchronous and continuous connections collapse immediately. Corporate systems relying on uninterrupted flows lose operational visibility, creating dangerous blind spots that prevent early detection of failures in critical machinery. To solve this structural vulnerability, engineers turn to a distributed architecture, decentralizing processing and placing robust computers, known as edge servers, physically close to the factory floor sensors.

The Role of the MQTT Protocol in Lightweight Communication

Originally created to monitor remote oil pipelines via satellite, the MQTT (Message Queuing Telemetry Transport) protocol has become the undisputed standard for the industrial internet of things due to its structural lightness. It operates on a publish-subscribe model, where sensors publish data to specific topics without needing to know who will consume them, and servers subscribe to those topics to receive the information. In practice, it is like a postal system where the sender leaves letters in a labeled mailbox and interested parties pick them up when needed, eliminating the need for direct and repetitive calls between machines.

Another fundamental asset of MQTT lies in the customization of delivery guarantee levels, known as Quality of Service or QoS. Level zero operates on a best-effort model, ideal for high-frequency temperature readings where losing an isolated point does not alter the overall picture. Level one introduces a simple acknowledgment mechanism, ensuring the message is retained and retransmitted until the receiver acknowledges receipt. This flexibility allows adjusting industrial network behavior according to the criticality of each specific data stream.

Edge Server Architecture as Network Shock Absorbers

When the connection to the central server or cloud disappears completely, the edge server assumes the role of guardian of data at the edge of the network. Instead of discarding information or crashing due to a lack of route, the edge server intercepts MQTT messages emitted by PLCs (programmable logic controllers) and sensors, storing them temporarily in fast local disks. In practice, the server acts like a water reservoir that accumulates liquid during dry spells to distribute it in a controlled manner as soon as rain returns.

This approach requires using lightweight local databases optimized for continuous writes, such as SQLite or compact time-series engines, capable of withstanding intense write cycles without corrupting storage. The edge server acts not only as a passive shock absorber but also as an autonomous processor capable of making local emergency decisions, such as shutting down an overheated motor even without contact with the main operations center.

Advanced Synchronization and Reconciliation Strategies

Re-establishing connection after an intermittency period requires extreme care to prevent the central server from being suffocated by an avalanche of accumulated data. If thousands of messages stored on the edge server are triggered all at once, available bandwidth collapses again, creating a vicious cycle of disconnection. To mitigate this risk, an exponential backoff algorithm is implemented, where the interval between retry attempts increases progressively if the network remains unstable.

In addition to controlled sending rates, state reconciliation demands the rigorous use of precise timestamps in each data packet generated at the source. Since device clocks may experience small variations, the edge server acts by synchronizing time via local protocols before stamping events. This ensures that upon re-establishing connection, the central database chronologically orders events occurred during the outage, preserving the historical truth of the industrial operation without corrupting metrics and performance indicators. To complement this implementation in production environments, see our related technical analyses on Data Synchronization in Heterogeneous Industrial Networks with Edge Gateways and High Precision Clock Synchronization with PTP in Industrial Networks for Deterministic Data Acquisition.

Final Considerations on Operational Resilience

Modern industrial network engineering does not seek to eliminate connectivity intermittency, as external physical factors make that utopia unreachable. Instead, the real goal is to build resilient systems that assume failure as a normal operational state and know how to bypass it elegantly. By combining the lightness of the MQTT protocol with the absorption capacity of edge servers, engineers ensure that factory operations remain safe, predictable, and fully instrumented, regardless of the whims of the communication network.