How to configure HAProxy load balancing for high concurrency

Why HAProxy is essential in high-concurrency architectures

When a system grows and user traffic spikes, a single web server can no longer handle the load alone. This is where load balancing comes into play, acting as an invisible conductor that distributes network requests across several backend computers. Among the most respected open-source tools for this job is HAProxy, an extremely fast and stable software designed to handle millions of concurrent connections while consuming minimal memory.

In practice, HAProxy intercepts everything coming from the internet and decides which internal server should handle that request. To ensure the system keeps running even if a machine crashes, the tool performs constant health checks, taking problematic nodes out of rotation. Furthermore, just as we manage network resilience using tools like automatic network failover in systemd-networkd, we must ensure web traffic is smoothly redirected without human intervention when a node fails.

Choosing the right traffic distribution algorithm

The way HAProxy decides which server will handle the next client depends on the algorithm configured in its main configuration file. The default method is roundrobin, which performs a simple sequential rotation among available servers. Although it works well when all machines have identical processing power and handle tasks of similar duration, it can cause imbalances if some requests take significantly longer than others.

For scenarios where response times vary greatly, the leastconn algorithm is usually the smarter choice. It directs new requests to the server holding the lowest number of active connections at that exact moment. This prevents an already overloaded server from taking on more work while an idle one sits waiting. If you need the same user to always hit the same server due to active sessions, IP hash-based strategies ensure this persistence cleanly.

Structuring the configuration file and global sections

All HAProxy magic happens inside the haproxy.cfg file, typically located in the /etc/haproxy/ directory. The file is split into well-defined logical blocks, starting with the global section where low-level operating system parameters are set, such as the maximum number of simultaneous processes and the user running the service.

Right below it lies the defaults section, which establishes standard timeout rules and behaviors for subsequent blocks. It is crucial to configure timeout limits correctly. For instance, setting a limit on how long HAProxy waits for a server response prevents stalled connections from holding network ports open indefinitely and exhausting machine resources.

global
    log /dev/log local0
    maxconn 50000
    user haproxy
    group haproxy

defaults
    log     global
    mode    http
    option  httplog
    timeout connect 5s
    timeout client  50s
    timeout server  50s

Configuring frontends and backends for high availability

The operational core of HAProxy rests on the pairing of frontend and backend blocks. The frontend defines how the load balancer accepts client connections, listening on specific ports and public IP addresses. This is where we apply security rules, SSL certificates for encryption, and initial packet inspection—much like the care we take when inspecting TCP network traffic in real time with tcpdump to spot network bottlenecks.

Meanwhile, the backend groups the actual servers that will process the heavy lifting. Below is a practical example of how to declare these blocks, enabling the least-connections algorithm and configuring automated health checks to ensure traffic never hits a dead server.

frontend web_cluster_front
    bind *:80
    default_backend web_cluster_back

backend web_cluster_back
    balance leastconn
    option httpchk GET /healthz
    server web1 192.168.1.10:8080 check inter 3s rise 2 fall 3
    server web2 192.168.1.11:8080 check inter 3s rise 2 fall 3

Monitoring failures and maintaining operational resilience

Configuring load balancing is only the first step; ensuring the system continues operating smoothly requires active monitoring. The check parameter in the configuration file instructs HAProxy to send periodic requests—such as a simple HTTP test to the /healthz route—to confirm that the application is alive and responding correctly.

If a server fails three consecutive attempts (defined by fall 3), HAProxy immediately removes it from the distribution route, redirecting incoming clients to the remaining servers. Once the issue is resolved and the server responds successfully twice in a row (rise 2), it is automatically brought back in. This autonomous self-healing cycle drastically reduces the need for manual midnight intervention when incidents happen. To explore these architectural concepts further in production, see our related technical analysis on How to test motherboard CMOS battery with a multimeter.

Final thoughts on performance and scalability

Mastering HAProxy configuration allows you to turn a fragile infrastructure into an environment capable of absorbing massive traffic spikes without losing stability. The secret lies in aligning your distribution algorithm choice with the actual profile of your application, fine-tuning timeout parameters, and maintaining rigorous health checks across all backend nodes. With these practices in place, your architecture gains the robustness needed to scale sustainably and securely.