Slow disk on Linux? How to spot I/O bottlenecks with iostat

Why does your Linux server slow down without apparent reason?

When a web application starts responding slowly or the entire system freezes while processing a heavy task, the blame usually falls on the processor or a lack of RAM. However, in many cases, the real villain is the storage subsystem. Storage drives, whether traditional mechanical hard disks or modern solid-state units, have physical speed limits that, when exceeded, create an invisible waiting queue and freeze the entire operating system.

To investigate this type of issue without installing complex monitoring software, system administrators rely on native utilities built into the vast majority of Linux distributions. Tools like iostat and vmstat are true Swiss Army knives for inspecting the internal health of a machine in real time. They reveal exactly what is happening behind the scenes in the kernel, making it easy to distinguish a momentary load spike from a chronic bottleneck that demands immediate intervention, such as increasing simultaneous connection limits to ease data flow.

How to install and extract first metrics from iostat

The iostat command is part of a utility package focused on system performance statistics called sysstat. If the command is not available in your production environment, installation is quick using the distribution's standard package manager, such as APT on Debian or Ubuntu. In practice, it collects metrics directly from the operating system kernel regarding CPU workload and the volume of data transfer handled by each mounted partition or physical disk.

Running the basic command in the terminal displays a general summary since the machine was last booted. To monitor the actual behavior of the machine under stress, it is ideal to specify a time interval in seconds followed by the desired number of repetitions. For example, requesting readings every two seconds forces the tool to continuously update the panel, which greatly facilitates identifying sudden write or read spikes caused by heavy databases or backup routines.

sudo apt update && sudo apt install sysstat -y
iostat -xz 2 5

Interpreting vital metrics like await, %util and tps

Reading the output of iostat requires attention to specific columns that determine whether the disk is overloaded or not. The tps column indicates the number of transfers per second sent to the physical device, encompassing both read and write operations. However, a high tps value does not always mean a problem, as fast disks handle thousands of operations per second well if the controller is properly provisioned.

The real warning sign lies in the await column, which measures the average time in milliseconds that each I/O request took to be serviced, including time spent in the waiting queue and actual processing time on the disk. If the await value spikes dramatically, applications will start accumulating blocked threads. Another crucial indicator is the %util column, which shows the percentage of time the disk remained busy processing requests, revealing saturation when it consistently approaches one hundred percent.

Identifying excessive swap memory usage with vmstat

When physical RAM runs out, the operating system resorts to a technique called paging, moving inactive pieces of memory to a reserved area on the hard drive known as swap space. While this prevents applications from crashing due to out-of-memory errors, constant swapping severely degrades overall performance because reading and writing data to storage is orders of magnitude slower than accessing it directly in RAM sticks.

To diagnose this undesirable behavior, the vmstat command comes into play, displaying an integrated overview of processes, memory, paging, interrupts, and CPU activity. Running the command continuously reveals whether the system suffers from a chronic shortage of physical resources, helping decide whether it is time to optimize running services or monitor overall hardware temperature to prevent catastrophic thermal failures under heavy load.

vmstat 2 10

Analyzing si and so columns to detect throttling

Within the report generated by vmstat, two columns deserve constant monitoring by the administrator: si (swap in) and so (swap out). The si column indicates the amount of data blocks brought from disk into RAM per second, while the so column shows how many blocks are pushed from RAM to disk over the same interval.

If si and so values remain zero or close to it during normal operation, the server's memory is adequate for the current workload. On the other hand, if these columns show high and steady numbers, it means the server suffers from a chronic shortage of physical memory, a phenomenon commonly known as thrashing. In this scenario, the system spends more time moving data between RAM and disk than executing actual application code. To explore these architectural concepts further in production, see our related technical analysis on Dark laptop screen? How to test backlight on the workbench.

Final considerations on I/O diagnosis in Linux

Identifying input and output bottlenecks in Linux ceases to be an insurmountable mystery once we master the combined use of native, lightweight tools like iostat and vmstat. Monitoring disk queue waiting times and swap paging flow ensures administrators act at the root of the problem before slowness impacts end-user experience. Understanding these metrics prevents unnecessary expenses on premature hardware upgrades and directs optimization efforts exactly where the system needs them most.

Maintaining a proactive checking routine with these utilities helps map the baseline behavior of each application during peak hours. That way, any drastic deviation in storage performance or memory consumption can be isolated quickly, ensuring high availability, stability, and longevity for the entire organization's server ecosystem.