Marcio Cunha

Slow database? How to find disk IOPS bottlenecks in Linux

Learn how to diagnose disk latency and IOPS bottlenecks in relational databases under high write concurrency using native Linux tools.

Marcio Cunha•4 min
Also available in:PortuguêsEspañol
Summary
  • Concurrent write operations quickly exhaust the operating system I/O queue when disks reach their physical IOPS limits.
  • The await metric in the iostat command reveals the average time requests take to be serviced, exposing saturated disks.
  • Continuous monitoring with tools like pidstat helps isolate which specific process is generating the heaviest write load.
  • Properly tuning database checkpoint parameters and asynchronous commits mitigates the impact of sudden write spikes.
  • Replacing traditional SATA disk architectures with provisioned NVMe volumes resolves severe storage bottlenecks.

Why does the database freeze when there are many simultaneous writes?

As applications grow, it is common to blame lack of RAM or poorly written SQL queries for general system slowness. However, under high write concurrency scenarios, the real villain is usually the storage subsystem. Requests pile up because the disk hardware cannot process requests at the speed demanded by the database engine.

In practice, this means the database must record every committed transaction into disk log files to guarantee durability, adhering to ACID properties. When hundreds of clients try to modify data simultaneously, the disk wait queue grows exponentially. Understanding this behavior requires looking beyond usual CPU metrics and diving deep into Linux kernel behavior.

To deepen your investigation into query performance before exclusively blaming hardware, check out our practical guide on optimizing PostgreSQL indexes. After all, inefficient indexes force full table scans, generating even more unnecessary read and write traffic on your disks.

What are IOPS and why do they dictate your application lifespan

IOPS stands for Input/Output Operations Per Second. In simple terms, it is the number of times your disk can read or write blocks of data within a one-second interval. Unlike throughput measured in megabytes per second, IOPS measure the ability to handle many small, simultaneous requests.

Relational databases generate thousands of small write operations constantly, updating indexes, tables, and transaction logs. If a traditional mechanical hard drive offers around one hundred IOPS, it will instantly lock up if it receives one thousand write requests per second. Modern SSDs and cloud NVMe volumes dramatically expand this limit, yet they still possess a rigid operational ceiling.

How to use iostat to reveal real disk saturation

The command-line tool iostat is part of the sysstat package and is used to monitor CPU utilization and disk partition performance. It shows how busy disks are and how long requests take to be answered. To start an accurate diagnosis in real time, run the following command in your Linux server terminal:

iostat -xz 2 10

This command outputs detailed reports every two seconds, repeating the reading ten times. The -x argument adds fundamental extended metrics, while -z hides idle devices, keeping the screen clean with only active disks relevant for analysis.

Interpreting critical metrics: await, %util, and svctm

Within the output generated by iostat, certain columns deserve close attention from engineers. The %util column indicates the percentage of time the disk was busy processing requests. If this value is consistently close to one hundred percent, it means the device has reached its physical capacity limit.

Another vital indicator is the await column, which measures the average time in milliseconds that each I/O operation took to complete, including queue time and actual processing time. If await spikes to tens or hundreds of milliseconds, database connections begin to time out, causing application failures.

How to find out which process is choking your storage

Knowing the disk is overloaded is only the first step; the next challenge is figuring out which process is causing the issue. On servers running multiple services, such as databases, web servers, and monitoring tools, the culprit is not always obvious.

To isolate I/O consumption per process, the pidstat tool offers a surgical view of disk activity. Running the command below lets you track processes writing the most data to storage:

pidstat -d 2 5

With this output, you can identify exactly whether it is the PostgreSQL process, MySQL, or some background backup script monopolizing disk bandwidth and throttling the main database.

Practical strategies to mitigate write bottlenecks and save your system

When diagnostics point to a genuine IOPS bottleneck, there are immediate technical paths to relieve pressure on storage. The first step involves reviewing database checkpoint settings, allowing the system to accumulate more changes in RAM before flushing them to disk all at once.

Another alternative is enabling asynchronous writes on secondary log tables where the loss of a few milliseconds of data does not represent financial or legal risk. Additionally, migrating to NVMe-based storage volumes with provisioned IOPS capacity ensures seasonal access spikes won't crash production. To complement this implementation in production environments, see our related technical analyses on How to secure AI applications against prompt injection and How to use WebAssembly in frontend to speed up heavy tasks.

Conclusion and next steps to keep your database healthy

Diagnosing disk latency in databases requires looking beyond fancy cloud monitoring screens and understanding real operating system behavior. The correct use of commands like iostat and pidstat turns subjective guesses into concrete data, allowing you to act directly on the root performance issue.

Maintaining a culture of continuous storage subsystem monitoring prevents unpleasant surprises during traffic spikes. By combining fine-tuned database configurations with proper hardware infrastructure, your application gains the resilience needed to scale without sacrificing operational stability.