Ext4 file system: Delayed allocation, dirty data blocks, fs-verity

Part 1 — Ext4 filesystem: data blocks, super blocks, inode structure

Part 2 — Ext4 filesystem: extent, flex_bg, sparse_super

Photo by Mr Cup / Fabien Barral on Unsplash

Delayed Allocation

The delayed allocation feature is enabled by default. That’s why the data may not be committed to the disk immediately when you try to write or modify a data block.

I mounted the ext4 filesystem with the default parameters (delayed allocation enabled). Blocks allocated after a while:

I mounted the ext4 filesystem with the nodelalloc parameter (delayed allocation disabled). I checked the blocks right after the file was created. Blocks allocated immediately:

What are the dirty data blocks?

Data blocks that have not been committed to disk are called “dirty data blocks”.

ext4 has 3 modes for writing files (you can set the mode while mounting the disk):

1. data=ordered (default mode, delayed allocation supported)
2. data=journal (delayed allocation not supported)
3. data=writeback

data=ordered:

Ext4 commits the metadata blocks the journal. After that, ext4 commits the dirty blocks to the disk.

data=journal:

Ext4 commits the dirty blocks and the corresponding metadata blocks to the journal before commiting them to the disk.

data=writeback:

It is similar to the data=ordered mode. The difference is that ext4 may commit the dirty blocks to the disk and after that, ext4 may commit the corresponding metadata blocks to the journal.

How can I see the size and number of dirty blocks?

Dirty: Total memory size waiting to be written to disk.
nr_dirty: Number of dirty blocks in memory waiting to be written to disk
pagesize: Contiguous block space of memory in bytes.

38036 KB / 9509 blocks = 4 KB (4096 bytes)

P.S.: You may not get exactly 4 (your page size) when you divide the dirty block space size by the number of the dirty blocks. It should be closer to your page size though. e.g.: 4.01, 4.02

You can flush the dirty blocks to the disk with the sync command:

If you need to flush the dirty blocks to the disk in your system program you must call either sys_sync, sys_fsync or sys_fdatasync calls.

How often dirty blocks are getting flushed to the disk?

It depends on several system parameters: dirty_background_ratio, dirty_ratio, dirty_expire_centisecs, dirty_writeback_centisecs. (see each parameter’s description here)

Here is an example of the dirty_expire_centisecs system value for allocating blocks:

I set the value of dirty_expire_centisecs 1000 (10 seconds):

Dirty data blocks are flushed to disk after executing the sync command, even if the value of dirty_expire_centisecs is 3000 (30 seconds),

What is fs-verity?

fs-verity is something supported by ext4 and I really like it. Let’s say you are running a SIEM application and you want to sign the log files and make log files read-only. fs-verity may help you in this scenario.

fs-verity calculates a hash value based on the file's contents and makes the file a “read-only” file and can’t be disabled.

fs-verity is not enabled by default. You should enable it while creating the filesystem:

mkfs.ext4 -O verity /dev/sdb

Example usage of fs-verity:

The relevant attribute can be seen via lsattr.