Container Filesystem Under The Hood: OverlayFS

Photo by Jubal Kenneth Bernal on Unsplash

Why does Docker need a virtual file system?

  • Because the containers must be isolated from each other. A container must not interfere with another container’s file system.
  • OS images must be downloaded once and used multiple times. Thanks to OverlayFS’ lower/upper directory mechanism, OS images don’t need to be replicated.

The directory structure in OverlayFS

There are four types of directories in OverlayFS: lower directory, upper directory, work directory, and overlay directory (a.k.a. merge directory)

Let’s get our hands dirty

mkdir lower upper workdir overlaymount -t overlay -o lowerdir=lower,upperdir=upper,workdir=workdir none overlayecho "from lower" >> lower/file1.txt
echo "from upper" >> upper/file2.txt
root@adil:~# ls -li overlay/
549319 -rw-r--r—- 1 root root 11 Oct 16 18:45 file1.txt
549320 -rw-r--r—- 1 root root 11 Oct 16 18:45 file2.txt
root@adil:~# ls -li upper/file2.txt
549320 -rw--r--r—- 1 root root 11 Oct 16 18:45 upper/file2.txt
root@adil:~# ls -li lower/file1.txt
549319 -rw-r--r—- 1 root root 11 Oct 16 18:45 lower/file1.txt

Let’s make it a little complicated

root@adil:~# echo "from lower" >> lower/file2.txt
root@adil:~# cat overlay/file2.txt
from upper
root@adil:~# cat overlay/file1.txt
from lower
root@adil:~# rm -rf overlay/file1.txt overlay/file2.txt
root@adil:~# ls -la overlay/
#empty
root@adil:~# ls -la lower/
-rw-r—-r—- 1 root root 11 Oct 16 18:45 file1.txt
-rw-r—-r—- 1 root root 11 Oct 16 19:03 file2.txt
root@adil:~# ls -la upper/
c —-— —-- -—- 3 root root 0, 0 Oct 16 19:08 file1.txt
c —-— —-- -—- 3 root root 0, 0 Oct 16 19:08 file2.txt
root@adil:~# cat upper/file1.txt
cat: upper/file1.txt: No such device or address
root@adil:~# cat upper/file2.txt
cat: upper/file2.txt: No such device or address
root@adil:~# cat lower/file1.txt
from lower
root@adil:~# cat lower/file2.txt
from lower

Let’s run the unlink command in the upper directory

root@adil:~# unlink upper/file1.txt
root@adil:~# unlink upper/file2.txt
root@adil:~# ls -la upper/
#empty
root@adil:~# cat overlay/file1.txt
from lower
root@adil:~# cat overlay/file2.txt
from lower

Let’s create a Docker container

root@adil:~# docker run -dit --name=mycontainer -v /root/test:/root/test -v /run/lock/example:/root/example ubuntu
root@a8cab50df83c:/# node --version
v16.11.1
root@a8cab50df83c:/# touch /root/test/one.txt
var fs = require('fs');fs.rename('/root/test/one.txt', '/root/example/two.txt', (err) => {
if (err) throw err;
console.log('Succeed');
});
root@a8cab50df83c:~# node test.js
/root/test.js:4
if (err) throw err;
^
[Error: EXDEV: cross-device link not permitted, rename '/root/test/one.txt' -> '/root/example/two.txt'] {
errno: -18,
code: 'EXDEV',
syscall: 'rename',
path: '/root/test/one.txt',
dest: '/root/example/two.txt'
root@a8cab50df83c:~# cat test.js
var fs = require('fs');
fs.copyFile('/root/test/one.txt', '/root/example/two.txt', (err) => {
if (err) throw err;
console.log('Copy — Success');
fs.unlink('/root/test/one.txt', (err) => {
if (err) throw err;
console.log('Unlink — Success')
})
});
root@a8cab50df83c:~# ls -l test/one.txt
-rw-r--r-— 1 root root 0 Oct 16 20:04 test/one.txt
root@a8cab50df83c:~# ls -l example/two.txt
ls: cannot access 'example/two.txt': No such file or directory
root@a8cab50df83c:~# node test.js
Copy — Success
Unlink — Success
root@a8cab50df83c:~# ls -l test/one.txt
ls: cannot access 'test/one.txt': No such file or directory
root@a8cab50df83c:~# ls -l example/two.txt
-rw-r--r-— 1 root root 0 Oct 16 20:04 example/two.txt

Some additional information:

You can use multiple lower directories:

mount -t overlay -o lowerdir=lower1:lower2:lower3,upperdir=upper,workdir=workdir none overlay

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store