Examining the Closed Column in ss Command

People are switching from netstat to ss as netstat (net-tools actually) is deprecated.

I see a lot of people are confused by the output of the ss -s command.

The column named Closed is particularly mind-blowing.

Total: 246
TCP:   129 (estab 87, closed 28, orphaned 4, timewait 28)
Transport Total     IP        IPv6
RAW       1        0         1
UDP       6        4         2
TCP       101      94        7
INET      108      98        10
FRAG      0        0         0

This output is from my test server. The test server’s OS is Ubuntu 20.04.1 LTS, Linux Kernel version is 5.4.0 and iproute2 package version is 5.5.0

You may get different output in older/newer Linux servers.

The ss stands for socket statistics.

How does ss collect/calculate socket statistics?

A screenshot from ss’ source code

As you can see, ss calculates the number of closed connections with this simple mathematical operation:

s.tcp_total - (s.tcp4_hashed + s.tcp6_hashed - s.tcp_tws)

Let’s stop here.

ss is actually parsing the output of /proc/net/sockstat and /proc/net/sockstat6:

ss is parsing the output of the /proc/net/sockstat and proc/net/sockstat6

tcp_total is actually “alloc” in the output of /proc/net/sockstat.
tcp4_hashed is actually “inuse” in the output of /proc/net/sockstat
tcp6_hashed is actually “inuse” in the output of /proc/net/sockstat6
tcp_tws is actually “tw” in the output of /proc/net/sockstat

P.S. TWS stands for “timewaits”

So, the output of /proc/net/sockstat and sockstat6 must be consistent with the output of ss -s:

root@adil:~# cat /proc/net/sockstat && echo "---" && cat /proc/net/sockstat6 && echo "---" && ss -s
sockets: used 400
TCP:      inuse 143 orphan 1 tw 38 alloc 247 mem 86
UDP:      inuse 4 mem 3
UDPLITE:  inuse 0
RAW:      inuse 0
FRAG:     inuse 0 memory 0
---
TCP6:     inuse 4
UDP6:     inuse 2
UDPLITE6: inuse 0
RAW6:     inuse 1
FRAG6:    inuse 0 memory 0
---
Total: 400
TCP:   285 (estab 136, closed 138, orphaned 1, timewait 38)
Transport Total     IP        IPv6
RAW       1         0         1
UDP       6         4         2
TCP       147       143       4
INET      154       147       7
FRAG      0         0         0

Let’s manually parse the output of /proc/net/sockstat and sockstat6:

s.tcp_total   = 247
s.tcp4_hashed = 143
s.tcp6_hashed = 4
s.tcp_tws     = 38

Let’s recall the mathematical operation that calculates the number of closed connections:

s.tcp_total - (s.tcp4_hashed + s.tcp6_hashed - s.tcp_tws)

Let’s put the values in the right places:

247 - (143 + 4 - 38)
Result (the number of closed connections): 138

38 out of 138 closed TCP sockets are actually in the TIME_WAIT state.

What about the other 100 TCP sockets?

We know that the closed connections are calculated based on the “alloc” column. So now we need to examine the “alloc” column in the output of /proc/net/sockstat now.

What is the alloc column in the sockstat output?

In the socket statistics, there are 2 types of TCP sockets: allocated and in use.

All TCP socket states are counted as alloc.
All TCP socket states except TCP_CLOSE are counted as inuse.

So, we found out that the other 100 TCP sockets are in TCP_CLOSE state.

A TCP socket can be marked as TCP_CLOSE in many cases. However, I’d like to point out a common scenario.

In TCP socket creation, the required fields are set in the sock_init_data function in the source code

Kernel sets the initial state of a TCP socket as “TCP_CLOSE”

Let’s test it with a simple PHP script:

<?php
$socket = [];
for($i = 0; $i < 100; $i++){
    $socket[$i] = socket_create(AF_INET, SOCK_STREAM, 0);
}
sleep (30);

As you can see, we create 100 sockets and we do nothing else.

The output of ss -s before running this script is as follows:

root@adil:~# ss -s
Total: 145
TCP: 4 (estab 2, closed 0, orphaned 0, timewait 0)
Transport Total   IP   IPv6
RAW       1       0    1
UDP       1       1    0
TCP       4       3    1
INET      6       4    2
FRAG      0       0    0

The output of ss -s changed when I ran the script:

root@adil:~# ss -s
Total: 245
TCP:   104 (estab 2, closed 100, orphaned 0, timewait 0)
Transport Total     IP        IPv6
RAW       1         0         1
UDP       1         1         0
TCP       4         3         1
INET      6         4         2
FRAG      0         0         0

We see that the Kernel creates the TCP socket with the TCP_CLOSE state.

Hence, if the Closed column has a large value and the Timewait column has a low value your application may create TCP sockets and do nothing else.

Note, however, that that the kernel may mark a TCP socket as TCP_CLOSE in many cases. This case is one of those scenarios and the most common.