colin/nixpkgs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

nixos/modules/system/boot/networkd: enable socket activation

Browse Source 
Since cd1dedac67 systemd-networkd has it's
netlink socket created via a systemd.socket unit. One might think that
this doesn't make much sense since networkd is just going to create it's
own socket on startup anyway. The difference here is that we have
configuration-time control over things like socket buffer sizes vs
compile-time constants.

For larger setups where networkd has to create a lot of (virtual)
devices the default buffer size of currently 128MB is not enough.

A good example is a machine with >100 virtual interfaces (e.g.,
wireguard tunnels, VLANs, …) that all have to be brought up during
startup. The receive buffer size will spike due to all the generated
message from the new interfaces. Eventually some of the message will be
dropped since there is not enough (permitted) buffer space available.

By having networkd start through / with a netlink socket created by
systemd we can configure the `ReceiveBufferSize` parameter in the socket
options without recompiling networkd.

Since the actual memory requirements depend on hardware, timing, exact
configurations etc. it isn't currently possible to infer a good default
from within the NixOS module system. Administrators are advised to
monitor the logs of systemd-networkd for `rtnl: kernel receive buffer
overrun` spam and increase the memory as required.

Note: Increasing the ReceiveBufferSize doesn't allocate any memory.  It
just increases the upper bound on the kernel side. The memory allocation
depends on the amount of messages that are queued on the kernel side of
the netlink socket.
This commit is contained in:
Andreas Rammhold 2020-05-28 01:07:30 +02:00 committed by Florian Klink
parent 6d95cf3de4
commit 55c09a884a
1 changed files with 32 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

32
nixos/doc/manual/release-notes/rl-2009.xml
View File

@ -566,6 +566,38 @@ systemd.services.nginx.serviceConfig.ReadWritePaths = [ "/var/www" ];
was removed, as udev gained native support to handle FIDO security tokens.
</para>
</listitem>
<listitem>
<para>
With this release <literal>systemd-networkd</literal> (when enabled through <xref linkend="opt-networking.useNetworkd"/>)
has it's netlink socket created through a <literal>systemd.socket</literal> unit. This gives us control over
socket buffer sizes and other parameters. For larger setups where networkd has to create a lot of (virtual)
devices the default buffer size (currently 128MB) is not enough.
</para>
<para>
On a machine with &gt;100 virtual interfaces (e.g., wireguard tunnels, VLANs, …), that all have to
be brought up during system startup, the receive buffer size will spike for a brief period.
Eventually some of the message will be dropped since there is not enough (permitted) buffer
space available.
</para>
<para>
By having <literal>systemd-networkd</literal> start with a netlink socket created by
<literal>systemd</literal> we can configure the <literal>ReceiveBufferSize=</literal> parameter
in the socket options (i.e. <literal>systemd.sockets.systemd-networkd.socketOptions.ReceiveBufferSize</literal>)
without recompiling <literal>systemd-networkd</literal>.
</para>
<para>
Since the actual memory requirements depend on hardware, timing, exact
configurations etc. it isn't currently possible to infer a good default
from within the NixOS module system. Administrators are advised to
monitor the logs of <literal>systemd-networkd</literal> for <literal>rtnl: kernel receive buffer
overrun</literal> spam and increase the memory limit as they see fit.
</para>
<para>
Note: Increasing the <literal>ReceiveBufferSize=</literal> doesn't allocate any memory. It just increases
the upper bound on the kernel side. The memory allocation depends on the amount of messages that are
queued on the kernel side of the netlink socket.
</para>
</listitem>
</itemizedlist>
</section>
</section>