Background
==========
Imagine you run a container on your machine. Then the routing table
might look like:
default via 10.0.10.1 dev eth0 proto dhcp metric 100
10.0.10.0/28 dev eth0 proto kernel scope link src 10.0.10.5 metric 100
[...]
10.42.0.0/24 via 10.42.0.0 dev flannel.1 onlink
10.42.1.2 dev cali02ad7e68ce1 scope link
10.42.1.3 dev cali8fcecf5aaff scope link
10.42.2.0/24 via 10.42.2.0 dev flannel.1 onlink
10.42.3.0/24 via 10.42.3.0 dev flannel.1 onlink
That is, there are another interfaces with subnets and specific routes.
If nm-cloud-setup now configures rules:
0: from all lookup local
30400: from 10.0.10.5 lookup 30400
32766: from all lookup main
32767: from all lookup default
and
default via 10.0.10.1 dev eth0 table 30400 proto static metric 10
10.0.10.1 dev eth0 table 30400 proto static scope link metric 10
then these other subnets will also be reached via the default route.
This container example is just one case where this is a problem. In
general, if you have specific routes on another interface, then the
default route in the 30400+ table will interfere badly.
The idea of nm-cloud-setup is to automatically configure the network for
secondary IP addresses. When the user has special requirements, then
they should disable nm-cloud-setup and configure whatever they want.
But the container use case is popular and important. It is not something
where the user actively configures the network. This case needs to work better,
out of the box. In general, nm-cloud-setup should work better with the
existing network configuration.
Change
======
Add new routing tables 30200+ with the individual subnets of the
interface:
10.0.10.0/24 dev eth0 table 30200 proto static metric 10
[...]
default via 10.0.10.1 dev eth0 table 30400 proto static metric 10
10.0.10.1 dev eth0 table 30400 proto static scope link metric 10
Also add more important routing rules with priority 30200+, which select
these tables based on the source address:
30200: from 10.0.10.5 lookup 30200
These will do source based routing for the subnets on these
interfaces.
Then, add a rule with priority 30350
30350: lookup main suppress_prefixlength 0
which processes the routes from the main table, but ignores the default
routes. 30350 was chosen, because it's in between the rules 30200+ and
30400+, leaving a range for the user to configure their own rules.
Then, as before, the rules 30400+ again look at the corresponding 30400+
table, to find a default route.
Finally, process the main table again, this time honoring the default
route. That is for packets that have a different source address.
This change means that the source based routing is used for the
subnets that are configured on the interface and for the default route.
Whereas, if there are any more specific routes in the main table, they will
be preferred over the default route.
Apparently Amazon Linux solves this differently, by not configuring a
routing table for addresses on interface "eth0". That might be an
alternative, but it's not clear to me what is special about eth0 to
warrant this treatment. It also would imply that we somehow recognize
this primary interface. In practise that would be doable by selecting
the interface with "iface_idx" zero.
Instead choose this approach. This is remotely similar to what WireGuard does
for configuring the default route ([1]), however WireGuard uses fwmark to match
the packets instead of the source address.
[1] https://www.wireguard.com/netns/#improved-rule-based-routing
Configuration can have [device*] and [connection*] settings and both
can include a 'match-device=' key, which is a list of device-specs.
Introduce a new 'allowed-connections' key for [device*] sections,
which specifies a list of connection-specs to indicate which
connections can be activated on the device.
With this, it becomes possible to have a device configuration like:
[device-enp1s0]
match-device=interface-name:enp1s0
allowed-connections=except:origin:nm-initrd-generator
so that NM in the real root ignores connections created by the
nm-initrd-generator, and starts activating a persistent
connection. This requires also setting 'keep-configuration=no' to not
generate an assumed connection.
Add function nm_utils_connection_match_spec_list() to check whether a
connection matches a spec list. Also document the supported syntax in
the man page.
Add a new 'keep-configuration' device option, set to 'yes' by
default. When set to 'no', on startup NetworkManager ignores that the
interface is pre-configured and doesn't try to keep its
configuration. Instead, it activates one of the persistent
connections.
nmcli now accepts `nmcli device up|down` which works the same way as
`nmcli device connect|disconnect`
I also edited man pages of nmcli with new options.
`description-docbook` is the alternative tag to `description`, the
difference is that `description-docbook` expects docbook XML but not
plaintext.
Signed-off-by: Wen Liang <liangwen12year@gmail.com>
Since the [main].iwd-config-path functionality, where NM watches for
NMSettingsConnection changes and update IWD network config files with
new settings, has proven to work without issues so far, enable it by
default. Instead of hardcoding /var/lib/iwd as the value, and since the
value can't be probed at NM compile time, query it from IWD's recently-
added D-Bus interface for settings when [main].iwd-config-path is either
missing or set to the new value "auto".
If the configuration contains dns=none and resolv.conf is updated
through a dispatcher script, currently there is no way to tell NM that
the content of resolv.conf changed, so that it can restart a hostname
resolution.
Use SIGUSR1 (and SIGHUP) for that.
The code never set "iface_get_config->cidr_addr", despite
setting "cidr_prefix" and "has_cidr". As a result, cloud-setup
would think that the subnet is "0.0.0.0/$PLEN", and calculate
the gateway as "0.0.0.1".
As a result it would add a default route to table 30400 via 0.0.0.1,
which is obviously wrong.
How to detect the right gateway? Let's try obtain the subnet also via
the meta data. That seems mostly correct, except that we only access
subnet at index 0. What if there are multiple ones? I don't know.
https://bugzilla.redhat.com/show_bug.cgi?id=1912236
Watch for NMSettingConnection changes and creation signals and convert
them to IWD format and write them to the configured IWD profile storage
directory. The logic is off by default and gets enabled when the new
iwd-config-path setting in nm.conf's [main] group is set to a path to
an existing directory.
The idea here is that when a user edits an NM connection profile, the
change is immediately mirrored in IWD since IWD watches its
configuration directory using inotify. This way NM clients can be used
to edit 802.1x settings, the PSK passphrase or the SSID -- changes that
would previously not take effect with the IWD backend.
Some precautions are taken to not make connections owned by a user
available to other users, such connections are not converted at all.
In all other cases where a connection cannot be converted sufficiently
well to the IWD format, for various reasons, we also give up and not
mirror these connections.
Due to IWD limitations and design differences with NM this logic has
many problems where it may not do its task properly. It's meant to work
on a best-effort and "better than nothing" basis, but it should be safe
in that it shouldn't delete users data or reveal secrets, etc. The most
obvious limitation is that there can be multiple NM connections
referring to the same SSID+Security tuple and only one IWD profile can
exist because the filename is based on only the SSID+Security type. We
already had one NM connection selected for each IWD KnownNetwork and
referenced by a pointer, so we ignore changes in NM connections other
than that selected one.
Add support for `carrier-wait-timeout` setting from kernel cmdline.
This will create a new `15-carrier-timeout.conf` file in
/run/NetworkManager/conf.d with the parameter value as specified.
The setting also inserts `match-device` to `*`, matching all devices.
NB: The parameter on kernel cmdline is specified in seconds. This is
done to be backwards compatible with with network-legacy module. However
the generated setting will automatically multiply specified value by
1000 and store timeout value in ms.
https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/issues/626https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/730
It's questionable whether the manual page should explain
exactly what it does.
However, it's a good exercise writing this up (to review
what happens). Also, a manual page that simply says "it configures
the network automatically" without going into how exactly, is
not very useful either.
NetworkManager is now able to configure veth interfaces throught the
NMSettingVeth. Veth interfaces only have "peer" property.
In order to support Veth interfaces in NetworkManager the design need
to pass the following requirements:
* Veth setting only has "peer" attribute.
* Ethernet profiles must be applicable to Veth interfaces.
* When creating a veth interface, the peer will be managed by
NetworkManager but will not have a profile.
* Veth connection can reapply only if the peer has not been modified.
* In order to modify the veth peer, NetworkManager must deactivate the
connection and create a new one with peer modified.
In general, it should support the basis of veth interfaces but without
breaking any existing feature or use case. The users that are using veth
interfaces as ethernet should not notice anything changed unless they
specified the veth peer setting.
Creating a Veth interface in NetworkManager is useful even without the
support for namespaces for some use cases, e.g "connecting one side of
the veth to an OVS bridge and the other side to a Linux bridge" this is
done when using OVN kubernetes [1][2]. In addition, it would provide
persistent configuration and rollback support for Veth interfaces.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1885605
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1894139
Signed-off-by: Fernando Fernandez Mancera <ffmancera@riseup.net>
If this setting it true (or missing) we skip most of the D-Bus
Disconnect() calls whoe purpose was to keep IWD's internal autoconnect
mechanism always disabled. We use the IWD's Station.State property
updates, and secrets requets through our IWD agent, to find out when IWD
is trying to connect and create "assumed" activations on the NM side to
mirror the IWD state. This is quite complicated due to the many
possible combinations of NMDevice's state and IWD's state. A lot of
them are "impossible" but we try to be careful to consider all the
different possibilities.
NM has a nice API for "assuming connections" but it's designed for
slightly different use cases than what we have here and for now we
created normal "managed"-type activations when assuming an IWD automatic
connection.
