nm_close() is like close(), but throws an assertion if the input fd is
>=0 and invalid. Passing an invalid (i.e. already closed) fd to
close() is a programming error with potentially catastrophic effects,
as another thread may reuse the closed fd.
We often want to cascade hashing, meaning, to combine the
outcome of various hash functions in a larger hash.
Instead of having each hash function return a guint hash value,
accept a hash state argument. This saves the overhead of initializing
and completing the intermediate hash states.
It also avoids loosing entropy when we reduce the larger hash state
into the intermediate guint hash value.
By using a macro, we don't cast all the types to guint. Instead,
we use their native types directly. Hence, we don't need
nm_hash_update_uint64() nor nm_hash_update_ptr().
Also, for types smaller then guint like char, we save hashing
the all zero bytes.
siphash24() is wildly used by projects nowadays.
It's certainly slower then our djb hashing that we used before.
But quite likely it's fast enough for us, given how wildly it is
used. I think it would be hard to profile NetworkManager to show
that the performance of hash tables is the issue, be it with
djb or siphash24.
Certainly with siphash24() it's much harder to exploit the hashing
algorithm to cause worst case hash operations (provided that the
seed is kept private). Does this better resistance against a denial
of service matter for us? Probably not, but let's better be safe then
sorry.
Note that systemd's implementation uses a different seed for each hash
table (at least, after the hash table grows to a certain size).
We don't do that and use only one global seed.
The privious NM_HASH_* macros directly operated on a guint value
and were thus close to the actual implementation.
Replace them by adding a NMHashState struct and accessors to
update the hash state. This hides the implementation better
and would allow us to carry more state. For example, we could
switch to siphash24() transparently.
For now, we still do a form basically djb2 hashing, albeit with
differing start seed.
Also add nm_hash_str() and nm_str_hash():
- nm_hash_str() is our own string hashing implementation
- nm_str_hash() is our own string implementation, but with a
GHashFunc signature, suitable to pass it to g_hash_table_new().
Also, it has this name in order to remind you of g_str_hash(),
which it is replacing.
"nm-utils/nm-shared-utils.h" shall contain utility function without other
dependencies. It is intended to be used by other projects as-is.
nm_utils_random_bytes() requires getrandom() and a HAVE_GETRANDOM configure
check. That makes it more cumbersome to re-use "nm-shared-utils.h", in
cases where you don't care about nm_utils_random_bytes().
Split nm_utils_random_bytes() out to a separate file.
Same for hash utils, which depend on nm_utils_random_bytes(). Also, hash
utils will eventually be extended to use siphash24.
This makes hashing non-deterministic with the aim to
make it harder to exploit hash collisions.
Non-deterministic also means that for unit testing
we will get different values on each run. But since we
shall never assign any meaning to these hash values
nor rely on them being stable between restarts (or
upgrades), that doesn't hurt.
Introduce a NM_HASH_INIT() function. It makes the places
where we initialize a hash with a certain seed visually clear.
Also, move them from "shared/nm-utils/nm-shared-utils.h" to
"shared/nm-utils/nm-macros-internal.h". We might want to
have NM_HASH_INIT() non-inline (hence, define it in the
source file).
Add a new function nm_utils_random_bytes().
This function now preferably uses getrandom() syscall if it is
available.
As fallback, it always tries to fill the buffer from /dev/urandom.
If it cannot, as last fallback it uses GRand, which cannot fail.
Hence, the function always sets some (pseudo) random bytes.
It also returns FALSE if the obtained bytes are possibly not good
randomness.
Fixes the following:
shared/nm-utils/nm-shared-utils.c:136: Warning: NetworkManager: GTK-Doc comment block end token "*/" should not be preceded by comment text:
* Returns: the input buffer with the quoted string. */
We already have nm_strquote_a(). That is useful, but uses alloca(), hence it
is ill suited to be called from a macro, inside a loop, or from a function
that should be inlined.
Instead, add nm_strquote() that has the same purpose but writes to a provided
string buffer.
nm_dedup_multi_obj_ref() is a trivial function, that only uses the field
which is already declared in the same header file. Move it to the header
so that it can be inlined (without LTO).
A replacement for g_strsplit_set(). While g_strsplit_set()
does (n+1) malloc and n slice allocations, this needs
roughtly (O(log(n))) mallocs.
Another difference from g_strsplit_set() is that this function
treats multiple delimiters as one (and thus never returns empty
words). While I can see that sometimes you may want to keep empty
words (like parsing a CSV file and preserve empty cells), we usually
use this function for splitting user input. In such case, we want
to treat multiple delimiters as one.
Kernel does not allow to add IPv6 routes with "src", as long as the
corresponding address is still tentative (related bug rh#1457196).
The workaround for this is cumbersome. First, when we fail to add such a
route with "pref_src", we guess that it happend due to this issue. In
that case, nm_ip6_config_commit() returns the list of routes that could
not be added for the moment (but hopefully can be added later).
We track this list in NMDevice, and keep trying to merge the routes
back into ip6_config. In order to not try indefinitely, keep track of a
timestamp when we tried to add this route for the first time.
Another uglyness is that pending tentative routes don't explicitly block
activation. In practice they may do, because for these routes we also have
an IPv6 address that is still doing DAD, so the IP configuration is
still pending due to that.
https://bugzilla.redhat.com/show_bug.cgi?id=1452684
- nm_clear_g_object() is like g_clear_object() but:
- it returns a boolean value, indicating that something was cleared.
- it includes an nm_assert() to check that the pointer is still
valid.
- it uses typeof() instead of blindly casting the argument.
- nm_g_object_ref_set() combines nm_clear_g_object() and resetting
the pointer to a new object, including taking a reference.
- also returns a boolean, indicating whether something changed.
- it gets the order of operations right: first it increses the
ref-count, before unrefing the old object.
- like nm_clear_g_object() and nm_clear_g_free() it first sets
the destination to NULL, instead of leaving a dangling pointer
for the duraction of the unref/free call.
- fix nm_clear_g_free() not to use a possibly dangling pointer.
Striclty speaking, that is undefined behavior.
And relax the type for nm_auto_unref_gtypeclass macro. Like
g_type_class_unref() itself, you usually don't use it with a GTypeClass
base class, but some subtype like GObjectClass.
For RFC1918 private IPv4addresses, guess a better prefix length for
addresses and routes.
nmtui is an interactive program. It makes sense to be a bit smarter
about what the user probably meant.
It would be nice if nmtui would update the entry field immediately when
the cursor leaves the field, to show the guessed prefix length. However,
that is not easily possible, so lets to that another time.
For IPv6 addresses, default to /64 instead of /128.
https://bugzilla.redhat.com/show_bug.cgi?id=1474295
We rely on clearing the dirty flag. For example in nm_ip4_config_replace(),
we first mark all entries dirty, then force-append the ones we keep,
and finally remove the ones that are still dirty.
Since _nm_ip_config_add_obj() short-cuts nm_dedup_multi_index_add_full(),
it must clear the dirty flag on its own.
Compiler wouldn't recognize that the @route/@address argument is always
initialized. The right workaround seems to let the next() functions always
set the value.
In file included from src/nm-ip6-config.c:24:0:
src/nm-ip6-config.c: In function ‘nm_ip6_config_create_setting’:
src/nm-ip6-config.c:734:62: error: the address of ‘address’ will always evaluate as ‘true’ [-Werror=address]
nm_ip_config_iter_ip6_address_for_each (&ipconf_iter, self, &address) {
^
src/nm-ip6-config.h:60:17: note: in definition of macro ‘nm_ip_config_iter_ip6_address_for_each’
for (({ if (address) { *(((const NMPlatformIP6Address **) address)) = NULL; } }), nm_ip_config_iter_ip6_address_init ((iter), (self)); \
^
Fixes: 6e9aa9402a
Previously, we would add exclusive routes via netlink message flags
NLM_F_CREATE | NLM_F_REPLACE for RTM_NEWROUTE. Similar to `ip route replace`.
Using that form of RTM_NEWROUTE message, we could only add a certain
route with a certain network/plen,metric triple once. That was already
hugely inconvenient, because
- when configuring routes, multiple (managed) interfaces may get
conflicting routes (multihoming). Only one of the routes can be actually
configured using `ip route replace`, so we need to track routes that are
currently shadowed.
- when configuring routes, we might replace externally configured
routes on unmanaged interfaces. We should not interfere with such
routes.
That was worked around by having NMRouteManager (and NMDefaultRouteManager).
NMRouteManager would keep a list of the routes which NetworkManager would like
to configure, even if momentarily being unable to do so due to conflicting routes.
This worked mostly well but was complicated. It involved bumping metrics to
avoid conflicts for device routes, as we might require them for gateway routes.
Drop that now. Instead, use the corresponding of `ip route append` to configure
routes. This allows NetworkManager to confiure (almost) all routes that we care.
Especially, it can configure all routes on a managed interface, without
replacing/interfering with routes on other interfaces. Hence, NMRouteManager
becomes obsolete.
It practice it is a bit more complicated because:
- when adding an IPv4 address, kernel will automatically create a device route
for the subnet. We should avoid that by using the IFA_F_NOPREFIXROUTE flag for
IPv4 addresses (still to-do). But as kernel may not support that flag for IPv4
addresses yet (and we don't require such a kernel yet), we still need functionality
similar to nm_route_manager_ip4_route_register_device_route_purge_list().
This functionality is now handled via nm_platform_ip4_dev_route_blacklist_set().
- trying to configure an IPv6 route with a source address will be rejected
by kernel as long as the address is tentative (see related bug rh#1457196).
Preferably, NMDevice would keep the list of routes which should be configured,
while kernel would have the list of what actually is configured. There is a
feed-back loop where both affect each other (for example, when externally deleting
a route, NMDevice must forget about it too). Previously, NMRouteManager would have
the task of remembering all routes which we currently want to configure, but cannot
due to conflicting routes.
We get rid of that, because now we configure non-exclusive routes. We however still
will need to remember IPv6 routes with a source address, that currently cannot be
configured yet. Hence, we will need to keep track of routes that
currently cannot be configured, but later may be.
That is still not done yet, as NMRouteManager didn't handle this
correctly either.
For completeness of the API. remove_obj() is basically a shortcut
of nm_dedup_multi_index_lookup_obj() combined with
nm_dedup_multi_index_remove_entry(). As such, it is useful to return
the actually deleted object. Note that the lookup needle @obj is not
necessarily the same instance as the one that will be removed, it's
only an instance that compares equal according to the index's equality
operator.
The return value shall indicate whether the add-call changed anything.
Reordering shall count as a change too.
On the other hand, clearing the dirty flag of the entry does not count
as a change.
Until now, NetworkManager's platform cache for routes used the quadruple
network/plen,metric,ifindex for equaliy. That is not kernel's
understanding of how routes behave. For example, with `ip route append`
you can add two IPv4 routes that only differ by their gateway. To
the previous form of platform cache, these two routes would wrongly
look identical, as the cache could not contain both routes. This also
easily leads to cache-inconsistencies.
Now that we have NM_PLATFORM_IP_ROUTE_CMP_TYPE_ID, fix the route's
compare operator to match kernel's.
Well, not entirely. Kernel understands more properties for routes then
NetworkManager. Some of these properties may also be part of the ID according
to kernel. To NetworkManager such routes would still look identical as
they only differ in a property that is not understood. This can still
cause cache-inconsistencies. The only fix here is to add support for
all these properties in NetworkManager as well. However, it's less serious,
because with this commit we support several of the more important properties.
See also the related bug rh#1337855 for kernel.
Another difficulty is that `ip route replace` and `ip route change`
changes an existing route. The replaced route has the same
NM_PLATFORM_IP_ROUTE_CMP_TYPE_WEAK_ID, but differ in the actual
NM_PLATFORM_IP_ROUTE_CMP_TYPE_ID:
# ip -d -4 route show dev v
# ip monitor route &
# ip route add 192.168.5.0/24 dev v
192.168.5.0/24 dev v scope link
# ip route change 192.168.5.0/24 dev v scope 10
192.168.5.0/24 dev v scope 10
# ip -d -4 route show dev v
unicast 192.168.5.0/24 proto boot scope 10
Note that we only got one RTM_NEWROUTE message, although from NMPCache's
point of view, a new route (with a particular ID) was added and another
route (with a different ID) was deleted. The cumbersome workaround is,
to keep an ordered list of the routes, and figure out which route was
replaced in response to an RTM_NEWROUTE. In absence of bugs, this should
work fine. However, as we only rely on events, we might wrongly
introduce a cache-inconsistancy as well. See the related bug rh#1337860.
Also drop nm_platform_ip4_route_get() and the like. The ID of routes
is complex, so it makes little sense to look up a route directly.
Default g_log() logs to stdout for INFO level and higher, but logs to stderr
for DEBUG/TRACE. That is annoying, because especially when redirecting the streams,
the messages get mixed up. Install a log handler and just print to stdout for
the tests.
Reasons:
- it adds an O(1) lookup index for accessing NMIPxConfig's addresses.
Hence, operations like merge/intersect have now runtime O(n) instead
of O(n^2).
Arguably, we expect low numbers of addresses in general. For low
numbers, the O(n^2) doesn't matter and quite likely in those cases
the previous implementation was just fine -- maybe even faster.
But the simple case works fine either way. It's important to scale
well in the exceptional case.
- the tracked objects can be shared between the various NMPI4Config,
NMIP6Config instances with NMPlatform and everybody else.
- the NMPObject can be treated generically, meaning it enables code to
handle both IPv4 and IPv6, or addresses and routes. See for example
_nm_ip_config_add_obj().
- I want core to evolve to somewhere where we don't keep copies of
NMPlatformIP4Address, et al. instances. Instead they shall all be
shared. I hope this will reduce memory consumption (although tracking a
reference consumes some memory too). Also, it shortcuts nmp_object_equal()
when comparing the same object. Calling nmp_object_equal() on the
identical objects would be a common case after the hash function
pre-evaluates equality.
