Our own implementation of a string buffer like GString.
Advantages (in decreasing relevance):
- Since we are in control, we can easily let it nm_explicit_bzero()
the memory. The regular GString API cannot be used in such a case.
While nm_explicit_bzero() may or may not be of questionable benefit,
the problem is that if the underlying API counteracts the aim of
clearing memory, it gets impossible. As API like NMStrBuf supports
it, clearing memory is a easy as enable the right flag.
This would for example be useful for example when we read passwords
from a file or file descriptor (e.g. try_spawn_vpn_auth_helper()).
- We have API like
nmp_object_to_string (const NMPObject *obj,
NMPObjectToStringMode to_string_mode,
char *buf,
gsize buf_size);
which accept a fixed size output buffer. This has the problem of
how choosing the right sized buffer. With NMStrBuf such API could
be instead
nmp_object_to_string (const NMPObject *obj,
NMPObjectToStringMode to_string_mode,
NMStrBuf *buf);
which can automatically grow (using heap allocation). It would be
easy to extend NMStrBuf to use a fixed buffer or limiting the
maximum string length. The point is, that the to-string API wouldn't
have to change. Depending on the NMStrBuf passed in, you can fill
an unbounded heap allocated string, a heap allocated string up to
a fixed length, or a static string of fixed length. NMStrBuf currently
only implements the unbounded heap allocate string case, but it would
be simple to extend.
Note that we already have API like nm_utils_strbuf_*() to fill a buffer
of fixed size. GString is not useable for that (efficiently), hence
this API exists. NMStrBuf could be easily extended to replace this API
without usability or performance penalty. So, while this adds one new
API, it could replace other APIs.
- GString always requires a heap allocation for the container. In by far
most of the cases where we use GString, we use it to simply construct
a string dynamically. There is zero use for this overhead. If one
really needs a heap allocated buffer, NMStrBuf can easily embedded
in a malloc'ed memory and boxed that way.
- GString API supports inserting and removing range. We almost never
make use of that. We only require append-only, which is simple to
implement.
- GString needs to NUL terminate the buffer on every append. It
has unnecessary overhead for allowing a usage of where intermediate
buffer contents are valid strings too. That is not the case with
NMStrBuf: the API requires the user to call nm_str_buf_get_str() or
nm_str_buf_finalize(). In most cases, you would only access the string
once at the end, and not while constructing it.
- GString always grows the buffer size by doubling it. I don't think
that is optimal. I don't think there is one optimal approach for how
to grow the buffer, it depends on the usage patterns. However, trying
to make an optimal choice here makes a difference. QT also thinks so,
and I adopted their approach in nm_utils_get_next_realloc_size().
When growing a buffer by appending a previously unknown number
of elements, the often preferable strategy is growing it exponentially,
so that the amortized runtime and re-allocation costs scale linearly.
GString just always increases the buffer length to the next power of
two. That works.
I think there is value in trying to find an optimal next size. Because
while it doesn't matter in terms of asymptotic behavior, in practice
a better choice should make a difference. This is inspired by what QT
does ([1]), to take more care when growing the buffers:
- QString allocates 4 characters at a time until it reaches size 20.
- From 20 to 4084, it advances by doubling the size each time. More
precisely, it advances to the next power of two, minus 12. (Some memory
allocators perform worst when requested exact powers of two, because
they use a few bytes per block for book-keeping.)
- From 4084 on, it advances by blocks of 2048 characters (4096 bytes).
This makes sense because modern operating systems don't copy the entire
data when reallocating a buffer; the physical memory pages are simply
reordered, and only the data on the first and last pages actually needs
to be copied.
Note that a QT is talking about 12 characters, so we use 24 bytes
head room.
[1] https://doc.qt.io/qt-5/containers.html#growth-strategies
Sometimes these function may set errno to unexpected values like EAGAIN.
This causes confusion. Avoid that by using our own wrappers that retry
in that case. For example, in rhbz#1797915 we have failures like:
errno = 0;
v = g_ascii_strtoll ("10", 0, &end);
if (errno != 0)
g_assert_not_reached ();
as g_ascii_strtoll() would return 10, but also set errno to EAGAIN.
Work around that by using wrapper functions that retry. This certainly
should be fixed in glib (or glibc), but the issues are severe enough to
warrant a workaround.
Note that our workarounds are very defensive. We only retry 2 times, if
we get an unexpected errno value. This is in the hope to recover from
a spurious EAGAIN. It won't recover from other errors.
https://bugzilla.redhat.com/show_bug.cgi?id=1797915
Depending on the type, OVS interfaces also have a corresponding netdev
in kernel (e.g. type "internal" does, type "patch" does not).
Such a case is neither NMU_IFACE_OVS nor NMU_IFACE_KERNEL (alone). There should
be a special type to represent those cases.
Add NMU_IFACE_OVS_OR_KERNEL for that.
nm_utils_ifname_valid() is to validate "connection.interface-name"
property. But the exact validation depends on the connection type.
Add "NMU_IFACE_ANY" to validate the name to check whether it would be
valid for any connection type.
This is for completeness and for places where the caller might not know
the connection type.
"all" and "default" never works.
"bonding_masters" works if you unload the bonding module. Well,
that should not really be called working...
Reject these names.
Generally, it's dangerous to reject values that were accepted
previously. This will lead to NetworkManager being unable to load
a profile from disk, which was loadable previously.
On the other hand, kernel would not have treated this setting as
it was intended. So, I would argue that the such a setting was not
working (as intended) anyway.
We can only hope that users don't configure arbitrary interface names.
It generally isn't a good idea to do, so "breaking" such things is less
of a concern.
The interface name might come from the command line or from a filename
(like during nm_vpn_wireguard_import()). It's not clear that this
is valid UTF-8. Asserting against that requires the caller to ensure
that the encoding is valid. That is cumbersome, especially since we anyway
check. Just report a regular error.
quoting 'man ovs-vswitchd.conf.db':
"The name must be alphanumeric and must not contain forward or backward
slashes."
OVS actually accepts a wider range of chars (all printable UTF-8 chars),
NetworkManager restricts this to ASCII char as it's a safer option for
now since OVS is not well documented on this matter.
https://bugzilla.redhat.com/show_bug.cgi?id=1788432
Fixes: e7d72a14f6 ('libnm-core: use different ifname validation function for OVS bridges, ports and interfaces')
The only affected caller is nm_modem_manager_get_modems(), which
is used by NMDeviceBt for DUN connections. This is rather bad.
Fixes: 4154d9618c ('bluetooth: refactor BlueZ handling and let NMBluezManager cache ObjectManager data')
Fixes: e688e70b37 ('shared: add nm_utils_hash_values_to_array() helper')
Most callers would pass FALSE to nm_utils_error_is_cancelled(). That's
not very useful. Split the two functions and have nm_utils_error_is_cancelled()
and nm_utils_error_is_cancelled_is_disposing().
There are crashes where this assertion fails, but it's not clear
how that could happen (because the input text seems a usual IPv4 address).
Try to collect some more information about what failed. It's only
enabled with NM_MORE_ASSERTS anyway.
Related: https://bugzilla.redhat.com/show_bug.cgi?id=1797915
The abbreviations "ns" and "ms" seem not very clear to me. Spell them
out to nsec/msec. Also, in parts we already used the longer abbreviations,
so it wasn't consistent.
inet_aton() also supports IPv4 addresses in octal (with a leading '0')
or where not all 4 digits of the address are present.
Add nm_utils_parse_inaddr_bin_full() to optionally fallback to
parse the address with inet_aton().
Note taht inet_aton() also supports all crazy formats, including
ignoring trailing garbage after a whitespace. We don't want to accept
that in general.
Note that even in legacy format we:
- accept everything that inet_pton() would accept
- additionally, we also accept some forms which inet_aton() would
accept, but not all.
That means, the legacy format that we accept is a superset of
inet_pton() and a subset of inet_aton(). Which is desirable.
We will rework NMClient entirely. Then, the synchronous initialization will also use
the asynchronous code paths. The difference will be that with synchronous initialization,
all D-Bus interaction will be done with an internal GMainContext as current thread default,
and that internal context will run until initialization completes.
Note that even after initialization completes, it cannot be swapped back to the user's
(outer) GMainContext. That is because contexts are essentially the queue for our
D-Bus events, and we cannot swap from one queue to the other in a race
free manner (or a full resync). In other words, the two contexts are not in sync,
so after using the internal context NMClient needs to stick to that (at least, until
the name owner gets lost, which gives an opportunity to resync and switch back to the
user's main context).
We thus need to hook the internal (inner) GMainContext with the user's (outer) context,
so when the user iterates the outer context, events on the inner context get dispatched.
Add nm_utils_g_main_context_create_integrate_source() to create such a GSource for
integrating two contexts.
Note that the use-case here is limited: the integrated, inner main context must
not be explicitly iterated except from being dispatched by the integrating
source. Otherwise, you'd get recursive runs, possible deadlocks and general
ugliness. NMClient must show restrain how to use the inner context while it is
integrated.
Several points.
- We spawn the dnsmasq process directly. That has several downsides:
- The lifetime of the process is tied to NetworkManager's. When
stopping NetworkManager, we usually also stop dnsmasq. Or we keep
the process running, but later the process is no longer a child process
of NetworkManager and we can only kill it using the pidfile.
- We don't do special sandboxing of the dnsmasq process.
- Note that we want to ensure that only one dnsmasq process is running
at any time. We should track that in a singletone. Note that NMDnsDnsmasq
is not a singleton. While there is only one instance active at any time,
the DNS plugin can be swapped (e.g. during SIGHUP). Hence, don't track the
process per-NMDnsDnsmasq instance, but in a global variable "gl_pid".
- Usually, when NetworkManager quits, it also stops the dnsmasq process.
Previously, we would always try to terminate the process based on the
pidfile. That is wrong. Most of the time, NetworkManager spawned the
process itself, as a child process. Hence, the PID is known and NetworkManager
will get a signal when dnsmasq exits. The only moment when NetworkManager should
use the pidfile, is the first time when checking to kill the previous instance.
That is: only once at the beginning, to kill instances that were
intentionally or unintentionally (crash) left running earlier.
This is now done by _gl_pid_kill_external().
- Previously, before starting a new dnsmasq instance we would kill a
possibly already running one, and block while waiting for the process to
disappear. We should never block. Especially, since we afterwards start
the process also in non-blocking way, there is no reason to kill the
existing process in a blocking way. For the most part, starting dnsmasq
is already asynchronous and so should be the killing of the dnsmasq
process.
- Drop GDBusProxy and only use GDBusConnection. It fully suffices.
- When we kill a dnsmasq instance, we actually don't have to wait at
all. That can happen fully in background. The only pecularity is that
when we restart a new instance before the previous instance is killed,
then we must wait for the previous process to terminate first. Also, if
we are about to exit while killing the dnsmasq instance, we must register
nm_shutdown_wait_obj_*() to wait until the process is fully gone.
Coverity says
CID 202453 (#1 of 1): Wrong sizeof argument (SIZEOF_MISMATCH)suspicious_sizeof:
Passing argument user_data of type gconstpointer and argument (gsize)nargs * 8UL /* sizeof (gconstpointer) */ to function g_slice_free1 is suspicious.
Let's pass instead the "data" pointer. It's the same, but maybe that
avoids the warning.
(cherry picked from commit d76df4c139)
Coverity doesn't like us ignoring the return value, although
we really only care about the "p" output pointer.
Try casting the result to (void), maybe that silences Coverity.
(cherry picked from commit e6fa3ce2df)
- don't let no_auto_default_from_file() do any preprocessing of
the lines that it reads. It merely splits the lines at '\n'
and utf8safe-unescapes them.
This was previously duplicated also by NMConfigData's property
setter. We don't need to do it twice.
- sort the lines. This makes the entire handling O(n*ln(n)) instead
of O(n^2). Also, sorting effectively normalizes the content, and
it's desirable to have one true representation of what we write.
In particular when calling nm_utils_strv_sort() with a positive length
argument, then this is not a %NULL terminated strv arrary. That may mean
that it makes sense for the input array to contain %NULL strings.
Use a strcmp() function that accepts %NULL too.
While this is not used at the moment, I think nm_utils_strv_sort()
should accept %NULL strings beause otherwise it's a possibly unexpected
restriction of its API. The function should handle sensible input gracefully.
It is like strcmp(), but has a signature suitable for GCompareDataFunc.
This is necessary with nm_utils_ptrarray_find_binary_search()
to search a sorted strv array.
The fault is here really C, which doesn't allow inline static functions.
So, you need all kinds of slightly different flavors for the same
callbacks (with or without user-data).
Note that glib2 internally just casts strcmp() to GCompareDataFunc ([1]),
relying on the fact how arguments are passed to the function and
ignoring the additional user-data argument. But I find that really
ugly and probably not permissible in general C. Dunno whether POSIX
would guarantee for this to work. I'd rather not do such function
pointer casts.
[1] 0c0cf59858/glib/garray.c (L1792)
Usually we avoid GSList, because I think it's not a great data type.
Anyway, our match-specs are just a GSList of strings, so we need some
API to handle them.
We no longer add these. If you use Emacs, configure it yourself.
Also, due to our "smart-tab" usage the editor anyway does a subpar
job handling our tabs. However, on the upside every user can choose
whatever tab-width he/she prefers. If "smart-tabs" are used properly
(like we do), every tab-width will work.
No manual changes, just ran commands:
F=($(git grep -l -e '-\*-'))
sed '1 { /\/\* *-\*- *[mM]ode.*\*\/$/d }' -i "${F[@]}"
sed '1,4 { /^\(#\|--\|dnl\) *-\*- [mM]ode/d }' -i "${F[@]}"
Check remaining lines with:
git grep -e '-\*-'
The ultimate purpose of this is to cleanup our files and eventually use
SPDX license identifiers. For that, first get rid of the boilerplate lines.
