Reasons:
- it adds an O(1) lookup index for accessing NMIPxConfig's addresses.
Hence, operations like merge/intersect have now runtime O(n) instead
of O(n^2).
Arguably, we expect low numbers of addresses in general. For low
numbers, the O(n^2) doesn't matter and quite likely in those cases
the previous implementation was just fine -- maybe even faster.
But the simple case works fine either way. It's important to scale
well in the exceptional case.
- the tracked objects can be shared between the various NMPI4Config,
NMIP6Config instances with NMPlatform and everybody else.
- the NMPObject can be treated generically, meaning it enables code to
handle both IPv4 and IPv6, or addresses and routes. See for example
_nm_ip_config_add_obj().
- I want core to evolve to somewhere where we don't keep copies of
NMPlatformIP4Address, et al. instances. Instead they shall all be
shared. I hope this will reduce memory consumption (although tracking a
reference consumes some memory too). Also, it shortcuts nmp_object_equal()
when comparing the same object. Calling nmp_object_equal() on the
identical objects would be a common case after the hash function
pre-evaluates equality.
Add a stable, recursive merge sort for CList.
This could be improved by doing an iterative implementation.
The recursive implementation's stack depth is not an issue,
as it is bound by O(ln(n)). But an iterative implementation
would safe the overhead of O(n*log(n)) function calls and be
potentially faster.
And get rid of the unused obj_full_equality_allows_different_class.
It's hard to grasp how to implement different object types that can compare
despite having different klasses. The idea was, that stack allocated
objects (used as lookup needles), are some small lightweight objects,
that still compare equal to the full instance. But it's unused. Drop it.
by moving the core functionality to "nm-dedup-multi.c".
As the ref-counting mechanism now is part of "nm-dedup-multi.c",
this works better and is reusable outside of platform.
Implement the reference counting of NMPObject as part of
NMDedupMultiObj and get rid of NMDedupMultiBox.
With this change, the NMPObject is aware in which NMDedupMultiIndex
instance it is tracked.
- this saves an additional GSlice allocation for the NMDedupMultiBox.
- it is immediately known, whether an NMPObject is tracked by a
certain NMDedupMultiIndex or not. This saves an additional hash
lookup.
- previously, when all idx-types cease to reference an NMDedupMultiObj
instance, it was removed. Now, a tracked objects stays in the
NMDedupMultiIndex until it's last reference is deleted. This possibly
extends the lifetime of the object and we may reuse it better.
- it is no longer possible to add one object to more then one
NMDedupMultiIndex instance. As we anyway want to have only one
instance to deduplicate the objects, this is fine.
- the ref-counting implementation is now part of NMDedupMultiObj.
Previously, NMDedupMultiIndex could also track objects that were
not ref-counted. Hoever, the object anyway *must* implement the
NMDedupMultiObj API, so this flexibility is unneeded and was not
used.
- a downside is, that NMPObject grows by one pointer size, even if
it isn't tracked in the NMDedupMultiIndex. But we really want to
put all objects into the index for sharing and deduplication. So
this downside should be acceptable. Still, code like
nmp_object_stackinit*() needs to handle a larger object.
Add the NMDedupMultiIndex cache. It basically tracks
objects as doubly linked list. With the addition that
each object and the list head is indexed by a hash table.
Also, it supports tracking multiple distinct lists,
all indexed by the idx-type instance.
It also deduplicates the tracked objects and shares them.
- the objects that can be put into the cache must be immutable
and ref-counted. That is, the cache will deduplicate them
and share the reference. Also, as these objects are immutable
and ref-counted, it is safe that users outside the cache
own them too (as long as they keep them immutable and manage
their reference properly).
The deduplication uses obj_id_hash_func() and obj_id_equal_func().
These functions must cover *every* aspect of the objects when
comparing equality. For example nm_platform_ip4_route_cmp()
would be a function that qualifies as obj_id_equal_func().
The cache creates references to the objects as needed and
gives them back. This happens via obj_get_ref() and
obj_put_ref(). Note that obj_get_ref() is free to create
a new object, for example to convert a stack-allocated object
to a (ref-counted) heap allocated one.
The deduplication process creates NMDedupIndexBox instances
which are the ref-counted entity. In principle, the objects
themself don't need to be ref-counted as that is handled by
the boxing instance.
- The cache doesn't only do deduplication. It is a multi-index,
meaning, callers add objects using a index handle NMDedupMultiIdxType.
The NMDedupMultiIdxType instance is the access handle to lookup
the list and objects inside the cache. Note that the idx-type
instance may partition the objects in distinct lists.
For all operations there are cross-references and hash table lookups.
Hence, every operation of this data structure is O(1) and the memory
overhead for an index tracking an object is constant.
The cache preserves ordering (due to linked list) and exposes the list
as public API. This allows users to iterate the list without any
additional copying of elements.
Platform has it's own, simple implementation of object types:
NMPObject. Extract a base type and move it to "shared/nm-utils/nm-obj.h"
so it can be reused.
The base type is trival, but it allows us to implement other objects
which are compatible with NMPObjects. Currently there is no API for generic
NMObjBaseInst type, so compatible in this case only means, that they
can be used in the same context (see example below).
The only thing that you can do with a NMObjBaseInst is check it's
NMObjBaseClass.
Incidentally, NMObjBaseInst is also made compatible to GTypeInstance.
It means, an NMObjBaseInst is not necessarily a valid GTypeInstance (like NMPObject
is not), but it could be implemented as such.
For example, you could do:
if (NMP_CLASS_IS_VALID ((NMPClass *) obj->klass)) {
/* is an NMPObject */
} else if (G_TYPE_CHECK_INSTANCE_TYPE (obj, NM_TYPE_SOMETHING)) {
/* it a NMSometing GType */
} else {
/* something else? */
}
The reason why NMPObject is not implemented as proper GTypeInstance is
because it would require us to register a GType (like
g_type_register_fundamental). However, then the NMPClass struct can
no longer be const and immutable memory. But we could.
NMObjBaseInst may or may not be a GTypeInstance. In a sense, it's
a base type of GTypeInstance and all our objects should be based
on it (optionally, they we may make them valid GTypes too).
Returning TRUE for zero makes no sense. Obviously, zero is not a power
of two.
Also, the function is used to check whether a number has only one bit
(flag) set, so, an alternative name would be "has-one-bit-set", which
also should return FALSE for zero. All callers didn't really care for
the previous meaning "has-at-most-one-bit-set".
This also avoids the issue of checking (x >= 0), which causes
-Wtype-limits warnings for unsigned types. Which was avoided
by doing (x == 0 || x > 0), which caused -Wlogical-op warning,
which then was avoided (x == 0 || (x > 0 && 1)). Just don't.
We recently added -Wlogical-op in our build process
(commit #41e7fca59762dc928c9d67b555b1409c3477b2b0).
Seems that old versions of gcc (4.8.x) will hit that warning with our
implementation of our "nm_utils_is_power_of_two" and
"test_nm_utils_is_power_of_two_do" macros.
Fool it just adding an always TRUE check.
Use C-style backslash escaping to sanitize non-UTF-8 strings.
The functions are compatible with glib's g_strcompress() and
g_strescape().
The difference is only that g_strescape() escapes all non-printable,
non ASCII character as well, while nm_utils_str_utf8safe_escape()
-- depending on the flags -- preserves valid UTF-8 sequence except
backslash.
The flags allow to optionally escape ASCII control characters and
all non-ASCII (valid UTF-8) characters. But the option to preserve
valid UTF-8 (non-ASCII) characters verbatim, is what distinguishes
from g_strescape().
UDev never creates such invalid escape sequences. Anyway,
we cannot accept a NUL character at this point. Just take
the ill escape verbatim -- it should never happen anyway.
Include the circular, doubly-linked list implementation from
c-util/c-list [1], commit 864051de6e7e1c93c782064fbe3a86b4c17ac466.
[1] https://github.com/c-util/c-list
CC shared/nm-utils/libnm_core_libnm_core_la-nm-udev-utils.lo
In file included from ./shared/nm-utils/nm-glib.h:27:0,
from ./shared/nm-utils/nm-macros-internal.h:29,
from ./shared/nm-default.h:178,
from shared/nm-utils/nm-udev-utils.c:21:
shared/nm-utils/nm-udev-utils.c: In function ‘nm_udev_client_enumerate_new’:
./shared/nm-utils/gsystem-local-alloc.h:53:50: error: ‘to_free’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
GS_DEFINE_CLEANUP_FUNCTION(void*, gs_local_free, g_free)
^~~~~~
shared/nm-utils/nm-udev-utils.c:147:18: note: ‘to_free’ was declared here
gs_free char *to_free;
^~~~~~~
In file included from ./shared/nm-utils/nm-glib.h:27:0,
from ./shared/nm-utils/nm-macros-internal.h:29,
from ./shared/nm-default.h:178,
from shared/nm-utils/nm-udev-utils.c:21:
shared/nm-utils/nm-udev-utils.c: In function ‘nm_udev_client_new’:
./shared/nm-utils/gsystem-local-alloc.h:53:50: error: ‘to_free’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
GS_DEFINE_CLEANUP_FUNCTION(void*, gs_local_free, g_free)
^~~~~~
shared/nm-utils/nm-udev-utils.c:243:20: note: ‘to_free’ was declared here
gs_free char *to_free;
^~~~~~~
Fixes: e32839838e
I used to use g_strv_length ((char **) p) instead, but that feels
ugly because it g_strv_length() is not designed to operate on
arbitrary pointer arrays.
Commit a8730c51c8 moved the enum
utils from libnm-core to shared/nm-utils.
However, three of those functions are part of public API in libnm.
So, when statically linking against "shared/nm-utils/nm-enum-utils.c"
and dynamically linking against libnm.so, those symbols are present
twice and cause a linker failure.
Fix that by moving the public API back to libnm-core.
Fixes: a8730c51c8
libnm contains the public function nm_utils_enum_from_str() et al.
The function is not flexible enough for nmcli's usecase. So, I would
need another public function like nm_utils_enum_from_str_full() that
has an extended API.
That was already required previously for ifcfg-rh writer, but in that
case I could just add it as internal API as libnm-core is linked statically
with NetworkManager.
I don't want to commit to a public API for an utility function. So move
the code instead to the shared directory, so that nmcli may link
statically against it and use the internal API.
These functions are only used by nm-meta-setting-desc.c. Make them internal.
Unfortunately, they are part of "common.h" which cannot be used without
the rest of nmcli. Still todo.
GUdevClient always creates a monitor instance, even if there are no subsystems
or handlers defined. Hence the first iteration of NMUdevClient did that as
well.
I think that can be avoided however. We only need a monitor when there is
a event handler subscribed. Contrary to GUdevClient, we know that from the
very beginning.
_NM_GET_PRIVATE() macro is used to implement a standard private-getter, but it
requires that "self" is a pointer of either "const type *" or "type *". That
is great in most cases, but sometimes we have predominatly self pointers of
different type, so it would require a lot of casts.
Add a different form _NM_GET_PRIVATE_VOID() where self pointer can be any
non-const pointer and returns a non-const private pointer after casting.
I think NM_CACHED_QUARK_FCN() is better because:
- the implementation is in our hand, meaning it is clear that
putting a "static" before NM_CACHED_QUARK_FCN() is guaranteed to
work -- without relying on G_DEFINE_QUARK() to be defined in a way
that this works (in fact, we currently never do that and instead
make all functions non-static).
- it does not construct function names by appending "_quark".
Thus you can grep for the entire function name and finding
the place where it is implemented.
- same with the stings, where the new macro doesn't stringify the
argument, which is less surpising. Again, now you can grep
for the string including the double quoting.
(yes, I really use grep to understand the source-code)
NM_CACHED_QUARK_FCN() is a replacement for G_DEFINE_QUARK().
G_DEFINE_QUARK() is mostly used to define GError quarks. As
such, it always appends _quark() to the function name, which
is unfavorable because it makes it harder to grep for the
definition of the function.
In general I think that macros that defined symbols by concatenating
something should be avoided because that makes it harder to locate
where the symbol was defined.
