Commit Graph

30304 Commits

Author SHA1 Message Date
Thomas Haller
b5f3d88e6f glib-aux: add path-utils from systemd
We use these functions, currently from our systemd fork. One day we want
to stop importing systemd code, so we need them ourselves.

Copy them, and adjust for NM style.
2022-04-20 12:07:03 +02:00
Thomas Haller
82cac62fe2 systemd: drop nm_sd_utils_unbase64{char,mem}() wrappers
They are unused now.
2022-04-20 12:07:03 +02:00
Thomas Haller
cdc3e3fa95 libnm: use own nm_unbase64mem_full() instead of systemd's in nm_utils_base64secret_decode() 2022-04-20 12:07:03 +02:00
Thomas Haller
bb0ba779f6 keyfile: use nm_unbase64char() instead of systemd code in _write_setting_wireguard() 2022-04-20 12:06:59 +02:00
Thomas Haller
04fc191922 glib-aux: refactor nm_unbase64mem_full()
Make the code more nm-like.
2022-04-20 12:05:17 +02:00
Thomas Haller
3571292d97 glib-aux: treat '=' as special character in nm_unbase64char()
This will be useful.
2022-04-20 12:05:17 +02:00
Thomas Haller
e3240781b1 glib-aux: use switch in nm_unbase64char()
This seams easier to read. And as we have a unit test that covers all
possible 256 input values, it's easy to refactor and ensure the code
still works.
2022-04-20 12:05:17 +02:00
Thomas Haller
63dcc5680b glib-aux/tests: add unbase64mem test
This is copied and adjusted from "src/core/tests/test-systemd.c",
where it currently tests the systemd implementation.
2022-04-20 12:05:16 +02:00
Thomas Haller
0aa7d59557 glib-aux: add nm_unbase64{char,mem,mem_full}() helpers
These are taken from systemd code. We want to stop using systemd code,
so we can eventually drop it.
2022-04-20 12:05:16 +02:00
Thomas Haller
c95fd646bc settings: merge branch 'th/settings-startup-complete-crash'
https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/1192
2022-04-20 11:53:51 +02:00
Thomas Haller
b2a4d706f8 settings: use GSource instead of numeric ID in NMSettings
I find it better style to use GSource pointers for tracking pending
sources.
2022-04-20 11:53:14 +02:00
Thomas Haller
9046975a81 settings: fix assertion failure in NMSettings' _startup_complete_check()
This probably has no bad effects when building without more-asserts.

  #0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
  #1  0x00007f7ead0564a3 in __pthread_kill_internal (signo=6, threadid=<optimized out>) at pthread_kill.c:78
  #2  0x00007f7ead009d06 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
  #3  0x00007f7eacfdc7d3 in __GI_abort () at abort.c:79
  #4  0x00007f7ead1fed4c in g_assertion_message (domain=<optimized out>, file=<optimized out>, line=<optimized out>, func=<optimized out>, message=<optimized out>) at ../glib/gtestutils.c:3065
  #5  0x00007f7ead25f98f in g_assertion_message_expr (domain=0x560964f8b7e9 "nm", file=0x560964f83da8 "src/core/settings/nm-settings.c", line=640, func=0x56096504a390 <__func__.44.lto_priv.1> "_startup_complete_check", expr=<optimized out>) at ../glib/gtestutils.c:3091
  #6  0x0000560964ed710e in _startup_complete_check (self=0x560966d1d030, now_msec=<optimized out>) at src/core/settings/nm-settings.c:640
  #7  0x0000560964ed7d9b in _startup_complete_notify_connection (self=0x560966d1d030, sett_conn=<optimized out>, forget=<optimized out>) at src/core/settings/nm-settings.c:704
  #8  0x0000560964edd070 in _connection_changed_delete (self=0x560966d1d030, storage=<optimized out>, sett_conn=0x560966cedbc0, allow_add_to_no_auto_default=<optimized out>) at src/core/settings/nm-settings.c:1244
  #9  0x0000560964edd948 in _connection_changed_process_one (update_reason=(NM_SETTINGS_CONNECTION_UPDATE_REASON_IGNORE_PERSIST_FAILURE | NM_SETTINGS_CONNECTION_UPDATE_REASON_CLEAR_SYSTEM_SECRETS | NM_SETTINGS_CONNECTION_UPDATE_REASON_UPDATE_NON_SECRET | unknown: 0x5400), override_sett_flags=0, sett_mask=NM_SETTINGS_CONNECTION_INT_FLAGS_NONE, sett_flags=1725440360, allow_add_to_no_auto_default=0, sett_conn_entry=0x560966d1d030, self=<optimized out>) at src/core/settings/nm-settings.c:1294
  #10 _connection_changed_process_all_dirty (self=<optimized out>, allow_add_to_no_auto_default=<optimized out>, sett_flags=<optimized out>, sett_mask=<optimized out>, override_sett_flags=<optimized out>, update_reason=<optimized out>) at src/core/settings/nm-settings.c:1335
  #11 0x0000560964eeb8ec in nm_settings_delete_connection (allow_add_to_no_auto_default=648659760, sett_conn=<optimized out>, self=0x560966d1d030) at src/core/settings/nm-settings.c:2457
  #12 nm_settings_connection_delete (self=<optimized out>, allow_add_to_no_auto_default=648659760) at src/core/settings/nm-settings-connection.c:637
  #13 0x0000560964eebebd in delete_auth_cb (self=0x560966cedbc0, context=0x7f7e9c0170a0, subject=0x560966cc5ed0, error=0x0, data=<optimized out>) at src/core/settings/nm-settings-connection.c:1877
  #14 0x0000560964ec9778 in pk_auth_cb (auth_manager=<optimized out>, auth_call_id=<optimized out>, is_authorized=1, is_challenge=<optimized out>, auth_error=<optimized out>, user_data=0x560966e16980) at src/core/settings/nm-settings-connection.c:1262
  #15 0x0000560964db9a28 in _call_id_invoke_callback (error=0x0, is_challenge=0, is_authorized=1, call_id=0x560966ddeb00) at src/core/nm-auth-manager.c:180
  #16 _call_on_idle (user_data=user_data@entry=0x560966ddeb00) at src/core/nm-auth-manager.c:284
  #17 0x00007f7ead23111b in g_idle_dispatch (source=0x560966e50190, callback=0x560964db9900 <_call_on_idle>, user_data=0x560966ddeb00) at ../glib/gmain.c:5848
  #18 0x00007f7ead234d4f in g_main_dispatch (context=0x560966cd1e20) at ../glib/gmain.c:3337
  #19 g_main_context_dispatch (context=0x560966cd1e20) at ../glib/gmain.c:4055
  #20 0x00007f7ead289608 in g_main_context_iterate.constprop.0 (context=0x560966cd1e20, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at ../glib/gmain.c:4131
  #21 0x00007f7ead234463 in g_main_loop_run (loop=0x560966caf010) at ../glib/gmain.c:4329
  #22 0x0000560964cb7515 in main (argc=<optimized out>, argv=<optimized out>) at src/core/main.c:509

Fixes: 3df662f534 ('settings: rework wait-device-timeout handling and consider device compatibility')
2022-04-20 11:53:13 +02:00
Thomas Haller
a3038d4f5a clients/tests: fix regular expression match in Util.replace_text()
Seems the previous code did not work properly:
With python36-3.6.8-38.module+el8.5.0+12207+5c5719bc.x86_6 on rhel-8.6:

  Traceback (most recent call last):
    File "/root/nm-build/NetworkManager/src/tests/client/test-client.py", line 1157, in f
      func(self)
    File "/root/nm-build/NetworkManager/src/tests/client/test-client.py", line 1724, in test_offline
      replace_stdout=replace_uuids,
    File "/root/nm-build/NetworkManager/src/tests/client/test-client.py", line 797, in call_nmcli
      frame,
    File "/root/nm-build/NetworkManager/src/tests/client/test-client.py", line 997, in _call_nmcli
      self.async_start(wait_all=sync_barrier)
    File "/root/nm-build/NetworkManager/src/tests/client/test-client.py", line 1032, in async_start
      async_job.wait_and_complete()
    File "/root/nm-build/NetworkManager/src/tests/client/test-client.py", line 670, in wait_and_complete
      self._complete_cb(self, return_code, stdout, stderr)
    File "/root/nm-build/NetworkManager/src/tests/client/test-client.py", line 919, in complete_cb
      stdout = Util.replace_text(stdout, replace_stdout)
    File "/root/nm-build/NetworkManager/src/tests/client/test-client.py", line 362, in replace_text
      if Util.is_regex_pattern(v_search):
    File "/root/nm-build/NetworkManager/src/tests/client/test-client.py", line 208, in is_regex_pattern
      t = re.Pattern
  AttributeError: module 're' has no attribute 'Pattern'

On this python version, re.compile() give an object of type
_sre.SRE_Pattern.

  # python -c 'import re; print(type(re.compile("a")))'
  <class '_sre.SRE_Pattern'>

Fixes: beebde9e56 ('client/test: allow matching and replacing regex-es in nmcli output')
2022-04-20 11:01:15 +02:00
Lubomir Rintel
e95482dfd0 merge: branch 'lr/nmcli-offline'
https://bugzilla.redhat.com/show_bug.cgi?id=1361145
https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/1183
2022-04-19 14:51:50 +02:00
Lubomir Rintel
97857dbacd client/test: add test for --offline behavior
Currently, only "add" and negative cases are tested. Testing "modify"
would require an ability to provide input. Perhaps at some later point.
2022-04-19 14:27:22 +02:00
Lubomir Rintel
e733357c91 client/test: add @nm_test_no_dbus decorator
Same as @nm_test, apart from that it doesn't spawn the mock D-Bus
service.
2022-04-19 14:12:43 +02:00
Lubomir Rintel
9108f8ecfc client/test: allow overriding all environment variables
Set extra variables after the pre-defined ones have been set. This
allows overriding then.

In particular, this allows overriding DBUS_SESSION_BUS_ADDRESS so that
the test can check the behavior of the client is correct when it's set
to some garbage.
2022-04-19 14:12:43 +02:00
Lubomir Rintel
beebde9e56 client/test: allow matching and replacing regex-es in nmcli output
This allows us to sanitize unpredictable UUIDs in client output in
--offline mode (where we can't just ask the mock service about the
actual UUID).
2022-04-19 14:12:43 +02:00
Lubomir Rintel
6fa1323ce5 nmcli: add --offline option for "add" and "modify"
This adds a global "--offline" option and allows its use with "add" and
"modify" commands. The "add" looks like this:

  $ nmcli --offline conn add type ethernet ens3 ipv4.dns 192.168.1.1 \
      >output.nmconnection

The "modify" is essentially implementing what's been suggested by
Beniamino in bugzilla ticked (referred to below):

  $ nmcli --offline connection modify ens3 ipv4.dns 192.168.1.1 \
      <input.nmconnection >output.nmconnection

Other commands don't support the argument at the moment:

  $ nmcli --offline c up ens3
  Error: 'up' command doesn't support --offline mode.

https://bugzilla.redhat.com/show_bug.cgi?id=1361145
2022-04-19 14:12:42 +02:00
Lubomir Rintel
438f52b459 nmcli/trivial: consistently order the options in process_command_line()
Make the order of nmc_complete_strings() arguments consistent with the
multi-way conditional below. Doesn't have any effect, just ensures the
ommisions and mistakes are hopefully easier to spot.
2022-04-19 14:12:20 +02:00
Lubomir Rintel
2a0231469f nmcli.h: tidy up boolean struct members
Use bitfields to save a few bytes. This involves swapping gboolean for
bool and some reordering in order to get them grouped together.

The patch looks horrible, because clang-format decides to put itself and
seem to go out of its way to make this whole file look idiotic.
What can you do.
2022-04-19 14:12:20 +02:00
Thomas Haller
a1ff31db3b contrib: install nmstate+nispor in "nm-in-container.sh" 2022-04-19 11:15:08 +02:00
Thomas Haller
4d53df2911 contrib/makerepo.sh: fix name for local cache of git repository
The $URL might already contain a ".git" suffix. Then $FULLNAME would
end up having two ".git" extensions. Fix that.
2022-04-15 11:24:19 +02:00
Thomas Haller
135bc5dd1f contrib/makerepo.sh: don't use unauthenticated github URL
It doesn't work anymore:

  $ git clone git://github.com/thom311/libnl.git
  Cloning into 'libnl'...
  fatal: remote error:
    The unauthenticated git protocol on port 9418 is no longer supported.
  Please see https://github.blog/2021-09-01-improving-git-protocol-security-github/ for more information.
2022-04-15 11:11:17 +02:00
Thomas Haller
f9f7f231c8 dns: merge branch 'th/dns-update-pending-rh2049421'
https://bugzilla.redhat.com/show_bug.cgi?id=2049421

https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/1189
2022-04-15 09:21:28 +02:00
Thomas Haller
80c9e2d9ec device: prevent IP state from reaching ready while DNS update pending
The goal would be to ensure that a device cannot move to activated,
while a DNS update is still pending.

This does not really work for most cases. That is, because NMDevice does
not directly push DNS updates to NMDnsManager, instead, NMPolicy is
watching all device changes, and doing it. But when NMPolicy decides to
to that, may not be the right moment.

We really should let NMDevice (or better, NML3Cfg) directly talk to
NMDnsManager. Why not? They have all the information when new DNS
configuration is available. The only thing that NMPolicy does on top of
that, is determining which device has the best default route. NMPolicy
could continue to do that (or maybe NMDnsManager could), but the update
needs to be directly triggered by NMDevice/NML3Cfg.
2022-04-15 09:04:31 +02:00
Thomas Haller
6c27e58d8d core: delay startup complete while we have pending DNS updates
While we have DNS updates pending, we cannot reach startup complete.
2022-04-15 09:04:31 +02:00
Thomas Haller
6e35cf4a7d core: add nm_manager_get_dns_manager() getter
nm_dns_manager_get() is already a singleton. So users usually
can just get it whenever they need -- except during shutdown
after the singleton was destroyed. This is usually fine, because
users really should not try to get it late during shutdown.

However, if you subscribe a signal handler on the singleton, then you
will also eventually want to unsubscribe it. While the moment when you
subscribe it is clearly not during late-shutdown, it's not clear how
to ensure that the signal listener gets destroyed before the DNS manager
singleton.

So usually, whenever you are going to subscribe a signal, you need to
make sure that the target object stays alive long enough. Which may
mean to keep a reference to it.

Next, we will have NMDevice subscribe to the singleton. With above said,
that would mean that potentially every NMDevice needs to keep a
reference to the NMDnsManager. That is not best. Also, later NMManager
will face the same problem, because it will also subscribe to
NMDnsManager.

So, instead let NMManager own a reference to the NMDnsManager. This
ensures the lifetimes are properly guarded (NMDevice also references
NMManager already).

Also, access nm_dns_manager_get() lazy on first use, to only initialize
it when needed the first time (which might be quite late).
2022-04-15 09:04:30 +02:00
Thomas Haller
cef5b8dd46 dns: prevent update-pending to hang indefinitely
For example, if you have a dnsmasq service running and bound to port 53, then
NetworkManager's [main].dns=dnsmasq will fail to start. And we keep retrying
to start it. But then update pending would hang indefinitely, and devices could
not become active. That must not happen.

Give the DNS update only 5 seconds. If it's not done by then, assume we
have a problem and unblock.
2022-04-15 09:04:30 +02:00
Thomas Haller
f2abcf2082 dns/dnsmasq: implement update-pending flag in NMDnsDnsmasq plugin
We want to know when we are busy (have an update pending or on-going).
Implement that.
2022-04-15 09:04:30 +02:00
Thomas Haller
ccf0e8d327 dns/dnsmasq: use GSource for timeout in NMDnsDnsmasq 2022-04-15 09:04:30 +02:00
Thomas Haller
5da17c689b dns/unbound: drop deprecated "unbound" DNS plugin
The "unbound" DNS plugin was very rudimentary and is deprecated since
commit 4a2fe09853 ('man: mark [main].dns=unbound as deprecated') (Jun
2021).

It is part of dnssec-trigger tool, but the dnssec-trigger tool doesn't
actually use it. Instead it installs a dispatcher script
"/usr/lib/NetworkManager/dispatcher.d/01-dnssec-trigger".

Especially, since the plugin requires "/usr/libexec/dnssec-trigger-script",
which is provided by "dnssec-trigger" package on Fedora. At the same
time, the package provides the dispatcher script. So I don't this works
or anybody is using this.

https://mail.gnome.org/archives/networkmanager-list/2022-April/msg00002.html
2022-04-15 09:04:30 +02:00
Thomas Haller
bbbb1b7339 dns/resolved: implement update-pending flag in NMDnsSystemdResolved plugin 2022-04-15 09:04:30 +02:00
Thomas Haller
a74a517f49 dns/resolved: track pending counter for D-Bus calls in NMDnsSystemdResolved
This is used to signal that an update is pending or in progress.
For this to work, we also need to implement the stop() handle.
Otherwise, we couldn't abort pending requests, which is necessary
during shutdown (not today, but in the future).
2022-04-15 09:04:30 +02:00
Thomas Haller
2f1feb9651 dns/resolved: use GPtrArray to collect ip datas in NMDnsSystemdResolved's update()
CList is a great, simple data structure. Especially, if we can embed it
into the data we track.

Here we just create a (temporary) list of pointers. A GPtrArray is the
better data structure for that.
2022-04-15 09:04:30 +02:00
Thomas Haller
eb25c9ecd2 dns/resolved: use nm_utils_hash_to_array_with_buffer() in NMDnsSystemdResolved's update()
We copy the content of the hash table to an array, so that we can sort
the entries and they have a defined order.

We are not only interested in the keys, but the keys and the values.
Hence, use nm_utils_hash_to_array_with_buffer() which gives both at
the same time.
2022-04-15 09:04:29 +02:00
Thomas Haller
51cec67253 dns/resolved: sort dirty interfaces to prune in "nm-dns-systemd-resolved.c"
When we do something where the order makes a visible difference,
we should do it in a consistent way, that does not depend on arbitray
things. Sort the ifindexes from dirty_interfaces hash table.
2022-04-15 09:04:29 +02:00
Thomas Haller
39b68d72d3 dns/resolved: add const to parameters in "nm-dns-systemd-resolved.c" 2022-04-15 09:04:29 +02:00
Thomas Haller
4564adfb53 dns/resolved: minor cleanups in "nm-dns-systemd-resolved.c" 2022-04-15 09:04:29 +02:00
Thomas Haller
a60b971003 dns: add update-pending property to NMDnsManager 2022-04-15 09:04:29 +02:00
Thomas Haller
b7ca08e971 dns: add "update-pending" state to NMDnsPlugin
Theoretically, this should be a GObject property, and not a signal.
But then I'd  also have to implement the get_property() function,
which is more hazzle than necessary. A signal will do nicely.
2022-04-15 09:04:29 +02:00
Thomas Haller
f68230fbe9 dns: call nm_dns_plugin_stop() also for NMDnsSystemdResolved instance
Currently NMDnsSystemdResolved does not implement "stop()". That is
about to change. Make sure to call stop before unreferencing the
instance.
2022-04-15 09:04:29 +02:00
Thomas Haller
0001a2fd0c dns: fix NMDnsPluginPrivate and drop unused fields
NM_DNS_PLUGIN_GET_PRIVATE() macro was broken. Also NMDnsPluginPrivate
contained unused fields. Fix that.

The private data is unused at the moment, but will be used next.
Hence it is fixed and not removed.
2022-04-15 09:04:28 +02:00
Thomas Haller
068ca09d16 dns: obfuscate pointer value for NMDnsPlugin logging 2022-04-15 09:04:28 +02:00
Thomas Haller
f7b41fc18c dns: avoid printing pointer value for NMDnsManager logging statements
We avoid printing raw pointer values. Also, in this case this is a
singleton, and we only create one instance of this type.

Note that we would still have printed the pointer instance while
constructing the instances, before setting it as singleton.

Just drop this.
2022-04-15 09:04:28 +02:00
Thomas Haller
bd95a5c0ec dns: register NMDnsPlugin instance as wait-obj for shutdown
nm_shutdown_wait_obj_register_object() today has no practical effect.
In the future it will block shutdown until the object gets destroyed.
We will want that NMDnsPlugin gets wrapped up during shut down, before
quitting.
2022-04-15 09:04:28 +02:00
Thomas Haller
de926723f0 glib-aux: add nm_utils_hash_to_array() helper
We effectively already have this function, with the name
nm_utils_named_values_from_strdict(). Which is a decent name,
if you have a strdict. But it seems odd to use for other dictionaries.

Instead, add a variant with a different name. Naming is important,
and just to have the better name, the function is effectively duplicated.
2022-04-15 09:04:28 +02:00
Thomas Haller
78aad6cf51 glib-aux: add "name_ptr" union field to NMUtilsNamedValue
NMUtilsNamedValue is a key-value tuple, usually the key is a string
(hence the name "Named").

But this struct is also useful for keys that are not strings.
Add another "name_ptr" union field to access the key that way.

The alternative would be to add another struct, which serves
a very similar purpose though.
2022-04-14 21:31:36 +02:00
Thomas Haller
e05f439ff4 dhcp: merge branch 'th/systemd-no-dhcp4'
https://bugzilla.redhat.com/show_bug.cgi?id=2073067

https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/1188
2022-04-14 21:22:49 +02:00
Thomas Haller
6150a495c9 dhcp/systemd: drop dhcp4 client (and related files)
This code is now unused.
2022-04-14 14:51:03 +02:00