Optimally, all list types properties in nmcli support proper escaping.
For that, we should use nm_utils_escaped_tokens_*() API.
For now, just change that for policy routes. They were just added recently,
so no change in behavior of released API.
(cherry picked from commit d59f046bb6)
Replace nm_utils_str_simpletokens_extract_next() by
nm_utils_escaped_tokens_split().
nm_utils_escaped_tokens_split() should become our first choice for
parsing and tokenizing.
Note that both nm_utils_str_simpletokens_extract_next() and
nm_utils_escaped_tokens_split() need to strdup the string once,
and tokenizing takes O(n). So, they are roughtly the same performance
wise. The only difference is, that as we iterate through the tokens,
we might abort early on error with nm_utils_str_simpletokens_extract_next()
and not parse the entire string. But that is a small benefit, since we
anyway always strdup() the string (being O(n) already).
Note that to-string will no longer escape ',' and ';'. This is a change
in behavior, of unreleased API. Also note, that escaping these is no
longer necessary, because nmcli soon will also use nm_utils_escaped_tokens_*().
Another change in behavior is that nm_utils_str_simpletokens_extract_next()
treated invalid escape sequences (backslashes followed by an arbitrary
character), buy stripping the backslash. nm_utils_escaped_tokens_*()
leaves such backslashes as is, and only honors them if they are followed
by a whitespace (the delimiter) or another backslash. The disadvantage
of the new approach is that backslashes are treated differently
depending on the following character. The benefit is, that most
backslashes can now be written verbatim, not requiring them to escape
them with a double-backslash.
Yes, there is a problem with these nested escape schemes:
- the caller may already need to escape backslash in shell.
- then nmcli will use backslash escaping to split the rules at ','.
- then nm_ip_routing_rule_from_string() will honor backslash escaping
for spaces.
- then iifname and oifname use backslash escaping for nm_utils_buf_utf8safe_escape()
to express non-UTF-8 characters (because interface names are not
necessarily UTF-8).
This is only redeamed because escaping is really only necessary for very
unusual cases, if you want to embed a backslash, a space, a comma, or a
non-UTF-8 character. But if you have to, now you will be able to express
that.
The other upside of these layers of escaping is that they become all
indendent from each other:
- shell can accept quoted/escaped arguments and will unescape them.
- nmcli can do the tokenizing for ',' (and escape the content
unconditionally when converting to string).
- nm_ip_routing_rule_from_string() can do its tokenizing without
special consideration of utf8safe escaping.
- NMIPRoutingRule takes iifname/oifname as-is and is not concerned
about nm_utils_buf_utf8safe_escape(). However, before configuring
the rule in kernel, this utf8safe escape will be unescaped to get
the interface name (which is non-UTF8 binary).
(cherry picked from commit b6d0be2d3b)
nmcli supports list options (optlist and multilist properties).
These commonly are individual items, concatenated by a delimiter.
It should be generally possibly to express every value. That means, we
need some for of escaping mechanism for delimiters.
Currently this is all inconsistent or no escaping is supported. I intend
to fix that (which will be a change in behavior).
For now, just add yet another style of tokenzing/concatenating list
items in nmcli. This is the style to replace all other styles.
(cherry picked from commit ba956bd499)
This escapes strings so that they can be concatenated with a delimiter
and without loss tokenized with nm_utils_escaped_tokens_split().
Note that this is similar to _nm_utils_escape_plain() and
_nm_utils_escape_spaces(). The difference is that
nm_utils_escaped_tokens_escape() also escapes the last trailing
whitespace. That means, if delimiters contains all NM_ASCII_SPACES, then
it is identical to _nm_utils_escape_spaces(). Otherwise, the trailing
space is treated specially. That is, because
nm_utils_escaped_tokens_split() uses NM_UTILS_STRSPLIT_SET_FLAGS_STRSTRIP,
to strip leading and trailing whitespace. To still express a trailing
whitespace, the last whitespace must be escaped. Note that
NM_UTILS_STRSPLIT_SET_FLAGS_STRSTRIP also honors escaping any whitespace
(not only at the last position), but when escaping we don't need to
escape them, because unescaped (non-trailing) whitespace are taken just
fine.
The pair nm_utils_escaped_tokens_split() and
nm_utils_escaped_tokens_escape() are proposed as default way of
tokenizing a list of items. For example, with
$ nmcli connection modify "$PROFILE" +ipv4.routing-rules 'priority 5 from 192.168.7.5/32 table 5, priority 6 iif a\, from 192.168.7.5/32 table 6'
Here we implement a to/from string function to handle one item
(nm_ip_routing_rule_{from,to}_string()). When such elements are combined with ',',
then we need to support an additional layer of escaping on top of that.
The advantage is that the indvidual to/from string functions are agnostic
to this second layer of escaping/tokenizing that nmcli employs to handle
a list of these items.
The disadvantage is that we possibly get multiple layers of backslash
escapings. That is only mitigated by the fact that nm_utils_escaped_tokens_*()
supports a syntax for which *most* characters don't need any special escaping.
Only delimiters, backslash, and the trailing space needs escaping, and
these are cases are expected to be few.
(cherry picked from commit e206e3d4d8)
Add a new flag that will remove escape characters after splitting
the string.
This implements a special kind of backslash escaping. It's not C escape
sequences (like '\n' or '\020'), but simply to take the special character
following the backslash verbatim. Note that the backslash is only
considered special, if it's followed by a delimiter, another backslash,
or a whitespace (in combination with %NM_UTILS_STRSPLIT_SET_FLAGS_STRSTRIP).
The main purpose of this form of escaping is nmcli's list options, like
$ nmcli connection modify "$PROFILE" +ipv4.routing-rules 'priority 5 from 192.168.7.5/32 table 5, priority 6 iif a\, from 192.168.7.5/32 table 6'
It's a contrieved example, but the list options are a list of IP
addresses, rules, etc. They implement their own syntax for one element,
and are concatenated by ','. To support that one element may have
arbitrary characters (including the delimiter and whitespaces), nmcli
employs a tokenization with this special kind of escaping.
(cherry picked from commit 9522aaf226)
This will essentially call g_strstrip() on each token.
There are some specialties:
- if the resulting word is empty after stripping, then according to
%NM_UTILS_STRSPLIT_SET_FLAGS_PRESERVE_EMPTY, the empty token will be
removed. If that results in an empty string array, %NULL will be
returned.
- if %NM_UTILS_STRSPLIT_SET_FLAGS_ALLOW_ESCAPING is set, then
whitespace that is backslash escaped is not removed.
Since this is a post-operation that happens after tokeninzing, it
could be done as a separate function. And we already have this function:
_nm_utils_unescape_plain() and _nm_utils_unescape_spaces().
However, that is ugly for several reasons:
- the stripping should be part of the tokenizing, you shouldn't need
several steps.
- nm_utils_strsplit_set_full() returns a "const char **" which
indicates the strings must not be freed. However, it is perfectly
valid to modify the string inplace. Hence, the post-op function
would need to cast the strings to "char *", which seems ugly
(although we do that on many places, and it's guaranteed to work).
- _nm_utils_unescape_plain()/_nm_utils_unescape_spaces() is indeed
already used together with nm_utils_strsplit_set_full(). However,
it requires to initialize the cb_lookup buffer twice. I would expect
that initializing the cb_lookup buffer is a large portion of what
the function does already (for short strings).
This issue will be solved in the next commit by adding yet another flag
which allows to unescape.
(cherry picked from commit 5b2b0dcadf)
This can be replaced by nm_utils_escaped_tokens_split().
Note that nm_utils_escaped_tokens_split() does not behave exactly
the same. For example, nm_utils_str_simpletokens_extract_next() would
remove all backslashes and leave only the following character.
nm_utils_escaped_tokens_split() instead only strips backslashes
that preceed a delimiter, whitespace or another backslash.
But we should have one preferred way of tokenizing, and I find this
preferable, because it allows for most backslashes to appear verbatim.
Optimally, all list types properties in nmcli support proper escaping.
For that, we should use nm_utils_escaped_tokens_*() API.
For now, just change that for policy routes. They were just added recently,
so no change in behavior of released API.
Replace nm_utils_str_simpletokens_extract_next() by
nm_utils_escaped_tokens_split().
nm_utils_escaped_tokens_split() should become our first choice for
parsing and tokenizing.
Note that both nm_utils_str_simpletokens_extract_next() and
nm_utils_escaped_tokens_split() need to strdup the string once,
and tokenizing takes O(n). So, they are roughtly the same performance
wise. The only difference is, that as we iterate through the tokens,
we might abort early on error with nm_utils_str_simpletokens_extract_next()
and not parse the entire string. But that is a small benefit, since we
anyway always strdup() the string (being O(n) already).
Note that to-string will no longer escape ',' and ';'. This is a change
in behavior, of unreleased API. Also note, that escaping these is no
longer necessary, because nmcli soon will also use nm_utils_escaped_tokens_*().
Another change in behavior is that nm_utils_str_simpletokens_extract_next()
treated invalid escape sequences (backslashes followed by an arbitrary
character), buy stripping the backslash. nm_utils_escaped_tokens_*()
leaves such backslashes as is, and only honors them if they are followed
by a whitespace (the delimiter) or another backslash. The disadvantage
of the new approach is that backslashes are treated differently
depending on the following character. The benefit is, that most
backslashes can now be written verbatim, not requiring them to escape
them with a double-backslash.
Yes, there is a problem with these nested escape schemes:
- the caller may already need to escape backslash in shell.
- then nmcli will use backslash escaping to split the rules at ','.
- then nm_ip_routing_rule_from_string() will honor backslash escaping
for spaces.
- then iifname and oifname use backslash escaping for nm_utils_buf_utf8safe_escape()
to express non-UTF-8 characters (because interface names are not
necessarily UTF-8).
This is only redeamed because escaping is really only necessary for very
unusual cases, if you want to embed a backslash, a space, a comma, or a
non-UTF-8 character. But if you have to, now you will be able to express
that.
The other upside of these layers of escaping is that they become all
indendent from each other:
- shell can accept quoted/escaped arguments and will unescape them.
- nmcli can do the tokenizing for ',' (and escape the content
unconditionally when converting to string).
- nm_ip_routing_rule_from_string() can do its tokenizing without
special consideration of utf8safe escaping.
- NMIPRoutingRule takes iifname/oifname as-is and is not concerned
about nm_utils_buf_utf8safe_escape(). However, before configuring
the rule in kernel, this utf8safe escape will be unescaped to get
the interface name (which is non-UTF8 binary).
nmcli supports list options (optlist and multilist properties).
These commonly are individual items, concatenated by a delimiter.
It should be generally possibly to express every value. That means, we
need some for of escaping mechanism for delimiters.
Currently this is all inconsistent or no escaping is supported. I intend
to fix that (which will be a change in behavior).
For now, just add yet another style of tokenzing/concatenating list
items in nmcli. This is the style to replace all other styles.
This escapes strings so that they can be concatenated with a delimiter
and without loss tokenized with nm_utils_escaped_tokens_split().
Note that this is similar to _nm_utils_escape_plain() and
_nm_utils_escape_spaces(). The difference is that
nm_utils_escaped_tokens_escape() also escapes the last trailing
whitespace. That means, if delimiters contains all NM_ASCII_SPACES, then
it is identical to _nm_utils_escape_spaces(). Otherwise, the trailing
space is treated specially. That is, because
nm_utils_escaped_tokens_split() uses NM_UTILS_STRSPLIT_SET_FLAGS_STRSTRIP,
to strip leading and trailing whitespace. To still express a trailing
whitespace, the last whitespace must be escaped. Note that
NM_UTILS_STRSPLIT_SET_FLAGS_STRSTRIP also honors escaping any whitespace
(not only at the last position), but when escaping we don't need to
escape them, because unescaped (non-trailing) whitespace are taken just
fine.
The pair nm_utils_escaped_tokens_split() and
nm_utils_escaped_tokens_escape() are proposed as default way of
tokenizing a list of items. For example, with
$ nmcli connection modify "$PROFILE" +ipv4.routing-rules 'priority 5 from 192.168.7.5/32 table 5, priority 6 iif a\, from 192.168.7.5/32 table 6'
Here we implement a to/from string function to handle one item
(nm_ip_routing_rule_{from,to}_string()). When such elements are combined with ',',
then we need to support an additional layer of escaping on top of that.
The advantage is that the indvidual to/from string functions are agnostic
to this second layer of escaping/tokenizing that nmcli employs to handle
a list of these items.
The disadvantage is that we possibly get multiple layers of backslash
escapings. That is only mitigated by the fact that nm_utils_escaped_tokens_*()
supports a syntax for which *most* characters don't need any special escaping.
Only delimiters, backslash, and the trailing space needs escaping, and
these are cases are expected to be few.
Add a new flag that will remove escape characters after splitting
the string.
This implements a special kind of backslash escaping. It's not C escape
sequences (like '\n' or '\020'), but simply to take the special character
following the backslash verbatim. Note that the backslash is only
considered special, if it's followed by a delimiter, another backslash,
or a whitespace (in combination with %NM_UTILS_STRSPLIT_SET_FLAGS_STRSTRIP).
The main purpose of this form of escaping is nmcli's list options, like
$ nmcli connection modify "$PROFILE" +ipv4.routing-rules 'priority 5 from 192.168.7.5/32 table 5, priority 6 iif a\, from 192.168.7.5/32 table 6'
It's a contrieved example, but the list options are a list of IP
addresses, rules, etc. They implement their own syntax for one element,
and are concatenated by ','. To support that one element may have
arbitrary characters (including the delimiter and whitespaces), nmcli
employs a tokenization with this special kind of escaping.
This will essentially call g_strstrip() on each token.
There are some specialties:
- if the resulting word is empty after stripping, then according to
%NM_UTILS_STRSPLIT_SET_FLAGS_PRESERVE_EMPTY, the empty token will be
removed. If that results in an empty string array, %NULL will be
returned.
- if %NM_UTILS_STRSPLIT_SET_FLAGS_ALLOW_ESCAPING is set, then
whitespace that is backslash escaped is not removed.
Since this is a post-operation that happens after tokeninzing, it
could be done as a separate function. And we already have this function:
_nm_utils_unescape_plain() and _nm_utils_unescape_spaces().
However, that is ugly for several reasons:
- the stripping should be part of the tokenizing, you shouldn't need
several steps.
- nm_utils_strsplit_set_full() returns a "const char **" which
indicates the strings must not be freed. However, it is perfectly
valid to modify the string inplace. Hence, the post-op function
would need to cast the strings to "char *", which seems ugly
(although we do that on many places, and it's guaranteed to work).
- _nm_utils_unescape_plain()/_nm_utils_unescape_spaces() is indeed
already used together with nm_utils_strsplit_set_full(). However,
it requires to initialize the cb_lookup buffer twice. I would expect
that initializing the cb_lookup buffer is a large portion of what
the function does already (for short strings).
This issue will be solved in the next commit by adding yet another flag
which allows to unescape.
This removes libnm-glib, libnm-glib-vpn, and libnm-util for good.
The it has been replaced with libnm since NetworkManager 1.0, disabled
by default since 1.12 and no up-to-date distributions ship it for years
now.
Removing the libraries allows us to:
* Remove the horrible hacks that were in place to deal with accidental use
of both the new and old library in a single process.
* Relief the translators of maintenance burden of similar yet different
strings.
* Get rid of known bad code without chances of ever getting fixed
(libnm-glib/nm-object.c and libnm-glib/nm-object-cache.c)
* Generally lower the footprint of the releases and our workspace
If there are some really really legacy users; they can just build
libnm-glib and friends from the NetworkManager-1.16 distribution. The
D-Bus API is stable and old libnm-glib will keep working forever.
https://github.com/NetworkManager/NetworkManager/pull/308
Scripts should refer to connections by UUID. That means, whenever a
UUID matches, that is really what the user wants. The specifier "uuid"
must have precedence over "id" (and all others). That means, we must
search all connections.
For example:
$ UUIDS=($(nmcli -g TYPE,UUID connection show | sed -n 's/802-11-wireless://p'))
$ nmcli -f connection.id,connection.uuid connection show "${UUIDS[@]}"
in this case we must preferrably match by UUID, regardless of whether
a "connection.id" exists.
Note that if you have:
$ nmcli connection show | grep fdf7b2d2-2858-3938-9b14-7f1b514a9a00
b fdf7b2d2-2858-3938-9b14-7f1b514a9a00 ethernet --
fdf7b2d2-2858-3938-9b14-7f1b514a9a00 ab9f3891-3420-335e-89da-f14c1b94c540 ethernet --
then certain commands will still select all matching connections:
$ nmcli -f connection.id,connection.uuid --mode multiline connection show fdf7b2d2-2858-3938-9b14-7f1b514a9a00
connection.id: fdf7b2d2-2858-3938-9b14-7f1b514a9a00
connection.uuid: ab9f3891-3420-335e-89da-f14c1b94c540
connection.id: b
connection.uuid: fdf7b2d2-2858-3938-9b14-7f1b514a9a00
This only makes a difference for commands that must pick only one profile:
$ nmcli connection modify fdf7b2d2-2858-3938-9b14-7f1b514a9a00 con-name new-name
Since commit 62b939de4e ('cli: add nmc_complete_strv() which takes a
string array for completion that may contain NULL'), the sentinel is
no longer needed.
b46392d25d build: use c-stdaux
61f21750be build: pull in c-stdaux
31fcf75afe build: sync with c-util
fec7b8f2da ci: add run with -DNDEBUG
git-subtree-dir: shared/c-rbtree
git-subtree-split: b46392d25de7a7bab67d48ef18bf8350b429cff5
c-stdaux is C11 and so is NetworkManager too.
But we build on Ubunut 14.04, where we have not all C11 parts
in place.
Since the header is not actually used (at the moment), patch out
the include.
