Commit Graph

152 Commits

Author SHA1 Message Date
Daniel Thwaites
e888c2133f
lib/strings: convert to string within hasInfix
This should fix the issue mentioned here:
https://github.com/NixOS/nixpkgs/pull/168175#issuecomment-1107830574
2022-05-01 17:39:42 +01:00
Naïm Camille Favier
535997fa52
lib/strings: fix quoting of example
Co-authored-by: Silvan Mosberger <github@infinisil.com>
2022-04-27 19:42:40 +02:00
Naïm Favier
226bc99659
lib/strings: add toShellVars
A straightforward piece of plumbing to safely inject Nix variables into
shell scripts:

''
  ${lib.toShellVars { inherit foo bar; }}
  cmd "$foo" --bar "$bar"
''
2022-04-27 16:04:17 +02:00
Daniel Thwaites
4f9b8a0702
lib/strings: optimise hasInfix function (#168175)
* lib/strings: optimise hasInfix function

* lib/strings: optimise hasInfix further using regex

* rstudio: call hasInfix with a string

* lib/strings: remove let from hasInfix

Co-authored-by: pennae <82953136+pennae@users.noreply.github.com>

Co-authored-by: pennae <82953136+pennae@users.noreply.github.com>
2022-04-22 12:43:25 -04:00
Robert Hensing
559ac3c9e7
Merge pull request #166383 from hercules-ci/always-sanitize-derivation-name
Always sanitize derivation name
2022-04-05 20:05:33 +02:00
Robert Hensing
fffabe7500 lib.sanitizeDerivationName: Simplify regex
`^` and `$` are implicit in `match`.
2022-04-05 19:14:09 +02:00
Silvan Mosberger
975a1ae339 lib/strings: Add levenshtein distance functions
Adds some functions related to string similarity:
- lib.strings.commonPrefixLength
- lib.strings.commonSuffixLength
- lib.strings.levenshtein
- lib.strings.levenshteinAtMost
2022-04-01 22:03:05 +02:00
Robert Hensing
342a3c32c9 lib.sanitizeDerivationName: Optimize the common case 2022-03-31 18:31:11 +02:00
Robert Helgesson
c7678aff1b
lib: fix escapeXML example in documentation
The previous example output was forgotten copy-paste from some other
function.
2021-11-02 22:43:25 +01:00
Robert Helgesson
e75f346aa3
lib: add function escapeXML
Given a string, this function returns a string that can be inserted
verbatim in an XML document.
2021-10-03 11:28:03 +02:00
polykernel
de81edf6ef lib/strings: fix infinite recursion on concatStringsSep fallback
The current implementation of the concatStringsSep fallback references
concatStrings whcih is just a partial application of concatStringsSep,
forming a circular dependency. Although this will almost never be
encountered as (assuming the user does not explicitly trigger it):
  1. the or operator will short circuit both in lazy and strict
     evaluation
  2. this can only occur in Nix versions prior to 1.10
     which is not compatible with various nix operations as of 2.3.15

However it is still important if scopedImport is used or the builtins
have been overwritten. lib.foldl' is used instead of builtins.foldl'
as the foldl' primops was introduced in the same release as concatStringsSep.
2021-08-26 20:08:05 -04:00
Niklas Hambüchen
5db6b909cd
strings.nix: Fix overly monomorphic type signature comments
These functions work on lists of anything, not just lists of strings.
2021-07-04 20:52:48 +02:00
sternenseemann
f39a5c4e50 lib/strings: forbid lists in isStorePath
When a list is passed to isStorePath this is most likely a mistake and
it is therefore better to just return false. There is one case where
this theoretically makes sense (if a list contains a single element for
which isStorePath elem), but since that case is also probably seldomly
intentional, it may save someone from debbuging unclear evaluation
errors.
2021-05-07 01:22:30 +02:00
sternenseemann
326d0970e0 lib/strings: fix example for isStorePath
Since it checks if dirOf x is the nix store dir, a trailing slash will
break this check and make it return false.
2021-05-07 01:22:30 +02:00
Lennart Spitzner
8c70a1a989
lib: fix documented type of fixedWidthString (#121396) 2021-05-01 17:29:00 +02:00
Alyssa Ross
a8afbb45c1 treewide: use lib.warnIf where appropriate 2021-04-28 21:44:21 +00:00
Alex Brandt
ff1700feda lib/string: drop redundant string in description of toInt.
Describing the string argument as a string is redundant and not needed
to describe what this function does.
2020-12-05 14:35:30 +00:00
Alex Brandt
4a5cb363ae lib/strings: fix typo in example
The example refers to a snake_case function name but Nix uses camelCase
function names.  This ensures the example is correct for the given
function.
2020-12-04 19:35:27 +00:00
Alex Brandt
2697a777e7 lib/strings: fix examples for enableFeatureAs
The As was missing in the examples on this library function.  This will
ensure the examples refer to the function they document.
2020-12-03 18:50:14 +00:00
Eelco Dolstra
f9dd8f04a0
Merge pull request #44928 from vcunat/p/configure-assert
lib/strings: guard against an easy mistake
2020-11-11 11:14:12 +01:00
Robert Hensing
afa6c51f27 lib: Use Nix's static scope checking, fix error message, optimize
Nix can perform static scope checking, but whenever code is inside
a `with` expression, the analysis breaks down, because it can't
know statically what's in the attribute set whose attributes were
brought into scope. In those cases, Nix has to assume that
everything works out.

Except it doesnt. Removing `with` from lib/ revealed an undefined
variable in an error message.

If that doesn't convince you that we're better off without `with`,
I can tell you that this PR results in a 3% evaluation performance
improvement because Nix can look up local variables by index.
This adds up with applications like the module system.

Furthermore, removing `with` makes the binding site of each
variable obvious, which helps with comprehension.
2020-10-22 13:46:47 +02:00
Etienne Laurin
d7464ab4bb lib.splitString: use builtin.split 2020-10-18 13:19:50 +00:00
V
423fc3f232 lib/strings: deprecate readPathsFromFile
> NOTE: This function is not performant and should be avoided.

It's not used at all in-tree now, so we can remove it completely after
any remaining users are given notice.
2020-09-03 19:15:10 +02:00
Silvan Mosberger
5ae3fb2c38
lib/strings: Add floatToString 2020-07-25 21:43:09 +02:00
Silvan Mosberger
f9eb3d158a
lib/strings: Introduce escapeNixIdentifier 2020-04-13 17:27:04 +02:00
Silvan Mosberger
f75c11cfdf
Merge pull request #83241 from Infinisil/valid-drv-name
lib/strings: Add `sanitizeDerivationName` function
2020-04-02 05:58:13 +02:00
Silvan Mosberger
4b206ac83b
lib/strings: Add sanitizeDerivationName function 2020-03-30 01:15:30 +02:00
Frederik Rietdijk
419bc0a4cd Revert "Revert "Merge master into staging-next""
In 87a19e9048 I merged staging-next into master using the GitHub gui as intended.
In ac241fb7a5 I merged master into staging-next for the next staging cycle, however, I accidentally pushed it to master.
Thinking this may cause trouble, I reverted it in 0be87c7979. This was however wrong, as it "removed" master.

This reverts commit 0be87c7979.
2020-02-05 19:41:25 +01:00
Frederik Rietdijk
0be87c7979 Revert "Merge master into staging-next"
I merged master into staging-next but accidentally pushed it to master.
This should get us back to 87a19e9048.

This reverts commit ac241fb7a5, reversing
changes made to 76a439239e.
2020-02-05 19:18:35 +01:00
Robert Helgesson
fc3bd1aeb9
lib/strings: fix comment typo 2020-02-05 18:09:37 +01:00
John Ericson
84a105254d lib: Add getName to mirror getVersion 2019-11-24 16:24:50 +00:00
Nathan van Doorn
8bf42f538e
Doc fix: use correct function name in type signature for concatIMapStringsSep 2019-03-18 12:14:39 +00:00
volth
bb9557eb7c lib.makePerlPath -> perlPackages.makePerlPath 2018-12-15 03:50:31 +00:00
Patrick Hilhorst
320c9c10de
make-derivation: use pname-version as default name if both are present 2018-11-06 00:04:21 +01:00
Graham Christensen
0c5d9e5c52
Merge pull request #49383 from tazjin/docs/lib-docstrings
Update library function "docstrings" for nixdoc generation
2018-10-29 11:00:02 +00:00
Vincent Ambo
0560caa578 lib/strings: Update documentation comments for doc generation
Updates documentation comments with extra information for nixdoc[1]
compatibility.

Some documentation strings have additionally been reworded for
clarity.

"Faux types" are added where applicable, but some functions do things
that are not trivially representable in the type notation used so they
were ignored for this purpose.

[1]: https://github.com/tazjin/nixdoc
2018-10-29 10:42:43 +01:00
Tim Cuthbertson
d984c55383 lib.isStorePath: fix false result when passed a path object
Since `isStorePath` relies on comparing against builtins.storeDir
(a string), we need to convert the input into a string as well.
2018-10-20 22:33:04 +11:00
Profpatsch
3e45b61a99 lib/trivial: add a few examples of usage of assertMsg/assertOneOf 2018-09-06 18:14:27 +02:00
Shea Levy
18337f3ece
Merge branch 'no-toPath' 2018-09-06 08:09:53 -04:00
Aaron Andersen
343e10aaa6 function rewritten by @Infinisil 2018-08-21 20:11:28 +02:00
Aaron Andersen
74d446176e as requested:
- moved function into strings.nix
- renamed function from makePerl5Lib
- removed duplicates entries in the resulting value
- rewrote the function from scratch after learning a few things (much cleaner now)
2018-08-21 20:11:28 +02:00
Vladimír Čunát
fecb444e18
lib/strings: guard against an easy mistake
This would catch the bad `fwknop` flags fixed in 580cab57e4,
during evaluation already.
2018-08-12 10:17:08 +02:00
volth
cc55a3ebcb treewide: fix build with disallowed aliases (#43872)
fixes build with disallowed aliases
2018-07-21 22:03:24 -04:00
Shea Levy
6da6accd30
treewide: Remove uses of builtins.toPath.
toPath has confusing semantics and is never necessary; it can always
either just be omitted or replaced by pre-concatenating `/.`. It has
been marked as "!!! obsolete?" for more than 10 years in a C++
comment, hopefully removing it will let us properly deprecate and,
eventually, remove it.
2018-05-22 16:42:02 -04:00
John Ericson
9e9cdd7027 lib: Add more configure flag helpers
Add with/without to match enable/disable, and add
`--{enable,with}-key=value` versions of both.
2018-05-11 17:43:35 +02:00
Matthew Bauer
8a98cf97e9 lib.makeSearchPath: allow null in search path
This makes things match ‘buildInputs’ where inputs are allowed to be
null.
2018-05-09 15:44:06 -05:00
Shea Levy
b66d7dc0ce
lib.isStorePath: Fix derivation detection 2018-03-09 17:21:31 -05:00
Ryan Trinkle
6281eb123b nixpkgs.lib: Add escapeNixString 2017-12-02 13:28:00 -05:00
Graham Christensen
152c63c9ff
Convert libs to a fixed-point
This does break the API of being able to import any lib file and get
its libs, however I'm not sure people did this.

I made this while exploring being able to swap out docFn with a stub
in #2305, to avoid functor performance problems. I don't know if that
is going to move forward (or if it is a problem or not,) but after
doing all this work figured I'd put it up anyway :)

Two notable advantages to this approach:

1. when a lib inherits another lib's functions, it doesn't
   automatically get put in to the scope of lib
2. when a lib implements a new obscure functions, it doesn't
   automatically get put in to the scope of lib

Using the test script (later in this commit) I got the following diff
on the API:

  + diff master fixed-lib
  11764a11765,11766
  > .types.defaultFunctor
  > .types.defaultTypeMerge
  11774a11777,11778
  > .types.isOptionType
  > .types.isType
  11781a11786
  > .types.mkOptionType
  11788a11794
  > .types.setType
  11795a11802
  > .types.types

This means that this commit _adds_ to the API, however I can't find a
way to fix these last remaining discrepancies. At least none are
_removed_.

Test script (run with nix-repl in the PATH):

  #!/bin/sh

  set -eux

  repl() {
      suff=${1:-}
      echo "(import ./lib)$suff" \
          | nix-repl 2>&1
  }

  attrs_to_check() {
      repl "${1:-}" \
          | tr ';'  $'\n' \
          | grep "\.\.\." \
          | cut -d' ' -f2 \
          | sed -e "s/^/${1:-}./" \
          | sort
  }

  summ() {
      repl "${1:-}" \
          | tr ' ' $'\n' \
          | sort \
          | uniq
  }

  deep_summ() {
      suff="${1:-}"
      depth="${2:-4}"
      depth=$((depth - 1))
      summ "$suff"

      for attr in $(attrs_to_check "$suff" | grep -v "types.types"); do
          if [ $depth -eq 0 ]; then
              summ "$attr" | sed -e "s/^/$attr./"
          else
              deep_summ "$attr" "$depth" | sed -e "s/^/$attr./"
          fi
      done
  }

  (
      cd nixpkgs

      #git add .
      #git commit -m "Auto-commit, sorry" || true
      git checkout fixed-lib
      deep_summ > ../fixed-lib
      git checkout master
      deep_summ > ../master
  )

  if diff master fixed-lib; then
      echo "SHALLOW MATCH!"
  fi

  (
      cd nixpkgs
      git checkout fixed-lib
      repl .types
  )
2017-09-16 21:36:43 -04:00
zimbatm
4d545297d8 lib: introduce imap0, imap1 (#25543)
* lib: introduce imap0, imap1

For historical reasons, imap starts counting at 1 and it's not
consistent with the rest of the lib.

So for now we split imap into imap0 that starts counting at zero and
imap1 that starts counting at 1. And imap is marked as deprecated.

See c71e2d4235 (commitcomment-21873221)

* replace uses of lib.imap

* lib: move imap to deprecated.nix
2017-07-04 23:29:23 +01:00
Profpatsch
2b0a8427e6 lib/string: make isStorePath total (#26216)
fix #9278
2017-05-30 00:20:37 +01:00
Tom Saeger
5989515b94 lib: trivial spelling fixes 2017-04-19 19:37:55 -05:00
Orivej Desh
02126b69e5
lib.readPathsFromFile: simplify, /cc #24036
This part isn't needed after 36de745e1b.
2017-03-19 14:52:47 +01:00
Vladimír Čunát
36de745e1b
readPathsFromFile: fixup after #23851
The final newline would produce an empty string;
let's filter all empty lines as well.
2017-03-19 10:19:20 +01:00
Léo Gaspard
da6619cffe libs: make splitString also split last separator (#23851)
* libs: make splitString also split last separator

* libs: add tests for splitStrings
2017-03-15 21:16:04 +00:00
Domen Kožar
661415d4eb getVersion: first try drv.version before parsing drv.name 2016-08-13 17:17:36 +02:00
Данило Глинський (Danylo Hlynskyi)
0a0b7eb5f2 fix documentation typo in lib/strings.nix (#17684) 2016-08-12 08:30:11 +00:00
Domen Kožar
221f7f18c1 hasSuffix: human readable inputs 2016-08-10 20:06:52 +02:00
Eric Sagnes
56575cc0ac lib: add fileContents function 2016-08-01 18:35:25 +09:00
aszlig
df475092e9
lib: Make escapeShellArg more robust
Quoting various characters that the shell *may* interpret specially is a
very fragile thing to do.

I've used something more robust all over the place in various Nix
expression I've written just because I didn't trust escapeShellArg.

Here is a proof of concept showing that I was indeed right in
distrusting escapeShellArg:

with import <nixpkgs> {};

let
  payload = runCommand "payload" {} ''
    # \x00 is not allowed for Nix strings, so let's begin at 1
    for i in $(seq 1 255); do
      echo -en "\\x$(printf %02x $i)"
    done > "$out"
  '';

  escapers = with lib; {
    current = escapeShellArg;
    better = arg: let
      backslashEscapes = stringToCharacters "\"\\ ';$`()|<>\r\t*[]&!~#";
      search = backslashEscapes ++ [ "\n" ];
      replace = map (c: "\\${c}") backslashEscapes ++ [ "'\n'" ];
    in replaceStrings search replace (toString arg);
    best = arg: "'${replaceStrings ["'"] ["'\\''"] (toString arg)}'";
  };

  testWith = escaper: let
    escaped = escaper (builtins.readFile payload);
  in runCommand "test" {} ''
    if ! r="$(bash -c ${escapers.best "echo -nE ${escaped}"} 2> /dev/null)"
    then
      echo bash eval error > "$out"
      exit 0
    fi
    if echo -n "$r" | cmp -s "${payload}"; then
      echo success > "$out"
    else
      echo failed > "$out"
    fi
  '';

in runCommand "results" {} ''
  echo "Test results:"
  ${lib.concatStrings (lib.mapAttrsToList (name: impl: ''
    echo "  ${name}: $(< "${testWith impl}")"
  '') escapers)}
  exit 1
''

The resulting output is the following:

Test results:
  best: success
  better: success
  current: bash eval error

I did the "better" implementation just to illustrate that the method of
quoting only "harmful" characters results in madness in terms of
implementation and performance.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
Cc: @edolstra, @zimbatm
2016-06-20 23:53:36 +02:00
zimbatm
b0f8416c5c Merge pull request #16180 from zimbatm/shell-escaping
Escape all shell arguments uniformly
2016-06-19 23:27:52 +01:00
zimbatm
bec28d748c Remove unecessary branching on old nix versions
All these builtins are available since 1.10 or earlier (1.10 being the
lib/minver.nix)
2016-06-17 11:06:48 +01:00
zimbatm
28fa4a2f03 Escape all shell arguments uniformly 2016-06-12 18:11:37 +01:00
Nikolay Amiantov
cdff1f61f0 lib: revert my bogus comment fix
See 5445e521b6 (commitcomment-17531945)
2016-05-19 01:26:04 +03:00
Nikolay Amiantov
5445e521b6 firefox: restore gstreamer support for older firefox releases 2016-05-19 00:28:49 +03:00
Nikolay Amiantov
c779e30d53 make*Path: implement via makeSearchPathOutput 2016-04-25 13:24:39 +03:00
Nikolay Amiantov
ab0a0c004e makeSearchPathOutputs: refactor to makeSearchPathOutput 2016-04-25 13:24:39 +03:00
Nikolay Amiantov
9b9fea305f makeSearchPath (and derivatives): check outputUnspecified 2016-04-13 22:09:41 +03:00
Nikolay Amiantov
e26119619f makePerlPath: try to guess outputs 2016-04-13 22:05:45 +03:00
Nikolay Amiantov
2a3b3d6f8b makeSearchPathOutputs: init function 2016-04-13 22:05:45 +03:00
Nikolay Amiantov
5ef56700b9 makeBinPath: guess outputs 2016-04-13 22:05:45 +03:00
Vladimír Čunát
30f14243c3 Merge branch 'master' into closure-size
Comparison to master evaluations on Hydra:
  - 1255515 for nixos
  - 1255502 for nixpkgs
2016-04-10 11:17:52 +02:00
Eelco Dolstra
fd2273e419 Stronger warning against usage of splitString 2016-04-01 13:52:54 +02:00
Vladimír Čunát
ab15a62c68 Merge branch 'master' into closure-size
Beware that stdenv doesn't build. It seems something more will be needed
than just resolution of merge conflicts.
2016-04-01 10:06:01 +02:00
zimbatm
7883ca774b lib/strings: document all the functions 2016-03-10 12:31:06 +00:00
Vladimír Čunát
716aac2519 Merge branch 'staging' into closure-size 2016-01-19 09:55:31 +01:00
Peter Simons
7ca8e13918 lib.getVersion: extend the function to cope with strings as well as derivations 2016-01-05 20:09:39 +01:00
Vladimír Čunát
f9f6f41bff Merge branch 'master' into closure-size
TODO: there was more significant refactoring of qtbase and plasma 5.5
on master, and I'm deferring pointing to correct outputs to later.
2015-12-31 09:53:02 +01:00
Thomas Tuegel
8baebb55d4 lib/strings: add readPathsFromFile 2015-12-16 15:00:34 -06:00
Luca Bruno
5b0352a6a4 Merge branch 'master' into closure-size 2015-12-11 18:31:00 +01:00
Domen Kožar
66c90e6e1f add makeBinPath function for abstracting away pkg/bin:pkg2/bin.. 2015-12-10 12:47:32 +01:00
Luca Bruno
a412927924 Merge remote-tracking branch 'origin/master' into closure-size 2015-11-25 21:37:30 +01:00
Christian Zagrodnick
1cdacc6aa2 lib/strings: add a toInt helper (close #11242)
While the function itself is pretty easy, it's not straitforward to find a way to convert string to int with nix.
2015-11-25 08:50:02 +01:00
Vladimír Čunát
5227fb1dd5 Merge commit staging+systemd into closure-size
Many non-conflict problems weren't (fully) resolved in this commit yet.
2015-10-03 13:33:37 +02:00
Eelco Dolstra
214d4fb73c Allow options with type "package" to be store paths
For example, this allows writing

  nix.package = /nix/store/786mlvhd17xvcp2r4jmmay6jj4wj6b7f-nix-1.10pre4206_896428c;

Also, document types.package in the manual.
2015-08-07 03:09:57 +02:00
Eelco Dolstra
c399f94bb3 intersperse: Fix quadratic performance 2015-07-24 15:58:34 +02:00
Eelco Dolstra
2d9885db9e Remove eqStrings
It's no longer needed. Also clean up some comments.
2015-07-24 15:58:34 +02:00
Eelco Dolstra
9cfd128a42 Use replaceStrings primop 2015-07-24 15:58:34 +02:00
Eelco Dolstra
6f6f66ac98 Use concatStringSep primop 2015-07-24 15:58:34 +02:00
Eelco Dolstra
0ae8b365b3 Rename misc.nix -> deprecated.nix 2015-07-23 18:31:54 +02:00
Eelco Dolstra
637e35deb9 Use foldl' instead of fold in some places 2015-07-23 18:31:54 +02:00
Nicolas B. Pierron
be65ac7c32 Issue 8152 - Reduce number of attribute set created by replaceChars. 2015-07-13 23:46:38 +02:00
Vladimír Čunát
c2fff72836 systemPackages, makeLibraryPath: try to guess outputs 2015-05-05 11:56:31 +02:00
Nikita Mikhailov
63ae447a0f Add 'fixedWidthString' and 'fixedWidthNumber' formatting functions 2015-03-08 19:02:32 +01:00
Eelco Dolstra
6c2bf141cf lib: Use arithmetic operators rather than builtins.add etc. 2014-10-05 01:10:06 +02:00
Igor Pashev
0383b57b3c Added concatMapStringsSep and concatImapStringsSep
Example: configure rewrite rules for Mediawiki

    RewriteEngine On
    RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} !-f
    RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} !-d
    ${concatMapStringsSep "\n" (u: "RewriteCond %{REQUEST_URI} !^${u.urlPath}") serverInfo.serverConfig.servedDirs}
    RewriteRule ${if config.enableUploads
2014-08-25 10:23:10 +02:00
Kirill Elagin
751a01063d lib: Do not take string context into account
Close #2883.
2014-06-11 20:22:58 +02:00
Vladimír Čunát
bf2bc0f0dc lib/strings: add hasPrefix and simplify hasSuffix
It was discussed as a part of #2570.
2014-05-13 11:05:37 +02:00
Vladimír Čunát
180bd65cd0 lib/strings: simplify removePrefix readability
It was discussed as a part of #2570.
2014-05-13 11:04:38 +02:00
Eelco Dolstra
ea36f3b868 fetchFromGitHub: Use .tar.gz instead of .zip
Also clean up the name attribute of fetchzip derivations a bit.
2014-05-09 15:53:44 +02:00