Regression introduced in PR #8119180729b6787. The file does not exist
somewhere during bootstrap of pkgsStatic.busybox which is used in nix
(by default).
I tested the builds.
If an empty string is passed to `-idirafter`, it breaks gcc. This commit makes
the stdenv less fragile by expanding out the shell glob and ensuring no empty
arguments get passed.
Before, we'd always use `cc = null`, and check for that. The problem is
this breaks for cross compilation to platforms that don't support a C
compiler.
It's a very subtle issue. One might think there is no problem because we
have `stdenvNoCC`, and presumably one would only build derivations that
use that. The problem is that one still wants to use tools at build-time
that are themselves built with a C compiler, and those are gotten via
"splicing". The runtime version of those deps will explode, but the
build time / `buildPackages` versions of those deps will be fine, and
splicing attempts to work this by using `builtins.tryEval` to filter out
any broken "higher priority" packages (runtime is the default and
highest priority) so that both `foo` and `foo.nativeDrv` works.
However, `tryEval` only catches certain evaluation failures (e.g.
exceptions), and not arbitrary failures (such as `cc.attr` when `cc` is
null). This means `tryEval` fails to let us use our build time deps, and
everything comes apart.
The right solution is, as usually, to get rid of splicing. Or, baring
that, to make it so `foo` never works and one has to explicitly do
`foo.*`. But that is a much larger change, and certaily one unsuitable
to be backported to stable.
Given that, we instead make an exception-throwing `cc` attribute, and
create a `hasCC` attribute for those derivations which wish to
condtionally use a C compiler: instead of doing `stdenv.cc or null ==
null` or something similar, one does `stdenv.hasCC`. This allows quering
without "tripping" the exception, while also allowing `tryEval` to work.
No platform without a C compiler is yet wired up by default. That will
be done in a following commit.
We want to make sure this value is explicitly set. Infering it for
every arch leads to annoying failures like:
https://hydra.nixos.org/build/92583832/
Perhaps we can enable it in the future with some smarter handling of
cc-wrapper.sh.
Adds pkgsCross.wasm32 and pkgsCross.wasm64. Use it to build Nixpkgs
with a WebAssembly toolchain.
stdenv/cross: use static overlay on isWasm
isWasm doesn’t make sense dynamically linked.
It is useful to make these dynamic and not bake them into gcc. This
means we don’t have to rebuild gcc to change these values. Instead, we
will pass cflags to gcc based on platform values. This was already
done hackily for android gcc (which is multi-target), but not for our
own gccs which are single target.
To accomplish this, we need to add a few things:
- add ‘arch’ to cpu
- add NIX_CFLAGS_COMPILE_BEFORE flag (goes before args)
- set -march everywhere
- set mcpu, mfpu, mmode, and mtune based on targetPlatform.gcc flags
cc-wrapper: only set -march when it is in the cpu type
Some architectures don’t have a good mapping of -march. For instance
POWER architecture doesn’t support the -march flag at all!
https://gcc.gnu.org/onlinedocs/gcc/RS_002f6000-and-PowerPC-Options.html#RS_002f6000-and-PowerPC-Options
this adds libc++ to the LLVM cross, giving us access to the full
Nixpkgs set. This requires 4 stages of wrapped compilers:
- Clang with no libraries
- Clang with just compiler-rt
- Clang with Libc, and compiler-rt
- Clang with Libc++, Libc, and compiler-rt
clang needs to find headers + libraries for compiling with libc++. We
need to add a libcxx argument to cc-wrapper. This means you do not
have to pass in c++ headers directly.
This resolves the last case remaining of #30670. Darwin clang++ now
works properly.
Fixes#30670
With the previous commit `propagateDoc` is now always given the correct value
(i.e. it is never set to `true` when there are no `man` and `info` outputs).
Hence, we can simply symlink the original outputs to the wrapper outputs.
Pros:
- simpler, less indirection compared to `propagated-user-env-packages`,
- uses less inodes (1 symlink, which nix then simply automatically resolves
and removes, vs. two directories and a file),
- makes direct references like "export MANPATH=${stdenv.cc.man}/share/man"
simply work.
Cons:
- I'm not aware of any.
This and the previous commit together almost completely revert commits
fde7296a47,
fa41297209, and
c981787db9.
- respect libc’s incdir and libdir
- make non-unix systems single threaded
- set LIMITS_H_TEST to false for avr
- misc updates to support new libc’s
- use multilib with avr
For threads we want to use:
- posix on unix systems
- win32 on windows
- single on everything else
For avr:
- add library directories for avrlibc
- to disable relro and bind
- avr5 should have precedence over avr3 - otherwise gcc uses the wrong one
02c09e0171 (NixOS/nixpkgs#44558) was reverted in
c981787db9 but, as it turns out, it fixed an issue
I didn't know about at the time: the values of `propagateDoc` options were
(and now again are) inconsistent with the underlying things those wrappers wrap
(see NixOS/nixpkgs#46119), which was (and now is) likely to produce more instances
of NixOS/nixpkgs#43547, if not now, then eventually as stdenv changes.
This patch (which is a simplified version of the original reverted patch) is the
simplest solution to this whole thing: it forces wrappers to directly inspect the
outputs of the things they are wrapping instead of making stdenv guess the correct
values.
The hack of using `crossConfig` to enforce stricter handling of
dependencies is replaced with a dedicated `strictDeps` for that purpose.
(Experience has shown that my punning was a terrible idea that made more
difficult and embarrising to teach teach.)
Now that is is clear, a few packages now use `strictDeps`, to fix
various bugs:
- bintools-wrapper and cc-wrapper
... binutils and gcc add it already anyway.
Without this it's easy to get cross-toolchain paths longer than 256
chars and nix-daemon will then fail to commit them to /nix/store on XFS.
Factor a bintools (i.e. binutils / cctools) wrapper out of cc-wrapper. While
only LD is wrapped, the setup hook defines environment variables on behalf of
other utilites.
On non-GNU (gcc) compilers, there is no "/lib/gcc/..."
so when this is eventually expanded this is empty
resulting in an incomplete "-idirafter " that
eats the next argument:
-idirafter -B/nix/store/wamjwwdvkmhbf4f2902nhw8jxxzv0hy3-clang-wrapper-4.0.1/bin/
Certain tools, e.g. compilers, are customarily prefixed with the name of
their target platform so that multiple builds can be used at once
without clobbering each other on the PATH. I was using identifiers named
`prefix` for this purpose, but that conflicts with the standard use of
`prefix` to mean the directory where something is installed. To avoid
conflict and confusion, I renamed those to `targetPrefix`.
If a dynamic linker for target is not found the generated script fails
due to unbound variable error (due to "set -u"). Correct by specifying
default value with dynamicLinker:- and not generating ldflagsBefore if
no linker is found.
This problem was found when cross compiling to mingw32 targets
cc-wrapper may wrap a cc-compiler, but it doesn't need one to build
itself. (c.f. expand-response-params is a separate derivation.) This
helps avoid cycles on the cross stuff, in addition to removing a
useless dependency edge.
I could have been super careful with overrides in the stdenv to avoid
the mass rebuild, but I don't think it's worth it.
1. `crossDrv` is now the default so we don't need to worry about that in
build != host builds.
2. shell is the build time shell, so `wrapCCCross` doesn't need to
worry, as build == host.
3. `shell.shellPath` will always be appended where useful.
4. Complicated `shell == ""` logic served no purpose.
ccPath is only defined below, so this condition would never be true.
Worse, that's not quite true: what if somebody happend to have `/clang`
and no sandboxing. Boy, wouldn't that be annoying to debug!
Having multiple compilers in the build environment would result in an
invalid LD_DYLD_PATH like /usr/lib/dyld/usr/lib/dyld.
Since the path is hardcoded in XNU it can't be anything but
/usr/lib/dyld anyway.
This is an ugly temp hack for cross compilation, but now we have something better on the way.
Bind `infixSalt` as an environment variable as it will be used in it.
As described in https://github.com/NixOS/nixpkgs/issues/18461, MacOS no
longer accepts dylibs which only reexport other dylibs, because their
symbol tables are empty. To get around this, we define an object file
with a single "private extern" symbol, which hopefully won't clobber
anything.
As @oxij points out in [1], this breakage is especially serious because
it changes the contents of built environments without a corresonding
change in their hashes. Also, the revert is easier than I thought.
This reverts commit 3cb745d5a6.
[1]: https://github.com/NixOS/nixpkgs/pull/27427#issuecomment-317293040
Besides deduplicating overlapping logic, clear warning messages were
added for:
- No glob/path for dynamic linker provided (use default glob)
- Glob did not expand to anything (don't append flag)
- glob expanded to multiple things (take first, like before)
This makes those files a bit easier to read. Also, for what it's worth,
it brings us one baby step closer to handling spaces in store paths.
Also, I optimized handling of many transitive deps with read. Probably,
not very beneficial, but nice to enforce the pkg-per-line structure.
Doing so let me find much dubious code and fix it.
Two misc notes:
- `propagated-user-env-packages` also needed to be adjusted as
sometimes it is copied to/from the propagated input files.
- `local fd` should ensure that file descriptors aren't clobbered
during recursion.
I think it's ok to export things which aren't wrapped. The cc-wrapper
can be thought of as responsible for all of binutils and the c
compiler, only wrapping those binaries which are necessary to
interposition---as opposed to all binaries it thinks are relevaant.
Conversely, adding the setup hook to the unwrapped compilers would be
unforunate as hooks are ugly hacks and the compilers themselves take
a long time to rebuild. Better to wholely separate "pure packages" from
hacks.
Eventually we should avoid this "pre-wrapping" and just update those
files in nixpkgs. This mass-rebuild change is best done along with
those needed to reduce the disparity between native and cross (i.e.
making native the "identity cross").
We now (on cross) require per-target flag interposition by putting the
triple in the names of the relevant environment variables, e.g:
export NIX_arm_unknown_linux_gnu_CFLAGS_COMPILE=...
The wrapper also has a `infixSalt` attribute (and "_" prefixed and
suffixed variants) to assist downstream packages.
Note how that the dashes are replaced to keep the identifier valid.
Using names like this allows us to keep the settings for different
compilers seperate.
I think it might be even better to use names like `NIX_{BUILD,HOST}...`
using the platform's role rather than the platform itself, but this
would be more work as the previous stages' tools would have to be re-
wrapped to take on their new role. I therefore didn't do this for now,
but that route should be thoroughly explored in the future.
This value is require to get c++ std include path for libclang based tools (vim plugins in my case).
I currently extract it this with this rather command:
```
eval echo $(nix-instantiate --eval --expr 'with (import <nixpkgs>) {}; clang.default_cxx_stdlib_compile')
```
it did not trigger any recompilation on my system.
The following parameters are now available:
* hardeningDisable
To disable specific hardening flags
* hardeningEnable
To enable specific hardening flags
Only the cc-wrapper supports this right now, but these may be reused by
other wrappers, builders or setup hooks.
cc-wrapper supports the following flags:
* fortify
* stackprotector
* pie (disabled by default)
* pic
* strictoverflow
* format
* relro
* bindnow
The ld-wrapper.sh script calls `readlink` in some circumstances. We need
to ensure that this is the `readlink` from the `coreutils` package so
that flag support is as expected.
This is accomplished by explicitly setting PATH at the top of each shell
script.
Without doing this, the following happens with a trivial `main.c`:
```
nix-env -f "<nixpkgs>" -iA pkgs.clang
$ clang main.c -L /nix/../nix/store/2ankvagznq062x1gifpxwkk7fp3xwy63-xnu-2422.115.4/Library -o a.out
readlink: illegal option -- f
usage: readlink [-n] [file ...]
```
The key element is the `..` in the path supplied to the linker via a
`-L` flag. With this patch, the above invocation works correctly on
darwin, whose native `/usr/bin/readlink` does not support the `-f` flag.
The explicit path also ensures that the `grep` called by `cc-wrapper.sh`
is the one from Nix.
Fixes#6447
... because cc-wrapper is meant to propagate man pages into user envs,
and info pages are rather large.
Also replace the duplicate g++ and gcc man1 pages by a symlink.
Note: -B argument seems more like for gcc's main output,
though it's used in a bit strange way here.
(Upstream default is /usr/lib/gcc/ which we don't move.)
This reverts d927da8dae. Having a copy
of gcc-wrapper/setup-hook.sh is bad for maintainability - it had
already started to diverge. Also, gccStdInc gave a nix-env conflict
with the standard gcc. And it wasn't actually used in Nixpkgs.
Instead, if you really need to change "-isystem" to "-I", you can now
set ccIncludeFlag to "-I".
Now development stuff is propagated from the first output,
and userEnvPkgs from the one with binaries.
Also don't move *.la files (yet). It causes problems, and they're small.
- there were many easy merge conflicts
- cc-wrapper needed nontrivial changes
Many other problems might've been created by interaction of the branches,
but stdenv and a few other packages build fine now.
- Add a conditional flag for the c++ std lib
- Build binaries that get linked by our own dyld (someday)
- Automatically add framework directories in the setup hook
