Commit Graph

182 Commits

Author SHA1 Message Date
Bernardo Meurer
6e086086d7
Merge pull request #188347 from zhaofengli/darwin-uuid 2023-11-30 10:51:28 -03:00
Minijackson
5581c0677c
bintools-wrapper: fix dynamic linker for powerpc64 big-endian
fixes #245162
2023-09-11 13:55:43 +02:00
Felix Bühler
0a2745684e
Merge pull request #239624 from Stunkymonkey/use-optionalString-then
treewide: use optionalString instead of 'then ""'
2023-07-22 13:02:47 +02:00
Artturi
8bf1b878cf
Merge pull request #239331 from pwaller/fix-pie-hardening 2023-07-06 02:14:35 +03:00
Felix Buehler
6672dde558 treewide: use optionalAttrs instead of 'else {}' 2023-06-25 11:01:34 -03:00
Felix Buehler
f3719756b5 treewide: use optionalString instead of 'then ""' 2023-06-24 20:19:19 +02:00
Peter Waller
e0d2053b87 build-support: Use response-expanded params in pie test
When a response file is in use, "$*" contains the response file and not
the parameters; both the linker and compiler wrappers are updated to use
the response-expanded params.

The compiler driver likes to pass parameters to the linker via a
response file, including -shared.

LLD rejects the combination of (-shared -pie), whereas other linkers
silently ignore the contradiction:

```
ld.lld: error: -shared and -pie may not be used together
```

This breaks certain configurations using LLD as a linker.

Changing `add-hardening.sh` results in a full rebuild. To avoid the
rebuild, here is a quick test case which shows the new hardening script
allows the link to succeed:

```
{ pkgs ? import <nixpkgs> {} }:

let
  # gcc silently accepts -shared -pie together, lld does not.
  linker = pkgs.wrapBintoolsWith { bintools = pkgs.llvmPackages.lld; };

  patchWrapper = prev: prev.overrideAttrs (final: prev: let
    prevScript = builtins.match (".*(/nix/store/[a-z0-9]+-add-hardening.sh).*") prev.postFixup;
  in {
    postFixup = (builtins.replaceStrings prevScript ["${./new-add-hardening.sh}"] prev.postFixup);
  });
in

pkgs.stdenv.mkDerivation {
  name = "nixpkgs-hardening-bug";

  src = pkgs.writeText "src.c" "int main(int argc, char* argv[]) { return 0; }";
  NIX_HARDENING_ENABLE = "pie";

  unpackPhase = ":";
  buildPhase = ''
    $CC -c -o src.o $src
    bash -x ${patchWrapper linker}/bin/ld.lld -o $out @${pkgs.writeText "responsefile" "-shared"} src.o
  '';
}
```

Fixes: #178162
Signed-off-by: Peter Waller <p@pwaller.net>
2023-06-23 11:09:49 +01:00
Alyssa Ross
194ddeefd5
wrapBintoolsWith: support LINK.EXE-style args in purity checks
LLD supports Windows-style linker arguments, but these previously
triggered purity check false positives, because it saw that they
started with a '/' and assumed they were paths.

This tweaks the path detection to allow through certain values that
could be paths, but are much more likely to be LINK.EXE-style flags.
The risk of false negatives here is low — the only things we'd now
fail to catch would be attempts to link with libraries in the root
directory, which doesn't happen in practice.

We also teach the wrapper how to apply its purity checks to library
paths specified with the /LIBPATH: option.

Tested that paths we expect to be rejected (like /lib/libfoo.so) still
are.
2023-04-29 01:37:00 +00:00
Weijia Wang
b2ef7956b6
Merge pull request #227560 from jackyliu16/loongnix-commit
lib.platforms.loongarch64: init
2023-04-28 13:21:42 +03:00
jackyliu16
b522b5a887 bintools: set dynamic linker 2023-04-28 12:09:43 +03:00
Manuel Mendez
31d6802773 gnatboot: rename to gnat-bootstrap
Most other bootstrap compilers are named -bootstrap so lets follow that
pattern.
2023-04-15 18:52:14 -04:00
Sandro
b04d4bad27
Merge pull request #216992 from SuperSandro2000/stdenvNative-fix-eval
{bintools,cc}-wrapper: don't fallback to version = null
2023-04-14 11:22:20 +02:00
Sandro Jäckel
7090651071
{bintools,cc}-wrapper: don't fallback to version = null
mkDerivation cannot handle that
2023-04-12 22:08:36 +02:00
Rahul Butani
b41933a1be
bintools-wrapper: specify SHA1 as the build-id hash style explicitly
NixOS/nixpkgs#146275 has more discussion on this; the abridged version
is that `lld` defaults to using `--build-id=fast` while GNU `ld` defaults
to `--build-id=sha1`. These differ in length and so
`separate-debug-info.sh`, as of this writing, errors on `lld`'s shorter
`--build-id=fast`-generated hashes.

`lld` offers the following `build-id` styles:
  - UUID (random; fast but bad for reproducibility)
  - fast (xxhash; fast but shorter hashes)
  - user provided hexstring
  - SHA1
  - MD5

GNU `ld` supports the latter three options, `mold` supports all of these
plus SHA256.

UUID is out because it's not reproducible, fast isn't supported by GNU
`ld`

Using a nix provided (sourced from the output base hash) hash as the
`build-id` seems tempting but would require a little extra work
(we have to include some characteristic of the binary being hashed
so that binaries within a derivation still have unique hashes; it
seems easy to get this wrong; i.e. a path based approach would make
two otherwise identical binaries that reside at different paths have
different `build-id` hashes)

That leaves SHA1 and MD5 and GNU `ld` already defaults to the former.

This commit adds `$NIX_BUILD_ID_STYLE` as an escape hatch, in case any
packages have strong opinions about which hash to use.

----

Note that if/when NixOS/nixpkgs#146275 goes through, this change can be
reverted if linker speed is a priority.
2023-02-25 12:49:40 -06:00
Gabriella Gonzalez
79484b1707
bintools: Add response file support to ld-wrapper (#213831)
The motivation behind this is to alleviate the problem
described in https://github.com/NixOS/nixpkgs/issues/41340.
I'm not sure if this completely fixes the problem, but it
eliminates one more area where we can exceed command line
length limits.

This is essentially the same change as in #112449,
except for `ld-wrapper.sh` instead of `cc-wrapper.sh`.

However, that change alone was not enough; on macOS the
`ld` provided by `darwin.cctools` fails if you use process
substitution to generate the response file, so I put up a
PR to fix that:

https://github.com/tpoechtrager/cctools-port/pull/131

… and I included a patch referencing that fix so that the
new `ld-wrapper` still works on macOS.
2023-02-23 17:05:18 -08:00
Artturi
ee54eb7d21
Merge pull request #216383 from Artturin/bintoolswrappermold 2023-02-17 19:32:06 +02:00
Artturin
b5abc3d090 bintools-wrapper: dont wrap ld if it doesn't exist
not all linkers have a ld binary in bin

also note the '${ld:-}' which allows users to set the ld path with a env
var

> '${foo:-val}' $foo, or val if unset (or null)
2023-02-17 04:56:48 +02:00
Artturin
3697ddeabe bintools-wrapper: wrap all 'ld.*'
allows using wrapBintoolsWith with all linkers

```
$ nix build ".#binutils"
$ ls ./result/bin/ld*
./result/bin/ld*  ./result/bin/ld.bfd*  ./result/bin/ld.gold*

$ nix build "nixpkgs#binutils"
$ ls ./result/bin/ld*
./result/bin/ld*  ./result/bin/ld.bfd* ./result/bin/ld.gold*
```
2023-02-15 20:51:16 +02:00
Felix Buehler
cdb39a86e0 treewide: use optionalString 2023-02-13 21:52:34 +01:00
Alyssa Ross
f5e63a0e9e bintools: add isGNU and isLLVM attributes 2023-01-20 18:35:25 +01:00
github-actions[bot]
49722fd14a
Merge master into staging-next 2023-01-13 18:01:34 +00:00
Boey Maun Suang
82b88d2db6 bintoolsDualAs: Add package
For reasons explained in the commit contents, in order to build the
native gnat package for x86_64-darwin, the native gnatboot package for
x86_64-darwin must have access to both the Clang integrated assembler
and the cctools GNU assembler for that platform.  This commit creates a
package with both of those assemblers that x86_64-darwin gnatboot can
then be wrapped with.
2023-01-07 18:32:12 +11:00
Artturin
adc8900df1 treewide: fix some core package structuredAttrs 2022-12-08 21:05:28 +02:00
Artturin
734d7df235 allow derivation attributes in env
derivations can be coerced to their output paths
2022-12-08 06:13:19 +02:00
Artturin
238a6053c4 stdenv: support opt-in __structuredAttrs
Co-authored-by: Robin Gloster <mail@glob.in>

stdenv: print message if structuredAttrs is enabled

stdenv: add _append

reduces the chance of a user doing it wrong

fix nix develop issue

output hooks don't work yet in nix develop though

making $outputs be the same on non-structuredAttrs and structuredAttrs
is too much trouble.

lets instead make a function that gets the output names

reading environment file '/nix/store/2x7m69a2sm2kh0r6v0q5s9z1dh41m4xf-xz-5.2.5-env-bin'
nix: src/nix/develop.cc:299: std::string Common::makeRcScript(nix::ref<nix::Store>, const BuildEnvironment&, const Path&): Assertion `outputs != buildEnvironment.vars.end()' failed.

use a function to get all output names instead of using $outputs

copy env functionality from https://github.com/NixOS/nixpkgs/pull/76732/commits
2022-12-08 06:13:19 +02:00
Zane van Iperen
1d0b9702fc bintools-wrapper: add dlltool, dllwrap, windmc, and windres 2022-12-03 00:28:59 +01:00
Victor Engmark
2a028c4f46 build-support: Use equivalent valid exit code
`exit -1` is equivalent to `exit 255`, since Bash does modulo 256 on the
number.

As per ShellCheck:

> SC2242 (error): Can only exit with status 0-255. Other data should be
> written to stdout/stderr.
2022-11-29 19:11:45 +13:00
Zhaofeng Li
06edf25685 bintools-wrapper: Set ZERO_AR_DATE and re-enable LC_UUID on Darwin
The LC_UUID is a hash over the output file contents.
2022-08-25 20:14:32 -06:00
Sergei Trofimovich
549e08c8e8 bintools-wrapper, cc-wrapper: avoid invalid export of 'expand-response-params'
POSIX sh (and `bash`) impose a restriction on environment variable name
format and disallow hypheps in the names. Normally it's not a problem
as nothing usually tries to refer nyphenated names.

One exception is `nix develop` (https://github.com/NixOS/nix/issues/6848):

    $ nix develop -f. gcc -L
    gcc-wrapper> ...-get-env.sh: line 70: expand-response-params: bad substitution

Note that bash usually uses explicitly created `expandResponseParams`
variant of the same variable.

To work the problem around let's avoid environment variable export and move
it to `passthru` for `cc` (used ina  few places) and remove it completely for
`binutils` (does not seem to be used at all).
2022-07-31 16:31:13 +01:00
github-actions[bot]
15686bdd94
Merge master into staging-next 2022-07-30 18:01:31 +00:00
Sergei Trofimovich
f2e3c9efff
Merge pull request #183415 from dramforever/uclibc-dyld
bintools-wrapper: Add dynamicLinker for uClibc
2022-07-30 18:39:06 +01:00
Artturin
0e16aa7b56 bintools-wrapper: symlink ar too
missed this in 1d44ac176c
2022-07-28 23:44:20 +03:00
dramforever
22ac2bce66 bintools-wrapper: Add dynamicLinker for uClibc 2022-07-29 01:43:36 +08:00
John Ericson
21966e13d2
Merge pull request #181943 from trofi/fix-cross-built-gcc
gcc: pass --with-build-sysroot=/
2022-07-23 23:52:07 -04:00
Sergei Trofimovich
34636efced gcc: pass --with-build-sysroot=/ for gcc builds
Without this change cross-built gcc fails to detect stack protector style:

    $ nix log -f pkgs/stdenv/linux/make-bootstrap-tools-cross.nix powerpc64le.bootGCC | fgrep __stack_chk_fail
    checking __stack_chk_fail in target C library... no
    checking __stack_chk_fail in target C library... no

It happens because gcc treats search paths differently:

    https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=gcc/configure.ac;h=446747311a6aec3c810ad6aa4190f7bd383b94f7;hb=HEAD#l2458

     if test x$host != x$target || test "x$TARGET_SYSTEM_ROOT" != x ||
        test x$build != x$host || test "x$with_build_sysroot" != x; then
       ...
       if test "x$with_build_sysroot" != "x"; then
         target_header_dir="${with_build_sysroot}${native_system_header_dir}"
       elif test "x$with_sysroot" = x; then
         target_header_dir="${test_exec_prefix}/${target_noncanonical}/sys-include"
       elif test "x$with_sysroot" = xyes; then
         target_header_dir="${test_exec_prefix}/${target_noncanonical}/sys-root${native_system_header_dir}"
       else
         target_header_dir="${with_sysroot}${native_system_header_dir}"
       fi
     else
       target_header_dir=${native_system_header_dir}
     fi

By passing --with-build-sysroot=/ we trick cross-case to use
`target_header_dir="${with_sysroot}${native_system_header_dir}"`
which makes it equivalent to non-cross
`target_header_dir="${with_build_sysroot}${native_system_header_dir}"`

Tested the following setups:
- cross-compiler without libc headers (powerpc64le-static)
- cross-compiler with libc headers (powerpc64le-debug)
- cross-build compiler with libc headers (powerpc64le bootstrapTools)

Before the change only 2 of 3 compilers detected libc headers.
After the change all 3 compilers detected libc headers.

For darwin we silently ignore '-syslibroot //' argument as it does not
introduce impurities.

While at it dropped mingw special case for no-libc build. Before the change
we passed both '--without-headers --with-native-system-headers-dir' for
no-libc gcc-static builds. This tricked darwin builds to find sys/sdt.h
and fail inhibid_libc builds. Now all targets avoid passing native headers
for gcc-static builds.

While at it fixed correct headers passing to
--with-native-system-headers-dir= in host != target case: we were passing
host's headers where intention was to pass target's headers.
Noticed the mismatch as a build failure on pkgsCross.powernv.stdenv.cc
on darwin where `sys/sdt.h` is present in host's headers (libSystem)
but not target's headers (`glibc`).

Co-authored-by: Adam Joseph <54836058+amjoseph-nixpkgs@users.noreply.github.com>
2022-07-23 18:40:07 +01:00
Artturin
e682dd84cf bintools-wrapper: symlink unsymlinked binaries from -unwrapped
this shouldn't change any binary available in the default build environment
because bintools-unwrapped is already in path ( idk where it comes from but i know because objcopy is in path but not in the wrapper )

this just makes all the binaries available under 'bintools' instead of
having to use 'bintools-unwrapped'

reduces confusion because now 'objcopy' and others will be in 'bintools'
2022-07-22 02:48:29 +03:00
Artturin
1d44ac176c treewide: add enableParallelBuilding's to bootstrap packages so hashes stay the same
when enableParallelBuildingByDefault is enabled

verified with
`nix-diff $(nix eval ".#gcc-unwrapped.drvPath") $(nix eval --expr 'with import ./. { config = { enableParallelBuildingByDefault = true; }; }; gcc-unwrapped.drvPath' --impure)`
2022-05-25 16:03:14 +03:00
Janne Heß
c911240e9c
Revert "Add mingwW64-llvm cross-system." 2022-05-18 13:50:23 +02:00
Shea Levy
89ad105c2e
Merge branch 'no-ldemulation' into staging 2022-05-18 06:30:39 -04:00
Shea Levy
d68a532d1b
Set a default machine type when using lld targeting Windows.
Fixes autotools checks parsing --help
2022-05-09 10:09:42 -04:00
Shea Levy
4c0d5f80fc
bintools-wrapper: Don't pass -z to lld targeting Windows 2022-05-09 10:05:45 -04:00
Shea Levy
dee9af9323
bintools: Add isXXX flags to check linker type 2022-05-09 10:03:48 -04:00
Shea Levy
b00016d9d9
bintools-wrapper: Remove LDEMULATION setting.
As far as I can tell, this has never actually done anything, as
LDEMULATION is not exported. I tried exporting it and builds broke,
and as it doesn't seem to have caused any problems as a noop all these
years it didn't seem worth investigating further.
2022-05-07 06:13:58 -04:00
Moritz Angermann
14996789a1
Check link type based on expanded parameters
So far we've ignored response files in arguments, and did not
check linkType against expanded parameters.  This means if
we have `-static` in a @reponse-file, linkType will not be
set to `-static` as we never check against the expanded arguments
from response files.
2022-01-01 20:30:56 +08:00
oxalica
3b069a2ef8
bintools-wrapper: add dynamicLinker for riscv 2021-11-26 00:32:33 +08:00
sterni
5d0972c63b
Merge pull request #138289 from sternenseemann/fix-non-gnu-strip
stdenv: move --enable-deterministic-archives flag into GNU wrapper
2021-10-07 12:29:59 +02:00
r-burns
16728283c3
Merge pull request #135619 from r-burns/fpie-musl-fixes
[staging] {cc,binutils}-wrapper: fixes for PIE hardening
2021-09-23 17:55:08 -07:00
Ryan Burns
37d089c1af {cc,binutils}-wrapper: match leading/trailing arguments
fixes e.g.:
pkgsMusl.libfsm
pkgsMusl.libiscsi
pkgsMusl.nsjail
pkgsMusl.pv

match strings have whitespace on either side, which wasn't
matching leading/trailing arguments previously
2021-09-20 17:24:04 -07:00
sternenseemann
11fe2fc3cf stdenv: move --enable-deterministic-archives flag into GNU wrapper
`--enable-deterministic-archives` is a GNU specific strip flag and
causes other strip implementations (for example LLVM's, see #138013)
to fail. Since strip failures are ignored, this means that stripping
doesn't work at all in certain situation (causing unnecessary
dependencies etc.).

To fix this, no longer pass `--enable-deterministic-archives`
unconditionally, but instead add it in a GNU binutils specific strip
wrapper only.

`commonStripFlags` was only used for this flag, so we can remove
it altogether.

Future work could be to make a generic strip wrapper, with support for
nix-support/strip-flags-{before,after} and NIX_STRIP_FLAGS_{BEFORE,AFTER}.
This possibly overkill and unnecessary though -- also with the
additional challenge of incorporating the darwin strip wrapper somehow.
2021-09-18 15:11:17 +02:00
sternenseemann
fb1348d433 bintools-wrapper: check if bintools to wrap isGNU, not stdenv
The wrapped bintools and the one used to build the wrapper can differ.
2021-09-17 17:06:24 +02:00