When native-compiling, gcc will install libraries into:
/nix/store/...-$targetConfig-gcc-$version-lib/lib
When cross-compiling, gcc will install libraries into:
/nix/store/...-$targetConfig-gcc-$version-lib/$targetConfig
When cross-compiling, we intended to create a link from $lib/lib to
$lib/$targetConfig, so that downstream users can always safely
assume that "${lib.getLib stdenv.cc.cc}/lib" is where the gcc
libraries are, regardless of whether `stdenv.cc.cc` is a cross
compiler or a native compiler.
Unfortunately, there were two problems with how we were trying to
create these links:
1. The link would be created only when `enableLibGccOutput==true`
2. The link was being created from the incorrect source
`$lib/lib/lib` instead of `$lib/lib`.
Both of these mistakes are my fault. This commit corrects them by
creating the link using `ln -Ts` (which is more predictable) and by
creating the link from `gcc/common/builder.nix` rather than from
`gcc/common/libgcc.nix`.
When build platform and host platform differ, but have the same
triple, the code in nixpkgs will consider it a cross compilation,
but gcc won't. This will lead some derivations to look for c++
headers in the wrong place. To solve this always output the headers
in the non-cross location, like we do for the other gcc headers
already.
The rest of our gcc expression prepends "${targetPlatform.config}-"
to paths and binaries if `hostPlatform!=targetPlatform`. The
`libgcc.nix` expression was using
'hostPlatform.config!=targetPlatform.config`, which caused it to
look in the wrong place when moving files. This commit corrects that.
It's not just gcc->clang cross compilation that produces different
platforms with equal configs — dynamic->static cross compilation can
have the same result (e.g. pkgsMusl.pkgsStatic). So, remove the check
for LLVM, but keep the check that the configs match.
This fixes pkgsMusl.nix on x86_64-linux, which depends on
pkgsMusl.pkgsStatic.buildPackages.gccWithoutTargetLibc.cc.
Fixes: 926c920c12 ("gcc: tighten condition for inhibit_libc=true")
For a cross-built native compiler, i.e. build!=(host==target), the
bundled libgfortran needs a gfortran which can run on the
buildPlatform and emit code for the targetPlatform. The compiler
which is built alongside gfortran in this configuration doesn't meet
that need: it runs on the hostPlatform.
This commit passes the necessary compiler via `GFORTRAN_FOR_TARGET`,
using `pkgsBuildTarget.gfortran`.
Cross-compiled binaries currently end up with two different libgcc
outpaths in their closure. This is harmless, but confusing.
The two libgccs are:
- One of them is the "first" targetPlatform libgcc, which is built
by the "first" cross-compiler. This "first libgcc" and "first
compiler" are used to build the targetPlatform glibc.
- Once glibc is built, we *rebuild* the cross-compiler, since gcc
can't enable most of its features unless you give it an
already-compiled targetPlatform glibc. When this "second"
compiler is built, it also builds an extra copy of libgcc.
This commit discards the second, extra libgcc, and instead puts a
reference to the first (correct) libgcc into the "second compiler"
`.passthru.libgcc`, so that anybody expecting `stdenv.cc.cc.libgcc`
to exist will still find it there.
Closes#249680
msvcrt is only one of the libcs in MinGW. We therefore
replace explictly testing for msvcrt with the isMinGW
predicate. This lays the foundation for ucrt64 support.
Without the change build on mips64-unknown-linux-gnu fails as:
$ nix-build -A buildPackages.gcc12 --argstr crossSystem mips64-linux
In file included from ...-glibc-mips64-unknown-linux-gnu-2.37-8-dev/include/bits/stat.h:25,
from ...-glibc-mips64-unknown-linux-gnu-2.37-8-dev/include/fcntl.h:78,
from ../../../../gcc-12.3.0/libsanitizer/sanitizer_common/sanitizer_linux.cpp:55:
...-glibc-mips64-unknown-linux-gnu-2.37-8-dev/include/bits/struct_stat.h:190:8: error: redefinition of 'struct stat64'
190 | struct stat64
| ^~~~~~
The change reverts commit 7df4387ebd
A few reasons to revert the commit:
1. The change was not enough to restore `-idirafter` override semantic
to match unwrapped compiler.
2. The change broke override semantics for cross-compilers
3. The change made override semantics different between cross-compiler
and native compiler
All of three have some overlap between, but I think it's important
to call all of them out.
The main fallout is the uboot builds, reported by cynerd.
Used the following test to check the override recovery:
$ nix shell github:NixOS/nixpkgs/release-22.05#pkgsCross.aarch64-multiplatform.stdenv.cc
$$ cat stdio.h
# empty
$$ printf "#include <stdio.h>" | aarch64-unknown-linux-gnu-gcc -E - -o - -idirafter . >/dev/null; echo $?
0
It failed before the change and succeded after.
Now that we use the standard builder, the commands produced by
pre-configure.nix are wrapped in a bash function. Inside of a bash
function, `export foo=` will still add `foo` to the environment of
any child processes forked after that point, but those variables
will *not* be visible to bash code which is outside of the
function-scope in which the `export` occurs.
Weird crap like this is yet another reason why we need to move away
from using bash for logic. Let's switch.
The bash `declare` builtin works differently when it occurs inside
function, as it now does due to breaking up the monolithic
`builder.sh` into separate phases. We have to add `-g` in order to
get the outside-of-a-bash-function behavior when using `declare`
within a bash function.
This commit adds arguments to `builder.nix`, making it a callable
function, and splits up the single massive shell script into
separate attributes for each phase. It also drops the first two
lines and the last line because these are part of the default
builder.
All script lines which were not part of a phase function have been
moved into `preUnpack` since this is the first phase that runs.
Subsequent commits will move parts of this to more sensible
locations.
This commit performs two search-and-replace operations:
- replace all `${` with `''${`
- replace `''` with `""` in shell scripts
This commit is left unsquashed to make review of the subsequent
commits easier.
The situation described in the comment preceding `export
inhibit_libc=true` does not match the conditional which follows it.
Specifically, the comment says that this line is meant for "clang
builds gcc" situations, yet it is enabled even when
`!stdenv.cc.isClang`.
This commit tightens the conditional to make it match the situation
described in the comment.
We want a `libgcc_s.so` to be built by the first stage
cross-compiler (withoutTargetLibc), since that is the compiler which
will compile the target libc.
This commit accomplishes that, by making three changes:
1. Replacing the `targetPlatform.libc == "msvcrt" &&` conditional
with `enableShared`, so that the code which cross-build
`libgcc_s.so` is used for all cross compilers capable of emitting
shared libraries.
2. Removing the `targetPlatform == hostPlatform` guard from the code
which produces the `libgcc` output.
3. Looking for build products in in "lib/${targetPlatform.config}/"
rather than "lib/", so we will find them when cross compiling.
This commit allows `gccCrossStageStatic` to build dynamically-linked
libraries. Since is no longer restricted to building static
libraries its name is no longer appropriate, and this commit also
renames it to the more-accurate `gccWithoutTargetLibc`.
By default, you can't build a gcc that knows how to create dynamic
libraries unless you have already built the targetPlatform libc.
Because of this, our gcc cross-compiler is built in two stages:
1. Build a cross-compiler (gccCrossStageStatic) that can build
only static libraries.
2. Use gccCrossStageStatic to compile the targetPlatform libc.
3. Use the targetPlatform libc to build a fully-capable cross
compiler.
You might notice that this pattern looks very similar to what we do
with `xgcc` in the stdenv bootstrap. Indeed it is! I would like to
work towards getting the existing stdenv bootstrap to handle cross
compilers as well. However we don't want to cripple `stdenv.xgcc`
by taking away its ability to build dynamic libraries.
It turns out that the only thing gcc needs the targetPlatform libc
for is to emit a DT_NEEDED for `-lc` into `libgcc.so`. That's it!
And since we don't use `gccCrossStageStatic` to build anything other
than libc, it's safe to omit the `DT_NEEDED` because that `libgcc`
will never be loaded by anything other than `libc`. So `libc` will
already be in the process's address space.
Other people have noticed this; crosstool-ng has been using this
approach for a very long time:
36ad0b17a7/scripts/build/cc/gcc.sh (L638-L640)
This commit deduplicates libgcc-related boilerplate which appears in
every version of our gcc expression, by moving it into libgcc.nix.
I will be submitting a separate PR which changes this boilerplate,
but that PR will be much easier to review if I can make the change
in just one place.
Meanwhile, *this* commit has no effect on eval:
$ for A in 10 11 12 13 4.8 4.9 6 7 8 9; do nix-instantiate . -A gcc$(echo $A | tr -d .); done 2>/dev/null | sort | tee before
/nix/store/1a37lnzpnz0dhm3lphiy2gcdrxgqa7ma-gcc-wrapper-4.8.5.drv
/nix/store/5szdivc8il0c3g94dq4wqnq5j77a9h6p-gcc-wrapper-11.4.0.drv
/nix/store/bmmc717wmnp1j2xkd3if5dfxicnflvn5-gcc-wrapper-7.5.0.drv
/nix/store/fc1ggpixv3wqcazchhl2hnn5zl5ds30l-gcc-wrapper-13.1.0.drv
/nix/store/j9c2b20w35r3ag5nxmklhagbwsgjhds2-gcc-wrapper-4.9.4.drv
/nix/store/nq7q57bxmsk2g457wr4b9449as3f216w-gcc-wrapper-12.3.0.drv
/nix/store/sqmkkfapzykapcs4azvxm83n786ga7q1-gcc-wrapper-10.4.0.drv
/nix/store/vxnz30i23mkl4ldsq485kxn7q0p2y4nf-gcc-wrapper-8.5.0.drv
/nix/store/yfhv0bv15cg5kj2xsb9fcgb6pdlw42v0-gcc-wrapper-6.5.0.drv
/nix/store/yi5gr75pb6kddnll10jg25hhndhkba7s-gcc-wrapper-9.5.0.drv
$ for A in 10 11 12 13 4.8 4.9 6 7 8 9; do nix-instantiate . -A gcc$(echo $A | tr -d .); done | sort | tee after
/nix/store/1a37lnzpnz0dhm3lphiy2gcdrxgqa7ma-gcc-wrapper-4.8.5.drv
/nix/store/5szdivc8il0c3g94dq4wqnq5j77a9h6p-gcc-wrapper-11.4.0.drv
/nix/store/bmmc717wmnp1j2xkd3if5dfxicnflvn5-gcc-wrapper-7.5.0.drv
/nix/store/fc1ggpixv3wqcazchhl2hnn5zl5ds30l-gcc-wrapper-13.1.0.drv
/nix/store/j9c2b20w35r3ag5nxmklhagbwsgjhds2-gcc-wrapper-4.9.4.drv
/nix/store/nq7q57bxmsk2g457wr4b9449as3f216w-gcc-wrapper-12.3.0.drv
/nix/store/sqmkkfapzykapcs4azvxm83n786ga7q1-gcc-wrapper-10.4.0.drv
/nix/store/vxnz30i23mkl4ldsq485kxn7q0p2y4nf-gcc-wrapper-8.5.0.drv
/nix/store/yfhv0bv15cg5kj2xsb9fcgb6pdlw42v0-gcc-wrapper-6.5.0.drv
/nix/store/yi5gr75pb6kddnll10jg25hhndhkba7s-gcc-wrapper-9.5.0.drv
$ diff -u after before
$
Before this change a Darwin gcc would output binaries that avoid
aligned_alloc, regardless of target platform. That's fine for Darwin
targets, but not for non-darwin targets such as pkgsCross.raspberryPi.
This change replaces the single configure flag with a flag for
each of build, host, target.
Idea by @trofi.
The 4aa95e3312 commit added support for
aarch64-darwin but also ignored platform flags if the build platform
is aarch64-darwin. This leads to confusing errors such as
`pkgsCross.raspberryPi` packages compiled with soft-float even though
the platform supports hard-float (and is built as such on other
platforms).
The correct way to ignore platform flags is to check `targetPlatform`,
not the build platform. This change fixes that.
While we're here, tigthen the special-case to cover only the problematic
flags: `-with-cpu` and `-with-arch`.
On MIPS, libsanitizer appears to have very detailed knowledge of the
Linux system call ABI manually encoded, by hand, into the library.
This encoding covers the o32 and 64 abis, but does not cover n32.
Adding support for n32 would be a major undertaking.
Without adding this encoding, builds of gcc will fail with errors
including (but not limited to) the following:
```
In file included from ../../../../gcc-12.2.0/libsanitizer/sanitizer_common/sanitizer_platform_limits_linux.cpp:21:
../../../../gcc-12.2.0/libsanitizer/sanitizer_common/sanitizer_platform_limits_linux.cpp:75:38: error: static assertion failed
75 | COMPILER_CHECK(struct_kernel_stat_sz == sizeof(struct stat));
| ~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
../../../../gcc-12.2.0/libsanitizer/sanitizer_common/sanitizer_internal_defs.h:348:44: note: in definition of macro 'COMPILER_CHECK'
348 | #define COMPILER_CHECK(pred) static_assert(pred, "")
| ^~~~
../../../../gcc-12.2.0/libsanitizer/sanitizer_common/sanitizer_platform_limits_linux.cpp:75:38: note: the comparison reduces to '(144 == 160)'
75 | COMPILER_CHECK(struct_kernel_stat_sz == sizeof(struct stat));
| ~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
../../../../gcc-12.2.0/libsanitizer/sanitizer_common/sanitizer_internal_defs.h:348:44: note: in definition of macro 'COMPILER_CHECK'
348 | #define COMPILER_CHECK(pred) static_assert(pred, "")
| ^~~~
...
In file included from ../../../../gcc-12.2.0/libsanitizer/sanitizer_common/sanitizer_linux.cpp:184:
../../../../gcc-12.2.0/libsanitizer/sanitizer_common/sanitizer_linux.cpp: In function '__sanitizer::uptr __sanitizer::internal_mmap(void*, uptr, int, int, int, u64)':
../../../../gcc-12.2.0/libsanitizer/sanitizer_common/sanitizer_syscall_generic.inc:19:24: error: '__NR_mmap2' was not declared in this scope
19 | # define SYSCALL(name) __NR_ ## name
| ^~~~~
../../../../gcc-12.2.0/libsanitizer/sanitizer_common/sanitizer_linux.cpp:198:27: note: in expansion of macro 'SYSCALL'
198 | return internal_syscall(SYSCALL(mmap2), addr, length, prot, flags, fd,
| ^~~~~~~
../../../../gcc-12.2.0/libsanitizer/sanitizer_common/sanitizer_linux.cpp: In function 'void __sanitizer::stat64_to_stat(stat64*, stat*)':
../../../../gcc-12.2.0/libsanitizer/sanitizer_common/sanitizer_linux.cpp:279:23: error: 'struct stat64' has no member named 'st_atim'; did you mean 'st_atime'?
279 | out->st_atime = in->st_atime;
| ^~~~~~~~
../../../../gcc-12.2.0/libsanitizer/sanitizer_common/sanitizer_linux.cpp:280:23: error: 'struct stat64' has no member named 'st_mtim'; did you mean 'st_mtime'?
280 | out->st_mtime = in->st_mtime;
| ^~~~~~~~
../../../../gcc-12.2.0/libsanitizer/sanitizer_common/sanitizer_linux.cpp:281:23: error: 'struct stat64' has no member named 'st_ctim'; did you mean 'st_ctime'?
281 | out->st_ctime = in->st_ctime;
| ^~~~~~~~
../../../../gcc-12.2.0/libsanitizer/sanitizer_common/sanitizer_linux.cpp: In function '__sanitizer::uptr __sanitizer::internal_stat(const char*, void*)':
../../../../gcc-12.2.0/libsanitizer/sanitizer_common/sanitizer_syscall_generic.inc:19:24: error: '__NR_stat64' was not declared in this scope
19 | # define SYSCALL(name) __NR_ ## name
| ^~~~~
../../../../gcc-12.2.0/libsanitizer/sanitizer_common/sanitizer_linux.cpp:353:30: note: in expansion of macro 'SYSCALL'
353 | int res = internal_syscall(SYSCALL(stat64), path, &buf64);
| ^~~~~~~
../../../../gcc-12.2.0/libsanitizer/sanitizer_common/sanitizer_linux.cpp: In function '__sanitizer::uptr __sanitizer::internal_lstat(const char*, void*)':
../../../../gcc-12.2.0/libsanitizer/sanitizer_common/sanitizer_syscall_generic.inc:19:24: error: '__NR_lstat64' was not declared in this scope
19 | # define SYSCALL(name) __NR_ ## name
| ^~~~~
../../../../gcc-12.2.0/libsanitizer/sanitizer_common/sanitizer_linux.cpp:378:30: note: in expansion of macro 'SYSCALL'
378 | int res = internal_syscall(SYSCALL(lstat64), path, &buf64);
| ^~~~~~~
../../../../gcc-12.2.0/libsanitizer/sanitizer_common/sanitizer_linux.cpp: In function '__sanitizer::uptr __sanitizer::internal_fstat(fd_t, void*)':
../../../../gcc-12.2.0/libsanitizer/sanitizer_common/sanitizer_syscall_generic.inc:19:24: error: '__NR_fstat64' was not declared in this scope
19 | # define SYSCALL(name) __NR_ ## name
| ^~~~~
../../../../gcc-12.2.0/libsanitizer/sanitizer_common/sanitizer_linux.cpp:397:30: note: in expansion of macro 'SYSCALL'
397 | int res = internal_syscall(SYSCALL(fstat64), fd, &buf64);
| ^~~~~~~
../../../../gcc-12.2.0/libsanitizer/sanitizer_common/sanitizer_linux.cpp: At global scope:
../../../../gcc-12.2.0/libsanitizer/sanitizer_common/sanitizer_linux.cpp:303:13: warning: 'void __sanitizer::kernel_stat_to_stat(kernel_stat*, stat*)' defined but not used [8;;https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-Wunused-function-Wunused-function8;;]
303 | static void kernel_stat_to_stat(struct kernel_stat *in, struct stat *out) {
| ^~~~~~~~~~~~~~~~~~~
mv -f .deps/sanitizer_mutex.Tpo .deps/sanitizer_mutex.Plo
make[4]: *** [Makefile:617: sanitizer_linux.lo] Error 1
mv -f .deps/sancov_flags.Tpo .deps/sancov_flags.Plo
../../../../gcc-12.2.0/libsanitizer/sanitizer_common/sanitizer_posix.cpp: In function '__sanitizer::fd_t __sanitizer::OpenFile(const char*, FileAccessMode, error_t*)':
../../../../gcc-12.2.0/libsanitizer/sanitizer_common/sanitizer_posix.cpp:162:27: warning: 'flags' may be used uninitialized [8;;https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-Wmaybe-uninitialized-Wmaybe-uninitialized8;;]
162 | fd_t res = internal_open(filename, flags, 0660);
| ~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~
../../../../gcc-12.2.0/libsanitizer/sanitizer_common/sanitizer_posix.cpp:156:7: note: 'flags' was declared here
156 | int flags;
| ^~~~~
```
