colin/nixpkgs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge pull request #22107 from Ericson2314/cross-tepid

Browse Source 
Somewhat saner cross-compiling through bootstrapping
This commit is contained in:
John Ericson 2017-01-24 15:09:56 -05:00 committed by GitHub
commit 5ad696b067
28 changed files with 719 additions and 355 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

153
doc/cross-compilation.xml Normal file
View File

@ -0,0 +1,153 @@
<chapter xmlns="http://docbook.org/ns/docbook"
xmlns:xlink="http://www.w3.org/1999/xlink"
xml:id="chap-cross">
<title>Cross-compilation</title>
<section xml:id="sec-cross-intro">
<title>Introduction</title>
<para>
"Cross-compilation" means compiling a program on one machine for another type of machine.
For example, a typical use of cross compilation is to compile programs for embedded devices.
These devices often don't have the computing power and memory to compile their own programs.
One might think that cross-compilation is a fairly niche concern, but there are advantages to being rigorous about distinguishing build-time vs run-time environments even when one is developing and deploying on the same machine.
Nixpkgs is increasingly adopting this opinion in that packages should be written with cross-compilation in mind, and nixpkgs should evaluate in a similar way (by minimizing cross-compilation-specific special cases) whether or not one is cross-compiling.
</para>
<para>
This chapter will be organized in three parts.
First, it will describe the basics of how to package software in a way that supports cross-compilation.
Second, it will describe how to use Nixpkgs when cross-compiling.
Third, it will describe the internal infrastructure supporting cross-compilation.
</para>
</section>
<!--============================================================-->
<section xml:id="sec-cross-packaging">
<title>Packing in a cross-friendly manner</title>
<section>
<title>Platform parameters</title>
<para>
The three GNU Autoconf platforms, <wordasword>build</wordasword>, <wordasword>host</wordasword>, and <wordasword>cross</wordasword>, are historically the result of much confusion.
<link xlink:href="https://gcc.gnu.org/onlinedocs/gccint/Configure-Terms.html" /> clears this up somewhat but there is more to be said.
An important advice to get out the way is, unless you are packaging a compiler or other build tool, just worry about the build and host platforms.
Dealing with just two platforms usually better matches people's preconceptions, and in this case is completely correct.
</para>
<para>
In Nixpkgs, these three platforms are defined as attribute sets under the names <literal>buildPlatform</literal>, <literal>hostPlatform</literal>, and <literal>targetPlatform</literal>.
All are guaranteed to contain at least a <varname>platform</varname> field, which contains detailed information on the platform.
All three are always defined at the top level, so one can get at them just like a dependency in a function that is imported with <literal>callPackage</literal>:
<programlisting>{ stdenv, buildPlatform, hostPlatform, fooDep, barDep, .. }: ...</programlisting>
</para>
<warning><para>
These platforms should all have the same structure in all scenarios, but that is currently not the case.
When not cross-compiling, they will each contain a <literal>system</literal> field with a short 2-part, hyphen-separated summering string name for the platform.
But, when when cross compiling, <literal>hostPlatform</literal> and <literal>targetPlatform</literal> may instead contain <literal>config</literal> with a fuller 3- or 4-part string in the manner of LLVM.
We should have all 3 platforms always contain both, and maybe give <literal>config</literal> a better name while we are at it.
</para></warning>
<variablelist>
<varlistentry>
<term><varname>buildPlatform</varname></term>
<listitem><para>
The "build platform" is the platform on which a package is built.
Once someone has a built package, or pre-built binary package, the build platform should not matter and be safe to ignore.
</para></listitem>
</varlistentry>
<varlistentry>
<term><varname>hostPlatform</varname></term>
<listitem><para>
The "host platform" is the platform on which a package is run.
This is the simplest platform to understand, but also the one with the worst name.
</para></listitem>
</varlistentry>
<varlistentry>
<term><varname>targetPlatform</varname></term>
<listitem>
<para>
The "target platform" is black sheep.
The other two intrinsically apply to all compiled software—or any build process with a notion of "build-time" followed by "run-time".
The target platform only applies to programming tools, and even then only is a good for for some of them.
Briefly, GCC, Binutils, GHC, and certain other tools are written in such a way such that a single build can only compiler code for a single platform.
Thus, when building them, one must think ahead about what platforms they wish to use the tool to produce machine code for, and build binaries for each.
</para>
<para>
There is no fundamental need to think about the target ahead of time like this.
LLVM, for example, was designed from the beginning with cross-compilation in mind, and so a normal LLVM binary will support every architecture that LLVM supports.
If the tool supports modular or pluggable backends, one might imagine specifying a <emphasis>set</emphasis> of target platforms / backends one wishes to support, rather than a single one.
</para>
<para>
The biggest reason for mess, if there is one, is that many compilers have the bad habit a build process that builds the compiler and standard library/runtime together.
Then the specifying target platform is essential, because it determines the host platform of the standard library/runtime.
Nixpkgs tries to avoid this where possible too, but still, because the concept of a target platform is so ingrained now in Autoconf and other tools, it is best to support it as is.
Tools like LLVM that don't need up-front target platforms can safely ignore it like normal packages, and it will do no harm.
</para>
</listitem>
</varlistentry>
</variablelist>
<note><para>
If you dig around nixpkgs, you may notice there is also <varname>stdenv.cross</varname>.
This field defined as <varname>hostPlatform</varname> when the host and build platforms differ, but otherwise not defined at all.
This field is obsolete and will soon disappear—please do not use it.
</para></note>
</section>
<section>
<title>Specifying Dependencies</title>
<para>
As mentioned in the introduction to this chapter, one can think about a build time vs run time distinction whether cross-compiling or not.
In the case of cross-compilation, this corresponds with whether a derivation running on the native or foreign platform is produced.
An interesting thing to think about is how this corresponds with the three Autoconf platforms.
In the run-time case, the depending and depended-on package simply have matching build, host, and target platforms.
But in the build-time case, one can imagine "sliding" the platforms one over.
The depended-on package's host and target platforms (respectively) become the depending package's build and host platforms.
This is the most important guiding principle behind cross-compilation with Nixpkgs, and will be called the <wordasword>sliding window principle</wordasword>.
In this manner, given the 3 platforms for one package, we can determine the three platforms for all its transitive dependencies.
</para>
<note><para>
The depending package's target platform is unconstrained by the sliding window principle, which makes sense in that one can in principle build cross compilers targeting arbitrary platforms.
</para></note>
<warning><para>
From the above, one would surmise that if a package is being built with a <literal>(build, host, target)</literal> platform triple of <literal>(foo, bar, bar)</literal>, then its build-time dependencies would have a triple of <literal>(foo, foo, bar)</literal>, and <emphasis>those packages'</emphasis> build-time dependencies would have triple of <literal>(foo, foo, foo)</literal>.
In other words, it should take two "rounds" of following build-time dependency edges before one reaches a fixed point where, by the sliding window principle, the platform triple no longer changes.
Unfortunately, at the moment, we do <emphasis>not</emphasis> implement this correctly, and after only one round of following build-time dependencies is the fixed point reached, with target incorrectly kept different than the others.
</para></warning>
<para>
How does this work in practice? Nixpkgs is now structured so that build-time dependencies are taken from from <varname>buildPackages</varname>, whereas run-time dependencies are taken from the top level attribute set.
For example, <varname>buildPackages.gcc</varname> should be used at build time, while <varname>gcc</varname> should be used at run time.
Now, for most of Nixpkgs's history, there was no <varname>buildPackages</varname>, and most packages have not been refactored to use it explicitly.
Instead, one can use the four attributes used for specifying dependencies as documented in <link linkend="ssec-stdenv-attributes" />.
We "splice" together the run-time and build-time package sets with <varname>callPackage</varname>, and then <varname>mkDerivation</varname> for each of four attributes pulls the right derivation out.
This splicing can be skipped when not cross compiling as the package sets are the same, but is a bit slow for cross compiling.
Because of this, a best-of-both-worlds solution is in the works with no splicing or explicit access of <varname>buildPackages</varname> needed.
For now, feel free to use either method.
</para>
</section>
</section>
<!--============================================================-->
<section xml:id="sec-cross-usage">
<title>Cross-building packages</title>
<para>
To be written.
This is basically unchanged so see the old wiki for now.
</para>
</section>
<!--============================================================-->
<section xml:id="sec-cross-infra">
<title>Cross-compilation infrastructure</title>
<para>To be written.</para>
<note><para>
If one explores nixpkgs, they will see derivations with names like <literal>gccCross</literal>.
Such <literal>*Cross</literal> derivations is a holdover from before we properly distinguished between the host and target platforms
—the derivation with "Cross" in the name covered the <literal>build = host != target</literal> case, while the other covered the <literal>host = target</literal>, with build platform the same or not based on whether one was using its <literal>.nativeDrv</literal> or <literal>.crossDrv</literal>.
This ugliness will disappear soon.
</para></note>
</section>
</chapter>

1
doc/manual.xml
View File

@ -13,6 +13,7 @@
<xi:include href="quick-start.xml" />
<xi:include href="stdenv.xml" />
<xi:include href="multiple-output.xml" />
<xi:include href="cross-compilation.xml" />
<xi:include href="configuration.xml" />
<xi:include href="functions.xml" />
<xi:include href="meta.xml" />

90
doc/stdenv.xml
View File

@ -194,33 +194,52 @@ genericBuild
tools.</para></listitem>
</varlistentry>
</variablelist>
<variablelist>
<title>Variables specifying dependencies</title>
<varlistentry>
<term><varname>nativeBuildInputs</varname></term>
<listitem><para>
A list of dependencies used by the new derivation at <emphasis>build</emphasis>-time.
I.e. these dependencies should not make it into the package's runtime-closure, though this is currently not checked.
For each dependency <replaceable>dir</replaceable>, the directory <filename><replaceable>dir</replaceable>/bin</filename>, if it exists, is added to the <envar>PATH</envar> environment variable.
Other environment variables are also set up via a pluggable mechanism.
For instance, if <varname>buildInputs</varname> contains Perl, then the <filename>lib/site_perl</filename> subdirectory of each input is added to the <envar>PERL5LIB</envar> environment variable.
See <xref linkend="ssec-setup-hooks"/> for details.
</para></listitem>
</varlistentry>
<varlistentry>
<term><varname>buildInputs</varname></term>
<listitem><para>A list of dependencies used by
<literal>stdenv</literal> to set up the environment for the build.
For each dependency <replaceable>dir</replaceable>, the directory
<filename><replaceable>dir</replaceable>/bin</filename>, if it
exists, is added to the <envar>PATH</envar> environment variable.
Other environment variables are also set up via a pluggable
mechanism. For instance, if <varname>buildInputs</varname>
contains Perl, then the <filename>lib/site_perl</filename>
subdirectory of each input is added to the <envar>PERL5LIB</envar>
environment variable. See <xref linkend="ssec-setup-hooks"/> for
details.</para></listitem>
<listitem><para>
A list of dependencies used by the new derivation at <emphasis>run</emphasis>-time.
Currently, the build-time environment is modified in the exact same way as with <varname>nativeBuildInputs</varname>.
This is problematic in that when cross-compiling, foreign executables can clobber native ones on the <envar>PATH</envar>.
Even more confusing is static-linking.
A statically-linked library should be listed here because ultimately that generated machine code will be used at run-time, even though a derivation containing the object files or static archives will only be used at build-time.
A less confusing solution to this would be nice.
</para></listitem>
</varlistentry>
<varlistentry>
<term><varname>propagatedNativeBuildInputs</varname></term>
<listitem><para>
Like <varname>nativeBuildInputs</varname>, but these dependencies are <emphasis>propagated</emphasis>:
that is, the dependencies listed here are added to the <varname>nativeBuildInputs</varname> of any package that uses <emphasis>this</emphasis> package as a dependency.
So if package Y has <literal>propagatedBuildInputs = [X]</literal>, and package Z has <literal>buildInputs = [Y]</literal>, then package X will appear in Z’s build environment automatically.
</para></listitem>
</varlistentry>
<varlistentry>
<term><varname>propagatedBuildInputs</varname></term>
<listitem><para>Like <varname>buildInputs</varname>, but these
dependencies are <emphasis>propagated</emphasis>: that is, the
dependencies listed here are added to the
<varname>buildInputs</varname> of any package that uses
<emphasis>this</emphasis> package as a dependency. So if package
Y has <literal>propagatedBuildInputs = [X]</literal>, and package
Z has <literal>buildInputs = [Y]</literal>, then package X will
appear in Z’s build environment automatically.</para></listitem>
<listitem><para>
Like <varname>buildInputs</varname>, but propagated just like <varname>propagatedNativeBuildInputs</varname>.
This inherits <varname>buildInputs</varname>'s flaws of clobbering native executables when cross-compiling and being confusing for static linking.
</para></listitem>
</varlistentry>
</variablelist>
@ -322,7 +341,7 @@ executed and in what order:
$preInstallPhases installPhase fixupPhase $preDistPhases
distPhase $postPhases</literal>.
</para>
<para>Usually, if you just want to add a few phases, it’s more
convenient to set one of the variables below (such as
<varname>preInstallPhases</varname>), as you then don’t specify
@ -706,7 +725,7 @@ makeFlagsArray=(CFLAGS="-O0 -g" LDFLAGS="-lfoo -lbar")
</variablelist>
<para>
<para>
You can set flags for <command>make</command> through the
<varname>makeFlags</varname> variable.</para>
@ -773,7 +792,7 @@ doCheck = true;</programlisting>
</variablelist>
</section>
@ -840,12 +859,12 @@ install phase. The default <function>fixupPhase</function> does the
following:
<itemizedlist>
<listitem><para>It moves the <filename>man/</filename>,
<filename>doc/</filename> and <filename>info/</filename>
subdirectories of <envar>$out</envar> to
<filename>share/</filename>.</para></listitem>
<listitem><para>It strips libraries and executables of debug
information.</para></listitem>
@ -1091,13 +1110,13 @@ functions.</para>
<variablelist>
<varlistentry xml:id='fun-substitute'>
<term><function>substitute</function>
<replaceable>infile</replaceable>
<replaceable>outfile</replaceable>
<replaceable>subs</replaceable></term>
<listitem>
<para>Performs string substitution on the contents of
<replaceable>infile</replaceable>, writing the result to
@ -1125,7 +1144,7 @@ functions.</para>
<literal>@<replaceable>...</replaceable>@</literal> in the
template as placeholders.</para></listitem>
</varlistentry>
<varlistentry>
<term><option>--subst-var-by</option>
<replaceable>varName</replaceable>
@ -1134,7 +1153,7 @@ functions.</para>
<literal>@<replaceable>varName</replaceable>@</literal> by
the string <replaceable>s</replaceable>.</para></listitem>
</varlistentry>
</variablelist>
</para>
@ -1162,7 +1181,7 @@ substitute ./foo.in ./foo.out \
</listitem>
</varlistentry>
<varlistentry xml:id='fun-substituteInPlace'>
<term><function>substituteInPlace</function>
@ -1173,7 +1192,7 @@ substitute ./foo.in ./foo.out \
<replaceable>file</replaceable>.</para></listitem>
</varlistentry>
<varlistentry xml:id='fun-substituteAll'>
<term><function>substituteAll</function>
<replaceable>infile</replaceable>
@ -1233,7 +1252,7 @@ echo @foo@
<listitem><para>Strips the directory and hash part of a store
path, outputting the name part to <literal>stdout</literal>.
For example:
<programlisting>
# prints coreutils-8.24
stripHash "/nix/store/9s9r019176g7cvn2nvcw41gsp862y6b4-coreutils-8.24"
@ -1241,7 +1260,7 @@ stripHash "/nix/store/9s9r019176g7cvn2nvcw41gsp862y6b4-coreutils-8.24"
If you wish to store the result in another variable, then the
following idiom may be useful:
<programlisting>
name="/nix/store/9s9r019176g7cvn2nvcw41gsp862y6b4-coreutils-8.24"
someVar=$(stripHash $name)
@ -1250,7 +1269,7 @@ someVar=$(stripHash $name)
</para></listitem>
</varlistentry>
</variablelist>
</section>
@ -1607,4 +1626,3 @@ Arch Wiki</link>.
</section>
</chapter>

9
nixos/doc/manual/release-notes/rl-1703.xml
View File

@ -30,6 +30,15 @@ has the following highlights: </para>
following incompatible changes:</para>
<itemizedlist>
<listitem>
<para>
Cross compilation has been rewritten. See the nixpkgs manual for
details. The most obvious breaking change is that derivations absent a
<literal>.nativeDrv</literal> or <literal>.crossDrv</literal> are now
cross by default, not native.
</para>
</listitem>
<listitem>
<para>
<literal>stdenv.overrides</literal> is now expected to take <literal>self</literal>

3
pkgs/build-support/cc-wrapper/default.nix
View File

@ -281,9 +281,6 @@ stdenv.mkDerivation {
crossAttrs = {
shell = shell.crossDrv + shell.crossDrv.shellPath;
libc = stdenv.ccCross.libc;
coreutils = coreutils.crossDrv;
binutils = binutils.crossDrv;
cc = cc.crossDrv;
#
# This is not the best way to do this. I think the reference should be
# the style in the gcc-cross-wrapper, but to keep a stable stdenv now I

7
pkgs/development/compilers/ghc/head.nix
View File

@ -1,5 +1,6 @@
{ stdenv, fetchgit, bootPkgs, perl, gmp, ncurses, libiconv, binutils, coreutils
, autoconf, automake, happy, alex, python3, crossSystem, selfPkgs, cross ? null
, autoconf, automake, happy, alex, python3, buildPlatform, targetPlatform
, selfPkgs, cross ? null
}:
let
@ -68,9 +69,9 @@ in stdenv.mkDerivation (rec {
passthru = {
inherit bootPkgs;
} // stdenv.lib.optionalAttrs (crossSystem != null) {
} // stdenv.lib.optionalAttrs (targetPlatform != buildPlatform) {
crossCompiler = selfPkgs.ghc.override {
cross = crossSystem;
cross = targetPlatform;
bootPkgs = selfPkgs;
};
};

4
pkgs/development/libraries/openssl/default.nix
View File

@ -1,4 +1,4 @@
{ stdenv, fetchurl, perl
{ stdenv, fetchurl, buildPackages, perl
, withCryptodev ? false, cryptodevHeaders
, enableSSL2 ? false }:
@ -76,7 +76,7 @@ let
postFixup = ''
# Check to make sure the main output doesn't depend on perl
if grep -r '${perl}' $out; then
if grep -r '${buildPackages.perl}' $out; then
echo "Found an erroneous dependency on perl ^^^" >&2
exit 1
fi

21
pkgs/os-specific/gnu/default.nix
View File

@ -3,7 +3,8 @@
args@{ fetchgit, stdenv, autoconf, automake, automake111x, libtool
, texinfo, glibcCross, hurdPartedCross, libuuid, samba
, gccCrossStageStatic, gccCrossStageFinal
, forceNativeDrv, forceSystem, newScope, platform, config, crossSystem
, forcedNativePackages, forceSystem, newScope, platform, config
, targetPlatform, buildPlatform
, overrides ? {} }:
with args;
@ -12,25 +13,25 @@ let
callPackage = newScope gnu;
gnu = {
hurdCross = forceNativeDrv (callPackage ./hurd {
hurdCross = forcedNativePackages.callPackage ./hurd {
inherit fetchgit stdenv autoconf libtool texinfo
glibcCross hurdPartedCross;
inherit (gnu) machHeaders mig;
libuuid = libuuid.crossDrv;
automake = automake111x;
headersOnly = false;
cross = assert crossSystem != null; crossSystem;
cross = assert targetPlatform != buildPlatform; targetPlatform;
gccCross = gccCrossStageFinal;
});
};
hurdCrossIntermediate = forceNativeDrv (callPackage ./hurd {
hurdCrossIntermediate = forcedNativePackages.callPackage ./hurd {
inherit fetchgit stdenv autoconf libtool texinfo glibcCross;
inherit (gnu) machHeaders mig;
hurdPartedCross = null;
libuuid = null;
automake = automake111x;
headersOnly = false;
cross = assert crossSystem != null; crossSystem;
cross = assert targetPlatform != buildPlatform; targetPlatform;
# The "final" GCC needs glibc and the Hurd libraries (libpthread in
# particular) so we first need an intermediate Hurd built with the
@ -42,7 +43,7 @@ let
# libshouldbeinlibc.
buildTarget = "libihash libstore libshouldbeinlibc";
installTarget = "libihash-install libstore-install libshouldbeinlibc-install";
});
};
hurdHeaders = callPackage ./hurd {
automake = automake111x;
@ -58,13 +59,13 @@ let
hurd = null;
};
libpthreadCross = forceNativeDrv (callPackage ./libpthread {
libpthreadCross = forcedNativePackages.callPackage ./libpthread {
inherit fetchgit stdenv autoconf automake libtool glibcCross;
inherit (gnu) machHeaders hurdHeaders;
hurd = gnu.hurdCrossIntermediate;
gccCross = gccCrossStageStatic;
cross = assert crossSystem != null; crossSystem;
});
cross = assert targetPlatform != buildPlatform; targetPlatform;
};
# In theory GNU Mach doesn't have to be cross-compiled. However, since it
# has to be built for i586 (it doesn't work on x86_64), one needs a cross

102
pkgs/stdenv/adapters.nix
View File

@ -56,69 +56,59 @@ rec {
# Return a modified stdenv that adds a cross compiler to the
# builds.
makeStdenvCross = stdenv: cross: binutilsCross: gccCross: stdenv //
{ mkDerivation = {name ? "", buildInputs ? [], nativeBuildInputs ? [],
propagatedBuildInputs ? [], propagatedNativeBuildInputs ? [],
selfNativeBuildInput ? false, ...}@args: let
makeStdenvCross = stdenv: cross: binutilsCross: gccCross: stdenv // {
# *BuildInputs exists temporarily as another name for
# *HostInputs.
mkDerivation =
{ name ? "", buildInputs ? [], nativeBuildInputs ? []
, propagatedBuildInputs ? [], propagatedNativeBuildInputs ? []
, selfNativeBuildInput ? false, ...
} @ args:
# In nixpkgs, sometimes 'null' gets in as a buildInputs element,
# and we handle that through isAttrs.
getNativeDrv = drv: drv.nativeDrv or drv;
getCrossDrv = drv: drv.crossDrv or drv;
nativeBuildInputsDrvs = map getNativeDrv nativeBuildInputs;
buildInputsDrvs = map getCrossDrv buildInputs;
propagatedBuildInputsDrvs = map getCrossDrv propagatedBuildInputs;
propagatedNativeBuildInputsDrvs = map getNativeDrv propagatedNativeBuildInputs;
let
# *BuildInputs exists temporarily as another name for
# *HostInputs.
# The base stdenv already knows that nativeBuildInputs and
# buildInputs should be built with the usual gcc-wrapper
# And the same for propagatedBuildInputs.
nativeDrv = stdenv.mkDerivation args;
# The base stdenv already knows that nativeBuildInputs and
# buildInputs should be built with the usual gcc-wrapper
# And the same for propagatedBuildInputs.
nativeDrv = stdenv.mkDerivation args;
# Temporary expression until the cross_renaming, to handle the
# case of pkgconfig given as buildInput, but to be used as
# nativeBuildInput.
hostAsNativeDrv = drv:
builtins.unsafeDiscardStringContext drv.nativeDrv.drvPath
== builtins.unsafeDiscardStringContext drv.crossDrv.drvPath;
buildInputsNotNull = stdenv.lib.filter
(drv: builtins.isAttrs drv && drv ? nativeDrv) buildInputs;
nativeInputsFromBuildInputs = stdenv.lib.filter hostAsNativeDrv buildInputsNotNull;
# Temporary expression until the cross_renaming, to handle the
# case of pkgconfig given as buildInput, but to be used as
# nativeBuildInput.
hostAsNativeDrv = drv:
builtins.unsafeDiscardStringContext drv.nativeDrv.drvPath
== builtins.unsafeDiscardStringContext drv.crossDrv.drvPath;
buildInputsNotNull = stdenv.lib.filter
(drv: builtins.isAttrs drv && drv ? nativeDrv) buildInputs;
nativeInputsFromBuildInputs = stdenv.lib.filter hostAsNativeDrv buildInputsNotNull;
in
stdenv.mkDerivation (args // {
name = name + "-" + cross.config;
nativeBuildInputs = nativeBuildInputs
++ nativeInputsFromBuildInputs
++ [ gccCross binutilsCross ]
++ stdenv.lib.optional selfNativeBuildInput nativeDrv
# without proper `file` command, libtool sometimes fails
# to recognize 64-bit DLLs
++ stdenv.lib.optional (cross.config == "x86_64-w64-mingw32") pkgs.file
;
# We should overwrite the input attributes in crossDrv, to overwrite
# the defaults for only-native builds in the base stdenv
crossDrv = if cross == null then nativeDrv else
stdenv.mkDerivation (args // {
name = name + "-" + cross.config;
nativeBuildInputs = nativeBuildInputsDrvs
++ nativeInputsFromBuildInputs
++ [ gccCross binutilsCross ]
++ stdenv.lib.optional selfNativeBuildInput nativeDrv
# without proper `file` command, libtool sometimes fails
# to recognize 64-bit DLLs
++ stdenv.lib.optional (cross.config == "x86_64-w64-mingw32") pkgs.file
;
# Cross-linking dynamic libraries, every buildInput should
# be propagated because ld needs the -rpath-link to find
# any library needed to link the program dynamically at
# loader time. ld(1) explains it.
buildInputs = [];
propagatedBuildInputs = propagatedBuildInputs ++ buildInputs;
propagatedNativeBuildInputs = propagatedNativeBuildInputs;
# Cross-linking dynamic libraries, every buildInput should
# be propagated because ld needs the -rpath-link to find
# any library needed to link the program dynamically at
# loader time. ld(1) explains it.
buildInputs = [];
propagatedBuildInputs = propagatedBuildInputsDrvs ++ buildInputsDrvs;
propagatedNativeBuildInputs = propagatedNativeBuildInputsDrvs;
crossConfig = cross.config;
} // args.crossAttrs or {});
crossConfig = cross.config;
} // args.crossAttrs or {});
in nativeDrv // {
inherit crossDrv nativeDrv;
};
} // {
inherit cross gccCross binutilsCross;
ccCross = gccCross;
};
inherit gccCross binutilsCross;
ccCross = gccCross;
};
/* Modify a stdenv so that the specified attributes are added to

15
pkgs/stdenv/booter.nix
View File

@ -57,12 +57,17 @@ stageFuns: let
# debugging purposes.
folder = stageFun: finalSoFar: let
args = stageFun finalSoFar;
stdenv = args.stdenv // {
# For debugging
__bootPackages = finalSoFar;
args' = args // {
stdenv = args.stdenv // {
# For debugging
__bootPackages = finalSoFar;
};
};
args' = args // { inherit stdenv; };
in
(if args.__raw or false then lib.id else allPackages) args';
if args.__raw or false
then args'
else allPackages ((builtins.removeAttrs args' ["selfBuild"]) // {
buildPackages = if args.selfBuild or true then null else finalSoFar;
});
in lib.lists.fold folder {} withAllowCustomOverrides

27
pkgs/stdenv/cross/default.nix
View File

@ -1,10 +1,10 @@
{ lib
, system, platform, crossSystem, config, overlays
, localSystem, crossSystem, config, overlays
}:
let
bootStages = import ../. {
inherit lib system platform overlays;
inherit lib localSystem overlays;
crossSystem = null;
# Ignore custom stdenvs when cross compiling for compatability
config = builtins.removeAttrs config [ "replaceStdenv" ];
@ -12,25 +12,28 @@ let
in bootStages ++ [
# Build Packages.
#
# For now, this is just used to build the native stdenv. Eventually, it
# should be used to build compilers and other such tools targeting the cross
# platform. Then, `forceNativeDrv` can be removed.
# Build Packages
(vanillaPackages: {
inherit system platform crossSystem config overlays;
buildPlatform = localSystem;
hostPlatform = localSystem;
targetPlatform = crossSystem;
inherit config overlays;
# Should be false, but we're trying to preserve hashes for now
selfBuild = true;
# It's OK to change the built-time dependencies
allowCustomOverrides = true;
stdenv = vanillaPackages.stdenv // {
# Needed elsewhere as a hacky way to pass the target
cross = crossSystem;
overrides = _: _: {};
};
})
# Run packages
# Run Packages
(buildPackages: {
inherit system platform crossSystem config overlays;
buildPlatform = localSystem;
hostPlatform = crossSystem;
targetPlatform = crossSystem;
inherit config overlays;
selfBuild = false;
stdenv = if crossSystem.useiOSCross or false
then let
inherit (buildPackages.darwin.ios-cross {

9
pkgs/stdenv/custom/default.nix
View File

@ -1,12 +1,12 @@
{ lib
, system, platform, crossSystem, config, overlays
, localSystem, crossSystem, config, overlays
}:
assert crossSystem == null;
let
bootStages = import ../. {
inherit lib system platform crossSystem overlays;
inherit lib localSystem crossSystem overlays;
# Remove config.replaceStdenv to ensure termination.
config = builtins.removeAttrs config [ "replaceStdenv" ];
};
@ -15,7 +15,10 @@ in bootStages ++ [
# Additional stage, built using custom stdenv
(vanillaPackages: {
inherit system platform crossSystem config overlays;
buildPlatform = localSystem;
hostPlatform = localSystem;
targetPlatform = localSystem;
inherit config overlays;
stdenv = config.replaceStdenv { pkgs = vanillaPackages; };
})

17
pkgs/stdenv/darwin/default.nix
View File

@ -1,11 +1,12 @@
{ lib
, system, platform, crossSystem, config, overlays
, localSystem, crossSystem, config, overlays
# Allow passing in bootstrap files directly so we can test the stdenv bootstrap process when changing the bootstrap tools
, bootstrapFiles ? let
fetch = { file, sha256, executable ? true }: import <nix/fetchurl.nix> {
url = "http://tarballs.nixos.org/stdenv-darwin/x86_64/33f59c9d11b8d5014dfd18cc11a425f6393c884a/${file}";
inherit sha256 system executable;
inherit (localSystem) system;
inherit sha256 executable;
}; in {
sh = fetch { file = "sh"; sha256 = "1rx4kg6358xdj05z0m139a0zn4f4zfmq4n4vimlmnwyfiyn4x7wk"; };
bzip2 = fetch { file = "bzip2"; sha256 = "104qnhzk79vkbp2yi0kci6lszgfppvrwk3rgxhry842ly1xz2r7l"; };
@ -18,6 +19,8 @@
assert crossSystem == null;
let
inherit (localSystem) system platform;
libSystemProfile = ''
(import "${./standard-sandbox.sb}")
'';
@ -98,7 +101,10 @@ in rec {
};
in {
inherit system platform crossSystem config overlays;
buildPlatform = localSystem;
hostPlatform = localSystem;
targetPlatform = localSystem;
inherit config overlays;
stdenv = thisStdenv;
};
@ -316,7 +322,10 @@ in rec {
stage3
stage4
(prevStage: {
inherit system crossSystem platform config overlays;
buildPlatform = localSystem;
hostPlatform = localSystem;
targetPlatform = localSystem;
inherit config overlays;
stdenv = stdenvDarwin prevStage;
})
];

4
pkgs/stdenv/default.nix
View File

@ -7,7 +7,7 @@
{ # Args just for stdenvs' usage
lib
# Args to pass on to the pkgset builder, too
, system, platform, crossSystem, config, overlays
, localSystem, crossSystem, config, overlays
} @ args:
let
@ -51,4 +51,4 @@ in
"i686-cygwin" = stagesNative;
"x86_64-cygwin" = stagesNative;
"x86_64-freebsd" = stagesFreeBSD;
}.${system} or stagesNative
}.${localSystem.system} or stagesNative

8
pkgs/stdenv/freebsd/default.nix
View File

@ -1,8 +1,9 @@
{ lib
, system, platform, crossSystem, config, overlays
, localSystem, crossSystem, config, overlays
}:
assert crossSystem == null;
let inherit (localSystem) system; in
[
@ -58,7 +59,10 @@ assert crossSystem == null;
})
(prevStage: {
inherit system crossSystem platform config overlays;
buildPlatform = localSystem;
hostPlatform = localSystem;
targetPlatform = localSystem;
inherit config overlays;
stdenv = import ../generic {
name = "stdenv-freebsd-boot-3";
inherit system config;

14
pkgs/stdenv/generic/default.nix
View File

@ -115,7 +115,19 @@ let
, sandboxProfile ? ""
, propagatedSandboxProfile ? ""
, ... } @ attrs:
let
let # Rename argumemnts to avoid cycles
buildInputs__ = buildInputs;
nativeBuildInputs__ = nativeBuildInputs;
propagatedBuildInputs__ = propagatedBuildInputs;
propagatedNativeBuildInputs__ = propagatedNativeBuildInputs;
in let
getNativeDrv = drv: drv.nativeDrv or drv;
getCrossDrv = drv: drv.crossDrv or drv;
nativeBuildInputs = map getNativeDrv nativeBuildInputs__;
buildInputs = map getCrossDrv buildInputs__;
propagatedBuildInputs = map getCrossDrv propagatedBuildInputs__;
propagatedNativeBuildInputs = map getNativeDrv propagatedNativeBuildInputs__;
in let
pos' =
if pos != null then
pos

30
pkgs/stdenv/linux/default.nix
View File

@ -4,21 +4,23 @@
# compiler and linker that do not search in default locations,
# ensuring purity of components produced by it.
{ lib
, system, platform, crossSystem, config, overlays
, localSystem, crossSystem, config, overlays
, bootstrapFiles ?
if system == "i686-linux" then import ./bootstrap-files/i686.nix
else if system == "x86_64-linux" then import ./bootstrap-files/x86_64.nix
else if system == "armv5tel-linux" then import ./bootstrap-files/armv5tel.nix
else if system == "armv6l-linux" then import ./bootstrap-files/armv6l.nix
else if system == "armv7l-linux" then import ./bootstrap-files/armv7l.nix
else if system == "mips64el-linux" then import ./bootstrap-files/loongson2f.nix
else abort "unsupported platform for the pure Linux stdenv"
, bootstrapFiles ? { # switch
"i686-linux" = import ./bootstrap-files/i686.nix;
"x86_64-linux" = import ./bootstrap-files/x86_64.nix;
"armv5tel-linux" = import ./bootstrap-files/armv5tel.nix;
"armv6l-linux" = import ./bootstrap-files/armv6l.nix;
"armv7l-linux" = import ./bootstrap-files/armv7l.nix;
"mips64el-linux" = import ./bootstrap-files/loongson2f.nix;
}.${localSystem.system}
or (abort "unsupported platform for the pure Linux stdenv")
}:
assert crossSystem == null;
let
inherit (localSystem) system platform;
commonPreHook =
''
@ -91,7 +93,10 @@ let
};
in {
inherit system platform crossSystem config overlays;
buildPlatform = localSystem;
hostPlatform = localSystem;
targetPlatform = localSystem;
inherit config overlays;
stdenv = thisStdenv;
};
@ -246,7 +251,10 @@ in
# dependency (`nix-store -qR') on bootstrapTools or the first
# binutils built.
(prevStage: {
inherit system crossSystem platform config overlays;
buildPlatform = localSystem;
hostPlatform = localSystem;
targetPlatform = localSystem;
inherit config overlays;
stdenv = import ../generic rec {
inherit system config;

21
pkgs/stdenv/linux/make-bootstrap-tools-cross.nix
View File

@ -55,11 +55,12 @@ let
if toolsArch == "armv6l" then raspberrypiCrossSystem else
if toolsArch == "armv7l" then armv7l-hf-multiplatform-crossSystem else null;
pkgs = pkgsFun ({inherit system;} // selectedCrossSystem);
pkgsUnspliced = pkgsFun ({inherit system;} // selectedCrossSystem);
pkgs = pkgsUnspliced.splicedPackages;
inherit (pkgs) stdenv nukeReferences cpio binutilsCross;
inherit (pkgsUnspliced.buildPackages) stdenv nukeReferences cpio binutilsCross;
glibc = pkgs.libcCross;
glibc = pkgs.libcCross.nativeDrv;
bash = pkgs.bash.crossDrv;
findutils = pkgs.findutils.crossDrv;
diffutils = pkgs.diffutils.crossDrv;
@ -71,7 +72,7 @@ let
gnumake = pkgs.gnumake.crossDrv;
patch = pkgs.patch.crossDrv;
patchelf = pkgs.patchelf.crossDrv;
gcc = pkgs.gcc.cc.crossDrv;
gcc = pkgs.gcc.crossDrv.cc;
gmpxx = pkgs.gmpxx.crossDrv;
mpfr = pkgs.mpfr.crossDrv;
zlib = pkgs.zlib.crossDrv;
@ -86,17 +87,17 @@ in
rec {
coreutilsMinimal = (pkgs.coreutils.override (args: {
coreutilsMinimal = pkgs.coreutils.override (args: {
# We want coreutils without ACL/attr support.
aclSupport = false;
attrSupport = false;
# Our tooling currently can't handle scripts in bin/, only ELFs and symlinks.
singleBinary = "symlinks";
})).crossDrv;
});
tarMinimal = (pkgs.gnutar.override { acl = null; }).crossDrv;
tarMinimal = pkgs.gnutar.override { acl = null; };
busyboxMinimal = (pkgs.busybox.override {
busyboxMinimal = pkgs.busybox.override {
useMusl = true;
enableStatic = true;
enableMinimal = true;
@ -109,13 +110,13 @@ rec {
CONFIG_TAR y
CONFIG_UNXZ y
'';
}).crossDrv;
};
build =
stdenv.mkDerivation {
name = "stdenv-bootstrap-tools-cross";
crossConfig = stdenv.cross.config;
crossConfig = pkgsUnspliced.hostPlatform.config;
buildInputs = [nukeReferences cpio binutilsCross];

13
pkgs/stdenv/native/default.nix
View File

@ -1,10 +1,11 @@
{ lib
, system, platform, crossSystem, config, overlays
, localSystem, crossSystem, config, overlays
}:
assert crossSystem == null;
let
inherit (localSystem) system platform;
shell =
if system == "i686-freebsd" || system == "x86_64-freebsd" then "/usr/local/bin/bash"
@ -134,7 +135,10 @@ in
# First build a stdenv based only on tools outside the store.
(prevStage: {
inherit system crossSystem platform config overlays;
buildPlatform = localSystem;
hostPlatform = localSystem;
targetPlatform = localSystem;
inherit config overlays;
stdenv = makeStdenv {
inherit (prevStage) cc fetchurl;
} // { inherit (prevStage) fetchurl; };
@ -143,7 +147,10 @@ in
# Using that, build a stdenv that adds the ‘xz’ command (which most systems
# don't have, so we mustn't rely on the native environment providing it).
(prevStage: {
inherit system crossSystem platform config overlays;
buildPlatform = localSystem;
hostPlatform = localSystem;
targetPlatform = localSystem;
inherit config overlays;
stdenv = makeStdenv {
inherit (prevStage.stdenv) cc fetchurl;
extraPath = [ prevStage.xz ];

4
pkgs/stdenv/nix/default.nix
View File

@ -9,9 +9,9 @@ assert crossSystem == null;
bootStages ++ [
(prevStage: let
inherit (prevStage) stdenv;
inherit (stdenv) system platform;
in {
inherit system platform crossSystem config;
inherit (prevStage) buildPlatform hostPlatform targetPlatform;
inherit config overlays;
stdenv = import ../generic rec {
inherit config;

185
pkgs/tools/misc/coreutils/default.nix
View File

@ -1,4 +1,4 @@
{ lib, stdenv, fetchurl, perl, xz, gmp ? null
{ lib, stdenv, buildPackages, fetchurl, perl, xz, gmp ? null
, aclSupport ? false, acl ? null
, attrSupport ? false, attr ? null
, selinuxSupport? false, libselinux ? null, libsepol ? null
@ -12,104 +12,101 @@ assert selinuxSupport -> libselinux != null && libsepol != null;
with lib;
let
self = stdenv.mkDerivation rec {
name = "coreutils-8.26";
stdenv.mkDerivation rec {
name = "coreutils-8.26";
src = fetchurl {
url = "mirror://gnu/coreutils/${name}.tar.xz";
sha256 = "13lspazc7xkviy93qz7ks9jv4sldvgmwpq36ghrbrqpq93br8phm";
};
src = fetchurl {
url = "mirror://gnu/coreutils/${name}.tar.xz";
sha256 = "13lspazc7xkviy93qz7ks9jv4sldvgmwpq36ghrbrqpq93br8phm";
};
# FIXME needs gcc 4.9 in bootstrap tools
hardeningDisable = [ "stackprotector" ];
# FIXME needs gcc 4.9 in bootstrap tools
hardeningDisable = [ "stackprotector" ];
patches = optional stdenv.isCygwin ./coreutils-8.23-4.cygwin.patch;
patches = optional stdenv.isCygwin ./coreutils-8.23-4.cygwin.patch;
# The test tends to fail on btrfs and maybe other unusual filesystems.
postPatch = optionalString (!stdenv.isDarwin) ''
sed '2i echo Skipping dd sparse test && exit 0' -i ./tests/dd/sparse.sh
sed '2i echo Skipping cp sparse test && exit 0' -i ./tests/cp/sparse.sh
sed '2i echo Skipping rm deep-2 test && exit 0' -i ./tests/rm/deep-2.sh
sed '2i echo Skipping du long-from-unreadable test && exit 0' -i ./tests/du/long-from-unreadable.sh
# The test tends to fail on btrfs and maybe other unusual filesystems.
postPatch = optionalString (!stdenv.isDarwin) ''
sed '2i echo Skipping dd sparse test && exit 0' -i ./tests/dd/sparse.sh
sed '2i echo Skipping cp sparse test && exit 0' -i ./tests/cp/sparse.sh
sed '2i echo Skipping rm deep-2 test && exit 0' -i ./tests/rm/deep-2.sh
sed '2i echo Skipping du long-from-unreadable test && exit 0' -i ./tests/du/long-from-unreadable.sh
'';
outputs = [ "out" "info" ];
nativeBuildInputs = [ perl xz.bin ];
configureFlags =
optional (singleBinary != false)
("--enable-single-binary" + optionalString (isString singleBinary) "=${singleBinary}")
++ optional stdenv.isSunOS "ac_cv_func_inotify_init=no"
++ optional withPrefix "--program-prefix=g";
buildInputs = [ gmp ]
++ optional aclSupport acl
++ optional attrSupport attr
++ optionals stdenv.isCygwin [ autoconf automake114x texinfo ] # due to patch
++ optionals selinuxSupport [ libselinux libsepol ];
crossAttrs = {
buildInputs = [ gmp.crossDrv ]
++ optional aclSupport acl.crossDrv
++ optional attrSupport attr.crossDrv
++ optionals selinuxSupport [ libselinux.crossDrv libsepol.crossDrv ]
++ optional (stdenv.ccCross.libc ? libiconv)
stdenv.ccCross.libc.libiconv.crossDrv;
# Prevents attempts of running 'help2man' on cross-built binaries.
PERL = "missing";
# Works around a bug with 8.26:
# Makefile:3440: *** Recursive variable 'INSTALL' references itself (eventually). Stop.
preInstall = ''
sed -i Makefile -e 's|^INSTALL =.*|INSTALL = ${buildPackages.coreutils}/bin/install -c|'
'';
outputs = [ "out" "info" ];
postInstall = ''
rm $out/share/man/man1/*
cp ${buildPackages.coreutils}/share/man/man1/* $out/share/man/man1
'';
nativeBuildInputs = [ perl xz.bin ];
configureFlags =
optional (singleBinary != false)
("--enable-single-binary" + optionalString (isString singleBinary) "=${singleBinary}")
++ optional stdenv.isSunOS "ac_cv_func_inotify_init=no"
++ optional withPrefix "--program-prefix=g";
buildInputs = [ gmp ]
++ optional aclSupport acl
++ optional attrSupport attr
++ optionals stdenv.isCygwin [ autoconf automake114x texinfo ] # due to patch
++ optionals selinuxSupport [ libselinux libsepol ];
crossAttrs = {
buildInputs = [ gmp.crossDrv ]
++ optional aclSupport acl.crossDrv
++ optional attrSupport attr.crossDrv
++ optionals selinuxSupport [ libselinux.crossDrv libsepol.crossDrv ]
++ optional (stdenv.ccCross.libc ? libiconv)
stdenv.ccCross.libc.libiconv.crossDrv;
# Prevents attempts of running 'help2man' on cross-built binaries.
PERL = "missing";
# Works around a bug with 8.26:
# Makefile:3440: *** Recursive variable 'INSTALL' references itself (eventually). Stop.
preInstall = ''
sed -i Makefile -e 's|^INSTALL =.*|INSTALL = ${self}/bin/install -c|'
'';
postInstall = ''
rm $out/share/man/man1/*
cp ${self}/share/man/man1/* $out/share/man/man1
'';
# Needed for fstatfs()
# I don't know why it is not properly detected cross building with glibc.
configureFlags = [ "fu_cv_sys_stat_statfs2_bsize=yes" ];
doCheck = false;
};
# The tests are known broken on Cygwin
# (http://thread.gmane.org/gmane.comp.gnu.core-utils.bugs/19025),
# Darwin (http://thread.gmane.org/gmane.comp.gnu.core-utils.bugs/19351),
# and {Open,Free}BSD.
# With non-standard storeDir: https://github.com/NixOS/nix/issues/512
doCheck = stdenv ? glibc && builtins.storeDir == "/nix/store";
# Saw random failures like ‘help2man: can't get '--help' info from
# man/sha512sum.td/sha512sum’.
enableParallelBuilding = false;
NIX_LDFLAGS = optionalString selinuxSupport "-lsepol";
FORCE_UNSAFE_CONFIGURE = optionalString stdenv.isSunOS "1";
makeFlags = optionalString stdenv.isDarwin "CFLAGS=-D_FORTIFY_SOURCE=0";
meta = {
homepage = http://www.gnu.org/software/coreutils/;
description = "The basic file, shell and text manipulation utilities of the GNU operating system";
longDescription = ''
The GNU Core Utilities are the basic file, shell and text
manipulation utilities of the GNU operating system. These are
the core utilities which are expected to exist on every
operating system.
'';
license = licenses.gpl3Plus;
platforms = platforms.all;
maintainers = [ maintainers.eelco ];
};
# Needed for fstatfs()
# I don't know why it is not properly detected cross building with glibc.
configureFlags = [ "fu_cv_sys_stat_statfs2_bsize=yes" ];
doCheck = false;
};
in
self
# The tests are known broken on Cygwin
# (http://thread.gmane.org/gmane.comp.gnu.core-utils.bugs/19025),
# Darwin (http://thread.gmane.org/gmane.comp.gnu.core-utils.bugs/19351),
# and {Open,Free}BSD.
# With non-standard storeDir: https://github.com/NixOS/nix/issues/512
doCheck = stdenv ? glibc && builtins.storeDir == "/nix/store";
# Saw random failures like ‘help2man: can't get '--help' info from
# man/sha512sum.td/sha512sum’.
enableParallelBuilding = false;
NIX_LDFLAGS = optionalString selinuxSupport "-lsepol";
FORCE_UNSAFE_CONFIGURE = optionalString stdenv.isSunOS "1";
makeFlags = optionalString stdenv.isDarwin "CFLAGS=-D_FORTIFY_SOURCE=0";
meta = {
homepage = http://www.gnu.org/software/coreutils/;
description = "The basic file, shell and text manipulation utilities of the GNU operating system";
longDescription = ''
The GNU Core Utilities are the basic file, shell and text
manipulation utilities of the GNU operating system. These are
the core utilities which are expected to exist on every
operating system.
'';
license = licenses.gpl3Plus;
platforms = platforms.all;
maintainers = [ maintainers.eelco ];
};
}

126
pkgs/top-level/all-packages.nix
View File

@ -10,26 +10,12 @@ self: pkgs:
with pkgs;
let
defaultScope = pkgs // pkgs.xorg;
in
{
# Allow callPackage to fill in the pkgs argument
inherit pkgs;
# We use `callPackage' to be able to omit function arguments that
# can be obtained from `pkgs' or `pkgs.xorg' (i.e. `defaultScope').
# Use `newScope' for sets of packages in `pkgs' (see e.g. `gnome'
# below).
callPackage = newScope {};
callPackages = lib.callPackagesWith defaultScope;
newScope = extra: lib.callPackageWith (defaultScope // extra);
# Override system. This is useful to build i686 packages on x86_64-linux.
forceSystem = system: kernel: nixpkgsFun {
inherit system;
@ -39,15 +25,9 @@ in
# Used by wine, firefox with debugging version of Flash, ...
pkgsi686Linux = forceSystem "i686-linux" "i386";
callPackage_i686 = lib.callPackageWith (pkgsi686Linux // pkgsi686Linux.xorg);
callPackage_i686 = pkgsi686Linux.callPackage;
forceNativeDrv = drv:
# Even when cross compiling, some packages come from the stdenv's
# bootstrapping package set. Those packages are only built for the native
# platform.
if crossSystem != null && drv ? crossDrv
then drv // { crossDrv = drv.nativeDrv; }
else drv;
forcedNativePackages = if hostPlatform == buildPlatform then pkgs else buildPackages;
# A stdenv capable of building 32-bit binaries. On x86_64-linux,
# it uses GCC compiled with multilib support; on i686-linux, it's
@ -3274,7 +3254,7 @@ in
pngout = callPackage ../tools/graphics/pngout { };
hurdPartedCross =
if crossSystem != null && crossSystem.config == "i586-pc-gnu"
if targetPlatform != buildPlatform && targetPlatform.config == "i586-pc-gnu"
then (makeOverridable
({ hurd }:
(parted.override {
@ -4779,44 +4759,48 @@ in
gccApple = throw "gccApple is no longer supported";
gccCrossStageStatic = let
gccCrossStageStatic = assert targetPlatform != buildPlatform; let
libcCross1 =
if stdenv.cross.libc == "msvcrt" then windows.mingw_w64_headers
else if stdenv.cross.libc == "libSystem" then darwin.xcode
else null;
in wrapGCCCross {
gcc = forceNativeDrv (gcc.cc.override {
cross = crossSystem;
gcc = forcedNativePackages.gcc.cc.override {
cross = targetPlatform;
crossStageStatic = true;
langCC = false;
libcCross = libcCross1;
enableShared = false;
});
# Why is this needed?
inherit (forcedNativePackages) binutilsCross;
};
libc = libcCross1;
binutils = binutilsCross;
cross = crossSystem;
cross = targetPlatform;
};
# Only needed for mingw builds
gccCrossMingw2 = wrapGCCCross {
gccCrossMingw2 = assert targetPlatform != buildPlatform; wrapGCCCross {
gcc = gccCrossStageStatic.gcc;
libc = windows.mingw_headers2;
binutils = binutilsCross;
cross = assert crossSystem != null; crossSystem;
cross = targetPlatform;
};
gccCrossStageFinal = wrapGCCCross {
gcc = forceNativeDrv (gcc.cc.override {
cross = crossSystem;
gccCrossStageFinal = assert targetPlatform != buildPlatform; wrapGCCCross {
gcc = forcedNativePackages.gcc.cc.override {
cross = targetPlatform;
crossStageStatic = false;
# XXX: We have troubles cross-compiling libstdc++ on MinGW (see
# <http://hydra.nixos.org/build/4268232>), so don't even try.
langCC = crossSystem.config != "i686-pc-mingw32";
});
langCC = targetPlatform.config != "i686-pc-mingw32";
# Why is this needed?
inherit (forcedNativePackages) binutilsCross;
};
libc = libcCross;
binutils = binutilsCross;
cross = crossSystem;
cross = targetPlatform;
};
gcc45 = lowPrio (wrapCC (callPackage ../development/compilers/gcc/4.5 {
@ -4834,7 +4818,7 @@ in
# and host != build), `cross' must be null but the cross-libc must still
# be passed.
cross = null;
libcCross = if crossSystem != null then libcCross else null;
libcCross = if targetPlatform != buildPlatform then libcCross else null;
}));
gcc48 = lowPrio (wrapCC (callPackage ../development/compilers/gcc/4.8 {
@ -4847,7 +4831,7 @@ in
# and host != build), `cross' must be null but the cross-libc must still
# be passed.
cross = null;
libcCross = if crossSystem != null then libcCross else null;
libcCross = if targetPlatform != buildPlatform then libcCross else null;
isl = if !stdenv.isDarwin then isl_0_14 else null;
cloog = if !stdenv.isDarwin then cloog else null;
@ -4864,7 +4848,7 @@ in
# and host != build), `cross' must be null but the cross-libc must still
# be passed.
cross = null;
libcCross = if crossSystem != null then libcCross else null;
libcCross = if targetPlatform != buildPlatform then libcCross else null;
isl = if !stdenv.isDarwin then isl_0_11 else null;
@ -4881,7 +4865,7 @@ in
# and host != build), `cross' must be null but the cross-libc must still
# be passed.
cross = null;
libcCross = if crossSystem != null then libcCross else null;
libcCross = if targetPlatform != buildPlatform then libcCross else null;
isl = if !stdenv.isDarwin then isl_0_14 else null;
}));
@ -4896,7 +4880,7 @@ in
# and host != build), `cross' must be null but the cross-libc must still
# be passed.
cross = null;
libcCross = if crossSystem != null then libcCross else null;
libcCross = if targetPlatform != buildPlatform then libcCross else null;
isl = if !stdenv.isDarwin then isl_0_14 else null;
}));
@ -5029,7 +5013,7 @@ in
# Haskell and GHC
haskell = callPackage ./haskell-packages.nix { inherit crossSystem; };
haskell = callPackage ./haskell-packages.nix { };
haskellPackages = haskell.packages.ghc801.override {
overrides = config.haskellPackageOverrides or (self: super: {});
@ -5507,12 +5491,12 @@ in
wrapGCCCross =
{gcc, libc, binutils, cross, shell ? "", name ? "gcc-cross-wrapper"}:
forceNativeDrv (callPackage ../build-support/gcc-cross-wrapper {
forcedNativePackages.callPackage ../build-support/gcc-cross-wrapper {
nativeTools = false;
nativeLibc = false;
noLibc = (libc == null);
inherit gcc binutils libc shell name cross;
});
};
# prolog
yap = callPackage ../development/compilers/yap { };
@ -6092,12 +6076,12 @@ in
gold = false;
});
binutilsCross = assert crossSystem != null; lowPrio (forceNativeDrv (
if crossSystem.libc == "libSystem" then darwin.cctools_cross
else binutils.override {
binutilsCross = assert targetPlatform != buildPlatform; lowPrio (
if targetPlatform.libc == "libSystem" then darwin.cctools_cross
else forcedNativePackages.binutils.override {
noSysDirs = true;
cross = crossSystem;
}));
cross = targetPlatform;
});
bison2 = callPackage ../development/tools/parsing/bison/2.x.nix { };
bison3 = callPackage ../development/tools/parsing/bison/3.x.nix { };
@ -6566,9 +6550,9 @@ in
cross_renaming: we should make all programs use pkgconfig as
nativeBuildInput after the renaming.
*/
pkgconfig = forceNativeDrv (callPackage ../development/tools/misc/pkgconfig {
pkgconfig = forcedNativePackages.callPackage ../development/tools/misc/pkgconfig {
fetchurl = fetchurlBoot;
});
};
pkgconfigUpstream = lowPrio (pkgconfig.override { vanilla = true; });
postiats-utilities = callPackage ../development/tools/postiats-utilities {};
@ -6742,7 +6726,7 @@ in
gdbGuile = lowPrio (gdb.override { inherit guile; });
gdbCross = lowPrio (callPackage ../development/tools/misc/gdb {
target = crossSystem;
target = if targetPlatform != buildPlatform then targetPlatform else null;
});
gdb-multitarget = lowPrio (gdb.override { multitarget = true; });
@ -7344,10 +7328,10 @@ in
withGd = true;
};
glibcCross = forceNativeDrv (glibc.override {
glibcCross = forcedNativePackages.glibc.override {
gccCross = gccCrossStageStatic;
linuxHeaders = linuxHeadersCross;
});
};
# We can choose:
libcCrossChooser = name: if name == "glibc" then glibcCross
@ -7356,7 +7340,7 @@ in
else if name == "libSystem" then darwin.xcode
else throw "Unknown libc";
libcCross = assert crossSystem != null; libcCrossChooser crossSystem.libc;
libcCross = assert targetPlatform != buildPlatform; libcCrossChooser targetPlatform.libc;
# Only supported on Linux
glibcLocales = if stdenv.isLinux then callPackage ../development/libraries/glibc/locales.nix { } else null;
@ -8265,9 +8249,9 @@ in
# glibc provides libiconv so systems with glibc don't need to build libiconv
# separately, but we also provide libiconvReal, which will always be a
# standalone libiconv, just in case you want it
libiconv = if crossSystem != null then
(if crossSystem.libc == "glibc" then libcCross
else if crossSystem.libc == "libSystem" then darwin.libiconv
libiconv = if stdenv ? cross then
(if stdenv.cross.libc == "glibc" then libcCross
else if stdenv.cross.libc == "libSystem" then darwin.libiconv
else libiconvReal)
else if stdenv.isGlibc then glibcIconv stdenv.cc.libc
else if stdenv.isDarwin then darwin.libiconv
@ -10918,8 +10902,8 @@ in
cmdline = callPackage ../os-specific/darwin/command-line-tools {};
apple-source-releases = callPackage ../os-specific/darwin/apple-source-releases { };
in apple-source-releases // rec {
cctools_cross = callPackage (forceNativeDrv (callPackage ../os-specific/darwin/cctools/port.nix {}).cross) {
cross = assert crossSystem != null; crossSystem;
cctools_cross = callPackage (forcedNativePackages.callPackage ../os-specific/darwin/cctools/port.nix {}).cross {
cross = assert targetPlatform != buildPlatform; targetPlatform;
inherit maloader;
xctoolchain = xcode.toolchain;
};
@ -10992,7 +10976,7 @@ in
libossp_uuid = callPackage ../development/libraries/libossp-uuid { };
libuuid =
if crossSystem != null && crossSystem.config == "i586-pc-gnu"
if targetPlatform != buildPlatform && targetPlatform.config == "i586-pc-gnu"
then (utillinuxMinimal // {
crossDrv = lib.overrideDerivation utillinuxMinimal.crossDrv (args: {
# `libblkid' fails to build on GNU/Hurd.
@ -11077,7 +11061,7 @@ in
# GNU/Hurd core packages.
gnu = recurseIntoAttrs (callPackage ../os-specific/gnu {
inherit platform crossSystem;
inherit platform;
});
hwdata = callPackage ../os-specific/linux/hwdata { };
@ -11156,13 +11140,13 @@ in
linuxHeaders = linuxHeaders_4_4;
linuxHeaders24Cross = forceNativeDrv (callPackage ../os-specific/linux/kernel-headers/2.4.nix {
cross = assert crossSystem != null; crossSystem;
});
linuxHeaders24Cross = forcedNativePackages.callPackage ../os-specific/linux/kernel-headers/2.4.nix {
cross = assert targetPlatform != buildPlatform; targetPlatform;
};
linuxHeaders26Cross = forceNativeDrv (callPackage ../os-specific/linux/kernel-headers/4.4.nix {
cross = assert crossSystem != null; crossSystem;
});
linuxHeaders26Cross = forcedNativePackages.callPackage ../os-specific/linux/kernel-headers/4.4.nix {
cross = assert targetPlatform != buildPlatform; targetPlatform;
};
linuxHeaders_3_18 = callPackage ../os-specific/linux/kernel-headers/3.18.nix { };
@ -11173,8 +11157,8 @@ in
else if ver == "2.6" then linuxHeaders26Cross
else throw "Unknown linux kernel version";
linuxHeadersCross = assert crossSystem != null;
linuxHeadersCrossChooser crossSystem.platform.kernelMajor;
linuxHeadersCross = assert targetPlatform != buildPlatform;
linuxHeadersCrossChooser targetPlatform.platform.kernelMajor;
kernelPatches = callPackage ../os-specific/linux/kernel/patches.nix { };
@ -11839,7 +11823,7 @@ in
uclibcCross = lowPrio (callPackage ../os-specific/linux/uclibc {
linuxHeaders = linuxHeadersCross;
gccCross = gccCrossStageStatic;
cross = assert crossSystem != null; crossSystem;
cross = assert targetPlatform != buildPlatform; targetPlatform;
});
udev = systemd;

19
pkgs/top-level/default.nix
View File

@ -83,7 +83,24 @@ in let
boot = import ../stdenv/booter.nix { inherit lib allPackages; };
stages = stdenvStages {
inherit lib system platform crossSystem config overlays;
# One would think that `localSystem` and `crossSystem` overlap horribly with
# the three `*Platforms` (`buildPlatform`, `hostPlatform,` and
# `targetPlatform`; see `stage.nix` or the manual). Actually, those
# identifiers I, @Ericson2314, purposefully not used here to draw a subtle
# but important distinction:
#
# While the granularity of having 3 platforms is necessary to properly
# *build* packages, it is overkill for specifying the user's *intent* when
# making a build plan or package set. A simple "build vs deploy" dichotomy
# is adequate: the "sliding window" principle described in the manual shows
# how to interpolate between the these two "end points" to get the 3
# platform triple for each bootstrapping stage.
#
# Also, less philosophically but quite practically, `crossSystem` should be
# null when one doesn't want to cross-compile, while the `*Platform`s are
# always non-null. `localSystem` is always non-null.
localSystem = { inherit system platform; };
inherit lib crossSystem config overlays;
};
pkgs = boot stages;

4
pkgs/top-level/haskell-packages.nix
View File

@ -1,4 +1,4 @@
{ pkgs, callPackage, stdenv, crossSystem }:
{ pkgs, callPackage, stdenv, buildPlatform, targetPlatform }:
rec {
@ -55,7 +55,7 @@ rec {
ghcHEAD = callPackage ../development/compilers/ghc/head.nix rec {
bootPkgs = packages.ghc7103;
inherit (bootPkgs) alex happy;
inherit crossSystem;
inherit buildPlatform targetPlatform;
selfPkgs = packages.ghcHEAD;
};
ghcjs = packages.ghc7103.callPackage ../development/compilers/ghcjs {

14
pkgs/top-level/release-cross.nix
View File

@ -32,8 +32,10 @@ let
in
{
# These `nativeDrv`s should be identical to their vanilla ones --- cross
# compiling should not affect the native derivation.
# These derivations from a cross package set's `buildPackages` should be
# identical to their vanilla equivalents --- none of these package should
# observe the target platform which is the only difference between those
# package sets.
ensureUnaffected = let
# Absurd values are fine here, as we are not building anything. In fact,
# there probably a good idea to try to be "more parametric" --- i.e. avoid
@ -47,8 +49,12 @@ in
# good idea lest there be some irrelevant pass-through debug attrs that
# cause false negatives.
testEqualOne = path: system: let
f = attrs: builtins.toString (lib.getAttrFromPath path (allPackages attrs));
in assert f { inherit system; } == f { inherit system crossSystem; }; true;
f = path: attrs: builtins.toString (lib.getAttrFromPath path (allPackages attrs));
in assert
f path { inherit system; }
==
f (["buildPackages"] ++ path) { inherit system crossSystem; };
true;
testEqual = path: systems: forAllSupportedSystems systems (testEqualOne path);

5
pkgs/top-level/release-lib.nix
View File

@ -76,8 +76,9 @@ rec {
* parameter for allPackages, defining the target platform for cross builds,
* and triggering the build of the host derivation (cross built - crossDrv). */
mapTestOnCross = crossSystem: mapAttrsRecursive
(path: systems: testOnCross crossSystem systems
(pkgs: addMetaAttrs { maintainers = crossMaintainers; } (getAttrFromPath path pkgs)));
(path: systems: testOnCross crossSystem systems (pkgs: addMetaAttrs
{ maintainers = crossMaintainers; }
(getAttrFromPath path pkgs.splicedPackages)));
/* Recursively map a (nested) set of derivations to an isomorphic

83
pkgs/top-level/splice.nix Normal file
View File

@ -0,0 +1,83 @@
# The `splicedPackages' package set, and its use by `callPackage`
#
# The `buildPackages` pkg set is a new concept, and the vast majority package
# expression (the other *.nix files) are not designed with it in mind. This
# presents us with a problem with how to get the right version (build-time vs
# run-time) of a package to a consumer that isn't used to thinking so cleverly.
#
# The solution is to splice the package sets together as we do below, so every
# `callPackage`d expression in fact gets both versions. Each# derivation (and
# each derivation's outputs) consists of the run-time version, augmented with a
# `nativeDrv` field for the build-time version, and `crossDrv` field for the
# run-time version.
#
# We could have used any names we want for the disambiguated versions, but
# `crossDrv` and `nativeDrv` were somewhat similarly used for the old
# cross-compiling infrastructure. The names are mostly invisible as
# `mkDerivation` knows how to pull out the right ones for `buildDepends` and
# friends, but a few packages use them directly, so it seemed efficient (to
# @Ericson2314) to reuse those names, at least initially, to minimize breakage.
#
# For performance reasons, rather than uniformally splice in all cases, we only
# do so when `pkgs` and `buildPackages` are distinct. The `actuallySplice`
# parameter there the boolean value of that equality check.
lib: pkgs: actuallySplice:
let
defaultBuildScope = pkgs.buildPackages // pkgs.buildPackages.xorg;
# TODO(@Ericson2314): we shouldn't preclude run-time fetching by removing
# these attributes. We should have a more general solution for selecting
# whether `nativeDrv` or `crossDrv` is the default in `defaultScope`.
pkgsWithoutFetchers = lib.filterAttrs (n: _: !lib.hasPrefix "fetch" n) pkgs;
defaultRunScope = pkgsWithoutFetchers // pkgs.xorg;
splicer = buildPkgs: runPkgs: let
mash = buildPkgs // runPkgs;
merge = name: {
inherit name;
value = let
defaultValue = mash.${name};
buildValue = buildPkgs.${name} or {};
runValue = runPkgs.${name} or {};
augmentedValue = defaultValue
// (lib.optionalAttrs (buildPkgs ? ${name}) { nativeDrv = buildValue; })
// (lib.optionalAttrs (runPkgs ? ${name}) { crossDrv = runValue; });
# Get the set of outputs of a derivation
getOutputs = value:
lib.genAttrs (value.outputs or []) (output: value.${output});
in
# Certain *Cross derivations will fail assertions, but we need their
# nativeDrv. We are assuming anything that fails to evaluate is an
# attrset (including derivation) and thus can be unioned.
if !(builtins.tryEval defaultValue).success then augmentedValue
# The derivation along with its outputs, which we recur
# on to splice them together.
else if lib.isDerivation defaultValue then augmentedValue
// splicer (getOutputs buildValue) (getOutputs runValue)
# Just recur on plain attrsets
else if lib.isAttrs defaultValue then splicer buildValue runValue
# Don't be fancy about non-derivations. But we could have used used
# `__functor__` for functions instead.
else defaultValue;
};
in lib.listToAttrs (map merge (lib.attrNames mash));
splicedPackages =
if actuallySplice
then splicer defaultBuildScope defaultRunScope
else pkgs // pkgs.xorg;
in
{
splicedPackages = splicedPackages // { recurseForDerivations = false; };
# We use `callPackage' to be able to omit function arguments that can be
# obtained `pkgs` or `buildPackages` and their `xorg` package sets. Use
# `newScope' for sets of packages in `pkgs' (see e.g. `gnome' below).
callPackage = pkgs.newScope {};
callPackages = lib.callPackagesWith splicedPackages;
newScope = extra: lib.callPackageWith (splicedPackages // extra);
}

86
pkgs/top-level/stage.nix
View File

@ -9,8 +9,45 @@
import `pkgs/default.nix` or `default.nix`. */
{ # The system (e.g., `i686-linux') for which to build the packages.
system
{ ## Misc parameters kept the same for all stages
##
# Utility functions, could just import but passing in for efficiency
lib
, # Use to reevaluate Nixpkgs; a dirty hack that should be removed
nixpkgsFun
## Platform parameters
##
## The "build" "host" "target" terminology below comes from GNU Autotools. See
## its documentation for more information on what those words mean. Note that
## each should always be defined, even when not cross compiling.
##
## For purposes of bootstrapping, think of each stage as a "sliding window"
## over a list of platforms. Specifically, the host platform of the previous
## stage becomes the build platform of the current one, and likewise the
## target platform of the previous stage becomes the host platform of the
## current one.
##
, # The platform on which packages are built. Consists of `system`, a
# string (e.g.,`i686-linux') identifying the most import attributes of the
# build platform, and `platform` a set of other details.
buildPlatform
, # The platform on which packages run.
hostPlatform
, # The platform which build tools (especially compilers) build for in this stage,
targetPlatform
## Other parameters
##
, # The package set used at build-time. If null, `buildPackages` will
# be defined internally as the produced package set as itself.
buildPackages
, # The standard environment to use for building packages.
stdenv
@ -21,21 +58,19 @@
allowCustomOverrides
, # Non-GNU/Linux OSes are currently "impure" platforms, with their libc
# outside of the store. Thus, GCC, GFortran, & co. must always look for
# files in standard system directories (/usr/include, etc.)
noSysDirs ? (system != "x86_64-freebsd" && system != "i686-freebsd"
&& system != "x86_64-solaris"
&& system != "x86_64-kfreebsd-gnu")
# outside of the store. Thus, GCC, GFortran, & co. must always look for files
# in standard system directories (/usr/include, etc.)
noSysDirs ? buildPlatform.system != "x86_64-freebsd"
&& buildPlatform.system != "i686-freebsd"
&& buildPlatform.system != "x86_64-solaris"
&& buildPlatform.system != "x86_64-kfreebsd-gnu"
, # The configuration attribute set
config
, overlays # List of overlays to use in the fix-point.
, crossSystem
, platform
, lib
, nixpkgsFun
, # A list of overlays (Additional `self: super: { .. }` customization
# functions) to be fixed together in the produced package set
overlays
}:
let
@ -50,11 +85,28 @@ let
};
stdenvBootstappingAndPlatforms = self: super: {
stdenv = stdenv // { inherit platform; };
inherit
system platform crossSystem;
buildPackages = (if buildPackages == null then self else buildPackages)
// { recurseForDerivations = false; };
inherit stdenv
buildPlatform hostPlatform targetPlatform;
};
# The old identifiers for cross-compiling. These should eventually be removed,
# and the packages that rely on them refactored accordingly.
platformCompat = self: super: let
# TODO(@Ericson2314) this causes infinite recursion
#inherit (self) buildPlatform hostPlatform targetPlatform;
in {
stdenv = super.stdenv // {
inherit (buildPlatform) platform;
} // lib.optionalAttrs (targetPlatform != buildPlatform) {
cross = targetPlatform;
};
inherit (buildPlatform) system platform;
};
splice = self: super: import ./splice.nix lib self (buildPackages != null);
allPackages = self: super:
let res = import ./all-packages.nix
{ inherit lib nixpkgsFun noSysDirs config; }
@ -83,8 +135,10 @@ let
# The complete chain of package set builders, applied from top to bottom
toFix = lib.foldl' (lib.flip lib.extends) (self: {}) ([
stdenvBootstappingAndPlatforms
platformCompat
stdenvAdapters
trivialBuilders
splice
allPackages
aliases
stdenvOverrides