We need this stuff to be available in lib so make-derivation.nix can
access it to construct the Meson cross file.
This has a couple of other advantages:
- It makes Rust less special. Now figuring out what Rust calls a
platform is the same as figuring out what Linux or QEMU call it.
- We can unify the schema used to define Rust targets, and the schema
used to access those values later. Just like you can set "config"
or "system" in a platform definition, and then access those same
keys on the elaborated platform, you can now set "rustcTarget" in
your crossSystem, and then access "stdenv.hostPlatform.rustcTarget"
in your code.
"rustcTarget", "rustcTargetSpec", "cargoShortTarget", and
"cargoEnvVarTarget" have the "rustc" and "cargo" prefixes because
these are not exposed to code by the compiler, and are not
standardized. The arch/os/etc. variables are all named to match the
forms in the Rust target spec JSON.
The new rust.target-family only takes a list, since we don't need to
worry about backwards compatibility when that name is used.
The old APIs are all still functional with no warning for now, so that
it's possible for external code to use a single API on both 23.05 and
23.11. We can introduce the warnings once 23.05 is EOL, and make them
hard errors when 23.11 is EOL.
We are replicating one mechanism behind `-Z build-std`.
There isn't yet crate2nix support for this, but one can (and I do) add
the missing stdlib deps (for this feature to pick up) with overrides.
This parameter is being set to `$NIX_BUILD_CORES` by default. This is a
standard practice but there's a suspicion that this can produce broken
builds. For some details see
https://github.com/cargo2nix/cargo2nix/issues/184 . As a
work-around/test, it'd be good if codegen-units can be set to something
constant, such as `1`. This PR allows it.
Note that the default of `$NIX_BUILD_CORES` is preserved so this MR
causes no change in default behaviour and no rebuilds.
According to rustc implementation[1], `-C incremental=no` enables
incremental builds with directory name `no`. This patch removes the
`-C incremental` argument to disable incremental builds.
[1]: ee86f96ba1/compiler/rustc_session/src/options.rs (L918-L919)
Cargo sets `CARGO_FEATURE_*` for all features when running a build
script:
https://doc.rust-lang.org/cargo/reference/environment-variables.html#environment-variables-cargo-sets-for-build-scripts
Some crates have build scripts (e.g. openblas-src) that rely on the
feature variables being properly set.
Since we now need several representations of features, this change
also updates `createFeatures` to be a list of features, rather than
`rustc` feature arguments. `configureCrate` and `buildCrate` then
build the required representations as-needed.
Fixes#68978
* Make errors include the crate name and make them much more prominent.
* Move more code into lib.sh
* Already source generated logging code and lib.sh in configure
It used to be the case (ref missing) that cargo did treat
`src/$libName.rs` as an alternative to `src/lib.rs` when the latter
wasn't present. Recently I failed to reproduce that with vanilla cargo
and it started to cause pain with some crates of the form:
some_crate/
`- src
`- main.rs
`- some_crate.rs
We would build `src/some_crate.rs` and thing it is a library while that
might not be the actual case. This crate is a valid `bin` crate not a
`lib` crate as far as I can tell from the samples I took.
I removed support for the previously required heuristic and commented
out the test cases in case we will need them again. We could crawl in
the Git history but chances are that the next person looking into this
doesn't know about the history.
This helps us instruct rustc to build tests instead of binaries. The
actual build will then ONLY produce test executables. This is a first
step towards having rust crate tests within nixpkgs.
We default back to only a single output in test cases since that is the
only reasonable thing to do here.
Producing libraries or binaries in addition to tests would theoretically
be feasible but usually generates different dependency trees. It is very
common to have some libraries in `[dev-depdendencies]` within Cargo.toml
just for your tests. To not start mixing things up going with a
dedicated derivation for the test build sounds like the best choice for
now.
To use this you must provide a proper test dependency chain to
`buildRustCrate` (as you would usually do with your non-test inputs).
And then set the `buildTests` attribute to `true`. The derivation will
then contain all tests that were built in `$out/tests`. All common test
patterns and directories should be supported and tested by this change.
Below is an example how you would run a single test from the derivation.
This commit contains some more examples in the `buildRustCrateTests`
attribute set that might be helpful.
```
let
drv = buildRustCrate {
…
buildTests true;
};
in runCommand "test-my-crate" {} ''
touch $out
exec ${drv}/tests/my-test
''
```
We can get rid of a bunch of workarounds that were in the build script
before by just passing on the `crateBin` attribute.
Before we converted the list of attributes to a string only to convert
it back in bash during the build phase. We can do the entire looping
through builds in Nix and thus need no conversion and parsing of
attributes over and over again.
The big part that still remains bash is the heuristic that cargo
introduced and that we can't do at eval time.
The expression is already long and confusing enough without the color
stuff sprinkled in. Moving it to a dedicated file makes sense.
I switched a bit of the color support code to pure Nix since there
wasn't much point in doing that in bash while we can just do it in Nix.
We can just use `lib` instead of `builtins` in all cases but the
`hashString` case. Also changed a few lines to make use of some optional
helpers from lib.
This cuts down the dependency tree on some rust builds where a crate not
just exposes a binary but also a library. `$out/lib` contained a bunch
of extra support files that among other information carry linker flags
(including the full path to link-time dependencies). Worst case this led
to some binary outputs depending on the full build closure of rust
crates.
Moving all the `$out/lib` files to `$lib/lib` solves this nicely.
`lib` might be a bit weird here as they are most of the time just rlib
files (rust libraries). Those are essential only required during
compilation but they can also be shared objects (like with traditional
C-style packages). Which is why I went with `lib` for the new output.
One of the caveats we are running into here is that we do not (always)
know ahead of time of a crate produces just a library or just a binary.
Cargo allows for some ambiguity regarding whether or not a crate
provides one, two, … binaries and libraries as it's outputs. Ideally we
would be able to rely on the `crateType` entirely but so far that isn't
the case. More work on that area might show how difficult that actually
is.
Before this change, buildRustCrate always called rustc with
--extern libName=[...]libName[...]
However, Cargo permits using a different name under which a dependency
is known to a crate. For example, rand 0.7.0 uses:
[dependencies]
getrandom_package = { version = "0.1.1", package = "getrandom", optional = true }
Which introduces the getrandom dependency such that it is known as
getrandom_package to the rand crate. In this case, the correct extern
flag is of the form
--extern getrandom_package=[...]getrandom[...]
which is currently not supported. In order to support such cases, this
change introduces a crateRenames argument to buildRustCrate. This
argument is an attribute set of dependencies that should be renamed. In
this case, crateRenames would be:
{
"getrandom" = "getrandom_package";
}
The extern options are then built such that if the libName occurs as
an attribute in this set, it value will be used as the local
name. Otherwise libName will be used as before.
This commit adds test based on real-world crates (brotli).
There were a few more edge cases that were missing beforehand. Also it
turned out that we can get rid of the `finalBins` list since that will
now be handled during runtime.
Cargo has a few odd (old) ways of picking source files if the `bin.path`
attribute isn't given in the Cargo.toml. This commit adds support for
some of those. The previous behaviour always defaulted to `src/main.rs`
which was not always the right choice.
Since there is look-ahead into the unpacked sources before running the
actual builder the path selection logic has to be embedded within the
build script.
`buildRustCrate` currently supports two ways of running building
binaries when processing a crate:
- Explicit definition of all the binaries (& optionally the paths to
their respective `main.rs`) and,
- if not binary was explictly configured all files matching the patterns
`src/main.rs`, `src/bin/*.rs`.
When the explicit list is given without path information paths are now
being picked from a list of candidates. The first match wins. The order
is the same as within the cargo compatibility code.
If the crate does not provide any libraries the path `src/{bin_name}.rs`
is also considered.
All underscores within the binary names are translated into dashes (`-`)
before the lookups are made. This seems to be a common convention.
