By restoring and diverting to the old version.
Previously the newer language features and use of more modern stdlib
imports broke the hook on Python<3.10.
We need this stuff to be available in lib so make-derivation.nix can
access it to construct the Meson cross file.
This has a couple of other advantages:
- It makes Rust less special. Now figuring out what Rust calls a
platform is the same as figuring out what Linux or QEMU call it.
- We can unify the schema used to define Rust targets, and the schema
used to access those values later. Just like you can set "config"
or "system" in a platform definition, and then access those same
keys on the elaborated platform, you can now set "rustcTarget" in
your crossSystem, and then access "stdenv.hostPlatform.rustcTarget"
in your code.
"rustcTarget", "rustcTargetSpec", "cargoShortTarget", and
"cargoEnvVarTarget" have the "rustc" and "cargo" prefixes because
these are not exposed to code by the compiler, and are not
standardized. The arch/os/etc. variables are all named to match the
forms in the Rust target spec JSON.
The new rust.target-family only takes a list, since we don't need to
worry about backwards compatibility when that name is used.
The old APIs are all still functional with no warning for now, so that
it's possible for external code to use a single API on both 23.05 and
23.11. We can introduce the warnings once 23.05 is EOL, and make them
hard errors when 23.11 is EOL.
To escape the pkg_resources API deprecation:
> catch-conflicts.py:1: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
Also remove exceptions for the previus bootstrap packages.
With this change it is possible to pass in `stdenv` directly to
`buildPython*` or override it using e.g.
```
numpy.overridePythonAttrs(_: {
stdenv = clangStdenv;
})
```
The situation:
Python <3.11: under Linux the abi string is always -gnu*
Python 3.11-3.12: musl is treated as its own abi in the python build system, but when cross-compiling the build host's libc is used for the target abi string. Cross compiling from glibc to musl gives a -gnu* target abi string and vice versa.
Python >=3.13: musl is treated as its own abi, and when cross-compiling the target libc is used for the target abi string
We backport the fix for python 3.11-3.12, since the intermediate state is almost impossible to model in the nix expression
This modifies the pypaBuildHook to not propagate its own python dependencies into the build environment. This prevents package conflicts.
- modify pypa-build-hook.sh to call pyproject-build via an absolute path. This removes the need of putting the dependencies inside the hook's propagatedBuildInputs
- remove the hook's dependencies from propagatedBuildInputs
- add a passthru test to the hook testing for the fix
The usage of wheel should be restricted to the hook. I discovered this
when trying to remove wheel from the Python bootstrap. Some packages
that needed wheel did not need it added explicitly because they use this
hook. This implicit change to the dependency tree shouldn't happen (even
though it is mostly harmless).
As we continue iterating on Python infrastructure for Python 3, some
code is starting to diverge for Python 2. If we copy the current state
of mk-python-derivation.nix and freeze it for Python 2, we can iterate
on it for Python 3 with more freedom.
Upstream's recommended "python -m build" way of invoking build fails
when the working directory contains a file named "build.py". This is
common for poetry projects that build C extensions.
Since the wheel file name is based on the package metadata instead of
what we set in the derivation, if you set the version as `unstable-YYYYMMDD`,
for example, the hook would silently fail. We will ignore the version
now and just use a glob instead to match anything that has the package
name in its path.
Fixes#248185.
Python always uses "gnu" prefixed ABI names, and this patch handles this by
doing a musl->gnu string replacement.
With pkgsCross.armv7l-hf-multiplatform.pkgsStatic, the previous name was:
_sysconfigdata__linux_arm-linux-gnu
Now, the corrected name is:
_sysconfigdata__linux_arm-linux-gnueabihf
This reverts commit 2ffca30cde.
`793cc9d982415b71cdba729cf779bfc49e9d2ae7` `python3: splice python within python3Packages.callPackage`
Was reverted because it broke
`pkgsCross.aarch64-multiplatform.python3Packages.cryptography`
But by reverting the revert `pkgsCross.aarch64-multiplatform.python3Packages.cryptography` still builds.
I tried reverting the 2 cryptography update commits, and it still
worked, so I do not know why this would work now.
Deeply-curried functions are pretty error-prone in untyped languages
like Nix. This is a particularly bad case because
`top-level/splice.nix` *also* declares a makeScopeWithSplicing, but
it takes *two fewer arguments*.
Let's switch to attrset-passing form, to provide some minimal level
of sanity-checking.
`lib.extends hooks pythonPackagesFun` includes python which we want to
splice
what this fixes:
```
nix-repl> lib.elemAt pkgsCross.aarch64-multiplatform.python3Packages.xpybutil.nativeBuildInputs 0
«derivation /nix/store/39dkb51rciw6zwg0c2c44gpmpjapddxc-python3-aarch64-unknown-linux-gnu-3.10.12.drv»
```
to
```
nix-repl> lib.elemAt pkgsCross.aarch64-multiplatform.python3Packages.xpybutil.nativeBuildInputs 0
«derivation /nix/store/21ldw2dp26xvv9iyxn9x77a8yh4waqz5-python3-3.10.12.drv»
```
Before https://github.com/NixOS/nixpkgs/pull/194205 The keep python used to work by accident because self was passed from
__splicedPackages https://github.com/NixOS/nixpkgs/pull/196052/files#diff-44ce3495c4f983ce64dd47c86a9d3e77bad210b2709c098a3806998dcd9b000bR213
But now it does not work because overrideAttrs is used on python in
python-packages-base.nix
Preferably we would [ splice the hooks too ](https://github.com/NixOS/nixpkgs/pull/228139) but we cannot do that until [Package sets within derivations (i.e. python3.pkgs) are not spliced](https://github.com/NixOS/nixpkgs/issues/211340) is fixed, because people often use `python3.pkgs.wrapPython` in `nativeBuildInputs` (it's correct but python3.pkgs should be python3Packages to get splicing.
The setuptools-rust requires some environment variables to really
perform cross build, otherwise it just builds for build platform.
This adds setup hook that introduces these environment variables.
There are three variables.
The PYO3_CROSS_LIB_DIR has to point to the target's Python library
directory. This has to be directory for the target not for the build or
host. We have to choose the correct target Python. I am unsure how to do
that simply in nixpkgs and this this implementations just delays this
and waits for the correct Python when package using this hook is build.
The CARGO_BUILD_TARGET triggers cross compilation in setuptools-rust.
This is simply the Rust target specification.
The CARGO_TARGET_*_LINKER variable should not be essentially required
but setuptools-rust probably mangles the Rust build environment somewhat
and that results to the missing linker. By explicitly specifying it
using the environment variable we force the correct linker.
Since python documentation requires Sphinx to build, documentation is built in
separate derivation to avoid hard recursion/bootstrapping questions and plugged
into main derivation. Care is taken that "python3.doc" derivation feels and
looks as close as possible to as if it were main derivation were multi-output
one.
This change does not cause rebuild of python3 itself.
Add a marker file to the python outputs that tells pip and other tooling
following PEP 668 that they should not be installing things system-wide.
This provides better feedback to the user and also potentially avoids issues in
single-user installs where the /nix/store is owned by the user.
For more details, see <https://peps.python.org/pep-0668/>
Here is how it currently looks like:
$ pip install requests
error: externally-managed-environment
× This environment is externally managed
╰─> This command has been disabled as it tries to modify the immutable
`/nix/store` filesystem.
To use Python with Nix and nixpkgs, have a look at the online documentation:
<https://nixos.org/manual/nixpkgs/stable/#python>.
mkPythonDerivation: apply checks to buildInputs as well
fixup: add a suggestion about nativeBuildInputs
Clean up the wording, add reasons for the suggestions
switch to throw
Adjust error to clarify build-time vs. run-time
