Commit Graph

111 Commits

Author SHA1 Message Date
Lily Foster
ddb94deafa prefetch-npm-deps: switch to data-encoding 2024-04-22 23:14:15 +10:00
Lily Foster
c588edaf25 prefetch-npm-deps: remove runtime dependency on nix 2024-04-22 23:14:15 +10:00
Sandro Jäckel
6181939cdf
fixup-yarn-lock: split out from prefetch-yarn-deps, cleanup installing
To reduce dependencies (mainly nix-prefetch-git and through that git,
git-lfs) when we just need to fixup a lock file, eg when building electron.
This also tries to avoid needless rebuilds when eg. golang is updated.

Also this cleans up and combined the build/installPhase of both tools to
be a lot simpler.
2024-04-17 11:35:22 -04:00
stuebinm
ff1a94e523 treewide: add meta.mainProgram to packages with a single binary
The nixpkgs-unstable channel's programs.sqlite was used to identify
packages producing exactly one binary, and these automatically added
to their package definitions wherever possible.
2024-03-19 03:14:51 +01:00
Lily Foster
ae86a507ed npmHooks.npmInstallHook: ignore bundle deps when calculating files to install
This assumes that downstream users of `buildNpmPackage` would rather our
own built `node_modules` be copied to the output rather than only the
`bundleDependencies` specified in the `package.json` file.

Having the latter behavior seems unexpected and unintuitive, and would
not work as installing from an `npm pack` is intended to (since doing
that would not do a `rebuild` step on those dependencies and it would
skip reifying a full dependency tree).
2024-03-14 14:04:51 +01:00
adisbladis
b6e4b86809 importNpmLock: init
This is an alternative to `fetchNpmDeps` that is notably different in that it uses metadata from `package.json` & `package-lock.json` instead of specifying a fixed-output hash.

Notable features:
- IFD free.
- Only fetches a node dependency once. No massive FODs.
- Support for URL, Git and path dependencies.
- Uses most of the existing `npmHooks`

`importNpmLock` can be used _only_ in the cases where we need to check in a `package-lock.json` in the tree.
Currently this means that we have 13 packages that would be candidates to use this function, though I expect most usage to be in private repositories.

This is upstreaming the builder portion of https://github.com/adisbladis/buildNodeModules into nixpkgs (different naming but the code is the same).
I will archive this repository and consider nixpkgs the new upstream once it's been merged.

For more explanations and rationale see https://discourse.nixos.org/t/buildnodemodules-the-dumbest-node-to-nix-packaging-tool-yet/35733

Example usage:
``` nix
stdenv.mkDerivation {
  pname = "my-nodejs-app";
  version = "0.1.0";

  src = ./.;

  nativeBuildInputs = [
    importNpmLock.hooks.npmConfigHook
    nodejs
    nodejs.passthru.python # for node-gyp
    npmHooks.npmBuildHook
    npmHooks.npmInstallHook
  ];

  npmDeps = buildNodeModules.fetchNodeModules {
    npmRoot = ./.;
  };
}
```
2024-03-05 12:23:28 +13:00
Lily Foster
8999c9d930 npmHooks.npmInstallHook: ignore rather than error for missing bin/man keys in package.json 2023-12-24 22:58:32 +01:00
Lily Foster
b5ca84b450 npmHooks.npmInstallHook: only overwrite npm cache for npm pack rather than for entire hook 2023-12-05 21:53:59 +01:00
Lily Foster
63fabdebd0
Merge pull request #267912 from reinismu/fetch-npm-deps-fix-bug
prefetch-npm-deps: add support for npm alias schema in version spec
2023-12-05 14:39:04 -05:00
Janik
b678419eac
Merge pull request #269078 from lilyinstarlight/fix/prefetch-npm-deps-more-bad-lockfiles 2023-12-03 06:53:59 +01:00
Reinis Muiznieks
02dd7c7bb3 prefetch-npm-deps: add support for npm alias schema in version spec 2023-11-29 13:14:59 +02:00
Lily Foster
25596bd419
Merge pull request #257337 from lorenzleutgeb/yarn-file
prefetch-yarn-deps: Fix handling of scoped packages
2023-11-26 16:49:33 -05:00
Lily Foster
09081aa859
fetchNpmDeps: add test case where empty default lockfile packages is needed 2023-11-23 20:52:00 -05:00
Lily Foster
77571a847f
prefetch-npm-deps: use default value when lockfile has no deps 2023-11-23 20:51:59 -05:00
Lily Foster
daec4bf734
prefetch-npm-deps: instrument some logging 2023-11-23 20:51:58 -05:00
Lily Foster
81ed58b0fe
prefetch-npm-deps: make cargo happy 2023-11-23 20:51:22 -05:00
Lily Foster
ba656ad84e
prefetch-npm-deps: bump deps 2023-11-23 20:51:14 -05:00
Nick Cao
a68bc4feaf
Merge pull request #266296 from yayayayaka/element-1.11.48
element-{web,desktop}: 1.11.47 -> 1.11.50
2023-11-23 09:27:14 -05:00
Janik
8e2c9abca8
Merge pull request #243458 from lilyinstarlight/fix/prefetch-npm-deps-error-when-no-cached-deps 2023-11-23 13:55:03 +01:00
Sandro Jäckel
f16843cb31
fetch-yarn-deps: fix missing cert when fetching packages
Found while updating element-web
2023-11-22 21:25:36 +01:00
Lily Foster
8e3009d95c
buildNpmPackage: add forceEmptyCache option 2023-11-22 15:18:49 -05:00
Lily Foster
05dc145e80
fetchNpmDeps: add forceEmptyCache option 2023-11-22 15:18:39 -05:00
Lily Foster
ec51a56dfc
prefetch-npm-deps: detect and error out when generating an empty cache 2023-11-22 15:18:16 -05:00
Lorenz Leutgeb
29cf6a0422 prefetch-yarn-deps: Fix access to .resolved
... and simplify `prefetchYarnDeps`
2023-11-22 18:03:48 +01:00
Lorenz Leutgeb
3843224a55 prefetch-yarn-deps: Fix parsing of scoped packages 2023-11-22 18:03:06 +01:00
Lily Foster
ab99231a36
prefetch-yarn-deps: add cacert to provide certificates during fetches
Ideally fetch-yarn-deps could do like some other fetchers and support
using SSL_CERT_FILE if it exists and also only verify integrity on FOD
hash unless using an empty/test hash.

But this should keep at least the same semantics as before the recent
Node.js change to stop using the built-in certificate store in favor of
the system one (which does not exist by default in the build sandbox).
2023-11-21 16:24:57 -05:00
midchildan
79459354cb buildNpmPackages: add missing deependencies 2023-11-17 07:58:58 +00:00
Felix Bühler
05a19f48a9
Merge pull request #267778 from lilyinstarlight/fix/yay-more-fixup-yarn-lock-stuff
prefetch-yarn-deps: re-add git hash for fixup
2023-11-16 21:06:38 +01:00
happysalada
7fb490674f buildNpmPackage: allow passing npmDeps 2023-11-16 11:57:48 +00:00
Lily Foster
d02eb2d0e5
prefetch-yarn-deps: re-add git hash for fixup 2023-11-15 21:02:52 -05:00
Janik
582f1e7321
Merge pull request #254420 from lilyinstarlight/fix/npm-pack-pls-😭
npmHooks.npmInstallHook: avoid script output in npm pack command
2023-11-13 22:43:57 +01:00
Yt
5aaeafbe26
buildNpmPackage: make nodejs overridable (#265171)
* buildNpmPackage: allow nodejs to be passed as argument

* Update doc/languages-frameworks/javascript.section.md

Co-authored-by: Lily Foster <lily@lily.flowers>

---------

Co-authored-by: Lily Foster <lily@lily.flowers>
2023-11-09 16:19:24 +00:00
Weijia Wang
093f098d26
Merge pull request #260011 from lilyinstarlight/fix/prefetch-npm-deps-network-error-recovery
prefetch-npm-deps: read url bodies within the retry loop
2023-10-12 19:58:20 +02:00
Lily Foster
554e2412e0
prefetch-npm-deps: read url bodies within the retry loop 2023-10-10 19:40:43 -04:00
Lily Foster
a6df5a7719
prefetch-npm-deps: bump deps 2023-10-09 11:37:16 -04:00
Doron Behar
d090cd9af4
Merge pull request #258384 from doronbehar/buildNpmPackage-makeWrapperArgs
buildNpmPackage: support makeWrapperArgs
2023-10-08 12:38:33 +03:00
Lily Foster
57087b8346
Merge pull request #255984 from SuperSandro2000/patch-1
build-npm-package: give a hint when npm prune fails
2023-10-07 17:51:52 -04:00
Sandro
30992a8c51
buildNpmPackage: give a hint when npm prune fails
Co-authored-by: Lily Foster <lily@lily.flowers>
2023-10-07 21:53:56 +02:00
Doron Behar
fc96f6d0fd buildNpmPackage: support makeWrapperArgs 2023-10-07 14:09:05 +03:00
Dan Buch
14f76a96e8
fetch-yarn-deps: warn on undefined expected hash
instead of rejecting, given that the expected hash may not be known/provided.
2023-09-27 00:17:40 +00:00
nicoo
fab52fca51 treewide: sha512 → hash 2023-09-22 18:37:42 +02:00
Rick van Schijndel
a31a3eca58
Merge pull request #251066 from lilyinstarlight/feature/prefetch-npm-deps-tokens
prefetch-npm-deps: add support for NIX_NPM_TOKENS env var
2023-09-12 17:31:26 +02:00
Lily Foster
7f76ac6e09
fetchNpmDeps: pass NIX_NPM_TOKENS as an impure env var 2023-09-11 16:50:17 -04:00
Lily Foster
e271266179
prefetch-npm-deps: add support for NIX_NPM_TOKENS env var 2023-09-11 16:49:36 -04:00
Lily Foster
9b2ca17abc
npmHooks.npmInstallHook: avoid script output in npm pack command 2023-09-10 10:31:41 -04:00
sternenseemann
bd374243c0 npmHooks: use adjacent packages, not buildPackages
Hooks are essentially implemented as special shell packages that run on
their respective host platform. When they are used, they appear as
nativeBuildInputs (as they need to be executed as part of the build of a
package using them) so are taken from buildPackages relative to the
derivation using them.

Since the override in buildNpmPackage nullifies splicing, we take
npmHooks from buildPackages manually.

Fixes pkgsCross.ghcjs.buildPackages.emscripten and thus
pkgsCross.ghcjs.haskellPackages.ghc.
2023-09-09 17:55:37 +02:00
Robert Schütz
96e5dce777 buildNpmPackage: use installManPage 2023-08-19 10:04:00 -07:00
Pierre Bourdon
28693fb1c3
buildNpmPackage: symlink manpages to the correct output directory 2023-08-18 12:41:05 +02:00
Lily Foster
785ed11d0a
prefetch-npm-deps: fix error typo and unnecessary name qualifier 2023-08-09 19:13:26 -04:00
MicroBlock
c2b8bf7f4e
feat: implement redirect for fetch-yarn-deps
This would fix NixOS/nixpkgs#245415
2023-07-26 15:13:37 +08:00