On the daemon side, nixStore gets set to the environment variable
NIX_STORE_DIR, else the environment variable NIX_STORE else the compile time
macro NIX_STORE_DIR (see the Settings::processEnvironment method in
nix/libstore/globals.cc). When creating a build environment, it sets
NIX_STORE with the value computed as described above. Hence, it's safer to
look for both NIX_STORE_DIR and NIX_STORE in (guix build utils), so that it
works in any context (build context or external context).
* guix/build/utils.scm (%store-directory): Consider both NIX_STORE_DIR and
NIX_STORE as environment variables.
Previously, 'patch-and-repack' would always create a tar.xz archive as a
result, even if the input was a directory (a checkout). This change
reduces gratuitous CPU and storage overhead.
* guix/packages.scm (patch-and-repack)[tarxz-name]: Remove 'checkout?' case.
[build](repack): New procedure, with "tar" invocation formerly at the
top level.
If SOURCE is a directory, call 'copy-recursively'; otherwise, call
'repack'.
Change NAME to ORIGINAL-FILE-NAME when it matches 'checkout?'.
This is a followup to 04baa011e9.
* guix/build/gnu-build-system.scm (gnu-build): Really ignore the return
value of PROC. Wrap PROC call in 'with-throw-handler'. Add
'end-of-phase' procedure and use it.
This is the case for macros (formats), for which the TeX Directory Structure
specifies to be located at the root of the source tree directly.
* guix/build-system/texlive.scm (texlive-ref): Make the ID parameter
optional. Update doc.
Before this change, compiling the module would emit the following message:
"guix/packages.scm:213:25: warning: macro `base32' used before definition".
* guix/packages.scm (define-compile-time-decoder)
(base32, base64): Move definitions to the top of the module.
This follows the fix to <https://bugs.gnu.org/30116>.
* guix/build/emacs-build-system.scm (patch-el-files)
[file-contains-nul-char]: Remove nested procedure.
[el-file]: Do no filter out files containing NUL characters.
Fixes <https://issues.guix.gnu.org/30116>.
Before this change, the presence of a NUL character on a line meant that
the (glibc) regexp engine used by Guile would either 1. stop scanning the
string or 2. crash with the error "string contains #\\nul character",
depending on the locale used.
This change works around this limitation by first replacing the NUL character
by an unused Unicode code point, doing the substitution, then reverting the
replacement.
* guix/build/utils.scm (unused-private-use-code-point)
(replace-char): New procedures.
(substitute): Make use of the above procedures to work around the NUL
character regexp engine limitation.
* tests/build-utils.scm: Add tests.
Co-authored-by: Maxim Cournoyer <maxim.cournoyer@gmail.com>
Signed-off-by: Maxim Cournoyer <maxim.cournoyer@gmail.com>
Fixes <https://bugs.gnu.org/45709>.
This is a follow-up to commit cf289d7cfa.
* guix/ui.scm (run-guix-command): Match for #false first.
Signed-off-by: Ricardo Wurmus <rekado@elephly.net>
* guix/upstream.scm (lookup-updater, package-latest-release)
(package-latest-release*, package-update): Make 'updaters' an optional
parameter.
* guix/lint.scm (check-for-updates): Remove second argument to
'package-latest-release*'.
* guix/scripts.scm (%command-categories): Add extension category.
* guix/ui.scm (source-file-command): Also parse extensions files.
(command-files): Accept an optional directory argument.
(extension-directories): New procedure.
(commands): Use it.
(show-guix-help): Hide empty categories.
(run-guix-command): Try loading an extension if there is no matching Guix
command.
Reported by Christopher Baines <mail@cbaines.net>
at <https://issues.guix.gnu.org/45323#2>.
* guix/scripts/substitute.scm (call-with-cached-connection): Handle
'gnutls-error and ERROR/INVALID-SESSION.
* guix/scripts/hash.scm (guix-hash): When 'recursive? is true, use
'open-hash-port' instead of 'open-sha256-port'.
* tests/guix-hash.sh: Add test for 'guix hash -r -H sha512'.
Otherwise man and info readers are not able to read their content as they rely
on the file extension to detect if compression is used.
* guix/build/gnu-build-system.scm (compress-documentation)
[retarget-symlink]: Append the compressor file extension to the link file name.
Suggested by Philippe Swartvagher <philippe.swartvagher@inria.fr>.
* guix/transformations.scm (transform-package-patches): New procedure.
(%transformations): Add it as 'with-patch'.
(%transformation-options, show-transformation-options-help/detailed):
Add '--with-patch'.
* tests/transformations.scm ("options->transformation, with-patch"): New
test.
* doc/guix.texi (Package Transformation Options): Document it.
Fixes <https://bugs.gnu.org/45453>.
* guix/import/utils.scm (alist->package): Find plain license names in (guix
licenses) first, and only fall back to SPDX names on error.
* tests/import-utils.scm ("alist->package with SPDX license name 1/2",
"alist->package with SPDX license name 2/2"): New tests.
This simplifies setup of build machines: no need to install Guile in
addition to Guix, no need to set 'GUILE_LOAD_PATH' & co., leading to
fewer failure modes.
* guix/ssh.scm (remote-run): New procedure.
(remote-daemon-channel): Use it instead of 'open-remote-pipe*'.
(store-import-channel)[import]: Remove check for module availability.
Add call to 'primitive-exit'.
Use 'remote-run' instead of 'open-remote-pipe'.
(store-export-channel)[export]: Remove check for module availability.
Add calls to 'primitive-exit'.
Use 'remote-run' instead of 'open-remote-pipe'.
(handle-import/export-channel-error): Remove 'module-error' clause.
(report-module-error): Remove.
* guix/scripts/offload.scm (assert-node-has-guix): Replace call to
'report-module-error' by 'leave'.
* doc/guix.texi (Daemon Offload Setup): Remove mention of Guile.
This significantly speeds up things like substituting the closure of a
.drv. This is a followup to 5ff521452b.
* guix/scripts/substitute.scm (http-multiple-get): Add #:open-connection
and #:keep-alive? and honor them.
(open-connection-for-uri/maybe): Use 'open-connection-for-uri/cached'
instead of 'guix:open-connection-for-uri'. Call 'http-multiple-get'
within 'call-with-cached-connection'.
(open-connection-for-uri/cached): Add #:timeout and #:verify-certificate?
and honor them.
(call-with-cached-connection): Add 'open-connection' parameter and
honor it.
* guix/build-system/gnu.scm (gnu-build, gnu-cross-build): Accept
bootstrap-scripts keyword argument and pass it to gnu-build.
(%bootstrap-scripts): New variable.
* guix/build/gnu-build-system.scm (%bootstrap-scripts): Remove variable.
(bootstrap): Remove default value for bootstrap-scripts argument.
* guix/import/cran.scm (%input-style): New parameter.
(format-inputs): Use it.
* guix/scripts/import/cran.scm (guix-import-cran): Set the %input-style
parameter.
(%options): Add "--style" option.
* doc/guix.texi (Invoking guix import): Document it.
* gnu/bootloader/grub.scm (grub-configuration-file): New parameter
store-crypto-devices.
[crypto-devices]: New helper function.
[builder]: Use crypto-devices.
* gnu/machine/ssh.scm (roll-back-managed-host): Use
boot-parameters-store-crypto-devices to provide its contents to the
bootloader configuration generation process.
* gnu/tests/install.scm (%encrypted-root-not-boot-os,
%encrypted-root-not-boot-os): New os declaration.
(%encrypted-root-not-boot-installation-script): New script, whose contents
were initially taken from %encrypted-root-installation-script.
(%test-encrypted-root-not-boot-os): New test.
* gnu/system.scm (define-module): Export
operating-system-bootoader-crypto-devices and
boot-parameters-store-crypto-devices.
(<boot-parameters>): Add field store-crypto-devices.
(read-boot-parameters): Parse store-crypto-devices field.
[uuid-sexp->uuid]: New helper function extracted from
device-sexp->device.
(operating-system-bootloader-crypto-devices): New function.
(operating-system-bootcfg): Use
operating-system-bootloader-crypto-devices to provide its contents to
the bootloader configuration generation process.
(operating-system-boot-parameters): Add store-crypto-devices to the
generated boot-parameters.
(operating-system-boot-parameters-file): Likewise to the file with
the serialized structure.
* guix/scripts/system.scm (reinstall-bootloader): Use
boot-parameters-store-crypto-devices to provide its contents to the
bootloader configuration generation process.
* tests/boot-parameters.scm (%default-store-crypto-devices): New
variable.
(%grub-boot-parameters, test-read-boot-parameters): Use
%default-store-crypto-devices.
(tests store-crypto-devices): New tests.
Fixes <https://issues.guix.gnu.org/45314>.
* guix/avahi.scm (avahi-browse-service-thread): Change timeout default value
to false when no "stop-loop?" procedure is passed. Adapt "iterate-simple-poll"
call accordingly.
Signed-off-by: Mathieu Othacehe <othacehe@gnu.org>
This removes the main source of latency between subsequent downloads.
* nix/libstore/build.cc (SubstitutionGoal::tryToRun): Add a
"deduplicate" key to ENV.
(SubstitutionGoal::finished): Remove call to 'optimisePath'.
* guix/scripts/substitute.scm (process-substitution)[destination-in-store?]
[dump-file/deduplicate*]: New variables.
Pass #:dump-file to 'restore-file'.
* guix/scripts/substitute.scm (guix-substitute)[deduplicate?]: New
variable.
Pass #:deduplicate? to 'process-substitution'.
* guix/serialization.scm (dump-file): Export and augment 'dump-file'.
This way, the hash of the store item can be computed as it is restored,
thereby avoiding an additional file tree traversal ('hashPath' call)
later on in the daemon. Consequently, it should reduce latency between
subsequent substitute downloads.
This is a followup to 5ff521452b.
* guix/scripts/substitute.scm (narinfo-hash-algorithm+value): New
procedure.
(process-substitution): Wrap INPUT into a hash input port, 'hashed', and
read from it. Compare the actual and expected hashes, and print a
"hash-mismatch" status line when they differ. When they match, print
not just "success" but also the nar hash and size.
* nix/libstore/build.cc (class SubstitutionGoal)[expectedHashStr]:
Remove.
(SubstitutionGoal::finished): Tokenize 'status'. Parse it and handle
"success" and "hash-mismatch" accordingly. Call 'hashPath' only when
the returned hash is not SHA256.
(SubstitutionGoal::handleChildOutput): Remove 'expectedHashStr'
handling.
* tests/substitute.scm ("substitute, invalid hash"): Rename to...
("substitute, invalid narinfo hash"): ... this.
("substitute, invalid hash"): New test.
Fixes <https://bugs.gnu.org/44985>.
Reported by Simon Josefsson <simon@josefsson.org>.
* guix/progress.scm (display-download-progress): Add #:tty? and honor it.
* guix/status.scm (print-build-event): Pass #:tty? to
'display-download-progress'.
* guix/import/elpa.scm (default-files-spec): New variable.
(download-git-repository, package-name->melpa-recipe, file-hash, vcs-file?,
git-repository->origin, melpa-recipe->origin, melpa-recipe->maybe-arguments):
New procedures.
(elpa-package->sexp): Add optional repo argument, and use it to determine
whether to attempt to construct a source using the MELPA recipe.
(elpa->guix-package): Pass repo to elpa-package->sexp.
Signed-off-by: Christopher Baines <mail@cbaines.net>
* guix/import/cran.scm (%cran-canonical-url): New variable.
(description->package): Construct home-page using canonical URL.
Signed-off-by: Christopher Baines <mail@cbaines.net>
This avoids repeated (mkdir-p "/gnu/store/.links") calls when
deduplicating lots of files.
* guix/store/deduplication.scm (deduplicate): Remove initial call to
'mkdir-p'. Add ENOENT case in 'link' exception handler. Reindent.
* tests/store-deduplication.scm ("deduplicate, ENOSPC"): Check
for (<= links 4) to account for the initial 'link' call.
* guix/store/database.scm (timestamp): New procedure.
(sqlite-register): Use it as the default for #:time.
(register-items): Likewise for #:registeration-time.
Fixes <https://bugs.gnu.org/44760>.
Previously, the 'register-path' call would re-traverse ITEM to compute
its nar hash, even though that hash is already known in the initial
store. This patch also avoids repeated opening/closing of the
database.
* guix/store/database.scm (call-with-database): Export.
* guix/scripts/system.scm (copy-item): Add 'db' parameter. Call
'sqlite-register' instead of 'register-path'.
(copy-closure): Remove redundant call to 'references*'. Call
'call-with-database' and pass the database to 'copy-item'.
Partly fixes <https://bugs.gnu.org/44760>.
* guix/build/store-copy.scm (copy-store-item): New procedure.
(populate-store): Use it instead of the inline 'copy-recursively' call.
* guix/scripts/system.scm (copy-item): Likewise.
Pass #:reset-timestamps? and #:deduplicate? to 'register-path'.
It is now up to the caller to deduplicate store contents.
* guix/store/database.scm (register-items): Remove #:deduplicate?
parameter and call to 'deduplicate'.
(register-path): Call 'deduplicate' when #:deduplicate? is true.
* gnu/build/image.scm (register-closure): Adjust call accordingly.
* gnu/build/vm.scm (register-closure): Likewise.
* guix/nar.scm (finalize-store-file): Likewise.
* guix/scripts/pack.scm (store-database): Likewise.
Until now deduplication was performed as an additional pass after
copying files, which involve re-traversing all the files that had just
been copied.
* guix/store/deduplication.scm (copy-file/deduplicate): New procedure.
* tests/store-deduplication.scm ("copy-file/deduplicate"): New test.
* guix/build/store-copy.scm (populate-store): Add #:deduplicate?
parameter and honor it.
* tests/gexp.scm ("gexp->derivation, store copy"): Pass #:deduplicate? #f
to 'populate-store'.
* gnu/build/image.scm (initialize-root-partition): Pass #:deduplicate?
to 'populate-store'. Pass #:deduplicate? #f to 'register-closure'.
* gnu/build/vm.scm (root-partition-initializer): Likewise.
* gnu/build/install.scm (populate-single-profile-directory): Pass
#:deduplicate? #f to 'populate-store'.
* gnu/build/linux-initrd.scm (build-initrd): Likewise.
* guix/scripts/pack.scm (self-contained-tarball)[import-module?]: New
procedure.
[build]: Pass it as an argument to 'source-module-closure'.
* guix/scripts/pack.scm (squashfs-image)[build]: Wrap in
'with-extensions'.
* gnu/system/linux-initrd.scm (expression->initrd)[import-module?]: New
procedure.
[builder]: Pass it to 'source-module-closure'.
* gnu/system/install.scm (cow-store-service-type)[import-module?]: New
procedure. Pass it to 'source-module-closure'.
Maxim Cournoyer
Ludovic Courtès
Mark H Weaver
Simon Tournier
Ricardo Wurmus
guy fleury iteriteka
Zhu Zihao
Efraim Flashner
Ludovic Courtès
Nicolas Goaziou
Miguel Ángel Arruga Vivas
Mathieu Othacehe
Carlo Zancanaro
Lars-Dominik Braun