Fixes <https://issues.guix.gnu.org/54723>.
Reported by Guillaume Le Vaillant <glv@posteo.net>.
Regression introduced in f743f2046b.
With commit f743f2046b, responses to
pipelined GETs would end up being written concurrently by many threads.
Thus the body of those responses could be interleaved and garbled.
* guix/scripts/publish.scm: Revert
f743f2046b.
* tests/publish.scm ("/*.narinfo pipeline"): New test.
This will allow mirror operators to alter these non-normative bits of a
narinfo without having to resign narinfos.
* guix/scripts/publish.scm (narinfo-string): Remove
URL/Compression/FileSize from BASE-INFO. Move them after "Signature".
* tests/publish.scm ("/*.narinfo")
("/*.narinfo with properly encoded '+' sign")
("/*.narinfo with lzip + gzip")
("with cache, lzip + gzip"): Adjust accordingly.
* tests/substitute.scm ("query narinfo with signature over relevant subset"):
New test.
Fixes <https://bugs.gnu.org/47239>.
Reported by Konrad Hinsen <konrad.hinsen@fastmail.net>.
* tests/publish.scm ("with cache"): Pass the result of 'stat:perms' to
'logand' to be umask-insensitive.
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
* guix/scripts/publish.scm (compress-nar)[write-compressed-file]: New
procedure.
Use it for 'gzip' and 'lzip'. Add 'zstd.
(nar-response-port, string->compression-type): Add case for 'zstd'.
* tests/publish.scm (zstd-supported?): New procedure.
("/nar/zstd/*"): New test.
* doc/guix.texi (Invoking guix publish): Document zstd compression.
(Base Services): Add cross-reference to the above node.
Reported by Ricardo Wurmus <rekado@elephly.net>.
* guix/scripts/publish.scm (compress-nar): Add 'chmod' call to ensure
PORT is #o644, in the uncompressed case.
(bake-narinfo+nar): Likewise for the narinfo file.
* tests/publish.scm ("with cache"): Check permissions on CACHED and NAR.
Fixes <https://bugs.gnu.org/44442>.
Regression introduced in 2b2ab7796a.
* guix/scripts/publish.scm (render-nar/cached): Do not call
'bypass-cache?' when ITEM is the empty string.
* tests/publish.scm ("with cache, cache bypass, unmapped hash part"):
New test.
* guix/scripts/publish.scm (render-signing-key): New procedure.
(render-home-page): Add link to it.
(make-request-handler): Handle it.
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
* Makefile.am (MODULES): Remove guix/zlib.scm and guix/lzlib.scm,
(SCM_TESTS): remove tests/zlib.scm, tests/lzlib.scm.
* build-aux/build-self.scm (make-config.scm): Remove unused %libz variable.
* configure.ac: Remove LIBZ and LIBLZ variables and check instead for
Guile-zlib and Guile-lzlib.
* doc/guix.texi ("Requirements"): Remove zlib requirement and add Guile-zlib
and Guile-lzlib instead.
* gnu/packages/package-management.scm (guix)[native-inputs]: Add "guile-zlib"
and "guile-lzlib",
[inputs]: remove "zlib" and "lzlib",
[propagated-inputs]: ditto,
[arguments]: add "guile-zlib" and "guile-lzlib" to Guile load path.
* guix/config.scm.in (%libz, %liblz): Remove them.
* guix/lzlib.scm: Remove it.
* guix/man-db.scm: Use (zlib) instead of (guix zlib).
* guix/profiles.scm (manual-database): Do not stub (guix config) in imported
modules list, instead add "guile-zlib" to the extension list.
* guix/scripts/publish.scm: Use (zlib) instead of (guix zlib) and (lzlib)
instead of (guix lzlib),
(string->compression-type, effective-compression): do not check for zlib and
lzlib availability.
* guix/scripts/substitute.scm (%compression-methods): Do not check for lzlib
availability.
* guix/self.scm (specification->package): Add "guile-zlib" and "guile-lzlib"
and remove "zlib" and "lzlib",
(compiled-guix): remove "zlib" and "lzlib" arguments and add guile-zlib and
guile-lzlib to the dependencies, also do not pass "zlib" and "lzlib" to
"make-config.scm" procedure,
(make-config.scm): remove "zlib" and "lzlib" arguments as well as %libz and
%liblz variables.
* guix/utils.scm (lzip-port): Use (lzlib) instead of (guix lzlib) and do not
check for lzlib availability.
* guix/zlib.scm: Remove it.
* m4/guix.m4 (GUIX_LIBZ_LIBDIR, GUIX_LIBLZ_FILE_NAME): Remove them.
* tests/lzlib.scm: Use (zlib) instead of (guix zlib) and (lzlib)
instead of (guix lzlib), and do not check for zlib and lzlib availability.
* tests/publish.scm: Ditto.
* tests/substitute.scm: Do not check for lzlib availability.
* tests/utils.scm: Ditto.
* tests/zlib.scm: Remove it.
This allows 'guix publish' to compress and advertise multiple
compression methods from which users can choose.
* guix/scripts/publish.scm (actual-compression): Rename to...
(actual-compressions): ... this. Expect REQUESTED to be a list, and
always return a list.
(%default-options): Remove 'compression.
(store-item->recutils): New procedure.
(narinfo-string): Change #:compression to #:compressions (plural).
Adjust accordingly.
(render-narinfo, render-narinfo/cached): Likewise.
(bake-narinfo+nar): Change #:compression to #:compressions.
[compressed-nar-size]: New procedure.
Call 'compress-nar' for each item returned by 'actual-compressions'.
Create a narinfo for each compression.
(effective-compression): New procedure.
(make-request-handler): Change #:compression to #:compressions.
Use 'effective-compression' to determine the applicable compression.
(guix-publish): Adjust handling of '--compression'.
Print a message for each compression that is enabled.
* tests/publish.scm ("/*.narinfo"): Adjust to new narinfo field
ordering.
("/*.narinfo with properly encoded '+' sign"): Likewise.
("/*.narinfo with lzip + gzip"): New test.
("with cache, lzip + gzip"): New test.
* doc/guix.texi (Invoking guix publish): Document it.
Fixes <https://bugs.gnu.org/33721>.
Reported by Chris Marusich <cmmarusich@gmail.com>.
* guix/scripts/publish.scm (render-nar/cached): Add #:ttl and honor it.
(make-request-handler): Pass #:ttl to 'render-nar/cached'.
* tests/publish.scm ("with cache, uncompressed"): Pass "--ttl=42h" to
'guix publish'. Check 'Cache-Control' on narinfo response and on nar
response.
Fixes <https://bugs.gnu.org/30868>.
Reported by Martin Castillo <castilma@uni-bremen.de>.
* tests/publish.scm ("/log/NAME"): Use #$(random-text) in the builder.
Fixes <https://bugs.gnu.org/30184>.
Regression introduced in 297e04d660.
Reported by Christopher Baines <mail@cbaines.net>.
* guix/scripts/publish.scm (nar-response-port): Add 'compression'
parameter and honor it.
(http-write): Get 'x-nar-compression' from the initial RESPONSE.
Fixes <https://bugs.gnu.org/28664>.
Reported by Maxim Cournoyer <maxim.cournoyer@gmail.com>.
* guix/scripts/publish.scm (bake-narinfo+nar): When COMPRESSION is
'none, write NAR instead of doing nothing.
(make-request-handler): Use 'render-nar/cached' for /nar URLs with no
compression.
* tests/publish.scm ("with cache", "with cache, uncompressed"): Adjust
expected result accordingly.
Fixes <http://bugs.gnu.org/26975>.
Reported by Mark H Weaver <mhw@netris.org>.
* guix/scripts/publish.scm (bake-narinfo+nar): Pass #f as the 2nd
argument to 'stat' and properly handle #f.
* tests/publish.scm (wait-for-file): New procedure.
("with cache"): Remove 'wait-for-file' procedure.
("with cache, uncompressed"): New test.
* guix/scripts/publish.scm (not-found): Add #:phrase and #:ttl
parameters and honor them.
* tests/publish.scm ("with cache"): Check the 'cache-control' header on
of the 404 response.
* guix/scripts/publish.scm (narinfo-string): Add #:file-size parameter.
Produce a "FileSize" field when COMPRESSION is eq? to '%no-compression'
or when FILE-SIZE is true.
(bake-narinfo+nar): Pass #:file-size.
* tests/publish.scm ("/*.narinfo")
("/*.narinfo with properly encoded '+' sign")
("with cache"): Check for "FileSize".
Fixes <http://bugs.gnu.org/26201>.
Reported by <dian_cecht@zoho.com>.
These options allow nars to be "baked" off-line and cached instead of
being compressed on the fly. As a side-effect, this allows us to
provide a 'Content-Length' header for nars.
* guix/scripts/publish.scm (show-help, %options): Add '--cache' and
'--workers'.
(%default-options): Add 'workers'.
(nar-cache-file, narinfo-cache-file, run-single-baker): New procedures.
(single-baker): New macro.
(render-narinfo/cached, bake-narinfo+nar)
(render-nar/cached): New procedures.
(make-request-handler): Add #:cache and #:pool parameters and honor
them.
(run-publish-server): Likewise.
(guix-publish): Honor '--cache' and '--workers'.
* tests/publish.scm ("with cache"): New test.
* doc/guix.texi (Invoking guix publish): Document it.
* guix/scripts/publish.scm (narinfo-string): Force %NO-COMPRESSION when
STORE-PATH matches 'compressed-file?'.
* guix/utils.scm (compressed-file?): New procedure.
* tests/publish.scm ("/*.narinfo for a compressed file"): New test.
Fixes <http://bugs.gnu.org/24060>.
Reported by Chris Marusich <cmmarusich@gmail.com>.
* tests/publish.scm (http-get-port): Explicitly call
'open-socket-for-uri' and add calls to 'setvbuf'.
* guix/scripts/publish.scm (show-help, %options): Add '--compression'.
(<compression>): New record type.
(%no-compression, %default-gzip-compression): New variables.
(%default-options): Add 'compression' key.
(narinfo-string): Add #:compression parameter and honor it.
(render-narinfo): Likewise.
(render-nar): Likewise.
<top level>: Add call to 'declare-header!'.
(swallow-zlib-error): New macro.
(nar-response-port): New procedure.
(http-write): Add call to 'force-output'. Use 'nar-response-port'
instead of 'response-port'. Use 'swallow-zlib-error'.
(make-request-handler): Add #:compression parameter and honor it. Add
"nar/gzip" URL handler.
(run-publish-server): Add #:compression parameter and honor it.
(guix-publish): Honor --compression.
* tests/publish.scm (http-get-port, wait-until-ready): New procedures.
<top level>: Run main server with "-C0". Call 'wait-until-ready'.
("/nar/gzip/*", "/*.narinfo with compression"): New tests.
* doc/guix.texi (Invoking guix publish): Document it.
Fixes <http://bugs.gnu.org/21888>.
Reported by iyzsong@member.fsf.org (宋文武).
* guix/scripts/publish.scm (narinfo-string): Use
'encode-and-join-uri-path' instead of 'string-append' to compute URL.
* tests/publish.scm ("/*.narinfo with properly encoded '+' sign"):
("/nar/ with properly encoded '+' sign"): New tests.
Before that '.log' files for scheme tests were fragmented and not
included in test-suite.log. This unifies the semantics of SRFI-64 API
with Automake test suite.
* build-aux/test-driver.scm: New file.
* Makefile.am (SCM_LOG_DRIVER, AM_SCM_LOG_DRIVER_FLAGS): New variables.
(SCM_LOG_COMPILER, AM_SCM_LOG_FLAGS): Delete variables.
(AM_TESTS_ENVIRONMENT): Set GUILE_AUTO_COMPILE to 0.
* test-env.in: Silence guix-daemon.
* doc/guix.texi (Running the Test Suite): Describe how to display the
detailed results. Bug reports require only 'test-suite.log' file.
* tests/base32.scm, tests/build-utils.scm, tests/builders.scm,
tests/challenge.scm, tests/cpan.scm, tests/cpio.scm, tests/cran.scm,
tests/cve.scm, tests/derivations.scm, tests/elpa.scm,
tests/file-systems.scm, tests/gem.scm, tests/gexp.scm,
tests/gnu-maintenance.scm, tests/grafts.scm, tests/graph.scm,
tests/gremlin.scm, tests/hackage.scm, tests/hash.scm,
tests/import-utils.scm, tests/lint.scm, tests/monads.scm, tests/nar.scm,
tests/packages.scm, tests/pk-crypto.scm, tests/pki.scm,
tests/profiles.scm, tests/publish.scm, tests/pypi.scm,
tests/records.scm, tests/scripts-build.scm, tests/scripts.scm,
tests/services.scm, tests/sets.scm, tests/size.scm, tests/snix.scm,
tests/store.scm, tests/substitute.scm, tests/syscalls.scm,
tests/system.scm, tests/ui.scm, tests/union.scm, tests/upstream.scm,
tests/utils.scm: Don't exit at the end of test groups.
* tests/containers.scm: Likewise. Use 'test-skip' instead of exiting
with error code 77.
Before that, /nar requests could succeed if the requested store item
exists but is invalid (although such requests were unlikely because the
corresponding narinfo request would have failed.)
* guix/scripts/publish.scm (render-nar): Add 'store' parameter. Use
'valid-path?' instead of 'file-exists?'.
(make-request-handler): Adjust 'render-nar' call accordingly.
* tests/publish.scm ("/nar/invalid"): New test.
Ludovic Courtès
Cees de Groot
Amar Singh
Mathieu Othacehe
Mathieu Lirzin
Alex Kost
David Thompson