This fixes a bug whereby "guix build texlive -n" would report:
0.0 MB would be downloaded:
/gnu/store/…-texlive-20180414
instead of:
The following derivation would be built:
/gnu/store/…-texlive-texmf-20180414.drv
2,595.2 MB would be downloaded:
/gnu/store/…-texlive-20180414-texmf.tar.xz
/gnu/store/…-texlive-20180414
where 'texlive-texmf' is a non-substitutable dependency of 'texlive'.
* guix/derivations.scm (dependencies-of-substitutables): New procedure.
(derivation-build-plan): When 'input-substitutable-info' returns true,
append the subset of DEPS that corresponds to SUBSTITUABLES to the first
argument of 'loop'.
* guix/ui.scm (show-what-to-build): Remove half-baked traversal of
DOWNLOAD.
* tests/derivations.scm ("derivation-build-plan and substitutes,
non-substitutable dep"): New test.
* guix/derivations.scm (build-derivations): Accept <derivation-input>
records among DERIVATIONS.
* tests/derivations.scm ("build-derivations with specific output"): Test
it.
Consequently, the whole graph of <derivation> object is readily
available without having to go through 'read-derivation-from-file',
which could have cache misses if the requested <derivation> object had
been GC'd in the meantime. This is an important property for the
performance of things like 'derivation-build-plan' that traverse the
derivation graph.
* guix/derivations.scm (<derivation-input>): Replace 'path' field by
'derivation'.
(derivation-input-path): Adjust accordingly.
(derivation-input-key): New procedure.
(derivation-input-output-paths): Adjust accordingly.
(coalesce-duplicate-inputs): Likewise.
(derivation-prerequisites): Use 'derivation-input-key' to compute keys
for INPUT-SET.
(derivation-build-plan): Likewise.
(read-derivation): Add optional 'read-derivation-from-file' parameter.
[make-input-drvs]: Call it.
(write-derivation)[write-input]: Adjust to new <derivation-input>.
(derivation/masked-inputs): Likewise, and remove redundant
'coalesce-duplicate-inputs' call.
(derivation)[input->derivation-input]: Change to consider only the
derivation case. Update call to 'make-derivation-input'.
[input->source]: New procedure.
Separate sources from inputs.
(map-derivation): Adjust to new <derivation-input>.
* tests/derivations.scm ("parse & export"): Pass a second argument to
'read-derivation'.
("build-expression->derivation and derivation-prerequisites")
("derivation-prerequisites and valid-derivation-input?"): Adjust to new
<derivation-input>.
The new 'derivation-build-plan' procedure has a more appropriate
signature: it takes a list of <derivation-inputs> instead of taking one
<derivation>. Its body is also much simpler.
* guix/derivations.scm (derivation-build-plan): New procedure.
(derivation-prerequisites-to-build): Express in terms of
'derivation-build-plan' and mark as deprecated.
* tests/derivations.scm: Change 'derivation-prerequisites-to-build'
tests to 'derivation-build-plan' and adjust accordingly.
* tests/derivations.scm ("derivation-prerequisites-to-build when outputs
already present"): Insert random text in BUILDER, and remove two calls
to 'delete-paths'. Use 'derivation->output-path' for INPUT-PATH instead
of an old idiom.
This allows callers to request the substitution of a single derivation
output.
* guix/store.scm (build-things): Accept derivation/output pairs among
THINGS.
* guix/derivations.scm (build-derivations): Likewise.
* tests/store.scm ("substitute + build-things with specific output"):
New test.
* tests/derivations.scm ("build-derivations with specific output"):
New test.
* doc/guix.texi (The Store): Adjust accordingly.
* guix/derivations.scm (derivation-prerequisites-to-build): Rename
#:substitutable? to #:substitutable-info.
[derivation-substitutable?]: Rename to...
[derivation-substitutable-info]: ... this. Return a list of <substitutable>.
Second return value is now a list of <substitutable> instead of a list
of strings.
* guix/ui.scm (show-what-to-build)[substitutable?]: Rename to...
[substitutable-info]: ... this.
Adjust to new 'derivation-prerequisites-to-build' return value type.
* tests/derivations.scm ("derivation-prerequisites-to-build and
substitutes"): Adjust.
("derivation-prerequisites-to-build and substitutes, local build"):
Likewise.
* guix/derivations.scm (substitution-oracle): Use
'substitution-path-info' instead of 'substitution-paths'. Turn SUBST
into a vhash from path to <substitutable>. Change the returned
procedure to provide a <substitutable> instead of a Boolean.
* tests/derivations.scm ("substitution-oracle and #:substitute? #f"):
Mock 'substitutable-path-info' instead of 'substitutable-paths'.
This is a followup to 9231ef12f2.
* tests/derivations.scm ("build-expression->derivation and builder
encoding"): Set '%default-port-encoding' to "UTF-8".
Reported by Mark H Weaver <mhw@netris.org>
at <https://lists.gnu.org/archive/html/guix-devel/2017-04/msg00457.html>.
* guix/derivations.scm (build-expression->derivation): Use a string
output port for the expression. This reverts part of
2dce88d5bb.
* tests/derivations.scm ("build-expression->derivation and builder
encoding"): New test.
This avoids connections to substitute servers for derivations that are
not substitutable anyway, such as profiles.
Reported by Andy Wingo.
* guix/derivations.scm (substitution-oracle): Skip derivations that do
not pass 'substitutable-derivation?'.
* tests/derivations.scm ("substitution-oracle and #:substitute? #f"):
New test.
* guix/derivations.scm (derivation-input<?): New procedure.
(write-derivation)[coalesce-duplicate-inputs]: Remove.
Remove calls to 'sort'.
(coalesce-duplicate-inputs): New procedure.
(derivation-hash): Sort INPUTS and use 'coalesce-duplicate-inputs'.
(derivation)[input->derivation-input]
[coalesce-duplicate-inputs]: New procedures.
Sort OUTPUTS, INPUTS, and ENV-VARS.
* tests/derivations.scm ("read-derivation vs. derivation"): New test.
Before that '.log' files for scheme tests were fragmented and not
included in test-suite.log. This unifies the semantics of SRFI-64 API
with Automake test suite.
* build-aux/test-driver.scm: New file.
* Makefile.am (SCM_LOG_DRIVER, AM_SCM_LOG_DRIVER_FLAGS): New variables.
(SCM_LOG_COMPILER, AM_SCM_LOG_FLAGS): Delete variables.
(AM_TESTS_ENVIRONMENT): Set GUILE_AUTO_COMPILE to 0.
* test-env.in: Silence guix-daemon.
* doc/guix.texi (Running the Test Suite): Describe how to display the
detailed results. Bug reports require only 'test-suite.log' file.
* tests/base32.scm, tests/build-utils.scm, tests/builders.scm,
tests/challenge.scm, tests/cpan.scm, tests/cpio.scm, tests/cran.scm,
tests/cve.scm, tests/derivations.scm, tests/elpa.scm,
tests/file-systems.scm, tests/gem.scm, tests/gexp.scm,
tests/gnu-maintenance.scm, tests/grafts.scm, tests/graph.scm,
tests/gremlin.scm, tests/hackage.scm, tests/hash.scm,
tests/import-utils.scm, tests/lint.scm, tests/monads.scm, tests/nar.scm,
tests/packages.scm, tests/pk-crypto.scm, tests/pki.scm,
tests/profiles.scm, tests/publish.scm, tests/pypi.scm,
tests/records.scm, tests/scripts-build.scm, tests/scripts.scm,
tests/services.scm, tests/sets.scm, tests/size.scm, tests/snix.scm,
tests/store.scm, tests/substitute.scm, tests/syscalls.scm,
tests/system.scm, tests/ui.scm, tests/union.scm, tests/upstream.scm,
tests/utils.scm: Don't exit at the end of test groups.
* tests/containers.scm: Likewise. Use 'test-skip' instead of exiting
with error code 77.
Suggested by Jookia.
* guix/derivations.scm (&file-search-error): New error condition.
(search-path*): Raise it when 'search-path' returns #f.
* guix/gexp.scm (search-path*): Remove.
* guix/ui.scm (call-with-error-handling): Add case for
'file-search-error?'.
* tests/derivations.scm ("build-expression->derivation and invalid
module name"): New test.
This allows tests to run as expected even in the presence of
replacements among the bootstrap packages, such as Perl (commit
d8173f21f7).
* tests/cpan.scm: Add (%graft? #f).
* tests/derivations.scm: Likewise.
* tests/graph.scm: Likewise.
* tests/monads.scm: Likewise.
* tests/profiles.scm: Likewise.
* tests/gexp.scm: Likewise.
("gexp->derivation vs. grafts"): Explicitly reenable grafting before,
and disable it after, using 'set-grafting'.
* guix/derivations.scm (substitution-oracle): Add #:mode parameter and
honor it.
(derivation-prerequisites-to-build): Likewise.
[derivation-built?]: Take it into account.
* guix/ui.scm (show-what-to-build): Add #:mode parameter. Pass it to
'substitute-oracle' and 'derivations-prerequisites-to-build'.
* tests/derivations.scm ("derivation-prerequisites-to-build in 'check'
mode"): New test.
* guix/scripts/substitute.scm (or*): New macro.
(%cache-url): Honor "untrusted-substitute-urls".
* guix/tests.scm (%test-substitute-urls): New variable.
(open-connection-for-tests): Use it.
* tests/derivations.scm ("derivation-prerequisites-to-build and substitutes",
"derivation-prerequisites-to-build and substitutes, non-substitutable
build", "derivation-prerequisites-to-build and substitutes, local build"):
Pass it to 'set-build-options'.
* tests/guix-daemon.sh: Likewise.
* tests/store.scm ("substitute query, alternating URLs"): New test.
("substitute query", "substitute", "substitute + build-things with output
path", "substitute, corrupt output hash", "substitute --fallback"): Pass
#:substitute-urls to 'set-build-options'.
* tests/derivations.scm ("derivation-prerequisites and
valid-derivation-input?"): Renamed from "derivation-prerequisites and
derivation-input-is-valid?". Build %BOOTSTRAP-GUILE. This fixes a
regression introduced in cdb5b07 when running the whole test suite.
* guix/derivations.scm (valid-derivation-input?): New procedure.
(derivation-prerequisites): Add 'cut?' parameter and honor it.
* tests/derivations.scm ("derivation-prerequisites and
derivation-input-is-valid?"): New test.
* guix/tests.scm (network-reachable?): New procedure.
* tests/builders.scm (network-reachable?): Remove.
Replace references to it with calls to the new 'network-reachable?'
procedure.
* tests/derivations.scm (%coreutils): Use 'network-reachable?' instead
of 'getaddrinfo'.
* tests/packages.scm: Likewise.
* tests/union.scm: Likewise.
* guix/tests.scm (derivation-narinfo): Turn 'nar' into a keyword
parameter. Add #:sha256 parameter, and honor it.
(call-with-derivation-narinfo): Add #:sha256 and pass it to
'derivation-narinfo'.
(with-derivation-narinfo): Extend with support for (sha256 => value).
* tests/store.scm ("substitute query"): Use 'with-derivation-narinfo'.
("substitute"): Likewise.
("substitute, corrupt output hash"): Likewise.
("substitute --fallback"): Likewise.
* tests/derivations.scm: Remove Emacs local variable.
This makes 'guix environment PACKAGE' significantly faster when
substitutes are enabled. Before that, it would lead to many invocations
of 'guix substitute-binary', one per 'derivation-prerequisites-to-build'
call. Now, all these are replaced by a single invocation.
* guix/derivations.scm (derivation-output-paths, substitution-oracle):
New procedures.
(derivation-prerequisites-to-build): Replace #:use-substitutes? with
#:substitutable?. Remove the local 'derivation-output-paths' and
'substitutable?'.
* guix/ui.scm (show-what-to-build): Add 'substitutable?'. Pass it to
'derivation-prerequisites-to-build'.
[built-or-substitutable?]: Use it instead of 'has-substitutes?'.
* tests/derivations.scm ("derivation-prerequisites-to-build and
substitutes"): Use #:substitutable? instead of #:use-substitutes?.
* guix/derivations.scm (derivation-output-names): New procedure.
(derivation-prerequisites-to-build): Use it for #:outputs.
(map-derivation): Likewise.
* tests/derivations.scm ("derivation-output-names"): New test.
* guix/tests.scm (derivation-narinfo, call-with-derivation-narinfo): New
procedures.
(with-derivation-narinfo): New macro.
* tests/derivations.scm ("derivation-prerequisites-to-build and
substitutes"): Use them.
Ludovic Courtès
Marius Bakke
Mathieu Lirzin