me
/
guix
Archived
1
0
Fork 0

offload: Modify the build-machine record to accept multiple systems.

* guix/scripts/offload.scm (<build-machine>)[systems]: New field.
[system]: Accessor changed to %build-machine-system.  Default to #f.
* guix/scripts/offload.scm (build-machine-system): Wrap %build-machine-system
with a deprecation warning.
(build-machine-systems): Access the new systems field or fallback to use
build-machine-system, for backward compatibility.
(machine-matches?): Adjust.
* tests/offload.scm: Add tests...
* Makefile.am (SCM_TESTS): ...and register them.
* doc/guix.texi (Daemon Offload Setup): Update doc.
master
Maxim Cournoyer 2020-08-24 16:26:14 -04:00
parent a5ccf1b522
commit 4b5a6fbc9b
No known key found for this signature in database
GPG Key ID: 1260E46482E63562
4 changed files with 111 additions and 20 deletions

View File

@ -433,6 +433,7 @@ SCM_TESTS = \
tests/monads.scm \
tests/nar.scm \
tests/networking.scm \
tests/offload.scm \
tests/opam.scm \
tests/openpgp.scm \
tests/packages.scm \

View File

@ -1043,29 +1043,31 @@ When desired, the build daemon can @dfn{offload} derivation builds to
other machines running Guix, using the @code{offload} @dfn{build
hook}@footnote{This feature is available only when
@uref{https://github.com/artyom-poptsov/guile-ssh, Guile-SSH} is
present.}. When that
feature is enabled, a list of user-specified build machines is read from
@file{/etc/guix/machines.scm}; every time a build is requested, for
instance via @code{guix build}, the daemon attempts to offload it to one
of the machines that satisfy the constraints of the derivation, in
particular its system type---e.g., @file{x86_64-linux}. Missing
prerequisites for the build are copied over SSH to the target machine,
which then proceeds with the build; upon success the output(s) of the
build are copied back to the initial machine.
present.}. When that feature is enabled, a list of user-specified build
machines is read from @file{/etc/guix/machines.scm}; every time a build
is requested, for instance via @code{guix build}, the daemon attempts to
offload it to one of the machines that satisfy the constraints of the
derivation, in particular its system types---e.g., @code{x86_64-linux}.
A single machine can have multiple system types, either because its
architecture natively supports it, via emulation (@pxref{Transparent
Emulation with QEMU}), or both. Missing prerequisites for the build are
copied over SSH to the target machine, which then proceeds with the
build; upon success the output(s) of the build are copied back to the
initial machine.
The @file{/etc/guix/machines.scm} file typically looks like this:
@lisp
(list (build-machine
(name "eightysix.example.org")
(system "x86_64-linux")
(systems (list "x86_64-linux" "i686-linux"))
(host-key "ssh-ed25519 AAAAC3Nza@dots{}")
(user "bob")
(speed 2.)) ;incredibly fast!
(build-machine
(name "armeight.example.org")
(system "aarch64-linux")
(systems (list "aarch64-linux"))
(host-key "ssh-rsa AAAAB3Nza@dots{}")
(user "alice")
(private-key
@ -1075,8 +1077,8 @@ The @file{/etc/guix/machines.scm} file typically looks like this:
@noindent
In the example above we specify a list of two build machines, one for
the @code{x86_64} architecture and one for the @code{aarch64}
architecture.
the @code{x86_64} and @code{i686} architectures and one for the
@code{aarch64} architecture.
In fact, this file is---not surprisingly!---a Scheme file that is
evaluated when the @code{offload} hook is started. Its return value
@ -1096,8 +1098,9 @@ builds. The important fields are:
@item name
The host name of the remote machine.
@item system
The system type of the remote machine---e.g., @code{"x86_64-linux"}.
@item systems
The system types the remote machine supports---e.g., @code{(list
"x86_64-linux" "i686-linux")}.
@item user
The user account to use when connecting to the remote machine over SSH.
@ -25025,7 +25028,7 @@ Maximum number of backup files to keep.
Defaults to @samp{3}
@end deftypevr
@node Transparent Emulation with QEMU
@subsubheading Transparent Emulation with QEMU
@cindex emulation

View File

@ -1,6 +1,7 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2014, 2015, 2016, 2017, 2018, 2019, 2020 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2017 Ricardo Wurmus <rekado@elephly.net>
;;; Copyright © 2020 Maxim Cournoyer <maxim.cournoyer@gmail.com>
;;;
;;; This file is part of GNU Guix.
;;;
@ -66,14 +67,16 @@
;;;
;;; Code:
(define-record-type* <build-machine>
build-machine make-build-machine
build-machine?
(name build-machine-name) ; string
(port build-machine-port ; number
(default 22))
(system build-machine-system) ; string
(systems %build-machine-systems ; list of strings
(default #f)) ; drop default after system is removed
(system %build-machine-system ; deprecated
(default #f))
(user build-machine-user) ; string
(private-key build-machine-private-key ; file name
(default (user-openssh-private-key)))
@ -91,6 +94,19 @@
(features build-machine-features ; list of strings
(default '())))
;;; Deprecated.
(define (build-machine-system machine)
(warning (G_ "The 'system' field is deprecated, \
please use 'systems' instead.~%"))
(%build-machine-system machine))
;;; TODO: Remove after the deprecated 'system' field is removed.
(define (build-machine-systems machine)
(or (%build-machine-systems machine)
(list (build-machine-system machine))
(leave (G_ "The build-machine object lacks a value for its 'systems'
field."))))
(define-record-type* <build-requirements>
build-requirements make-build-requirements
build-requirements?
@ -359,8 +375,8 @@ of free disk space on '~a'~%")
(define (machine-matches? machine requirements)
"Return #t if MACHINE matches REQUIREMENTS."
(and (string=? (build-requirements-system requirements)
(build-machine-system machine))
(and (member (build-requirements-system requirements)
(build-machine-systems machine))
(lset<= string=?
(build-requirements-features requirements)
(build-machine-features machine))))

71
tests/offload.scm 100644
View File

@ -0,0 +1,71 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2020 Maxim Cournoyer <maxim.cournoyer@gmail.com>
;;;
;;; This file is part of GNU Guix.
;;;
;;; GNU Guix is free software; you can redistribute it and/or modify it
;;; under the terms of the GNU General Public License as published by
;;; the Free Software Foundation; either version 3 of the License, or (at
;;; your option) any later version.
;;;
;;; GNU Guix is distributed in the hope that it will be useful, but
;;; WITHOUT ANY WARRANTY; without even the implied warranty of
;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
;;; GNU General Public License for more details.
;;;
;;; You should have received a copy of the GNU General Public License
;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
(define-module (tests offload)
#:use-module (guix scripts offload)
#:use-module (srfi srfi-64))
(test-begin "offload")
(define-syntax-rule (expose-internal-definitions s1 s2 ...)
(begin
(define s1 (@@ (guix scripts offload) s1))
(define s2 (@@ (guix scripts offload) s2)) ...))
(expose-internal-definitions machine-matches?
build-requirements-system
build-requirements-features
build-machine-system
build-machine-systems
%build-machine-system
%build-machine-systems
build-machine-features)
(define (deprecated-build-machine system)
(build-machine
(name "m1")
(user "dummy")
(host-key "some-key")
(system system)))
(define (new-build-machine systems)
(build-machine
(name "m1")
(user "dummy")
(host-key "some-key")
(systems systems)))
;;; Test that deprecated build-machine definitions still work.
(test-assert (machine-matches? (deprecated-build-machine "i686-linux")
(build-requirements
(system "i686-linux"))))
(test-assert (machine-matches? (new-build-machine '("i686-linux"))
(build-requirements
(system "i686-linux"))))
;;; A build machine can act as more than one system type, thanks to QEMU
;;; emulation.
(test-assert (machine-matches? (new-build-machine '("armhf-linux"
"aarch64-linux"
"i686-linux"
"x86_64-linux"))
(build-requirements
(system "armhf-linux"))))