me
/
guix
Archived
1
0
Fork 0
Code

doc: Add chapter on containers to Cookbook. 

* doc/guix-cookbook.texi (Containers): New chapter.
Ricardo Wurmus 2022-10-13 00:23:39 +02:00
parent 9c97647446
commit 56915dc275
No known key found for this signature in database
GPG Key ID: 197A5888235FACAC
1 changed files with 401 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

402
doc/guix-cookbook.texi
View File

@ -11,7 +11,7 @@
@set SUBSTITUTE-TOR-URL https://4zwzi66wwdaalbhgnix55ea3ab4pvvw66ll2ow53kjub6se4q2bclcyd.onion
@copying
Copyright @copyright{} 2019 Ricardo Wurmus@*
Copyright @copyright{} 2019, 2022 Ricardo Wurmus@*
Copyright @copyright{} 2019 Efraim Flashner@*
Copyright @copyright{} 2019 Pierre Neidhardt@*
Copyright @copyright{} 2020 Oleg Pykhalov@*
@ -71,6 +71,7 @@ Weblate} (@pxref{Translating Guix,,, guix, GNU Guix reference manual}).
* Scheme tutorials:: Meet your new favorite language!
* Packaging:: Packaging tutorials
* System Configuration:: Customizing the GNU System
* Containers:: Isolated environments and nested systems
* Advanced package management:: Power to the users!
* Environment management:: Control environment
@ -2454,6 +2455,405 @@ ngx.say(stdout)
#$(local-file "index.lua"))))))))))))))
@end lisp
@c *********************************************************************
@node Containers
@chapter Containers
The kernel Linux provides a number of shared facilities that are
available to processes in the system. These facilities include a shared
view on the file system, other processes, network devices, user and
group identities, and a few others. Since Linux 3.19 a user can choose
to @emph{unshare} some of these shared facilities for selected
processes, providing them (and their child processes) with a different
view on the system.
A process with an unshared @code{mount} namespace, for example, has its
own view on the file system --- it will only be able to see directories
that have been explicitly bound in its mount namespace. A process with
its own @code{proc} namespace will consider itself to be the only
process running on the system, running as PID 1.
Guix uses these kernel features to provide fully isolated environments
and even complete Guix System containers, lightweight virtual machines
that share the host system's kernel. This feature comes in especially
handy when using Guix on a foreign distribution to prevent interference
from foreign libraries or configuration files that are available
system-wide.
@menu
* Guix Containers:: Perfectly isolated environments
* Guix System Containers:: A system inside your system
@end menu
@node Guix Containers
@section Guix Containers
The easiest way to get started is to use @command{guix shell} with the
@option{--container} option. @xref{Invoking guix shell,,, guix, GNU
Guix Reference Manual} for a reference of valid options.
The following snippet spawns a minimal shell process with most
namespaces unshared from the system. The current working directory is
visible to the process, but anything else on the file system is
unavailable. This extreme isolation can be very useful when you want to
rule out any sort of interference from environment variables, globally
installed libraries, or configuration files.
@example
guix shell --container
@end example
It is a bleak environment, barren, desolate. You will find that not
even the GNU coreutils are available here, so to explore this deserted
wasteland you need to use built-in shell commands. Even the usually
gigantic @file{/gnu/store} directory is reduced to a faint shadow of
itself.
@example sh
$ echo /gnu/store/*
/gnu/store/@dots{}-gcc-10.3.0-lib
/gnu/store/@dots{}-glibc-2.33
/gnu/store/@dots{}-bash-static-5.1.8
/gnu/store/@dots{}-ncurses-6.2.20210619
/gnu/store/@dots{}-bash-5.1.8
/gnu/store/@dots{}-profile
/gnu/store/@dots{}-readline-8.1.1
@end example
@cindex exiting a container
There isn't much you can do in an environment like this other than
exiting it. You can use @key{^D} or @command{exit} to terminate this
limited shell environment.
@cindex exposing directories, container
@cindex sharing directories, container
@cindex mapping locations, container
You can make other directories available inside of the container
environment; use @option{--expose=DIRECTORY} to bind-mount the given
directory as a read-only location inside the container, or use
@option{--share=DIRECTORY} to make the location writable. With an
additional mapping argument after the directory name you can control the
name of the directory inside the container. In the following example we
map @file{/etc} on the host system to @file{/the/host/etc} inside a
container in which the GNU coreutils are installed.
@example sh
$ guix shell --container --share=/etc=/the/host/etc coreutils
$ ls /the/host/etc
@end example
Similarly, you can prevent the current working directory from being
mapped into the container with the @option{--no-cwd} option. Another
good idea is to create a dedicated directory that will serve as the
container's home directory, and spawn the container shell from that
directory.
@cindex hide system libraries, container
@cindex avoid ABI mismatch, container
On a foreign system a container environment can be used to compile
software that cannot possibly be linked with system libraries or with
the system's compiler toolchain. A common use-case in a research
context is to install packages from within an R session. Outside of a
container environment there is a good chance that the foreign compiler
toolchain and incompatible system libraries are found first, resulting
in incompatible binaries that cannot be used by R. In a container shell
this problem disappears, as system libraries and executables simply
aren't available due to the unshared @code{mount} namespace.
Let's take a comprehensive manifest providing a comfortable development
environment for use with R:
@lisp
(specifications->manifest
(list "r-minimal"
;; base packages
"bash-minimal"
"glibc-locales"
"nss-certs"
;; Common command line tools lest the container is too empty.
"coreutils"
"grep"
"which"
"wget"
"sed"
;; R markdown tools
"pandoc"
;; Toolchain and common libraries for "install.packages"
"gcc-toolchain@@10"
"gfortran-toolchain"
"gawk"
"tar"
"gzip"
"unzip"
"make"
"cmake"
"pkg-config"
"cairo"
"libxt"
"openssl"
"curl"
"zlib"))
@end lisp
Let's use this to run R inside a container environment. For convenience
we share the @code{net} namespace to use the host system's network
interfaces. Now we can build R packages from source the traditional way
without having to worry about ABI mismatch or incompatibilities.
@example sh
$ guix shell --container --network --manifest=manifest.scm -- R
R version 4.2.1 (2022-06-23) -- "Funny-Looking Kid"
Copyright (C) 2022 The R Foundation for Statistical Computing
@dots{}
> e <- Sys.getenv("GUIX_ENVIRONMENT")
> Sys.setenv(GIT_SSL_CAINFO=paste0(e, "/etc/ssl/certs/ca-certificates.crt"))
> Sys.setenv(SSL_CERT_FILE=paste0(e, "/etc/ssl/certs/ca-certificates.crt"))
> Sys.setenv(SSL_CERT_DIR=paste0(e, "/etc/ssl/certs"))
> install.packages("Cairo", lib=paste0(getwd()))
@dots{}
* installing *source* package 'Cairo' ...
@dots{}
* DONE (Cairo)
The downloaded source packages are in
'/tmp/RtmpCuwdwM/downloaded_packages'
> library("Cairo", lib=getwd())
> # success!
@end example
Using container shells is fun, but they can become a little cumbersome
when you want to go beyond just a single interactive process. Some
tasks become a lot easier when they sit on the rock solid foundation of
a proper Guix System and its rich set of system services. The next
section shows you how to launch a complete Guix System inside of a
container.
@node Guix System Containers
@section Guix System Containers
The Guix System provides a wide array of interconnected system services
that are configured declaratively to form a dependable stateless GNU
System foundation for whatever tasks you throw at it. Even when using
Guix on a foreign distribution you can benefit from the design of Guix
System by running a system instance as a container. Using the same
kernel features of unshared namespaces mentioned in the previous
section, the resulting Guix System instance is isolated from the host
system and only shares file system locations that you explicitly
declare.
A Guix System container differs from the shell process created by
@command{guix shell --container} in a number of important ways. While
in a container shell the containerized process is a Bash shell process,
a Guix System container runs the Shepherd as PID 1. In a system
container all system services (@pxref{Services,,, guix, GNU Guix
Reference Manual}) are set up just as they would be on a Guix System in
a virtual machine or on bare metal---this includes daemons managed by
the GNU@tie{}Shepherd (@pxref{Shepherd Services,,, guix, GNU Guix
Reference Manual}) as well as other kinds of extensions to the operating
system (@pxref{Service Composition,,, guix, GNU Guix Reference Manual}).
The perceived increase in complexity of running a Guix System container
is easily justified when dealing with more complex applications that
have higher or just more rigid requirements on their execution
contexts---configuration files, dedicated user accounts, directories for
caches or log files, etc. In Guix System the demands of this kind of
software are satisfied through the deployment of system services.
@node A Database Container
@subsection A Database Container
A good example might be a PostgreSQL database server. Much of the
complexity of setting up such a database server is encapsulated in this
deceptively short service declaration:
@lisp
(service postgresql-service-type
(postgresql-configuration
(postgresql postgresql-14)))
@end lisp
A complete operating system declaration for use with a Guix System
container would look something like this:
@lisp
(use-modules (gnu))
(use-package-modules databases)
(use-service-modules databases)
(operating-system
(host-name "container")
(timezone "Europe/Berlin")
(file-systems (cons (file-system
(device (file-system-label "does-not-matter"))
(mount-point "/")
(type "ext4"))
%base-file-systems))
(bootloader (bootloader-configuration
(bootloader grub-bootloader)
(targets '("/dev/sdX"))))
(services
(cons* (service postgresql-service-type
(postgresql-configuration
(postgresql postgresql-14)
(config-file
(postgresql-config-file
(log-destination "stderr")
(hba-file
(plain-file "pg_hba.conf"
"\
local all all trust
host all all 10.0.0.1/32 trust"))
(extra-config
'(("listen_addresses" "*")
("log_directory" "/var/log/postgresql")))))))
(service postgresql-role-service-type
(postgresql-role-configuration
(roles
(list (postgresql-role
(name "test")
(create-database? #t))))))
%base-services)))
@end lisp
With @code{postgresql-role-service-type} we define a role ``test'' and
create a matching database, so that we can test right away without any
further manual setup. The @code{postgresql-config-file} settings allow
a client from IP address 10.0.0.1 to connect without requiring
authentication---a bad idea in production systems, but convenient for
this example.
Let's build a script that will launch an instance of this Guix System as
a container. Write the @code{operating-system} declaration above to a
file @file{os.scm} and then use @command{guix system container} to build
the launcher. (@pxref{Invoking guix system,,, guix, GNU Guix Reference
Manual}).
@example
$ guix system container os.scm
The following derivations will be built:
/gnu/store/@dots{}-run-container.drv
@dots{}
building /gnu/store/@dots{}-run-container.drv...
/gnu/store/@dots{}-run-container
@end example
Now that we have a launcher script we can run it to spawn the new system
with a running PostgreSQL service. Note that due to some as yet
unresolved limitations we need to run the launcher as the root user, for
example with @command{sudo}.
@example
$ sudo /gnu/store/@dots{}-run-container
system container is running as PID 5983
@dots{}
@end example
Background the process with @key{Ctrl-z} followed by @command{bg}. Note
the process ID in the output; we will need it to connect to the
container later. You know what? Let's try attaching to the container
right now. We will use @command{nsenter}, a tool provided by the
@code{util-linux} package:
@example
$ guix shell util-linux
$ sudo nsenter -a -t 5983
root@@container /# pgrep -a postgres
49 /gnu/store/@dots{}-postgresql-14.4/bin/postgres -D /var/lib/postgresql/data --config-file=/gnu/store/@dots{}-postgresql.conf -p 5432
51 postgres: checkpointer
52 postgres: background writer
53 postgres: walwriter
54 postgres: autovacuum launcher
55 postgres: stats collector
56 postgres: logical replication launcher
root@@container /# exit
@end example
The PostgreSQL service is running in the container!
@node Container Networking
@subsection Container Networking
@cindex container networking
What good is a Guix System running a PostgreSQL database service as a
container when we can only talk to it with processes originating in the
container? It would be much better if we could talk to the database
over the network.
The easiest way to do this is to create a pair of connected virtual
Ethernet devices (known as @code{veth}). We move one of the devices
(@code{ceth-test}) into the @code{net} namespace of the container and
leave the other end (@code{veth-test}) of the connection on the host
system.
@example
pid=5983
ns="guix-test"
host="veth-test"
client="ceth-test"
# Attach the new net namespace "guix-test" to the container PID.
sudo ip netns attach $ns $pid
# Create the pair of devices
sudo ip link add $host type veth peer name $client
# Move the client device into the container's net namespace
sudo ip link set $client netns $ns
@end example
Then we configure the host side:
@example
sudo ip link set $host up
sudo ip addr add 10.0.0.1/24 dev $host
@end example
@dots{}and then we configure the client side:
@example
sudo ip netns exec $ns ip link set lo up
sudo ip netns exec $ns ip link set $client up
sudo ip netns exec $ns ip addr add 10.0.0.2/24 dev $client
@end example
At this point the host can reach the container at IP address 10.0.0.2,
and the container can reach the host at IP 10.0.0.1. This is all we
need to talk to the database server inside the container from the host
system on the outside.
@example
$ psql -h 10.0.0.2 -U test
psql (14.4)
Type "help" for help.
test=> CREATE TABLE hello (who TEXT NOT NULL);
CREATE TABLE
test=> INSERT INTO hello (who) VALUES ('world');
INSERT 0 1
test=> SELECT * FROM hello;
who
-------
world
(1 row)
@end example
Now that we're done with this little demonstration let's clean up:
@example
sudo kill $pid
sudo ip netns del $ns
sudo ip link del $host
@end example
@c *********************************************************************
@node Advanced package management
@chapter Advanced package management