Fixes <http://bugs.gnu.org/15890>.
* nix/libstore/globals.hh (Settings) Add clientUid and clientGid.
* nix/nix-daemon/nix-daemon.cc (daemonLoop] Store UID and GID of the
caller in settings.
* nix/libstore/build.cc (_chown): New function.
(DerivationGoal::deleteTmpDir): Use it, change ownership of build
directory if it is kept and the new owner is not root.
Before that we'd have STDERR_WRITE round trips for very small amounts of
data, ranging from a few bytes for the metadata of nars to the size of
one file being exported.
With this change, something like:
guix archive --export /gnu/store/5rrsbaghh5ix1vjcicsl60gsxilhjnf2-coreutils-8.25 | dd of=/dev/null
reports a throughput of 35 MB/s instead of 25 MB/s before.
* nix/nix-daemon/nix-daemon.cc (TunnelSink): Inherit from 'BufferedSink'
rather than 'Sink'. Rename 'operator ()' to 'write'.
(performOp) <wopExportPath>: Add 'sink.flush' call.
* .gitignore: add etc/guix-publish.conf and /etc/guix-publish.service.
* etc/guix-publish.conf.in: New file.
* etc/guix-publish.service.in: New file.
* nix/local.mk (etc/guix-%.service, etc/guix-%.conf): Generalized former
build-rules for by using patterns.
(nodist_systemdservice_DATA): Add etc/guix-publish.service, update
comment.
(nodist_upstartjob_DATA): Add etc/guix-publish.conf, update comment.
* doc/guix.texi (Invoking guix publish): Add description for enabling
"guix publish" on host distros using the new files.
This solves a problem whereby if /gnu/store/.links had enough entries,
ext4's directory index would be full, leading to link(2) returning
ENOSPC.
* nix/libstore/optimise-store.cc (LocalStore::optimisePath_): Upon
ENOSPC from link(2), print a message and return instead of throwing a
'SysError'.
In particular, this eliminates a bunch of boilerplate code.
Also integrates these Nix commits:
80da7a6 Probably fix SQLITE_BUSY errors
37a337b throwSQLiteError(): Check for SIGINT so we don't loop forever
Co-authored-by: Ludovic Courtès <ludo@gnu.org>
Reported by Federico Beffa <beffa@ieee.org>
at <https://lists.gnu.org/archive/html/guix-devel/2016-05/msg00928.html>.
* nix/libstore/local-store.cc (LocalStore::querySubstitutablePaths)
(LocalStore::querySubstitutablePathInfos): Return when
'settings.useSubstitutes' is false.
* tests/store.scm ("references/substitutes missing reference info"):
Make sure to return #f on failure.
* tests/store.scm ("substitutable-path-info when substitutes are turned off"):
("substitutable-paths when substitutes are turned off"): New tests.
Previously files in the Nix store were owned by root or by nixbld,
depending on whether they were created by a substituter or by a
builder. This doesn't matter much, but causes spurious diffoscope
differences. So use root everywhere.
E.g.
$ nix-build pkgs/stdenv/linux/ -A stage1.pkgs.perl --check
nix-store: src/libstore/build.cc:1323: void nix::DerivationGoal::tryToBuild(): Assertion `buildMode != bmCheck || validPaths.size() == drv->outputs.size()' failed.
when perl.out exists but perl.man doesn't. The fix is to only check
the outputs that exist. Note that "nix-build -A stage1.pkgs.all
--check" will still give a (proper) error in this case.
If repair found a corrupted/missing path that depended on a
multiple-output derivation, and some of the outputs of the latter were
not present, it failed with a message like
error: path ‘/nix/store/cnfn9d5fjys1y93cz9shld2xwaibd7nn-bash-4.3-p42-doc’ is not valid
Co-authored-by: Ludovic Courtès <ludo@gnu.org>
Otherwise, since the call to write a "d" character to the lock file
can fail with ENOSPC, we can get an unhandled exception resulting in a
call to terminate().
* doc/local.mk: Use "%D%" for the directory of the fragment relative to
the base 'Makefile.am'.
* emacs/local.mk: Likewise.
* gnu/local.mk: Likewise.
* nix/local.mk: Likewise.
This follows a convention used by some other GNU packages like Autoconf,
Bison, Coreutils, and Gnulib.
* doc.am: Rename to ...
* doc/local.mk: ... this.
* emacs.am: Rename to ...
* emacs/local.mk: ... this.
* gnu-system.am: Rename to ...
* gnu/local.mk: ... this.
* daemon.am: Rename to ...
* nix/local.mk: ... this.
* Makefile.am: Adapt to them.
* doc/guix.texi (Porting to a New Platform): Adapt documentation.
* guix/config.scm.in (%state-directory, %config-directory): Adapt comments.
* emacs/guix-config.el.in (guix-config-state-directory): Likewise.
In Nix itself, the new 'BuildResult' type is returned by the new
'buildDerivation' method, which we don't have and need.
* nix/libstore/build.cc (Goal)[cancel]: Remove.
[timeOut]: New pure virtual method.
(DerivationGoal)[result]: New field.
[cancel]: Remove.
[timedOut, getResult, done]: New methods.
(DerivationGoal::cancel): Remove.
(DerivationGoal::timedOut): New method.
(DerivationGoal::haveDerivation): Call 'done' instead of 'amDone'.
(DerivationGoal::outputsSubstituted): Ditto.
(DerivationGoal::inputsRealised): Ditto.
(DerivationGoal::buildDone): Ditto.
(DerivationGoal::handleChildOutput): Call 'timedOut' instead of
'cancel'.
(DerivationGoal::done): New method.
(SubstitutionGoal)[cancel]: Remove.
[timedOut]: New method.
(SubstitutionGoal::cancel): Remove.
(SubstitutionGoal::timedOut): New method.
(Worker::waitForInput): Use it.
* nix/libstore/store-api.hh (BuildResult): New struct.
Co-authored-by: Ludovic Courtès <ludo@gnu.org>
Rather than using $<host-TMPDIR>/nix-build-<drvname>-<number>, the
temporary directory is now always /tmp/nix-build-<drvname>-0. This
improves bitwise-exact reproducibility for builds that store $TMPDIR
in their build output. (Of course, those should still be fixed...)
* nix/libstore/build.cc (DerivationGoal)[tmpDirInSandbox]: New field.
(DerivationGoal::startBuilder): Initialize 'useChroot' earlier. Compute
'tmpDirInSandbox', and use it when populating 'dirsInChroot'.
* doc/guix.texi (Build Environment Setup): Document it.
Co-authored-by: Ludovic Courtès <ludo@gnu.org>
This makes it easy to detect non-deterministic builds.
* nix/libstore/build.cc (DerivationGoal): Remove 'InodesSeen'; add
'curRound', 'nrRound', and 'prevInfos'.
(DerivationGoal::inputsRealised): Initialize 'nrRound'.
(NotDeterministic): New error type.
(DerivationGoal::buildDone): Check whether we need to repeat.
(DerivationGoal::startBuilder): Adjust message.
(DerivationGoal::registerOutputs): Check whether we get the same result.
* nix/libstore/globals.cc (Settings::get(const string & name, int def)):
New method.
* nix/libstore/globals.hh (Settings): Add it.
* nix/libstore/store-api.hh (ValidPathInfo): Add operator ==.
* nix/nix-daemon/nix-daemon.cc (performOp): Allow "build-repeat" for
"untrusted" users.
Co-authored-by: Ludovic Courtès <ludo@gnu.org>
* nix/libstore/worker-protocol.hh (PROTOCOL_VERSION): Bump to 0x10f.
* nix/libstore/remote-store.cc (RemoteStore::buildPaths): Send the
BUILDMODE when the daemon supports it. Reject invalid values of
BUILDMODE for old daemons.
* nix/nix-daemon/nix-daemon.cc (performOp) <wopBuildPaths>: Read the
build mode when the client supports it.
If automatic store optimisation is enabled, and a hard-linked file in
the store gets corrupted, then the corresponding .links entry will
also be corrupted. In that case, trying to repair with --repair or
--repair-path won't work, because the new "good" file will be replaced
by a hard link to the corrupted file. We can catch most of these cases
by doing a sanity-check on the file sizes.
Fixes https://github.com/NixOS/nixpkgs/issues/9504.
Note that this means we may have a non-functional /bin/sh in the
chroot while rebuilding Bash or one of its dependencies. Ideally those
packages don't rely on /bin/sh though.
This reinstates commit aa0f8409, which was inadvertently undone in commit
322eeb87.
Running 'guix archive --import' as root would have let corrupt or unauthentic
store items through.
Reported by Eric Hanchrow <eric.hanchrow@gmail.com>
at <http://bugs.gnu.org/21354>.
* nix/nix-daemon/nix-daemon.cc (performOp) <wopImportPaths>: Pass true as the
first argument to 'importPaths'.
This commit had been inadvertently reverted in 322eeb8.
* nix/libstore/local-store.cc (LocalStore::getLineFromSubstituter):
Flush when the line contains '\r'.
This is a squashed commit of the following:
commit 0dccab9f417b406f5d4aedc81900fc7b2f16c9f6
Author: Eelco Dolstra <eelco.dolstra@logicblox.com>
Date: Thu Jul 2 00:30:16 2015 +0200
Typo
commit 2cd28517b13524c242c7758783b0b2d8250fdded
Author: Ludovic Courtès <ludo@gnu.org>
Date: Wed Jul 1 14:56:34 2015 +0200
Preserve supplementary groups of build users
The following patch is an attempt to address this bug (see
<http://bugs.gnu.org/18994>) by preserving the supplementary groups of
build users in the build environment.
In practice, I would expect that supplementary groups would contain only
one or two groups: the build users group, and possibly the “kvm” group.
[Changed &at(0) to data() and removed tabs - Eelco]
commit 6e38685ef65284093df79ebe7378bac33b0e7e5d
Author: Eelco Dolstra <eelco.dolstra@logicblox.com>
Date: Tue Jun 30 21:41:26 2015 +0200
GC: Handle ENOSPC creating/moving to the trash directory
Issue #564.
commit 5e0a9ae2e25a1016389f4893a6ed6682aadcf51d
Author: Eelco Dolstra <eelco.dolstra@logicblox.com>
Date: Mon Jun 22 15:54:55 2015 +0200
Use posix_fallocate to create /nix/var/nix/db/reserved
commit 4e5ab98d6d14f8b0e3bd1d77b2f4f2354e7a49a8
Author: Eelco Dolstra <eelco.dolstra@logicblox.com>
Date: Mon Jun 22 15:47:40 2015 +0200
Make /nix/var/nix/db/reserved bigger
Issue #564.
commit 60bda60fc06135aa97a93301b1a9e2270768f5b3
Author: Eelco Dolstra <eelco.dolstra@logicblox.com>
Date: Wed Jun 10 16:17:06 2015 +0200
Export outputPaths function
This is useful for the new hydra-queue-runner.
commit 5dfea34048aa8541f20aeb2fbcd163561b609a49
Author: Eelco Dolstra <eelco.dolstra@logicblox.com>
Date: Thu Jul 2 22:51:33 2015 +0200
Use std::vector::data()
commit 2459458bc8257734ca78cb7a2db3df20bd730ec0
Author: Eelco Dolstra <eelco.dolstra@logicblox.com>
Date: Thu Jun 4 16:04:41 2015 +0200
Allow substitutes for builds that have preferLocalBuild set
Not substituting builds with "preferLocalBuild = true" was a bad idea,
because it didn't take the cost of dependencies into account. For
instance, if we can't substitute a fetchgit call, then we have to
download/build git and all its dependencies.
Partially reverts 5558652709f27e8a887580b77b93c705659d7a4b and adds a
new derivation attribute "allowSubstitutes" to specify whether a
derivation may be substituted.
Fixes <http://bugs.gnu.org/19044>.
* nix/guix-register/guix-register.cc (register_validity): Change the (optimize
&& prefix.empty ()) condition to just (optimize). Change settings.nixStore
around the loop. Prepend PREFIX to the argument passed to 'optimisePath'.
* tests/guix-register.sh: Use 'guix-register -p' with duplicate files, and
make sure they are deduplicated on the target. Adjust existing tests.
Ludovic Courtès
Hartmut Goebel
Eelco Dolstra
David Craven
Mathieu Lirzin
Manolis Ragkousis