Archived
1
0
Fork 0

derivations: Make 'coalesce-duplicate-inputs' linear in the number of inputs.

Partly fixes <https://issues.guix.gnu.org/49439>.
Reported by Ricardo Wurmus <rekado@elephly.net>.

When running the command:

  guix environment pigx-scrnaseq --search-paths --no-grafts

this change reduces total heap allocations from 1.4GiB to 717MiB (49%)
and wall-clock time from 7.5s to 5.7s (24%).

Without '--no-grafts', heap allocations go from 2.1GiB to 1.4GiB (33%)
and wall-clock time from 12.1s to 10.9s (10%).

* guix/derivations.scm (coalesce-duplicate-inputs): Rewrite using a hash
table to make it O(N) rather than O(N²).
This commit is contained in:
Ludovic Courtès 2021-07-27 17:58:40 +02:00
parent 5dc4fe1004
commit 78daf9e02e
No known key found for this signature in database
GPG key ID: 090B11993D9AEBB5

View file

@ -241,32 +241,29 @@ the store."
"Return a list of inputs, such that when INPUTS contains the same DRV twice, "Return a list of inputs, such that when INPUTS contains the same DRV twice,
they are coalesced, with their sub-derivations merged. This is needed because they are coalesced, with their sub-derivations merged. This is needed because
Nix itself keeps only one of them." Nix itself keeps only one of them."
(define (find pred lst) ;inlinable copy of 'find' (define table
(let loop ((lst lst)) (make-hash-table 25))
(match lst
(() #f)
((head . tail)
(if (pred head) head (loop tail))))))
(fold (lambda (input result) (for-each (lambda (input)
(match input (let* ((drv (derivation-input-path input))
(($ <derivation-input> (= derivation-file-name path) sub-drvs) (sub-drvs (derivation-input-sub-derivations input)))
;; XXX: quadratic (match (hash-get-handle table drv)
(match (find (match-lambda (#f
(($ <derivation-input> (= derivation-file-name p) (hash-set! table drv input))
s) ((and handle (key . ($ <derivation-input> drv sub-drvs2)))
(string=? p path))) ;; Merge DUP with INPUT.
result) (let* ((sub-drvs (delete-duplicates
(#f (append sub-drvs sub-drvs2)))
(cons input result)) (input
((and dup ($ <derivation-input> drv sub-drvs2)) (make-derivation-input drv
;; Merge DUP with INPUT. (sort sub-drvs string<?))))
(let ((sub-drvs (delete-duplicates (set-cdr! handle input))))))
(append sub-drvs sub-drvs2)))) inputs)
(cons (make-derivation-input drv (sort sub-drvs string<?))
(delq dup result)))))))) (hash-fold (lambda (key input lst)
'() (cons input lst))
inputs)) '()
table))
(define* (derivation-prerequisites drv #:optional (cut? (const #f))) (define* (derivation-prerequisites drv #:optional (cut? (const #f)))
"Return the list of derivation-inputs required to build DRV, recursively. "Return the list of derivation-inputs required to build DRV, recursively.