path: root/_posts/2018-08-01-verifying-npm-ci-reproducibility.md

---
title: Verifying "npm ci" reproducibility
date: 2018-08-01
layout: post
---
When
[npm\@5](https://blog.npmjs.org/post/161081169345/v500) came bringing
[package-locks](https://docs.npmjs.com/files/package-locks) with it, I
was confused about the benefits it provided, since running `npm install`
more than once could resolve all the dependencies again and yield yet
another fresh `package-lock.json` file. The message saying \"you should
add this file to version control\" left me hesitant on what to do[^1].

However the [addition of
`npm ci`](https://blog.npmjs.org/post/171556855892/introducing-npm-ci-for-faster-more-reliable)
filled this gap: it\'s a stricter variation of `npm install` which
guarantees that \"[subsequent installs are able to generate identical
trees](https://docs.npmjs.com/files/package-lock.json)\". But are they
really identical? I could see that I didn\'t have the same problems of
different installation outputs, but I didn\'t know for **sure** if it
was really identical.

Computing the hash of a directory\'s content
--------------------------------------------

I quickly searched for a way to check for the hash signature of an
entire directory tree, but I couldn\'t find one. I\'ve made a poor
man\'s [Merkle tree](https://en.wikipedia.org/wiki/Merkle_tree)
implementation using `sha256sum` and a few piped commands at the
terminal:

``` {.bash .numberLines startFrom=""}
merkle-tree () {
  dirname="${1-.}"
  pushd "$dirname"
  find . -type f              | \
    sort                      | \
    xargs -I{} sha256sum "{}" | \
    sha256sum                 | \
    awk '{print $1}'
  popd
}
```

Going through it line by line:

-   \#1 we define a Bash function called `merkle-tree`;
-   \#2 it accepts a single argument: the directory to compute the
    merkle tree from. If nothing is given, it runs on the current
    directory (`.`);
-   \#3 we go to the directory, so we don\'t get different prefixes in
    `find`\'s output (like `../a/b`);
-   \#4 we get all files from the directory tree. Since we\'re using
    `sha256sum` to compute the hash of the file contents, we need to
    filter out folders from it;
-   \#5 we need to sort the output, since different file systems and
    `find` implementations may return files in different orders;
-   \#6 we use `xargs` to compute the hash of each file individually
    through `sha256sum`. Since a file may contain spaces we need to
    escape it with quotes;
-   \#7 we compute the hash of the combined hashes. Since `sha256sum`
    output is formatted like `<hash> <filename>`, it produces a
    different final hash if a file ever changes name without changing
    it\'s content;
-   \#8 we get the final hash output, excluding the `<filename>` (which
    is `-` in this case, aka `stdin`).

### Positive points:

1.  ignore timestamp: running more than once on different installation
    yields the same hash;
2.  the name of the file is included in the final hash computation.

### Limitations:

1.  it ignores empty folders from the hash computation;
2.  the implementation\'s only goal is to represent using a digest
    whether the content of a given directory is the same or not. Leaf
    presence checking is obviously missing from it.

### Testing locally with sample data

``` {.bash .numberLines startFrom=""}
mkdir /tmp/merkle-tree-test/
cd /tmp/merkle-tree-test/
mkdir -p a/b/ a/c/ d/
echo "one"   > a/b/one.txt
echo "two"   > a/c/two.txt
echo "three" > d/three.txt
merkle-tree . # output is       be343bb01fe00aeb8fef14a3e16b1c3d1dccbf86d7e41b4753e6ccb7dc3a57c3
merkle-tree . # output still is be343bb01fe00aeb8fef14a3e16b1c3d1dccbf86d7e41b4753e6ccb7dc3a57c3
echo "four"  > d/four.txt
merkle-tree . # output is now   b5464b958969ed81815641ace96b33f7fd52c20db71a7fccc45a36b3a2ae4d4c
rm d/four.txt
merkle-tree . # output back to  be343bb01fe00aeb8fef14a3e16b1c3d1dccbf86d7e41b4753e6ccb7dc3a57c3
echo "hidden-five" > a/b/one.txt
merkle-tree . # output changed  471fae0d074947e4955e9ac53e95b56e4bc08d263d89d82003fb58a0ffba66f5
```

It seems to work for this simple test case.

You can try copying and pasting it to verify the hash signatures.

Using `merkle-tree` to check the output of `npm ci`
---------------------------------------------------

*I\'ve done all of the following using Node.js v8.11.3 and npm\@6.1.0.*

In this test case I\'ll take the main repo of
[Lerna](https://lernajs.io/)[^2]:

``` {.bash .numberLines startFrom=""}
cd /tmp/
git clone https://github.com/lerna/lerna.git
cd lerna/
git checkout 57ff865c0839df75dbe1974971d7310f235e1109
npm ci
merkle-tree node_modules/ # outputs 11e218c4ac32fac8a9607a8da644fe870a25c99821167d21b607af45699afafa
rm -rf node_modules/
npm ci
merkle-tree node_modules/ # outputs 11e218c4ac32fac8a9607a8da644fe870a25c99821167d21b607af45699afafa
npm ci      # test if it also works with an existing node_modules/ folder
merkle-tree node_modules/ # outputs 11e218c4ac32fac8a9607a8da644fe870a25c99821167d21b607af45699afafa
```

Good job `npm ci` :)

\#6 and \#9 take some time to run (21 seconds in my machine), but this
specific use case isn\'t performance sensitive. The slowest step is
computing the hash of each individual file.

Conclusion
----------

`npm ci` really \"generates identical trees\".

I\'m not aware of any other existing solution for verifying the hash
signature of a directory. If you know any I\'d [like to
know](mailto:eu@euandre.org).

*Edit*
------

2019/05/22: Fix spelling.

[^1]: The
    [documentation](https://docs.npmjs.com/cli/install#description)
    claims `npm install` is driven by the existing `package-lock.json`,
    but that\' actually [a little bit
    tricky](https://github.com/npm/npm/issues/17979#issuecomment-332701215).

[^2]: Finding a big known repo that actually committed the
    `package-lock.json` file was harder than I expected.