1 files changed, 146 insertions, 0 deletions
diff --git a/_posts/2018-08-01-verifying-npm-ci-reproducibility.md b/_posts/2018-08-01-verifying-npm-ci-reproducibility.md
new file mode 100644
index 0000000..f1fd1dd
--- /dev/null
+++ b/_posts/2018-08-01-verifying-npm-ci-reproducibility.md
@@ -0,0 +1,146 @@
+---
+title: Verifying "npm ci" reproducibility
+date: 2018-08-01
+layout: post
+lang: en
+ref: veryfing-npm-ci-reproducibility
+updated_at: 2019-05-22
+---
+When [npm@5](https://blog.npmjs.org/post/161081169345/v500) came bringing
+[package-locks](https://docs.npmjs.com/files/package-locks) with it, I
+was confused about the benefits it provided, since running `npm install`
+more than once could resolve all the dependencies again and yield yet
+another fresh `package-lock.json` file. The message saying "you should
+add this file to version control" left me hesitant on what to do[^1].
+
+However the [addition of `npm ci`](https://blog.npmjs.org/post/171556855892/introducing-npm-ci-for-faster-more-reliable)
+filled this gap: it's a stricter variation of `npm install` which
+guarantees that "[subsequent installs are able to generate identical trees](https://docs.npmjs.com/files/package-lock.json)". But are they
+really identical? I could see that I didn't have the same problems of
+different installation outputs, but I didn't know for **sure** if it
+was really identical.
+
+## Computing the hash of a directory's content
+
+I quickly searched for a way to check for the hash signature of an
+entire directory tree, but I couldn't find one. I've made a poor
+man's [Merkle tree](https://en.wikipedia.org/wiki/Merkle_tree)
+implementation using `sha256sum` and a few piped commands at the
+terminal:
+
+```bash
+merkle-tree () {
+  dirname="${1-.}"
+  pushd "$dirname"
+  find . -type f              | \
+    sort                      | \
+    xargs -I{} sha256sum "{}" | \
+    sha256sum                 | \
+    awk '{print $1}'
+  popd
+}
+```
+
+Going through it line by line:
+
+- #1 we define a Bash function called `merkle-tree`;
+- #2 it accepts a single argument: the directory to compute the
+  merkle tree from. If nothing is given, it runs on the current
+  directory (`.`);
+- #3 we go to the directory, so we don't get different prefixes in
+  `find`'s output (like `../a/b`);
+- #4 we get all files from the directory tree. Since we're using
+  `sha256sum` to compute the hash of the file contents, we need to
+  filter out folders from it;
+- #5 we need to sort the output, since different file systems and
+  `find` implementations may return files in different orders;
+- #6 we use `xargs` to compute the hash of each file individually
+  through `sha256sum`. Since a file may contain spaces we need to
+  escape it with quotes;
+- #7 we compute the hash of the combined hashes. Since `sha256sum`
+  output is formatted like `<hash> <filename>`, it produces a
+  different final hash if a file ever changes name without changing
+  it's content;
+- #8 we get the final hash output, excluding the `<filename>` (which
+    is `-` in this case, aka `stdin`).
+
+### Positive points:
+
+1.  ignore timestamp: running more than once on different installation
+    yields the same hash;
+2.  the name of the file is included in the final hash computation.
+
+### Limitations:
+
+1.  it ignores empty folders from the hash computation;
+2.  the implementation's only goal is to represent using a digest
+    whether the content of a given directory is the same or not. Leaf
+    presence checking is obviously missing from it.
+
+### Testing locally with sample data
+
+```bash
+mkdir /tmp/merkle-tree-test/
+cd /tmp/merkle-tree-test/
+mkdir -p a/b/ a/c/ d/
+echo "one"   > a/b/one.txt
+echo "two"   > a/c/two.txt
+echo "three" > d/three.txt
+merkle-tree . # output is       be343bb01fe00aeb8fef14a3e16b1c3d1dccbf86d7e41b4753e6ccb7dc3a57c3
+merkle-tree . # output still is be343bb01fe00aeb8fef14a3e16b1c3d1dccbf86d7e41b4753e6ccb7dc3a57c3
+echo "four"  > d/four.txt
+merkle-tree . # output is now   b5464b958969ed81815641ace96b33f7fd52c20db71a7fccc45a36b3a2ae4d4c
+rm d/four.txt
+merkle-tree . # output back to  be343bb01fe00aeb8fef14a3e16b1c3d1dccbf86d7e41b4753e6ccb7dc3a57c3
+echo "hidden-five" > a/b/one.txt
+merkle-tree . # output changed  471fae0d074947e4955e9ac53e95b56e4bc08d263d89d82003fb58a0ffba66f5
+```
+
+It seems to work for this simple test case.
+
+You can try copying and pasting it to verify the hash signatures.
+
+## Using `merkle-tree` to check the output of `npm ci`
+
+*I've done all of the following using Node.js v8.11.3 and npm@6.1.0.*
+
+In this test case I'll take the main repo of
+[Lerna](https://lernajs.io/)[^2]:
+
+```bash
+cd /tmp/
+git clone https://github.com/lerna/lerna.git
+cd lerna/
+git checkout 57ff865c0839df75dbe1974971d7310f235e1109
+npm ci
+merkle-tree node_modules/ # outputs 11e218c4ac32fac8a9607a8da644fe870a25c99821167d21b607af45699afafa
+rm -rf node_modules/
+npm ci
+merkle-tree node_modules/ # outputs 11e218c4ac32fac8a9607a8da644fe870a25c99821167d21b607af45699afafa
+npm ci      # test if it also works with an existing node_modules/ folder
+merkle-tree node_modules/ # outputs 11e218c4ac32fac8a9607a8da644fe870a25c99821167d21b607af45699afafa
+```
+
+Good job `npm ci` :)
+
+#6 and #9 take some time to run (21 seconds in my machine), but this
+specific use case isn't performance sensitive. The slowest step is
+computing the hash of each individual file.
+
+## Conclusion
+
+`npm ci` really "generates identical trees".
+
+I'm not aware of any other existing solution for verifying the hash
+signature of a directory. If you know any I'd [like to know](mailto:eu@euandre.org).
+
+## *Edit*
+
+2019/05/22: Fix spelling.
+
+[^1]: The [documentation](https://docs.npmjs.com/cli/install#description)
+    claims `npm install` is driven by the existing `package-lock.json`,
+    but that's actually [a little bit tricky](https://github.com/npm/npm/issues/17979#issuecomment-332701215).
+
+[^2]: Finding a big known repo that actually committed the
+    `package-lock.json` file was harder than I expected.