Add til on git notes on CI

1 files changed, 118 insertions, 0 deletions

diff --git a/_tils/2020-11-30-storing-ci-data-on-git-notes.md b/_tils/2020-11-30-storing-ci-data-on-git-notes.md
new file mode 100644
index 0000000..8ffeab8
--- /dev/null
+++ b/_tils/2020-11-30-storing-ci-data-on-git-notes.md
@@ -0,0 +1,118 @@
+---
+
+title: Storing CI data on Git notes
+
+date: 2020-11-30
+
+layout: post
+
+lang: en
+
+ref: storing-ci-data-on-git-notes
+
+---
+
+Extending the bare bones CI server I've [talked about before][previous-article],
+divoplade on Freenode suggested storing CI artifacts on [Git notes][git-notes],
+such as tarballs, binaries, logs, *etc*.
+
+I've written a small script that will put log files and CI job data on Git notes,
+and make it visible on the porcelain log. It is a simple extension of the
+previous article:
+
+```shell
+#!/usr/bin/env bash
+set -Eeuo pipefail
+set -x
+
+PREFIX='/srv/ci/vps'
+mkdir -p "$PREFIX"
+read -r _ SHA _ # oldrev newrev refname
+FILENAME="$(date -Is)-$SHA.log"
+LOGFILE="$PREFIX/$FILENAME"
+exec &> >(tee -a "$LOGFILE")
+
+echo "Starting CI job at: $(date -Is)"
+
+finish() {
+  STATUS="$?"
+  printf "\n\n>>> exit status was %s\n" "$STATUS"
+  echo "Finishing CI job at: $(date -Is)"
+  popd
+  NOTE=$(cat <<EOF
+See CI logs with:
+  git notes --ref=refs/notes/ci-logs show $SHA
+  git notes --ref=refs/notes/ci-data show $SHA
+EOF
+)
+  git notes --ref=refs/notes/ci-data add -f -m "$STATUS $FILENAME"
+  git notes --ref=refs/notes/ci-logs add -f -F "$LOGFILE"
+  git notes add -f -m "$NOTE"
+  printf "\n\n>>> CI logs added as Git note."
+}
+trap finish EXIT
+
+unset GIT_DIR
+CLONE="$(mktemp -d)"
+git clone . "$CLONE"
+pushd "$CLONE"
+git config --global user.email git@euandre.org
+git config --global user.name 'EuAndreh CI'
+
+./container make check site
+./container make publish
+```
+
+The important part is in the `finish()` function:
+- #25 stores the exit status and the generated filename separated by spaces;
+- #26 adds the log file in a note using the `refs/notes/ci-logs` ref;
+- #27 it adds a note to the commit saying how to see the logs.
+
+A commit now has an attached note, and shows it whenever you look at it:
+
+```diff
+$ git show 930ba1888f49f11e52a4a715438cd9f5f413dd9c
+commit 930ba1888f49f11e52a4a715438cd9f5f413dd9c (oldvps/master)
+Author: EuAndreh <eu@euandre.org>
+Date:   Mon Nov 30 01:11:38 2020 -0300
+
+    vps.scm: Uncomment mcron job time marker
+
+Notes:
+    See CI logs with:
+      git notes --ref=refs/notes/ci-logs show 930ba1888f49f11e52a4a715438cd9f5f413dd9c
+      git notes --ref=refs/notes/ci-data show 930ba1888f49f11e52a4a715438cd9f5f413dd9c
+
+diff --git a/sync/vps.scm b/sync/vps.scm
+index 3f6ca69..02b9cc6 100644
+--- a/sync/vps.scm
++++ b/sync/vps.scm
+@@ -280,7 +280,7 @@ pki " mail-domain " key  \"" (tls-priv-for mail-domain) "\"")))
+              tls-prefixes)))
+
+ (define generate-ci-index-html-job
+-  #~(job "* * * * *" ;; "*/5 * * * *"
++  #~(job "*/5 * * * *"
+          #$(program-file
+             "generate-ci-index-html.scm"
+             (with-imported-modules (modules:source-module-closure
+```
+
+Other tools such as [cgit][cgit] will also show notes on the web interface:
+[https://git.euandreh.xyz/vps/commit?id=930ba1888f49f11e52a4a715438cd9f5f413dd9c](https://git.euandreh.xyz/vps/commit?id=930ba1888f49f11e52a4a715438cd9f5f413dd9c)
+
+You can go even further: since cgit can serve raw blob directly, you can even
+serve such artifacts (log files, release artifacts, binaries) from cgit itself:
+
+```shell
+$ SHA="$(git notes --ref=refs/notes/ci-logs list 930ba1888f49f11e52a4a715438cd9f5f413dd9c)"
+$ echo "https://git.euandreh.xyz/vps/blob?id=$SHA"
+https://git.euandreh.xyz/vps/blob?id=b3a6438a0c7a47864c54c61359b6ef50e864dbff
+```
+
+And like that you'll have cgit serving the artifacts for you:
+[https://git.euandreh.xyz/vps/blob?id=b3a6438a0c7a47864c54c61359b6ef50e864dbff](https://git.euandreh.xyz/vps/blob?id=b3a6438a0c7a47864c54c61359b6ef50e864dbff)
+
+[previous-article]: {% link _tils/2020-11-12-diy-bare-bones-ci-server-with-bash-and-nix.md %}
+[git-notes]: https://git-scm.com/docs/git-notes
+[cgit]: https://git.zx2c4.com/cgit/about/