diff options
Diffstat (limited to '')
| -rw-r--r-- | _tils/2020-11-30-storing-ci-data-on-git-notes.md | 118 | ||||
| -rw-r--r-- | scripts/spelling/international.txt | 2 |
2 files changed, 120 insertions, 0 deletions
diff --git a/_tils/2020-11-30-storing-ci-data-on-git-notes.md b/_tils/2020-11-30-storing-ci-data-on-git-notes.md new file mode 100644 index 0000000..8ffeab8 --- /dev/null +++ b/_tils/2020-11-30-storing-ci-data-on-git-notes.md @@ -0,0 +1,118 @@ +--- + +title: Storing CI data on Git notes + +date: 2020-11-30 + +layout: post + +lang: en + +ref: storing-ci-data-on-git-notes + +--- + +Extending the bare bones CI server I've [talked about before][previous-article], +divoplade on Freenode suggested storing CI artifacts on [Git notes][git-notes], +such as tarballs, binaries, logs, *etc*. + +I've written a small script that will put log files and CI job data on Git notes, +and make it visible on the porcelain log. It is a simple extension of the +previous article: + +```shell +#!/usr/bin/env bash +set -Eeuo pipefail +set -x + +PREFIX='/srv/ci/vps' +mkdir -p "$PREFIX" +read -r _ SHA _ # oldrev newrev refname +FILENAME="$(date -Is)-$SHA.log" +LOGFILE="$PREFIX/$FILENAME" +exec &> >(tee -a "$LOGFILE") + +echo "Starting CI job at: $(date -Is)" + +finish() { + STATUS="$?" + printf "\n\n>>> exit status was %s\n" "$STATUS" + echo "Finishing CI job at: $(date -Is)" + popd + NOTE=$(cat <<EOF +See CI logs with: + git notes --ref=refs/notes/ci-logs show $SHA + git notes --ref=refs/notes/ci-data show $SHA +EOF +) + git notes --ref=refs/notes/ci-data add -f -m "$STATUS $FILENAME" + git notes --ref=refs/notes/ci-logs add -f -F "$LOGFILE" + git notes add -f -m "$NOTE" + printf "\n\n>>> CI logs added as Git note." +} +trap finish EXIT + +unset GIT_DIR +CLONE="$(mktemp -d)" +git clone . "$CLONE" +pushd "$CLONE" +git config --global user.email git@euandre.org +git config --global user.name 'EuAndreh CI' + +./container make check site +./container make publish +``` + +The important part is in the `finish()` function: +- #25 stores the exit status and the generated filename separated by spaces; +- #26 adds the log file in a note using the `refs/notes/ci-logs` ref; +- #27 it adds a note to the commit saying how to see the logs. + +A commit now has an attached note, and shows it whenever you look at it: + +```diff +$ git show 930ba1888f49f11e52a4a715438cd9f5f413dd9c +commit 930ba1888f49f11e52a4a715438cd9f5f413dd9c (oldvps/master) +Author: EuAndreh <eu@euandre.org> +Date: Mon Nov 30 01:11:38 2020 -0300 + + vps.scm: Uncomment mcron job time marker + +Notes: + See CI logs with: + git notes --ref=refs/notes/ci-logs show 930ba1888f49f11e52a4a715438cd9f5f413dd9c + git notes --ref=refs/notes/ci-data show 930ba1888f49f11e52a4a715438cd9f5f413dd9c + +diff --git a/sync/vps.scm b/sync/vps.scm +index 3f6ca69..02b9cc6 100644 +--- a/sync/vps.scm ++++ b/sync/vps.scm +@@ -280,7 +280,7 @@ pki " mail-domain " key \"" (tls-priv-for mail-domain) "\""))) + tls-prefixes))) + + (define generate-ci-index-html-job +- #~(job "* * * * *" ;; "*/5 * * * *" ++ #~(job "*/5 * * * *" + #$(program-file + "generate-ci-index-html.scm" + (with-imported-modules (modules:source-module-closure +``` + +Other tools such as [cgit][cgit] will also show notes on the web interface: +[https://git.euandreh.xyz/vps/commit?id=930ba1888f49f11e52a4a715438cd9f5f413dd9c](https://git.euandreh.xyz/vps/commit?id=930ba1888f49f11e52a4a715438cd9f5f413dd9c) + +You can go even further: since cgit can serve raw blob directly, you can even +serve such artifacts (log files, release artifacts, binaries) from cgit itself: + +```shell +$ SHA="$(git notes --ref=refs/notes/ci-logs list 930ba1888f49f11e52a4a715438cd9f5f413dd9c)" +$ echo "https://git.euandreh.xyz/vps/blob?id=$SHA" +https://git.euandreh.xyz/vps/blob?id=b3a6438a0c7a47864c54c61359b6ef50e864dbff +``` + +And like that you'll have cgit serving the artifacts for you: +[https://git.euandreh.xyz/vps/blob?id=b3a6438a0c7a47864c54c61359b6ef50e864dbff](https://git.euandreh.xyz/vps/blob?id=b3a6438a0c7a47864c54c61359b6ef50e864dbff) + +[previous-article]: {% link _tils/2020-11-12-diy-bare-bones-ci-server-with-bash-and-nix.md %} +[git-notes]: https://git-scm.com/docs/git-notes +[cgit]: https://git.zx2c4.com/cgit/about/ diff --git a/scripts/spelling/international.txt b/scripts/spelling/international.txt index 93bf254..cd545a0 100644 --- a/scripts/spelling/international.txt +++ b/scripts/spelling/international.txt @@ -43,6 +43,7 @@ FTS Fastmail Feldman Forsgren +Freenode GADTs GTK GitLab @@ -141,6 +142,7 @@ clojure datalog datom datoms +divoplade dl e earned |