diff options
Diffstat (limited to 'src/content/tils/2020/11')
| -rw-r--r-- | src/content/tils/2020/11/08/find-broken-symlink.adoc | 25 | ||||
| -rw-r--r-- | src/content/tils/2020/11/12/diy-nix-bash-ci.adoc | 63 | ||||
| -rw-r--r-- | src/content/tils/2020/11/12/git-bisect-automation.adoc | 25 | ||||
| -rw-r--r-- | src/content/tils/2020/11/12/useful-bashvars.adoc | 61 | ||||
| -rw-r--r-- | src/content/tils/2020/11/14/gpodder-media.adoc | 21 | ||||
| -rw-r--r-- | src/content/tils/2020/11/30/git-notes-ci.adoc | 112 |
6 files changed, 0 insertions, 307 deletions
diff --git a/src/content/tils/2020/11/08/find-broken-symlink.adoc b/src/content/tils/2020/11/08/find-broken-symlink.adoc deleted file mode 100644 index 624d24a..0000000 --- a/src/content/tils/2020/11/08/find-broken-symlink.adoc +++ /dev/null @@ -1,25 +0,0 @@ -= Find broken symlinks with "find" -:categories: shell - -:annex: https://git-annex.branchable.com/ -:annex-wanted: https://git-annex.branchable.com/git-annex-wanted/ - -The `find` command knows how to show broken symlinks: - -[source,sh] ----- -find . -xtype l ----- - -This was useful to me when combined with {annex}[Git Annex]. Its -{annex-wanted}[`wanted`] option allows you to have a "sparse" checkout of the -content, and save space by not having to copy every annexed file locally: - -[source,sh] ----- -git annex wanted . 'exclude=Music/* and exclude=Videos/*' ----- - -You can `find` any broken symlinks outside those directories by querying with -Git Annex itself, but `find . -xtype l` works on other places too, where broken -symlinks might be a problem. diff --git a/src/content/tils/2020/11/12/diy-nix-bash-ci.adoc b/src/content/tils/2020/11/12/diy-nix-bash-ci.adoc deleted file mode 100644 index 97ace30..0000000 --- a/src/content/tils/2020/11/12/diy-nix-bash-ci.adoc +++ /dev/null @@ -1,63 +0,0 @@ -= DIY bare bones CI server with Bash and Nix -:categories: ci -:sort: 2 - -:post-receive: https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks -:example-project: https://euandreh.xyz/remembering/ci.html - -With a server with Nix installed (no need for NixOS), you can leverage its build -isolation for running CI jobs by adding a {post-receive}[post-receive] Git hook -to the server. - -In most of my project I like to keep a `test` attribute which runs the test with -`nix-build -A test`. This way, a post-receive hook could look like: - -[source,sh] ----- -#!/usr/bin/env bash -set -Eeuo pipefail -set -x - -LOGS_DIR="/data/static/ci-logs/libedn" -mkdir -p "$LOGS_DIR" -LOGFILE="${LOGS_DIR}/$(date -Is)-$(git rev-parse master).log" -exec &> >(tee -a "${LOGFILE}") - -unset GIT_DIR -CLONE="$(mktemp -d)" -git clone . "$CLONE" -pushd "$CLONE" - -finish() { - printf "\n\n>>> exit status was %s\n" "$?" -} -trap finish EXIT - -nix-build -A test ----- - -We initially (lines #5 to #8) create a log file, named after _when_ the run is -running and for _which_ commit it is running for. The `exec` and `tee` combo -allows the output of the script to go both to `stdout` _and_ the log file. This -makes the logs output show up when you do a `git push`. - -Lines #10 to #13 create a fresh clone of the repository and line #20 runs the -test command. - -After using a similar post-receive hook for a while, I now even generate a -simple HTML file to make the logs available ({example-project}[example project]) -through the browser. - -== Upsides - -No vendor lock-in, as all you need is a server with Nix installed. - -And if you pin the Nixpkgs version you're using, this very simple setup yields -extremely sandboxed runs on a very hermetic environment. - -== Downsides - -Besides the many missing shiny features of this very simplistic CI, `nix-build` -can be very resource intensive. Specifically, it consumes too much memory. So -if it has to download too many things, or the build closure gets too big, the -server might very well run out of memory. diff --git a/src/content/tils/2020/11/12/git-bisect-automation.adoc b/src/content/tils/2020/11/12/git-bisect-automation.adoc deleted file mode 100644 index dff8737..0000000 --- a/src/content/tils/2020/11/12/git-bisect-automation.adoc +++ /dev/null @@ -1,25 +0,0 @@ -= Git bisect automation -:categories: git -:sort: 1 - -It is good to have an standardized way to run builds and tests on the repository -of a project, so that you can find when a bug was introduced by using -`git bisect run`. - -I've already been in the situation when a bug was introduced and I didn't know -how it even was occurring, and running Git bisect over hundreds of commits to -pinpoint the failing commit was very empowering: - -[source,sh] ----- -$ GOOD_COMMIT_SHA=e1fd0a817d192c5a5df72dd7422e36558fa78e46 -$ git bisect start HEAD $GOOD_COMMIT_SHA -$ git bisect run sn -c './build.sh && ./run-failing-case.sh' ----- - -Git will than do a binary search between the commits, and run the commands you -provide it with to find the failing commit. - -Instead of being afraid of doing a bisect, you should instead leverage it, and -make Git help you dig through the history of the repository to find the bad -code. diff --git a/src/content/tils/2020/11/12/useful-bashvars.adoc b/src/content/tils/2020/11/12/useful-bashvars.adoc deleted file mode 100644 index fb148fb..0000000 --- a/src/content/tils/2020/11/12/useful-bashvars.adoc +++ /dev/null @@ -1,61 +0,0 @@ -= Useful Bash variables -:categories: shell - -:bash: https://www.gnu.org/software/bash/ -:bash-bang-bang: https://www.gnu.org/software/bash/manual/bash.html#Event-Designators -:bash-dollar-underscore: https://www.gnu.org/software/bash/manual/bash.html#Special-Parameters - -{bash}[GNU Bash] has a few two letter variables that may be useful when typing -on the terminal. - -== `!!`: the text of the last command - -The {bash-bang-bang}[`!!` variable] refers to the previous command, and I find -useful when following chains for symlinks: - -[source,sh] ----- -$ which git -/run/current-system/sw/bin/git -$ readlink $(!!) -readlink $(which git) -/nix/store/5bgr1xpm4m0r72h9049jbbhagxdyrnyb-git-2.28.0/bin/git ----- - -It is also useful when you forget to prefix `sudo` to a command that requires -it: - -[source,sh] ----- -$ requires-sudo.sh -requires-sudo.sh: Permission denied -$ sudo !! -sudo ./requires-sudo.sh -# all good ----- - -Bash prints the command expansion before executing it, so it is better for you -to follow along what it is doing. - -== `$_`: most recent parameter - -The {bash-dollar-underscore}[`$_` variable] will give you the most recent -parameter you provided to a previous argument, which can save you typing -sometimes: - -[source,sh] ----- -# instead of... -$ mkdir -p a/b/c/d/ -$ cd a/b/c/d/ - -# ...you can: -$ mkdir -p a/b/c/d/ -$ cd $_ ----- - -== Conclusion - -I wouldn't use those in a script, as it would make the script terser to read, I -find those useful shortcut that are handy when writing at the interactive -terminal. diff --git a/src/content/tils/2020/11/14/gpodder-media.adoc b/src/content/tils/2020/11/14/gpodder-media.adoc deleted file mode 100644 index f722f35..0000000 --- a/src/content/tils/2020/11/14/gpodder-media.adoc +++ /dev/null @@ -1,21 +0,0 @@ -= gPodder as a media subscription manager - -:re-discover: https://www.charlieharrington.com/unexpected-useless-and-urgent -:gpodder: https://gpodder.github.io/ - -As we {re-discover}[re-discover] the value of Atom/RSS feeds, most useful feed -clients I know of don't support media, specifically audio and video. - -{gpodder}[gPodder] does. - -It is mostly know as a desktop podcatcher. But the thing about podcasts is that -the feed is provided through an RSS/Atom feed. So you can just use gPodder as -your media feed client, where you have control of what you look at. - -I audio and video providers I know of offer an RSS/Atom view of their content, -so you can, say, treat any YouTube channel like a feed on its own. - -gPodder will then managed your feeds, watched/unwatched, queue downloads, etc. - -Being obvious now, it was a big finding for me. If it got you interested, I -recommend you giving gPodder a try. diff --git a/src/content/tils/2020/11/30/git-notes-ci.adoc b/src/content/tils/2020/11/30/git-notes-ci.adoc deleted file mode 100644 index 48a996b..0000000 --- a/src/content/tils/2020/11/30/git-notes-ci.adoc +++ /dev/null @@ -1,112 +0,0 @@ -= Storing CI data on Git notes -:categories: git ci - -:git-notes: https://git-scm.com/docs/git-notes -:nix-bash-ci: link:../12/diy-nix-bash-ci.html -:cgit: https://git.zx2c4.com/cgit/ - -Extending the bare bones CI server I've {nix-bash-ci}[talked about before], -divoplade on Freenode suggested storing CI artifacts on {git-notes}[Git notes], -such as tarballs, binaries, logs, _etc_. - -I've written a small script that will put log files and CI job data on Git -notes, and make it visible on the porcelain log. It is a simple extension of -the previous article: - -[source,sh] ----- -#!/usr/bin/env bash -set -Eeuo pipefail -set -x - -PREFIX='/srv/ci/vps' -mkdir -p "$PREFIX" -read -r _ SHA _ # oldrev newrev refname -FILENAME="$(date -Is)-$SHA.log" -LOGFILE="$PREFIX/$FILENAME" -exec &> >(tee -a "$LOGFILE") - -echo "Starting CI job at: $(date -Is)" - -finish() { - STATUS="$?" - printf "\n\n>>> exit status was %s\n" "$STATUS" - echo "Finishing CI job at: $(date -Is)" - popd - NOTE=$(cat <<EOF -See CI logs with: - git notes --ref=refs/notes/ci-logs show $SHA - git notes --ref=refs/notes/ci-data show $SHA -EOF -) - git notes --ref=refs/notes/ci-data add -f -m "$STATUS $FILENAME" - git notes --ref=refs/notes/ci-logs add -f -F "$LOGFILE" - git notes add -f -m "$NOTE" - printf "\n\n>>> CI logs added as Git note." -} -trap finish EXIT - -unset GIT_DIR -CLONE="$(mktemp -d)" -git clone . "$CLONE" -pushd "$CLONE" -git config --global user.email git@euandre.org -git config --global user.name 'EuAndreh CI' - -./container make check site -./container make publish ----- - -The important part is in the `finish()` function: - #25 stores the exit status -and the generated filename separated by spaces; - #26 adds the log file in a -note using the `refs/notes/ci-logs` ref; - #27 it adds a note to the commit -saying how to see the logs. - -A commit now has an attached note, and shows it whenever you look at it: - -[source,diff] ----- -$ git show 87c57133abd8be5d7cc46afbf107f59b26066575 -commit 87c57133abd8be5d7cc46afbf107f59b26066575 -Author: EuAndreh <eu@euandre.org> -Date: Wed Feb 24 21:58:28 2021 -0300 - - vps/machines.scm: Change path to cronjob files - -Notes: - See CI logs with: - git notes --ref=refs/notes/ci-logs show 87c57133abd8be5d7cc46afbf107f59b26066575 - git notes --ref=refs/notes/ci-data show 87c57133abd8be5d7cc46afbf107f59b26066575 - -diff --git a/servers/vps/machines.scm b/servers/vps/machines.scm -index d1830ca..a4ccde7 100644 ---- a/servers/vps/machines.scm -+++ b/servers/vps/machines.scm -@@ -262,8 +262,8 @@ pki " mail-domain " key \"" (tls-priv-for mail-domain) "\"")) - (service mcron-service-type - (mcron-configuration - (jobs -- (list #~(job "30 1 * * 1" "guix gc -d") -- #~(job "30 0 * * *" "/var/lib/euandreh/backup.sh"))))) -+ (list #~(job "30 1 * * 1" "/opt/bin/gc.sh") -+ #~(job "30 0 * * *" "/opt/bin/backup.sh"))))) - (service dhcp-client-service-type) - #; - (service opensmtpd-service-type ----- - -Other tools such as {cgit}[cgit] will also show notes on the web interface: -https://euandre.org/git/servers/commit?id=87c57133abd8be5d7cc46afbf107f59b26066575. - -You can go even further: since cgit can serve raw blob directly, you can even -serve such artifacts (log files, release artifacts, binaries) from cgit itself: - -[source,sh] ----- -$ SHA="$(git notes --ref=refs/notes/ci-logs list 87c57133abd8be5d7cc46afbf107f59b26066575)" -$ echo "https://euandre.org/git/servers/blob?id=$SHA" -https://euandre.org/git/servers/blob?id=1707a97bae24e3864fe7943f8dda6d01c294fb5c ----- - -And like that you'll have cgit serving the artifacts for you: -https://euandre.org/git/servers/blob?id=1707a97bae24e3864fe7943f8dda6d01c294fb5c. |