diff options
Diffstat (limited to 'src')
125 files changed, 13238 insertions, 121 deletions
diff --git a/src/bin/pb b/src/bin/pb new file mode 100755 index 0000000..c934124 --- /dev/null +++ b/src/bin/pb @@ -0,0 +1,60 @@ +#!/bin/sh +set -eu + +usage() { + cat <<-'EOF' + Usage: pb FILENAME.adoc [CONTENT] + EOF +} + + +TITLE=FIXME +while getopts 't:' flag; do + case "$flag" in + (t) + TITLE="$OPTARG" + ;; + (*) + usage >&2 + exit 2 + ;; + esac +done +shift $((OPTIND - 1)) + + +FILENAME="${1:-}" +SOURCE="${2:-}" +eval "$(assert-arg -- "$FILENAME" 'FILENAME')" +FILENAME="${FILENAME%.adoc}.adoc" + +cd "$(dirname "$(readlink -f -- "$0")")"/../../ + +DIR=src/content/en/pastebins/"$(timestamp | sed 's|-|/|g')" +OUT="$DIR"/"$FILENAME" + +if [ -e "$OUT" ]; then + printf '"%s" already exists.\n' "$OUT" >&2 + exit 2 +fi + +if [ -n "$SOURCE" ]; then + CONTENT="$(cat "$SOURCE")" +else + if [ -t 0 ]; then + CONTENT='FIXME' + else + printf 'Reading from STDIN...\n' >&2 + CONTENT="$(cat)" + exec 0</dev/tty + fi +fi + +mkdir -p "$DIR" +printf '= %s\n\n[source,sh]\n----\n%s\n----\n' "$TITLE" "$CONTENT" > "$OUT" +${VISUAL:-${EDITOR:-vi}} "$OUT" + +sh mkdeps.sh > deps.mk +git reset . +git add "$OUT" deps.mk +git commit -m "pb(1): Auto-add \"$OUT\"" diff --git a/src/comments.lemmy.json b/src/comments.lemmy.json new file mode 100644 index 0000000..b8606fd --- /dev/null +++ b/src/comments.lemmy.json @@ -0,0 +1,288 @@ +[ + { + "id": "https://programming.dev/post/56115670", + "source": "fediverse", + "author": "EuAndreh", + "handle": "EuAndreh@programming.dev", + "profile": "https://programming.dev/u/EuAndreh", + "url": "https://programming.dev/post/56115670", + "page": "en/til/2026/09/05/multiline-sh-comments.html", + "published": "2026-09-05T16:06:50Z", + "content": "Multiline comments in POSIX sh\n\nBack writing in my website.\n" + }, + { + "id": "https://programming.dev/comment/25835338", + "in_reply_to": "https://programming.dev/post/56115670", + "source": "fediverse", + "author": "FizzyOrange", + "handle": "FizzyOrange@programming.dev", + "profile": "https://programming.dev/u/FizzyOrange", + "url": "https://programming.dev/comment/25835338", + "published": "2026-09-05T16:37:44Z", + "content": "Yeah but... Don't. Multiline comments are a bit of an anti-pattern at the best of times. This is a war crime.\n" + }, + { + "id": "https://programming.dev/comment/25835411", + "in_reply_to": "https://programming.dev/comment/25835338", + "source": "fediverse", + "author": "EuAndreh", + "handle": "EuAndreh@programming.dev", + "profile": "https://programming.dev/u/EuAndreh", + "url": "https://programming.dev/comment/25835411", + "published": "2026-09-05T16:43:32Z", + "content": "I use this only when developing, mostly while debugging. IIRC I've never commited code like this.\n" + }, + { + "id": "https://programming.dev/comment/25835523", + "in_reply_to": "https://programming.dev/comment/25835411", + "source": "fediverse", + "author": "FizzyOrange", + "handle": "FizzyOrange@programming.dev", + "profile": "https://programming.dev/u/FizzyOrange", + "url": "https://programming.dev/comment/25835523", + "published": "2026-09-05T16:50:23Z", + "content": "Close notepad.exe. Open a real editor. Select the text you want to comment out. Press Ctrl-/\n" + }, + { + "id": "https://programming.dev/comment/25835605", + "in_reply_to": "https://programming.dev/comment/25835523", + "source": "fediverse", + "author": "EuAndreh", + "handle": "EuAndreh@programming.dev", + "profile": "https://programming.dev/u/EuAndreh", + "url": "https://programming.dev/comment/25835605", + "published": "2026-09-05T16:55:43Z", + "content": "No notepad.exe, nvi: `'a,s/^/# /`.\n" + }, + { + "id": "https://beehaw.org/comment/6198058", + "in_reply_to": "https://programming.dev/comment/25835338", + "source": "fediverse", + "author": "TehPers", + "handle": "TehPers@beehaw.org", + "profile": "https://beehaw.org/u/TehPers", + "url": "https://beehaw.org/comment/6198058", + "published": "2026-09-05T18:07:40Z", + "content": "\u003e Multiline comments are a bit of an anti-pattern at the best of times.\n\nIf you're making this as a general statement, it varies from language to language. For example, CSS (and arguably HTML and Markdown) only supports multiline comments, and multiline comments are often used for documentation (as with jsdoc and, though only comments in the same way that heredocs to /dev/null are, Python).\n\nBut yeah, if you *actually* do this in Bash, [wat](https://www.destroyallsoftware.com/talks/wat).\n" + }, + { + "id": "https://lemmy.ml/comment/27651659", + "in_reply_to": "https://programming.dev/post/56115670", + "source": "fediverse", + "author": "thingsiplay", + "handle": "thingsiplay@lemmy.ml", + "profile": "https://lemmy.ml/u/thingsiplay", + "url": "https://lemmy.ml/comment/27651659", + "published": "2026-09-05T18:36:21Z", + "content": "I think this solves an issue, that's not an issue at all and makes it more complicated and harder to read. What do I mean by that?\n\nIt requires more text and exact correct syntax of multiple elements. It makes it harder to see where the start and end is, and one has to look for any lines that could potentially be a comment in a heredoc. The syntax highlighter does not show this in comment color. Bash still executes those lines to read the \"comment\", as it thinks its normal code. In example variables can still be expanded in heredocs. Other tools removing comments from script to count lines would be fooled too.\n\nCompare this to a single hashtag symbol in the beginning of each comment. That's all. This is also good practice everyone understands. From practical point of view, why would anyone use heredocs for multi line comments over single line comments?\n" + }, + { + "id": "https://lemmy.ml/comment/27651807", + "in_reply_to": "https://beehaw.org/comment/6198058", + "source": "fediverse", + "author": "thingsiplay", + "handle": "thingsiplay@lemmy.ml", + "profile": "https://lemmy.ml/u/thingsiplay", + "url": "https://lemmy.ml/comment/27651807", + "published": "2026-09-05T18:46:28Z", + "content": "\u003e CSS (and arguably HTML and Markdown) only supports multiline comments, and multiline comments are often used for documentation\n\nThose aren't programming languages. Because, especially in HTML, a line end does not end an instruction and is just for formatting. So single line comments would be very awkward in HTML, so they had to use multi only. And because CSS is meant to be used close to HTML, I think they opt in to the same. JavaScript in example has single line comments again.\n\nDoes any programming language exist that supports multiline comments only without single line comments?\n" + }, + { + "id": "https://programming.dev/comment/25837399", + "in_reply_to": "https://lemmy.ml/comment/27651807", + "source": "fediverse", + "author": "EuAndreh", + "handle": "EuAndreh@programming.dev", + "profile": "https://programming.dev/u/EuAndreh", + "url": "https://programming.dev/comment/25837399", + "published": "2026-09-05T18:51:39Z", + "content": "C89, AKA ANSI C famously only supports `/* */`, having `// ` as a cross-pollinated idea from C++.\n" + }, + { + "id": "https://lemmy.ml/comment/27652362", + "in_reply_to": "https://programming.dev/comment/25837399", + "source": "fediverse", + "author": "thingsiplay", + "handle": "thingsiplay@lemmy.ml", + "profile": "https://lemmy.ml/u/thingsiplay", + "url": "https://lemmy.ml/comment/27652362", + "published": "2026-09-05T19:19:44Z", + "content": "Good example, at least historical. ;-) I wonder why they never added single line comments before. I mean it took them quite a long time to do so. Languages before 1960 even had single lines too. \n\nEdit: BTW it was common in C to write multiline comments to have them look like single line comments too, because that's easier to read and understand. Maybe that's why they didn't feel the need for, because in practical usage it could be replicated. In example as:\n\n```c\n/* Title\n * description\n * variables\n */\n```\n\nThis could easily be replaced by single lines as:\n\n```c\n// Title\n// description\n// variables\n```\n" + }, + { + "id": "https://beehaw.org/comment/6198279", + "in_reply_to": "https://lemmy.ml/comment/27651807", + "source": "fediverse", + "author": "TehPers", + "handle": "TehPers@beehaw.org", + "profile": "https://beehaw.org/u/TehPers", + "url": "https://beehaw.org/comment/6198279", + "published": "2026-09-05T19:42:00Z", + "content": "Both HTML and CSS (when together) are turing complete, and JSX syntax only supports HTML comments in many places.\n\nJavaScript *having* single line comments isn't really relevant. Multiline comments are in heavy use there, arguably moreso than single line comments.\n\nEdit: and I should add that \"programming languages\" is moving the goalpost. The person I responded to didn't qualify their statement to programming languages.\n" + }, + { + "id": "https://lemmy.ml/comment/27652766", + "in_reply_to": "https://beehaw.org/comment/6198279", + "source": "fediverse", + "author": "thingsiplay", + "handle": "thingsiplay@lemmy.ml", + "profile": "https://lemmy.ml/u/thingsiplay", + "url": "https://lemmy.ml/comment/27652766", + "published": "2026-09-05T19:50:27Z", + "content": "I don't think that being turing complete means anything in this discussion, because it does not change what the language is for and how its used and interpreted. HTML and CSS are not instructions for the CPU, they are markup languages describing text for humans, not compiling it down to execute machine code for CPUs.\n" + }, + { + "id": "https://beehaw.org/comment/6198316", + "in_reply_to": "https://lemmy.ml/comment/27652766", + "source": "fediverse", + "author": "TehPers", + "handle": "TehPers@beehaw.org", + "profile": "https://beehaw.org/u/TehPers", + "url": "https://beehaw.org/comment/6198316", + "published": "2026-09-05T20:05:25Z", + "content": "Again, the discussion was about multiline comments, not programming languages.\n\nAnd regardless of all that, JSX remains one of the most used languages in the world and supports only multiline comments in certain contexts, and JavaScript as a whole uses multiline comments generously.\n\nThe discussion was around multiline comments being an anti-pattern, which is entirely untrue. They just serve a different purpose than single-line comments, and usage varies from language to language.\n" + }, + { + "id": "https://lemmy.ml/comment/27653021", + "in_reply_to": "https://beehaw.org/comment/6198316", + "source": "fediverse", + "author": "thingsiplay", + "handle": "thingsiplay@lemmy.ml", + "profile": "https://lemmy.ml/u/thingsiplay", + "url": "https://lemmy.ml/comment/27653021", + "published": "2026-09-05T20:09:41Z", + "content": "\u003e The discussion was around multiline comments being an anti-pattern, which is entirely untrue. \n\nIt is an antipattern in Bash. He did not say that for HTML or JavaScript.\n" + }, + { + "id": "https://beehaw.org/comment/6198326", + "in_reply_to": "https://lemmy.ml/comment/27653021", + "source": "fediverse", + "author": "TehPers", + "handle": "TehPers@beehaw.org", + "profile": "https://beehaw.org/u/TehPers", + "url": "https://beehaw.org/comment/6198326", + "published": "2026-09-05T20:11:36Z", + "content": "Bash doesn't have multiline comments. Even the article acknowledges this lol. From context, it was a general statement, not specific to Bash.\n" + }, + { + "id": "https://programming.dev/comment/25841603", + "in_reply_to": "https://programming.dev/post/56115670", + "source": "fediverse", + "author": "CameronDev", + "handle": "CameronDev@programming.dev", + "profile": "https://programming.dev/u/CameronDev", + "url": "https://programming.dev/comment/25841603", + "published": "2026-09-06T00:06:07Z", + "content": "Your website is dead.\n" + }, + { + "id": "https://programming.dev/comment/25842210", + "in_reply_to": "https://programming.dev/comment/25841603", + "source": "fediverse", + "author": "EuAndreh", + "handle": "EuAndreh@programming.dev", + "profile": "https://programming.dev/u/EuAndreh", + "url": "https://programming.dev/comment/25842210", + "published": "2026-09-06T01:05:26Z", + "content": "Oops, renamed the www dir, should be back now.\n" + }, + { + "id": "https://programming.dev/comment/25845713", + "in_reply_to": "https://beehaw.org/comment/6198058", + "source": "fediverse", + "author": "FizzyOrange", + "handle": "FizzyOrange@programming.dev", + "profile": "https://programming.dev/u/FizzyOrange", + "url": "https://programming.dev/comment/25845713", + "published": "2026-09-06T08:39:35Z", + "content": "The fact that CSS only supports multiline comments doesn't mean multiline comments are good, it means CSS is badly designed.\n" + }, + { + "id": "https://beehaw.org/comment/6199598", + "in_reply_to": "https://programming.dev/comment/25845713", + "source": "fediverse", + "author": "TehPers", + "handle": "TehPers@beehaw.org", + "profile": "https://beehaw.org/u/TehPers", + "url": "https://beehaw.org/comment/6199598", + "published": "2026-09-06T08:54:26Z", + "content": "It means neither of those, actually, just that CSS supports only multiline comments.\n\nMultiline comments remain prevalent across ecosystems though (like in JS and Python). If you believe they're bad there too, that's fine, but the fact is that it would be an anti-pattern to go against those established standards.\n" + }, + { + "id": "https://lemmy.world/comment/25692482", + "in_reply_to": "https://programming.dev/post/56115670", + "source": "fediverse", + "author": "talkingpumpkin", + "handle": "talkingpumpkin@lemmy.world", + "profile": "https://lemmy.world/u/talkingpumpkin", + "url": "https://lemmy.world/comment/25692482", + "published": "2026-09-06T16:10:53Z", + "content": "is\n```bash\n: \u003c\u003cEOF\n text\n text\nEOF\n```\n\na bash-ism?\n" + }, + { + "id": "https://programming.dev/comment/25852776", + "in_reply_to": "https://lemmy.world/comment/25692482", + "source": "fediverse", + "author": "EuAndreh", + "handle": "EuAndreh@programming.dev", + "profile": "https://programming.dev/u/EuAndreh", + "url": "https://programming.dev/comment/25852776", + "published": "2026-09-06T18:54:06Z", + "content": "AFAICT, not really, looks line POSIX sh at a glance. Just don't forget to `: \u003c\u003c'EOF'` instead of `: \u003c\u003cEOF`.\n\nBut `:` is arguably simpler than `cat`, one fewer fork/exec call plus no `open(2)` on `/dev/null`.\n" + }, + { + "id": "https://programming.dev/comment/25852783", + "in_reply_to": "https://programming.dev/comment/25852776", + "source": "fediverse", + "author": "EuAndreh", + "handle": "EuAndreh@programming.dev", + "profile": "https://programming.dev/u/EuAndreh", + "url": "https://programming.dev/comment/25852783", + "published": "2026-09-06T18:54:29Z", + "content": "I like it, thanks :)\n" + }, + { + "id": "https://lemmy.ml/comment/27668420", + "in_reply_to": "https://beehaw.org/comment/6198279", + "source": "fediverse", + "author": "thingsiplay", + "handle": "thingsiplay@lemmy.ml", + "profile": "https://lemmy.ml/u/thingsiplay", + "url": "https://lemmy.ml/comment/27668420", + "published": "2026-09-06T20:12:59Z", + "content": "\u003e Edit: and I should add that “programming languages” is moving the goalpost. The person I responded to didn’t qualify their statement to programming languages.\n\nI think from context it is apparent that we are not talking about HTML and CSS (which you brought this up yourself, not the person you responded to). I think there was no need to specify his statement was to programming languages, by context we know it is. Also we are in Programming.dev. What context do you need more?\n" + }, + { + "id": "https://beehaw.org/comment/6200940", + "in_reply_to": "https://lemmy.ml/comment/27668420", + "source": "fediverse", + "author": "TehPers", + "handle": "TehPers@beehaw.org", + "profile": "https://beehaw.org/u/TehPers", + "url": "https://beehaw.org/comment/6200940", + "published": "2026-09-06T20:45:35Z", + "content": "TIL programmers don't write HTML and CSS.\n\n(I guess if all you use are LLMs to write it, maybe that's true.)\n" + }, + { + "id": "https://lemmy.ml/comment/27670051", + "in_reply_to": "https://beehaw.org/comment/6200940", + "source": "fediverse", + "author": "thingsiplay", + "handle": "thingsiplay@lemmy.ml", + "profile": "https://lemmy.ml/u/thingsiplay", + "url": "https://lemmy.ml/comment/27670051", + "published": "2026-09-06T22:09:56Z", + "content": "You understand the difference between a markup language to display text with HTML and CSS, and a programming language to execute programs. Right? And this has nothing to do with LLMs. I don't know why you are in an attacking position for no reason. I didn't want hurt anyone, just discussing here.\n" + }, + { + "id": "https://beehaw.org/comment/6201202", + "in_reply_to": "https://lemmy.ml/comment/27670051", + "source": "fediverse", + "author": "TehPers", + "handle": "TehPers@beehaw.org", + "profile": "https://beehaw.org/u/TehPers", + "url": "https://beehaw.org/comment/6201202", + "published": "2026-09-06T22:57:49Z", + "content": "\u003e I don't know why you are in an attacking position for no reason. I didn't want hurt anyone, just discussing here.\n\nYou are being excessively pedantic over a topic that has absolutely nothing to do with the discussion. I don't know how you want me to respond to these comments.\n\nAs a reminder, what I was responding to was whether multiline comments are an anti-pattern, *not* whether HTML+CSS is a programming language.\n" + } +] diff --git a/src/comments.mastodon.json b/src/comments.mastodon.json new file mode 100644 index 0000000..3107d28 --- /dev/null +++ b/src/comments.mastodon.json @@ -0,0 +1,13 @@ +[ + { + "id": "https://hachyderm.io/ap/users/117133834156130647/statuses/117218686716307964", + "source": "fediverse", + "author": "EuAndreh", + "handle": "euandreh@hachyderm.io", + "profile": "https://hachyderm.io/@euandreh", + "url": "https://hachyderm.io/@euandreh/117218686716307964", + "page": "en/til/2026/09/05/multiline-sh-comments.html", + "published": "2026-09-05T13:33:34Z", + "content": "Fun hack: multi-line comments in sh\n\nhttps://euandre.org/en/til/2026/09/05/multiline-sh-comments.html\n\n#sh\n" + } +] diff --git a/src/comments.mbox b/src/comments.mbox new file mode 100644 index 0000000..edf49e0 --- /dev/null +++ b/src/comments.mbox @@ -0,0 +1,3464 @@ +From rory@roryokane.com Mon Aug 31 16:44:13 2020 +Received: from mail-qv1-f42.google.com (mail-qv1-f42.google.com [209.85.219.42]) + by mail-b.sr.ht (Postfix) with ESMTPS id 1E84AFF0FB + for <~euandreh/public-inbox@lists.sr.ht>; Mon, 31 Aug 2020 16:44:13 +0000 (UTC) +Received: by mail-qv1-f42.google.com with SMTP id m14so1468478qvt.1 + for <~euandreh/public-inbox@lists.sr.ht>; Mon, 31 Aug 2020 09:44:13 -0700 (PDT) +X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; + d=1e100.net; s=20161025; + h=x-gm-message-state:mime-version:from:date:message-id:subject:to; + bh=1N/5MWE9d7/I+SxiuWn81KliArG9XSFF8duwpxyY0oA=; + b=GiBnVI+Pf7DSl9ycPdfwPRjfdrlqlQQdgGKKyGsiIXPB1mLh6sBlzhoEQASxa/9hJm + rfhT8wRfQyBwscG2FpECYK09N/0ks7a71Pi3vy9ePKvSHJl0FYhywyAWzZ9Z7Xfp0/SF + oohs7Vzovs/sA9D9m2oKPcyfrj6lHWqf5qHJdfatP4LfVHCdRg1uwj3CmtCJWzmIFzuM + aIkUXLrK5YyBbfVl2L6EvEU1L7EDqOaTrXJt/+hWOev813kGYdYWsxMbYmirBAmO8sal + rgaF9vR/Jp8C+bxV/LSONsVOi4CA8azpLd1SYYpmS5yM4H5ISjaIOZfrZycx/Zd5Rle8 + tn6Q== +X-Gm-Message-State: AOAM530nGnF5g0fpYzDDTNbp5T67sHNZCqeZcVlHkD7CLIqa78AzYkW0 + 44VZGvmhPwGnRAVlt7ZPbGJQAlmieuw= +X-Google-Smtp-Source: ABdhPJwgZnnyjMFpfNBttjOyxb0b4VZgNwyJNN7HkBIHn457LHtZ8CSNy1RJMts49AmqIUIw2YT9Yg== +X-Received: by 2002:a0c:b255:: with SMTP id k21mr1811354qve.134.1598892252561; + Mon, 31 Aug 2020 09:44:12 -0700 (PDT) +Received: from mail-qt1-f176.google.com (mail-qt1-f176.google.com. [209.85.160.176]) + by smtp.gmail.com with ESMTPSA id n33sm10800778qtd.43.2020.08.31.09.44.12 + for <~euandreh/public-inbox@lists.sr.ht> + (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); + Mon, 31 Aug 2020 09:44:12 -0700 (PDT) +Received: by mail-qt1-f176.google.com with SMTP id z2so5166925qtv.12 + for <~euandreh/public-inbox@lists.sr.ht>; Mon, 31 Aug 2020 09:44:12 -0700 (PDT) +X-Received: by 2002:ac8:73d0:: with SMTP id v16mr2161160qtp.384.1598892251883; + Mon, 31 Aug 2020 09:44:11 -0700 (PDT) +MIME-Version: 1.0 +From: =?UTF-8?Q?Rory_O=E2=80=99Kane?= <rory@roryokane.com> +Date: Mon, 31 Aug 2020 12:44:01 -0400 +X-Gmail-Original-Message-ID: <CAD_ag1NQoRdA3CyDPFyXBs-KcuU36dKqtZ8HCk-bKMeKRCe_UA@mail.gmail.com> +Message-ID: <CAD_ag1NQoRdA3CyDPFyXBs-KcuU36dKqtZ8HCk-bKMeKRCe_UA@mail.gmail.com> +Subject: Re: The database I wish I had +To: ~euandreh/public-inbox@lists.sr.ht +Content-Type: text/plain; charset="UTF-8" + +In this discussion of your post on Lobsters, there are some comments +that may aid you in your quest: +https://lobste.rs/s/m9vkg4/database_i_wish_i_had + +From eu@euandre.org Mon Aug 31 23:00:25 2020 +Received: from box.euandre.org (box.euandre.org [46.101.160.115]) + by mail-b.sr.ht (Postfix) with ESMTPS id 95690FF10A + for <~euandreh/public-inbox@lists.sr.ht>; Mon, 31 Aug 2020 23:00:24 +0000 (UTC) +Authentication-Results: mail-b.sr.ht; + dkim=pass (2048-bit key) header.d=euandre.org header.i=@euandre.org header.b=UPSgUUXq +Received: from authenticated-user (box.euandre.org [46.101.160.115]) + (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) + (No client certificate requested) + by box.euandre.org (Postfix) with ESMTPSA id 70152FC81E; + Mon, 31 Aug 2020 20:00:20 -0300 (-03) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=euandre.org; s=mail; + t=1598914822; bh=f4y8R0BR6plzUtLJhSLdKJQrnDt5TiYAaxpeGwaCRBQ=; + h=From:To:Cc:Subject:In-Reply-To:References:Date:From; + b=UPSgUUXqESOz5UhGXtlYTFrg0Vy42raDUdQSF1jRvRJpFdVQ7MWErphZhOvLjMrSB + UxSspR/+WTYJDMf61hcqMfCqFYqGLpFH6Q6I6V4M7MABddX6mKgoyL16fBJl2V/FcH + DVJBcrI+FeDrr3eoLTbXR5ZZzddSTSzfS2w93/QLad95xyd443VBEtUXEvl+G6pLC1 + Bzv7oeG07V2OZF+jIsfW116e0tyUsT1JTY377yiWTIxoqP27+PIyP2SZjPyBsS/Xzb + 0MqK6PhL3IbwQ8xvv/h2SklKD9PYXtrjU5hupBXJSb/puN7fZHoaSv/qM69mfNuMxI + yTfUF1RUxJy5A== +From: EuAndreh <eu@euandre.org> +To: Rory =?utf-8?Q?O=E2=80=99Kane?= <rory@roryokane.com>, + ~euandreh/public-inbox@lists.sr.ht +Cc: +Subject: Re: The database I wish I had +In-Reply-To: <CAD_ag1NQoRdA3CyDPFyXBs-KcuU36dKqtZ8HCk-bKMeKRCe_UA@mail.gmail.com> +References: <CAD_ag1NQoRdA3CyDPFyXBs-KcuU36dKqtZ8HCk-bKMeKRCe_UA@mail.gmail.com> +Date: Mon, 31 Aug 2020 19:55:51 -0300 +Message-ID: <87r1rmzaq0.fsf@euandre.org> +MIME-Version: 1.0 +Content-Type: text/plain; charset=utf-8 +Content-Transfer-Encoding: quoted-printable + +Rory O=E2=80=99Kane <rory@roryokane.com> writes: + +> In this discussion of your post on Lobsters, there are some comments +> that may aid you in your quest: +> https://lobste.rs/s/m9vkg4/database_i_wish_i_had + +I don't have an account to discuss things there, and I don't have an +invite, either. + +If you could invite me there I would gladly engage on the discussion =E2=98= +=BA=EF=B8=8F. + +From rory@roryokane.com Mon Aug 31 23:43:09 2020 +Received: from mail-qk1-f180.google.com (mail-qk1-f180.google.com [209.85.222.180]) + by mail-b.sr.ht (Postfix) with ESMTPS id 43D60FF107 + for <~euandreh/public-inbox@lists.sr.ht>; Mon, 31 Aug 2020 23:43:09 +0000 (UTC) +Received: by mail-qk1-f180.google.com with SMTP id w186so7938858qkd.1 + for <~euandreh/public-inbox@lists.sr.ht>; Mon, 31 Aug 2020 16:43:09 -0700 (PDT) +X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; + d=1e100.net; s=20161025; + h=x-gm-message-state:mime-version:references:in-reply-to:from:date + :message-id:subject:to:cc:content-transfer-encoding; + bh=gaaBELP8x2iHF+2fExDi20fg3n/w86cYg/IGy9Qn3xc=; + b=DRAjD5Bo+V1b4ugqTEX8uvyep/1C0WyUO+myK/uHn6v95bQRDYNS1+whm98sXMPXrb + qcafP0m0kbUfsTjEboLvrqZBZa3Ztv+8GFW6XmRjrtfJuZXx7RL0wXj16daqa2ik/dEe + RA0WM9UrrczXtTBjwnrG7ozNOGy1SKDKHO/c4f7aylfYGP+wVxyDl6nAaMQk+wHGkJcY + I37uXMATBkcsNPO+R8/lsyiCLkgk3wED+XJpYKZekgRA8p2I4S7wOP1SqmIpIyQlNllq + cYtPJkv8wi5MNtwR8t0k+DU6QZwYk8H1U+3YhWfwuVl+S+zHN1DC3Tw5ZrlxcIlpsP5v + 0fQA== +X-Gm-Message-State: AOAM530oi09/wtTwog1b67r3kLqfg4eevHc/vZXa69bokmICUCS/wXgK + ZU0PXrAbUDFg2UNYNvv/oKkU5//k/mo= +X-Google-Smtp-Source: ABdhPJyzZixNEwElMWvVTgTATohR8FEqbOW0p/68jnNsN8u+gCY6xUFeVZhDVtPlvGcDFDsynVK1gQ== +X-Received: by 2002:a05:620a:2224:: with SMTP id n4mr4016163qkh.18.1598917388635; + Mon, 31 Aug 2020 16:43:08 -0700 (PDT) +Received: from mail-qk1-f178.google.com (mail-qk1-f178.google.com. [209.85.222.178]) + by smtp.gmail.com with ESMTPSA id u15sm12587433qtj.3.2020.08.31.16.43.08 + for <~euandreh/public-inbox@lists.sr.ht> + (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); + Mon, 31 Aug 2020 16:43:08 -0700 (PDT) +Received: by mail-qk1-f178.google.com with SMTP id p25so7942117qkp.2 + for <~euandreh/public-inbox@lists.sr.ht>; Mon, 31 Aug 2020 16:43:08 -0700 (PDT) +X-Received: by 2002:a37:b6c6:: with SMTP id g189mr4006715qkf.491.1598917387885; + Mon, 31 Aug 2020 16:43:07 -0700 (PDT) +MIME-Version: 1.0 +References: <CAD_ag1NQoRdA3CyDPFyXBs-KcuU36dKqtZ8HCk-bKMeKRCe_UA@mail.gmail.com> + <87r1rmzaq0.fsf@euandre.org> +In-Reply-To: <87r1rmzaq0.fsf@euandre.org> +From: =?UTF-8?Q?Rory_O=E2=80=99Kane?= <rory@roryokane.com> +Date: Mon, 31 Aug 2020 19:42:57 -0400 +X-Gmail-Original-Message-ID: <CAD_ag1Obqre+uRN+fQgX1-O7WWkT=2dTo4-HxabLXMBE9od6DQ@mail.gmail.com> +Message-ID: <CAD_ag1Obqre+uRN+fQgX1-O7WWkT=2dTo4-HxabLXMBE9od6DQ@mail.gmail.com> +Subject: Re: The database I wish I had +To: EuAndreh <eu@euandre.org> +Cc: ~euandreh/public-inbox@lists.sr.ht +Content-Type: text/plain; charset="UTF-8" +Content-Transfer-Encoding: quoted-printable + +EuAndreh <eu@euandre.org> wrote: + +> I don't have an account to discuss things there, and I don't have an +> invite, either. +> +> If you could invite me there I would gladly engage on the discussion =EF= +=B8=8F. + +I checked out your website and based on your writing, I can probably +trust you with an invite. Invite sent=E2=80=94enjoy. + +From eu@euandre.org Tue Sep 1 01:00:13 2020 +Received: from box.euandre.org (box.euandre.org [46.101.160.115]) + by mail-b.sr.ht (Postfix) with ESMTPS id B6E38FF11C + for <~euandreh/public-inbox@lists.sr.ht>; Tue, 1 Sep 2020 01:00:12 +0000 (UTC) +Authentication-Results: mail-b.sr.ht; + dkim=pass (2048-bit key) header.d=euandre.org header.i=@euandre.org header.b=P61dJynV +Received: from authenticated-user (box.euandre.org [46.101.160.115]) + (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) + (No client certificate requested) + by box.euandre.org (Postfix) with ESMTPSA id 167EDFC820; + Mon, 31 Aug 2020 22:00:09 -0300 (-03) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=euandre.org; s=mail; + t=1598922011; bh=VOay/ul6frlnGv4X4iPDMI75WLW6Uop6PCjAV5nwAac=; + h=From:To:Cc:Cc:Subject:In-Reply-To:References:Date:From; + b=P61dJynVkkTLwtYleZjejOi5dPaIMpaJR1omX34om5nTFtkTU/7ckXbqlU/AlUxfb + IC516Xu9/pkk1uW45SdeoU8+9nYnK9/KK7VCiifC5yQeMLZwlzNupanCYeycFes500 + D5Ygal4+Kz6prDX2cKrF4O97/nBKQKG6QQDzRG61Q1v+Tg2OMoJ45XENtQ7ONYzt9H + RkbSKiNA4ipJDzLx1vd9VX+cKSC9VljZn9dFxIBA6KQ+nmbMNXtNmirc10sREJ2g3u + zYefnUA3vRF1csdbIkTjfbiWnD2w2RN3Zj/oMwhNiAjHix4FyIg89WXwzv/CHmP9gr + 3+SrXBIC6XxzQ== +From: EuAndreh <eu@euandre.org> +To: Rory =?utf-8?Q?O=E2=80=99Kane?= <rory@roryokane.com> +Cc: ~euandreh/public-inbox@lists.sr.ht +Cc: +Subject: Re: The database I wish I had +In-Reply-To: <CAD_ag1Obqre+uRN+fQgX1-O7WWkT=2dTo4-HxabLXMBE9od6DQ@mail.gmail.com> +References: <CAD_ag1NQoRdA3CyDPFyXBs-KcuU36dKqtZ8HCk-bKMeKRCe_UA@mail.gmail.com> + <87r1rmzaq0.fsf@euandre.org> + <CAD_ag1Obqre+uRN+fQgX1-O7WWkT=2dTo4-HxabLXMBE9od6DQ@mail.gmail.com> +Date: Mon, 31 Aug 2020 21:42:38 -0300 +Message-ID: <87mu2az5s1.fsf@euandre.org> +MIME-Version: 1.0 +Content-Type: text/plain; charset=utf-8 +Content-Transfer-Encoding: quoted-printable + +Rory O=E2=80=99Kane <rory@roryokane.com> writes: + +> I checked out your website and based on your writing, I can probably +> trust you with an invite. Invite sent=E2=80=94enjoy. + +Thanks for the invitation, I hope I can add to the discussion and the +community.=20 + +From me@josephg.com Tue Sep 1 03:22:08 2020 +Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) + by mail-b.sr.ht (Postfix) with ESMTPS id 85748FF119 + for <~euandreh/public-inbox@lists.sr.ht>; Tue, 1 Sep 2020 03:22:08 +0000 (UTC) +Authentication-Results: mail-b.sr.ht; + dkim=pass (1024-bit key) header.d=josephg.com header.i=@josephg.com header.b=XpX/7nfb; + dkim=fail reason="key not found in DNS" (0-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=MSacOUNm +Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) + by mailout.nyi.internal (Postfix) with ESMTP id 4403D5C0190 + for <~euandreh/public-inbox@lists.sr.ht>; Mon, 31 Aug 2020 23:22:08 -0400 (EDT) +Received: from imap35 ([10.202.2.85]) + by compute1.internal (MEProxy); Mon, 31 Aug 2020 23:22:08 -0400 +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=josephg.com; h= + mime-version:message-id:date:from:to:subject:content-type; s= + mesmtp; bh=D0CRtslwgxf19V66w3zMnP44hZ3wW3VGv7uaTI0Tszs=; b=XpX/7 + nfbLe6I0o40Nfy5xKR+AcUanUComIWaSJMa5Gz5V2bCbIFdj6j2sSHEJWMJ/OvnG + K4hgJmu5ZXcYOmhZBRIrGpjackRdPfipduHOMYuWsT/tdMecLsxBU79WhNR71xfi + gCQK/U2ChiHuu2JvN2lhP9UZHk7EoaAkBGVtwQ= +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= + messagingengine.com; h=content-type:date:from:message-id + :mime-version:subject:to:x-me-proxy:x-me-proxy:x-me-sender + :x-me-sender:x-sasl-enc; s=fm3; bh=D0CRtslwgxf19V66w3zMnP44hZ3wW + 3VGv7uaTI0Tszs=; b=MSacOUNm/cU+7m2AkVNHn56i7XS7rZXVQlRheFQ10MXFQ + gY7AEoJYghtK1hl+FZhe4yTOOUJfj7xGQsnp7ENjd0yHSZPIqh4w1WDWMeWgvIwg + p3v7xHQc/ce/1zKCV7NNftJh9ietQ63JTujrUTGC0Hszv5+D+7mkLO+OGtMwAmVR + auNg0sFFa6BdTfaQLRWAX/BqQkiYcJ9oFW+Lxw1wiN6D/K005aRgHbbOwCAHGi9v + 6qkg/cVFzzqR52ydek4Oq0AtGwRBVMGK7llRBDRe83VDOcjzRNZScU/Z1hlEhDWF + fYm0B4nWev8MbeQUup3DjryE9U0WrMv/1MmI+G25A== +X-ME-Sender: <xms:X75NX0NkXIq-arNl9tbSsnRDUhjMNpg-2ozeoZqLY6_2_W3DklLffQ> +X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduiedrudefiedgjedtucetufdoteggodetrfdotf + fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen + uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfffhffvufgtsehttdertd + erredtnecuhfhrohhmpedfufgvphhhucfivghnthhlvgdfuceomhgvsehjohhsvghphhhg + rdgtohhmqeenucggtffrrghtthgvrhhnpeekgffhuedttedvjeelkeffkeeuueelkeeite + dttefgleejledtueetgfeijefhueenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgr + mhepmhgrihhlfhhrohhmpehmvgesjhhoshgvphhhghdrtghomh +X-ME-Proxy: <xmx:X75NX6_-JLu-Eneod1EKsHyKkOEIRYFaAZQH4uBldS7GxRe_0WzDdw> + <xmx:X75NX7QCRm4jcdxXVZf_-SgheExet0j7JbTijuDCSZfm7EpG8GoXxg> + <xmx:X75NX8tKELWH7ryIzS_tU-Gi7kQXm8uw2XHfCRdbB4HJ08Y4s4Nheg> + <xmx:YL5NX4HEO3VR6t7Yz1-vKIi99YuUZV2t1wRHH3Tqv8f1bMEvEjAbAw> +Received: by mailuser.nyi.internal (Postfix, from userid 501) + id C51F714C00E5; Mon, 31 Aug 2020 23:22:07 -0400 (EDT) +X-Mailer: MessagingEngine.com Webmail Interface +User-Agent: Cyrus-JMAP/3.3.0-232-g4bdb081-fm-20200825.002-g4bdb081a +Mime-Version: 1.0 +Message-Id: <d78064d2-5441-4275-96d9-463a08dfd7a9@www.fastmail.com> +Date: Tue, 01 Sep 2020 03:21:47 +0000 +From: "Seph Gentle" <me@josephg.com> +To: ~euandreh/public-inbox@lists.sr.ht +Subject: Making a database +Content-Type: text/plain + +I've been thinking along similar lines, and am talking to some folks about building something on top of CRDTs. Modern CRDTs like automerge and y.js are just getting to the point where they're fast and small enough to work fine. I can see something really valuable that looks kind of like git, except where you can store arbitrary data. And live-sync peers - so if I make changes you see those changes live on your computer too - without (necessarily) needing a centralized server. + +Love to chat if you & others are keen - I/we are still in the process of figuring out what this thing will look like. But, I feel like the database piece is really core here too, and I like a lot of your thinking in this space. + +-Seph + +From me@jonas-schuermann.name Tue Sep 1 09:26:09 2020 +Received: from mout-p-102.mailbox.org (mout-p-102.mailbox.org [80.241.56.152]) + by mail-b.sr.ht (Postfix) with ESMTPS id CEBA2FF127 + for <~euandreh/public-inbox@lists.sr.ht>; Tue, 1 Sep 2020 09:26:08 +0000 (UTC) +Received: from smtp2.mailbox.org (smtp2.mailbox.org [IPv6:2001:67c:2050:105:465:1:2:0]) + (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) + (No client certificate requested) + by mout-p-102.mailbox.org (Postfix) with ESMTPS id 4BghWz4k2gzKmTx + for <~euandreh/public-inbox@lists.sr.ht>; Tue, 1 Sep 2020 11:26:07 +0200 (CEST) +X-Virus-Scanned: amavisd-new at heinlein-support.de +Received: from smtp2.mailbox.org ([80.241.60.241]) + by spamfilter04.heinlein-hosting.de (spamfilter04.heinlein-hosting.de [80.241.56.122]) (amavisd-new, port 10030) + with ESMTP id lwr5j4QAvkHQ for <~euandreh/public-inbox@lists.sr.ht>; + Tue, 1 Sep 2020 11:26:02 +0200 (CEST) +To: ~euandreh/public-inbox@lists.sr.ht +Subject: Re: The database I wish I had +From: =?UTF-8?Q?Jonas_Sch=c3=bcrmann?= <me@jonas-schuermann.name> +Message-ID: <e0b3b042-5eda-bcde-59a5-a5b61fecd2e5@jonas-schuermann.name> +Date: Tue, 1 Sep 2020 11:26:02 +0200 +MIME-Version: 1.0 +Content-Type: text/plain; charset=utf-8; format=flowed +Content-Language: en-US +X-MBO-SPAM-Probability: +X-Rspamd-Score: -1.02 / 15.00 / 15.00 +X-Rspamd-Queue-Id: 6015B17AB +X-Rspamd-UID: af255d +Content-Transfer-Encoding: quoted-printable + +Hello EuAndreh, + +I read your blog post with excitement because I'm currently on a very +similar quest. My goal is to bring back application data into the hands +of the users (and onto their devices) to replace centralized, +cloud-based data stores. I wrote down what I wanted in this blog post +[1]. + +Like you, I didn't find anything that seemed to work like I wanted, so I +decided to start exploring the problem myself. While you go for the +Git-like approach of three-way-merges, I am trying to model conflict +situations with completed categories. + +I can't say that I've come very far yet. I realized that properly +modelling conflicts with completed categories is a hard problem and +while I had some progress already, there isn't anything that I would +consider ready to be shared just yet (if you're curious you can look at +the source [2]). + +But through discussions on lobste.rs [3] and further research I built a +list of references, prior work and concepts in this space [4]. From your +blog post I even got a few more references that I just added :) Perhaps +this list is also useful for you. + +I wish you the best of luck with your endeavor and I hope to see further +posts about your progress on the blog :) When I have more time I will +also write more about what I've found and what's next. + +Cordially, Jonas Sch=C3=BCrmann + +[1]=20 +https://jonas-schuermann.name/projects/dvcs-for-structured-data/blog/2020= +-03-22-manifesto.html +[2]=20 +https://gitlab.com/MazeChaZer/dvcs-for-structured-data/-/blob/master/sour= +ce-code/EventSourcing.hs +[3] https://lobste.rs/s/qsehiu/i_want_decentralized_version_control_for +[4]=20 +https://jonas-schuermann.name/projects/dvcs-for-structured-data/notes/ref= +erences.html + +From eu@euandre.org Tue Sep 1 10:00:15 2020 +Received: from box.euandre.org (box.euandre.org [46.101.160.115]) + by mail-b.sr.ht (Postfix) with ESMTPS id 6FC96FF127 + for <~euandreh/public-inbox@lists.sr.ht>; Tue, 1 Sep 2020 10:00:14 +0000 (UTC) +Authentication-Results: mail-b.sr.ht; + dkim=pass (2048-bit key) header.d=euandre.org header.i=@euandre.org header.b=Fm0cCCjb +Received: from authenticated-user (box.euandre.org [46.101.160.115]) + (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) + (No client certificate requested) + by box.euandre.org (Postfix) with ESMTPSA id 82F61FCB45; + Tue, 1 Sep 2020 07:00:11 -0300 (-03) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=euandre.org; s=mail; + t=1598954412; bh=d3MsFaC9js1laWdI/WHjnTEpvQ9IMPR4/17eUYzh5Z8=; + h=From:To:Cc:Subject:In-Reply-To:References:Date:From; + b=Fm0cCCjbaKJpKhoedrPl0XWIYJujunIk9LXBI7OUyABb7hgxRqEkLmW93LCA60QAE + +aZMzweJYdcNtmRBMeQmlkFYwnZZq8L9OBlsqi+BOILAEA5jv8Ly8RMQ6ZV6kogRpY + rV0RwyBEvskQ0vao5a/IWLuCSSalXShntwRIy44liQYZqxjFwJTQEXPDeb+gvmYaz1 + 9aEvg5edL/OEGfo0VZpRk//f47V6a8o/3/uEaLpJAtEfCtQY1YYk5BdQGafCp5LrVR + fdp/6ju5YnM3yjM52MxTUvEjFpvK+FL/FSQDJ2DhdtbrH/EHHmNpImLP9WL1Bw9mNY + N2ar7lD/+9A6g== +From: EuAndreh <eu@euandre.org> +To: Seph Gentle <me@josephg.com>, ~euandreh/public-inbox@lists.sr.ht +Cc: +Subject: Re: Making a database +In-Reply-To: <d78064d2-5441-4275-96d9-463a08dfd7a9@www.fastmail.com> +References: <d78064d2-5441-4275-96d9-463a08dfd7a9@www.fastmail.com> +Date: Tue, 01 Sep 2020 06:39:19 -0300 +Message-ID: <87ft81zvi0.fsf@euandre.org> +MIME-Version: 1.0 +Content-Type: text/plain + +"Seph Gentle" <me@josephg.com> writes: + +> I've been thinking along similar lines, and am talking to some folks +> about building something on top of CRDTs. Modern CRDTs like automerge +> and y.js are just getting to the point where they're fast and small +> enough to work fine. I can see something really valuable that looks +> kind of like git, except where you can store arbitrary data. And +> live-sync peers - so if I make changes you see those changes live on +> your computer too - without (necessarily) needing a centralized +> server. +> +> Love to chat if you & others are keen - I/we are still in the process +> of figuring out what this thing will look like. But, I feel like the +> database piece is really core here too, and I like a lot of your +> thinking in this space. + +Thanks for the encouragement :) + +I've looked at CRDTs, but I haven't done a deep dive yet, because my +first impression was that they generally choose to limit the types of +data structures and operations that you can perform on them in favour of +being conflict-free. + +That's why I didn't mention it on my article directly, but I admit to +having it as pre-requisite to making an informed decision on the +subject. + +WDYT? + +From eu@euandre.org Tue Sep 1 11:00:18 2020 +Received: from box.euandre.org (box.euandre.org [46.101.160.115]) + by mail-b.sr.ht (Postfix) with ESMTPS id 7E055FF122 + for <~euandreh/public-inbox@lists.sr.ht>; Tue, 1 Sep 2020 11:00:17 +0000 (UTC) +Authentication-Results: mail-b.sr.ht; + dkim=pass (2048-bit key) header.d=euandre.org header.i=@euandre.org header.b=aexrWU2E +Received: from authenticated-user (box.euandre.org [46.101.160.115]) + (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) + (No client certificate requested) + by box.euandre.org (Postfix) with ESMTPSA id A3E67FD6B9; + Tue, 1 Sep 2020 08:00:14 -0300 (-03) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=euandre.org; s=mail; + t=1598958016; bh=RLLLMwMzFeXBwtfwXnEsHK/UTMXFBGSC5qgHaz/ZmOE=; + h=From:To:Cc:Subject:In-Reply-To:References:Date:From; + b=aexrWU2Ee5zBZdVCWHCoLentiEWPHDdm7cDXSVNCrsIO+wkZPDXWbT+YwGbX7pldB + rs//2AD5Bz0VOPByW33jD53/lmCCBeZ8eBjSLpfN2/w3sXHKPXh348R6/YLLRMSuwj + 9bCKuxK3X8qJ2SMS3fg8OGIzHgBq8rL59U0pFgCiV6WR5Z92mG80P8iogtSI8LzU/g + yBhu4sv4gJ3IlLhTiwX4HgAQkSoOCu6rdm5AbDdkwPwteUyy+7F2KlHc6eD7UTktU8 + ic7l2oVEyQrUYddwtZmUfJJCG9tffy5RDAyBO/TaLjVJpesdFEY5nRIG/Xm+1geH1j + fv++ZMqsLTgxQ== +From: EuAndreh <eu@euandre.org> +To: Jonas =?utf-8?Q?Sch=C3=BCrmann?= <me@jonas-schuermann.name>, + ~euandreh/public-inbox@lists.sr.ht +Cc: +Subject: Re: The database I wish I had +In-Reply-To: <e0b3b042-5eda-bcde-59a5-a5b61fecd2e5@jonas-schuermann.name> +References: <e0b3b042-5eda-bcde-59a5-a5b61fecd2e5@jonas-schuermann.name> +Date: Tue, 01 Sep 2020 07:26:47 -0300 +Message-ID: <87y2ltyeqg.fsf@euandre.org> +MIME-Version: 1.0 +Content-Type: text/plain; charset=utf-8 +Content-Transfer-Encoding: quoted-printable + +Jonas Sch=C3=BCrmann <me@jonas-schuermann.name> writes: + +> Hello EuAndreh, + +Hi there! + +> I read your blog post with excitement because I'm currently on a very +> similar quest. My goal is to bring back application data into the hands +> of the users (and onto their devices) to replace centralized, +> cloud-based data stores. I wrote down what I wanted in this blog post +> [1]. +> +> Like you, I didn't find anything that seemed to work like I wanted, so I +> decided to start exploring the problem myself. While you go for the +> Git-like approach of three-way-merges, I am trying to model conflict +> situations with completed categories. + +Hmmm, I haven't heard this term before. Are these [0] the complete +categories that you mean? + +[0]: https://en.wikipedia.org/wiki/Complete_category +=20 +> I can't say that I've come very far yet. I realized that properly +> modelling conflicts with completed categories is a hard problem and +> while I had some progress already, there isn't anything that I would +> consider ready to be shared just yet (if you're curious you can look at +> the source [2]). +> +> But through discussions on lobste.rs [3] and further research I built a +> list of references, prior work and concepts in this space [4]. From your +> blog post I even got a few more references that I just added :) Perhaps +> this list is also useful for you. + +Thanks for all the links, I'll try to use them to enrich my +understanding on the subject. + +From me@josephg.com Tue Sep 1 11:25:21 2020 +Received: from wout2-smtp.messagingengine.com (wout2-smtp.messagingengine.com [64.147.123.25]) + by mail-b.sr.ht (Postfix) with ESMTPS id 50AC3FF122 + for <~euandreh/public-inbox@lists.sr.ht>; Tue, 1 Sep 2020 11:25:21 +0000 (UTC) +Authentication-Results: mail-b.sr.ht; + dkim=pass (1024-bit key) header.d=josephg.com header.i=@josephg.com header.b=O0rYtvTA; + dkim=fail reason="key not found in DNS" (0-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=f1mPF/1x +Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) + by mailout.west.internal (Postfix) with ESMTP id E4956E77; + Tue, 1 Sep 2020 07:25:19 -0400 (EDT) +Received: from imap35 ([10.202.2.85]) + by compute1.internal (MEProxy); Tue, 01 Sep 2020 07:25:20 -0400 +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=josephg.com; h= + mime-version:message-id:in-reply-to:references:date:from:to + :subject:content-type:content-transfer-encoding; s=mesmtp; bh=BO + dAgwWsH67HEzbojnfogot17D0WKCAzyItvgTLmJCw=; b=O0rYtvTAoINKZTJObn + vUnmONdChl7CPOEklbKnqUQ+Sk8qaAoCe0YyqwTLq0YZywk49u56Zwff5QaSJfLm + SYA9bd2RUEagih/gPTR4BHJarqfWSHQeUsHsvMmVtj4w8jrJt261gcmh1Git8Jkt + lJSUIZ9tpOecI/nFsnhxCleA0= +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= + messagingengine.com; h=content-transfer-encoding:content-type + :date:from:in-reply-to:message-id:mime-version:references + :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender + :x-sasl-enc; s=fm3; bh=BOdAgwWsH67HEzbojnfogot17D0WKCAzyItvgTLmJ + Cw=; b=f1mPF/1xX+9SrbduF4hyrnDZlojUnotl/ZuV6F+yuzbQlOHFTtLgMSJ5u + ghKQ7iTGJdFGqxzM87y1JJLCWb6BZjbRw2TjhDK+z24JeMwOVOpZgDsZKZ4vWlHI + x6C8Ca/TnlF8Glsyvepoczx5SP6kbYCOe/JrBGp3gTIfTGohNHuxCsxxHvW3gn35 + hQLIa42DMt3y1p0q2rl4ltdrCYpCzQVvfbhKssX3AmCkwndYDhbP6l4jLnWH6jeM + oBw77aR3keR94fhUnSCsnlX5ZSljRkIecR7Z9sGsR8KeB5j72A/ND3rncJyOOQ/e + +DXRIAj+KVgqvbY2AimyrweAA9mQg== +X-ME-Sender: <xms:nS9OX8TbEfFCRuUyS6F4hwgBx8v03SzB2Ggzuy63tMjyFVg-UrVkNQ> + <xme:nS9OX5zGsoiG5l8lthnVpFgOEb1gqC7oeFR2NwTbn4uDQFjSZ-ZvMZeo0aSeRF8GV + exRk5tGsHuiPRtDjw> +X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduiedrudefjedggedtucetufdoteggodetrfdotf + fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen + uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfgjfhffhffvufgtgfesth + hqredtreerjeenucfhrhhomhepfdfuvghphhcuifgvnhhtlhgvfdcuoehmvgesjhhoshgv + phhhghdrtghomheqnecuggftrfgrthhtvghrnhepkeegtdfhgfdvheetveetleejgfdvhf + egtdegfedvhfegveffgfdthfehgedukeegnecuffhomhgrihhnpehslhgrtghkrdgtohhm + necuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepmhgvse + hjohhsvghphhhgrdgtohhm +X-ME-Proxy: <xmx:nS9OX50QvymIMNNSVjvYuwnELZacWmVFdJvm4gYZ7wZ28Ysp8XeqaA> + <xmx:nS9OXwCN5irAMSLyp8Oejw_bDz4dedc6-GQj4ThImfuyCt5P3RUUhA> + <xmx:nS9OX1jNG59H8yiSSKrEdc830ToS-wnjIO8Dqrr2hhrFB4opADJCoQ> + <xmx:ny9OX6tPG_S_xI2Z3IHEmzknqb1-PlAmuX_wv6n9qglqXIcgp0MqSQ> +Received: by mailuser.nyi.internal (Postfix, from userid 501) + id 72BAB14C034C; Tue, 1 Sep 2020 07:25:17 -0400 (EDT) +X-Mailer: MessagingEngine.com Webmail Interface +User-Agent: Cyrus-JMAP/3.3.0-248-gcd102cb-fm-20200901.001-gcd102cb9 +Mime-Version: 1.0 +Message-Id: <9e513706-7ebc-4a40-8e5e-a5f94d84bc77@www.fastmail.com> +In-Reply-To: <87ft81zvi0.fsf@euandre.org> +References: <d78064d2-5441-4275-96d9-463a08dfd7a9@www.fastmail.com> + <87ft81zvi0.fsf@euandre.org> +Date: Tue, 01 Sep 2020 21:24:55 +1000 +From: "Seph Gentle" <me@josephg.com> +To: EuAndreh <eu@euandre.org>, ~euandreh/public-inbox@lists.sr.ht +Subject: Re: Making a database +Content-Type: text/plain;charset=utf-8 +Content-Transfer-Encoding: quoted-printable + +On Tue, Sep 1, 2020, at 7:39 PM, EuAndreh wrote: +> "Seph Gentle" <me@josephg.com> writes: +>=20 +> > I've been thinking along similar lines, and am talking to some folks= + +> > about building something on top of CRDTs. Modern CRDTs like automerg= +e +> > and y.js are just getting to the point where they're fast and small +> > enough to work fine. I can see something really valuable that looks +> > kind of like git, except where you can store arbitrary data. And +> > live-sync peers - so if I make changes you see those changes live on= + +> > your computer too - without (necessarily) needing a centralized +> > server. +> > +> > Love to chat if you & others are keen - I/we are still in the proces= +s +> > of figuring out what this thing will look like. But, I feel like the= + +> > database piece is really core here too, and I like a lot of your +> > thinking in this space. +>=20 +> Thanks for the encouragement :) +>=20 +> I've looked at CRDTs, but I haven't done a deep dive yet, because my +> first impression was that they generally choose to limit the types of +> data structures and operations that you can perform on them in favour = +of +> being conflict-free. +>=20 +> That's why I didn't mention it on my article directly, but I admit to +> having it as pre-requisite to making an informed decision on the +> subject. +>=20 +> WDYT? + +Well, they=E2=80=99re conflict-free in the sense that (unlike Postgres/f= +oundationdb/etc) the database doesn=E2=80=99t reject conflicting concurr= +ent updates. Which is a property you need if you=E2=80=99re going to com= +mit local writes while you=E2=80=99re offline. And if I=E2=80=99m readin= +g your blog post right it sounds like that=E2=80=99s what you=E2=80=99re= + going for. + +You kinda gave a good description of them in your post already. They wor= +k by requiring all peers to resolve concurrent edits to an object the sa= +me way. That could be last-writer-wins (based on time stamps like you sa= +id in your article). Or it could store all conflicting versions together= + to be resolved by the next reader (Eg riak). Or it could do something m= +ore clever (like merge changes using automerge or y.js or the like). Git= + is sort of a CRDT - it=E2=80=99s just it=E2=80=99s 3 way merge algorith= +m is a bit of a dogs breakfast as far as these things go. + +Anyway, happy to chat about this stuff in more detail if you wanna zoom = +or something. Or hop into the automerge slack (with pvh from ink&switch,= + and Martin Kleppmann, and others) and we can talk about this stuff - ht= +tps://join.slack.com/t/automerge/shared_invite/zt-e4p3760n-kKh7r3KRH1Yww= +NfiZM8ktw + +As I said - I=E2=80=99m looking to build something like this too and it=E2= +=80=99d be good to compare notes on our approaches! + +From rnewman@twinql.com Tue Sep 1 15:46:34 2020 +Received: from a27-232.smtp-out.us-west-2.amazonses.com (a27-232.smtp-out.us-west-2.amazonses.com [54.240.27.232]) + by mail-b.sr.ht (Postfix) with ESMTPS id 02A70FF0E3 + for <~euandreh/public-inbox@lists.sr.ht>; Tue, 1 Sep 2020 15:46:33 +0000 (UTC) +Authentication-Results: mail-b.sr.ht; + dkim=pass (1024-bit key) header.d=twinql.com header.i=@twinql.com header.b=cNOZcfp4; + dkim=pass (1024-bit key) header.d=amazonses.com header.i=@amazonses.com header.b=Hg/JwA02 +DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; + s=qvs2k2sd2ivtrflyu2nswmravuqizjml; d=twinql.com; t=1598975192; + h=Subject:From:To:Date:Mime-Version:Content-Type:Content-Transfer-Encoding:References:Message-Id; + bh=h5tYwgdiFvKfB58ME/qKUDlZfMWWDjBD1us0jqb+obo=; + b=cNOZcfp4gLnUosw6qaEpaunrxtpsBSNsQgvh/C3StK1VPirNXCoHKVx5ZtOO/szG + PdOufKL9NM0bQu6OiYFBxUdp2fuZ2FGwONd2nzGBOGcMN4V2aSOuMAGcuClGe2Kf+SM + xfSfiY2iyxEUtlFoNU9qfvTtJ5Wutm+/EkIRuJ+g= +DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; + s=hsbnp7p3ensaochzwyq5wwmceodymuwv; d=amazonses.com; t=1598975192; + h=Subject:From:To:Date:Mime-Version:Content-Type:Content-Transfer-Encoding:References:Message-Id:Feedback-ID; + bh=h5tYwgdiFvKfB58ME/qKUDlZfMWWDjBD1us0jqb+obo=; + b=Hg/JwA02/i8iV50zfzagoO6qkIy3fWh3xaS3Ea/Mbew1Ve1B70ilQdJIrthnSRCF + S50Nn0YzMQlyzXXd8rCHSfwTcqDUYV14hdN+GPbGPF+O+6w/X7KG6PxWnYTE/5w/cE9 + 5HAQ9xp3FdpTqa2RvVsLtEJVweoEtHmNR5319VRs= +Subject: Re: The database I wish I had +From: =?UTF-8?Q?Richard_Newman?= <rnewman@twinql.com> +To: =?UTF-8?Q?=7Eeuandreh/public-inbox=40lists=2Esr=2Eht?= + <~euandreh/public-inbox@lists.sr.ht> +Date: Tue, 1 Sep 2020 15:46:31 +0000 +Mime-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: quoted-printable +References: <7454DA45-C5A4-4A54-8D02-1F3BD37726E9@twinql.com> +X-Priority: 3 (Normal) +X-Mailer: Amazon WorkMail +Thread-Index: AQHWgHcPJ4udfXYrRaOwhoO7EcsDJQ== +Thread-Topic: The database I wish I had +X-Original-Mailer: Apple Mail (2.3445.104.14) +X-Wm-Sent-Timestamp: 1598975191 +Message-ID: <010101744a592b75-1dce9281-f0b8-4226-9d50-fd2c7901fa72-000000@us-west-2.amazonses.com> +X-SES-Outgoing: 2020.09.01-54.240.27.232 +Feedback-ID: 1.us-west-2.An468LAV0jCjQDrDLvlZjeAthld7qrhZr+vow8irkvU=:AmazonSES + +Hello! Original author of Mentat here.=0D=0A=0D=0AThe use of SQLite for M= +entat was intended to be an accelerant for development: by leveraging SQL= +ite we could ignore large parts of the tricky problem of getting bits on = +disk across platforms, and also get things like transactionality and full= +-text search for free.=0D=0A=0D=0AWe used the same leverage trick when we= + built the first proof of concept using Clojure and ClojureScript =E2=80=94= + we could run it in a JVM locally, compare to Datomic, and test in Node, = +and entirely avoid having to build parsers and lexers. This allowed us to= + focus on the problem of representing a datom/log/index data model in SQL= +ite.=0D=0A=0D=0AThe downside of using SQLite was a large amount of time w= +riting code to compile queries =E2=80=94 which were written in much the s= +ame language as Datomic=E2=80=99s =E2=80=94 into SQL for execution. That = +Datomic compatibility also pushed us towards building an EDN parser in Ru= +st, and we found some under-specified parts of Datomic, too.=0D=0A=0D=0AP= +ortability to the web itself was not an initial goal, but it was somethin= +g we thought about. With adequate resourcing, and the benefit of hindsigh= +t, we would have instead built a chunk-based backing store abstraction, u= +sing something like rkv/LMDB for embedded use and indexeddb on the web. W= +e did not feel like we had the engineering capacity to take on building a= + truly portable, battle-tested storage layer like SQLite=E2=80=99s *and* = +a bits-on-disk query executor =E2=80=94 I=E2=80=99ve done it in the past,= + and it takes a lot of time and is hard to get right.=0D=0A=0D=0AMentat a= +ttempted to address a number of things that you didn=E2=80=99t cover in y= +our brief post. A robust definition of identity, end-to-end encryption, a= +nd evolution of schema (even simple schema like cardinality and uniquenes= +s constraints on attributes, which are the foundation of identity) are th= +ree such. Schema evolution becomes tremendously important when you have m= +ultiple teams using the same storage, or a long-lived product =E2=80=94 t= +he more useful the database is, the more likely you are to need this feat= +ure!=0D=0A=0D=0AThose three things cascade into the problem of synchroniz= +ation, which was my focus for a number of years.=0D=0A=0D=0ASynchronizati= +on is a hard problem. Doing it without server assistance is even harder, = +but is necessary for a portable and secure product. Synchronization of ge= +neric data =E2=80=94 that is, data where the synchronization tool itself = +doesn=E2=80=99t have its own model of the world =E2=80=94 is extremely tr= +icky: everything from identifiers through to basic constraints needs to b= +e aware of the possibility of change, and naturally it=E2=80=99s an itera= +tive and distributed situation. Most of the approaches I found rely on a = +server and are for systems that omit schema, and are typically incomplete= +: e.g., detecting conflicts on individual attributes, leading to nonsensi= +cal results, and offload to the developer the problem of separately handl= +ing stable identifiers.=0D=0A=0D=0AI still have a persistent itch to star= +t with a clean slate and solve this problem. No large company has the rig= +ht incentives to solve the much harder problems faced by a truly secure s= +tore, most hobbyists lack the understanding of the importance of managing= + change and concurrent collaboration over time, and the current trend is = +towards using =E2=80=98dumb=E2=80=99 object/key-value stores or specialis= +t systems like time-series databases, so I think it=E2=80=99s unlikely th= +at anybody else is going to tackle all three in the same solution. Here=E2= +=80=99s hoping! + +From eu@euandre.org Tue Sep 1 18:00:18 2020 +Received: from box.euandre.org (box.euandre.org [46.101.160.115]) + by mail-b.sr.ht (Postfix) with ESMTPS id CD25BFF0C9 + for <~euandreh/public-inbox@lists.sr.ht>; Tue, 1 Sep 2020 18:00:17 +0000 (UTC) +Authentication-Results: mail-b.sr.ht; + dkim=pass (2048-bit key) header.d=euandre.org header.i=@euandre.org header.b=q+qyYbsV +Received: from authenticated-user (box.euandre.org [46.101.160.115]) + (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) + (No client certificate requested) + by box.euandre.org (Postfix) with ESMTPSA id F050CFCB45; + Tue, 1 Sep 2020 15:00:14 -0300 (-03) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=euandre.org; s=mail; + t=1598983216; bh=HeMLnOZIeRgOrLdGepra14BxoBWpvE7fXfdvB9MFFjw=; + h=From:To:Cc:Subject:In-Reply-To:References:Date:From; + b=q+qyYbsVLZrz3+swol7Y+Sc59kNUZkM1tSaXTcgLVIOp3CSJ+96XlP7i6oKZdeubH + fZIdw2fmFCMQotaXCnLJKBUNauX5Jrvt+4t2TrujIie5i5o/VSp08DxtxyWAmUWqdt + nPZrAand3styW0vvpWFYCZ7RjJKs4Ujek2G4H713X/eDOTooKalFNIXZ+A7t/xzq+9 + mYojhphqHBDFQeqNKs9500YqetgTg4hsYfqRmGHzOs5ADw1dkQegQhjduodWvj2SpI + YeUu4OoXYDYUR9o+6VkXV+KtC4zDnoXyvgdSfHaLrahnkfiCBgg1Y6Kmcxy8iKnXjP + q5baz6m+cXC9Q== +From: EuAndreh <eu@euandre.org> +To: Richard Newman <rnewman@twinql.com>, + "~euandreh/public-inbox@lists.sr.ht" <~euandreh/public-inbox@lists.sr.ht> +Cc: +Subject: Re: The database I wish I had +In-Reply-To: <010101744a592b75-1dce9281-f0b8-4226-9d50-fd2c7901fa72-000000@us-west-2.amazonses.com> +References: <7454DA45-C5A4-4A54-8D02-1F3BD37726E9@twinql.com> + <010101744a592b75-1dce9281-f0b8-4226-9d50-fd2c7901fa72-000000@us-west-2.amazonses.com> +Date: Tue, 01 Sep 2020 14:50:07 -0300 +Message-ID: <87sgc1xu7k.fsf@euandre.org> +MIME-Version: 1.0 +Content-Type: text/plain; charset=utf-8 +Content-Transfer-Encoding: quoted-printable + +Richard Newman <rnewman@twinql.com> writes: + +> Hello! Original author of Mentat here. + +Hi there! You wouldn't be suprised if I told I recognize you by name, +would you? + +Great job on Mentat, and please write more installments on "Thinking +about Syncing" if possible =E2=98=BA=EF=B8=8F + +> The use of SQLite for Mentat was intended to be an accelerant for +> development: by leveraging SQLite we could ignore large parts of the +> tricky problem of getting bits on disk across platforms, and also get +> things like transactionality and full-text search for free. + +For your goal of supporting other teams working on Firefox, SQLite is a +perfectly reasonable choice. + +> We used the same leverage trick when we built the first proof of +> concept using Clojure and ClojureScript =E2=80=94 we could run it in a JVM +> locally, compare to Datomic, and test in Node, and entirely avoid +> having to build parsers and lexers. This allowed us to focus on the +> problem of representing a datom/log/index data model in SQLite.=20 +> +> The downside of using SQLite was a large amount of time writing code +> to compile queries =E2=80=94 which were written in much the same language= + as +> Datomic=E2=80=99s =E2=80=94 into SQL for execution. That Datomic compatib= +ility also +> pushed us towards building an EDN parser in Rust, and we found some +> under-specified parts of Datomic, too.=20 + +Hmm, I'd be interested in learning more about those under-specified +pitfalls to try to avoid them if possible.=20 + +> Portability to the web itself was not an initial goal, but it was +> something we thought about. With adequate resourcing, and the benefit +> of hindsight, we would have instead built a chunk-based backing store +> abstraction, using something like rkv/LMDB for embedded use and +> indexeddb on the web. We did not feel like we had the engineering +> capacity to take on building a truly portable, battle-tested storage +> layer like SQLite=E2=80=99s *and* a bits-on-disk query executor =E2=80=94= + I=E2=80=99ve done it +> in the past, and it takes a lot of time and is hard to get right.=20 + +I don't expect to match SQLite's robustness for quite some time, and +having plugable storage APIs makes me consider having a SQLite storage +API to have as a correctness reference. + +What do you mean by "chunk-based backing store abstraction"? Could you +elaborate further? + +> Mentat attempted to address a number of things that you didn=E2=80=99t co= +ver +> in your brief post. A robust definition of identity, end-to-end +> encryption, and evolution of schema (even simple schema like +> cardinality and uniqueness constraints on attributes, which are the +> foundation of identity) are three such. Schema evolution becomes +> tremendously important when you have multiple teams using the same +> storage, or a long-lived product =E2=80=94 the more useful the database i= +s, +> the more likely you are to need this feature!=20 + +What do you mean by "identity"? As in ":db.unique/value" and +":db.unique/identity"? Couldn't you just use UUIDs for +":db.unique/identity" values? + +For schema evolution wouldn't the "Grow your schema, and never break +it"[0] approach be enough? + +On encryption, my first approach would be to see if git-remote-gcrypt[1] +functionality can be translated, and have the local database be ignorant +of all encryption whatsoever. However, I haven't explored it enough to +be able to declare it to be possible, so I would welcome a more mature +view on the subject. + +[0]: https://blog.datomic.com/2017/01/the-ten-rules-of-schema-growth.html=20 +[1]: https://spwhitton.name/tech/code/git-remote-gcrypt/ + +> Those three things cascade into the problem of synchronization, which +> was my focus for a number of years.=20 +> +> Synchronization is a hard problem. Doing it without server assistance +> is even harder, but is necessary for a portable and secure product. +> Synchronization of generic data =E2=80=94 that is, data where the +> synchronization tool itself doesn=E2=80=99t have its own model of the wor= +ld =E2=80=94 +> is extremely tricky: everything from identifiers through to basic +> constraints needs to be aware of the possibility of change, and +> naturally it=E2=80=99s an iterative and distributed situation. Most of the +> approaches I found rely on a server and are for systems that omit +> schema, and are typically incomplete: e.g., detecting conflicts on +> individual attributes, leading to nonsensical results, and offload to +> the developer the problem of separately handling stable identifiers.=20 + +If sound and robust, what's the downside of detecting conflicts on +individual attributes? + +If an Datomic-like entity with 10 attributes has 4 of them change by +different database instances, but only 1 change is on the same +attribute, would this type of detection offer more granularity instead? + +Wouldn't the 5-tuple be the equivalent of a single line of code on git, +and aggregates are built on top of it? + +From eu@euandre.org Tue Sep 1 18:00:22 2020 +Received: from box.euandre.org (box.euandre.org [46.101.160.115]) + by mail-b.sr.ht (Postfix) with ESMTPS id 097B1FF0C9 + for <~euandreh/public-inbox@lists.sr.ht>; Tue, 1 Sep 2020 18:00:22 +0000 (UTC) +Authentication-Results: mail-b.sr.ht; + dkim=pass (2048-bit key) header.d=euandre.org header.i=@euandre.org header.b=bsr8vYSW +Received: from authenticated-user (box.euandre.org [46.101.160.115]) + (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) + (No client certificate requested) + by box.euandre.org (Postfix) with ESMTPSA id C1FCCFD6BB; + Tue, 1 Sep 2020 15:00:19 -0300 (-03) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=euandre.org; s=mail; + t=1598983221; bh=VxPcwqhwGX+tAOR9QBGKCYzKImUIi+BqaSfmrBhsR6g=; + h=From:To:Cc:Subject:In-Reply-To:References:Date:From; + b=bsr8vYSWGIarl0IY64Ew+GLFGsQhZe971tKRr72HG9/UJru3FMjjA0VXK5LmyRGAh + ZjamXQpzM0biS0NNhY6mhfrvHXGKaGr19E6I5NDokNZFc/T/hXD1qMzY4fgEkDmayl + fteqI7T//DT06yseOXOkfrKzNCDWKmSmlsOo9PZgSsAicwEsqxb/UqehXN6ON9vpG4 + VCXsOW8y8/aQGR7/WL2M9rkTd2cf6p/csDiUTkGf8YMFu8Qz/u5MgzY/14LCShXzfL + S+iz36YnA/kIu1RjrOzR22rPPcQRfZE1MyS9B+h9QeHHWR7A1SnsfE8WnlhBCCRxU7 + ZcKgKfiihKPxQ== +From: EuAndreh <eu@euandre.org> +To: Seph Gentle <me@josephg.com>, ~euandreh/public-inbox@lists.sr.ht +Cc: +Subject: Re: Making a database +In-Reply-To: <9e513706-7ebc-4a40-8e5e-a5f94d84bc77@www.fastmail.com> +References: <d78064d2-5441-4275-96d9-463a08dfd7a9@www.fastmail.com> + <87ft81zvi0.fsf@euandre.org> + <9e513706-7ebc-4a40-8e5e-a5f94d84bc77@www.fastmail.com> +Date: Tue, 01 Sep 2020 14:56:23 -0300 +Message-ID: <87pn75xtx4.fsf@euandre.org> +MIME-Version: 1.0 +Content-Type: text/plain; charset=utf-8 +Content-Transfer-Encoding: quoted-printable + +> You kinda gave a good description of them in your post already. They +> work by requiring all peers to resolve concurrent edits to an object +> the same way. That could be last-writer-wins (based on time stamps +> like you said in your article). Or it could store all conflicting +> versions together to be resolved by the next reader (Eg riak). Or it +> could do something more clever (like merge changes using automerge or +> y.js or the like). Git is sort of a CRDT - it=E2=80=99s just it=E2=80=99s= + 3 way merge +> algorithm is a bit of a dogs breakfast as far as these things go.=20 + +My ignorace is really on the applicability of it. The "store all +conflicting versions together or to be resolved by the next reader" +sounds like what I'm thinking about, so maybe I'm trying to re-invent +CRDTs after all. + +I've just bumped the priority of this topic to better understand those +nuances.=20 + +From rnewman@twinql.com Tue Sep 1 19:30:22 2020 +Received: from a27-132.smtp-out.us-west-2.amazonses.com (a27-132.smtp-out.us-west-2.amazonses.com [54.240.27.132]) + by mail-b.sr.ht (Postfix) with ESMTPS id 67FA4FF130 + for <~euandreh/public-inbox@lists.sr.ht>; Tue, 1 Sep 2020 19:30:21 +0000 (UTC) +Authentication-Results: mail-b.sr.ht; + dkim=pass (1024-bit key) header.d=twinql.com header.i=@twinql.com header.b=HXFRpUpY; + dkim=pass (1024-bit key) header.d=amazonses.com header.i=@amazonses.com header.b=Fknn4Oqd +DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; + s=qvs2k2sd2ivtrflyu2nswmravuqizjml; d=twinql.com; t=1598988619; + h=Subject:From:To:Cc:Date:Mime-Version:Content-Type:Content-Transfer-Encoding:In-Reply-To:References:Message-Id; + bh=Y4wqQqAQDJ1Sa/E5OhrtvJ+Yk3Uij2laWmG/aD8wLfw=; + b=HXFRpUpY+Dt9MJW6Z60km/fTpPfIYak8NKLDvv6UeroaBfvWClO0loOtSYksmT88 + dWeTwau14008g4YQwDwEY8msngXXDx/9W05ArClEsNXDygcmOYea3eHhqVMcvS/dtuA + QYV4fjMoIj9QsGiHAP/QKjampkgK6nHx+Qfhlti4= +DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; + s=hsbnp7p3ensaochzwyq5wwmceodymuwv; d=amazonses.com; t=1598988619; + h=Subject:From:To:Cc:Date:Mime-Version:Content-Type:Content-Transfer-Encoding:In-Reply-To:References:Message-Id:Feedback-ID; + bh=Y4wqQqAQDJ1Sa/E5OhrtvJ+Yk3Uij2laWmG/aD8wLfw=; + b=Fknn4Oqdb29M5jY4pJV5V9gX2yK3n+WkgnRWmjQl46DdAzcaL4dHoZvIbk8cQxSt + v6z1Rdu/S1IvHVSIHLYloebUmPBBfmLzEksz9sh5yCGO4IJhfdIEk0CPgPvNNkXC7as + DpyhbHSQzUMFT1EBYwmgHNK/DQMSwe+2Jy7SpfPg= +Subject: Re: The database I wish I had +From: =?UTF-8?Q?Richard_Newman?= <rnewman@twinql.com> +To: =?UTF-8?Q?EuAndreh?= <eu@euandre.org> +Cc: =?UTF-8?Q?=7Eeuandreh/public-inbox=40lists=2Esr=2Eht?= + <~euandreh/public-inbox@lists.sr.ht> +Date: Tue, 1 Sep 2020 19:30:19 +0000 +Mime-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: quoted-printable +In-Reply-To: <87sgc1xu7k.fsf@euandre.org> +References: <7454DA45-C5A4-4A54-8D02-1F3BD37726E9@twinql.com> + <010101744a592b75-1dce9281-f0b8-4226-9d50-fd2c7901fa72-000000@us-west-2.amazonses.com> + <87sgc1xu7k.fsf@euandre.org> + <9652AAE0-10DF-4D13-ABE8-B30667BE27DD@twinql.com> +X-Priority: 3 (Normal) +X-Mailer: Amazon WorkMail +Thread-Index: AQHWgHcPJ4udfXYrRaOwhoO7EcsDJQAErSAlAAfQ7+g= +Thread-Topic: The database I wish I had +X-Original-Mailer: Apple Mail (2.3445.104.14) +X-Wm-Sent-Timestamp: 1598988618 +Message-ID: <010101744b260d62-1bb508ab-647f-4877-bdf9-f988667d56e1-000000@us-west-2.amazonses.com> +X-SES-Outgoing: 2020.09.01-54.240.27.132 +Feedback-ID: 1.us-west-2.An468LAV0jCjQDrDLvlZjeAthld7qrhZr+vow8irkvU=:AmazonSES + +> Hi there! You wouldn't be suprised if I told I recognize you by name,=0D= +=0A> would you=3F=0D=0A>=20=0D=0A> Great job on Mentat, and please write = +more installments on "Thinking=0D=0A> about Syncing" if possible =E2=98=BA= +=EF=B8=8F=0D=0A=0D=0AOne day I do hope to return to them! Thank you for t= +he kind words.=0D=0A=0D=0A=0D=0A> Hmm, I'd be interested in learning more= + about those under-specified=0D=0A> pitfalls to try to avoid them if poss= +ible.=20=0D=0A=0D=0AMostly the usual stuff you get when a thing is define= +d by its implementation:=0D=0A=0D=0Ahttps://github.com/mozilla/mentat/pul= +l/665=0D=0Ahttps://github.com/mozilla/mentat/issues/412=0D=0A=0D=0AThere = +are probably more if you poke around the issue tracker.=0D=0A=0D=0A=0D=0A= +> What do you mean by "chunk-based backing store abstraction"=3F Could yo= +u=0D=0A> elaborate further=3F=0D=0A=0D=0AI'm familiar with the way Allegr= +oGraph does storage, and fairly familiar with how Datomic does it.=0D=0A=0D= +=0AIn short, a datom store like this consists of the log (all datoms, wit= +h retractions, ordered by ID), and some current state (which Mentat calle= +d `datoms`), which consists of all current asserted datoms.=0D=0A=0D=0ATh= +e current state is typically stored in a number of redundant permutations= + to support querying. In AllegroGraph (subject, predicate, object, graph,= + id), it would maintain multiple indices: `spogi`, `posgi`, etc. Each of = +these store the same datoms in different orders.=0D=0A=0D=0AIf we have th= +e data=0D=0A=0D=0A```=0D=0A :bob foaf:knows :alice=0D=0A :alice foaf:na= +me "Alice"=0D=0A :bob foaf:name "Bob"=0D=0A :bob foaf:age 26=0D=0A :al= +ice foaf:age 30=0D=0A```=0D=0A=0D=0Ait might appear in the `spogi` index = +as:=0D=0A=0D=0A```=0D=0A :alice foaf:age 30=0D=0A :alice foaf:name "Ali= +ce"=0D=0A :bob foaf:age 26=0D=0A :bob foaf:knows :alice=0D=0A :bob foa= +f:name "Bob"=0D=0A```=0D=0A=0D=0Aand in the `posgi` index as:=0D=0A=0D=0A= +```=0D=0A foaf:age 26 :bob=0D=0A foaf:age 30 :alice=20=0D=0A foaf:name= + "Alice" :alice=20=0D=0A foaf:name "Bob" :bob=20=0D=0A foaf:knows :alic= +e :bob=20=0D=0A```=0D=0A=0D=0ANaturally these would be using efficient (i= +nterned) sortable representations of each term, with rows of fixed size, = +memory mapped on disk.=0D=0A=0D=0AThis allows you to implement query patt= +ern execution as an index walk.=0D=0A=0D=0A`SELECT =3Fs WHERE =3Fs foaf:n= +ame 'Alice'`=0D=0A=0D=0Afor example turns into a walk on `posgi`: you fin= +d the spot that corresponds to `foaf:name`, find the next place which is = +`Alice`, and then read out the `s`.=0D=0A=0D=0AWhen you insert data, you = +typically don't want to immediately regenerate all of these indices. Inst= +ead you accrue new *chunks*, which are just all of the datoms asserted in= + some window of time, or within a particular transaction. The query engin= +e walks all of the relevant chunks and merges the results together. One c= +hunk might have Bob's name, and one might have Alice's.=0D=0A=0D=0ATypica= +lly _metaindices_ are used to help the query engine navigate which chunks= + to use.=0D=0A=0D=0AAllegroGraph periodically merges chunks together: a f= +ully merged store is faster to query. Datomic goes the other way: it buil= +ds new merged chunks and keeps the old ones around, because those old chu= +nks represent the state of the database at a point in time! This is how y= +ou efficiently query past states in Datomic: it trades space in your Cass= +andra cluster for the ability to grab the right chunks for any query.=0D=0A= +=0D=0AYou can easily imagine that any system that allows you to put a nam= +e on some region of data can be used to store these chunks, and their met= +aindices, and the log itself. The only requirement is that the chunk stor= +e be durable, and ideally replicated and relatively fast to read from.=0D= +=0A=0D=0A=0D=0A> What do you mean by "identity"=3F As in ":db.unique/valu= +e" and=0D=0A> ":db.unique/identity"=3F Couldn't you just use UUIDs for=0D= +=0A> ":db.unique/identity" values=3F=0D=0A=0D=0AIf on Computer A I visit = +https://aclu.org, and on Computer B I visit https://aclu.org, my two comp= +uters will each give the same history item a different internal identifie= +r =E2=80=94 1234 and 4567, say.=0D=0A=0D=0AWhen I sync them, I need the d= +ata store to know that the visits I made to each page _were visits to the= + same thing_.=0D=0A=0D=0AThere is more value to it than that (e.g., Datom= +ic's "lookup refs"), but it's important to recognize that identity is _do= +main specific_, and it is often _compound_ and even contextual or conditi= +onal.=0D=0A=0D=0AA large part of synchronization, and a large part of dat= +a modeling in general, consists of figuring out exactly what it is you're= + trying to record data about, and how to identify it. It's tremendously i= +mportant in NoSQL systems because there is no foreign key constraint to h= +elp you, and changing an identifier later is difficult. It's just as impo= +rtant in syncing systems because these identifiers are the only thing you= + have to help you merge data =E2=80=94 to zip two graphs together.=0D=0A=0D= +=0A=0D=0A> For schema evolution wouldn't the "Grow your schema, and never= + break=0D=0A> it"[0] approach be enough=3F=0D=0A=0D=0AIt definitely helps= +; this is how teams generally try to work with DynamoDB, for example, tho= +ugh there the schema is implicit rather than explict.=0D=0A=0D=0AHowever,= + you inevitably have data migrations to worry about=E2=80=A6 not to menti= +on that syncing makes this a distributed system, and so you cannot easily= + coordinate schema changes in more than one place, and they will race wit= +h new data!=0D=0A=0D=0AThere's also the argument that coordinating change= + in consumers and engineers is sometimes harder than evolving a vocabular= +y!=0D=0A=0D=0AIt's a complicated topic.=0D=0A=0D=0A=0D=0A> On encryption,= + my first approach would be to see if git-remote-gcrypt[1]=0D=0A> functio= +nality can be translated, and have the local database be ignorant=0D=0A> = +of all encryption whatsoever. However, I haven't explored it enough to=0D= +=0A> be able to declare it to be possible, so I would welcome a more matu= +re=0D=0A> view on the subject.=0D=0A=0D=0AI have been relatively happy wi= +th SQLCipher, and I think the idea of block-level encryption is a good pl= +ace to put the abstraction. The thing you lose with any of these mechanis= +ms is, by definition, the ability for a central entity to help you with c= +onflict resolution. The best you can get is ordering of opaque chunks and= + collision detection.=0D=0A=0D=0A=0D=0A> If sound and robust, what's the = +downside of detecting conflicts on=0D=0A> individual attributes=3F=0D=0A=0D= +=0AIt relies on extensive reification to achieve the property that we wan= +t: that each semantically conflicting write results in a conflicting attr= +ibute change.=0D=0A=0D=0AAs a contrived example example: notes have both = +a title and a body.=0D=0A=0D=0AA user has two computers.=0D=0A=0D=0AOn Co= +mputer A, the user decides to change their first note to something else, = +updating both the title and body.=0D=0A=0D=0AOn Computer B, they do the s= +ame thing to a different title and body.=0D=0A=0D=0AWith attribute-based = +conflict resolution it is possible to end up with a note that has a combi= +nation of A's title and B's body =E2=80=94 a combination that never exist= +ed on either computer.=0D=0A=0D=0AThis gets even worse when you treat del= +etion specially, when new assertions don't conflict but do alter domain s= +emantics, or when some of the conflicts affect the way the rest of the da= +ta is interpreted (e.g., when you have an identity property or a lookup r= +ef).=0D=0A=0D=0AThe only data modeling solution to this case is to introd= +uce an immutable 'note node', and have the _change_ be to have Note One p= +oint to a different note node =E2=80=94 one attribute change!=0D=0A=0D=0A= +In a JSON-ish system like Pouch you'd swap an entire JS object within the= + same attribute, rather than having two different attributes.=0D=0A=0D=0A= +Developers =E2=80=94 at least, non-Clojure developers =E2=80=94 rarely th= +ink in these terms, and now everywhere else you need to make really caref= +ul decisions about whether you point to the 'note' or the 'node'. Getting= + it wrong means dangling pointers.=0D=0A=0D=0AUsing per-attribute automat= +ic merging without having an expert ontologist model your domain is a rec= +ipe for weird edge-case conflict resolution bugs that result in data soup= + or data loss.=0D=0A=20=0D=0A=0D=0A> If an Datomic-like entity with 10 at= +tributes has 4 of them change by=0D=0A> different database instances, but= + only 1 change is on the same=0D=0A> attribute, would this type of detect= +ion offer more granularity instead=3F=0D=0A=0D=0AMy thinking is mostly: y= +ou need some kind of schema mechanism to be able to recognize which prope= +rties can be reconciled safely, and you also cannot examine each tuple in= +dependently in many cases. It's complicated. Something like Dropbox's dat= +astore conflict resolution (if I remember right) is fine much of the time= +, but wrong some of the time.=0D=0A=0D=0A=0D=0A> Wouldn't the 5-tuple be = +the equivalent of a single line of code on git,=0D=0A> and aggregates are= + built on top of it=3F=0D=0A=0D=0AEven source code merging isn't this sim= +ple, no=3F A conceptual operation like "add this line underneath this oth= +er line" is the thing you're trying to represent, but that line has _cont= +ext_ =E2=80=94 it might have moved, the file might have been renamed, or = +someone else already put a line there.=0D=0A=0D=0AYou've probably seen th= +is with b0rked merges in software version control, where the result of a = +merge won't compile because of duplicate identifiers thanks to hunks movi= +ng around=E2=80=A6=0D=0A=0D=0ABut trying to apply text-based merging to s= +emantic data only works if there are no constraints in your data, or if y= +ou have fast-forward merges =E2=80=94 merging can violate uniqueness prop= +erties in all kinds of ways.=20=0D=0A + +From me@josephg.com Tue Sep 1 22:13:15 2020 +Received: from wout4-smtp.messagingengine.com (wout4-smtp.messagingengine.com [64.147.123.20]) + by mail-b.sr.ht (Postfix) with ESMTPS id 53A0EFF124 + for <~euandreh/public-inbox@lists.sr.ht>; Tue, 1 Sep 2020 22:13:14 +0000 (UTC) +Authentication-Results: mail-b.sr.ht; + dkim=pass (1024-bit key) header.d=josephg.com header.i=@josephg.com header.b=2Kj6nHib; + dkim=fail reason="key not found in DNS" (0-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=eH2pUNJ6 +Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) + by mailout.west.internal (Postfix) with ESMTP id CB8F377E; + Tue, 1 Sep 2020 18:13:12 -0400 (EDT) +Received: from imap35 ([10.202.2.85]) + by compute1.internal (MEProxy); Tue, 01 Sep 2020 18:13:12 -0400 +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=josephg.com; h= + mime-version:message-id:in-reply-to:references:date:from:to + :subject:content-type:content-transfer-encoding; s=mesmtp; bh=Q9 + PUfqkL153ICDLSRfraWMnWRXOclTHibxQiOyAV3lA=; b=2Kj6nHibSzEldHCTUN + lgQhb0piuopatnU3wV94ozxZ95sUW7zyCnSE/s9gqFgTgYg0o2ZeqMffp92mRZis + frDqWjWNhxv4WN78vO8dcwA8U/a+Uwp4cfxUyjuyY/zw7NHHM8LORDQCfembOUFj + O6a5SW/fQRgU9XJIemyvxSDlM= +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= + messagingengine.com; h=content-transfer-encoding:content-type + :date:from:in-reply-to:message-id:mime-version:references + :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender + :x-sasl-enc; s=fm3; bh=Q9PUfqkL153ICDLSRfraWMnWRXOclTHibxQiOyAV3 + lA=; b=eH2pUNJ6n7/Ng/S9WTH+X0QePTCyzr5SjXdLgG3eN4a4yktWOtaMr44AU + nW7SE782cX3xP2TEyvVSJ58tnx6r+6M5Jzauja+b2OJiTtnrJyWfIZ6iK/URp/y2 + v++7NoAfWmyXOfUQAiLbK92FCRrRkR8nGx0EksjMInRiDUozhHcg1mRYRipvULUx + UsaQ0gtdGNDm76c6VwZU7YwBxssaeVQNP8ROlvm/y/KC3gpLYILLdNJPcI/r1846 + FrhxPQLgAQIe+4UV6NYtjNIRbFNPjdIWHZvWZ4LdMRPQ4yFdbpD6aHhKa+wLBPyB + sIgGBCGwzMhfTDSRZQw8DrWpcx21w== +X-ME-Sender: <xms:dsdOX2zlGD9BWCFDAMf7fpxK8tc8KYm5XDsDnDguZM2Sy8xflOH9kA> + <xme:dsdOXyTTPqu0-Pa3UYN2l_7sXVE04FX8AClSMYgSkAoeLeTspezWpQUlQYsja54Jc + wrAfUl58HrBhQRp6w> +X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduiedrudefkedgtdeiucetufdoteggodetrfdotf + fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen + uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfgjfhffhffvufgtgfesth + hqredtreerjeenucfhrhhomhepfdfuvghphhcuifgvnhhtlhgvfdcuoehmvgesjhhoshgv + phhhghdrtghomheqnecuggftrfgrthhtvghrnhepteeugfeufeeludekieduiefggfehhf + eiffelfeetteejheehveelteffhefffeeunecuffhomhgrihhnpehklhgvphhpmhgrnhhn + rdgtohhmnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomh + epmhgvsehjohhsvghphhhgrdgtohhm +X-ME-Proxy: <xmx:dsdOX4WWAjDIjt91xpvOW8wGgg-eiGlTvG2yYFGPVBe2qPkuU169eg> + <xmx:dsdOX8gA1SFXeEhDY-io8js7YHP8z7rKmyRtM81cBA0MFgLBRHhtow> + <xmx:dsdOX4DJCCv4jHxex2u1y3w6cvLGKhAfSjHmtdhvaRTHlJcfSu4J5w> + <xmx:eMdOX7M12Nj2n8pxrtLuGPyovCEq1wtzixBdJWACKU6e8qYcMiQKKg> +Received: by mailuser.nyi.internal (Postfix, from userid 501) + id 762F414C00B3; Tue, 1 Sep 2020 18:13:10 -0400 (EDT) +X-Mailer: MessagingEngine.com Webmail Interface +User-Agent: Cyrus-JMAP/3.3.0-248-gcd102cb-fm-20200901.001-gcd102cb9 +Mime-Version: 1.0 +Message-Id: <bab08df6-be8e-47bd-a254-7b86c41a9b47@www.fastmail.com> +In-Reply-To: <87pn75xtx4.fsf@euandre.org> +References: <d78064d2-5441-4275-96d9-463a08dfd7a9@www.fastmail.com> + <87ft81zvi0.fsf@euandre.org> + <9e513706-7ebc-4a40-8e5e-a5f94d84bc77@www.fastmail.com> + <87pn75xtx4.fsf@euandre.org> +Date: Wed, 02 Sep 2020 08:12:45 +1000 +From: "Seph Gentle" <me@josephg.com> +To: EuAndreh <eu@euandre.org>, ~euandreh/public-inbox@lists.sr.ht +Subject: Re: Making a database +Content-Type: text/plain;charset=utf-8 +Content-Transfer-Encoding: quoted-printable + +On Wed, Sep 2, 2020, at 3:56 AM, EuAndreh wrote: +> > You kinda gave a good description of them in your post already. They= + +> > work by requiring all peers to resolve concurrent edits to an object= + +> > the same way. That could be last-writer-wins (based on time stamps +> > like you said in your article). Or it could store all conflicting +> > versions together to be resolved by the next reader (Eg riak). Or it= + +> > could do something more clever (like merge changes using automerge o= +r +> > y.js or the like). Git is sort of a CRDT - it=E2=80=99s just it=E2=80= +=99s 3 way merge +> > algorithm is a bit of a dogs breakfast as far as these things go.=20= + +>=20 +> My ignorace is really on the applicability of it. The "store all +> conflicting versions together or to be resolved by the next reader" +> sounds like what I'm thinking about, so maybe I'm trying to re-invent +> CRDTs after all. +>=20 +> I've just bumped the priority of this topic to better understand those= + +> nuances.=20 + +This is worth watching if you haven=E2=80=99t seen it. Martin is at the = +forefront of the field - but this gives you a sense of where the researc= +h tech is at: + +https://martin.kleppmann.com/2020/07/06/crdt-hard-parts-hydra.html + +From eu@euandre.org Wed Sep 2 00:00:16 2020 +Received: from box.euandre.org (box.euandre.org [46.101.160.115]) + by mail-b.sr.ht (Postfix) with ESMTPS id E53C0FF14B + for <~euandreh/public-inbox@lists.sr.ht>; Wed, 2 Sep 2020 00:00:15 +0000 (UTC) +Authentication-Results: mail-b.sr.ht; + dkim=pass (2048-bit key) header.d=euandre.org header.i=@euandre.org header.b=Ts/gQOWS +Received: from authenticated-user (box.euandre.org [46.101.160.115]) + (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) + (No client certificate requested) + by box.euandre.org (Postfix) with ESMTPSA id E5F6DFC052; + Tue, 1 Sep 2020 21:00:11 -0300 (-03) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=euandre.org; s=mail; + t=1599004813; bh=GS02iBK+iDwoGCfVqsjbqMenu+Wpn9OO6GpSXGHcpsY=; + h=From:To:Cc:Subject:In-Reply-To:References:Date:From; + b=Ts/gQOWSiM5Vq/pHxaNuXKM4mXoy3VW9rD5mv/uLMGHYQcslItj1TNcvxINLzU/yj + ZaBmqtENnSd9AXap4PHxsFz48JlDI5oYcn98HE9nLsAZrD7/9GkhJMNjRDsf0ODAl9 + W7do6QS8LRAZo0RhTguAzgoTJXf/IxuRqykG9Nm8imNuLDN4T04EbGvRw4ibN4UhlT + DED1G5O4Ul96BT6mY/im9tBQPGGNURLLcNcUkqG4fpddJrg58/yH9JOPMhnNhatEiV + 39+m3xyL2rM8HSBfKun87TXjE6CD4/kQ0U6OpSB9ej/4zdNJ7Ei9K5NSuF78KRvkWJ + qHKRHqP/84rQw== +From: EuAndreh <eu@euandre.org> +To: Seph Gentle <me@josephg.com>, ~euandreh/public-inbox@lists.sr.ht +Cc: +Subject: Re: Making a database +In-Reply-To: <bab08df6-be8e-47bd-a254-7b86c41a9b47@www.fastmail.com> +References: <d78064d2-5441-4275-96d9-463a08dfd7a9@www.fastmail.com> + <87ft81zvi0.fsf@euandre.org> + <9e513706-7ebc-4a40-8e5e-a5f94d84bc77@www.fastmail.com> + <87pn75xtx4.fsf@euandre.org> + <bab08df6-be8e-47bd-a254-7b86c41a9b47@www.fastmail.com> +Date: Tue, 01 Sep 2020 20:18:08 -0300 +Message-ID: <87eenlxf0v.fsf@euandre.org> +MIME-Version: 1.0 +Content-Type: text/plain; charset=utf-8 +Content-Transfer-Encoding: quoted-printable + +"Seph Gentle" <me@josephg.com> writes: + +> This is worth watching if you haven=E2=80=99t seen it. Martin is at the f= +orefront of the field - but this gives you a sense of where the research te= +ch is at: +> +> https://martin.kleppmann.com/2020/07/06/crdt-hard-parts-hydra.html + +Coincidentally, that's the third item on my list =F0=9F=98=AC + +From jeff@rhyason.org Wed Sep 2 01:35:17 2020 +Received: from mx3.rhyason.org (mx3.rhyason.org [104.245.39.21]) + by mail-b.sr.ht (Postfix) with ESMTPS id 6C175FF0E7 + for <~euandreh/public-inbox@lists.sr.ht>; Wed, 2 Sep 2020 01:35:17 +0000 (UTC) +Authentication-Results: mail-b.sr.ht; + dkim=pass (1024-bit key) header.d=rhyason.org header.i=@rhyason.org header.b=weEVa7Zv +Received: from rhyason.org (d173-183-32-226.bchsia.telus.net [173.183.32.226]) + by mx3.rhyason.org (8.15.2/8.15.2) with ESMTP id 0821ZI7G073010 + for <~euandreh/public-inbox@lists.sr.ht>; Tue, 1 Sep 2020 21:35:19 -0400 (EDT) + (envelope-from jeff@rhyason.org) +Received: from rhyason.org (localhost [127.0.0.1]); + by rhyason.org (OpenSMTPD) with ESMTP id 4aa4e076; + for <~euandreh/public-inbox@lists.sr.ht>; + Tue, 1 Sep 2020 19:35:08 -0600 (MDT) +DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=rhyason.org; h=from + :content-type:content-transfer-encoding:mime-version:subject + :message-id:date:to; s=selector1; bh=61RPMANfo/dmUC5BfkkO8NNOMrg + =; b=weEVa7ZvWqlxTjSrAxMCoiPy3GBkyNONviKR6lPNlGM3/eV5+0g3KhjKIKs + g5+xXXB1G69y1xvNyo5hpOYY4Gk51OeGCQ/QxtE3tD2IgTxYjWlsB5SuNTrWI6Kd + Onz8i5Fdan7ZigJ1ambirCGyI/IWPb8Q8gG7+i76Wb2vj3W8= +DomainKey-Signature: a=rsa-sha1; c=nofws; d=rhyason.org; h=from + :content-type:content-transfer-encoding:mime-version:subject + :message-id:date:to; q=dns; s=selector1; b=mpt0BK56C+LRHqEw8Bx8M + y7nJWXZqD5kxPw4J7wh3mh8VcMYiBuA/2BpbrT7wYH9IxpMGFjlSP/lIZIHUPFTt + cpbgqiL6CrW1zvP4CJ+6ygSQkfnsEGekcmEgLGa/E49lcKxA/y7Ep1G7maJJIZxN + 3OqgdtRLK1MiajxwW/MvBA= +Received: from [10.0.88.117] (10.0.88.117 [10.0.88.117]); + by rhyason.org (OpenSMTPD) with ESMTPSA id d7674b26; + TLS version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO; + for <~euandreh/public-inbox@lists.sr.ht>; + Tue, 1 Sep 2020 19:35:08 -0600 (MDT) +From: Jeff Rhyason <jeff@rhyason.org> +Content-Type: text/plain; + charset=us-ascii +Content-Transfer-Encoding: quoted-printable +Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\)) +Subject: building MSTs->CRDTs-> Re: The database I wish I had +Message-Id: <99C92E06-E0BC-4DB0-BF63-20FD44DC4062@rhyason.org> +Date: Tue, 1 Sep 2020 18:35:08 -0700 +To: ~euandreh/public-inbox@lists.sr.ht +X-Mailer: Apple Mail (2.3608.120.23.2.1) + +It's amazing to see this discussion. Let's keep it going! + +My friend and I started Indelible[1] (now defunct) for a lot of the same = +reasons you want to build a datastore -- there aren't enough options for = +a datastore that's immutable, relational, highly-vaailable, and = +private.. We're still super-passionate about the ideas, love Datomic but = +want something a bit different, decided not to commercialize it and = +start more simply with just the data structure. We don't want a huge = +thing, just a library, to keep coordination down; the complexity doesn't = +spread out of the client. + +Where we are at now is we have a great versioned immutable B-Tree-like = +structure that is the foundation for the verioned and diffable database = +we want. FWIW The Go implementation of the tree is at = +https://github.com/jrhy/mast . We have plans to do a CRDT based on it, = +possibly add an SQL layer, but frankly, having a strongly-consistent, = +verioned + diffable tree, is pretty good for now. We'd love to = +collaborate! + + + +[1] Indelible API Reference = +http://htmlpreview.github.io/?https://github.com/indeliblesystems/indelibl= +e/blob/master/doc/000-API-ref.html + + +From eu@euandre.org Wed Sep 2 08:00:14 2020 +Received: from box.euandre.org (box.euandre.org [46.101.160.115]) + by mail-b.sr.ht (Postfix) with ESMTPS id E2229FF148 + for <~euandreh/public-inbox@lists.sr.ht>; Wed, 2 Sep 2020 08:00:13 +0000 (UTC) +Authentication-Results: mail-b.sr.ht; + dkim=pass (2048-bit key) header.d=euandre.org header.i=@euandre.org header.b=Gv++l+b4 +Received: from authenticated-user (box.euandre.org [46.101.160.115]) + (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) + (No client certificate requested) + by box.euandre.org (Postfix) with ESMTPSA id BBE37FC052; + Wed, 2 Sep 2020 05:00:10 -0300 (-03) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=euandre.org; s=mail; + t=1599033612; bh=L0rF3hNESOLacI6plS3F0G5McLpB7MuPNAKgItTuBzg=; + h=From:To:Cc:Cc:Subject:In-Reply-To:References:Date:From; + b=Gv++l+b4mbVug6D7ywconYDZ/EzLKxn27zY5F1FhWm2A97mClGgf20SVzjqrxZNOE + LHVxQX1qwB7I/MzwWDf0ACHFQsOu0Y2VgQyffVE0BHInz/TPbXlv9Qbfe1L9yRFvD7 + Z+3BoX5//m/1Na1ZKeuP1KT5neo8dF75wyyUhG0Zh/hFEDa1ILdR8wrnovtouqmwxA + JoMH2EzjLJ8ad2pox2VU0BskunOUvIX/CRXsrnXDCxybseFECE7h2jghegme7MlLae + /kvzmmg6Ow1z/lzvftKWGJRPb47MvQYSFIbnLb2kkkhTb6evJ5eXPO2nPY6/vJ8t6c + jhgzwfN/DzLeA== +From: EuAndreh <eu@euandre.org> +To: Richard Newman <rnewman@twinql.com> +Cc: "~euandreh/public-inbox@lists.sr.ht" <~euandreh/public-inbox@lists.sr.ht> +Cc: +Subject: Re: The database I wish I had +In-Reply-To: <010101744b260d62-1bb508ab-647f-4877-bdf9-f988667d56e1-000000@us-west-2.amazonses.com> +References: <7454DA45-C5A4-4A54-8D02-1F3BD37726E9@twinql.com> + <010101744a592b75-1dce9281-f0b8-4226-9d50-fd2c7901fa72-000000@us-west-2.amazonses.com> + <87sgc1xu7k.fsf@euandre.org> + <9652AAE0-10DF-4D13-ABE8-B30667BE27DD@twinql.com> + <010101744b260d62-1bb508ab-647f-4877-bdf9-f988667d56e1-000000@us-west-2.amazonses.com> +Date: Wed, 02 Sep 2020 04:30:43 -0300 +Message-ID: <878sdsy6sc.fsf@euandre.org> +MIME-Version: 1.0 +Content-Type: text/plain; charset=utf-8 +Content-Transfer-Encoding: quoted-printable + +This is some rich discussion, thanks for engaging! + +Richard Newman <rnewman@twinql.com> writes: + +> Mostly the usual stuff you get when a thing is defined by its +> implementation: +> +> https://github.com/mozilla/mentat/pull/665 +> https://github.com/mozilla/mentat/issues/412 +> +> There are probably more if you poke around the issue tracker. + +Thanks for the references. + +> I'm familiar with the way AllegroGraph does storage, and fairly +> familiar with how Datomic does it. + +I've used Common Lisp a lot before, and I heard a lot about +AllegroGraph, but I've never worked with it. Would you recommed it as a +place to take inspirations from? + +> In short, a datom store like this consists of the log (all datoms, +> with retractions, ordered by ID), and some current state (which Mentat +> called `datoms`), which consists of all current asserted datoms. +> +> The current state is typically stored in a number of redundant +> permutations to support querying. In AllegroGraph (subject, predicate, +> object, graph, id), it would maintain multiple indices: `spogi`, +> `posgi`, etc. Each of these store the same datoms in different orders. +> +> If we have the data +> +> ``` +> :bob foaf:knows :alice +> :alice foaf:name "Alice" +> :bob foaf:name "Bob" +> :bob foaf:age 26 +> :alice foaf:age 30 +> ``` +> +> it might appear in the `spogi` index as: +> +> ``` +> :alice foaf:age 30 +> :alice foaf:name "Alice" +> :bob foaf:age 26 +> :bob foaf:knows :alice +> :bob foaf:name "Bob" +> ``` +> +> and in the `posgi` index as: +> +> ``` +> foaf:age 26 :bob +> foaf:age 30 :alice +> foaf:name "Alice" :alice +> foaf:name "Bob" :bob +> foaf:knows :alice :bob +> ``` + +These look just like the EAVT and AVET indexes in Datomic. + +> Naturally these would be using efficient (interned) sortable +> representations of each term, with rows of fixed size, memory mapped +> on disk. +> +> This allows you to implement query pattern execution as an index walk. +> +> `SELECT ?s WHERE ?s foaf:name 'Alice'` +> +> for example turns into a walk on `posgi`: you find the spot that +> corresponds to `foaf:name`, find the next place which is `Alice`, and +> then read out the `s`. +> +> When you insert data, you typically don't want to immediately +> regenerate all of these indices. Instead you accrue new *chunks*, +> which are just all of the datoms asserted in some window of time, or +> within a particular transaction. The query engine walks all of the +> relevant chunks and merges the results together. One chunk might have +> Bob's name, and one might have Alice's. +> +> Typically _metaindices_ are used to help the query engine navigate +> which chunks to use. +> +> AllegroGraph periodically merges chunks together: a fully merged store +> is faster to query. Datomic goes the other way: it builds new merged +> chunks and keeps the old ones around, because those old chunks +> represent the state of the database at a point in time! This is how +> you efficiently query past states in Datomic: it trades space in your +> Cassandra cluster for the ability to grab the right chunks for any +> query. +> +> You can easily imagine that any system that allows you to put a name +> on some region of data can be used to store these chunks, and their +> metaindices, and the log itself. The only requirement is that the +> chunk store be durable, and ideally replicated and relatively fast to +> read from. + +The most intriguing part of this for me is trying to mimic Datomic's +implementation of those merged chunks, specifically how Rich Hickey +describes it as an equivalent of persistent data structures used for +storage, with sharing of previous versions, etc. I believe he calls it +"durable persistent" datastructure, like making things on disk to be +a bit similar on how they are on memory. + +In fact, that's exactly where I'm focused at right now: deepening my +grasp on persistent data structures to enable me to navigate this +subject with more fluidity, and try to implement such durable persistent +data structures. + +The merging of the AllegroGraph that you describe sounds like the flat +files merge of indexes that RH describes, and contrasts with his +proposal. I'll try to find this quote, and re-read the paper he mentions +on the talk (IIRC it's the BigTable one). + +I was considering having all indexes be derived from the raw data +transported between instances, but it might get computationally +expensive, and making those indexes replicable sounds like a good idea. + +> If on Computer A I visit https://aclu.org, and on Computer B I visit +> https://aclu.org, my two computers will each give the same history +> item a different internal identifier =E2=80=94 1234 and 4567, say. +> +> When I sync them, I need the data store to know that the visits I made +> to each page _were visits to the same thing_. + +Isn't that a domain-specifc problem of identity? + +I would solve this by allowing those two entries to cooexist without any +conflict, and do aggregation queries to answers those questions, +something like (pseudo-SQL): + +SELECT DISTINCT(url) FROM urls; + +...to show the all the visited URLs, and: + +SELECT SUM(1) from URLS GROUP BY url + +...to show the visit count for each URL. + +WDYT? + +> It definitely helps; this is how teams generally try to work with +> DynamoDB, for example, though there the schema is implicit rather than +> explict. +> +> However, you inevitably have data migrations to worry about=E2=80=A6 not = +to +> mention that syncing makes this a distributed system, and so you +> cannot easily coordinate schema changes in more than one place, and +> they will race with new data! +> +> There's also the argument that coordinating change in consumers and +> engineers is sometimes harder than evolving a vocabulary! +> +> It's a complicated topic. + +On migrations: + +I used to share your view on needing a migration scape hatch, but I +don't anymore. On the company I work those kind of database migrations +never really occur. People either create a new attribute, copy data from +a database to another, process it using ETL, compute things on the fly +with functions, and many other strategies. We've already had to comply +with excisions in production, but never on schema, only on data itself. + +So I don't think that's an actual issue, and I would insist that the +"Grow your schema" approach actually does work at scale, on the long +run, for multiple scenarios.=20 + +> I have been relatively happy with SQLCipher, and I think the idea of +> block-level encryption is a good place to put the abstraction. The +> thing you lose with any of these mechanisms is, by definition, the +> ability for a central entity to help you with conflict resolution. The +> best you can get is ordering of opaque chunks and collision detection. + +I believe that's not what I meant by comparing it to git-remote-gcrypt, +but maybe I misunderstood your point. + +Here's a code snippet to try to highlight what I was referring to: + +--8<---------------cut here---------------start------------->8--- +$ pushd $(mktemp -d) +/tmp/tmp.iapFd5iBCr ~/ +$ mkdir repo-1 +$ cd repo-1/ +$ git init +$ echo 'a' > file +$ git add file +$ git commit -m commit-a +[master (commit racine) c4112e4] commit-a + 1 file changed, 1 insertion(+) + create mode 100644 file +$ git init --bare ../remote/ +D=C3=A9p=C3=B4t Git vide initialis=C3=A9 dans /tmp/tmp.iapFd5iBCr/remote/ +$ git remote add origin gcrypt::../remote +$ git push origin +gcrypt: Repository not found: ../remote +gcrypt: Setting up new repository +gcrypt: Remote ID is :id:KjDStP/7m18wJagDO0/o +=C3=89num=C3=A9ration des objets: 3, fait. +D=C3=A9compte des objets: 100% (3/3), fait. +Total 3 (delta 0), r=C3=A9utilis=C3=A9s 0 (delta 0), r=C3=A9utilis=C3=A9s d= +u pack 0 +gcrypt: Encrypting to: --throw-keyids --default-recipient-self +gcrypt: Requesting manifest signature +To gcrypt::../remote + * [new branch] master -> master +$ cd .. +$ git clone gcrypt::remote repo-2 +Clonage dans 'repo-2'... +gcrypt: Decrypting manifest +gpg: selecting card failed: No such device +gpg: Signature faite le mer. 02 sept. 2020 03:43:12 -03 +gpg: avec la clef RSA 5BDAE9B8B2F6C6BCBB0D6CE581F90EC3CD3560= +60 +gpg: Bonne signature de =C2=AB=C2=A0EuAndreh <eu@euandre.org>=C2=A0=C2=BB [= +ultime] +gcrypt: Remote ID is :id:KjDStP/7m18wJagDO0/o +R=C3=A9ception d'objets: 100% (3/3), fait. +$ cd repo-2/ +$ echo 'b' > file +$ git add file +$ git commit -m commit-b +[master a5f5f0b] commit-b + 1 file changed, 1 insertion(+), 1 deletion(-) +$ git remote set-url origin gcrypt::../remote +$ git push origin +gcrypt: Decrypting manifest +gpg: selecting card failed: No such device +gpg: Signature faite le mer. 02 sept. 2020 03:43:12 -03 +gpg: avec la clef RSA 5BDAE9B8B2F6C6BCBB0D6CE581F90EC3CD3560= +60 +gpg: Bonne signature de =C2=AB=C2=A0EuAndreh <eu@euandre.org>=C2=A0=C2=BB [= +ultime] +=C3=89num=C3=A9ration des objets: 3, fait. +D=C3=A9compte des objets: 100% (3/3), fait. +Total 3 (delta 0), r=C3=A9utilis=C3=A9s 0 (delta 0), r=C3=A9utilis=C3=A9s d= +u pack 0 +gcrypt: Encrypting to: --throw-keyids --default-recipient-self +gcrypt: Requesting manifest signature +To gcrypt::../remote + c4112e4..a5f5f0b master -> master +$ cd ../repo-1/ +$ echo 'c' > file +$ git add file +$ git commit -m commit-c +[master 3d71137] commit-c + 1 file changed, 1 insertion(+), 1 deletion(-) +--8<---------------cut here---------------end--------------->8--- + +At this point, repo-1 has changes that conflict with repo-2, and remote +doesn't know about it, but we can resolve it locally: + +--8<---------------cut here---------------start------------->8--- +$ git pull origin master +gcrypt: Decrypting manifest +gpg: selecting card failed: No such device +gpg: Signature faite le mer. 02 sept. 2020 03:44:34 -03 +gpg: avec la clef RSA 5BDAE9B8B2F6C6BCBB0D6CE581F90EC3CD3560= +60 +gpg: Bonne signature de =C2=AB=C2=A0EuAndreh <eu@euandre.org>=C2=A0=C2=BB [= +ultime] +Depuis gcrypt::../remote + * branch master -> FETCH_HEAD +Fusion automatique de file +CONFLIT (contenu) : Conflit de fusion dans file +La fusion automatique a =C3=A9chou=C3=A9 ; r=C3=A9glez les conflits et vali= +dez le r=C3=A9sultat. +$ cat file +<<<<<<< HEAD +c +=3D=3D=3D=3D=3D=3D=3D +b +>>>>>>> a5f5f0b6c780a7ab3f7e28f373b06fa9a10c443f +--8<---------------cut here---------------end--------------->8--- + +This is just the general behaviour that I'm aiming towards, but that is +really just a long shot, I still need to reasearch more on it. + +At a glance I can't really tell if that is what you mean when citing +SQLCipher.=20 + +>> If sound and robust, what's the downside of detecting conflicts on +>> individual attributes? +> +> It relies on extensive reification to achieve the property that we +> want: that each semantically conflicting write results in a +> conflicting attribute change.=20 +> +> As a contrived example example: notes have both a title and a body.=20 +> +> A user has two computers. +> +> On Computer A, the user decides to change their first note to +> something else, updating both the title and body.=20 +> +> On Computer B, they do the same thing to a different title and body.=20 +> +> With attribute-based conflict resolution it is possible to end up with +> a note that has a combination of A's title and B's body =E2=80=94 a +> combination that never existed on either computer.=20 +> +> This gets even worse when you treat deletion specially, when new +> assertions don't conflict but do alter domain semantics, or when some +> of the conflicts affect the way the rest of the data is interpreted +> (e.g., when you have an identity property or a lookup ref).=20 +>=20 +> The only data modeling solution to this case is to introduce an +> immutable 'note node', and have the _change_ be to have Note One point +> to a different note node =E2=80=94 one attribute change!=20 +>=20 +> In a JSON-ish system like Pouch you'd swap an entire JS object within +> the same attribute, rather than having two different attributes.=20 +> +> Developers =E2=80=94 at least, non-Clojure developers =E2=80=94 rarely th= +ink in these +> terms, and now everywhere else you need to make really careful +> decisions about whether you point to the 'note' or the 'node'. Getting +> it wrong means dangling pointers.=20 +> +> Using per-attribute automatic merging without having an expert +> ontologist model your domain is a recipe for weird edge-case conflict +> resolution bugs that result in data soup or data loss.=20 + +TBH I'm not aiming for automatic conflict resolution. + +Why not use the granularity of attributes to detect conflict, but not to +resolve them? + +Let me add to the scenario you presented: notes have a title, a body and +an author. + +A user has two computers. + +On Computer A, the user decides to change their first note to +something else, updating both the title and body.=20 + +On Computer B, they do the same thing to a different title and body.=20 + +On both cases, the author stays the same. + +When the DB instance on Computer A gets data from Computer B, the query: + +SELECT * FROM notes WHERE id =3D 1234; + +would return both, and some metadata saying that it has a conflict. Now +the programmer needs to decide how to deal with these: use timestamps, +pick a winning device, sort by revision ID and pick the first, show them +on a UI to the user to resolve, etc. + +Either way, the DB knows that the note has the title and body attributes +in conflict, but the author is fine. The aggregate entity itself isn't +in conflict, but some of it's attributes. + +The situation that you describe where one would end up with a note that +is a combination of A's title and B's body would only happen if the DB +tried to solve things "automagically". + +However, I agree on the costly constant reification being required, and +for long disconnected histories to be merged together this might be +expensive.=20 + +>> If an Datomic-like entity with 10 attributes has 4 of them change by +>> different database instances, but only 1 change is on the same +>> attribute, would this type of detection offer more granularity instead? +> +> My thinking is mostly: you need some kind of schema mechanism to be +> able to recognize which properties can be reconciled safely, and you +> also cannot examine each tuple independently in many cases. It's +> complicated. Something like Dropbox's datastore conflict resolution +> (if I remember right) is fine much of the time, but wrong some of the +> time.=20 + +Couldn't you leverage the schema for that? If the said conflicting +attribute has: + +- cardinality one and both DBs changed it, we have a conflict; +- cardinality many and both changed the same attribute, we have a + conflict; +- cardinality may and Computer A changed an entry and Computer B changed + another one, we don't have a conflict. + +and so on. + +I wouldn't build it with an exaustive search, but just leverage the +schema to make those decisions. + +I don't know Dropbox's datastore, is this[0] the one you're refferring +to?=20 + +[0]: https://dropbox.tech/developers/how-the-datastore-api-handles-conflict= +s-part-1-basics-of-offline-conflict-handling + +> Even source code merging isn't this simple, no? A conceptual operation +> like "add this line underneath this other line" is the thing you're +> trying to represent, but that line has _context_ =E2=80=94 it might have +> moved, the file might have been renamed, or someone else already put a +> line there.=20 +> +> You've probably seen this with b0rked merges in software version +> control, where the result of a merge won't compile because of +> duplicate identifiers thanks to hunks moving around=E2=80=A6=20 +> +> But trying to apply text-based merging to semantic data only works if +> there are no constraints in your data, or if you have fast-forward +> merges =E2=80=94 merging can violate uniqueness properties in all kinds of +> ways.=20 + +Yep, that was an oversimplification. + +<hr /> + +Again, thanks for engaging, this discussion feels like a golden nugget. + +From me@jonas-schuermann.name Wed Sep 2 12:48:41 2020 +Received: from mout-p-201.mailbox.org (mout-p-201.mailbox.org [80.241.56.171]) + by mail-b.sr.ht (Postfix) with ESMTPS id E843EFF14B + for <~euandreh/public-inbox@lists.sr.ht>; Wed, 2 Sep 2020 12:48:40 +0000 (UTC) +Received: from smtp1.mailbox.org (smtp1.mailbox.org [80.241.60.240]) + (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) + (No client certificate requested) + by mout-p-201.mailbox.org (Postfix) with ESMTPS id 4BhNzC5xW2zQlX2; + Wed, 2 Sep 2020 14:48:39 +0200 (CEST) +X-Virus-Scanned: amavisd-new at heinlein-support.de +Received: from smtp1.mailbox.org ([80.241.60.240]) + by spamfilter03.heinlein-hosting.de (spamfilter03.heinlein-hosting.de [80.241.56.117]) (amavisd-new, port 10030) + with ESMTP id 3fpZQfKdp0ux; Wed, 2 Sep 2020 14:48:36 +0200 (CEST) +Subject: Re: The database I wish I had +To: EuAndreh <eu@euandre.org>, ~euandreh/public-inbox@lists.sr.ht +References: <e0b3b042-5eda-bcde-59a5-a5b61fecd2e5@jonas-schuermann.name> + <87y2ltyeqg.fsf@euandre.org> +From: =?UTF-8?Q?Jonas_Sch=c3=bcrmann?= <me@jonas-schuermann.name> +Message-ID: <acd49073-c46e-2bae-da55-de1d8cc42533@jonas-schuermann.name> +Date: Wed, 2 Sep 2020 14:48:35 +0200 +MIME-Version: 1.0 +In-Reply-To: <87y2ltyeqg.fsf@euandre.org> +Content-Type: text/plain; charset=utf-8; format=flowed +Content-Language: en-US +Content-Transfer-Encoding: 7bit +X-MBO-SPAM-Probability: +X-Rspamd-Score: -0.85 / 15.00 / 15.00 +X-Rspamd-Queue-Id: 6F42037F +X-Rspamd-UID: dfaa19 + +> Hmmm, I haven't heard this term before. Are these [0] the complete +> categories that you mean? +> +> [0]: https://en.wikipedia.org/wiki/Complete_category + +Yes, exactly. The idea is pretty abstract and while I know a little bit +about category theory I'm by no means an expert on the subject. This +blog post [1] is a very down-to-earth, practical explanation how the +concept of (co)completed categories can be applied to conflict +representation in version control systems. + +[1] https://jneem.github.io/merging/ + +From rnewman@twinql.com Wed Sep 2 16:14:41 2020 +Received: from a58-46.smtp-out.us-west-2.amazonses.com (a58-46.smtp-out.us-west-2.amazonses.com [54.240.58.46]) + by mail-b.sr.ht (Postfix) with ESMTPS id AF4DCFF12F + for <~euandreh/public-inbox@lists.sr.ht>; Wed, 2 Sep 2020 16:14:40 +0000 (UTC) +Authentication-Results: mail-b.sr.ht; + dkim=pass (1024-bit key) header.d=twinql.com header.i=@twinql.com header.b=grgVRwHH; + dkim=pass (1024-bit key) header.d=amazonses.com header.i=@amazonses.com header.b=SGue23U5 +DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; + s=qvs2k2sd2ivtrflyu2nswmravuqizjml; d=twinql.com; t=1599063278; + h=Subject:From:To:Cc:Date:Mime-Version:Content-Type:Content-Transfer-Encoding:In-Reply-To:References:Message-Id; + bh=UZ3b7SzbsIcKi0Si4ZISwM274bq+ODQFNXqhUN8KRVo=; + b=grgVRwHHOiZAo7asJgQxtXN5anM8LDtht/oVf7XVjh3IJNFPui4FSOPJK+Eh4Xdf + MniQ9lndbRWU+3IR3fr3G6NKqYNdfpJb74/j/LvjnnrwZqxQQJFt99pQmwAKADGJAyG + ZIwzp6kjG9pz3Uk1Bh+kRa2mHBIMmgfenRoqjiAI= +DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; + s=hsbnp7p3ensaochzwyq5wwmceodymuwv; d=amazonses.com; t=1599063278; + h=Subject:From:To:Cc:Date:Mime-Version:Content-Type:Content-Transfer-Encoding:In-Reply-To:References:Message-Id:Feedback-ID; + bh=UZ3b7SzbsIcKi0Si4ZISwM274bq+ODQFNXqhUN8KRVo=; + b=SGue23U5gqkhyLMKyKwkBGrqzxh9Ds7ua97mgDIB1BcWDmf58EDy2KhBrlkOtn0y + IP8gxNfgYZDYdJFvPt8GmlkDg8nQNYW29iWNcf8ppMHCCEchNB85KrXAOtyBXYiR2oa + nKriOwTP4/GeXifOl7+T8eIEkFuZI6DVL5Gas5SA= +Subject: Re: The database I wish I had +From: =?UTF-8?Q?Richard_Newman?= <rnewman@twinql.com> +To: =?UTF-8?Q?EuAndreh?= <eu@euandre.org> +Cc: =?UTF-8?Q?=7Eeuandreh/public-inbox=40lists=2Esr=2Eht?= + <~euandreh/public-inbox@lists.sr.ht> +Date: Wed, 2 Sep 2020 16:14:38 +0000 +Mime-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: quoted-printable +In-Reply-To: <878sdsy6sc.fsf@euandre.org> +References: <7454DA45-C5A4-4A54-8D02-1F3BD37726E9@twinql.com> + <010101744a592b75-1dce9281-f0b8-4226-9d50-fd2c7901fa72-000000@us-west-2.amazonses.com> + <87sgc1xu7k.fsf@euandre.org> + <9652AAE0-10DF-4D13-ABE8-B30667BE27DD@twinql.com> + <010101744b260d62-1bb508ab-647f-4877-bdf9-f988667d56e1-000000@us-west-2.amazonses.com> + <878sdsy6sc.fsf@euandre.org> + <7027FD8B-1B30-435F-A840-7DC61A722E0B@twinql.com> +X-Priority: 3 (Normal) +X-Mailer: Amazon WorkMail +Thread-Index: AQHWgHcPJ4udfXYrRaOwhoO7EcsDJQAErSAlAAfQ7+gAIgKd3wAzRhy+ +Thread-Topic: The database I wish I had +X-Original-Mailer: Apple Mail (2.3445.104.14) +X-Wm-Sent-Timestamp: 1599063277 +Message-ID: <010101744f99448f-fa8dd88e-0daf-41f8-b3ca-6ea551c300e4-000000@us-west-2.amazonses.com> +X-SES-Outgoing: 2020.09.02-54.240.58.46 +Feedback-ID: 1.us-west-2.An468LAV0jCjQDrDLvlZjeAthld7qrhZr+vow8irkvU=:AmazonSES + +> I've used Common Lisp a lot before, and I heard a lot about=0D=0A> Alle= +groGraph, but I've never worked with it. Would you recommed it as a=0D=0A= +> place to take inspirations from=3F=0D=0A=0D=0AHelping to build it was a= + very formative experience for me. twinql was the name of my first SPARQL= + query engine, which became the genesis of AllegroGraph's, and it stuck a= +s a domain name!=0D=0A=0D=0ASome of AllegroGraph's performance is the res= +ult of decades of experience (Steve Haflich comes to mind) and having som= +e of the same exceptional developers work on it as worked on the compiler= +, so its internals are excellent inspiration. The UPI (how EAV are repres= +ented) is a good idea executed well. AllegroGraph is also a good example = +of how important tooling is for exploring this kind of semi-structured da= +ta.=0D=0A=0D=0A=0D=0A> These look just like the EAVT and AVET indexes in = +Datomic.=0D=0A=0D=0AYes, exactly! It's a very natural way to model a fixe= +d-width tuple store.=0D=0A=0D=0A=0D=0A> I was considering having all inde= +xes be derived from the raw data=0D=0A> transported between instances, bu= +t it might get computationally=0D=0A> expensive, and making those indexes= + replicable sounds like a good idea.=0D=0A=0D=0AIt depends where you're r= +eplicating. In Datomic's case each of the query nodes gets a replicated c= +opy of the index chunks it needs (and you can thus control cache warmth b= +y routing queries to different nodes), because those nodes are meant to b= +e fungible, and reads from S3 (or wherever) are cheap, so there's no sens= +e doing the work twice.=0D=0A=0D=0AIf you were to stand up a hot spare, y= +ou would probably just stream the log and have the far end create its own= + indices. That's particularly true if the other store needs different ind= +ices, or is materializing views.=0D=0A=0D=0ASidenote: it surprises me tha= +t Datomic Cloud stores its log in DynamoDB. It's hard to imagine a good p= +rimary key scheme that wouldn't at some point exhaust write capacity, and= + DDB makes different availability and durability tradeoffs than S3. I'm c= +urious how that decision was made.=0D=0A=0D=0AObligatory disclaimer: I wo= +rk at AWS, but the above is not a statement on behalf of AWS!=0D=0A=0D=0A= +=0D=0A>> When I sync them, I need the data store to know that the visits = +I made=0D=0A>> to each page _were visits to the same thing_.=0D=0A>=20=0D= +=0A> Isn't that a domain-specifc problem of identity=3F=0D=0A=0D=0AYes, a= +bsolutely. I take the position that the goal of a database is to help me = +efficiently, accurately, and ergonomically represent my domain. If it doe= +sn't, every developer has to solve that problem at the application level,= + and that limits interoperability, correctness, performance, and expressi= +veness.=0D=0A=0D=0AThe ability for this data layer to be able to help you= + with problems like syncing also relies on its ability to leverage some i= +nformation about your domain =E2=80=94 compare to when you check in an im= +age file or a tarball to Git, and it can't help you detect or resolve con= +flicts at anything more granular than the containing file, even if that f= +ile has its own internal structure.=0D=0A=0D=0A=0D=0A> I would solve this= + by allowing those two entries to cooexist without any=0D=0A> conflict, a= +nd do aggregation queries to answers those questions,=0D=0A> something li= +ke (pseudo-SQL):=0D=0A>=20=0D=0A> SELECT DISTINCT(url) FROM urls;=0D=0A>=20= +=0D=0A> ...to show the all the visited URLs, and:=0D=0A>=20=0D=0A> SELECT= + SUM(1) from URLS GROUP BY url=0D=0A>=20=0D=0A> ...to show the visit coun= +t for each URL.=0D=0A>=20=0D=0A> WDYT=3F=0D=0A=0D=0AThe advantage is inde= +ed that you have removed the ability for data to conflict.=0D=0A=0D=0AThe= + disadvantage is that you have pushed a domain identity and merging into = +application code, where:=0D=0A=0D=0A* Every query needs to implement this= + kind of unification in multiple places, which is more expensive and horr= +endously difficult to get right, not to mention very hard for developers = +to read.=0D=0A=0D=0A* Application developers can get it wrong (e.g., dele= +ting from History needs to retract every entry with a given URL), and you= + make domain operations more complex as a result. Sometimes this adds val= +ue =E2=80=94 e.g., this kind of accumulative storage gives you magic prop= +erties like lifetime undo! =E2=80=94 but I suspect that putting that in t= +he log is better.=0D=0A=0D=0A* You lose some conveniences like lookup ref= +s, because you've removed the concept of a 1:1 lookup altogether.=0D=0A=0D= +=0A* You lose the trivial ability to detect and model _changes_, which I = +find is natural for developers: this approach essentially makes the log p= +art of the datom state. You can still do retract/assert locally, for a pa= +rticular entity, but if you want to e.g., get the latest title for a page= +, you need to reduce over the log with some ordering.=0D=0A=0D=0A=0D=0AI = +agree with your position in principle: the idea of figuring out the curre= +nt state dynamically by rolling up history from different machines is ver= +y event-sourcey and is theoretically pleasant. Every device would simply = +append new non-conflicting _observations_ to the log. I do advocate for t= +his in some circumstances.=0D=0A=0D=0AHowever, in practice we usually hav= +e some stable idea of what we want current state to be (particularly of s= +chema itself!), and some domain concepts around identity and cardinality = +that we want to enforce and use to build concepts like change, conflict, = +and merging, and we need that maintained state to be fast and obvious bec= +ause we use it when making new assertions.=0D=0A=0D=0AIn essence, the pro= +blem I was trying to solve with syncing in Mentat was: how do I allow dev= +elopers to talk about their schema in such a way that they could get sync= +ing for nearly free=3F That a sync engine could, in most cases, automatic= +ally figure out which entities were the same, decide on a stable ordering= + of operations, and rewrite history to produced the expected merged state= +, such that in the above example all the visits stitch together, we know = +the latest title, and the log has all the historical titles=3F=0D=0A=0D=0A= +Using observation-structured storage to remove conflict is definitely a v= +aluable tool in the toolbox for doing that: the more you can structure yo= +ur data as new observations about stably identified things, the easier ti= +me you have with syncing. I believe, but have not proved, that a system w= +ith only identity properties and cardinality-many properties, perhaps wit= +h some additional restrictions on retraction and uniqueness, will never h= +ave conflicts when merging.=0D=0A=0D=0ABut I also think that the best dev= +eloper ergonomics are actually slightly _more_ expressive than even Datom= +ic or Mentat =E2=80=94 developers told me that they wanted cardinality-N,= + and to be able to express types or domain restrictions on properties, an= +d nesting, and so on =E2=80=94 and delivering those ergonomics means taki= +ng care of turning the log into a simple, concrete current state for them= + to query.=0D=0A=0D=0APhrased differently: in the above, your developers = +want some way to write=0D=0A=0D=0A SELECT url, title FROM pages=0D=0A=0D= +=0Aand=0D=0A=0D=0A SET title FOR page WITH url 'foo.com' TO 'whatever'=0D= +=0A=0D=0AWhether you do that by figuring out at sync time the current tit= +le for pages with a given URL, or whether you give them some affordance t= +o dynamically roll that up from the log, gets to be the same thing if you= + think of Mentat's 'datoms' table and schema as a way for developers to d= +escribe how Mentat should roll up current state from the log!=0D=0A=0D=0A= +=0D=0A> I used to share your view on needing a migration scape hatch, but= + I=0D=0A> don't anymore. On the company I work those kind of database mig= +rations=0D=0A> never really occur. People either create a new attribute, = +copy data from=0D=0A> a database to another, process it using ETL, comput= +e things on the fly=0D=0A> with functions, and many other strategies. We'= +ve already had to comply=0D=0A> with excisions in production, but never o= +n schema, only on data itself.=0D=0A=0D=0AI actually view what you descri= +be as migration: the point at which you're defining a new attribute and s= +toring things twice, or doing a batch job to move things over, is exactly= + a data migration, if perhaps smeared over space and time!=0D=0A=0D=0AThe= +se still have all the problems that traditional schema migrations have: d= +ealing with data that doesn't conform, and dealing with clients that don'= +t know about the new format.=0D=0A=0D=0A=0D=0A> So I don't think that's a= +n actual issue, and I would insist that the=0D=0A> "Grow your schema" app= +roach actually does work at scale, on the long=0D=0A> run, for multiple s= +cenarios.=20=0D=0A=0D=0AI agree. I also think that one of the ways it doe= +s so is by pushing the work of tracking versions and vocabulary into appl= +ication code, either formally or informally (e.g., checking for the exist= +ence of a property, coordinating the upgrade of client code, etc.) =E2=80= +=94 the clients need to know how to work with the schema as it grows!=0D=0A= +=0D=0AThis was the motivation for me spending time on vocabulary manageme= +nt, which we did end up building:=0D=0A=0D=0Ahttps://github.com/mozilla/m= +entat/wiki/Proposal:-application-schema-coordination-and-versioning=0D=0A= +=0D=0A=0D=0A> At a glance I can't really tell if that is what you mean wh= +en citing=0D=0A> SQLCipher.=20=0D=0A=0D=0AAh, we were talking about two d= +ifferent kinds of encryption!=0D=0A=0D=0AWe were using SQLCipher for loca= +l block-level encryption.=0D=0A=0D=0AAt Mozilla we would additionally use= + Firefox Account encryption to end-to-end encrypt data as it passed throu= +gh Sync. The two are complementary. Sync acts exactly as you demonstrate = +with git-remote-gcrypt: two local devices can access cleartext and are re= +sponsible for resolving conflicts, and the service itself sees only ciphe= +rtext and does not involve itself in how conflicts are resolved.=0D=0A=0D= +=0AMy point in bringing this up is that many systems have a central entit= +y that can help in resolving conflicts; if the central entity can only ev= +er see ciphertext, it's limited to whatever ordering or collision propert= +ies you enable via the envelope. For that Git system I imagine it tracks = +refs, and the Git object store is opaque.=0D=0A=0D=0A=0D=0A> TBH I'm not = +aiming for automatic conflict resolution.=0D=0A>=20=0D=0A> Why not use th= +e granularity of attributes to detect conflict, but not to=0D=0A> resolve= + them=3F=0D=0A=0D=0AIn short:=0D=0A=0D=0A* End user software is no longer= + built (as iSync was) to present data conflicts to end users for resoluti= +on. The goal is conflict resolution.=0D=0A* You cannot leave a store in a= +n inconsistent state until conflicts have been addressed, because you sti= +ll need to handle local writes. You cannot disconnect a device until conf= +licts have been addressed; users expect continuity of service. Whenever y= +ou are able, you have to do _something_.=0D=0A* _Someone_ has to write th= +e code to resolve conflicts. Eventually you're going to have some dev com= +e up with the bright idea of annotating "data objects" with some kind of = +conflict resolution attributes, and then have an automated "conflict reso= +lver"=E2=80=A6 at which point you've Greenspun yourself a sync engine and= + schema language. This works much better when it's part of the storage an= +d sync later, and can participate in transaction boundaries and annotate = +writes.=0D=0A=0D=0AThis is the doc I wrote to outline the motivations for= + Mentat at Mozilla:=0D=0A=0D=0Ahttps://mozilla.github.io/firefox-browser-= +architecture/text/0008-sync-and-storage-review-packet.html=0D=0A=0D=0AThe= + Lay Summary in particular is worth a read.=0D=0A=0D=0AWhen we built Fire= +fox for iOS we managed to put storage and syncing inside the same compone= +nt, so finally we were in a position to track changes during writes, and = +could get it right. Mentat was the next logical step: how do we let other= + teams use a storage system that will still do a good job of syncing thei= +r data without them having to get elbow deep in how a sync engine works, = +and without them having to put all the syncing gunk =E2=80=94 timestamps/= +versions, UUIDs, etc. =E2=80=94 in their data model=3F=0D=0A=0D=0AWhat do= +es Mentat need to know, schema-wise, to be able to do that kind of declar= +ative conflict resolution=3F We didn't finish answering that question, bu= +t I think the intended outcome is still valid.=0D=0A=0D=0A=0D=0A> Either = +way, the DB knows that the note has the title and body attributes=0D=0A> = +in conflict, but the author is fine. The aggregate entity itself isn't=0D= +=0A> in conflict, but some of it's attributes.=0D=0A=0D=0AThis is what Co= +uch/Pouch do, I think. Well, it actually resolves the conflict and leaves= + a conflict marker=E2=80=A6 no idea whether anyone actually writes code t= +o check that!=0D=0A=0D=0A=0D=0A> Couldn't you leverage the schema for tha= +t=3F If the said conflicting=0D=0A> attribute has:=0D=0A>=20=0D=0A> - car= +dinality one and both DBs changed it, we have a conflict;=0D=0A> - cardin= +ality many and both changed the same attribute, we have a=0D=0A> conflic= +t;=0D=0A> - cardinality may and Computer A changed an entry and Computer = +B changed=0D=0A> another one, we don't have a conflict.=0D=0A>=20=0D=0A>= + and so on.=0D=0A=0D=0AYes, that's exactly the approach we were taking. T= +he important difference is that you also need to zip together identities = +during this process =E2=80=94 for example, two assertions of a cardinalit= +y-one attribute are not a conflict if the two entities at the other end t= +urn out to be the same. Couch etc. don't have that concept.=0D=0A=0D=0A(I= +n an open-world system like OWL-DL, this cardinality conflict actually be= +comes implication of equality!)=0D=0A=0D=0A> I don't know Dropbox's datas= +tore, is this[0] the one you're refferring=0D=0A> to=3F=20=0D=0A=0D=0AYep= +! It's a really great bit of thinking.=0D=0A=0D=0A + +From eu@euandre.org Thu Sep 3 01:00:21 2020 +Received: from box.euandre.org (box.euandre.org [46.101.160.115]) + by mail-b.sr.ht (Postfix) with ESMTPS id 71E69FF0FA + for <~euandreh/public-inbox@lists.sr.ht>; Thu, 3 Sep 2020 01:00:20 +0000 (UTC) +Authentication-Results: mail-b.sr.ht; + dkim=pass (2048-bit key) header.d=euandre.org header.i=@euandre.org header.b=rMNL9qSC +Received: from authenticated-user (box.euandre.org [46.101.160.115]) + (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) + (No client certificate requested) + by box.euandre.org (Postfix) with ESMTPSA id E3DD6102122; + Wed, 2 Sep 2020 22:00:16 -0300 (-03) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=euandre.org; s=mail; + t=1599094818; bh=VQ0pHLJALciNNh1+kpFy1DVLAgL5kn8a36k3GAV6u7E=; + h=From:To:Cc:Subject:In-Reply-To:References:Date:From; + b=rMNL9qSC0p+9DH1RAD5wEPqn9/v63VAiwAiMKQuTvccTrH5EnO2qlB7mKU594brWM + P+5qCkBJymi/N2hK+BvoROdWT24A6NPyG0rORpg00rmuJsrpyn6HkpbHzOcx3CwYdH + WUVzXoZQndMFNa4nN4P/KE4Mh/jA/4TvVO/4QvfrJqpnpfKdPJeIgFEdrCEQL+3ZIP + b/DjqLRRSkV3fljHnB2MhsCMe/od5BD+wDEza/x+HvOwTxFHvalbWT1QVj3saKxiTG + VbD8xe43tls9agJsSh/z8MErjmNI4Ngd0ygfIu8QbbLNswkpQe3RMru5lYi2o49N55 + 1bnUqfFRkHs/w== +From: EuAndreh <eu@euandre.org> +To: Jonas =?utf-8?Q?Sch=C3=BCrmann?= <me@jonas-schuermann.name>, + ~euandreh/public-inbox@lists.sr.ht +Cc: +Subject: Re: The database I wish I had +In-Reply-To: <acd49073-c46e-2bae-da55-de1d8cc42533@jonas-schuermann.name> +References: <e0b3b042-5eda-bcde-59a5-a5b61fecd2e5@jonas-schuermann.name> + <87y2ltyeqg.fsf@euandre.org> + <acd49073-c46e-2bae-da55-de1d8cc42533@jonas-schuermann.name> +Date: Wed, 02 Sep 2020 21:34:51 -0300 +Message-ID: <87d033g0k4.fsf@euandre.org> +MIME-Version: 1.0 +Content-Type: text/plain; charset=utf-8 +Content-Transfer-Encoding: quoted-printable + +Jonas Sch=C3=BCrmann <me@jonas-schuermann.name> writes: + +> Yes, exactly. The idea is pretty abstract and while I know a little bit +> about category theory I'm by no means an expert on the subject. This +> blog post [1] is a very down-to-earth, practical explanation how the +> concept of (co)completed categories can be applied to conflict +> representation in version control systems. +> +> [1] https://jneem.github.io/merging/ + +This sounds interesting, thanks for the link. + +From eu@euandre.org Thu Sep 3 01:07:15 2020 +Received: from box.euandre.org (box.euandre.org [46.101.160.115]) + by mail-b.sr.ht (Postfix) with ESMTPS id 20958FF0FA + for <~euandreh/public-inbox@lists.sr.ht>; Thu, 3 Sep 2020 01:07:15 +0000 (UTC) +Authentication-Results: mail-b.sr.ht; + dkim=pass (2048-bit key) header.d=euandre.org header.i=@euandre.org header.b=Y4O+t3yg +Received: from authenticated-user (box.euandre.org [46.101.160.115]) + (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) + (No client certificate requested) + by box.euandre.org (Postfix) with ESMTPSA id B209DFC81E; + Wed, 2 Sep 2020 22:07:12 -0300 (-03) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=euandre.org; s=mail; + t=1599095234; bh=Y2H9I+SgM0rzOycDHDHlfDDvDGVB79YoDxvEzTudos4=; + h=From:To:Cc:Subject:In-Reply-To:References:Date:From; + b=Y4O+t3yg4rBzXtK6CL570EiO+7TXdlHZ1OAjSDRUjr71MbaIt+HonABTUommz90+U + +FkpruCMrLNQIyuveS07REukjQVp/zKHrH41FLFDaRUoBIg1OuxvIp4I+Ug1M+HQAC + tS1tPA0oqbqmrI7l2ByFEP58VyYhRNpYAbrpSMJUI4gt/Busfq5TFLQfnuTZSaGbNy + 9AsJYnD04/B+VRQHcrST55I8sVyaHkxQM0xck0J7iEnJesbTxGU6adg89UL7fTd4Jb + LP4PognYY5bvYrsRP1eTQVkm8Iefevp0X+fgNdQsvBkee8WLAykSq4jkMpPaBEyMl3 + WIq9JcFaLACew== +From: EuAndreh <eu@euandre.org> +To: Jeff Rhyason <jeff@rhyason.org>, ~euandreh/public-inbox@lists.sr.ht +Cc: +Subject: Re: building MSTs->CRDTs-> Re: The database I wish I had +In-Reply-To: <99C92E06-E0BC-4DB0-BF63-20FD44DC4062@rhyason.org> +References: <99C92E06-E0BC-4DB0-BF63-20FD44DC4062@rhyason.org> +Date: Wed, 02 Sep 2020 22:06:59 -0300 +Message-ID: <87v9gveki4.fsf@euandre.org> +MIME-Version: 1.0 +Content-Type: text/plain + +Jeff Rhyason <jeff@rhyason.org> writes: + +> It's amazing to see this discussion. Let's keep it going! +> +> My friend and I started Indelible[1] (now defunct) for a lot of the +> same reasons you want to build a datastore -- there aren't enough +> options for a datastore that's immutable, relational, +> highly-vaailable, and private.. We're still super-passionate about the +> ideas, love Datomic but want something a bit different, decided not to +> commercialize it and start more simply with just the data structure. +> We don't want a huge thing, just a library, to keep coordination down; +> the complexity doesn't spread out of the client. +> +> Where we are at now is we have a great versioned immutable B-Tree-like +> structure that is the foundation for the verioned and diffable +> database we want. FWIW The Go implementation of the tree is at +> https://github.com/jrhy/mast . We have plans to do a CRDT based on +> it, possibly add an SQL layer, but frankly, having a +> strongly-consistent, verioned + diffable tree, is pretty good for now. +> We'd love to collaborate! + +This looks interesting! I'll take a deeper look later. + +I'll be particularly interested in seeing the implementation of +persistent B-Trees that are durable. + +From eu@euandre.org Thu Sep 3 23:39:44 2020 +Received: from box.euandre.org (box.euandre.org [46.101.160.115]) + by mail-b.sr.ht (Postfix) with ESMTPS id B8167FF1D6 + for <~euandreh/public-inbox@lists.sr.ht>; Thu, 3 Sep 2020 23:39:43 +0000 (UTC) +Authentication-Results: mail-b.sr.ht; + dkim=pass (2048-bit key) header.d=euandre.org header.i=@euandre.org header.b=oQR0LJk6 +Received: from authenticated-user (box.euandre.org [46.101.160.115]) + (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) + (No client certificate requested) + by box.euandre.org (Postfix) with ESMTPSA id 2F2B2FCB45; + Thu, 3 Sep 2020 20:39:39 -0300 (-03) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=euandre.org; s=mail; + t=1599176381; bh=0OTcSYr5H5tf2Nj44ZROMSLLvq65wMQATwFWuwPcGcA=; + h=From:To:Cc:Cc:Subject:In-Reply-To:References:Date:From; + b=oQR0LJk66tmm7FUnmKKEUg3GVM6KyB0ybBX7bvke8hAkcTr/UOADsGEa64ZOgq47F + 5+6d43VfXXhxXQie9IypcAHdqe5eYTXL0Yh6ymknjM9GIPHEhGRgzbK+jeTe6EK2KH + y2bC4HqyQ0Xn9Ijj+m+OEQCACknw1vsIvVH3qQveSft3mRmIKIibpDL5uxMaGbNRHK + V0ts3LFy07Knz4QI5fsPRqBwsKNQV9t+F5eL5ITa6OSC3JY+QclEbcYheBW80V42IT + 8+hb6HzscpFlxvBtdi1fG7PbnUtWAp/7j6sK8AtVYRCK5OybOAy0VpJhGGbFCZdale + s3u7XVTPdzmeQ== +From: EuAndreh <eu@euandre.org> +To: Richard Newman <rnewman@twinql.com> +Cc: "~euandreh/public-inbox@lists.sr.ht" <~euandreh/public-inbox@lists.sr.ht> +Cc: +Subject: Re: The database I wish I had +In-Reply-To: <010101744f99448f-fa8dd88e-0daf-41f8-b3ca-6ea551c300e4-000000@us-west-2.amazonses.com> +References: <7454DA45-C5A4-4A54-8D02-1F3BD37726E9@twinql.com> + <010101744a592b75-1dce9281-f0b8-4226-9d50-fd2c7901fa72-000000@us-west-2.amazonses.com> + <87sgc1xu7k.fsf@euandre.org> + <9652AAE0-10DF-4D13-ABE8-B30667BE27DD@twinql.com> + <010101744b260d62-1bb508ab-647f-4877-bdf9-f988667d56e1-000000@us-west-2.amazonses.com> + <878sdsy6sc.fsf@euandre.org> + <7027FD8B-1B30-435F-A840-7DC61A722E0B@twinql.com> + <010101744f99448f-fa8dd88e-0daf-41f8-b3ca-6ea551c300e4-000000@us-west-2.amazonses.com> +Date: Thu, 03 Sep 2020 20:39:23 -0300 +Message-ID: <87h7se5t1w.fsf@euandre.org> +MIME-Version: 1.0 +Content-Type: text/plain; charset=utf-8 +Content-Transfer-Encoding: quoted-printable + +I started quoting many parts of your email, and things started to get +confusing. Each answer that I gave was referring to each other, and I +almost made a loop. + +So I've rewritten everything as a single piece of prose. I added some +snippets of your text to better contextualize what I was talking about, +but I tried to address what you said directly. Here it goes. + +# On migrations + +I still don't think that I've insisted enough on *strict* schema growth. + +The wiki page you pointed to reinforces that: it talks a lot about how +to coordinate attributes that changed their names, change in vocabulary, +shared ownership of changes in schema and even schema versioning! + +All of them are the opposite of what I want, and they're "schema +breakage", not "schema growth". + +What I mean by that is the same thing that Rich Hickey proposes as a +what growth and breakage means in his talk "Spec-ulation"[0]. + +All of those forms of schema change are instances of breakage, which +should have no place here. They're all sources of undesired +incompatibility. I'm not sure if Datomic allows you to change the type +of an attribute, and if it does this can't be copied, since we don't +have a transactor to coordinate such change. + +If you want to rename an attribute, don't! Create a new one! Maybe you +could retract schema and assert back the same schema, but you shouldn't +overwrite it with an incompatible name, or attribute type. This is a +trap that you shoudln't put yourself in, and maybe even this could be +enforced. + +This model of an ever-growing vocabulary is how HTML works. At some +point people realised that using <b> to mark something as bold was not +such a good idea, since it coupled semantics with presentation. So a new +tag was created, <strong>, which only represents the semantic part of +<b>. And <b> still continues to exist, and that's fine. + +The same can be said about JavaScript. I've written a fair share of it +in the past 2 years, and I don't think I ever used the 'var' keyword, +only 'let' and 'const'. And yet, every JavaScript runtime still +recognizes it, and even transpiler emit it as their output, and that's +also fine. + +Maybe a linter or the browser tooling would warn you against the usage +of those old features, but they continue to work. If an equivalent of a +"vocabulary change" happened, web pages would be brittle by design more +brittle, and old pages would stop working as time passes. That isn't a +principle I would like to adopt. + +When more than one database instance can add data, the programmer needs +to think of it as an open world, where new things are accrued, but old +things don't disapper. Changing the vocabulary mid-air only makes this +open world brittle so "migrations" can be performed. + +I see no reason for having this type of migration you describe, and +insist on *strict* schema growth, it being a saner, more robust and +completly fine alternative. + +[0]: https://www.youtube.com/watch?v=3DoyLBGkS5ICk + +# On conflicts and ergonomics + +You mentioned ergonomics thrice on your message, and I believe that your +insistence of it is a sign of the coupling of two different layers of +the database, and makes ease of use drive the fundamentals. I also think +that your statement on conflict resolution being a requirement is not a +good idea. Let me expand on that. + +I'd argue that conflict resolution is inherit complexity [1] of creating +a decentralized client-side application that can sync data. + +That means that the responsability to solve conflicts is not the +database's, but the application's responsability, and making decisions +of conflict resolution on the database limits its application, and I +wouldn't consider it to be general purpose. + +When two offline database instances change the same field, there's no +escaping from the bottom reality, which is: there is a conflict, and it +needs to be resolved somewhere. There's no layer of ergonomics that can +change this fact, and picking a strategy for conflict resolution is +undoubtly domain-specific. + +You mentioned that I proposed a solution on URL history tracking that +fowards all of the responsability to the programmer, but I was only +giving a simple example, and that same problem could be tackled in many +different ways, and the person creating the application has a decision +to make on it. + +Let's expand on the URL history on the browser example from earlier, and +explore how to handle identity. + +The application developer now has to decide what they want to consider a +URL being "the same" as another, and that's what I think you meant when +you talked about pushing domain identity to the application. + +What does the programmer wants the identity of a URL to be? Is it a +UUID attributed to it when asserting the entitiy on the database? If so, +you would need to do things like the SELECT DISTINCT and SELECT SUM I +proposed, inevitably. The programmer may otherwise try to avoid this by +always starting from an existing database, so the history could be +common whenever a new database instance starts. One would just not +simply 'git init', they would 'git clone' in order to work with shared +identities that are created in a distributed way. + +Is the identity of a URL its string? If so, the URL would have to be +treated as an URI, and that string would be effectively global, and +updates would have to be handled with lots of care. + +The database *can't* make those decision. What it *can* do is provide +tools that allow the programmer to work with this inherent complexity. +And I see no way around it. No amout of ergonomics can solve this, and +removing the ability to have 1:1 lookups is the nature of decentralized +client-side applications, it's not me trying to forward a problem that +the database should solve to the programmer.=20 + +A simple comparison is: whenever we talk about code, we use SHAs to +pinpoint which piece of code we mean. What does a SHA say about the +existence of other branches that disagree? Nothing! You can't ever +assume that the commit you're on is the latest version of a file, or +even is the only version of a file. While you were not looking, any +number of new versions could have been created, which means that when +you fetch, there isn't a single README. There are *at least* two of +them: README from master, and README from origin/master. They may be the +same, and they may not be the same, but saying just "the README file" is +an incomplete depiction of it: you either say "my README file" or talk +about both of them, which do coexist. + +There's no denial that this model is much more complex than the usual +1:1 lookup we're used to, but trying to hide this in the database +implementation somehow would be a bad idea. The programmer brought this +to themself when they choose to build a decentralized client-side +application.=20 + +However, with a solid foundation, there are still lots of room for work +on ergonomics, like:=20 + +1. diffing algorithms for text like you mention would totally +make sense to be in a "contrib/" directory, so that people could pick +different diffing algorithms for different string fields. + +2. whenever a 'dev comes up with the bright idea of annotating "data +objects" with some kind of conflict resolution', I could point them to a +"IConflictResolutionStrategy" interface for they to implement. Their +implementation could even end-up under said "contrib/" directory. Even +better if "IConflictResolutionStrategy" could be composed! A programmer +would then pick different strategies for different entities in their +domain, like in a buffet. + +3. offer some low-level building blocks that can be applied to different +domains, like tombstones, max(a.last-modified, b.last-modified), etc. + +...and so on. + +The point is: the database is not in a position to make this type of +decision. Only the programmer is, and they must be aware of it. + +You said that: + +> the goal of a database is to help me efficiently, accurately, and +> ergonomically represent my domain=20 + +For me, the crucial part is the accurately comes first, and ergonomics +comes second. No accuracy should be sacrificed in name of ergonomics, +since the inherent complexity is already too big to be dealt with. + +The syncing part is not free. + +[1]: As defined in http://curtclifton.net/papers/MoseleyMarks06a.pdf + +# On history rewriting + +You talked about how Mentat was trying to reconcile changes by rewriting +the history in a stable and consistent order. + +I'd argue that instead of a linear log where things are rebased, a DAG +log where things are merged is a more flexible approach. Let me show you +where I'm coming from first. + +You said that: + +> Phrased differently: in the above, your developers want some way to +> write +> +> SELECT url, title FROM pages +> +> and +> +> SET title FOR page WITH url 'foo.com' TO 'whatever' + +This may still be true, but with a caveat: you'd be asserting facts to +the local "timeline", in a somewhat similar way that the Mentat wiki +describes[2] (I feel funny giving you a link to Mentat's wiki).=20 + +What I'd do differently is to don't try so hard to make everyone +converge on the same thing always, and represent timelines in a more +flexible way. + +Instead of having two types of timelines, =CE=B1 and =CE=B2, there would be= + a +multitude of timelines, one for each instance of the database. Let's say +we have 3 of them for the purpose of explanation. + +Instead of adding an extra 'timeline_id' identifier, I'd add a +':db/txParent' to the transaction entity, with cardinality many, and the +database instance would always know it's latest local parent. This is +effectivelly like a branch, but not a shared one, and transations with +more than one ':db/txParent' represent merges. Those would only happen +when explicitly solving conflicts, and any given database instances +could know about a multitude of timelines. + +This way, how can one know the current state of the database?=20 + +>From the first root datom, the database would rollup every datom taking +the timelines and merges in consideration, and it would be the sum of +data from all timelines. When a conflict exists, the local timeline is +always picked so the database can still serve writes, and the conflict +would be presented when serving reads.=20 + +The obvious problem of always favouring the local timeline is that each +timeline would show different "primary" answers, but that's sounds +somewhat reasonable to me to enable the database to keep working while +offline, and while the conflict isn't resolved. + +This also means that "sync" are two distinct things: exchanging timeline +data, and reconciling that data locally. The only part that's online is +the "exchange", and instances resolve conflict locally. This way, the +database is always available, even when offline, and even when conflict +is present. + +But how would queries over history behave? No clue yet =F0=9F=98=AC. + +I tried not to sound too hand wavy, and I've only prototyped a small +part of this in the past, so there are probably many more problems with +this approach. I don't claim this is a sound solution yet, but it does +handle some of the problems that Mentat faced in a different way. I'd be +interested to see you poking holes on it.=20 + +When combined with strict schema growth, new data always comes after +it's schema declaration, so the application doesn't have to coordinate +around distributed schema evolution. + +[2]: https://github.com/mozilla/mentat/wiki/Multiple-timeline-support + +# On CouchDB/PouchDB's entity conflicts + +CouchDB/PouchDB's most fundamental value is the document. This means +that a single document write is atomic (not true for clustered CouchDB), +but you end up rewritting the full document ([3], on time 3:35). If you +represent an entity as a document, the whole entity is in conflict, not +the attribute that changed. + +So it doesn't know the difference between two changes on unrelated +attributes and full rewrites of the full document: everything is a +conflict in CouchDB/PouchDB's view. + +On another talk ([4] on time 5:30, also by Joan Touzet), she mentions +the desire to add partial updates to CouchDB, but I don't know if it was +pursued. + +Checking for conflicts on revisions is considered good practice ([1], on +time 10:21) on the community. + +[3]: https://www.youtube.com/watch?v=3DBKQ9kXKoHS8 +[4]: https://www.youtube.com/watch?v=3D0gAZHIYBh-g + +# On zipping identities together + +=F0=9F=91=8D + +# Misc + +TIL: I didn't know you worked on AllegroGraph. + +P.S.: I haven't reviewed this text too much, and it might have gotten a +bit confusing. I'm happy to clarify any points, and I'll consider +writing a more formal version of this braindump later. I chose to send +this like this over taking more time and letting the discussion cool +down.=20 + +From eu@euandre.org Fri Sep 4 00:00:13 2020 +Received: from box.euandre.org (box.euandre.org [46.101.160.115]) + by mail-b.sr.ht (Postfix) with ESMTPS id 7A675FF1B9 + for <~euandreh/public-inbox@lists.sr.ht>; Fri, 4 Sep 2020 00:00:12 +0000 (UTC) +Authentication-Results: mail-b.sr.ht; + dkim=pass (2048-bit key) header.d=euandre.org header.i=@euandre.org header.b=teyCS1z1 +Received: from authenticated-user (box.euandre.org [46.101.160.115]) + (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) + (No client certificate requested) + by box.euandre.org (Postfix) with ESMTPSA id 3507CFCB45; + Thu, 3 Sep 2020 21:00:09 -0300 (-03) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=euandre.org; s=mail; + t=1599177611; bh=mYcY4i3Y9/GUAx3/nY2Lu91JMvyDWaWOxTADQnXB+MM=; + h=From:To:Cc:Cc:Subject:In-Reply-To:References:Date:From; + b=teyCS1z1iKrOm7S9ivB92sQQ5uq6Kb9TRtkrIvyFYqPWIP9zFVq4sYuuU5NQZwDSi + GHHeO9otqioa+dWLVcXYITaxTvgD3uAaiI/Xa9Nzj3swN1DWuzGwJ7Ovqf8vQDBSQI + 0VROFULtzElG2iNTv3tyWy6C7Hu12whmbfteUtq6aTzonf9pFIP6Wio6wGakmS9px9 + xwKDfuZgrdN8tUWhjAM+Joz9bwRgRg08+hJOf9Jd9JKShTgZGosTIOYDl8HnfQwR7F + fZbYza/n4vBwLdkpDlOpX9wVdVprXmvM3cD+zoPQ/ukRomUmjQ6KFETBOksZ4t7pY1 + E/KZ/wEEBxRJA== +From: EuAndreh <eu@euandre.org> +To: Richard Newman <rnewman@twinql.com> +Cc: "~euandreh/public-inbox@lists.sr.ht" <~euandreh/public-inbox@lists.sr.ht> +Cc: +Subject: Re: The database I wish I had +In-Reply-To: <87h7se5t1w.fsf@euandre.org> +References: <7454DA45-C5A4-4A54-8D02-1F3BD37726E9@twinql.com> + <010101744a592b75-1dce9281-f0b8-4226-9d50-fd2c7901fa72-000000@us-west-2.amazonses.com> + <87sgc1xu7k.fsf@euandre.org> + <9652AAE0-10DF-4D13-ABE8-B30667BE27DD@twinql.com> + <010101744b260d62-1bb508ab-647f-4877-bdf9-f988667d56e1-000000@us-west-2.amazonses.com> + <878sdsy6sc.fsf@euandre.org> + <7027FD8B-1B30-435F-A840-7DC61A722E0B@twinql.com> + <010101744f99448f-fa8dd88e-0daf-41f8-b3ca-6ea551c300e4-000000@us-west-2.amazonses.com> + <87h7se5t1w.fsf@euandre.org> +Date: Thu, 03 Sep 2020 20:57:28 -0300 +Message-ID: <87eeni5s7r.fsf@euandre.org> +MIME-Version: 1.0 +Content-Type: text/plain + +I better get a POC so I can stop talking in "what ifs". + +I was more focused on the storage implementation side of it, but I'm +considering not implementing a storage backend for this and just using +SQLite instead (more detailed in [0]). + +Building durable persistent data structures on top of SQLite backed +storage is still a bit distant, but the model I presented could benefit +from a POC. + +I'll try starting that instead of giving so much focus on storage, and +share it later. + +[0]: https://euandre.org/2020/08/31/the-database-i-wish-i-had.html#fn:posix-sqlite + +From eu@euandre.org Fri Sep 4 02:12:34 2020 +Received: from box.euandre.org (box.euandre.org [46.101.160.115]) + by mail-b.sr.ht (Postfix) with ESMTPS id DE901FF196 + for <~euandreh/public-inbox@lists.sr.ht>; Fri, 4 Sep 2020 02:12:33 +0000 (UTC) +Authentication-Results: mail-b.sr.ht; + dkim=pass (2048-bit key) header.d=euandre.org header.i=@euandre.org header.b=aOTLZVo3 +Received: from authenticated-user (box.euandre.org [46.101.160.115]) + (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) + (No client certificate requested) + by box.euandre.org (Postfix) with ESMTPSA id 13742FCB45; + Thu, 3 Sep 2020 23:12:30 -0300 (-03) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=euandre.org; s=mail; + t=1599185552; bh=0eOLRgKRFwQtp0D1PSgenzUsGttO7assNfrY8hnPqz8=; + h=From:To:Cc:Cc:Subject:In-Reply-To:References:Date:From; + b=aOTLZVo3JPMEsqZSTB2G6Y+T0rDJDcBXrDuU5+R0SjJN9lX+c01/ir+CO/+Av22VH + IhKMuNJU1nVjzOf4Qs0CnmJO1Agzeuq+pf41gU0qIx0kZVSaUZP8PgXzpG1DUvfQ5T + tEKNEHhRVFapOZ9HYeYzbMcUdO4z4haaEkuxerFOwwvIudLPkmCCeJH7xFouMvkMSN + vxyKaJK7RdWzBkiXZutwKzS6BEbvbalzdMf7IcLXAlmYsUdV3bMPH0Bvg5HsxvLY1I + YQNTiPp8yRiojv69hKqGKByUvhPveXVtbZNt3k8y4cwU5UAaAXnTQrNwSFhy3pxB+9 + fwaRNRHiXi1mQ== +From: EuAndreh <eu@euandre.org> +To: Richard Newman <rnewman@twinql.com> +Cc: "~euandreh/public-inbox@lists.sr.ht" <~euandreh/public-inbox@lists.sr.ht> +Cc: +Subject: Re: The database I wish I had +In-Reply-To: <87eeni5s7r.fsf@euandre.org> +References: <7454DA45-C5A4-4A54-8D02-1F3BD37726E9@twinql.com> + <010101744a592b75-1dce9281-f0b8-4226-9d50-fd2c7901fa72-000000@us-west-2.amazonses.com> + <87sgc1xu7k.fsf@euandre.org> + <9652AAE0-10DF-4D13-ABE8-B30667BE27DD@twinql.com> + <010101744b260d62-1bb508ab-647f-4877-bdf9-f988667d56e1-000000@us-west-2.amazonses.com> + <878sdsy6sc.fsf@euandre.org> + <7027FD8B-1B30-435F-A840-7DC61A722E0B@twinql.com> + <010101744f99448f-fa8dd88e-0daf-41f8-b3ca-6ea551c300e4-000000@us-west-2.amazonses.com> + <87h7se5t1w.fsf@euandre.org> <87eeni5s7r.fsf@euandre.org> +Date: Thu, 03 Sep 2020 23:02:55 -0300 +Message-ID: <87a6y65meo.fsf@euandre.org> +MIME-Version: 1.0 +Content-Type: text/plain + +Here's what I was referring to when talking about "durable persistent +data structures": https://www.youtube.com/watch?v=Cym4TZwTCNU + +From eu@euandre.org Sat Sep 5 09:07:44 2020 +Received: from box.euandre.org (box.euandre.org [46.101.160.115]) + by mail-b.sr.ht (Postfix) with ESMTPS id 6A917FF1F5 + for <~euandreh/public-inbox@lists.sr.ht>; Sat, 5 Sep 2020 09:07:43 +0000 (UTC) +Authentication-Results: mail-b.sr.ht; + dkim=pass (2048-bit key) header.d=euandre.org header.i=@euandre.org header.b=grYKK93t +Received: from authenticated-user (box.euandre.org [46.101.160.115]) + (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) + (No client certificate requested) + by box.euandre.org (Postfix) with ESMTPSA id 92EB0FC83F; + Sat, 5 Sep 2020 06:07:40 -0300 (-03) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=euandre.org; s=mail; + t=1599296862; bh=RjoAEI4vfrQ1VCfrySR+wMnwePv+AnUVT1ZbWcm8O1A=; + h=From:To:Cc:Cc:Subject:In-Reply-To:References:Date:From; + b=grYKK93td5WpquSSrkfBXBzUeJ25jSXwwjOE/x5c8rfR1DVAT1sDu7NfGVP+80pBW + oG7IM1+Kpjz7PL15Bg01iDOXTLXMD/4GeikxqSUUG2I9Pj0NEO4eNzl6hNTfNDBuMA + E7cz3qVVonEDHIYGlXvavHAWxWU6MXoSdDvEn0K2zNPU3qmo5PJ0vTKYNQTPkInY1N + DRSUHv6DgfTUUYyrXdcw0rIPhQdNvobloBT29Q7LOtvXcBtiMhWlLMG30ifV+ETmhU + h6AIw5myD387Nf7GRL6TfmtM1zWotf92jSop38NRnUCOJQ8iBpUrTt74qaSsuhcDbx + p4C1GSTXxTj2g== +From: EuAndreh <eu@euandre.org> +To: Richard Newman <rnewman@twinql.com> +Cc: "~euandreh/public-inbox@lists.sr.ht" <~euandreh/public-inbox@lists.sr.ht> +Cc: +Subject: Re: The database I wish I had +In-Reply-To: <87a6y65meo.fsf@euandre.org> +References: <7454DA45-C5A4-4A54-8D02-1F3BD37726E9@twinql.com> + <010101744a592b75-1dce9281-f0b8-4226-9d50-fd2c7901fa72-000000@us-west-2.amazonses.com> + <87sgc1xu7k.fsf@euandre.org> + <9652AAE0-10DF-4D13-ABE8-B30667BE27DD@twinql.com> + <010101744b260d62-1bb508ab-647f-4877-bdf9-f988667d56e1-000000@us-west-2.amazonses.com> + <878sdsy6sc.fsf@euandre.org> + <7027FD8B-1B30-435F-A840-7DC61A722E0B@twinql.com> + <010101744f99448f-fa8dd88e-0daf-41f8-b3ca-6ea551c300e4-000000@us-west-2.amazonses.com> + <87h7se5t1w.fsf@euandre.org> <87eeni5s7r.fsf@euandre.org> + <87a6y65meo.fsf@euandre.org> +Date: Sat, 05 Sep 2020 06:07:31 -0300 +Message-ID: <878sdosib0.fsf@euandre.org> +MIME-Version: 1.0 +Content-Type: text/plain; charset=utf-8 +Content-Transfer-Encoding: quoted-printable + +> The obvious problem of always favouring the local timeline is that each +> timeline would show different "primary" answers, but that's sounds +> somewhat reasonable to me to enable the database to keep working while +> offline, and while the conflict isn't resolved. + +Also a possible ergonomics layer that can be built on top of some lower lev= +el +primitive: + +Suppose we want to add a profile page to an app, but the user will be +able to edit their profile while offline. + +Such profile page contains 3 parts: name, bio and links. Here's how it +could look like: + +{:profile/name "Foo Bar" + :profile/links #{{:link/title "Personal webpage" + :link/position 0 + :link/address #uri "https://example.com"} + {:link/title "Mastodon profile" + :link/position 1 + :link/address #uri "https://example.org"} + {:link/title "Photo album" + :link/position 2 + :link/address #uri "https://example.net"}} + :profile/bio "My lengthy bio (...)"} + +For each of those attributes, one could want to handle conflicts in a diffe= +rent +way: + +1. For the :profile/name, you could want to be fully conservative, and never + resolve any conflict whatsoever. If the user changes their name to "Foo = +Bax" + in one device and to "Foo Baz" in another, you may choose to not solve t= +his + conflict on the application. Instead you could just pick one of them to = +show + to the user on their profile page, say "Foo Baz", but add an exclamation= + mark + icon =E2=9D=97=EF=B8=8F beside it. When the user touches the icon, the a= +pp asks: "do you want + to keep 'Foo Baz' or would you like to change it to 'Foo Bax'?". This co= +uld + be done multiple times for multiple concurrent conflicts, and when user + confirms their choice, that's a merge that resolves the conflict for all + instances when propagated. + + The upside is that the app doesn't try to be smart about reconciling the + :profile/name, showing things like "Foo Baxz" or "Foo BazFooBax". The + downside is that if the app doesn't handle this explicitly, they'll be + perpetually inconsistent. + +2. For the :profile/links, specifically the :link/position attribute, you m= +ight + try an intermediate approach. Say you want to handle reordering less + conservatively. Say the user changes the position of the "Photo album" l= +ink + to 1 in a device, and to 0 on another device, and that is their actual + desired final position for this link. You could solve it by doing + something trivial, like picking the transaction UUID with the lowest + alphanumerical value. This gives you consistent choices across instances= +, and + a potential nuisance for the user: if the winning final state is with "P= +hoto + album" a 0, nothing needs to be done, and all instances agree with this.= + If + it isn't, then the user should reorder again, and all instances would st= +ill agree. + + The upside of this is that this conflict resolution strategy allows you = +to + never stay conflicted for this particular attribute, and you could have = +1:1 + lookups back. The downside is that this assumes that the transactions th= +at + are reordering links contain only :link/position attribute asserts. + +3. For the :profile/bio, you could be extremely optimistic, and try to comb= +ine + all edits. This could be done by modeling it using commutative edits, si= +milar + to how in [0], and let everything always converge. Maybe even adopt the + binary enconding of edits proposed in [1], and encode all of the edit hi= +story + in the database itself. They talk a lot about the metadata overhead of t= +he + edit history, and recording every edit as they come in would also add a + datom-related overhead, too. It might be more advantegeous to accumulate= + the + edit history, and when the app closes or a relevent event occurs, all ed= +its + are combined into a single binary, and that would be a single datom entr= +y. + This would remove the datom-related overhead while still persisting the = +full + edit history. + + The upside of this is that the text could always converge, you would get + 1:1 lookups back and you wouldn't have to deal with diffs on a huge stri= +ng, + neither would the user. The downside is that you'll need a layer on top = +of the + transactions to actually derive the text, and querying would get trickie= +r. + The database would give you the ordered list of all binary enconding thi= +ngs, + and a function would turn that into text. + + Another downside is that this is very much an open research problem, and + probably has a lot of maturity to be gained. It would be very powerful if + this could be built on top of some database primitive, without the datab= +ase + having to explicitly support it. + +All of that to handle the conflicts of *each attribute*, and they're all +the programmer's responsability to choose. + +If the database was versatile enough, an ideal world the programmer could w= +rite: + +;; schema definition +[{:db/ident :profile/name + :db/valueType :db.type/string + :db/cardinality :db.cardinality/one + :db/conflict-strategy :db.conflict-strategy/none} + {:db/ident :link/position + :db/valueType :db.type/long + :db/cardinality :db.cardinality/one + :db/conflict-strategy :db.conflict-strategy/simple-winner} + {:db/ident :profile/bio + :db/valueType :db.type/string + :db/cardinality :db.cardinality/one + :db/conflict-strategy :db.conflict-strategy/automerge} + ...] + +And be done with it. + +If those :db/conflict-strategy fields could be open and extensible, the dat= +abase +could learn new algorithms to deal with conflict without changing it's +internals, and people could teach how translate the :profile/bio from +:db.type/string the actual :db.type/bytes that would be actually stored. + +Also, I think this is what I was trying to say with "coupling two differente +layers of the database". However, I haven't tested any of this yet. + +[0]: https://doi.org/10.1016/j.jpdc.2010.12.006 +[1]: https://github.com/automerge/automerge-perf/tree/master/columnar + +From rnewman@twinql.com Sat Sep 5 20:04:34 2020 +Received: from a58-61.smtp-out.us-west-2.amazonses.com (a58-61.smtp-out.us-west-2.amazonses.com [54.240.58.61]) + by mail-b.sr.ht (Postfix) with ESMTPS id D36DAFF13B + for <~euandreh/public-inbox@lists.sr.ht>; Sat, 5 Sep 2020 20:04:32 +0000 (UTC) +Authentication-Results: mail-b.sr.ht; + dkim=pass (1024-bit key) header.d=twinql.com header.i=@twinql.com header.b=qiZnhSkC; + dkim=pass (1024-bit key) header.d=amazonses.com header.i=@amazonses.com header.b=MQAUAk7Q +DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; + s=qvs2k2sd2ivtrflyu2nswmravuqizjml; d=twinql.com; t=1599336271; + h=Subject:From:To:Cc:Date:Mime-Version:Content-Type:Content-Transfer-Encoding:In-Reply-To:References:Message-Id; + bh=lPgy0nUilFxkCQGCuQxBgZW5Udn+JVGxl6bDms54Sk4=; + b=qiZnhSkCMRBNxcfwIf1Ecic54ZZmlMpioyl5vuE02tb5tFBnHk+bAFJcpIwJInMS + EdJiWj0xSnX7x4dBpD6JqwFHvQfx3wvs+tW6yxyAvjIQcFYb4qoC8ufiI33atGIOW/R + rrhRvIaDsmSyeT53ab+mytd/6WCxBnufMhHh5rrE= +DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; + s=hsbnp7p3ensaochzwyq5wwmceodymuwv; d=amazonses.com; t=1599336271; + h=Subject:From:To:Cc:Date:Mime-Version:Content-Type:Content-Transfer-Encoding:In-Reply-To:References:Message-Id:Feedback-ID; + bh=lPgy0nUilFxkCQGCuQxBgZW5Udn+JVGxl6bDms54Sk4=; + b=MQAUAk7QsnWFnb7gUs7do0tFKJ/ZEjJ+XsIFxth+m525oY5TIo0jyYTBcFxAeQU3 + ruo+SqNRgJBJVtzqIHoghOuG8UqNq50MkyFLgv+agjPe95SKqQPIgnHeraWBXXccyjN + C42JTZ/q9uSxe/Kxy2m7iHmbpGF3+O3tP1PMHXLk= +Subject: Re: The database I wish I had +From: =?UTF-8?Q?Richard_Newman?= <rnewman@twinql.com> +To: =?UTF-8?Q?EuAndreh?= <eu@euandre.org> +Cc: =?UTF-8?Q?=7Eeuandreh/public-inbox=40lists=2Esr=2Eht?= + <~euandreh/public-inbox@lists.sr.ht> +Date: Sat, 5 Sep 2020 20:04:31 +0000 +Mime-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: quoted-printable +In-Reply-To: <87h7se5t1w.fsf@euandre.org> +References: <7454DA45-C5A4-4A54-8D02-1F3BD37726E9@twinql.com> + <010101744a592b75-1dce9281-f0b8-4226-9d50-fd2c7901fa72-000000@us-west-2.amazonses.com> + <87sgc1xu7k.fsf@euandre.org> + <9652AAE0-10DF-4D13-ABE8-B30667BE27DD@twinql.com> + <010101744b260d62-1bb508ab-647f-4877-bdf9-f988667d56e1-000000@us-west-2.amazonses.com> + <878sdsy6sc.fsf@euandre.org> + <7027FD8B-1B30-435F-A840-7DC61A722E0B@twinql.com> + <010101744f99448f-fa8dd88e-0daf-41f8-b3ca-6ea551c300e4-000000@us-west-2.amazonses.com> + <87h7se5t1w.fsf@euandre.org> + <B032A2E0-0679-4FF6-9A80-06450A8F36B1@twinql.com> +X-Priority: 3 (Normal) +X-Mailer: Amazon WorkMail +Thread-Index: AQHWgHcPJ4udfXYrRaOwhoO7EcsDJQAErSAlAAfQ7+gAIgKd3wAzRhy+AHUc9SwA0i0RQQ== +Thread-Topic: The database I wish I had +X-Original-Mailer: Apple Mail (2.3445.104.14) +X-Wm-Sent-Timestamp: 1599336269 +Message-ID: <010101745fdecd5f-52ab4d88-26f1-4a6d-879d-3234c1956260-000000@us-west-2.amazonses.com> +X-SES-Outgoing: 2020.09.05-54.240.58.61 +Feedback-ID: 1.us-west-2.An468LAV0jCjQDrDLvlZjeAthld7qrhZr+vow8irkvU=:AmazonSES + +Ah, thank you for the conversation refactor!=0D=0A=0D=0A> The wiki page y= +ou pointed to reinforces that: it talks a lot about how=0D=0A> to coordin= +ate attributes that changed their names, change in vocabulary,=0D=0A> sha= +red ownership of changes in schema and even schema versioning!=0D=0A>=20=0D= +=0A> All of them are the opposite of what I want, and they're "schema=0D=0A= +> breakage", not "schema growth".=0D=0A=0D=0AMy two observations are thes= +e:=0D=0A=0D=0AOn the one hand, front-end developers tend to think in term= +s of change =E2=80=94 "this thing's title was X and is now Y" =E2=80=94 a= +nd states =E2=80=94 "give me a list of things and their current titles so= + I can search them or show them in a table".=0D=0A =20=0D=0AThat view of = +change extends to schema/data, because the schema stretches most of the w= +ay towards the front-end itself, sometimes all the way to a "view model" = +in the MVVM sense.=0D=0A =20=0D=0AAnd it involves migration, because no f= +ront-end developer (or back-end developer, for that matter) wants to have= + to write queries that accommodate all of the different ways a particular= + concept might have been represented in the last ten years. Doing so is m= +entally taxing, is very difficult to test, and results in wildly variable= + performance. They want to fix their data once, or define how to roll it = +up into a single state once, and make sure the old representation never g= +ets used again. And these representations do change, often significantly;= + making sure they don't requires tremendous up-front deep thought (for ex= +ample, my modeling attempt for embedded videos), and results in something= + that is often normalized to the point of non-viability (already a concer= +n with tuple stores).=0D=0A=0D=0AOn the other hand, storage/syncing devel= +opers learn to love immutable logs, because it makes a ton of sense. Sync= +able storage systems should be based on an immutable timeline of appended= + observations of the world.=0D=0A=0D=0A=0D=0AThere is obvious conflict be= +tween these two. Resolving that conflict involves some complexity, and I = +felt it better to address that complexity once, inside the storage system= +=2E=0D=0A=0D=0AI didn't feel that it was worthwhile to build a storage sy= +stem that required front-end developers to adjust their mental model so f= +ar from what they're trying to achieve on a daily basis, because they hav= +e a choice in which tools they use.=0D=0A=0D=0AFrom my perspective, the p= +rimary goal has to be to give them a system they're willing to use (a cle= +ar, current-focused schema and the ability to represent change), with the= + additional goal of a useful log and better formalisms underneath to supp= +ort syncing. Putting it the other way 'round would simply lead to them us= +ing SQLite or JSON files, omitting syncing, building bad ad hoc sync solu= +tions, and otherwise continuing in the way that Mozilla had been doing th= +ings.=0D=0A=0D=0AMentat, like Datomic, is a stateful layer on top of log-= +structured data, with the current schema =E2=80=94 like the current state= + of the world! =E2=80=94 being materialized from the log.=0D=0A=0D=0A=0D=0A= +> When more than one database instance can add data, the programmer needs= +=0D=0A> to think of it as an open world, where new things are accrued, bu= +t old=0D=0A> things don't disapper. Changing the vocabulary mid-air only = +makes this=0D=0A> open world brittle so "migrations" can be performed.=0D= +=0A=0D=0AI am a fan of open-world systems =E2=80=94 I spent 7+ years work= +ing on semantic web stuff! =E2=80=94 but I'm not aware of any client appl= +ications with any degree of adoption that use the open-world model. Havin= +g multiple components share state doesn't require an open world per se.=0D= +=0A=0D=0ABeing able to evolve the representation of existing data, just l= +ike being able to 'change' the state of the world, is regarded as a *feat= +ure* by developers, and they typically would rather sacrifice interoperab= +ility than take on the burden of dealing with a pile of data that only ge= +ts more crufty over time. In the 'schemaless' world (e.g., DynamoDB), dev= +elopers take a mix of painful approaches (ranging from adding version tag= +s that drive switches in application code, to doing manual migrations bet= +ween tables) to clean up old data when the previous representation cannot= + stretch far enough.=0D=0A=0D=0AI tried with Mentat to strike a balance h= +ere: to design a system that would allow multiple teams to interact, defi= +ning their vocabularies in an ever-growing way, with specific transformat= +ion steps as needed, and being able to detect and handle conflicting voca= +bulary usage within that closed world. You can absolutely use Mentat as a= + system with no vocabulary modification ever, complicating your queries t= +o accommodate, but it would not have met the needs of its audience:=0D=0A= +=0D=0AD: "So, I realized that having a page -> visit timestamp property d= +oesn't actually do what we want; we need to be able to mark each visit wi= +th which device visited the page. How do I change the schema and the exis= +ting data=3F"=0D=0AR: "You can't; make a new property. Every time you wan= +t to retrieve the visits for the page, everywhere in your app, your queri= +es need to look at both properties."=0D=0AD: "Can I write some function t= +hat seamlessly transforms the old data to the new format and works with s= +yncing=3F"=0D=0AR: "I wrote a doc about that, but we decided that purity = +was better. You can write code to manually retract the old stuff, but oth= +er syncing devices can still mess you up. Sorry."=0D=0A=0D=0AResolving co= +nflicts and supporting sync was a core part of Mentat's charter. The last= + thing I wanted was for developers to think the system was hard to use, o= +r to build a second storage layer on top to maintain a snapshot of the cu= +rrent data in an app-centric format.=0D=0A=0D=0A=0D=0A> You mentioned erg= +onomics thrice on your message, and I believe that your=0D=0A> insistence= + of it is a sign of the coupling of two different layers of=0D=0A> the da= +tabase, and makes ease of use drive the fundamentals.=0D=0A=0D=0AYes, exa= +ctly. The goal of Mentat wasn't to build a pure log-structured store: it = +was to build a store with a developer interface that wasn't too alien to = +an engineer who might store data in JSON or SQLite, but with internals th= +at made sharing data, syncing data, and evolving data easier and safer.=0D= +=0A=0D=0A=0D=0A> I also think=0D=0A> that your statement on conflict reso= +lution being a requirement is not a=0D=0A> good idea. Let me expand on th= +at.=0D=0A>=20=0D=0A> I'd argue that conflict resolution is inherit comple= +xity [1] of creating=0D=0A> a decentralized client-side application that = +can sync data.=0D=0A>=20=0D=0A> That means that the responsability to sol= +ve conflicts is not the=0D=0A> database's, but the application's responsa= +bility, and making decisions=0D=0A> of conflict resolution on the databas= +e limits its application, and I=0D=0A> wouldn't consider it to be general= + purpose.=0D=0A=0D=0AAh, here's an interesting point.=0D=0A=0D=0AThere ar= +e two positions here: am I vending a database, or am I vending a storage = +solution for a client app=3F=0D=0A=0D=0ABuilding Firefox, Firefox for And= +roid, and Firefox for iOS convinced me that implementing a synchronizatio= +n solution that is not tightly coupled with the storage layer =E2=80=94 t= +hat is, one that is just another consumer of the same storage interface t= +hat the rest of the app uses =E2=80=94 is difficult and error-prone. Sync= + needs to see writes as they happen, and synchronization has particular i= +mplications for how data is stored.=0D=0A=0D=0AI also found that offloadi= +ng the difficulties in syncing =E2=80=94 conflict resolution, tracking ch= +ange, merging identities, etc. =E2=80=94 to application developers leads = +to worse outcomes.=0D=0A=0D=0AMy feeling is that it should be possible fo= +r the application to educate the storage layer about how to handle most o= +f the conflicts it encounters, and to model its data in a way that makes = +conflicts meaningful and minimal, without sacrificing the kinds of data r= +epresentations with which developers feel comfortable.=0D=0A=0D=0A=0D=0A>= + When two offline database instances change the same field, there's no=0D= +=0A> escaping from the bottom reality, which is: there is a conflict, and= + it=0D=0A> needs to be resolved somewhere. There's no layer of ergonomics= + that can=0D=0A> change this fact, and picking a strategy for conflict re= +solution is=0D=0A> undoubtly domain-specific.=0D=0A=0D=0AThe approach I w= +as taking for syncing in Mentat involved rewriting (or permanently associ= +ating) identifiers based on uniqueness constraints. That is, as a develop= +er I could transact a schema fragment:=0D=0A=0D=0A=0D=0A {:history/url := +db/unique :db.unique/value}=0D=0A=0D=0A=0D=0Aand thereafter say things li= +ke=0D=0A=0D=0A=0D=0A {:history/url "https://foo.com/"=0D=0A :history/t= +itle "Today's title"}=0D=0A=0D=0A=0D=0AIf the log already contains this, = +with a local ID:=0D=0A=0D=0A=0D=0A [1234 :history/url "https://foo.com/"= +]=0D=0A=0D=0A=0D=0Athe new transact would add to the log something like:=0D= +=0A=0D=0A=0D=0A [1234 :history/title "Today's title"]=0D=0A=0D=0A=0D=0AA= +nother device can record similar data:=0D=0A=0D=0A=0D=0A {:history/url "= +https://foo.com/"=0D=0A :history/title "A different title"}=0D=0A=0D=0A= +=0D=0Awhich would transact=0D=0A=0D=0A=0D=0A [4567 :history/title "A dif= +ferent title"]=0D=0A=0D=0A=0D=0AA query on either platform can omit the I= +D, using a lookup ref instead:=0D=0A=0D=0A=0D=0A [[:history/url "https:/= +/foo.com/"] :history/title =3Ftitle]=0D=0A=0D=0A=0D=0AIn both cases we ge= +t a 1:1 association between a URL and a 'page'. The use of lookup refs an= +d the schema definition allows the developer to use the [attribute, URL] = +pair as an external identifier in this context, while using an efficient = +identifier internally.=0D=0A=0D=0AWhen the two transaction logs are merge= +d, it follows from the schema definition that B(4567) =3D=3D=3D A(1234), = +and that identity can be used in the store to zip the two graphs together= + in a way that is seamless to the developer: the 1:1 association is prese= +rved. In practice we found that holding on to those numeric IDs, which ca= +n change during syncing, in application code was rarely or never necessar= +y.=0D=0A=0D=0AThis example is what I mean by 'teaching' the database abou= +t domain definitions of identity. In a relational database, the schema is= + largely used to define structure and constraints. The way it feels to wr= +ite a schema in Datomic/Mentat is much closer to 'explaining' how various= + kinds of new assertions and sync behaviors will interact with the existi= +ng data =E2=80=94 the schema constraints are so often used to drive looku= +p refs and updates, and of course there's no tabular structure.=0D=0A=0D=0A= +This can get a merge all the way down to "this page's title changed from = +X to Y at time T on one device and from X to Z on another at time Y; whic= +h do you want=3F" which is _also_ something that you can teach the databa= +se to automatically resolve =E2=80=94 annotate the schema to say that a c= +onflict takes the time-latest value.=0D=0A=0D=0A=0D=0A> They may be the=0D= +=0A> same, and they may not be the same, but saying just "the README file= +" is=0D=0A> an incomplete depiction of it: you either say "my README file= +" or talk=0D=0A> about both of them, which do coexist.=0D=0A=0D=0AI actua= +lly used this as an example when explaining Mentat =E2=80=94 timelines ar= +e branches, but the `datoms` table (the data you actually work with) is a= + Git "working tree". Developers (and end users) need a working tree and a= + single state of the world.=0D=0A=0D=0A=0D=0A> The point is: the database= + is not in a position to make this type of=0D=0A> decision. Only the prog= +rammer is, and they must be aware of it.=0D=0A=0D=0AWhich kinds of progra= +mmers are your audience=3F=0D=0A=0D=0AIn my experience, most developers a= +re (a) not aware of the difficulties here, (b) do not have the time to di= +g in to the details, and (c) would rather take reasonable defaults (e.g.,= + latest wins) or even data duplication or corruption than sacrifice simpl= +icity to get it right. That is: if you presented them with the option of = +a distributed sync system where they had to handle multiple timelines, us= +e pluggable conflict-resolution modules, and write complicated queries to= + deal with that complexity, they are instead going to use JSON files.=0D=0A= +=0D=0AIt would not be inaccurate to say that Mentat is/was _already_ way = +too much of a shift for most developers; a tuple store with a log is expe= +nsive and confusing in ways that a key-value store is not.=0D=0A=0D=0A=0D= +=0A> You said that:=0D=0A>=20=0D=0A>> the goal of a database is to help m= +e efficiently, accurately, and=0D=0A>> ergonomically represent my domain=20= +=0D=0A>=20=0D=0A> For me, the crucial part is the accurately comes first,= + and ergonomics=0D=0A> comes second. No accuracy should be sacrificed in = +name of ergonomics,=0D=0A> since the inherent complexity is already too b= +ig to be dealt with.=0D=0A=0D=0ATo broaden your point a little bit, my po= +sition on that has shifted over the years, after working on systems that = +prioritized correctness over adoption or user experience (e.g., Firefox S= +ync's perfect crypto, which caused _so many users_ to lose data because t= +hey weren't prepared to manage long-lived secure keys, and couldn't under= +stand the setup flow=E2=80=A6).=0D=0A=0D=0AA system that is perfect but t= +hat people =E2=80=94 developers or end users =E2=80=94 don't use it is _w= +orse_ than a system that makes compromises and gets some adoption. As Guy= + Steele put it about Java:=20=0D=0A=0D=0A "We were not out to win over t= +he Lisp programmers; we were after the C++ programmers. We managed to dra= +g a lot of them about half-way to Lisp."=0D=0A=0D=0AI think a brief tange= +nt into ontology would be interesting. I know your point here is about ac= +curately modeling change, but a similar set of positions exists when mode= +ling information itself.=0D=0A=0D=0AThe 'correct' way to model the world = +is as a time-series sequence of independent observations, with everything= + else being materialized from it through a carefully designed ontological= + lens, such that one doesn't have to accidentally make false statements i= +n order to say something true.=0D=0A=0D=0AAt one end is the kind of ontol= +ogy I drew on a whiteboard once, nearly twenty years ago, in which the co= +ncept of a music "album" is broken down into issuances, releases, track o= +rders, renditions, recordings, performers, names, identities, collectives= +, etc. =E2=80=94 it's incredibly complex how one recording of a group of = +people in a room might be recorded multiple times, remastered over the ye= +ars, edited for various releases, and appear on multiple formats in diffe= +rent versions, and how different versions of songs might have different b= +ands or people playing different parts.=0D=0A=0D=0AMusic nerds really car= +e about this, though perhaps not down to the level of detail to which I m= +odeled it.=0D=0A=0D=0AAt the other end is how this kind of metadata is re= +ally represented in the real world.=0D=0A=0D=0ATitle: "Landslide"=0D=0AAr= +tist: "Fleetwood Mac"=0D=0APlay Count: 15=0D=0ATrack Number: 2=0D=0A=0D=0A= +The theoretically pure way to model all this stuff makes it possible to s= +ay "this recording of Landslide is of a performance by a band named Fleet= +wood Mac =E2=80=94 the particular instance of Fleetwood Mac that included= + Stevie Nicks, not the one that included Bob Welch". Let's not even get i= +nto the fact that there are two albums recorded by Fleetwood Mac titled "= +Fleetwood Mac" =E2=80=94 (title, artist) isn't compound-unique!=0D=0A=0D=0A= +But no user wants to say "Alexa, play Fleetwood Mac" and have her reply "= +do you mean the band 'Fleetwood Mac' that recorded songs released by Warn= +er Bros, or the band 'Fleetwood Mac' that recorded songs released by Blue= + Horizon"=3F They will exchange a certain amount of precision to cast a w= +ider and more usable net.=0D=0A=0D=0APut bluntly: apps have trouble getti= +ng the darn play counts on two devices to add up to the same number, let = +alone this degree of precision. I would rather make it _possible_ for the= +m to model data at a reasonable degree of accuracy, with no more difficul= +ty than they would have writing a CREATE TABLE statement, and help them g= +et their play counts right when they sync two devices, rather than force = +them to take an extreme position on accuracy and leave them to figure out= + how to list how many times the user played a song.=0D=0A=0D=0A=0D=0A> Th= +is way, how can one know the current state of the database=3F=20=0D=0A>=20= +=0D=0A> From the first root datom, the database would rollup every datom = +taking=0D=0A> the timelines and merges in consideration, and it would be = +the sum of=0D=0A> data from all timelines. When a conflict exists, the lo= +cal timeline is=0D=0A> always picked so the database can still serve writ= +es, and the conflict=0D=0A> would be presented when serving reads.=20=0D=0A= +>=20=0D=0A> The obvious problem of always favouring the local timeline is= + that each=0D=0A> timeline would show different "primary" answers, but th= +at's sounds=0D=0A> somewhat reasonable to me to enable the database to ke= +ep working while=0D=0A> offline, and while the conflict isn't resolved.=0D= +=0A=0D=0AMy view is that end users expect all of their devices to present= + the same canonical view of their data, and manual conflict resolution sh= +ould be rare/unnecessary/modeled as an explicit change or an explicit sta= +te (a write like any other).=0D=0A=0D=0AFurther, developers expect querie= +s to be fast and writes and queries to behave the same when devices are s= +ynced, regardless of which device performs the write or runs the query.=0D= +=0A=0D=0AThat makes it difficult to either diverge between devices for lo= +ng periods, or to expect routine manual involvement in conflicts.=0D=0A=0D= +=0AIt seems to me that the longer these distinct timelines exist, the mor= +e conflicts will breed, as writes reflect different local states, and so = +the difficulty of presenting a coherent view increases over time.=0D=0A=0D= +=0AI expected probably one place to do manual conflict resolution in Fire= +fox: password management. Even there it might be better to turn every con= +flict into an explicit "previous passwords" entry, or to expose the log a= +s a dynamic list of previous passwords =E2=80=94 that data is valuable fo= +r detecting website breaches. That is, it's better to surface the alterna= +tive data as a normal change and reach agreement with other devices, rath= +er than preserving some kind of pending conflict.=0D=0A=0D=0ABy modeling = +rebases or merges as a coherent set of retractions of conflicting data, i= +t's still possible to implement log-oriented features like querying *ever= +y* title a page has ever had, but eagerly reach a shared timeline. ("Show= + me that news article I remember reading".)=0D=0A=0D=0A=0D=0A> This also = +means that "sync" are two distinct things: exchanging timeline=0D=0A> dat= +a, and reconciling that data locally.=0D=0A=0D=0AAgreed.=0D=0A=0D=0A=0D=0A= +> The only part that's online is=0D=0A> the "exchange", and instances res= +olve conflict locally. This way, the=0D=0A> database is always available,= + even when offline, and even when conflict=0D=0A> is present.=0D=0A=0D=0A= +The difference IMO is that the goal of conflict resolution is that every = +device converges on the same canonical timeline, and does so as quickly a= +s possible.=0D=0A=0D=0A=0D=0A> But how would queries over history behave=3F= + No clue yet =F0=9F=98=AC.=0D=0A>=20=0D=0A> I tried not to sound too hand= + wavy, and I've only prototyped a small=0D=0A> part of this in the past, = +so there are probably many more problems with=0D=0A> this approach. I don= +'t claim this is a sound solution yet, but it does=0D=0A> handle some of = +the problems that Mentat faced in a different way. I'd be=0D=0A> interest= +ed to see you poking holes on it.=0D=0A=0D=0AIt's a really difficult prob= +lem! My suspicion is that querying history is either broadly a union ("th= +ese pages all had this title at some point") or very much an expert featu= +re, analogous to browsing your own 'undo' tree. But note that the union a= +pproach can benefit from identifier rewriting to represent derived equali= +ty at a particular point in time, even in the presence of conflicts.=0D=0A= +=0D=0A=0D=0A> When combined with strict schema growth, new data always co= +mes after=0D=0A> it's schema declaration, so the application doesn't have= + to coordinate=0D=0A> around distributed schema evolution.=0D=0A=0D=0AYes= +, that's broadly the position I took with Mentat: by definition, data on = +each timeline comes after its schema declaration, so we have an ordered m= +echanism for merging and also for migrating data as the schema evolves.=0D= +=0A=0D=0A=0D=0A> TIL: I didn't know you worked on AllegroGraph.=0D=0A=0D=0A= +Seems like every few years I end up back on data modeling and graph stora= +ge :D=0D=0A=0D=0A=0D=0A> If the database was versatile enough, an ideal w= +orld the programmer could write:=0D=0A>=20=0D=0A> ;; schema definition=0D= +=0A> [{:db/ident :profile/name=0D=0A> :db/valueType :db.type/string=0D=0A= +> :db/cardinality :db.cardinality/one=0D=0A> :db/conflict-strategy :db.= +conflict-strategy/none}=0D=0A=0D=0AYes, that's pretty much where we were = +headed with Mentat. My key observations were that (a) you need really goo= +d tree-zipping, and frequent syncs, to minimize conflicts; (b) tree-zippi= +ng and merges practically require that internal identifiers be rewritten = +when identity is determined; (c) it is undesirable to stop the world, and= + so when a conflict is detected you end up with continuing divergence whi= +ch will be even harder to merge later; (d) because this is a single graph= +, a conflict _must_ be resolved in some way before you can merge two head= +s; (e) lossy merges (squash then rebase or squash then merge) can help.=0D= +=0A=0D=0A + +From eu@euandre.org Sat Sep 5 22:51:10 2020 +Received: from box.euandre.org (box.euandre.org [46.101.160.115]) + by mail-b.sr.ht (Postfix) with ESMTPS id D9930FF12D + for <~euandreh/public-inbox@lists.sr.ht>; Sat, 5 Sep 2020 22:51:09 +0000 (UTC) +Authentication-Results: mail-b.sr.ht; + dkim=pass (2048-bit key) header.d=euandre.org header.i=@euandre.org header.b=w2kZ+s6p +Received: from authenticated-user (box.euandre.org [46.101.160.115]) + (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) + (No client certificate requested) + by box.euandre.org (Postfix) with ESMTPSA id 2130DFC822; + Sat, 5 Sep 2020 19:51:06 -0300 (-03) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=euandre.org; s=mail; + t=1599346268; bh=PWBfaRn3jOle7Z3Lc1INXcX50+v5RZ5HNwANJeQm8Js=; + h=From:To:Cc:Cc:Subject:In-Reply-To:References:Date:From; + b=w2kZ+s6pUTUZJSwLwi3NmcFMqy3a8mBj7PuNX5oJD9O3R4MZM4Td6AZVsjHQQLRVG + IRQHyT9t8jngQXu0mkRtU2F4IZoKEPMQAVrRBOBq+f4+J1OR5zynUcT7UKE+hmWDtr + lJqDHIbUEzNoS8yLpWXZqib6kwHFqF8B/TLtbl4scllrpf+Q0eq7ZbDpjYHfotNt7C + xZ8bFSDeeSP6Ec93nh82bOpXqAEhaTA/vf7qh7T1qWKdjmFvDV3WldUtxjmMcCDvgS + PYZ5Laxlrvoq8li4XF55rArXMOb8gftJMy3QY0rPRmyvpFLKFFB423i+NnIctnu93o + ewcl54UnP6Iaw== +From: EuAndreh <eu@euandre.org> +To: Richard Newman <rnewman@twinql.com> +Cc: "~euandreh/public-inbox@lists.sr.ht" <~euandreh/public-inbox@lists.sr.ht> +Cc: +Subject: Re: The database I wish I had +In-Reply-To: <010101745fdecd5f-52ab4d88-26f1-4a6d-879d-3234c1956260-000000@us-west-2.amazonses.com> +References: <7454DA45-C5A4-4A54-8D02-1F3BD37726E9@twinql.com> + <010101744a592b75-1dce9281-f0b8-4226-9d50-fd2c7901fa72-000000@us-west-2.amazonses.com> + <87sgc1xu7k.fsf@euandre.org> + <9652AAE0-10DF-4D13-ABE8-B30667BE27DD@twinql.com> + <010101744b260d62-1bb508ab-647f-4877-bdf9-f988667d56e1-000000@us-west-2.amazonses.com> + <878sdsy6sc.fsf@euandre.org> + <7027FD8B-1B30-435F-A840-7DC61A722E0B@twinql.com> + <010101744f99448f-fa8dd88e-0daf-41f8-b3ca-6ea551c300e4-000000@us-west-2.amazonses.com> + <87h7se5t1w.fsf@euandre.org> + <B032A2E0-0679-4FF6-9A80-06450A8F36B1@twinql.com> + <010101745fdecd5f-52ab4d88-26f1-4a6d-879d-3234c1956260-000000@us-west-2.amazonses.com> +Date: Sat, 05 Sep 2020 19:50:56 -0300 +Message-ID: <87r1rfsur3.fsf@euandre.org> +MIME-Version: 1.0 +Content-Type: text/plain; charset=utf-8 +Content-Transfer-Encoding: quoted-printable + +I see what to mean by being accessible, and it does make sense. + +If done properly this might be the reality, where nobody chooses to +handle conflicts manually because they'd rather have a consistent view +by picking one of the possible algorithms that are available for them to +pick for each attribute. I'll experiment with this a bit, and if=20 +feasible, the "perpetual inconsistency" might be an internal view, and +something available for those who choose to customize their conflict +resolution strategy. This would be ideal, and perfectly aligned with +what you said on "the application educating the storage layer about how +to handle most of the conflicts it encouters". The unique URL you showed +is a tempting challenge to reach, and the domain identity being +teachable sounds indeed really powerful. + +This also means that with those defaults the desired 1:1 lookups would +be restored. I don't think that doing manual resolution for each +attribute is desired, and I wouldn't impose this on the programmer. In +fact, I'd want to use it myself most of the time.=20 + +That would also mean an intersection between the two conflicting forces +you mentioned, where both sides would get what they wanted: a developer +could use it to quickly prototype something by picking the suggested +defaults and not finding many surprises, while allowing another +developer to fine tune their strategy when desired. + +It's a database, with a robust storage solution inside it, but primarily +a database. + +If not possible, I tend to err on the side of less ease. Not because I +want to be a purist, but because I want to do this for myself =F0=9F=98=AC.= + The +title of the article was actually honest on this, and I've actually had +this exact though a few times: "I'm struggling with this database, but +there must be more people like me with the same struggles. I bet someone +has already solved, or will solve soon enough". I'm the target audience, +and if I do something that solves my problem, than I would consider it a +success :). It may not be widespread but I'd feel better knowing that +I've built something that works over something that works most of the +time. Again, I don't want to be a purist, I just want to create a +database that I'd want for myself, and whoever sees themselves on the +same position could benefit. If ease can be added on top of it with some +choices made for you, great! From your comment, I think you'd agree with +that last part.=20=20=20=20=20 + +I'm not trying to convert anyone on this, though. I'd be happy to share +the code and learning, and welcome those who are interested and +attracted to this, but broad adoption and world domination are not +goals. + +The applications that I tried to write using the existing solutions were +just good enough to get me started, but I always felt abandoned when I +wanted to evolve it somehow, or add some specific functionality without +losing data, or some similar trade-off. Most of those times I'd rather +deal with that complexity than choose an easier path. + +Today whenever I see a client-side database that syncs, I go to the +documentation and search for "conflict". I'll either get a page with +some in-depth explanation, or there will be a list of choices they made +for me with no way out of it, like last write win, and permanent +deletion tombstones. Sure those are valuable, but I'd rather be able to +choose them myself. I already know by now that I will need some tweaked +conflict resolution someday, and they won't be able to provide it for +me without an ad-hoc approach.=20 + +If someone would value something more familiar and more +fragile/opinionated over something very different and more sound, then +maybe they're correct to pick SQLite or some JSON files, and having data +loss in some edge cases would be just fine in the problem they're +solving. I'd encourage them to stick to SQLite, even, its a great piece +of software.=20 + +Also, there's something more profound on this, I believe:=20 + +Having a solid foundation could potentially help people reason about the +problem with good tools, instead of looking like a solid foundation that +will crack if you step on the wrong place. Instead of helping, it could +be hurting the application, and the superficial ease that many solutions +offer sometimes feel more like a trap for me to figure out myself later. +To me it's like someone saying: "have this relational fully ACID +performant database!", and only later I realize that it isn't actually +ACID by default, and if all change all the knobs to make it ACID, it +becomes unusuably slow (or maybe even those knobs aren't there at all).=20 + +I'd argue that such approach may look weirder and more foreign to any +developer, but it would actually be "easier". This reminds me of +"Simple Made Easy" [0] + +[0]: https://www.infoq.com/presentations/Simple-Made-Easy/ + +Back on schema growth: one doesn't have to make a perfect plan of the +data models so no mental tax gets to them later. + +The solution you mentioned sounds just fine: write a function that +transforms the old data and feed it to the new attribute, and retract +everything else. From that point on, you've effectively migrated the +data, and you'll only need to consider the historical schema when doing +historical queries. Nothing broke on the process, and people might just +forget about how it used to be in the past. No mental tax, unless you +want to dig history. + +The existence of the new attribute serves the same purpose that the +version tag does. + +You still have an immutable log, and you've evolved your schema to +better represent the domain. I don't see how those conflict here. The +biggest objection would probably be the "wasted storage" of the old +data.=20 + +But changing schema types and versioning them is something else. It +might even be possible to implement, doing rollups of the schema just +like what's done with the rest of the data like you mentioned, but I +just don't like this idea overall, I think it has brittleness written +all over it. Maybe I'm just being grumpy.=20=20=20 + +About your album ontology: I like it! Coincidence or not, I did 5 years +of music major before dropping out of college, so I'd consider myself a +"music nerd"! But I agree that one needs the database to help them model +the domain in a useful way, and I wouldn't force everybody to have a +particular view of it. Ultimately the database should be a useful tool +to solve a problem.=20 + +From andi@notmuch.email Tue Oct 6 07:53:02 2020 +Received: from mx.h4ck.space (mx.h4ck.space [159.69.146.50]) + by mail-b.sr.ht (Postfix) with ESMTPS id 57F5AFF0FF + for <~euandreh/public-inbox@lists.sr.ht>; Tue, 6 Oct 2020 07:53:02 +0000 (UTC) +Authentication-Results: mail-b.sr.ht; + dkim=pass (1024-bit key) header.d=notmuch.email header.i=@notmuch.email header.b=iFvGMOSq +Date: Tue, 6 Oct 2020 09:52:58 +0200 +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=notmuch.email; + s=mail; t=1601970781; + bh=RfiC8xpAHKtbuatmn10k3tqQKuSGYPCep+XBVi1qDe0=; + h=Date:From:To:Subject; + b=iFvGMOSqx3JpTHGUue5rMzrKPzZiiAFpjJBgA/1P8lgnk/7gBNl51mU5M7m42XXS2 + cGLwhRQAF3y/x9IoyNGAAUy/hRg+u1VGeRPoV+wFWlitZNgH6dWMuos8XNxLv7Z+dr + X1MbaF5gSIQaRehQyg7oyDhd9TO2WcKj6jRCia60= +From: andi@notmuch.email +To: ~euandreh/public-inbox@lists.sr.ht +Subject: crate2nix +Message-ID: <20201006075258.52tz5u5a36fpbfwy@wrt> +MIME-Version: 1.0 +Content-Type: text/plain; charset=utf-8 +Content-Disposition: inline + +Hi, + +Just saw your crate2nix code and it looks very nice and clean! + +Just a heads-up, since you might not aware of all the alternatives out +there, there is a version from Eelco that basically does the same as +yours but also handles git dependencies. I did blog about that briefly +last month [1]. + +I am curious if you were aware of that and/or naersk [2] as both of them +also come with the promise of no generated files and no *2nix commands +to execute. + + +Regards, + +Andi + + + +[1]: https://andreas.rammhold.de/posts/nix-rust/#import-cargohttpsgithubcomedolstraimport-cargo +[2]: https://github.com/nmattia/naersk + +From eu@euandre.org Tue Oct 6 09:09:49 2020 +Received: from box.euandre.org (box.euandre.org [46.101.160.115]) + by mail-b.sr.ht (Postfix) with ESMTPS id 9ED40FF107 + for <~euandreh/public-inbox@lists.sr.ht>; Tue, 6 Oct 2020 09:09:48 +0000 (UTC) +Authentication-Results: mail-b.sr.ht; + dkim=pass (2048-bit key) header.d=euandre.org header.i=@euandre.org header.b=sXxvO0HC +Received: from authenticated-user (box.euandre.org [46.101.160.115]) + (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) + (No client certificate requested) + by box.euandre.org (Postfix) with ESMTPSA id DF0A2FC991; + Tue, 6 Oct 2020 06:09:45 -0300 (-03) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=euandre.org; s=mail; + t=1601975387; bh=R37p/WxSUhw6s3+NsQXWU1P6/NnfwRrO7khrmNnifPs=; + h=From:To:Cc:Subject:In-Reply-To:References:Date:From; + b=sXxvO0HCnYXM8ovt5uOk5bZ/3ZTzOR9HVWXgURy/MicF6fwxTJJfOMCC4+b/ZanpX + hcQc+ZSsiOgevD6oR8MZE630bHDpXNs/FskZFM0gRAwG68dfJ/lpj2ogo5GQ9fEbKy + oH0MD7Yd2suVGevV9oULcadlt5g8Y75Yuazzwx0HOoF12A4BQWtRrBsBV/Qp7yNoZm + xjNIf9MwbD2MsEWlF/K9JeDEdsHazcpQh2RiVd1/G497B9F4yEbE5HmkOP5aGP5jUv + yudSn0cBCWQ7S/FXeHXUyhZeebiQnFWkpJeIfUSEhGSOoim1GE/gMi1U7rA1qsWaoy + jVVWomlh//r5w== +From: EuAndreh <eu@euandre.org> +To: andi@notmuch.email, ~euandreh/public-inbox@lists.sr.ht +Cc: +Subject: Re: crate2nix +In-Reply-To: <20201006075258.52tz5u5a36fpbfwy@wrt> +References: <20201006075258.52tz5u5a36fpbfwy@wrt> +Date: Tue, 06 Oct 2020 06:09:27 -0300 +Message-ID: <87r1qbafg8.fsf@euandre.org> +MIME-Version: 1.0 +Content-Type: text/plain; charset=utf-8 +Content-Transfer-Encoding: quoted-printable + +andi@notmuch.email writes: + +> Hi, + +Hi there! + +> Just saw your crate2nix code and it looks very nice and clean! + +=E2=98=BA=EF=B8=8F + +> Just a heads-up, since you might not aware of all the alternatives out +> there, there is a version from Eelco that basically does the same as +> yours but also handles git dependencies. I did blog about that briefly +> last month [1]. + +I wasn't aware of it. At a first glance, it looks remarkably similar! +Maybe I'm on to something =F0=9F=98=AC. I'll try it later, thanks for the l= +inks. + +> I am curious if you were aware of that and/or naersk [2] as both of them +> also come with the promise of no generated files and no *2nix commands +> to execute. + +I did try naersk. It was the best one I found, in fact. It does use +Cargo under the hood to drive things, but it doesn't expose the +environment it builds. With it you can parameterize the Cargo commands, +but not run it by yourself. + +In fact, there's a snippet that I copied directly from naersk[0]. I +didn't mention it on the README, only on the commit log. + +[0]: https://git.euandreh.xyz/cargo2nix/commit/?id=3Dc75e7e2d3ee7f7e074ea9f= +8e5b959d6395338acc + +From m@kluv.in Tue Dec 22 16:29:11 2020 +Authentication-Results: mail-b.sr.ht; dkim=none +Received: from se15-1.privateemail.com (se15-1.privateemail.com [198.54.127.72]) + by mail-b.sr.ht (Postfix) with ESMTPS id B579CFF0D7 + for <~euandreh/public-inbox@lists.sr.ht>; Tue, 22 Dec 2020 16:29:11 +0000 (UTC) +Received: from new-01.privateemail.com ([68.65.122.22]) + by se15.registrar-servers.com with esmtpsa (TLSv1.2:AES128-GCM-SHA256:128) + (Exim 4.92) + (envelope-from <m@kluv.in>) + id 1krkX5-0004vZ-Jo + for ~euandreh/public-inbox@lists.sr.ht; Tue, 22 Dec 2020 08:29:10 -0800 +Received: from MTA-08-1.privateemail.com (unknown [10.20.147.18]) + (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) + (No client certificate requested) + by NEW-01.privateemail.com (Postfix) with ESMTPS id B5E4C60041 + for <~euandreh/public-inbox@lists.sr.ht>; Tue, 22 Dec 2020 16:29:05 +0000 (UTC) +Received: from MTA-08.privateemail.com (localhost [127.0.0.1]) + by MTA-08.privateemail.com (Postfix) with ESMTP id 91A866006F + for <~euandreh/public-inbox@lists.sr.ht>; Tue, 22 Dec 2020 11:29:05 -0500 (EST) +Received: from DESKTOP1997JCK (unknown [10.20.151.208]) + by MTA-08.privateemail.com (Postfix) with ESMTPA id F339F6004A + for <~euandreh/public-inbox@lists.sr.ht>; Tue, 22 Dec 2020 16:29:04 +0000 (UTC) +From: <m@kluv.in> +To: <~euandreh/public-inbox@lists.sr.ht> +Subject: Progress on 'The database I wish I had' +Date: Tue, 22 Dec 2020 17:29:03 +0100 +Message-ID: <000001d6d87f$90db0f90$b2912eb0$@kluv.in> +MIME-Version: 1.0 +Content-Type: text/plain; + charset="us-ascii" +Content-Transfer-Encoding: 7bit +X-Mailer: Microsoft Outlook 16.0 +Thread-Index: AdbYfzkizJpIDb+oQYeLozSx36pHnw== +Content-Language: no +X-Virus-Scanned: ClamAV using ClamSMTP +X-Originating-IP: 68.65.122.22 +X-SpamExperts-Domain: o1.privateemail.com +X-SpamExperts-Username: out-01 +Authentication-Results: registrar-servers.com; auth=pass (plain) smtp.auth=out-01@o1.privateemail.com +X-SpamExperts-Outgoing-Class: unsure +X-SpamExperts-Outgoing-Evidence: Combined (0.57) +X-Recommended-Action: accept +X-Filter-ID: Pt3MvcO5N4iKaDQ5O6lkdGlMVN6RH8bjRMzItlySaT8/DiuTODf6wrobfALkT2HqPUtbdvnXkggZ + 3YnVId/Y5jcf0yeVQAvfjHznO7+bT5wWykU6VVuoVZYG6cNLn3c4E9P7Wdo8qPSPEXiwPI9MYyaH + o/9wB5YLuaEkARV33qkqPXaGse7C78HuChWU138c4pWx7WxWFZpsjT/483ewgsz4HwYGTNBY7kFu + mOeXVIneVHayp+oaTkTmqro7//B+gpZK9MmGPsPG1IFhoAOccOwmVIoqJOmt2f1p/FeBBYf0It0W + sNcsjnTm+4g485SS8OTf6I2qJgqFlok7aJfKCMNBaM6YjzsuHtjxewJtlZX1S4orbC27wIbqb1J8 + wyDQ43DG7JDHGkUFlLvaUAJ9EsZg/9mPb0OnPglkYW72F/Odw+XGlIW1bb6iLQaqIs5BLfTttFI5 + MCNL/izpcNORuAUvossjam0/HVDFzCeLVAjI+ht+2XwDC3Hj+WjRz7duYp7tG8ix8mThYIoihNfI + Pg3O+76Hxe6qkGjOqN7KXlq8vamDuW8qjcPFOC9xtigG+MEYMo0NDeyXVO3tNSgVgckOnz0mswVX + ZkpFVe5ZgGA0MLjqgL337Aba0izaNK+kAlchHs+Xxee6P8LI+rvlGOeqKR5Od1/Dbcw8eAyTq1JN + 7i5WVjtE146jFUwt8Q10AQGCef4uNE610hEkYogGmsvnLjiRqfPZZiHIniXn+ZdiHE9X9GlwwRvN + 2KIObvd055RG8/vjrz5/4Qa1aRJkvZZIZN1+KRx7WOWIKK5eEM2Qqv1jJWtJXZNrnBE/zutvXj2E + w+ZVKNlbNM488ODolw/r6z7OkCVgacEoYdDR+UFRXxKF5tPxTxfD0dMN+t5ZjqKOEsXEdN1Fhcn9 + wF7BNjheMPxghFsjIwINNqkbXC/wv1jB3a9oGPa4BHVKDIPK97/T4LRRVYxF+VXiiOfHJN40eTXl + WiUAYdLmsJdAoPKCpWwKtkkGG+bEnfOEkWTNI3SjTCvjMfNBc9ze9o81pXKSQ+GI7QB7PH97h6/L + 6Wa2uSQw0nxsyGH4ASfmmhbNFFwa8AyQYqjO7qYtiXb+9Q== +X-Report-Abuse-To: spam@se16.registrar-servers.com + +Hi EuAndreh. + +I am curious if you found any existing solution for the proposed database? I +have been looking at irmin (which was mentioned in all forums) a lot lately, +largely as substitute to libgit2, but recently also as a database. + +It looks like the conversation have tapered off, or moved elsewhere? In any +case it was interesting stumbling across your post, seeing as I have been +having very similar ideas. + +Regards, +Martin + + + +From eu@euandre.org Tue Dec 22 21:37:15 2020 +Authentication-Results: mail-b.sr.ht; dkim=fail header.d=euandre.org header.i=@euandre.org +Received: from box.euandre.org (box.euandre.org [46.101.160.115]) + by mail-b.sr.ht (Postfix) with ESMTPS id 40985FF0DB + for <~euandreh/public-inbox@lists.sr.ht>; Tue, 22 Dec 2020 21:37:15 +0000 (UTC) +Received: from authenticated-user (box.euandre.org [46.101.160.115]) + (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) + (No client certificate requested) + by box.euandre.org (Postfix) with ESMTPSA id 4B3B9FC4BF; + Tue, 22 Dec 2020 18:37:11 -0300 (-03) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=euandre.org; s=mail; + t=1608673033; bh=9xQtiA8BJBLUgJEdjCq1SNS1YmrPaYDMrBvQ2oYoGpY=; + h=From:To:Cc:Subject:In-Reply-To:References:Date:From; + b=P++6DaNXwhc5G/perUnlDlWN9zHzoyzmASEDPcZ0nqbNUpKJy9sV5OmC7etQTyPMO + bAAwkUw3Wl/tn9CaLxzYJo+IClWYQEVJDmAMwlcCIJa3Ol+fSQsceBxK1a5za5EA8W + SzBrXwvpbYG9ISVwBUY+fNkqwAoRP2lQ8Sy3/pvrA0Kq894eXg08tbvlTJh3H6rh3Z + mla6VfP5qftYM6oWWhGSs6EJkGtmMDA+1UkAn+ASnSROejuxcPm1Cp2qmT2VoEPapP + kmHV3bjCM9v4nN/1bPy1rBvhwhx1O9UfNWvUYGWnhb24Tu0zK68FA20ZXlQc1bRnGA + yBrW5x0bzEZxg== +From: EuAndreh <eu@euandre.org> +To: m@kluv.in, ~euandreh/public-inbox@lists.sr.ht +Cc: +Subject: Re: Progress on 'The database I wish I had' +In-Reply-To: <000001d6d87f$90db0f90$b2912eb0$@kluv.in> +References: <000001d6d87f$90db0f90$b2912eb0$@kluv.in> +Date: Tue, 22 Dec 2020 18:36:56 -0300 +Message-ID: <87h7oda5fr.fsf@euandre.org> +MIME-Version: 1.0 +Content-Type: text/plain + +<m@kluv.in> writes: + +> Hi EuAndreh. + +Hi there! + +> I am curious if you found any existing solution for the proposed database? I +> have been looking at irmin (which was mentioned in all forums) a lot lately, +> largely as substitute to libgit2, but recently also as a database. + +I have written it down as a place to steal ideas from, which I plan to +do in the future as I mature more on the field, but irmin isn't a +solution, being written in OCaml. + +> It looks like the conversation have tapered off, or moved elsewhere? In any +> case it was interesting stumbling across your post, seeing as I have been +> having very similar ideas. + +I'm working on it, slowly but surely. + +The previous "comment" link used to point to my email address first, and +the this public-inbox second. Because of that I had many conversations +done in private messages, where I imagine that people just got the first +link. I responded those in private, but many had similar enough +comments, and I wrote an article on that [0]. FYI: I recently added a +"articles by category" Atom feed, if you prefer a filtered list [1] of +articles. + +On the ParsecC project mentioned on the article: +I'm already much more comfortable with creating a C API, but I got to a +point that I needed a proper error handling solution. I stumbled upon an +existing implementation of a parser combinator in C [2]. + +The point I'm at right now is to read and understand that source code. +In fact, it is 1 of 4 open tabs on my browser right now, before writing +this message. + +If I find its error handling good enough, I'll replicate it in ParsecC, +and then in libedn. + +I've also started reading a good book on the subject [3]. I can't speak +yet to all of it, but I like what I've seen so far. + +I happy to discuss more about it, or answer questions you may have. And +also, I'm happy to accept contributions! + +[0]: https://euandre.org/2020/11/12/durable-persistent-trees-and-parser-combinators-building-a-database.html +[1]: https://euandre.org/articles-by-category.html#mediator +[2]: https://github.com/wbhart/Cesium3/tree/combinators +[3]: https://www.immutablearchitecture.com/ + + diff --git a/src/comments.txt b/src/comments.txt new file mode 100644 index 0000000..99da91a --- /dev/null +++ b/src/comments.txt @@ -0,0 +1,17 @@ +## Threads whose subject names no page, and where they belong. +## +## The subject is the join between a list thread and a page: the +## Comment link puts the page's title in it. That breaks when +## somebody starts a new thread instead of replying, or writes the +## name of the thing down wrong. Both happened here. +## +## <THREAD-ROOT-MESSAGE-ID><TAB><the page's path> + +## Three follow-ups to "The database I wish I had", each begun as a +## fresh thread rather than a reply. +<d78064d2-5441-4275-96d9-463a08dfd7a9@www.fastmail.com> en/blog/2020/08/31/database-i-wish-i-had.html +<000001d6d87f$90db0f90$b2912eb0$@kluv.in> en/blog/2020/08/31/database-i-wish-i-had.html +<99C92E06-E0BC-4DB0-BF63-20FD44DC4062@rhyason.org> en/blog/2020/08/31/database-i-wish-i-had.html + +## "Just saw your crate2nix code" --- the tool is cargo2nix. +<20201006075258.52tz5u5a36fpbfwy@wrt> en/blog/2020/10/05/cargo2nix.html diff --git a/src/config.json b/src/config.json new file mode 100644 index 0000000..831d18f --- /dev/null +++ b/src/config.json @@ -0,0 +1,12 @@ +{ + "author": "EuAndreh", + "comments_json": "src/comments.mastodon.json src/comments.lemmy.json", + "discussions_url_prefix": "https://lists.sr.ht/~euandreh/public-inbox?search=", + "email": "eu@euandre.org", + "feed_alternate_url": "./", + "feed_title": "EuAndreh's blog", + "list_addr": "~euandreh/public-inbox@lists.sr.ht", + "me_urls": "https://hachyderm.io/@euandreh", + "site_name": "EuAndreh's website", + "sourcecode_url_prefix": "https://euandre.org/git/euandre.org/tree" +} diff --git a/src/content/.well-known/security.txt b/src/content/.well-known/security.txt deleted file mode 100644 index f588b54..0000000 --- a/src/content/.well-known/security.txt +++ /dev/null @@ -1,4 +0,0 @@ -Contact: mailto:eu@euandre.org -Expires: 2022-07-12T03:00:00.000Z -Encryption: https://euandre.org/public.asc -Preferred-Languages: en, pt, fr, eo, es diff --git a/src/content/b b/src/content/b new file mode 120000 index 0000000..dc1dc0c --- /dev/null +++ b/src/content/b @@ -0,0 +1 @@ +/dev/null
\ No newline at end of file diff --git a/src/content/en/about.adoc b/src/content/en/about.adoc new file mode 100644 index 0000000..14d5e22 --- /dev/null +++ b/src/content/en/about.adoc @@ -0,0 +1,11 @@ += About + +:mailto: mailto:~euandreh/public-inbox@lists.sr.ht +:archive: https://lists.sr.ht/~euandreh/public-inbox + +Hi, I'm EuAndreh. I write software and, occasionally, music. You can find my +contact information in the footer of this page, or mail my {mailto}[public +inbox] ({archive}[archive]). + +This is my personal website where I write articles, publish software and more +related work. diff --git a/src/content/en/blog/2018/07/17/guix-nixos.adoc b/src/content/en/blog/2018/07/17/guix-nixos.adoc new file mode 100644 index 0000000..42290f6 --- /dev/null +++ b/src/content/en/blog/2018/07/17/guix-nixos.adoc @@ -0,0 +1,197 @@ += Running Guix on NixOS + +:install-step: https://www.gnu.org/software/guix/manual/en/html_node/Binary-Installation.html#Binary-Installation + +I wanted to run Guix on a NixOS machine. Even though the Guix manual explains +how to do it {install-step}[step by step], I needed a few extra ones to make it +work properly. + +I couldn't just install GuixSD because my wireless network card doesn't have any +free drivers (yet). + +== Creating `guixbuilder` users + +:manual: https://www.gnu.org/software/guix/manual/en/html_node/Build-Environment-Setup.html#Build-Environment-Setup + +Guix requires you to create non-root users that will be used to perform the +builds in the isolated environments. + +The {manual}[manual] already provides you with a ready to run (as root) command +for creating the build users: + +[source,sh] +---- +groupadd --system guixbuild +for i in `seq -w 1 10`; +do + useradd -g guixbuild -G guixbuild \ + -d /var/empty -s `which nologin` \ + -c "Guix build user $i" --system \ + guixbuilder$i; +done +---- + +:mutable-users: https://nixos.org/nixos/manual/index.html#sec-user-management + +However, In my personal NixOS I have disabled +{mutable-users}[`users.mutableUsers`], which means that even if I run the above +command it means that they'll be removed once I rebuild my OS: + +[source,sh] +---- +$ sudo nixos-rebuild switch +(...) +removing user ‘guixbuilder7’ +removing user ‘guixbuilder3’ +removing user ‘guixbuilder10’ +removing user ‘guixbuilder1’ +removing user ‘guixbuilder6’ +removing user ‘guixbuilder9’ +removing user ‘guixbuilder4’ +removing user ‘guixbuilder2’ +removing user ‘guixbuilder8’ +removing user ‘guixbuilder5’ +(...) +---- + +Instead of enabling `users.mutableUsers` I could add the Guix users by adding +them to my system configuration: + +[source,nix] +---- +{ config, pkgs, ...}: + +{ + + # ... NixOS usual config ellided ... + + users = { + mutableUsers = false; + + extraUsers = + let + andrehUser = { + andreh = { + # my custom user config + }; + }; + buildUser = (i: + { + "guixbuilder${i}" = { # guixbuilder$i + group = "guixbuild"; # -g guixbuild + extraGroups = ["guixbuild"]; # -G guixbuild + home = "/var/empty"; # -d /var/empty + shell = pkgs.nologin; # -s `which nologin` + description = "Guix build user ${i}"; # -c "Guix buid user $i" + isSystemUser = true; # --system + }; + } + ); + in + # merge all users + pkgs.lib.fold (str: acc: acc // buildUser str) + andrehUser + # for i in `seq -w 1 10` + (map (pkgs.lib.fixedWidthNumber 2) (builtins.genList (n: n+1) 10)); + + extraGroups.guixbuild = { + name = "guixbuild"; + }; + }; +} +---- + +Here I used `fold` and the `//` operator to merge all of the configuration sets +into a single `extraUsers` value. + +== Creating the `systemd` service + +:service-file: https://git.savannah.gnu.org/cgit/guix.git/tree/etc/guix-daemon.service.in?id=00c86a888488b16ce30634d3a3a9d871ed6734a2 + +One other thing missing was the `systemd` service. + +First I couldn't just copy the `.service` file to `/etc` since in NixOS that +folder isn't writable. But also I wanted the service to be better integrated +with the OS. + +That was a little easier than creating the users, all I had to do was translate +the provided {service-file}[`guix-daemon.service.in`] configuration to an +equivalent Nix expression: + +[source,ini] +---- +# This is a "service unit file" for the systemd init system to launch +# 'guix-daemon'. Drop it in /etc/systemd/system or similar to have +# 'guix-daemon' automatically started. + +[Unit] +Description=Build daemon for GNU Guix + +[Service] +ExecStart=/var/guix/profiles/per-user/root/guix-profile/bin/guix-daemon --build-users-group=guixbuild +Environment=GUIX_LOCPATH=/root/.guix-profile/lib/locale +RemainAfterExit=yes +StandardOutput=syslog +StandardError=syslog + +# See <https://lists.gnu.org/archive/html/guix-devel/2016-04/msg00608.html>. +# Some package builds (for example, go@1.8.1) may require even more than +# 1024 tasks. +TasksMax=8192 + +[Install] +WantedBy=multi-user.target +---- + +This sample `systemd` configuration file became: + +[source,nix] +---- +guix-daemon = { + enable = true; + description = "Build daemon for GNU Guix"; + serviceConfig = { + ExecStart = "/var/guix/profiles/per-user/root/guix-profile/bin/guix-daemon --build-users-group=guixbuild"; + Environment="GUIX_LOCPATH=/root/.guix-profile/lib/locale"; + RemainAfterExit="yes"; + StandardOutput="syslog"; + StandardError="syslog"; + TaskMax= "8192"; + }; + wantedBy = [ "multi-user.target" ]; +}; +---- + +There you go! After running `sudo nixos-rebuild switch` I could get Guix up and +running: + +[source,sh] +---- +$ guix package -i hello +The following package will be installed: + hello 2.10 /gnu/store/bihfrh609gkxb9dp7n96wlpigiv3krfy-hello-2.10 + +substitute: updating substitutes from 'https://mirror.hydra.gnu.org'... 100.0% +The following derivations will be built: + /gnu/store/nznmdn6inpwxnlkrasydmda4s2vsp9hg-profile.drv + /gnu/store/vibqrvw4c8lacxjrkqyzqsdrmckv77kq-fonts-dir.drv + /gnu/store/hi8alg7wi0wgfdi3rn8cpp37zhx8ykf3-info-dir.drv + /gnu/store/cvkbp378cvfjikz7mjymhrimv7j12p0i-ca-certificate-bundle.drv + /gnu/store/d62fvxymnp95rzahhmhf456bsf0xg1c6-manual-database.drv +Creating manual page database... +1 entries processed in 0.0 s +2 packages in profile +$ hello +Hello, world! +---- + +:nixos-modules: https://nixos.org/nixos/manual/index.html#sec-writing-modules +:req: https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html#Requirements + +Some improvements to this approach are: + +. looking into {nixos-modules}[NixOS modules] and trying to bundle everything + together into a single logical unit; +. {req}[build Guix from source] and share the Nix store and daemon with Guix. + +Happy Guix/Nix hacking! diff --git a/src/content/en/blog/2018/08/01/npm-ci-reproducibility.adoc b/src/content/en/blog/2018/08/01/npm-ci-reproducibility.adoc new file mode 100644 index 0000000..7c1e837 --- /dev/null +++ b/src/content/en/blog/2018/08/01/npm-ci-reproducibility.adoc @@ -0,0 +1,148 @@ += Verifying "npm ci" reproducibility +:updatedat: 2019-05-22 + +:empty: +:npm-5: https://blog.npmjs.org/post/161081169345/v500 +:package-locks-old: https://docs.npmjs.com/files/package-locks +:package-lock: https://docs.npmjs.com/files/package-lock.json +:add-npm-ci: https://blog.npmjs.org/post/171556855892/introducing-npm-ci-for-faster-more-reliable +:cli-docs: https://docs.npmjs.com/cli/install#description +:tricky-issue: https://github.com/npm/npm/issues/17979#issuecomment-332701215 + +When {npm-5}[npm@5] came bringing {package-locks-old}[package-locks] with it, I +was confused about the benefits it provided, since running `npm install` more +than once could resolve all the dependencies again and yield yet another fresh +`package-lock.json` file. The message saying "you should add this file to +version control" left me hesitant on what to +do{empty}footnote:package-lock-message[ + {cli-docs}[documentation] claims `npm install` is driven by the existing + `package-lock.json`, but that's actually {tricky-issue}[a little bit tricky]. +]. + +However the {add-npm-ci}[addition of `npm ci`] filled this gap: it's a stricter +variation of `npm install` which guarantees that "{package-lock}[subsequent +installs are able to generate identical trees]". But are they really identical? +I could see that I didn't have the same problems of different installation +outputs, but I didn't know for *sure* if it was really identical. + +== Computing the hash of a directory's content + +:merkle-tree: https://en.wikipedia.org/wiki/Merkle_tree + +I quickly searched for a way to check for the hash signature of an entire +directory tree, but I couldn't find one. I've made a poor man's +{merkle-tree}[Merkle tree] implementation using `sha256sum` and a few piped +commands at the terminal: + +[source,sh] +---- +merkle-tree () { + dirname="${1-.}" + pushd "$dirname" + find . -type f | + sort | + xargs -I{} sha256sum "{}" | + sha256sum | + awk '{print $1}' + popd +} +---- + +Going through it line by line: + +* #1 we define a Bash function called `merkle-tree`; +* #2 it accepts a single argument: the directory to compute the merkle tree from + If nothing is given, it runs on the current directory (`.`); +* #3 we go to the directory, so we don't get different prefixes in `find`'s + output (like `../a/b`); +* #4 we get all files from the directory tree. Since we're using `sha256sum` to + compute the hash of the file contents, we need to filter out folders from it; +* #5 we need to sort the output, since different file systems and `find` + implementations may return files in different orders; +* #6 we use `xargs` to compute the hash of each file individually through + `sha256sum`. Since a file may contain spaces we need to escape it with + quotes; +* #7 we compute the hash of the combined hashes. Since `sha256sum` output is + formatted like `<hash> <filename>`, it produces a different final hash if a + file ever changes name without changing it's content; +* #8 we get the final hash output, excluding the `<filename>` (which is `-` in + this case, aka `stdin`). + +=== Positive points: + +. ignore timestamp: running more than once on different installation yields the + same hash; +. the name of the file is included in the final hash computation. + +=== Limitations: + +. it ignores empty folders from the hash computation; +. the implementation's only goal is to represent using a digest whether the + content of a given directory is the same or not. Leaf presence checking is + obviously missing from it. + +=== Testing locally with sample data + +[source,sh] +---- +mkdir /tmp/merkle-tree-test/ +cd /tmp/merkle-tree-test/ +mkdir -p a/b/ a/c/ d/ +echo "one" > a/b/one.txt +echo "two" > a/c/two.txt +echo "three" > d/three.txt +merkle-tree . # output is be343bb01fe00aeb8fef14a3e16b1c3d1dccbf86d7e41b4753e6ccb7dc3a57c3 +merkle-tree . # output still is be343bb01fe00aeb8fef14a3e16b1c3d1dccbf86d7e41b4753e6ccb7dc3a57c3 +echo "four" > d/four.txt +merkle-tree . # output is now b5464b958969ed81815641ace96b33f7fd52c20db71a7fccc45a36b3a2ae4d4c +rm d/four.txt +merkle-tree . # output back to be343bb01fe00aeb8fef14a3e16b1c3d1dccbf86d7e41b4753e6ccb7dc3a57c3 +echo "hidden-five" > a/b/one.txt +merkle-tree . # output changed 471fae0d074947e4955e9ac53e95b56e4bc08d263d89d82003fb58a0ffba66f5 +---- + +It seems to work for this simple test case. + +You can try copying and pasting it to verify the hash signatures. + +== Using `merkle-tree` to check the output of `npm ci` + +_I've done all of the following using Node.js v8.11.3 and npm@6.1.0_. + +In this test case I'll take the main repo of +https://lernajs.io/[Lerna]footnote:lerna-package-lock[ + Finding a big known repo that actually committed the `package-lock.json` file + was harder than I expected. +]: + +[source,sh] +---- +cd /tmp/ +git clone https://github.com/lerna/lerna.git +cd lerna/ +git checkout 57ff865c0839df75dbe1974971d7310f235e1109 +npm ci +merkle-tree node_modules/ # outputs 11e218c4ac32fac8a9607a8da644fe870a25c99821167d21b607af45699afafa +rm -rf node_modules/ +npm ci +merkle-tree node_modules/ # outputs 11e218c4ac32fac8a9607a8da644fe870a25c99821167d21b607af45699afafa +npm ci # test if it also works with an existing node_modules/ folder +merkle-tree node_modules/ # outputs 11e218c4ac32fac8a9607a8da644fe870a25c99821167d21b607af45699afafa +---- + +Good job `npm ci` :) + +#6 and #9 take some time to run (21 seconds in my machine), but this specific +use case isn't performance sensitive. The slowest step is computing the hash of +each individual file. + +== Conclusion + +`npm ci` really "generates identical trees". + +I'm not aware of any other existing solution for verifying the hash signature of +a directory. If you know any, shoot me an email, as I'd like to know it. + +== *Edit* + +2019-05-22: Fix spelling. diff --git a/src/content/en/blog/2018/12/21/ytdl-subs.adoc b/src/content/en/blog/2018/12/21/ytdl-subs.adoc new file mode 100644 index 0000000..10afbf6 --- /dev/null +++ b/src/content/en/blog/2018/12/21/ytdl-subs.adoc @@ -0,0 +1,279 @@ += Using "youtube-dl" to manage YouTube subscriptions + +:ytsm-ann: https://old.reddit.com/r/DataHoarder/comments/9sg8q5/i_built_a_selfhosted_youtube_subscription_manager/ +:ytsm-code: https://github.com/chibicitiberiu/ytsm +:ytdl: https://youtube-dl.org/ + +I've recently read the {ytsm-ann}[announcement] of a very nice +{ytsm-code}[self-hosted YouTube subscription manager]. I haven't used YouTube's +built-in subscriptions for a while now, and haven't missed it at all. When I +saw the announcement, I considered writing about the solution I've built on top +of {ytdl}[youtube-dl]. + +== Background: the problem with YouTube + +:net-giants: https://staltz.com/what-happens-when-you-block-internet-giants.html + +In many ways, I agree with {net-giants}[André Staltz's view on data ownership +and privacy]: + +____ +I started with the basic premise that "I want to be in control of my data". +Sometimes that meant choosing when to interact with an internet giant and how +much I feel like revealing to them. Most of times it meant not interacting with +them at all. I don't want to let them be in full control of how much they can +know about me. I don't want to be in autopilot mode. (...) Which leads us to +YouTube. While I was able to find alternatives to Gmail (Fastmail), Calendar +(Fastmail), Translate (Yandex Translate), _etc._ YouTube remains as the most +indispensable Google-owned web service. It is really really hard to avoid +consuming YouTube content. It was probably the smartest startup acquisition +ever. My privacy-oriented alternative is to watch YouTube videos through Tor, +which is technically feasible but not polite to use the Tor bandwidth for these +purposes. I'm still scratching my head with this issue. +____ + +Even though I don't use most alternative services he mentions, I do watch videos +from YouTube. But I also feel uncomfortable logging in to YouTube with a Google +account, watching videos, creating playlists and similar things. + +Using the mobile app is worse: you can't even block ads in there. You're in +less control on what you share with YouTube and Google. + +== youtube-dl + +:other-sites: https://rg3.github.io/youtube-dl/supportedsites.html + +youtube-dl is a command-line tool for downloading videos, from YouTube and +{other-sites}[many other sites]: + +[source,sh] +---- +$ youtube-dl https://www.youtube.com/watch?v=rnMYZnY3uLA +[youtube] rnMYZnY3uLA: Downloading webpage +[youtube] rnMYZnY3uLA: Downloading video info webpage +[download] Destination: A Origem da Vida _ Nerdologia-rnMYZnY3uLA.mp4 +[download] 100% of 32.11MiB in 00:12 +---- + +It can be used to download individual videos as showed above, but it also has +some interesting flags that we can use: + +* `--output`: use a custom template to create the name of the downloaded file; +* `--download-archive`: use a text file for recording and remembering which + videos were already downloaded; +* `--prefer-free-formats`: prefer free video formats, like `webm`, `ogv` and + Matroska `mkv`; +* `--playlist-end`: how many videos to download from a "playlist" (a channel, a + user or an actual playlist); +* `--write-description`: write the video description to a `.description` file, + useful for accessing links and extra content. + +Putting it all together: + +[source,sh] +---- +$ youtube-dl "https://www.youtube.com/channel/UClu474HMt895mVxZdlIHXEA" \ + --download-archive ~/Nextcloud/cache/youtube-dl-seen.conf \ + --prefer-free-formats \ + --playlist-end 20 \ + --write-description \ + --output "~/Downloads/yt-dl/%(uploader)s/%(upload_date)s - %(title)s.%(ext)s" +---- + +This will download the latest 20 videos from the selected channel, and write +down the video IDs in the `youtube-dl-seen.conf` file. Running it immediately +after one more time won't have any effect. + +If the channel posts one more video, running the same command again will +download only the last video, since the other 19 were already downloaded. + +With this basic setup you have a minimal subscription system at work, and you +can create some functions to help you manage that: + +[source,sh] +---- +#!/bin/sh + +export DEFAULT_PLAYLIST_END=15 + +download() { + youtube-dl "$1" \ + --download-archive ~/Nextcloud/cache/youtube-dl-seen.conf \ + --prefer-free-formats \ + --playlist-end "$2" \ + --write-description \ + --output "~/Downloads/yt-dl/%(uploader)s/%(upload_date)s - %(title)s.%(ext)s" +} +export -f download + + +download_user() { + download "https://www.youtube.com/user/$1" "${2-$DEFAULT_PLAYLIST_END}" +} +export -f download_user + + +download_channel() { + download "https://www.youtube.com/channel/$1" "${2-$DEFAULT_PLAYLIST_END}" +} +export -f download_channel + + +download_playlist() { + download "https://www.youtube.com/playlist?list=$1" "${2-$DEFAULT_PLAYLIST_END}" +} +export -f download_playlist +---- + +With these functions, you now can have a subscription fetching script to +download the latest videos from your favorite channels: + +[source,sh] +---- +#!/bin/sh + +download_user ClojureTV 15 +download_channel 'UCmEClzCBDx-vrt0GuSKBd9g' 100 +download_playlist 'PLqG7fA3EaMRPzL5jzd83tWcjCUH9ZUsbX' 15 +---- + +Now, whenever you want to watch the latest videos, just run the above script +and you'll get all of them in your local machine. + +== Tradeoffs + +=== I've made it for myself, with my use case in mind + + +[qanda] +Offline:: +My internet speed it somewhat +reasonable{empty}footnote:internet-speed[ + Considering how expensive it is and the many ways it could be better, but also + how much it has improved over the last years, I say it's reasonable. +], but it is really unstable. Either at work or at home, it's not uncommon to +loose internet access for 2 minutes 3~5 times every day, and stay completely +offline for a couple of hours once every week. ++ +Working through the hassle of keeping a playlist on disk has payed off many, +many times. Sometimes I even not notice when the connection drops for some +minutes, because I'm watching a video and working on some document, all on my +local computer. ++ +There's also no quality adjustment for YouTube's web player, I always pick the +higher quality and it doesn't change during the video. For some types of +content, like a podcast with some tiny visual resources, this doesn't change +much. For other types of content, like a keynote presentation with text written +on the slides, watching on 144p isn't really an option. ++ +If the internet connection drops during the video download, youtube-dl will +resume from where it stopped. ++ +This is an offline first benefit that I really like, and works well for me. + + +Sync the "seen" file:: +I already have a running instance of Nextcloud, so just dumping the +`youtube-dl-seen.conf` file inside Nextcloud was a no-brainer. ++ +You could try putting it in a dedicated git repository, and wrap the script with +an autocommit after every run. If you ever had a merge conflict, you'd simply +accept all changes and then run the following to tidy up the file: ++ +[source,sh] +---- +$ uniq youtube-dl-seen.conf > youtube-dl-seen.conf +---- + + +Doesn't work on mobile:: +My primary device that I use everyday is my laptop, not my phone. It works well +for me this way. ++ +Also, it's harder to add ad-blockers to mobile phones, and most mobile software +still depends on Google's and Apple's blessing. ++ +If you wish, you can sync the videos to the SD card periodically, but that's a +bit of extra manual work. + + +=== The Good + + +[qanda] +Better privacy:: +We don't even have to configure the ad-blocker to keep ads and trackers away! ++ +YouTube still has your IP address, so using a VPN is always a good idea. +However, a timing analysis would be able to identify you (considering the +current implementation). + + +No need to self-host:: +There's no host that needs maintenance. Everything runs locally. ++ +As long as you keep youtube-dl itself up to date and sync your "seen" file, +there's little extra work to do. + + +Track your subscriptions with git:: +After creating a `subscriptions.sh` executable that downloads all the videos, +you can add it to git and use it to track metadata about your subscriptions. + + +=== The Bad + + +[qanda] +Maximum playlist size is your disk size:: +This is a good thing for getting a realistic view on your actual "watch later" +list. However I've run out of disk space many times, and now I need to be more +aware of how much is left. + + +=== The Ugly + +We can only avoid all the bad parts of YouTube with youtube-dl as long as +YouTube keeps the videos public and programmatically accessible. If YouTube +ever blocks that we'd loose the ability to consume content this way, but also +loose confidence on considering YouTube a healthy repository of videos on the +internet. + + +== Going beyond + +Since you're running everything locally, here are some possibilities to be +explored: + + +=== A playlist that is too long for being downloaded all at once + +You can wrap the `download_playlist` function (let's call the wrapper +`inc_download`) and instead of passing it a fixed number to the `--playlist-end` +parameter, you can store the `$n` in a folder (something like +`$HOME/.yt-db/$PLAYLIST_ID`) and increment it by `$step` every time you run +`inc_download`. + +This way you can incrementally download videos from a huge playlist without +filling your disk with gigabytes of content all at once. + + +=== Multiple computer scenario + +The `download_playlist` function could be aware of the specific machine that it +is running on and apply specific policies depending on the machine: always +download everything; only download videos that aren't present anywhere else; +_etc._ + + +== Conclusion + +youtube-dl is a great tool to keep at hand. It covers a really large range of +video websites and works robustly. + +Feel free to copy and modify this code, and send me suggestions of improvements +or related content. + +== _Edit_ + +2019-05-22: Fix spelling. diff --git a/src/content/en/blog/2019/06/02/nixos-stateless-workstation.adoc b/src/content/en/blog/2019/06/02/nixos-stateless-workstation.adoc new file mode 100644 index 0000000..f89a106 --- /dev/null +++ b/src/content/en/blog/2019/06/02/nixos-stateless-workstation.adoc @@ -0,0 +1,146 @@ += Using NixOS as an stateless workstation + +:empty: +:nixos: https://nixos.org/ + +Last +week{empty}footnote:last-week[ + "Last week" as of the start of this writing, so around the end of May 2019. +] I changed back to an +old{empty}footnote:old-computer[ + I was using a 32GB RAM, i7 and 250GB SSD Samsung laptop. The switch was back + to a 8GB RAM, i5 and 500GB HDD Dell laptop. The biggest difference I noticed + was on faster memory, both RAM availability and the disk speed, but I had + 250GB less local storage space. +] Samsung laptop, and installed {nixos}[NixOS] on it. + +After using NixOS on another laptop for around two years, I wanted verify how +reproducible was my desktop environment, and how far does NixOS actually can go +on recreating my whole OS from my configuration files and personal data. I +gravitated towards NixOS after trying (and failing) to create an `install.sh` +script that would imperatively install and configure my whole OS using apt-get. +When I found a GNU/Linux distribution that was built on top of the idea of +declaratively specifying the whole OS I was automatically +convinced{empty}footnote:convincend-by-declarative-aspect[ + The declarative configuration aspect is something that I now completely take + for granted, and wouldn't consider using something which isn't declarative. A + good metric to show this is me realising that I can't pinpoint the moment when + I decided to switch to NixOS. It's like I had a distant past when this wasn't + true. +]. + +I was impressed. Even though I've been experiencing the benefits of Nix +isolation daily, I always felt skeptical that something would be missing, +because the devil is always on the details. But the result was much better than +expected! + +There were only 2 missing configurations: + +. tap-to-click on the touchpad wasn't enabled by default; +. the default theme from the gnome-terminal is "Black on white" instead of + "White on black". + +That's all. + +I haven't checked if I can configure those in NixOS GNOME module, but I guess +both are scriptable and could be set in a fictional `setup.sh` run. + +This makes me really happy, actually. More happy than I anticipated. + +Having such a powerful declarative OS makes me feel like my data is the really +important stuff (as it should be), and I can interact with it on any +workstation. All I need is an internet connection and a few hours to download +everything. It feels like my physical workstation and the installed OS are +serving me and my data, instead of me feeling as hostage to the specific OS +configuration at the moment. Having a few backup copies of everything important +extends such peacefulness. + +After this positive experience with recreating my OS from simple Nix +expressions, I started to wonder how far I could go with this, and started +considering other areas of improvements: + +== First run on a fresh NixOS installation + +Right now the initial setup relies on non-declarative manual tasks, like +decrypting some credentials, or manually downloading *this* git repository with +specific configurations before *that* one. + +I wonder what some areas of improvements are on this topic, and if investing on +it is worth it (both time-wise and happiness-wise). + +== Emacs + +:spacemacs: https://spacemacs.org/ +:emacs: https://www.gnu.org/software/emacs/ +:layers: https://spacemacs.org/doc/LAYERS.html +:there: https://nixos.org/nixos/manual/index.html#module-services-emacs-adding-packages +:packages: https://www.gnu.org/software/guix/manual/en/html_node/Application-Setup.html#Emacs-Packages + +Right now I'm using the {spacemacs}[Spacemacs], which is a community package +curation and configuration on top of {emacs}[Emacs]. + +Spacemacs does support the notion of {layers}[layers], which you can +declaratively specify and let Spacemacs do the rest. + +However this solution isn't nearly as robust as Nix: being purely functional, +Nix does describe everything required to build a derivation, and knows how to do +so. Spacemacs it closer to more traditional package managers: even though the +layers list is declarative, the installation is still very much imperative. +I've had trouble with Spacemacs not behaving the same on different computers, +both with identical configurations, only brought to convergence back again after +a `git clean -fdx` inside `~/.emacs.d/`. + +The ideal solution would be managing Emacs packages with Nix itself. After a +quick search I did found that {there}[there is support for Emacs packages in +Nix]. So far I was only aware of {packages}[Guix support for Emacs packages]. + +This isn't a trivial change because Spacemacs does include extra curation and +configuration on top of Emacs packages. I'm not sure the best way to improve +this right now. + +== myrepos + +:myrepos: https://myrepos.branchable.com/ + +I'm using {myrepos}[myrepos] to manage all my git repositories, and the general +rule I apply is to add any repository specific configuration in myrepos' +`checkout` phase: + +[source,sh] +---- +# sample ~/.mrconfig file snippet +[dev/guix/guix] +checkout = + git clone https://git.savannah.gnu.org/git/guix.git guix + cd guix/ + git config sendemail.to guix-patches@gnu.org +---- + +This way when I clone this repo again the email sending is already +pre-configured. + +This works well enough, but the solution is too imperative, and my `checkout` +phases tend to become brittle over time if not enough care is taken. + +== GNU Stow + +:not-at-all: https://euandre.org/git/dotfiles/tree/bash/symlinks.sh?id=316939aa215181b1d22b69e94241eef757add98d +:stow: https://www.gnu.org/software/stow/ + +For my home profile and personal configuration I already have a few dozens of +symlinks that I manage manually. This has worked so far, but the solution is +sometimes fragile and {not-at-all}[not declarative at all]. I wonder if +something like {stow}[GNU Stow] can help me simplify this. + +== Conclusion + +:nix: https://nixos.org/nix/ + +I'm really satisfied with NixOS, and I intend to keep using it. If what I've +said interests you, maybe try tinkering with the {nix}[Nix package manager] (not +the whole NixOS) on your current distribution (it can live alongside any other +package manager). + +If you have experience with declarative Emacs package managements, GNU Stow or +any similar tool, _etc._, mail me some tips]. If you don't have any experience +at all, I'd still love to hear from you. diff --git a/src/content/en/blog/2020/08/10/guix-srht.adoc b/src/content/en/blog/2020/08/10/guix-srht.adoc new file mode 100644 index 0000000..a89e86e --- /dev/null +++ b/src/content/en/blog/2020/08/10/guix-srht.adoc @@ -0,0 +1,128 @@ += Guix inside sourcehut builds.sr.ht CI +:updatedat: 2020-08-19 + +:nixos: https://man.sr.ht/builds.sr.ht/compatibility.md#nixos +:guix: https://guix.gnu.org/ +:binary-inst: https://guix.gnu.org/manual/en/guix.html#Binary-Installation +:shell-inst: https://git.savannah.gnu.org/cgit/guix.git/plain/etc/guix-install.sh + +After the release of the {nixos}[NixOS images in builds.sr.ht] and much usage of +it, I also started looking at {guix}[Guix] and wondered if I could get it on the +awesome builds.sr.ht service. + +The Guix manual section on the {binary-inst}[binary installation] is very +thorough, and even a {shell-inst}[shell installer script] is provided, but it is +built towards someone installing Guix on their personal computer, and relies +heavily on interactive input. + +I developed the following set of scripts that I have been using for some time to +run Guix tasks inside builds.sr.ht jobs. First, `install-guix.sh`: + +[source,sh] +---- +#!/usr/bin/env bash +set -x +set -Eeuo pipefail + +VERSION='1.0.1' +SYSTEM='x86_64-linux' +BINARY="guix-binary-${VERSION}.${SYSTEM}.tar.xz" + +cd /tmp +wget "https://ftp.gnu.org/gnu/guix/${BINARY}" +tar -xf "${BINARY}" + +sudo mv var/guix /var/ +sudo mv gnu / +sudo mkdir -p ~root/.config/guix +sudo ln -fs /var/guix/profiles/per-user/root/current-guix ~root/.config/guix/current + +GUIX_PROFILE="$(echo ~root)/.config/guix/current" +source "${GUIX_PROFILE}/etc/profile" + +groupadd --system guixbuild +for i in $(seq -w 1 10); +do + useradd -g guixbuild \ + -G guixbuild \ + -d /var/empty \ + -s "$(command -v nologin)" \ + -c "Guix build user ${i}" --system \ + "guixbuilder${i}"; +done + +mkdir -p /usr/local/bin +cd /usr/local/bin +ln -s /var/guix/profiles/per-user/root/current-guix/bin/guix . +ln -s /var/guix/profiles/per-user/root/current-guix/bin/guix-daemon . + +guix archive --authorize < ~root/.config/guix/current/share/guix/ci.guix.gnu.org.pub +---- + +Almost all of it is taken directly from the {binary-inst}[binary installation] +section from the manual, with the interactive bits stripped out: after +downloading and extracting the Guix tarball, we create some symlinks, add +guixbuild users and authorize the `ci.guix.gnu.org.pub` signing key. + +After installing Guix, we perform a `guix pull` to update Guix inside +`start-guix.sh`: + +[source,sh] +---- +#!/usr/bin/env bash +set -x +set -Eeuo pipefail + +sudo guix-daemon --build-users-group=guixbuild & +guix pull +guix package -u +guix --version +---- + +Then we can put it all together in a sample `.build.yml` configuration file I'm +using myself: + +[source,yaml] +---- +image: debian/stable +packages: + - wget +sources: + - https://git.sr.ht/~euandreh/songbooks +tasks: + - install-guix: | + cd ./songbooks/ + ./scripts/install-guix.sh + ./scripts/start-guix.sh + echo 'sudo guix-daemon --build-users-group=guixbuild &' >> ~/.buildenv + echo 'export PATH="${HOME}/.config/guix/current/bin${PATH:+:}$PATH"' >> ~/.buildenv + - tests: | + cd ./songbooks/ + guix environment -m build-aux/guix.scm -- make check + - docs: | + cd ./songbooks/ + guix environment -m build-aux/guix.scm -- make publish-dist +---- + +We have to add the `guix-daemon` to `~/.buildenv` so it can be started on every +following task run. Also, since we used `wget` inside `install-guix.sh`, we had +to add it to the images package list. + +After the `install-guix` task, you can use Guix to build and test your project, +or run any `guix environment --ad-hoc my-package -- my script` :) + +== Improvements + +:repository: https://git.sr.ht/~sircmpwn/builds.sr.ht + +When I originally created this code I had a reason why to have both a `sudo` +call for `sudo ./scripts/install-guix.sh` and `sudo` usages inside +`install-guix.sh` itself. I couldn't figure out why (it feels like my past self +was a bit smarter 😬), but it feels ugly now. If it is truly required I could +add an explanation for it, or remove this entirely in favor of a more elegant +solution. + +I could also contribute the Guix image upstream to builds.sr.ht, but there +wasn't any build or smoke tests in the original {repository}[repository], so I +wasn't inclined to make something that just ``works on my machine'' or add a +maintainence burden to the author. I didn't look at it again recently, though. diff --git a/src/content/en/blog/2020/08/31/database-i-wish-i-had.adoc b/src/content/en/blog/2020/08/31/database-i-wish-i-had.adoc new file mode 100644 index 0000000..7f010b9 --- /dev/null +++ b/src/content/en/blog/2020/08/31/database-i-wish-i-had.adoc @@ -0,0 +1,299 @@ += The database I wish I had +:categories: mediator +:updatedat: 2020-09-03 + +:empty: +:values-talk: https://vimeo.com/230142234 +:haskell-startup: https://www.youtube.com/watch?v=ZR3Jirqk6W8 + +I watched the talk "{values-talk}[Platform as a Reflection of Values: Joyent, +Node.js and beyond]" by Bryan Cantrill, and I think he was able to put into +words something I already felt for some time: if there's no piece of software +out there that reflects your values, it's time for you to build that +software{empty}footnote:talk-time[ + At the very end, at time 29:49. When talking about the draft of this article + with a friend, he noted that Bryan O'Sullivan (a different Bryan) says a + similar thing on his talk "{haskell-startup}[Running a startup on Haskell]", + at time 4:15. +]. + +I kind of agree with what he said, because this is already happening to me. I +long for a database with a certain set of values, and for a few years I was just +waiting for someone to finally write it. After watching his talk, Bryan is +saying to me: "time to stop waiting, and start writing it yourself". + +So let me try to give an overview of such database, and go over its values. + +== Overview + +I want a database that allows me to create decentralized client-side +applications that can sync data. + +The best one-line description I can give right now is: + +____ +It's sort of like PouchDB, Git, Datomic, SQLite and Mentat. +____ + +A more descriptive version could be: + +____ +An embedded, immutable, syncable relational database. +____ + +Let's go over what I mean by each of those aspects one by one. + +=== Embedded + +:sqlite: https://sqlite.org/index.html +:sqlite-whentouse: https://sqlite.org/whentouse.html +:pouchdb: https://pouchdb.com/ +:couchdb: https://couchdb.apache.org/ +:mentat: https://github.com/mozilla/mentat +:pouchdb-adapters: https://pouchdb.com/adapters.html +:datomic-storage-services: https://docs.datomic.com/on-prem/storage.html +:sqlite-amalgamation: https://www.sqlite.org/amalgamation.html +:pointed-out: https://news.ycombinator.com/item?id=24338881 + +I think the server-side database landscape is diverse and mature enough for my +needs (even though I end up choosing SQLite most of the time), and what I'm +after is a database to be embedded on client-side applications itself, be it +desktop, browser, mobile, _etc._ + +The purpose of such database is not to keep some local cache of data in case of +lost connectivity: we have good solutions for that already. It should serve as +the source of truth, and allow the application to work on top of it. + +{sqlite}[*SQLite*] is a great example of that: it is a very powerful relational +database that runs {sqlite-whentouse}[almost anywhere]. What I miss from it +that SQLite doesn't provide is the ability to run it on the browser: even though +you could compile it to WebAssembly, [line-through]#it assumes a POSIX +filesystem that would have to be +emulated#{empty}footnote:posix-sqlite[ + It was {pointed-out}[pointed out to me] that SQLite doesn't assume the + existence of a POSIX filesystem, as I wrongly stated. Thanks for the + correction. +pass:[</p><p>] + This makes me consider it as a storage backend all by itself. I initially + considered having an SQLite storage backend as one implementation of the POSIX + filesystem storage API that I mentioned. My goal was to rely on it so I could + validate the correctness of the actual implementation, given SQLite's + robustness. +pass:[</p><p>] + However it may even better to just use SQLite, and get an ACID backend without + recreating a big part of SQLite from scratch. In fact, both Datomic and + PouchDB didn't create an storage backend for themselves, they just plugged on + what already existed and already worked. I'm beginning to think that it would + be wiser to just do the same, and drop entirely the from scratch + implementation that I mentioned. +pass:[</p><p>] + That's not to say that adding an IndexedDB compatibility layer to SQLite would + be enough to make it fit the other requirements I mention on this page. SQLite + still is an implementation of a update-in-place, SQL, table-oriented database. + It is probably true that cherry-picking the relevant parts of SQLite (like + storage access, consistency, crash recovery, parser generator, *etc.*) and + leaving out the unwanted parts (SQL, tables, threading, *etc.*) would be + better than including the full SQLite stack, that's simply an optimization. + Both could even coexist, if desired. +pass:[</p><p>] + SQLite would have to be treated similarly to how Datomic treats SQL databases: + instead of having a table for each entities, spread attributes over the + tables, *etc.*, it treats SQL databases as a key-value storage so it doesn't + have to re-implement interacting with the disk that other databases do well. +pass:[</p><p>] + The tables would contain blocks of binary data, so there isn't a difference on + how the SQLite storage backend behaves and how the IndexedDB storage backend + behaves, much like how Datomic works the same regardless of the storage + backend, same for PouchDB. +pass:[</p><p>] + I welcome corrections on what I said above, too. +]. + +{pouchdb}[*PouchDB*] is another great example: it's a full reimplementation of +{couchdb}[CouchDB] that targets JavaScript environments, mainly the browser and +Node.js. However I want a tool that can be deployed anywhere, and not limit its +applications to places that already have a JavaScript runtime environment, or +force the developer to bundle a JavaScript runtime environment with their +application. This is true for GTK+ applications, command line programs, Android +apps, _etc._ + +{mentat}[*Mentat*] was an interesting project, but its reliance on SQLite makes +it inherit most of the downsides (and benefits too) of SQLite itself. + +Having such a requirement imposes a different approach to storage: we have to +decouple the knowledge about the intricacies of storage from the usage of +storage itself, so that a module (say query processing) can access storage +through an API without needing to know about its implementation. This allows +the database to target a POSIX filesystems storage API and an IndexedDB storage +API, and make the rest of the code agnostic about storage. PouchDB has such +mechanism (called {pouchdb-adapters}[adapters]) and Datomic has them too (called +{datomic-storage-services}[storage services]). + +This would allow the database to adapt to where it is embedded: when targeting +the browser the IndexedDB storage API would provide the persistence layer that +the database requires, and similarly the POSIX filesystem storage API would +provide the persistence layer when targeting POSIX systems (like desktops, +mobile, _etc._). + +But there's also an extra restriction that comes from by being embedded: it +needs to provide and embeddable artifact, most likely a binary library object +that exposes a C compatible FFI, similar to {sqlite-amalgamation}[how SQLite +does]. Bundling a full runtime environment is possible, but doesn't make it a +compelling solution for embedding. This rules out most languages, and leaves +us with C, Rust, Zig, and similar options that can target POSIX systems and +WebAssembly. + +=== Immutable + +:datomic: https://www.datomic.com/ +:day-of-datomic: https://vimeo.com/116315075 +:git: https://git-scm.com/ +:sqlite-limits: https://sqlite.org/limits.html +:datomic-no-history: https://docs.datomic.com/cloud/best.html#nohistory-for-high-churn + +Being immutable means that only new information is added, no in-place update +ever happens, and nothing is ever deleted. + +Having an immutable database presents us with similar trade-offs found in +persistent data structures, like lack of coordination when doing reads, caches +being always coherent, and more usage of space. + +{datomic}[*Datomic*] is the go to database example of this: it will only add +information (datoms) and allows you to query them in a multitude of ways. +Stuart Halloway calls it "accumulate-only" over +"append-only"{empty}footnote:accumulate-only[ + Video "{day-of-datomic}[Day of Datomic Part 2]" on Datomic's information + model, at time 12:28. +]: + +____ +It's accumulate-only, it is not append-only. So append-only, most people when +they say that they're implying something physical about what happens. +____ + +Also a database can be append-only and overwrite existing information with new +information, by doing clean-ups of "stale" data. I prefer to adopt the +"accumulate-only" naming and approach. + +{git}[*Git*] is another example of this: new commits are always added on top of +the previous data, and it grows by adding commits instead of replacing existing +ones. + +Git repositories can only grow in size, and that is not only an acceptable +condition, but also one of the reasons to use it. + +All this means that no in-place updates happens on data, and the database will +be much more concerned about how compact and efficiently it stores data than how +fast it does writes to disk. Being embedded, the storage limitation is either +a) how much storage the device has or b) how much storage was designed for the +application to consume. So even though the database could theoretically operate +with hundreds of TBs, a browser page or mobile application wouldn't have access +to this amount of storage. SQLite even {sqlite-limits}[says] that it does +support approximately 280 TBs of data, but those limits are untested. + +The upside of keeping everything is that you can have historical views of your +data, which is very powerful. This also means that applications should turn +this off when not +relevant{empty}footnote:no-history[ + Similar to {datomic-no-history}[Datomic's `:db/noHistory`]. +]. + +=== Syncable + +:3-way-merge: https://en.wikipedia.org/wiki/Merge_(version_control) +:git-remote-gcrypt: https://spwhitton.name/tech/code/git-remote-gcrypt/ + +This is a frequent topic when talking about offline-first solutions. When +building applications that: + +* can fully work offline, +* stores data, +* propagates that data to other application instances, + +then you'll need a conflict resolution strategy to handle all the situations +where different application instances disagree. Those application instances +could be a desktop and a browser version of the same application, or the same +mobile app in different devices. + +A three-way merge seems to be the best approach, on top of which you could add +application specific conflict resolution functions, like: + +* pick the change with higher timestamp; +* if one change is a delete, pick it; +* present the diff on the screen and allow the user to merge them. + +Some databases try to make this "easy", by choosing a strategy for you, but I've +found that different applications require different conflict resolution +strategies. Instead, the database should leave this up to the user to decide, +and provide tools for them to do it. + +{3-way-merge}[*Three-way merges in version control*] are the best example, +performing automatic merges when possible and asking the user to resolve +conflicts when they appear. + +The unit of conflict for a version control system is a line of text. The +database equivalent would probably be a single attribute, not a full entity or a +full row. + +Making all the conflict resolution logic be local should allow the database to +have encrypted remotes similar to how {git-remote-gcrypt}[git-remote-gcrypt] +adds this functionality to Git. This would enable users to sync the application +data across devices using an untrusted intermediary. + +=== Relational + +:datomic-datalog: https://docs.datomic.com/on-prem/query.html +:datomic-model: https://docs.datomic.com/cloud/whatis/data-model.html#datoms + +I want the power of relational queries on the client applications. + +Most of the arguments against traditional table-oriented relational databases +are related to write performance, but those don't apply here. The bottlenecks +for client applications usually aren't write throughput. Nobody is interested +in differentiating between 1 MB/s or 10 MB/s when you're limited to 500 MB +total. + +The relational model of the database could either be based on SQL and tables +like in SQLite, or maybe {datomic-datalog}[datalog] and {datomic-model}[datoms] +like in Datomic. + +== From aspects to values + +Now let's try to translate the aspects above into values, as suggested by Bryan +Cantrill. + +=== Portability + +Being able to target so many different platforms is a bold goal, and the +embedded nature of the database demands portability to be a core value. + +=== Integrity + +When the local database becomes the source of truth of the application, it must +provide consistency guarantees that enables applications to rely on it. + +=== Expressiveness + +The database should empower applications to slice and dice the data in any way +it wants to. + +== Next steps + +Since I can't find any database that fits these requirements, I've finally come +to terms with doing it myself. + +It's probably going to take me a few years to do it, and making it portable +between POSIX and IndexedDB will probably be the biggest challenge. I got +myself a few books on databases to start. + +I wonder if I'll ever be able to get this done. + +== External links + +:reddit: https://old.reddit.com/r/programming/comments/ijwz5b/the_database_i_wish_i_had/ +:lobsters: https://lobste.rs/s/m9vkg4/database_i_wish_i_had +:hn: https://news.ycombinator.com/item?id=24337244 +:list: https://lists.sr.ht/~euandreh/public-inbox/%3C010101744a592b75-1dce9281-f0b8-4226-9d50-fd2c7901fa72-000000%40us-west-2.amazonses.com%3E + +See discussions on {reddit}[Reddit], {lobsters}[lobsters], {hn}[HN] and {list}[a +lengthy email exchange]. diff --git a/src/content/en/blog/2020/10/05/cargo2nix-demo.tar.gz b/src/content/en/blog/2020/10/05/cargo2nix-demo.tar.gz Binary files differnew file mode 100644 index 0000000..43677ec --- /dev/null +++ b/src/content/en/blog/2020/10/05/cargo2nix-demo.tar.gz diff --git a/src/content/en/blog/2020/10/05/cargo2nix.adoc b/src/content/en/blog/2020/10/05/cargo2nix.adoc new file mode 100644 index 0000000..a2d478e --- /dev/null +++ b/src/content/en/blog/2020/10/05/cargo2nix.adoc @@ -0,0 +1,72 @@ += cargo2nix: Dramatically simpler Rust in Nix +:sort: 1 + +:empty: +:swift2nix: link:swift2nix.html +:cargo2nix: link:cargo2nix-demo.tar.gz + +In the same vein of my earlier post on {swift2nix}[swift2nix], I was able to +quickly prototype a Rust and Cargo variation of it: {cargo2nix}[cargo2nix]. + +The initial prototype is even smaller than swift2nix: it has only 37 lines of +code. + +Here's how to use it (snippet taken from the repo's README): + +[source,nix] +---- +let + niv-sources = import ./nix/sources.nix; + mozilla-overlay = import niv-sources.nixpkgs-mozilla; + pkgs = import niv-sources.nixpkgs { overlays = [ mozilla-overlay ]; }; + src = pkgs.nix-gitignore.gitignoreSource [ ] ./.; + cargo2nix = pkgs.callPackage niv-sources.cargo2nix { + lockfile = ./Cargo.lock; + }; +in pkgs.stdenv.mkDerivation { + inherit src; + name = "cargo-test"; + buildInputs = [ pkgs.latest.rustChannels.nightly.rust ]; + phases = [ "unpackPhase" "buildPhase" ]; + buildPhase = '' + # Setup dependencies path to satisfy Cargo + mkdir .cargo/ + ln -s ${cargo2nix.env.cargo-config} .cargo/config + ln -s ${cargo2nix.env.vendor} vendor + + # Run the tests + cargo test + touch $out + ''; +} +---- + +That `cargo test` part on line 20 is what I have been fighting with every +"*2nix" available for Rust out there. I don't want to bash any of them. All I +want is to have full control of what Cargo commands to run, and the "*2nix" tool +should only setup the environment for me. Let me drive Cargo myself, no need to +parameterize how the tool runs it for me, or even replicate its internal +behaviour by calling the Rust compiler directly. + +Sure it doesn't support private registries or Git dependencies, but how much +bigger does it has to be to support them? Also, it doesn't support those *yet*, +there's no reason it can't be extended. I just haven't needed it yet, so I +haven't added. Patches welcome. + +The layout of the `vendor/` directory is more explicit and public then what +swift2nix does: it is whatever the command `cargo vendor` returns. However I +haven't checked if the shape of the `.cargo-checksum.json` is specified, or +internal to Cargo. + +Try out the demo (also taken from the repo's README): + +[source,sh] +---- +pushd "$(mktemp -d)" +wget -O- https://euandre.org/static/attachments/cargo2nix-demo.tar.gz | + tar -xv +cd cargo2nix-demo/ +nix-build +---- + +Report back if you wish. diff --git a/src/content/en/blog/2020/10/05/cargo2nix.tar.gz b/src/content/en/blog/2020/10/05/cargo2nix.tar.gz Binary files differnew file mode 100644 index 0000000..d7224d9 --- /dev/null +++ b/src/content/en/blog/2020/10/05/cargo2nix.tar.gz diff --git a/src/content/en/blog/2020/10/05/swift2nix-demo.tar.gz b/src/content/en/blog/2020/10/05/swift2nix-demo.tar.gz Binary files differnew file mode 100644 index 0000000..cc8b4f1 --- /dev/null +++ b/src/content/en/blog/2020/10/05/swift2nix-demo.tar.gz diff --git a/src/content/en/blog/2020/10/05/swift2nix.adoc b/src/content/en/blog/2020/10/05/swift2nix.adoc new file mode 100644 index 0000000..9a3c6fe --- /dev/null +++ b/src/content/en/blog/2020/10/05/swift2nix.adoc @@ -0,0 +1,194 @@ += swift2nix: Run Swift inside Nix builds +:sort: 0 + +:empty: +:nix: https://nixos.org/ +:swift2nix: link:swift2nix.tar.gz + +While working on a Swift project, I didn't find any tool that would allow Swift +to run inside {nix}[Nix] builds. Even thought you _can_ run Swift, the real +problem arises when using the package manager. It has many of the same problems +that other package managers have when trying to integrate with Nix, more on this +below. + +I wrote a simple little tool called {swift2nix}[swift2nix] that allows you trick +Swift's package manager into assuming everything is set up. Here's the example +from swift2nix's README file: + +[source,nix] +---- +let + niv-sources = import ./nix/sources.nix; + pkgs = import niv-sources.nixpkgs { }; + src = pkgs.nix-gitignore.gitignoreSource [ ] ./.; + swift2nix = pkgs.callPackage niv-sources.swift2nix { + package-resolved = ./Package.resolved; + }; +in pkgs.stdenv.mkDerivation { + inherit src; + name = "swift-test"; + buildInputs = with pkgs; [ swift ]; + phases = [ "unpackPhase" "buildPhase" ]; + buildPhase = '' + # Setup dependencies path to satisfy SwiftPM + mkdir .build + ln -s ${swift2nix.env.dependencies-state-json} .build/dependencies-state.json + ln -s ${swift2nix.env.checkouts} .build/checkouts + + # Run the tests + swift test + touch $out + ''; +} +---- + +The key parts are lines 15~17: we just fake enough files inside `.build/` that +Swift believes it has already downloaded and checked-out all dependencies, and +just moves on to building them. + +I've worked on it just enough to make it usable for myself, so beware of +unimplemented cases. + +== Design + +What swift2nix does is just provide you with the bare minimum that Swift +requires, and readily get out of the way: + +. I explicitly did not want to generated a `Package.nix` file, since + `Package.resolved` already exists and contains the required information; +. I didn't want to have an "easy" interface right out of the gate, after + fighting with "*2nix" tools that focus too much on that. + +The final actual code was so small (46 lines) that it made me think about +package managers, "*2nix" tools and some problems with many of them. + +== Problems with package managers + +I'm going to talk about solely language package managers. Think npm and cargo, +not apt-get. + +Package managers want to do too much, or assume too much, or just want to take +control of the entire build of the dependencies. + +This is a recurrent problem in package managers, but I don't see it as an +intrinsic one. There's nothing about a "package manager" that prevents it from +_declaring_ what it expects to encounter and in which format. The _declaring_ +part is important: it should be data, not code, otherwise you're back in the +same problem, just like lockfiles are just data. Those work in any language, +and tools can cooperate happily. + +There's no need for this declarative expectation to be standardized, or be made +compatible across languages. That would lead to a poor format that no package +manager really likes. Instead, If every package manager could say out loud what +it wants to see exactly, than more tools like swift2nix could exist, and they +would be more reliable. + +This could even work fully offline, and be simply a mapping from the lockfile +(the `Package.resolved` in Swift's case) to the filesystem representation. For +Swift, the `.build/dependencies-state.json` comes very close, but it is internal +to the package manager. + +Even though this pain only exists when trying to use Swift inside Nix, it sheds +light into this common implicit coupling that package managers have. They +usually have fuzzy boundaries and tight coupling between: + +. resolving the dependency tree and using some heuristic to pick a package + version; +. generating a lockfile with the exact pinned versions; +. downloading the dependencies present on the lockfile into some local cache; +. arranging the dependencies from the cache in a meaningful way for itself + inside the project; +. work using the dependencies while _assuming_ that step 4 was done. + +When you run `npm install` in a repository with no lockfile, it does 1~4. If +you do the same with `cargo build`, it does 1~5. That's too much: many of those +assumptions are implicit and internal to the package manager, and if you ever +need to rearrange them, you're on your own. Even though you can perform some of +those steps, you can't compose or rearrange them. + +Instead a much saner approach could be: + +. this stays the same; +. this also stays the same; +. be able to generate some JSON/TOML/edn which represents the local expected + filesystem layout with dependencies (i.e. exposing what the package manager + expects to find), let's call it `local-registry.json`; +. if a `local-registry.json` was provided, do a build using that. Otherwise + generate its own, by downloading the dependencies, arranging them, _etc._ + +The point is just making what the package manager requires visible to the +outside world via some declarative data. If this data wasn't provided, it can +move on to doing its own automatic things. + +By making the expectation explicit and public, one can plug tools _à la carte_ +if desired, but doesn't prevent the default code path of doing things the exact +same way they are now. + +== Problems with "*2nix" tools + +:node2nix: https://github.com/svanderburg/node2nix + +I have to admit: I'm unhappy with most of they. + +They conflate "using Nix" with "replicating every command of the package manager +inside Nix". + +The avoidance of an "easy" interface that I mentioned above comes from me +fighting with some of the "*2nix" tools much like I have to fight with package +managers: I don't want to offload all build responsibilities to the "*2nix" +tool, I just want to let it download some of the dependencies and get out of the +way. I want to stick with `npm test` or `cargo build`, and Nix should only +provide the environment. + +This is something that {node2nix}[node2nix] does right. It allows you to build +the Node.js environment to satisfy NPM, and you can keep using NPM for +everything else: + +[source,sh] +---- +ln -s ${node2nix-package.shell.nodeDependencies}/lib/node_modules ./node_modules +npm test +---- + +Its natural to want to put as much things into Nix as possible to benefit from +Nix's advantages. Isn't that how NixOS itself was born? + +But a "*2nix" tool should leverage Nix, not be coupled with it. The above +example lets you run any arbitrary NPM command while profiting from isolation +and reproducibility that Nix provides. It is even less brittle: any changes to +how NPM runs some things will be future-compatible, since node2nix isn't trying +to replicate what NPM does, or fiddling with NPM's internal. + +**A "*2nix" tool should build the environment, preferably from the lockfile +directly and offload everything else to the package manager**. The rest is just +nice-to-have. + +swift2nix itself could provide an "easy" interface, something that allows you to +write: + +[source,sh] +---- +nix-build -A swift2nix.release +nix-build -A swift2nix.test +---- + +The implementation of those would be obvious: create a new +`pkgs.stdenv.mkDerivation` and call `swift build -c release` and `swift test` +while using `swift2nix.env` under the hood. + +== Conclusion + +Package managers should provide exact dependencies via a data representation, +i.e. lockfiles, and expose via another data representation how they expect those +dependencies to appear on the filesystem, i.e. `local-registry.json`. This +allows package managers to provide an API so that external tools can create +mirrors, offline builds, other registries, isolated builds, _etc._ + +"*2nix" tools should build simple functions that leverage that +`local-registry.json`{empty}footnote:local-registry[ + This `local-registry.json` file doesn't have to be checked-in the repository + at all. It could be always generated on the fly, much like how Swift's + `dependencies-state.json` is. +] data and offload all the rest back to the package manager itself. This allows +the "*2nix" to not keep chasing the package manager evolution, always trying to +duplicate its behaviour. diff --git a/src/content/en/blog/2020/10/05/swift2nix.tar.gz b/src/content/en/blog/2020/10/05/swift2nix.tar.gz Binary files differnew file mode 100644 index 0000000..a22aaa0 --- /dev/null +++ b/src/content/en/blog/2020/10/05/swift2nix.tar.gz diff --git a/src/content/en/blog/2020/10/19/feature-flags.adoc b/src/content/en/blog/2020/10/19/feature-flags.adoc new file mode 100644 index 0000000..8788407 --- /dev/null +++ b/src/content/en/blog/2020/10/19/feature-flags.adoc @@ -0,0 +1,306 @@ += Feature flags: differences between backend, frontend and mobile +:categories: presentation +:updatedat: 2020-11-03 + +:empty: +:slides: link:../../../../slide/2020/10/19/feature-flags.pdf +:fowler-article: https://martinfowler.com/articles/feature-toggles.html + +_This article is derived from a {slides}[presentation] on the same subject._ + +When discussing about feature flags, I find that their costs and benefits are +often well exposed and addressed. Online articles like +"{fowler-article}[Feature Toggle (aka Feature Flags)]" do a great job of +explaining them in detail, giving great general guidance of how to apply +techniques to adopt it. + +However the weight of those costs and benefits apply differently on backend, +frontend or mobile, and those differences aren't covered. In fact, many of them +stop making sense, or the decision of adopting a feature flag or not may change +depending on the environment. + +In this article I try to make the distinction between environments and how +feature flags apply to them, with some final best practices I've acquired when +using them in production. + +== Why feature flags + +:atlassian-cicd: https://www.atlassian.com/continuous-delivery/principles/continuous-integration-vs-delivery-vs-deployment + +Feature flags in general tend to be cited on the context of +{atlassian-cicd}[continuous deployment]: + +____ +A: With continuous deployment, you deploy to production automatically + +B: But how do I handle deployment failures, partial features, _etc._? + +A: With techniques like canary, monitoring and alarms, feature flags, _etc._ +____ + +Though adopting continuous deployment doesn't force you to use feature flags, it +creates a demand for it. The inverse is also true: using feature flags on the +code points you more obviously to continuous deployment. Take the following +code sample for example, that we will reference later on the article: + +[source,javascript] +---- +function processTransaction() { + validate(); + persist(); + // TODO: add call to notifyListeners() +} +---- + +While being developed, being tested for suitability or something similar, +`notifyListeners()` may not be included in the code at once. So instead of +keeping it on a separate, long-lived branch, a feature flag can decide when the +new, partially implemented function will be called: + +[source,javascript] +---- +function processTransaction() { + validate(); + persist(); + if (featureIsEnabled("activate-notify-listeners")) { + notifyListeners(); + } +} +---- + +This allows your code to include `notifyListeners()`, and decide when to call it +at runtime. For the price of extra things around the code, you get more +dynamicity. + +So the fundamental question to ask yourself when considering adding a feature +flag should be: + +____ +Am I willing to pay with code complexity to get dynamicity? +____ + +It is true that you can make the management of feature flags as straightforward +as possible, but having no feature flags is simpler than having any. What you +get in return is the ability to parameterize the behaviour of the application at +runtime, without doing any code changes. + +Sometimes this added complexity may tilt the balance towards not using a feature +flag, and sometimes the flexibility of changing behaviour at runtime is +absolutely worth the added complexity. This can vary a lot by code base, +feature, but fundamentally by environment: its much cheaper to deploy a new +version of a service than to release a new version of an app. + +So the question of which environment is being targeted is key when reasoning +about costs and benefits of feature flags. + +== Control over the environment + +:fdroid: https://f-droid.org/ +:bad-apple: https://www.paulgraham.com/apple.html + +The key differentiator that makes the trade-offs apply differently is how much +control you have over the environment. + +When running a *backend* service, you usually are paying for the servers +themselves, and can tweak them as you wish. This means you have full control do +to code changes as you wish. Not only that, you decide when to do it, and for +how long the transition will last. + +On the *frontend* you have less control: even though you can choose to make a +new version available any time you wish, you can't +force{empy}footnote:force[ + Technically you could force a reload with JavaScript using + `window.location.reload()`, but that not only is invasive and impolite, but + also gives you the illusion that you have control over the client when you + actually don't: clients with disabled JavaScript would be immune to such + tactics. +] clients to immediately switch to the new version. That means that a) clients +could skip upgrades at any time and b) you always have to keep backward and +forward compatibility in mind. + +Even though I'm mentioning frontend directly, it applies to other environment +with similar characteristics: desktop applications, command-line programs, +_etc_. + +On *mobile* you have even less control: app stores need to allow your app to be +updated, which could bite you when least desired. Theoretically you could make +you APK available on third party stores like {fdroid}[F-Droid], or even make the +APK itself available for direct download, which would give you the same +characteristics of a frontend application, but that happens less often. + +On iOS you can't even do that. You have to get Apple's blessing on every single +update. Even though we already know that is a {bad-apple}[bad idea] for over a +decade now, there isn't a way around it. This is where you have the least +control. + +In practice, the amount of control you have will change how much you value +dynamicity: the less control you have, the more valuable it is. In other words, +having a dynamic flag on the backend may or may not be worth it since you could +always update the code immediately after, but on iOS it is basically always +worth it. + +== Rollout + +:kubernetes-deployment: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#creating-a-deployment +:play-store-rollout: https://support.google.com/googleplay/android-developer/answer/6346149?hl=en +:app-store-rolllout: https://help.apple.com/app-store-connect/#/dev3d65fcee1 + +A rollout is used to _roll out_ a new version of software. + +They are usually short-lived, being relevant as long as the new code is being +deployed. The most common rule is percentages. + +On the *backend*, it is common to find it on the deployment infrastructure +itself, like canary servers, blue/green deployments, {kubernetes-deployment}[a +kubernetes deployment rollout], _etc_. You could do those manually, by having a +dynamic control on the code itself, but rollbacks are cheap enough that people +usually do a normal deployment and just give some extra attention to the metrics +dashboard. + +Any time you see a blue/green deployment, there is a rollout happening: most +likely a load balancer is starting to direct traffic to the new server, until +reaching 100% of the traffic. Effectively, that is a rollout. + +On the *frontend*, you can selectively pick which user's will be able to +download the new version of a page. You could use geographical region, IP, +cookie or something similar to make this decision. + +CDN propagation delays and people not refreshing their web pages are also +rollouts by themselves, since old and new versions of the software will coexist. + +On *mobile*, the Play Store allows you to perform fine-grained +{play-store-rollout}[staged rollouts], and the App Store allows you to perform +limited {app-store-rollout}[phased releases]. + +Both for Android and iOS, the user plays the role of making the download. + +In summary: since you control the servers on the backend, you can do rollouts at +will, and those are often found automated away in base infrastructure. On the +frontend and on mobile, there are ways to make new versions available, but users +may not download them immediately, and many different versions of the software +end up coexisting. + +== Feature flag + +A feature flag is a _flag_ that tells the application on runtime to turn on or +off a given _feature_. That means that the actual production code will have +more than one possible code paths to go through, and that a new version of a +feature coexists with the old version. The feature flag tells which part of the +code to go through. + +They are usually medium-lived, being relevant as long as the new code is being +developed. The most common rules are percentages, allow/deny lists, A/B groups +and client version. + +On the *backend*, those are useful for things that have a long development +cycle, or that needs to done by steps. Consider loading the feature flag rules +in memory when the application starts, so that you avoid querying a database or +an external service for applying a feature flag rule and avoid flakiness on the +result due to intermittent network failures. + +Since on the *frontend* you don't control when to update the client software, +you're left with applying the feature flag rule on the server, and exposing the +value through an API for maximum dynamicity. This could be in the frontend code +itself, and fallback to a "just refresh the page"/"just update to the latest +version" strategy for less dynamic scenarios. + +On *mobile* you can't even rely on a "just update to the latest version" +strategy, since the code for the app could be updated to a new feature and be +blocked on the store. Those cases aren't recurrent, but you should always +assume the store will deny updates on critical moments so you don't find +yourself with no cards to play. That means the only control you actually have +is via the backend, by parameterizing the runtime of the application using the +API. In practice, you should always have a feature flag to control any relevant +piece of code. There is no such thing as "too small code change for a feature +flag". What you should ask yourself is: + +____ +If the code I'm writing breaks and stays broken for around a month, do I care? +____ + +If you're doing an experimental screen, or something that will have a very small +impact you might answer "no" to the above question. For everything else, the +answer will be "yes": bug fixes, layout changes, refactoring, new screen, +filesystem/database changes, _etc_. + +== Experiment + +An experiment is a feature flag where you care about analytical value of the +flag, and how it might impact user's behaviour. A feature flag with analytics. + +They are also usually medium-lived, being relevant as long as the new code is +being developed. The most common rule is A/B test. + +On the *backend*, an experiment rely on an analytical environment that will pick +the A/B test groups and distributions, which means those can't be held in memory +easily. That also means that you'll need a fallback value in case fetching the +group for a given customer fails. + +On the *frontend* and on *mobile* they are no different from feature flags. + +== Operational toggle + +An operational toggle is like a system-level manual circuit breaker, where you +turn on/off a feature, fail over the load to a different server, _etc_. They +are useful switches to have during an incident. + +They are usually long-lived, being relevant as long as the code is in +production. The most common rule is percentages. + +They can be feature flags that are promoted to operational toggles on the +*backend*, or may be purposefully put in place preventively or after a +postmortem analysis. + +On the *frontend* and on *mobile* they are similar to feature flags, where the +"feature" is being turned on and off, and the client interprets this value to +show if the "feature" is available or unavailable. + +== Best practices + +=== Prefer dynamic content + +Even though feature flags give you more dynamicity, they're still somewhat +manual: you have to create one for a specific feature and change it by hand. + +If you find yourself manually updating a feature flags every other day, or +tweaking the percentages frequently, consider making it fully dynamic. Try +using a dataset that is generated automatically, or computing the content on the +fly. + +Say you have a configuration screen with a list of options and sub-options, and +you're trying to find how to better structure this list. Instead of using a +feature flag for switching between 3 and 5 options, make it fully dynamic. This +way you'll be able to perform other tests that you didn't plan, and get more +flexibility out of it. + +=== Use the client version to negotiate feature flags + +After effectively finishing a feature, the old code that coexisted with the new +one will be deleted, and all traces of the transition will vanish from the code +base. However if you just remove the feature flags from the API, all of the old +versions of clients that relied on that value to show the new feature will go +downgrade to the old feature. + +This means that you should avoid deleting client-facing feature flags, and +retire them instead: use the client version to decide when the feature is +stable, and return `true` for every client with a version greater or equal to +that. This way you can stop thinking about the feature flag, and you don't +break or downgrade clients that didn't upgrade past the transition. + +=== Beware of many nested feature flags + +Nested flags combine exponentially. + +Pick strategic entry points or transitions eligible for feature flags, and +beware of their nesting. + +=== Include feature flags in the development workflow + +Add feature flags to the list of things to think about during whiteboarding, and +deleting/retiring a feature flags at the end of the development. + +=== Always rely on a feature flag on the app + +Again, there is no such thing "too small for a feature flag". Too many feature +flags is a good problem to have, not the opposite. Automate the process of +creating a feature flag to lower its cost. diff --git a/src/content/en/blog/2020/10/20/wrong-interviewing.adoc b/src/content/en/blog/2020/10/20/wrong-interviewing.adoc new file mode 100644 index 0000000..4b8d855 --- /dev/null +++ b/src/content/en/blog/2020/10/20/wrong-interviewing.adoc @@ -0,0 +1,340 @@ += How not to interview engineers +:updatedat: 2020-10-24 + +:bad-article: https://defmacro.substack.com/p/how-to-interview-engineers +:satire-comment: https://defmacro.substack.com/p/how-to-interview-engineers/comments#comment-599996 +:double-down: https://twitter.com/spakhm/status/1315754730740617216 +:poes-law: https://en.wikipedia.org/wiki/Poe%27s_law +:hn-comment-1: https://news.ycombinator.com/item?id=24757511 + +This is a response to Slava's "{bad-article}[How to interview engineers]" +article. I initially thought it was a satire, {satire-comment}[as have others], +but he has [doubled down on it]: + +____ +(...) Some parts are slightly exaggerated for sure, but the essay isn't meant as +a joke. +____ + +That being true, he completely misses the point on how to improve hiring, and +proposes a worse alternative on many aspects. It doesn't qualify as +provocative, it is just wrong. + +I was comfortable taking it as a satire, and I would just ignore the whole thing +if it wasn't (except for the technical memo part), but friends of mine +considered it to be somewhat reasonable. This is a adapted version of parts of +the discussions we had, risking becoming a gigantic showcase of {poes-law}[Poe's +law]. + +In this piece, I will argument against his view, and propose an alternative +approach to improve hiring. + +It is common to find people saying how broken technical hiring is, as well put +in words by a phrase on {hn-comment-1}[this comment]: + +____ +Everyone loves to read and write about how developer interviewing is flawed, but +no one wants to go out on a limb and make suggestions about how to improve it. +____ + +I guess Slava was trying to not fall on this trap, and make a suggestion on how +to improve instead, which all went terribly wrong. + +== What not to do + +=== Time candidates + +:hammock-driven-talk: https://www.youtube.com/watch?v=f84n5oFoZBc + +Timing the candidate shows up on the "talent" and "judgment" sections, and they +are both bad ideas for the same reason: programming is not a performance. + +What do e-sports, musicians, actors and athletes have in common: performance +psychologists. + +For a pianist, their state of mind during concerts is crucial: they not only +must be able to deal with stage anxiety, but to become really successful they +will have to learn how to exploit it. The time window of the concert is what +people practice thousands of hours for, and it is what defines one's career, +since how well all the practice went is irrelevant to the nature of the +profession. Being able to leverage stage anxiety is an actual goal of them. + +That is also applicable to athletes, where the execution during a competition +makes them sink or swim, regardless of how all the training was. + +The same cannot be said about composers, though. They are more like book +writers, where the value is not on very few moments with high adrenaline, but on +the aggregate over hours, days, weeks, months and years. A composer may have a +deadline to finish a song in five weeks, but it doesn't really matter if it is +done on a single night, every morning between 6 and 9, at the very last week, or +any other way. No rigid time structure applies, only whatever fits best to the +composer. + +Programming is more like composing than doing a concert, which is another way of +saying that programming is not a performance. People don't practice algorithms +for months to keep them at their fingertips, so that finally in a single +afternoon they can sit down and write everything at once in a rigid 4 hours +window, and launch it immediately after. + +Instead software is built iteratively, by making small additions, than +refactoring the implementation, fixing bugs, writing a lot at once, _etc_. all +while they get a firmer grasp of the problem, stop to think about it, come up +with new ideas, _etc_. + +Some specifically plan for including spaced pauses, and call it +"{hammock-driven-talk}[Hammock Driven Development]", which is just artist's +"creative idleness" for hackers. + +Unless you're hiring for a live coding group, a competitive programming team, or +a professional live demoer, timing the candidate that way is more harmful than +useful. This type of timing doesn't find good programmers, it finds performant +programmers, which isn't the same thing, and you'll end up with people who can +do great work on small problems but who might be unable to deal with big +problems, and loose those who can very well handle huge problems, slowly. If +you are lucky you'll get performant people who can also handle big problems on +the long term, but maybe not. + +An incident is the closest to a "performance" that it gets, and yet it is still +dramatically different. Surely it is a high stress scenario, but while people +are trying to find a root cause and solve the problem, only the downtime itself +is visible to the exterior. It is like being part of the support staff +backstage during a play: even though execution matters, you're still not on the +spot. During an incident you're doing debugging in anger rather than live +coding. + +Although giving a candidate the task to write a "technical memo" has potential +to get a measure of the written communication skills of someone, doing so in a +hard time window also misses the point for the same reasons. + +=== Pay attention to typing speed + +:dijkstra-typing: https://www.cs.utexas.edu/users/EWD/transcriptions/EWD05xx/EWD512.html +:speech-to-text: https://www.youtube.com/watch?v=Mz3JeYfBTcY +:j-lang: https://www.jsoftware.com/#/ + +Typing is speed in never the bottleneck of a programmer, no matter how great +they are. + +As {dijkstra-typing}[Dijkstra said]: + +____ +But programming, when stripped of all its circumstantial irrelevancies, boils +down to no more and no less than very effective thinking so as to avoid +unmastered complexity, to very vigorous separation of your many different +concerns. +____ + +In other words, programming is not about typing, it is about thinking. + +Otherwise, the way to get those star programmers that can't type fast enough a +huge productivity boost is to give them a touch typing course. If they are so +productive with typing speed being a limitation, imagine what they could +accomplish if they had razor sharp touch typing skills? + +Also, why stop there? A good touch typist can do 90 WPM (words per minute), and +a great one can do 120 WPM, but with a stenography keyboard they get to 200 +WPM+. That is double the productivity! Why not try +{speech-to-text}[speech-to-text]? Make them all use {j-lang}[J] so they all +need to type less! How come nobody thought of that? + +And if someone couldn't solve the programming puzzle in the given time window, +but could come back in the following day with an implementation that is not only +faster, but uses less memory, was simpler to understand and easier to read than +anybody else? You'd be losing that person too. + +=== IQ + +:determination-article: https://www.paulgraham.com/determination.html +:scihub-article: https://sci-hub.do/https://psycnet.apa.org/doiLanding?doi=10.1037%2F1076-8971.6.1.33 + +For "building an extraordinary team at a hard technology startup", +intelligence is not the most important, +{determination-article}[determination is]. + +And talent isn't "IQ specialized for engineers". IQ itself isn't a measure of +how intelligent someone is. Ever since Alfred Binet with Théodore Simon started +to formalize what would become IQ tests years later, they already acknowledged +limitations of the technique for measuring intelligence, which is +{scihub-article}[still true today]. + +So having a high IQ tells only how smart people are for a particular aspect of +intelligence, which is not representative of programming. There are numerous +aspects of programming that are covered by IQ measurement: how to name variables +and functions, how to create models which are compatible with schema evolution, +how to make the system dynamic for runtime parameterization without making it +fragile, how to measure and observe performance and availability, how to pick +between acquiring and paying technical debt, _etc_. + +Not to say about everything else that a programmer does that is not purely +programming. Saying high IQ correlates with great programming is a stretch, at +best. + +=== Ditch HR + +Slava tangentially picks on HR, and I will digress on that a bit: + +____ +A good rule of thumb is that if a question could be asked by an intern in HR, +it's a non-differential signaling question. +____ + +Stretching it, this is a rather snobbish view of HR. Why is it that an intern +in HR can't make signaling questions? Could the same be said of an intern in +engineering? + +In other words: is the question not signaling because the one asking is from HR, +or because the one asking is an intern? If the latter, than he's just arguing +that interns have no place in interviewing, but if the former than he was +picking on HR. + +Extrapolating that, it is common to find people who don't value HR's work, and +only see them as inferiors doing unpleasant work, and who aren't capable enough +(or _smart_ enough) to learn programming. + +This is equivalent to people who work primarily on backend, and see others +working on frontend struggling and say: "isn't it just building views and +showing them on the browser? How could it possibly be that hard? I bet I could +do it better, with 20% of code". As you already know, the answer to it is +"well, why don't you go do it, then?". + +This sense of superiority ignores the fact that HR have actual professionals +doing actual hard work, not unlike programmers. If HR is inferior and so easy, +why not automate everything away and get rid of a whole department? + +I don't attribute this world view to Slava, this is only an extrapolation of a +snippet of the article. + +=== Draconian mistreating of candidates + +:bad-apple: https://www.paulgraham.com/apple.html +:be-good: https://www.paulgraham.com/good.html + +If I found out that people employed theatrics in my interview so that I could +feel I've "earned the privilege to work at your company", I would quit. + +If your moral compass is so broken that you are comfortable mistreating me while +I'm a candidate, I immediately assume you will also mistreat me as an employee, +and that the company is not a good place to work, as {bad-apple}[evil begets +stupidity]: + +____ +But the other reason programmers are fussy, I think, is that evil begets +stupidity. An organization that wins by exercising power starts to lose the +ability to win by doing better work. And it's not fun for a smart person to +work in a place where the best ideas aren't the ones that win. I think the +reason Google embraced "Don't be evil" so eagerly was not so much to impress the +outside world as to inoculate themselves against arrogance. +____ + +Paul Graham goes beyond "don't be evil" with a better motto: +"{be-good}[be good]". + +Abusing the asymmetric nature of an interview to increase the chance that the +candidate will accept the offer is, well, abusive. I doubt a solid team can +actually be built on such poor foundations, surrounded by such evil measures. + +And if you really want to give engineers "the measure of whoever they're going +to be working with", there are plenty of reasonable ways of doing it that don't +include performing fake interviews. + +=== Personality tests + +Personality tests around the world need to be a) translated, b) adapted and c) +validated. Even though a given test may be applicable and useful in a country, +this doesn't imply it will work for other countries. + +Not only tests usually come with translation guidelines, but also its +applicability needs to be validated again after the translation and adaptation +is done to see if the test still measures what it is supposed to. + +That is also true within the same language. If a test is shown to work in +England, it may not work in New Zealand, in spite of both speaking english. The +cultural context difference is influent to the point of invalidating a test and +making it be no longer valid. + +Irregardless of the validity of the proposed "big five" personality test, saying +"just use attributes x, y and z this test and you'll be fine" is a rough +simplification, much like saying "just use Raft for distributed systems, after +all it has been proven to work" shows he throws all of that background away. + +So much as applying personality tests themselves is not a trivial task, and +psychologists do need special training to become able to effectively apply one. + +=== More cargo culting + +:cult: https://calteches.library.caltech.edu/51/2/CargoCult.htm +:cult-archived: https://web.archive.org/web/20201003090303/https://calteches.library.caltech.edu/51/2/CargoCult.htm + +He calls the ill-defined "industry standard" to be cargo-culting, but his +proposal isn't sound enough to not become one. + +Even if the ideas were good, they aren't solid enough, or based on solid enough +things to make them stand out by themselves. Why is it that talent, judgment +and personality are required to determine the fitness of a good candidate? Why +not 2, 5, or 20 things? Why those specific 3? Why is talent defined like that? +Is it just because he found talent to be like that? + +Isn't that definitionally also +{cult}[cargo-culting]footnote:cargo-cult[ + {cult-archived}[Archived version]. +]? Isn't he just repeating whatever he found to work form him, without +understanding why? + +What Feynman proposes is actually the opposite: + +____ +In summary, the idea is to try to give *all* of the information to help others +to judge the value of your contribution; not just the information that leads to +judgment in one particular direction or another. +____ + +What Slava did was just another form of cargo culting, but this was one that he +believed to work. + +== What to do + +I will not give you a list of things that "worked for me, thus they are +correct". I won't either critique the current "industry standard", nor what +I've learned from interviewing engineers. + +Instead, I'd like to invite you to learn from history, and from what other +professionals have to teach us. + +Programming isn't an odd profession, where everything about it is different from +anything else. It is just another episode in the "technology" series, which has +seasons since before recorded history. It may be an episode where things move a +bit faster, but it is fundamentally the same. + +So here is the key idea: what people did _before_ software engineering? + +What hiring is like for engineers in other areas? Don't civil, electrical and +other types of engineering exist for much, much longer than software engineering +does? What have those centuries of accumulated experience thought the world +about technical hiring? + +What studies were performed on the different success rate of interviewing +strategies? What have they done right and what have they done wrong? + +What is the purpose of HR? Why do they even exist? Do we need them, and if so, +what for? What is the value they bring, since everybody insist on building an +HR department in their companies? Is the existence of HR another form of cargo +culting? + +What is industrial and organizational psychology? What is that field of study? +What do they specialize in? What have they learned since the discipline +appeared? What have they done right and wrong over history? Is is the current +academic consensus on that area? What is a hot debate topic in academia on that +area? What is the current bleeding edge of research? What can they teach us +about hiring? What can they teach us about technical hiring? + +== Conclusion + +If all I've said makes me a "no hire" in the proposed framework, I'm really +glad. + +This says less about my programming skills, and more about the employer's world +view, and I hope not to be fooled into applying for a company that adopts this +one. + +Claiming to be selecting "extraordinary engineers" isn't an excuse to reinvent +the wheel, poorly. diff --git a/src/content/en/blog/2020/11/07/diy-bugs.adoc b/src/content/en/blog/2020/11/07/diy-bugs.adoc new file mode 100644 index 0000000..5fbc920 --- /dev/null +++ b/src/content/en/blog/2020/11/07/diy-bugs.adoc @@ -0,0 +1,93 @@ += DIY an offline bug tracker with text files, Git and email +:updatedat: 2021-08-14 + +:attack-on-ytdl: https://github.com/github/dmca/blob/master/2020/10/2020-10-23-RIAA.md +:list-discussions: https://sourcehut.org/blog/2020-10-29-how-mailing-lists-prevent-censorship/ +:docs-in-repo: https://podcast.writethedocs.org/2017/01/25/episode-3-trends/ +:ci-in-notes: link:../../../../til/2020/11/30/git-notes-ci.html +:todos-mui: https://man.sr.ht/todo.sr.ht/#email-access +:git-bug-bridges: https://github.com/MichaelMure/git-bug#bridges + +When {attack-on-ytdl}[push comes to shove], the operational aspects of +governance of a software project matter a lot. And everybody likes to chime in +with their alternative of how to avoid single points of failure in project +governance, just like I'm doing right now. + +The most valuable assets of a project are: + +. source code +. discussions +. documentation +. builds +. tasks and bugs + +For *source code*, Git and other DVCS solve that already: everybody gets a full +copy of the entire source code. + +If your code forge is compromised, moving it to a new one takes a couple of +minutes, if there isn't a secondary remote serving as mirror already. In this +case, no action is required. + +If you're having your *discussions* by email, "{list-discussions}[taking this +archive somewhere else and carrying on is effortless]". + +Besides, make sure to backup archives of past discussions so that the history is +also preserved when this migration happens. + +The *documentation* should {docs-in-repo}[live inside the repository +itself]footnote:writethedocs-in-repo[ + Described as "the ultimate marriage of the two". Starts at time 31:50. +], so that not only it gets first class treatment, but also gets distributed to +everybody too. Migrating the code to a new forge already migrates the +documentation with it. + +As long as you keep the *builds* vendor neutral, the migration should only +involve adapting how you call your `tests.sh` from the format of +`provider-1.yml` uses to the format that `provider-2.yml` accepts. It isn't +valuable to carry the build history with the project, as this data quickly +decays in value as weeks and months go by, but for simple text logs +{ci-in-notes}[using Git notes] may be just enough, and they would be replicated +with the rest of the repository. + +But for *tasks and bugs* many rely on a vendor-specific service, where +you register and manage those issues via a web browser. Some provide an +{todos-mui}[interface for interacting via email] or an API for +{git-bug-bridges[bridging local bugs with vendor-specific services]. But +they're all layers around the service, that disguises it as being a central +point of failure, which when compromised would lead to data loss. When push +comes to shove, you'd loose data. + +== Alternative: text files, Git and email + +:todos-example: https://euandre.org/git/remembering/tree/TODOs.md?id=3f727802cb73ab7aa139ca52e729fd106ea916d0 +:todos-script: https://euandre.org/git/remembering/tree/aux/workflow/TODOs.sh?id=3f727802cb73ab7aa139ca52e729fd106ea916d0 +:todos-html: https://euandreh.xyz/remembering/TODOs.html +:fossil-tickets: https://fossil-scm.org/home/doc/trunk/www/bugtheory.wiki + +Why not do the same as documentation, and move tasks and bugs into the +repository itself? + +It requires no extra tool to be installed, and fits right in the already +existing workflow for source code and documentation. + +I like to keep a {todos-example}[`TODOs.md`] file at the repository top-level, +with two relevant sections: "tasks" and "bugs". Then when building the +documentation I'll just {todos-script}[generate an HTML file from it], and +{todos-html}[publish] it alongside the static website. All that is done on the +main branch. + +Any issues discussions are done in the mailing list, and a reference to a +discussion could be added to the ticket itself later on. External contributors +can file tickets by sending a patch. + +The good thing about this solution is that it works for 99% of projects out +there. + +For the other 1%, having Fossil's "{fossil-tickets}[tickets]" could be an +alternative, but you may not want to migrate your project to Fossil to get those +niceties. + +Even though I keep a `TODOs.md` file on the main branch, you can have a `tasks` +branch with a `task-n.md` file for each task, or any other way you like. + +These tools are familiar enough that you can adjust it to fit your workflow. diff --git a/src/content/en/blog/2020/11/08/paradigm-shift-review.adoc b/src/content/en/blog/2020/11/08/paradigm-shift-review.adoc new file mode 100644 index 0000000..1110085 --- /dev/null +++ b/src/content/en/blog/2020/11/08/paradigm-shift-review.adoc @@ -0,0 +1,154 @@ += The Next Paradigm Shift in Programming - video review +:categories: video-review + +:reviewed-video: https://www.youtube.com/watch?v=6YbK8o9rZfI + +This is a review with comments of "{reviewed-video}[The Next Paradigm Shift in +Programming]", by Richard Feldman. + +This video was _strongly_ suggested to me by a colleague. I wanted to discuss +it with her, and when drafting my response I figured I could publish it publicly +instead. + +Before anything else, let me just be clear: I really like the talk, and I think +Richard is a great public speaker. I've watched several of his talks over the +years, and I feel I've followed his career at a distance, with much respect. +This isn't a piece criticizing him personally, and I agree with almost +everything he said. These are just some comments but also nitpicks on a few +topics I think he missed, or that I view differently. + +== Structured programming + +:forgotten-art-video: https://www.youtube.com/watch?v=SFv8Wm2HdNM + +The historical overview at the beginning is very good. In fact, the very video +I watched previously was about structured programming! + +Kevlin Henney on "{forgotten-art-video}[The Forgotten Art of Structured +Programming]" does a deep-dive on the topic of structured programming, and how +on his view it is still hidden in our code, when we do a `continue` or a `break` +in some ways. Even though it is less common to see an explicit `goto` in code +these days, many of the original arguments of Dijkstra against explicit `goto`s +is applicable to other constructs, too. + +This is a very mature view, and I like how he goes beyond the "don't use +`goto`s" heuristic and proposes and a much more nuanced understanding of what +"structured programming" means. + +In a few minutes, Richard is able to condense most of the significant bits of +Kevlin's talk in a didactical way. Good job. + +== OOP like a distributed system + +:joe-oop: https://www.infoq.com/interviews/johnson-armstrong-oop/ +:rich-hickey-oop: https://www.youtube.com/watch?v=ROor6_NGIWU + +Richard extrapolates Alan Kay's original vision of OOP, and he concludes that it +is more like a distributed system that how people think about OOP these days. +But he then states that this is a rather bad idea, and we shouldn't pursue it, +given that distributed systems are known to be hard. + +However, his extrapolation isn't really impossible, bad or an absurd. In fact, +it has been followed through by Erlang. Joe Armstrong used to say that +"{joe-oop}[Erlang might the only OOP language]", since it actually adopted this +paradigm. + +But Erlang is a functional language. So this "OOP as a distributed system" view +is more about designing systems in the large than programs in the small. + +There is a switch of levels in this comparison I'm making, as can be done with +any language or paradigm: you can have a functional-like system that is built +with an OOP language (like a compiler, that given the same input will produce +the same output), or an OOP-like system that is built with a functional +language (Rich Hickey calls it "{rich-hickey-oop}[OOP in the +large]"footnote:langsys[ + From 24:05 to 27:45. +]). + +So this jump from in-process paradigm to distributed paradigm is rather a big +one, and I don't think you he can argue that OOP has anything to say about +software distribution across nodes. You can still have Erlang actors that run +independently and send messages to each other without a network between them. +Any OTP application deployed on a single node effectively works like that. + +I think he went a bit too far with this extrapolation. Even though I agree it +is a logical a fair one, it isn't evidently bad as he painted. I would be fine +working with a single-node OTP application and seeing someone call it "a _real_ +OOP program". + +== First class immutability + +:immer: https://sinusoid.es/immer/ +:immutable-js: https://immutable-js.github.io/immutable-js/ + +I agree with his view of languages moving towards the functional paradigm. But +I think you can narrow down the "first-class immutability" feature he points out +as present on modern functional programming languages to "first-class immutable +data structures". + +I wouldn't categorize a language as "supporting functional programming style" +without a library for functional data structures it. By discipline you can +avoid side-effects, write pure functions as much as possible, and pass functions +as arguments around is almost every language these days, but if when changing an +element of a vector mutates things in-place, that is still not functional +programming. + +To avoid that, you end-up needing to make clones of objects to pass to a +function, using freezes or other workarounds. All those cases are when the +underlying mix of OOP and functional programming fail. + +There are some languages with third-party libraries that provide functional data +structures, like {immer}[immer] for C++, or {immutable-js}[ImmutableJS] for +JavaScript. + +But functional programming is more easily achievable in languages that have them +built-in, like Erlang, Elm and Clojure. + +== Managed side-effects + +:redux: https://redux.js.org/ +:re-frame: https://github.com/Day8/re-frame + +His proposal of adopting managed side-effects as a first-class language concept +is really intriguing. + +This is something you can achieve with a library, like {redux}[Redux] for +JavaScript or {re-frame}[re-frame] for Clojure. + +I haven't worked with a language with managed side-effects at scale, and I don't +feel this is a problem with Clojure or Erlang. But is this me finding a flaw in +his argument or not acknowledging a benefit unknown to me? This is a +provocative question I ask myself. + +Also all FP languages with managed side-effects I know are statically-typed, and +all dynamically-typed FP languages I know don't have managed side-effects baked +in. + +== What about declarative programming? + +:tarpit-article: https://curtclifton.net/papers/MoseleyMarks06a.pdf + +In "{tarpit-article}[Out of the Tar Pit]", B. Moseley and P. Marks go beyond his +view of functional programming as the basis, and name a possible "functional +relational programming" as an even better solution. They explicitly call out +some flaws in most of the modern functional programming languages, and instead +pick declarative programming as an even better starting paradigm. + +If the next paradigm shift is towards functional programming, will the following +shift be towards declarative programming? + +== Conclusion + +:simple-made-easy: https://www.infoq.com/presentations/Simple-Made-Easy/ + +Beyond all Richard said, I also hear often bring up functional programming when +talking about utilizing all cores of a computer, and how FP can help with that. + +Rich Hickey makes a great case for single-process FP on his famous talk +"{simple-made-easy}[Simple Made Easy]". + +//// +I find this conclusion too short, and it doesn't revisits the main points +presented on the body of the article. I won't rewrite it now, but it would be +an improvement to extend it to do so. +//// diff --git a/src/content/en/blog/2020/11/12/database-parsers-trees.adoc b/src/content/en/blog/2020/11/12/database-parsers-trees.adoc new file mode 100644 index 0000000..47595e8 --- /dev/null +++ b/src/content/en/blog/2020/11/12/database-parsers-trees.adoc @@ -0,0 +1,226 @@ += Durable persistent trees and parser combinators - building a database +:categories: mediator +:updatedat: 2021-02-09 + +:empty: +:db-article: link:../../08/31/database-i-wish-i-had.html + +I've received with certain frequency messages from people wanting to know if +I've made any progress on the database project {db-article}[I've written about]. + +There are a few areas where I've made progress, and here's a public post on it. + +== Proof-of-concept: DAG log + +:mediator-permalink: https://euandre.org/git/mediator/tree/src/core/clojure/src/mediator.clj?id=db4a727bc24b54b50158827b34502de21dbf8948#n1 + +The main thing I wanted to validate with a concrete implementation was the +concept of modeling a DAG on a sequence of datoms. + +The notion of a _datom_ is a rip-off from Datomic, which models data with time +aware _facts_, which come from RDF. RDF's fact is a triple of +subject-predicate-object, and Datomic's datoms add a time component to it: +subject-predicate-object-time, A.K.A. entity-attribute-value-transaction: + +[source,clojure] +---- +[[person :likes "pizza" 0 true] + [person :likes "bread" 1 true] + [person :likes "pizza" 1 false]] +---- + +The above datoms say: - at time 0, `person` like pizza; - at time 1, `person` +stopped liking pizza, and started to like bread. + +Datomic ensures total consistency of this ever growing log by having a single +writer, the transactor, that will enforce it when writing. + +In order to support disconnected clients, I needed a way to allow multiple +writers, and I chose to do it by making the log not a list, but a directed +acyclic graph (DAG): + +[source,clojure] +---- +[[person :likes "pizza" 0 true] + [0 :parent :db/root 0 true] + [person :likes "bread" 1 true] + [person :likes "pizza" 1 false] + [1 :parent 0 1 true]] +---- + +The extra datoms above add more information to build the directionality to the +log, and instead of a single consistent log, the DAG could have multiple leaves +that coexist, much like how different Git branches can have different "latest" +commits. + +In order to validate this idea, I started with a Clojure implementation. The +goal was not to write the actual final code, but to make a proof-of-concept that +would allow me to test and stretch the idea itself. + +This code {mediator-permalink}[already exists], but is yet fairly incomplete: + +:commented-code: https://euandre.org/git/mediator/tree/src/core/clojure/src/mediator.clj?id=db4a727bc24b54b50158827b34502de21dbf8948#n295 +:more: https://euandre.org/git/mediator/tree/src/core/clojure/src/mediator.clj?id=db4a727bc24b54b50158827b34502de21dbf8948#n130 +:than: https://euandre.org/git/mediator/tree/src/core/clojure/src/mediator.clj?id=db4a727bc24b54b50158827b34502de21dbf8948#n146 +:one: https://euandre.org/git/mediator/tree/src/core/clojure/src/mediator.clj?id=db4a727bc24b54b50158827b34502de21dbf8948#n253 + +* the building of the index isn't done yet (with some {commented-code}[commented + code] on the next step to be implemented) +* the indexing is extremely inefficient, with {more}[more] {than}[than] + {one}[one] occurrence of `O²` functions; +* no query support yet. + +== Top-down _and_ bottom-up + +However, as time passed and I started looking at what the final implementation +would look like, I started to consider keeping the PoC around. + +The top-down approach (Clojure PoC) was in fact helping guide me with the +bottom-up, and I now have "promoted" the Clojure PoC into a "reference +implementation". It should now be a finished implementation that says what the +expected behaviour is, and the actual code should match the behaviour. + +The good thing about a reference implementation is that it has no performance of +resources boundary, so if it ends up being 1000× slower and using 500× more +memory, it should be find. The code can be also 10× or 100× simpler, too. + +== Top-down: durable persistent trees + +:pavlo-videos: https://www.youtube.com/playlist?list=PLSE8ODhjZXjbohkNBWQs_otTrBTrjyohi +:db-book: https://www.databass.dev/ + +In promoting the PoC into a reference implementation, this top-down approach now +needs to go beyond doing everything in memory, and the index data structure now +needs to be disk-based. + +Roughly speaking, most storage engines out there are based either on B-Trees or +LSM Trees, or some variations of those. + +But when building an immutable database, update-in-place B-Trees aren't an +option, as it doesn't accommodate keeping historical views of the tree. LSM +Trees may seem a better alternative, but duplication on the files with +compaction are also ways to delete old data which is indeed useful for a +historical view. + +I think the thing I'm after is a mix of a Copy-on-Write B-Tree, which would keep +historical versions with the write IO cost amortization of memtables of LSM +Trees. I don't know of any B-Tree variant out there that resembles this, so +I'll call it "Flushing Copy-on-Write B-Tree". + +I haven't written any code for this yet, so all I have is a high-level view of +what it will look like: + +. like Copy-on-Write B-Trees, changing a leaf involves creating a new leaf and + building a new path from root to the leaf. The upside is that writes a lock + free, and no coordination is needed between readers and writers, ever; +. the downside is that a single leaf update means at least `H` new nodes that + will have to be flushed to disk, where `H` is the height of the tree. To + avoid that, the writer creates these nodes exclusively on the in-memory + memtable, to avoid flushing to disk on every leaf update; +. a background job will consolidate the memtable data every time it hits X MB, + and persist it to disk, amortizing the cost of the Copy-on-Write B-Tree; +. readers than will have the extra job of getting the latest relevant + disk-resident value and merge it with the memtable data. + +The key difference to existing Copy-on-Write B-Trees is that the new trees are +only periodically written to disk, and the intermediate values are kept in +memory. Since no node is ever updated, the page utilization is maximum as it +doesn't need to keep space for future inserts and updates. + +And the key difference to existing LSM Trees is that no compaction is run: +intermediate values are still relevant as the database grows. So this leaves +out tombstones and value duplication done for write performance. + +One can delete intermediate index values to reclaim space, but no data is lost +on the process, only old B-Tree values. And if the database ever comes back to +that point (like when doing a historical query), the B-Tree will have to be +rebuilt from a previous value. After all, the database _is_ a set of datoms, +and everything else is just derived data. + +Right now I'm still reading about other data structures that storage engines +use, and I'll start implementing the "Flushing Copy-on-Write B-Tree" as I learn +more{empty}footnote:learn-more-db[ + If you are interested in learning more about this too, the very best two + resources on this subject are Andy Pavlo's "{pavlo-videos}[Intro to Database + Systems]" course and Alex Petrov's "{db-book}[Database Internals]" book. +] and mature it more. + +== Bottom-up: parser combinators and FFI + +:cbindgen: https://github.com/eqrion/cbindgen +:cbindgen-next: https://blog.eqrion.net/future-directions-for-cbindgen/ +:syn-crate: https://github.com/dtolnay/syn +:libedn: https://euandre.org/git/libedn/ + +I chose Rust as it has the best WebAssembly tooling support. + +My goal is not to build a Rust database, but a database that happens to be in +Rust. In order to reach client platforms, the primary API is the FFI one. + +I'm not very happy with current tools for exposing Rust code via FFI to the +external world: they either mix C with C++, which I don't want to do, or +provide no access to the intermediate representation of the FFI, which would be +useful for generating binding for any language that speaks FFI. + +I like better the path that the author of {cbindgen}[cbindgen] crate +{cbindgen-next}[proposes]: emitting an data representation of the Rust C API +(the author calls is a `ffi.json` file), and than building transformers from the +data representation to the target language. This way you could generate a C API +_and_ the node-ffi bindings for JavaScript automatically from the Rust code. + +So the first thing to be done before moving on is an FFI exporter that doesn't +mix C and C++, and generates said `ffi.json`, and than build a few transformers +that take this `ffi.json` and generate the language bindings, be it C, C++, +JavaScript, TypeScript, Kotlin, Swift, Dart, +_etc_footnote:ffi-langs[ + Those are, specifically, the languages I'm more interested on. My goal is + supporting client applications, and those languages are the most relevant for + doing so: C for GTK, C++ for Qt, JavaScript and TypeScript for Node.js and + browser, Kotlin for Android and Swing, Swift for iOS, and Dart for Flutter. +]. + +I think the best way to get there is by taking the existing code for cbindgen, +which uses the {syn-crate}[syn] crate to parse the Rust +code{empty}footnote:rust-syn[ + The fact that syn is an external crate to the Rust compiler points to a big + warning: procedural macros are not first class in Rust. They are just like + Babel plugins in JavaScript land, with the extra shortcoming that there is no + specification for the Rust syntax, unlike JavaScript. +pass:[</p><p>] + As flawed as this may be, it seems to be generally acceptable and adopted, + which works against building a solid ecosystem for Rust. +pass:[</p><p>] + The alternative that rust-ffi implements relies on internals of the Rust + compiler, which isn't actually worst, just less common and less accepted. +], and adapt it to emit the metadata. + +I've started a fork of cbindgen: +[line-through]#x-bindgen#{empty}footnote:x-bindgen[ + _EDIT_: now archived, the experimentation was fun. I've started to move more + towards C, so this effort became deprecated. +]. Right now it is just a copy of cbindgen verbatim, and I plan to remove all C +and C++ emitting code from it, and add a IR emitting code instead. + +When starting working on x-bindgen, I realized I didn't know what to look for in +a header file, as I haven't written any C code in many years. So as I was +writing {libedn}[libedn], I didn't know how to build a good C API to expose. So +I tried porting the code to C, and right now I'm working on building a _good_ C +API for a JSON parser using parser combinators: +[line-through]#ParsecC#{empty}footnote:parsecc[ + _EDIT_: now also archived. +]. + +After "finishing" ParsecC I'll have a good notion of what a good C API is, and +I'll have a better direction towards how to expose code from libedn to other +languages, and work on x-bindgen then. + +What both libedn and ParsecC are missing right now are proper error reporting, +and property-based testing for libedn. + +== Conclusion + +I've learned a lot already, and I feel the journey I'm on is worth going +through. + +If any of those topics interest you, message me to discuss more or contribute! +Patches welcome! diff --git a/src/content/en/blog/2020/11/14/local-first-review.adoc b/src/content/en/blog/2020/11/14/local-first-review.adoc new file mode 100644 index 0000000..2036069 --- /dev/null +++ b/src/content/en/blog/2020/11/14/local-first-review.adoc @@ -0,0 +1,305 @@ += Local-First Software: article review +:categories: presentation article-review + +:empty: +:presentation: link:../../../../slide/2020/11/14/local-first-hype.pdf +:reviewed-article: https://martin.kleppmann.com/papers/local-first.pdf + +_This article is derived from a {presentation}[presentation] given at a Papers +We Love meetup on the same subject._ + +This is a review of the article "{reviewed-article}[Local-First Software: You +Own Your Data, in spite of the Cloud]", by M. Kleppmann, A. Wiggins, P. Van +Hardenberg and M. F. McGranaghan. + +== Offline-first, local-first + +The "local-first" term they use isn't new, and I have used it myself in the past +to refer to this types of application, where the data lives primarily on the +client, and there are conflict resolution algorithms that reconcile data created +on different instances. + +Sometimes I see confusion with this idea and "client-side", "offline-friendly", +"syncable", etc. I have myself used this terms, also. + +There exists, however, already the "offline-first" term, which conveys almost +all of that meaning. In my view, "local-first" doesn't extend "offline-first" +in any aspect, rather it gives a well-defined meaning to it instead. I could +say that "local-first" is just "offline-first", but with 7 well-defined ideals +instead of community best practices. + +It is a step forward, and given the number of times I've seen the paper shared +around I think there's a chance people will prefer saying "local-first" in +_lieu_ of "offline-first" from now on. + +== Software licenses + +On a footnote of the 7th ideal ("You Retain Ultimate Ownership and Control"), +the authors say: + +____ +In our opinion, maintaining control and ownership of data does not mean that the +software must necessarily be open source. (...) as long as it does not +artificially restrict what users can do with their files. +____ + +They give examples of artificial restrictions, like this artificial restriction +I've come up with: + +[source,sh] +---- +#!/bin/sh + +TODAY=$(date +%s) +LICENSE_EXPIRATION=$(date -d 2020-11-15 +%s) + +if [ $TODAY -ge $LICENSE_EXPIRATION ]; then + echo 'License expired!' + exit 1 +fi + +echo $((2 + 2)) +---- + +Now when using this very useful program: + +[source,sh] +---- +# today +$ ./useful-adder.sh +4 +# tomorrow +$ ./useful-adder.sh +License expired! +---- + +This is obviously an intentional restriction, and it goes against the 5th ideal +("The Long Now"). This software would only be useful as long as the embedded +license expiration allowed. Sure you could change the clock on the computer, +but there are many other ways that this type of intentional restriction is in +conflict with that ideal. + +However, what about unintentional restrictions? What if a software had an equal +or similar restriction, and stopped working after days pass? Or what if the +programmer added a constant to make the development simpler, and this led to +unintentionally restricting the user? + +[source,sh] +---- +# today +$ useful-program +# ...useful output... + +# tomorrow, with more data +$ useful-program +ERROR: Panic! Stack overflow! +---- + +Just as easily as I can come up with ways to intentionally restrict users, I can +do the same for unintentionally restrictions. A program can stop working for a +variety of reasons. + +If it stops working due do, say, data growth, what are the options? Reverting +to an earlier backup, and making it read-only? That isn't really a "Long Now", +but rather a "Long Now as long as the software keeps working as expected". + +The point is: if the software isn't free, "The Long Now" isn't achievable +without a lot of wishful thinking. Maybe the authors were trying to be more +friendly towards business who don't like free software, but in doing so they've +proposed a contradiction by reconciling "The Long Now" with proprietary +software. + +It isn't the same as saying that any free software achieves that ideal, either. +The license can still be free, but the source code can become unavailable due to +cloud rot. Or maybe the build is undocumented, or the build tools had specific +configuration that one has to guess. A piece of free software can still fail to +achieve "The Long Now". Being free doesn't guarantee it, just makes it +possible. + +A colleague has challenged my view, arguing that the software doesn't really +need to be free, as long as there is an specification of the file format. This +way if the software stops working, the format can still be processed by other +programs. But this doesn't apply in practice: if you have a document that you +write to, and software stops working, you still want to write to the document. +An external tool that navigates the content and shows it to you won't allow you +to keep writing, and when it does that tool is now starting to re-implement the +software. + +An open specification could serve as a blueprint to other implementations, +making the data format more friendly to reverse-engineering. But the +re-implementation still has to exist, at which point the original software +failed to achieve "The Long Now". + +It is less bad, but still not quite there yet. + +== Denial of existing solutions + +:distgit: https://drewdevault.com/2018/07/23/Git-is-already-distributed.html + +When describing "Existing Data Storage and Sharing Models", on a +footnote{empty}footnote:devil[ + This is the second aspect that I'm picking on the article from a footnote. I + guess the devil really is on the details. +] the authors say: + +____ +In principle it is possible to collaborate without a repository service, e.g. by +sending patch files by email, but the majority of Git users rely on GitHub. +____ + +The authors go to a great length to talk about usability of cloud apps, and even +point to research they've done on it, but they've missed learning more from +local-first solutions that already exist. + +Say the automerge CRDT proves to be even more useful than what everybody +imagined. Say someone builds a local-first repository service using it. How +will it change anything of the Git/GitHub model? What is different about it +that prevents people in the future writing a paper saying: + +____ +In principle it is possible to collaborate without a repository service, e.g. by +using automerge and platform X, but the majority of Git users rely on GitHub. +____ + +How is this any better? + +If it is already {distgit}[possible] to have a local-first development workflow, +why don't people use it? Is it just fashion, or there's a fundamental problem +with it? If so, what is it, and how to avoid it? + +If sending patches by emails is perfectly possible but out of fashion, why even +talk about Git/GitHub? Isn't this a problem that people are putting themselves +in? How can CRDTs possibly prevent people from doing that? + +My impression is that the authors envision a better future, where development is +fully decentralized unlike today, and somehow CRDTs will make that happen. If +more people think this way, "CRDT" is next in line to the buzzword list that +solves everything, like "containers", "blockchain" or "machine learning". + +Rather than picturing an imaginary service that could be described like +"GitHub+CRDTs" and people would adopt it, I'd rather better understand why +people don't do it already, since Git is built to work like that. + +== Ditching of web applications + +:pouchdb: https://pouchdb.com/ +:instant-apps: https://developer.android.com/topic/google-play-instant + +The authors put web application in a worse position for building local-first +application, claiming that: + +____ +(...) the architecture of web apps remains fundamentally server-centric. +Offline support is an afterthought in most web apps, and the result is +accordingly fragile. +____ + +Well, I disagree. + +The problem isn't inherit to the web platform, but instead how people use it. + +I have myself built offline-first applications, leveraging IndexedDB, App Cache, +_etc_. I wanted to build an offline-first application on the web, and so I did. + +In fact, many people choose {pouchdb}[PouchDB] _because_ of that, since it is a +good tool for offline-first web applications. The problem isn't really the +technology, but how much people want their application to be local-first. + +Contrast it with Android {instant-apps}[Instant Apps], where applications are +sent to the phone in small parts. Since this requires an internet connection to +move from a part of the app bundle to another, a subset of the app isn't +local-first, despite being an app. + +The point isn't the technology, but how people are using it. Local-first web +applications are perfectly possible, just like non-local-first native +applications are possible. + +== Costs are underrated + +I think the costs of "old-fashioned apps" over "cloud apps" are underrated, +mainly regarding storage, and that this costs can vary a lot by application. + +Say a person writes online articles for their personal website, and puts +everything into Git. Since there isn't supposed to be any collaboration, all of +the relevant ideals of local-first are achieved. + +Now another person creates videos instead of articles. They could try keeping +everything local, but after some time the storage usage fills the entire disk. +This person's local-first setup would be much more complex, and would cost much +more on maintenance, backup and storage. + +Even though both have similar needs, a local-first video repository is much more +demanding. So the local-first thinking here isn't "just keep everything local", +but "how much time and money am I willing to spend to keep everything local". + +The convenience of "cloud apps" becomes so attractive that many don't even have +a local copy of their videos, and rely exclusively on service providers to +maintain, backup and store their content. + +The dial measuring "cloud apps" and "old-fashioned apps" needs to be specific to +use-cases. + +== Real-time collaboration is optional + +If I were the one making the list of ideals, I wouldn't focus so much on +real-time collaboration. + +Even though seamless collaboration is desired, it being real-time depends on the +network being available for that. But ideal 3 states that "The Network is +Optional", so real-time collaboration is also optional. + +The fundamentals of a local-first system should enable real-time collaboration +when network is available, but shouldn't focus on it. + +On many places when discussing applications being offline, it is common for me +to find people saying that their application works "even on a plane, subway or +elevator". That is a reflection of when said developers have to deal with +networks being unavailable. + +But this leaves out a big chunk of the world where internet connection is +intermittent, or only works every other day or only once a week, or stops +working when it rains, _etc_. For this audience, living without network +connectivity isn't such a discrete moment in time, but part of every day life. +I like the fact that the authors acknowledge that. + +When discussing "working offline", I'd rather keep this type of person in mind, +then the subset of people who are offline when on the elevator will naturally be +included. + +== On CRDTs and developer experience + +:archived-article: https://web.archive.org/web/20130116163535/https://labs.oracle.com/techrep/1994/smli_tr-94-29.pdf + +When discussing developer experience, the authors bring up some questions to be +answered further, like: + +____ +For an app developer, how does the use of a CRDT-based data layer compare to +existing storage layers like a SQL database, a filesystem, or CoreData? Is a +distributed system harder to write software for? +____ + +That is an easy one: yes. + +A distributed system _is_ harder to write software for, being a distributed +system. + +Adding a large layer of data structures and algorithms will make it more complex +to write software for, naturally. And if trying to make this layer transparent +to the programmer, so they can pretend that layer doesn't exist is a bad idea, +as RPC frameworks have tried, and failed. + +See "{archived-article}[A Note on Distributed Computing]" for a critique on RPC +frameworks trying to make the network invisible, which I think also applies in +equivalence for making the CRDTs layer invisible. + +== Conclusion + +I liked a lot the article, as it took the "offline-first" philosophy and ran +with it. + +But I think the authors' view of adding CRDTs and things becoming local-first is +a bit too magical. + +This particular area is one that I have large interest on, and I wish to see +more being done on the "local-first" space. diff --git a/src/content/en/blog/2021/01/26/remembering-ann.adoc b/src/content/en/blog/2021/01/26/remembering-ann.adoc new file mode 100644 index 0000000..6786b3c --- /dev/null +++ b/src/content/en/blog/2021/01/26/remembering-ann.adoc @@ -0,0 +1,216 @@ += ANN: remembering - Add memory to dmenu, fzf and similar tools +:categories: ann + +:remembering: https://euandreh.xyz/remembering/ +:dmenu: https://tools.suckless.org/dmenu/ +:fzf: https://github.com/junegunn/fzf + +Today I pushed v0.1.0 of {remembering}[remembering], a tool to enhance the +interactive usability of menu-like tools, such as {dmenu}[dmenu] and {fzf}[fzf]. + +== Previous solution + +:yeganesh: https://dmwit.com/yeganesh/ + +I previously used {yeganesh}[yeganesh] to fill this gap, but as I started to +rely less on Emacs, I added fzf as my go-to tool for doing fuzzy searching on +the terminal. But I didn't like that fzf always showed the same order of +things, when I would only need 3 or 4 commonly used files. + +For those who don't know: yeganesh is a wrapper around dmenu that will remember +your most used programs and put them on the beginning of the list of +executables. This is very convenient for interactive prolonged use, as with +time the things you usually want are right at the very beginning. + +But now I had this thing, yeganesh, that solved this problem for dmenu, but +didn't for fzf. + +I initially considered patching yeganesh to support it, but I found it more +coupled to dmenu than I would desire. I'd rather have something that knows +nothing about dmenu, fzf or anything, but enhances tools like those in a useful +way. + +== Implementation + +:v-010: https://euandre.org/git/remembering/tree/remembering?id=v0.1.0 +:getopts: https://www.opengroup.org/onlinepubs/9699919799/utilities/getopts.html +:sort: https://www.opengroup.org/onlinepubs/9699919799/utilities/sort.html +:awk: https://www.opengroup.org/onlinepubs/9699919799/utilities/awk.html +:spencer-quote: https://en.wikipedia.org/wiki/Henry_Spencer#cite_note-3 + +Other than being decoupled from dmenu, another improvement I though that could +be made on top of yeganesh is the programming language choice. Instead of +Haskell, I went with POSIX sh. Sticking to POSIX sh makes it require less +build-time dependencies. There aren't any, actually. Packaging is made much +easier due to that. + +The good thing is that the program itself is small enough ({v-010}[119 lines] on +v0.1.0) that POSIX sh does the job just fine, combined with other POSIX +utilities such as {getopts}[getopts], {sort}[sort] and {awk}[awk]. + +The behaviour is: given a program that will read from STDIN and write a single +entry to STDOUT, `remembering` wraps that program, and rearranges STDIN so that +previous choices appear at the beginning. + +Where you would do: + +[source,sh] +---- +$ seq 5 | fzf + + 5 + 4 + 3 + 2 +> 1 + 5/5 +> +---- + +And every time get the same order of numbers, now you can write: + +[source,sh] +---- +$ seq 5 | remembering -p seq-fzf -c fzf + + 5 + 4 + 3 + 2 +> 1 + 5/5 +> +---- + +On the first run, everything is the same. If you picked 4 on the previous +example, the following run would be different: + +[source,sh] +---- +$ seq 5 | remembering -p seq-fzf -c fzf + + 5 + 3 + 2 + 1 +> 4 + 5/5 +> +---- + +As time passes, the list would adjust based on the frequency of your choices. + +I aimed for reusability, so that I could wrap diverse commands with +`remembering` and it would be able to work. To accomplish that, a "profile" +(the `-p something` part) stores data about different runs separately. + +I took the idea of building something small with few dependencies to other +places too: - the manpages are written in troff directly; - the tests are just +more POSIX sh files; - and a POSIX Makefile to `check` and `install`. + +I was aware of the value of sticking to coding to standards, but I had past +experience mostly with programming language standards, such as ECMAScript, +Common Lisp, Scheme, or with IndexedDB or DOM APIs. It felt good to rediscover +these nice POSIX tools, which makes me remember of a quote by +{spencer-quote}[Henry Spencer]: + +____ +Those who do not understand Unix are condemned to reinvent it, poorly. +____ + +== Usage examples + +Here are some functions I wrote myself that you may find useful: + +=== Run a command with fzf on `$PWD` + +[source,sh] +---- +f() { + profile="$f-shell-function(pwd | sed -e 's_/_-_g')" + file="$(git ls-files | \ + remembering -p "$profile" \ + -c "fzf --select-1 --exit -0 --query \"$2\" --preview 'cat {}'")" + if [ -n "$file" ]; then + # shellcheck disable=2068 + history -s f $@ + history -s "$1" "$file" + "$1" "$file" +fi +} +---- + +This way I can run `f vi` or `f vi config` at the root of a repository, and the +list of files will always appear on the most used order. Adding `pwd` to the +profile allows it to not mix data for different repositories. + +=== Copy password to clipboard + +:pass: https://www.passwordstore.org/ + +[source,sh] +---- +choice="$(find "$HOME/.password-store" -type f | \ + grep -Ev '(.git|.gpg-id)' | \ + sed -e "s|$HOME/.password-store/||" -e 's/\.gpg$//' | \ + remembering -p password-store \ + -c 'dmenu -l 20 -i')" + + +if [ -n "$choice" ]; then + pass show "$choice" -c +fi +---- + +Adding the above to a file and binding it to a keyboard shortcut, I can access +the contents of my {pass}[password store], with the entries ordered by usage. + +=== Replacing yeganesh + +Where I previously had: + +[source,sh] +---- +exe=$(yeganesh -x) && exec $exe +---- + +Now I have: + +[source,sh] +---- +exe=$(dmenu_path | remembering -p dmenu-exec -c dmenu) && exec $exe +---- + +This way, the executables appear on order of usage. + +If you don't have `dmenu_path`, you can get just the underlying `stest` tool +that looks at the executables available in your `$PATH`. Here's a juicy +one-liner to do it: + +[source,sh] +---- +$ wget -O- https://dl.suckless.org/tools/dmenu-5.0.tar.gz | \ + tar Ozxf - dmenu-5.0/arg.h dmenu-5.0/stest.c | \ + sed 's|^#include "arg.h"$|// #include "arg.h"|' | \ + cc -xc - -o stest +---- + +With the `stest` utility you'll be able to list executables in your `$PATH` and +pipe them to dmenu or something else yourself: + +[source,sh] +---- +$ (IFS=:; ./stest -flx $PATH;) | sort -u | remembering -p another-dmenu-exec -c dmenu | sh +---- + +In fact, the code for `dmenu_path` is almost just like that. + +== Conclusion + +:packaged: https://euandre.org/git/package-repository/ + +For my personal use, I've {packaged}[packaged] `remembering` for GNU Guix and +Nix. Packaging it to any other distribution should be trivial, or just +downloading the tarball and running `[sudo] make install`. + +Patches welcome! diff --git a/src/content/en/blog/2021/02/17/fallible.adoc b/src/content/en/blog/2021/02/17/fallible.adoc new file mode 100644 index 0000000..1f2f641 --- /dev/null +++ b/src/content/en/blog/2021/02/17/fallible.adoc @@ -0,0 +1,285 @@ += ANN: fallible - Fault injection library for stress-testing failure scenarios +:updatedat: 2022-03-06 + +:fallible: https://euandreh.xyz/fallible/ + +Yesterday I pushed v0.1.0 of {fallible}[fallible], a miniscule library for +fault-injection and stress-testing C programs. + +== _EDIT_ + +:changelog: https://euandreh.xyz/fallible/CHANGELOG.html +:tarball: https://euandre.org/static/attachments/fallible.tar.gz + +2021-06-12: As of {changelog}[0.3.0] (and beyond), the macro interface improved +and is a bit different from what is presented in this article. If you're +interested, I encourage you to take a look at it. + +2022-03-06: I've {tarball}[archived] the project for now. It still needs some +maturing before being usable. + +== Existing solutions + +:gnu-std: https://www.gnu.org/prep/standards/standards.html#Semantics +:valgrind: https://www.valgrind.org/ +:so-alloc: https://stackoverflow.com/questions/1711170/unit-testing-for-failed-malloc + +Writing robust code can be challenging, and tools like static analyzers, fuzzers +and friends can help you get there with more certainty. As I would try to +improve some of my C code and make it more robust, in order to handle system +crashes, filled disks, out-of-memory and similar scenarios, I didn't find +existing tooling to help me get there as I expected to find. I couldn't find +existing tools to help me explicitly stress-test those failure scenarios. + +Take the "{gnu-std}[Writing Robust Programs]" section of the GNU Coding +Standards: + +____ +Check every system call for an error return, unless you know you wish to ignore +errors. (...) Check every call to malloc or realloc to see if it returned NULL. +____ + +From a robustness standpoint, this is a reasonable stance: if you want to have a +robust program that knows how to fail when you're out of memory and `malloc` +returns `NULL`, than you ought to check every call to `malloc`. + +Take a sample code snippet for clarity: + +[source,c] +---- +void a_function() { + char *s1 = malloc(A_NUMBER); + strcpy(s1, "some string"); + + char *s2 = malloc(A_NUMBER); + strcpy(s2, "another string"); +} +---- + +At a first glance, this code is unsafe: if any of the calls to `malloc` returns +`NULL`, `strcpy` will be given a `NULL` pointer. + +My first instinct was to change this code to something like this: + +[source,diff] +---- +@@ -1,7 +1,15 @@ + void a_function() { + char *s1 = malloc(A_NUMBER); ++ if (!s1) { ++ fprintf(stderr, "out of memory, exitting\n"); ++ exit(1); ++ } + strcpy(s1, "some string"); + + char *s2 = malloc(A_NUMBER); ++ if (!s2) { ++ fprintf(stderr, "out of memory, exitting\n"); ++ exit(1); ++ } + strcpy(s2, "another string"); + } +---- + +As I later found out, there are at least 2 problems with this approach: + +. *it doesn't compose*: this could arguably work if `a_function` was `main`. + But if `a_function` lives inside a library, an `exit(1);` is an inelegant way + of handling failures, and will catch the top-level `main` consuming the + library by surprise; +. *it gives up instead of handling failures*: the actual handling goes a bit + beyond stopping. What about open file handles, in-memory caches, unflushed + bytes, etc.? + +If you could force only the second call to `malloc` to fail, +{valgrind}[Valgrind] would correctly complain that the program exitted with +unfreed memory. + +So the last change to make the best version of the above code is: + +[source,diff] +---- +@@ -1,15 +1,14 @@ +-void a_function() { ++bool a_function() { + char *s1 = malloc(A_NUMBER); + if (!s1) { +- fprintf(stderr, "out of memory, exitting\n"); +- exit(1); ++ return false; + } + strcpy(s1, "some string"); + + char *s2 = malloc(A_NUMBER); + if (!s2) { +- fprintf(stderr, "out of memory, exitting\n"); +- exit(1); ++ free(s1); ++ return false; + } + strcpy(s2, "another string"); + } +---- + +Instead of returning `void`, `a_function` now returns `bool` to indicate whether +an error ocurred during its execution. If `a_function` returned a pointer to +something, the return value could be `NULL`, or an `int` that represents an +error code. + +The code is now a) safe and b) failing gracefully, returning the control to the +caller to properly handle the error case. + +After seeing similar patterns on well designed APIs, I adopted this practice for +my own code, but was still left with manually verifying the correctness and +robustness of it. + +How could I add assertions around my code that would help me make sure the +`free(s1);` exists, before getting an error report? How do other people and +projects solve this? + +From what I could see, either people a) hope for the best, b) write safe code +but don't strees-test it or c) write ad-hoc code to stress it. + +The most proeminent case of c) is SQLite: it has a few wrappers around the +familiar `malloc` to do fault injection, check for memory limits, add warnings, +create shim layers for other environments, etc. All of that, however, is +tightly couple with SQLite itself, and couldn't be easily pulled off for using +somewhere else. + +When searching for it online, an {so-alloc}[interesting thread] caught my +atention: fail the call to `malloc` for each time it is called, and when the +same stacktrace appears again, allow it to proceed. + +== Implementation + +:mallocfail: https://github.com/ralight/mallocfail +:should-fail-fn: https://euandre.org/git/fallible/tree/src/fallible.c?id=v0.1.0#n16 + +A working implementation of that already exists: {mallocfail}[mallocfail]. It +uses `LD_PRELOAD` to replace `malloc` at run-time, computes the SHA of the +stacktrace and fails once for each SHA. + +I initially envisioned and started implementing something very similar to +mallocfail. However I wanted it to go beyond out-of-memory scenarios, and using +`LD_PRELOAD` for every possible corner that could fail wasn't a good idea on the +long run. + +Also, mallocfail won't work together with tools such as Valgrind, who want to do +their own override of `malloc` with `LD_PRELOAD`. + +I instead went with less automatic things: starting with a +`fallible_should_fail(char *filename, int lineno)` function that fails once for +each `filename`+`lineno` combination, I created macro wrappers around common +functions such as `malloc`: + +[source,c] +---- +void *fallible_malloc(size_t size, const char *const filename, int lineno) { +#ifdef FALLIBLE + if (fallible_should_fail(filename, lineno)) { + return NULL; + } +#else + (void)filename; + (void)lineno; +#endif + return malloc(size); +} + +#define MALLOC(size) fallible_malloc(size, __FILE__, __LINE__) +---- + +With this definition, I could replace the calls to `malloc` with `MALLOC` (or +any other name that you want to `#define`): + +[source,diff] +---- +--- 3.c 2021-02-17 00:15:38.019706074 -0300 ++++ 4.c 2021-02-17 00:44:32.306885590 -0300 +@@ -1,11 +1,11 @@ + bool a_function() { +- char *s1 = malloc(A_NUMBER); ++ char *s1 = MALLOC(A_NUMBER); + if (!s1) { + return false; + } + strcpy(s1, "some string"); + +- char *s2 = malloc(A_NUMBER); ++ char *s2 = MALLOC(A_NUMBER); + if (!s2) { + free(s1); + return false; +---- + +With this change, if the program gets compiled with the `-DFALLIBLE` flag the +fault-injection mechanism will run, and `MALLOC` will fail once for each +`filename`+`lineno` combination. When the flag is missing, `MALLOC` is a very +thin wrapper around `malloc`, which compilers could remove entirely, and the +`-lfallible` flags can be omitted. + +This applies not only to `malloc` or other `stdlib.h` functions. If +`a_function` is important or relevant, I could add a wrapper around it too, that +checks if `fallible_should_fail` to exercise if its callers are also doing the +proper clean-up. + +The actual code is just this single function, +{should-fail-fn}[`fallible_should_fail`], which ended-up taking only ~40 lines. +In fact, there are more lines of either Makefile (111), README.md (82) or troff +(306) on this first version. + +The price for such fine-grained control is that this approach requires more +manual work. + +== Usage examples + +=== `MALLOC` from the `README.md` + +:fallible-check: https://euandreh.xyz/fallible/fallible-check.1.html + +[source,c] +---- +// leaky.c +#include <string.h> +#include <fallible_alloc.h> + +int main() { + char *aaa = MALLOC(100); + if (!aaa) { + return 1; + } + strcpy(aaa, "a safe use of strcpy"); + + char *bbb = MALLOC(100); + if (!bbb) { + // free(aaa); + return 1; + } + strcpy(bbb, "not unsafe, but aaa is leaking"); + + free(bbb); + free(aaa); + return 0; +} +---- + +Compile with `-DFALLIBLE` and run {fallible-check}[`fallible-check.1`]: + +[source,sh] +---- +$ c99 -DFALLIBLE -o leaky leaky.c -lfallible +$ fallible-check ./leaky +Valgrind failed when we did not expect it to: +(...suppressed output...) +# exit status is 1 +---- + +== Conclusion + +:package: https://euandre.org/git/package-repository/ + +For my personal use, I'll {package}[package] them for GNU Guix and Nix. +Packaging it to any other distribution should be trivial, or just downloading +the tarball and running `[sudo] make install`. + +Patches welcome! diff --git a/src/content/en/blog/2021/02/17/fallible.tar.gz b/src/content/en/blog/2021/02/17/fallible.tar.gz Binary files differnew file mode 100644 index 0000000..211cadd --- /dev/null +++ b/src/content/en/blog/2021/02/17/fallible.tar.gz diff --git a/src/content/en/blog/2021/04/29/relational-review.adoc b/src/content/en/blog/2021/04/29/relational-review.adoc new file mode 100644 index 0000000..4b53737 --- /dev/null +++ b/src/content/en/blog/2021/04/29/relational-review.adoc @@ -0,0 +1,144 @@ += A Relational Model of Data for Large Shared Data Banks - article-review + +:empty: +:reviewed-article: https://www.seas.upenn.edu/~zives/03f/cis550/codd.pdf + +This is a review of the article "{reviewed-article}[A Relational Model of Data +for Large Shared Data Banks]", by E. F. Codd. + +== Data Independence + +Codd brings the idea of _data independence_ as a better approach to use on +databases. This is contrast with the existing approaches, namely hierarquical +(tree-based) and network-based. + +His main argument is that queries in applications shouldn't depende and be +coupled with how the data is represented internally by the database system. +This key idea is very powerful, and something that we strive for in many other +places: decoupling the interface from the implementation. + +If the database system has this separation, it can kep the querying interface +stable, while having the freedom to change its internal representation at will, +for better performance, less storage, etc. + +This is true for most modern database systems. They can change from B-Trees +with leafs containing pointers to data, to B-Trees with leafs containing the raw +data , to hash tables. All that without changing the query interface, only its +performance. + +Codd mentions that, from an information representation standpoint, any index is +a duplication, but useful for perfomance. + +This data independence also impacts ordering (a _relation_ doesn't rely on the +insertion order). + +== Duplicates + +His definition of relational data is a bit differente from most modern database +systems, namely *no duplicate rows*. + +I couldn't find a reason behind this restriction, though. For practical +purposes, I find it useful to have it. + +== Relational Data + +:edn: https://github.com/edn-format/edn + +In the article, Codd doesn't try to define a language, and today's most popular +one is SQL. + +However, there is no restriction that says that "SQL database" and "relational +database" are synonyms. One could have a relational database without using SQL +at all, and it would still be a relational one. + +The main one that I have in mind, and the reason that led me to reading this +paper in the first place, is Datomic. + +Is uses an {edn}[edn]-based representation for datalog +queries{empty}footnote:edn-queries[ + You can think of it as JSON, but with a Clojure taste. +], and a particular schema used to represent data. + +Even though it looks very weird when coming from SQL, I'd argue that it ticks +all the boxes (except for "no duplicates") that defines a relational database, +since building relations and applying operations on them is possible. + +Compare and contrast a contrived example of possible representations of SQL and +datalog of the same data: + +[source,sql] +---- +-- create schema +CREATE TABLE people ( + id UUID PRIMARY KEY, + name TEXT NOT NULL, + manager_id UUID, + FOREIGN KEY (manager_id) REFERENCES people (id) +); + +-- insert data +INSERT INTO people (id, name, manager_id) VALUES + ("d3f29960-ccf0-44e4-be66-1a1544677441", "Foo", "076356f4-1a0e-451c-b9c6-a6f56feec941"), + ("076356f4-1a0e-451c-b9c6-a6f56feec941", "Bar"); + +-- query data, make a relation + +SELECT employees.name AS 'employee-name', + managers.name AS 'manager-name' +FROM people employees +INNER JOIN people managers ON employees.manager_id = managers.id; +---- + +[source,clojure] +---- +;; create schema +#{{:db/ident :person/id + :db/valueType :db.type/uuid + :db/cardinality :db.cardinality/one + :db/unique :db.unique/value} + {:db/ident :person/name + :db/valueType :db.type/string + :db/cardinality :db.cardinality/one} + {:db/ident :person/manager + :db/valueType :db.type/ref + :db/cardinality :db.cardinality/one}} + +;; insert data +#{{:person/id #uuid "d3f29960-ccf0-44e4-be66-1a1544677441" + :person/name "Foo" + :person/manager [:person/id #uuid "076356f4-1a0e-451c-b9c6-a6f56feec941"]} + {:person/id #uuid "076356f4-1a0e-451c-b9c6-a6f56feec941" + :person/name "Bar"}} + +;; query data, make a relation +{:find [?employee-name ?manager-name] + :where [[?person :person/name ?employee-name] + [?person :person/manager ?manager] + [?manager :person/name ?manager-name]]} +---- + +(forgive any errors on the above SQL and datalog code, I didn't run them to +check. Patches welcome!) + +This employee example comes from the paper, and both SQL and datalog +representations match the paper definition of "relational". + +Both "Foo" and "Bar" are employees, and the data is normalized. SQL represents +data as tables, and Datomic as datoms, but relations could be derived from both, +which we could view as: + +[source,sql] +---- +employee_name | manager_name +---------------------------- +"Foo" | "Bar" +---- + +== Conclusion + +The article also talks about operators, consistency and normalization, which are +now so widespread and well-known that it feels a bit weird seeing someone +advocating for it. + +I also stablish that `relational != SQL`, and other databases such as Datomic +are also relational, following Codd's original definition. diff --git a/src/content/en/blog/categories.adoc b/src/content/en/blog/categories.adoc new file mode 100644 index 0000000..feb64ff --- /dev/null +++ b/src/content/en/blog/categories.adoc @@ -0,0 +1,2 @@ += Articles by category +:type: categories diff --git a/src/content/en/blog/index.adoc b/src/content/en/blog/index.adoc new file mode 100644 index 0000000..e24d614 --- /dev/null +++ b/src/content/en/blog/index.adoc @@ -0,0 +1,2 @@ += Blog +:type: collection diff --git a/src/content/en/index.adoc b/src/content/en/index.adoc new file mode 100644 index 0000000..5fde68a --- /dev/null +++ b/src/content/en/index.adoc @@ -0,0 +1,3 @@ += Home + +The newest entry of each collection is listed below. diff --git a/src/content/en/pastebin/2016/04/05/rpn.adoc b/src/content/en/pastebin/2016/04/05/rpn.adoc new file mode 100644 index 0000000..c567d0d --- /dev/null +++ b/src/content/en/pastebin/2016/04/05/rpn.adoc @@ -0,0 +1,23 @@ += RPN macro setup + +[source,lisp] +---- +(defmacro rpn (body) + (rpn-expander body)) + +(defun rpn-expander (body) + (mapcar (lambda (x) + (if (listp x) + (rpn-expander x) + x)) + (reverse body))) + +(rpn ((2 1 +) 2 *)) +; => 6 + +#| +Just a quick stub. + +One could easily improve #'RPN-EXPANDER in order to better suit one's needs. +|# +---- diff --git a/src/content/en/pastebin/2018/07/11/nix-pinning.adoc b/src/content/en/pastebin/2018/07/11/nix-pinning.adoc new file mode 100644 index 0000000..63b1ac9 --- /dev/null +++ b/src/content/en/pastebin/2018/07/11/nix-pinning.adoc @@ -0,0 +1,26 @@ += Nix pinning +:categories: nix + +[source,nix] +---- +let + # Pin the nixpkgs version + stdenv = pkgs.stdenv; + pkgsOriginal = import <nixpkgs> {}; + pkgsSrc = pkgsOriginal.fetchzip { + url = "https://github.com/NixOS/nixpkgs/archive/18.03.zip"; + sha256 = "0hk4y2vkgm1qadpsm4b0q1vxq889jhxzjx3ragybrlwwg54mzp4f"; + }; + + pkgs = import (pkgsSrc) {}; + + buildNodeJS = pkgs.callPackage <nixpkgs/pkgs/development/web/nodejs/nodejs.nix> {}; + +in rec { + nodeFromNVMRC = buildNodeJS { + version = "8.7.0"; + sha256 = "16mml3cwjnq7yf9yd67d2dybav3nvbnk89fkixs1wz7fd26d05ss"; + patches = []; + }; +} +---- diff --git a/src/content/en/pastebin/2018/07/13/guix-nixos-systemd.adoc b/src/content/en/pastebin/2018/07/13/guix-nixos-systemd.adoc new file mode 100644 index 0000000..fa4226a --- /dev/null +++ b/src/content/en/pastebin/2018/07/13/guix-nixos-systemd.adoc @@ -0,0 +1,22 @@ += GNU Guix systemd daemon for NixOS +:catgories: nix, guix +:sort: 3 + +[source,nix] +---- + # Derived from Guix guix-daemon.service.in + # https://git.savannah.gnu.org/cgit/guix.git/tree/etc/guix-daemon.service.in?id=00c86a888488b16ce30634d3a3a9d871ed6734a2 + systemd.services.guix-daemon = { + enable = true; + description = "Build daemon for GNU Guix"; + serviceConfig = { + ExecStart = "/var/guix/profiles/per-user/root/guix-profile/bin/guix-daemon --build-users-group=guixbuild"; + Environment="GUIX_LOCPATH=/root/.guix-profile/lib/locale"; + RemainAfterExit="yes"; + StandardOutput="syslog"; + StandardError="syslog"; + TaskMax= 8192; + }; + wantedBy = [ "multi-user.target" ]; + }; +---- diff --git a/src/content/en/pastebin/2018/07/13/guixbuilder-nixos.adoc b/src/content/en/pastebin/2018/07/13/guixbuilder-nixos.adoc new file mode 100644 index 0000000..65dbcc7 --- /dev/null +++ b/src/content/en/pastebin/2018/07/13/guixbuilder-nixos.adoc @@ -0,0 +1,42 @@ += Guix users in NixOS system configuration +:categories: nix guix +:sort: 2 + +[source,nix] +---- + users = { + mutableUsers = false; + + extraUsers = + let + andrehUser = { + andreh = { + # my custom user config + }; + }; + # From the Guix manual: + # https://www.gnu.org/software/guix/manual/en/html_node/Build-Environment-Setup.html#Build-Environment-Setup + buildUser = (i: + { + "guixbuilder${i}" = { # guixbuilder$i + group = "guixbuild"; # -g guixbuild + extraGroups = ["guixbuild"]; # -G guixbuild + home = "/var/empty"; # -d /var/empty + shell = pkgs.nologin; # -s `which nologin` + description = "Guix build user ${i}"; # -c "Guix buid user $i" + isSystemUser = true; # --system + }; + } + ); + in + # merge all users + pkgs.lib.fold (str: acc: acc // buildUser str) + andrehUser + # for i in `seq -w 1 10` + (map (pkgs.lib.fixedWidthNumber 2) (builtins.genList (n: n+1) 10)); + + extraGroups.guixbuild = { + name = "guixbuild"; + }; + }; +---- diff --git a/src/content/en/pastebin/2018/07/13/guixbuilder.adoc b/src/content/en/pastebin/2018/07/13/guixbuilder.adoc new file mode 100644 index 0000000..35057f9 --- /dev/null +++ b/src/content/en/pastebin/2018/07/13/guixbuilder.adoc @@ -0,0 +1,14 @@ += Guix builder user creation commands +:categories: guix + +[source,sh] +---- +groupadd --system guixbuild +for i in `seq -w 1 10`; +do + useradd -g guixbuild -G guixbuild \ + -d /var/empty -s `which nologin` \ + -c "Guix build user $i" --system \ + guixbuilder$i; +done +---- diff --git a/src/content/en/pastebin/2018/07/13/nix-strpad.adoc b/src/content/en/pastebin/2018/07/13/nix-strpad.adoc new file mode 100644 index 0000000..71e8168 --- /dev/null +++ b/src/content/en/pastebin/2018/07/13/nix-strpad.adoc @@ -0,0 +1,8 @@ += Nix string padding +:categories: nix +:sort: 1 + +[source,nix] +---- +padString = (n: if n < 10 then "0" + toString n else toString n) +---- diff --git a/src/content/en/pastebin/2018/07/25/nix-exps.adoc b/src/content/en/pastebin/2018/07/25/nix-exps.adoc new file mode 100644 index 0000000..04cb7f4 --- /dev/null +++ b/src/content/en/pastebin/2018/07/25/nix-exps.adoc @@ -0,0 +1,46 @@ += Nix exps +:categories: nix + +[source,nix] +---- +let + pkgsOriginal = import <nixpkgs> {}; + pkgsSrc = pkgsOriginal.fetchzip { + url = "https://github.com/NixOS/nixpkgs/archive/18.03.zip"; + sha256 = "0hk4y2vkgm1qadpsm4b0q1vxq889jhxzjx3ragybrlwwg54mzp4f"; + }; + pkgs = import (pkgsSrc) {}; + stdenv = pkgs.stdenv; + + # Taken from: + # http://www.cs.yale.edu/homes/lucas.paul/posts/2017-04-10-hakyll-on-nix.html + websiteBuilder = pkgs.stdenv.mkDerivation { + name = "website-builder"; + src = ./hakyll; + phases = "unpackPhase buildPhase"; + buildInputs = [ + (pkgs.haskellPackages.ghcWithPackages (p: with p; [ hakyll ])) + ]; + buildPhase = '' + mkdir -p $out/bin + ghc -O2 -dynamic --make Main.hs -o $out/bin/generate-site + ''; + }; +in rec { + euandrehWebsite = stdenv.mkDerivation rec { + name = "euandreh-website"; + src = ./site; + phases = "unpackPhase buildPhase"; + # version = "0.1"; + buildInputs = [ websiteBuilder ]; + buildPhase = '' + export LOCALE_ARCHIVE="${pkgs.glibcLocales}/lib/locale/locale-archive"; + export LANG=en_US.UTF-8 + generate-site build + + mkdir $out + cp -r _site/* $out + ''; + }; +} +---- diff --git a/src/content/en/pastebin/2018/07/25/nix-showdrv.adoc b/src/content/en/pastebin/2018/07/25/nix-showdrv.adoc new file mode 100644 index 0000000..b62e526 --- /dev/null +++ b/src/content/en/pastebin/2018/07/25/nix-showdrv.adoc @@ -0,0 +1,75 @@ += nix show-derivation sample output +:categories: nix +:sort: 1 + +[source,nix] +---- +$ nix show-derivation /nix/store/zzz9cl2ly0mb2njr7vwa5528fxmn29m8-combofont-0.2.drv +{ + "/nix/store/zzz9cl2ly0mb2njr7vwa5528fxmn29m8-combofont-0.2.drv": { + "outputs": { + "out": { + "path": "/nix/store/dc897j29s5pl5mcw064n5b07bydacfm5-combofont-0.2", + "hashAlgo": "r:sha1", + "hash": "06be9cab7176fe6d99dd773315d9ec5c62f6a71b" + } + }, + "inputSrcs": [ + "/nix/store/b6ill8amfg0gki49zapm4asrrw9zzgz9-builder.sh" + ], + "inputDrvs": { + "/nix/store/3s0crp8826gwvfap6kjjyh9a7wq92awk-stdenv.drv": [ + "out" + ], + "/nix/store/fafsh2hx1xxqgm8gwkj3bw3czz6dcvvw-mirrors-list.drv": [ + "out" + ], + "/nix/store/qqla9sd8p8qwgl2a1wpn75bwp2vw70mm-bash-4.4-p12.drv": [ + "out" + ], + "/nix/store/v8fxvb0wlsa5pmrfawa3dg501mglw43c-curl-7.59.0.drv": [ + "dev" + ] + }, + "platform": "x86_64-linux", + "builder": "/nix/store/lw7xaqhakk0i1c631m3cvac3x4lc5gr5-bash-4.4-p12/bin/bash", + "args": [ + "-e", + "/nix/store/b6ill8amfg0gki49zapm4asrrw9zzgz9-builder.sh" + ], + "env": { + "buildInputs": "", + "builder": "/nix/store/lw7xaqhakk0i1c631m3cvac3x4lc5gr5-bash-4.4-p12/bin/bash", + "configureFlags": "", + "curlOpts": "", + "depsBuildBuild": "", + "depsBuildBuildPropagated": "", + "depsBuildTarget": "", + "depsBuildTargetPropagated": "", + "depsHostBuild": "", + "depsHostBuildPropagated": "", + "depsTargetTarget": "", + "depsTargetTargetPropagated": "", + "downloadToTemp": "1", + "executable": "", + "impureEnvVars": "http_proxy https_proxy ftp_proxy all_proxy no_proxy NIX_CURL_FLAGS NIX_HASHED_MIRRORS NIX_CONNECT_TIMEOUT NIX_MIRRORS_apache NIX_MIRRORS_bioc NIX_MIRRORS_bitlbee NIX_MIRRORS_cpan NIX_MIRRORS_debian NIX_MIRRORS_fedora NIX_MIRRORS_gcc NIX_MIRRORS_gentoo NIX_MIRRORS_gnome NIX_MIRRORS_gnu NIX_MIRRORS_gnupg NIX_MIRRORS_hackage NIX_MIRRORS_hashedMirrors NIX_MIRRORS_imagemagick NIX_MIRRORS_kde NIX_MIRRORS_kernel NIX_MIRRORS_maven NIX_MIRRORS_metalab NIX_MIRRORS_mozilla NIX_MIRRORS_mysql NIX_MIRRORS_oldsuse NIX_MIRRORS_openbsd NIX_MIRRORS_opensuse NIX_MIRRORS_postgresql NIX_MIRRORS_pypi NIX_MIRRORS_roy NIX_MIRRORS_sagemath NIX_MIRRORS_samba NIX_MIRRORS_savannah NIX_MIRRORS_sourceforge NIX_MIRRORS_sourceforgejp NIX_MIRRORS_steamrt NIX_MIRRORS_ubuntu NIX_MIRRORS_xfce NIX_MIRRORS_xorg", + "mirrorsFile": "/nix/store/36pk3fz566c2zj6bj8qy7gxl1z14xc4f-mirrors-list", + "name": "combofont-0.2", + "nativeBuildInputs": "/nix/store/hgv54iw72sgpqmzgv30s6gsfc4rd4wzp-curl-7.59.0-dev", + "out": "/nix/store/dc897j29s5pl5mcw064n5b07bydacfm5-combofont-0.2", + "outputHash": "3fkzcqjwxkciacvpvncnvzknf6mrrgh6", + "outputHashAlgo": "sha1", + "outputHashMode": "recursive", + "postFetch": "mkdir \"$out\";tar -xf $downloadedFile \\\n '--strip-components=0' \\\n -C \"$out\" --anchored --exclude=tlpkg --keep-old-files\n", + "preferHashedMirrors": "1", + "preferLocalBuild": "1", + "propagatedBuildInputs": "", + "propagatedNativeBuildInputs": "", + "showURLs": "", + "stdenv": "/nix/store/i3kgk0nibrbpgmzdwdfi2ym50i8m3lww-stdenv", + "system": "x86_64-linux", + "urls": "http://146.185.144.154/texlive-2017/combofont.tar.xz http://gateway.ipfs.io/ipfs/QmRLK45EC828vGXv5YDaBsJBj2LjMjjA2ReLVrXsasRzy7/texlive-2017/combofont.tar.xz" + } + } +} +---- diff --git a/src/content/en/pastebin/2019/06/08/inconsistent-hash.adoc b/src/content/en/pastebin/2019/06/08/inconsistent-hash.adoc new file mode 100644 index 0000000..8dc5794 --- /dev/null +++ b/src/content/en/pastebin/2019/06/08/inconsistent-hash.adoc @@ -0,0 +1,1053 @@ += Inconsistent hash of buildGoModule +:categories: nix + +:commit: https://euandre.org/git/servers/commit?id=6ba76140238b5e3c7009c201f9f80ac86063f438 + +'''' + +FIXED: The `<nixpkgs>` was different on different environments. See +https://discourse.nixos.org/t/inconsistent-hash-of-buildgomodule/3127/2. + +'''' + +The {commit}[commit that made this visible]. + +== Offending derivation: + +:orig-src: https://euandre.org/git/servers/tree/default.nix?id=6ba76140238b5e3c7009c201f9f80ac86063f438#n3 + +{orig-src}[Full source code on the repository]: + +[source,nix] +---- +terraform-godaddy = pkgs.buildGoModule rec { + name = "terraform-godaddy-${version}"; + version = "1.6.4"; + src = pkgs.fetchFromGitHub { + owner = "n3integration"; + repo = "terraform-godaddy"; + rev = "v${version}"; + sha256 = "00blqsan74s53dk9ab4hxi1kzxi46k57dr65dmbiradfa3yz3852"; + }; + modSha256 = "0p81wqw2n8vraxk20xwg717582ijwq2k7v5j3n13y4cd5bxd8hhz"; + postInstall = + "mv $out/bin/terraform-godaddy $out/bin/terraform-provider-godaddy"; +}; +---- + +== Local build: + +[source,sh] +---- +$ nix-build -A terraform-godaddy +these derivations will be built: + /nix/store/3hs274i9qdsg3hsgp05j7i5cqxsvpcqx-terraform-godaddy-1.6.4-go-modules.drv + /nix/store/y5961vv6y9c0ps2sbd8xfnpqvk0q7qhq-terraform-godaddy-1.6.4.drv +building '/nix/store/3hs274i9qdsg3hsgp05j7i5cqxsvpcqx-terraform-godaddy-1.6.4-go-modules.drv'... +unpacking sources +unpacking source archive /nix/store/m62ydk4wy6818sysfys0qz20cx5nzj7h-source +source root is source +patching sources +configuring +building +go: finding github.com/mitchellh/copystructure v1.0.0 +go: finding github.com/blang/semver v3.5.1+incompatible +go: finding github.com/posener/complete v1.2.1 +go: finding github.com/apparentlymart/go-cidr v1.0.0 +go: finding github.com/agext/levenshtein v1.2.1 +go: finding github.com/mitchellh/reflectwalk v1.0.0 +go: finding github.com/mitchellh/mapstructure v1.1.2 +go: finding github.com/hashicorp/hil v0.0.0-20170627220502-fa9f258a9250 +go: finding github.com/bgentry/go-netrc v0.0.0-20140422174119-9fd32a8b3d3d +go: finding github.com/bgentry/speakeasy v0.1.0 +go: finding github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af +go: finding github.com/hashicorp/errwrap v1.0.0 +go: finding github.com/hashicorp/hcl2 v0.0.0-20181220012050-6631d7cd0a68 +go: finding google.golang.org/grpc v1.17.0 +go: finding golang.org/x/crypto v0.0.0-20181203042331-505ab145d0a9 +go: finding github.com/hashicorp/go-version v1.0.0 +go: finding google.golang.org/appengine v1.4.0 +go: finding golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4 +go: finding honnef.co/go/tools v0.0.0-20180920025451-e3ad64cb4ed3 +go: finding github.com/hashicorp/terraform v0.11.11 +go: finding google.golang.org/genproto v0.0.0-20181221175505-bd9b4fb69e2f +go: finding github.com/mitchellh/go-wordwrap v1.0.0 +go: finding github.com/hashicorp/go-cleanhttp v0.5.0 +go: finding github.com/kylelemons/godebug v0.0.0-20170820004349-d65d576e9348 +go: finding golang.org/x/oauth2 v0.0.0-20181203162652-d668ce993890 +go: finding github.com/mitchellh/go-wordwrap v0.0.0-20150314170334-ad45545899c7 +go: finding github.com/kr/pty v1.1.3 +go: finding github.com/hashicorp/yamux v0.0.0-20181012175058-2f1d1f20f75d +go: finding github.com/aws/aws-sdk-go v1.16.11 +go: finding cloud.google.com/go v0.26.0 +go: finding google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8 +go: finding github.com/sergi/go-diff v1.0.0 +go: finding golang.org/x/sys v0.0.0-20181228144115-9a3f9b0469bb +go: finding github.com/go-ini/ini v1.40.0 +go: finding github.com/golang/protobuf v1.2.0 +go: finding github.com/satori/go.uuid v1.2.0 +go: finding github.com/mitchellh/cli v1.0.0 +go: finding google.golang.org/appengine v1.1.0 +go: finding honnef.co/go/tools v0.0.0-20180728063816-88497007e858 +go: finding golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f +go: finding github.com/mitchellh/iochan v1.0.0 +go: finding github.com/mitchellh/go-homedir v1.0.0 +go: finding github.com/spf13/pflag v1.0.2 +go: finding github.com/kr/pretty v0.1.0 +go: finding github.com/go-test/deep v1.0.1 +go: finding github.com/hashicorp/go-multierror v1.0.0 +go: finding github.com/spf13/pflag v1.0.3 +go: finding github.com/onsi/ginkgo v1.7.0 +go: finding github.com/onsi/gomega v1.4.3 +go: finding github.com/zclconf/go-cty v0.0.0-20181218225846-4fe1e489ee06 +go: finding gopkg.in/yaml.v2 v2.2.2 +go: finding github.com/mitchellh/gox v0.4.0 +go: finding github.com/zclconf/go-cty v0.0.0-20181129180422-88fbe721e0f8 +go: finding golang.org/x/crypto v0.0.0-20180816225734-aabede6cba87 +go: finding golang.org/x/net v0.0.0-20181220203305-927f97764cc3 +go: finding golang.org/x/net v0.0.0-20180826012351-8a410e7b638d +go: finding github.com/google/go-cmp v0.2.0 +go: finding golang.org/x/sys v0.0.0-20180830151530-49385e6e1522 +go: finding github.com/onsi/ginkgo v1.6.0 +go: finding gopkg.in/fsnotify.v1 v1.4.7 +go: finding gopkg.in/yaml.v2 v2.2.1 +go: finding github.com/hashicorp/go-plugin v0.0.0-20181212150838-f444068e8f5a +go: finding github.com/armon/go-radix v1.0.0 +go: finding golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be +go: finding github.com/golang/mock v1.1.1 +go: finding github.com/ulikunitz/xz v0.5.5 +go: finding golang.org/x/tools v0.0.0-20180828015842-6cd1fcedba52 +go: finding github.com/davecgh/go-spew v1.1.1 +go: finding golang.org/x/net v0.0.0-20180906233101-161cd47e91fd +go: finding gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 +go: finding github.com/hpcloud/tail v1.0.0 +go: finding golang.org/x/lint v0.0.0-20181217174547-8f45f776aaf1 +go: finding github.com/mattn/go-colorable v0.0.9 +go: finding google.golang.org/grpc v1.16.0 +go: finding github.com/vmihailenco/msgpack v3.3.3+incompatible +go: finding github.com/posener/complete v1.1.1 +go: finding github.com/mitchellh/go-testing-interface v1.0.0 +go: finding github.com/golang/protobuf v1.1.0 +go: finding github.com/mattn/go-isatty v0.0.3 +go: finding github.com/kr/text v0.1.0 +go: finding golang.org/x/net v0.0.0-20181106065722-10aee1819953 +go: finding github.com/hashicorp/go-hclog v0.0.0-20181001195459-61d530d6c27f +go: finding github.com/oklog/run v1.0.0 +go: finding github.com/mitchellh/hashstructure v1.0.0 +go: finding golang.org/x/tools v0.0.0-20181221235234-d00ac6d27372 +go: finding github.com/hashicorp/go-getter v0.0.0-20181213035916-be39683deade +go: finding github.com/kisielk/gotool v1.0.0 +go: finding howett.net/plist v0.0.0-20181124034731-591f970eefbb +go: finding github.com/vmihailenco/msgpack v4.0.1+incompatible +go: finding golang.org/x/sync v0.0.0-20181108010431-42b317875d0f +go: finding golang.org/x/net v0.0.0-20180724234803-3673e40ba225 +go: finding gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 +go: finding github.com/fatih/color v1.7.0 +go: finding cloud.google.com/go v0.34.0 +go: finding github.com/hashicorp/yamux v0.0.0-20180604194846-3520598351bb +go: finding github.com/hashicorp/hcl v1.0.0 +go: finding github.com/hashicorp/go-uuid v1.0.0 +go: finding github.com/hashicorp/go-multierror v0.0.0-20180717150148-3d5d8f294aa0 +go: finding github.com/mattn/go-isatty v0.0.4 +go: finding github.com/hashicorp/errwrap v0.0.0-20180715044906-d6c0cd880357 +go: finding github.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310 +go: finding golang.org/x/net v0.0.0-20180811021610-c39426892332 +go: finding github.com/fsnotify/fsnotify v1.4.7 +go: finding github.com/bsm/go-vlq v0.0.0-20150828105119-ec6e8d4f5f4e +go: finding github.com/golang/mock v1.2.0 +go: finding golang.org/x/net v0.0.0-20181129055619-fae4c4e3ad76 +go: finding github.com/apparentlymart/go-dump v0.0.0-20180507223929-23540a00eaa3 +go: finding github.com/aws/aws-sdk-go v1.15.78 +go: finding github.com/golang/lint v0.0.0-20180702182130-06c8688daad7 +go: finding golang.org/x/text v0.3.0 +go: finding github.com/pmezard/go-difflib v1.0.0 +go: finding golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc +go: finding github.com/kr/pty v1.1.1 +go: finding github.com/client9/misspell v0.3.4 +go: finding github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b +go: finding golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3 +go: finding gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 +go: finding github.com/jessevdk/go-flags v1.4.0 +go: finding github.com/stretchr/testify v1.2.2 +go: finding github.com/hashicorp/go-hclog v0.0.0-20180709165350-ff2cf002a8dd +go: finding golang.org/x/net v0.0.0-20181114220301-adae6a3d119a +go: finding github.com/apparentlymart/go-textseg v1.0.0 +go: finding golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e +go: finding github.com/mitchellh/go-testing-interface v0.0.0-20171004221916-a61a99592b77 +go: finding google.golang.org/grpc v1.14.0 +go: finding golang.org/x/lint v0.0.0-20180702182130-06c8688daad7 +go: finding github.com/hashicorp/go-safetemp v1.0.0 +go: finding github.com/jmespath/go-jmespath v0.0.0-20160202185014-0b12d6b521d8 +installing +hash mismatch in fixed-output derivation '/nix/store/jgbfkhlsz6bmq724p5cqqcgfyc7l6sdv-terraform-godaddy-1.6.4-go-modules': + wanted: sha256:0p81wqw2n8vraxk20xwg717582ijwq2k7v5j3n13y4cd5bxd8hhz + got: sha256:10n2dy7q9kk1ly58sw965n6qa8l0nffh8vyd1vslx0gdlyj25xxs +cannot build derivation '/nix/store/y5961vv6y9c0ps2sbd8xfnpqvk0q7qhq-terraform-godaddy-1.6.4.drv': 1 dependencies couldn't be built +error: build of '/nix/store/y5961vv6y9c0ps2sbd8xfnpqvk0q7qhq-terraform-godaddy-1.6.4.drv' failed +---- + +== Build https://builds.sr.ht/~euandreh/job/67836#task-setup-0[on CI]: + +The `setup.sh` script contains a call to `nix-shell` which in turns build the +same `terraform-godaddy` derivation: + +[source,sh] +---- +$ cd vps/ +$ ./scripts/ci/setup.sh +warning: Nix search path entry '/nix/var/nix/profiles/per-user/root/channels' does not exist, ignoring +these derivations will be built: + /nix/store/as9r3n55czsdiq82iacs0hq12alxb2m0-remove-references-to.drv + /nix/store/fdh1ahjdh3fgsz4qz386klsa9bsqil48-source.drv + /nix/store/x7r5kh20ajlnj6vw6fg649w0iypcg1ga-terraform-godaddy-1.6.4-go-modules.drv + /nix/store/w4ghinrmpq524k3617ikfc8i42aa0dbb-terraform-godaddy-1.6.4.drv +these paths will be fetched (868.72 MiB download, 4262.91 MiB unpacked): + /nix/store/01aggsi1ndjhnr93gcy8c4s1xbxab8dn-unzip-6.0 + /nix/store/02nzlzdw0kiici9368jp5s84cpbqxkva-python3.7-certifi-2018.11.29 + /nix/store/0bdf1xn7p6xzk008yr6cahq3wjlvah5g-terraform-provider-flexibleengine-1.5.0-bin + /nix/store/0jl2dhydfh3jbfpkgkrixisqkhj12d4y-libffi-3.2.1-dev + /nix/store/0jwyd55g8nfhm25a0bh1j1by6afdriic-perl5.28.2-File-Listing-6.04 + /nix/store/0rc1jyfbxwffmsphyv2pfnxd6smysc1l-terraform-provider-ansible-0.0.4-bin + /nix/store/0w6l8kh3d30kg3nxc8xyi84gmrfxjnns-git-2.21.0 + /nix/store/0z8i6sq8mg138qnifr1z37y780xkk8hf-terraform-provider-cloudstack-0.2.0-bin + /nix/store/15fv1623h1vcn5z0nq42v5rgjirbp5r0-terraform-provider-rancher-1.2.1-bin + /nix/store/18rr3rg32imsnfyx6zb6s8lc8qpkdr74-nghttp2-1.38.0-bin + /nix/store/1dydqkwswavzkyvr1qr62zmx3nqpmpp4-gnutls-3.6.7 + /nix/store/1fl7yd9chgswnabbsvva7xvg5ak1q44p-terraform-provider-vault-1.8.0-bin + /nix/store/1hml3hx7qlbkv139khazb24jh69nngcd-terraform-provider-bigip-0.12.2-bin + /nix/store/1kz91g5mfj271lj5kxz2m1axcs2yqafy-thin-provisioning-tools-0.7.6 + /nix/store/1wh5wgw6a3w91mk2avvn9ssw32nlw9kd-terraform-provider-openstack-1.18.0-bin + /nix/store/206dvjl6595dk40dli12ziv393ww54wl-bzip2-1.0.6.0.1 + /nix/store/20wmykp8fj2izxdj8lic8ggcfpdid5ka-tzdata-2019a + /nix/store/2ar3zk5fjr34ys2dqnsfbb678x6fdlj4-openssh-7.9p1 + /nix/store/2dfjlvp38xzkyylwpavnh61azi0d168b-binutils-2.31.1 + /nix/store/2j9jm3jaxfn2g6wxak61wkhmrg6c4nn5-unbound-1.9.1-lib + /nix/store/2k46270d0h3gqj1c0wgx8prnj51jqryd-db-5.3.28 + /nix/store/2lh08897y86kxvyjdd1vlnkg8fz88nkd-terraform-provider-rundeck-0.1.0-bin + /nix/store/2xhsrw4ws6kc4x3983wdwwlnim27c6iz-shadow-4.6 + /nix/store/2yy3pv77rwbxk7b2mpysmiqdzhmgmphg-terraform-provider-http-1.1.1-bin + /nix/store/31l04a1yxxdbdpzdp8mpfk96rhj3bg2c-python3.7-netaddr-0.7.19 + /nix/store/35mdgd1wc67g60azsrghzgn4fjhr5d2r-zfs-user-0.7.13-lib + /nix/store/3mgn9jnjzj1rgxclbixk5xa0kkx9xpw3-openssl-1.0.2r-dev + /nix/store/3qjz5kfri8sa0dj1213rap75alpqsm2l-terraform-provider-mailgun-0.1.0-bin + /nix/store/3s4fr71ykyw54kyyqavd0ba42klg0bhf-libXcursor-1.2.0 + /nix/store/3xq3w5fgz99rhp3rxfkbp0ahg37mgmly-pango-1.43.0 + /nix/store/3xzkc4wyadr3vrva2q320axjr6cyb43n-python-2.7.16 + /nix/store/43i41p1n1sxssmqpf9jp5x4gcy6r2fl6-git-2.21.0 + /nix/store/479dvd7q6c18l3jl2myhfxmfsjbqjjch-python3.7-dopy-2016-01-04 + /nix/store/4i1mw6av3d6pr9bqggb4hnv6cykbrhhi-kexec-tools-2.0.19 + /nix/store/4jw2677fvb11aj1bal9a2iksqz0mk80m-expat-2.2.6 + /nix/store/4qq5hh1r6sqb0kpxc305rb468s45j4aw-libICE-1.0.9 + /nix/store/4z62pandn85xhcc5vazmi29cs2yps47b-iproute2-5.0.0 + /nix/store/50gfgyi2rxi4n25if8cqvlxlh5czl0wd-yajl-2.1.0 + /nix/store/50rywa1m6asdz1y78a6dpa0xf98vm01v-perl5.28.2-LWP-MediaTypes-6.04 + /nix/store/50wggbbr0wdg21hrvl4icwlppvk4464b-terraform-provider-opc-1.3.6-bin + /nix/store/5b0s7hhp52vq4psmicf8m8y2jr5jsiaz-terraform-provider-ucloud-1.6.0-bin + /nix/store/5k67y2lglsnswrya21z51d4h87a081k5-terraform-provider-kubernetes-1.6.2-bin + /nix/store/5l3967kll8m6s66zprzwb2p6vf2mh5yd-libtasn1-4.13 + /nix/store/5lcz7p2xz1zp8iyd9yjmrg1kxw5yygnx-terraform-provider-panos-1.5.1-bin + /nix/store/5nkxpwdgpxs97yqh2fxz9y0rm80rc280-terraform-provider-heroku-1.9.0-bin + /nix/store/5pjazw71xk4kysxrzacgjl4iai691k25-curl-7.64.1 + /nix/store/5qnlfx9qncn0fcw6mbfj6j58pz0cv0p3-binutils-wrapper-2.31.1 + /nix/store/5x1551gw825apcsnwx8gzfnmiapbz8yl-perl5.28.2-IO-HTML-1.001 + /nix/store/5ydkc9jcaaxlz58dr7gvyhi3gcmafsfy-python3.7-pyparsing-2.3.1 + /nix/store/5ygnx64lyv5a8pnpmlj7bs8s2dz2hkxd-terraform-provider-spotinst-1.13.2-bin + /nix/store/5z3s6zbi98gh8cfliaplnmv15j568c46-terraform-provider-null-2.1.2-bin + /nix/store/61shjilahl0d237fg9b3z3chza2lgms4-patchelf-0.9 + /nix/store/63gjp25l4cmdkl63zy0rcgmsvd2p2p34-terraform-0.11.14 + /nix/store/63k736kr346ncpzv5yiqiyyyiqpa2h8m-terraform-provider-bitbucket-1.0.0-bin + /nix/store/6554dpyahvcs49dmv434aky6bfkmqb30-gnumake-4.2.1 + /nix/store/69msrhi85iay3cb7c3nksr0s8l0xpsc7-util-linux-2.33.2 + /nix/store/69vq0a9sqynmz335apm8zgyjdmq34s5j-libX11-1.6.7 + /nix/store/6b2jabk1scwhhk9bz7wjzycvmkiw419d-libapparmor-2.13.1 + /nix/store/6brahzfjri338n3fggplfrsmf63mrwnx-terraform-provider-nutanix-1.0.1-bin + /nix/store/6bvd29jny80ka8df9prr5hrl5yz7d98k-systemd-239.20190219 + /nix/store/6hv1yfwyydyg2lzqcllwjb68xl4mrppw-terraform-provider-tencentcloud-1.5.0-bin + /nix/store/6hwdmzpspbnb7ix5z6m9h60jyy42kj90-dbus-1.12.12-lib + /nix/store/6mz512j183wj7qas2qm6zkrks5k8rh00-gettext-0.19.8.1 + /nix/store/6whclwjzwg46s0dkxwk1xz8cdcxnkd3y-db-4.8.30 + /nix/store/715lcljfyp8grxlmaf51pn0n3ml3dwgg-bash-interactive-4.4-p23 + /nix/store/7256h1y98mmzsckwk2x7i3v3cxmvgrmq-python3.7-pyOpenSSL-19.0.0-dev + /nix/store/749qksf79hvn0aprcznd9bwfv550qwh3-go-1.12.1 + /nix/store/7axz4xwz0vfrdgjyk59xg998bdqbvg5x-terraform-provider-random-2.1.2-bin + /nix/store/7b7nbb0w2iwskwhzjhfwrif631h4smia-libpciaccess-0.14 + /nix/store/7crry947d1xvp1f15c6q089l0gcy5hpc-stdenv-linux + /nix/store/7fd40sykaxj6dvya7mvif3f16wrqijr9-terraform-provider-terraform-1.0.2-bin + /nix/store/7gwvcm8dc24vnphbx85q1afaxhfhac28-perl5.28.2-HTTP-Cookies-6.04 + /nix/store/7k3hvg4sfpr6y2bg8b7x9mkb0d2p3scr-terraform-provider-clc-0.1.0-bin + /nix/store/7mmn8ri08z48vfj69c2h66f3g349ilq1-mailcap-2.1.48 + /nix/store/7mqpbfs391s9hbnfzkpgw3inj8mkldr8-terraform-provider-azurerm-1.27.1-bin + /nix/store/85hh7apv9n3gganpnnq36zvlwm126mdh-openssl-1.0.2r-bin + /nix/store/89wg3f6hk41gxm4n6cikj6r7gr2k7h8j-terraform-provider-nixos-0.0.1-bin + /nix/store/8aylwgi9nb6hsgz6620fzsllbc7h2rx1-c-ares-1.15.0 + /nix/store/8bxvyvd3ky0w5gk3k0lq2fmvj30fbzj8-zfs-user-0.7.13 + /nix/store/8dl5c7n4555lr6qipki2424652gf8az8-ebtables-2.0.10-4 + /nix/store/8gn2b5vvlazg608cj1y5l4igp9rckmnq-dejavu-fonts-minimal-2.37 + /nix/store/8krs5vbid0ic6vvlvjvndvjb815q8hbd-terraform-provider-ns1-1.3.1-bin + /nix/store/8ndwixznskf4zbf6h96ww4j8ap4j8067-fribidi-1.0.5 + /nix/store/8vqr6vbvyzlpsk6q0mnj93sf5j1wr5qa-perl5.28.2-Test-RequiresInternet-0.05 + /nix/store/8xy69pkisipvdmrpm1nmbi6qa2c6lhn0-bash-interactive-4.4-p23-info + /nix/store/8yy3wngrdcpmjyw2ryas1y4wwhbd1356-patch-2.7.6 + /nix/store/8z5vplmgshflm7yixhp8q7hy11xxxd8a-terraform-provider-elasticsearch-0.6.0-bin + /nix/store/91a8wnca647kfw67sk0iykdbyjpr8430-perl5.28.2-libwww-perl-6.38 + /nix/store/920nr51iw7qaplqjlqrlnql9g5ljq3vp-gdbm-1.18.1 + /nix/store/93rldbdly90q16lzk7hzilhk4qkdpqfq-keyutils-1.6-lib + /nix/store/94nrq9paz335s155x9za8n7kb0q3y211-libgcrypt-1.8.4 + /nix/store/95rd64vii9j6h31fcr9lba8m8940zfpj-terraform-provider-github-2.0.0-bin + /nix/store/97d3r4a7v1nal53x0gv17hrbbcp0rb21-util-linux-2.33.2-bin + /nix/store/9dk1gh07pwkvg62rns4k670h54bhfhgh-zlib-1.2.11-dev + /nix/store/9fvjgcjn1d0c9476qlr05psvwljwzq59-python3.7-cryptography-2.6.1 + /nix/store/9gjpg5lsdhgrhi805948c648nhn39l8z-terraform-provider-template-2.1.2-bin + /nix/store/9hysgvp7qrfcil4b5qhwdq2vm9hism13-libxcb-1.13.1 + /nix/store/9pa3p1rqhnvlrngaqsx09766cl1j6zf3-python3.7-httpretty-0.9.6 + /nix/store/9xb22l3577nznvd6dqqis6ixgmwq9ygh-python3.7-pycparser-2.19 + /nix/store/9yb9whkdgf3zyy85xac248kwq1wm6qd6-harfbuzz-2.3.1 + /nix/store/a185xh0jcx7il7hw2gfh0pmvrah3x67y-systemd-239.20190219-lib + /nix/store/a2cg0faxbwnicf41vwmw467jw7i9ix46-pinentry-1.1.0 + /nix/store/a2rr0irv2ssvvphvafgrxy4di0pkkagn-audit-2.8.5 + /nix/store/ag3fp30cz58ijm2yyy5adp1f3kw814b5-libXcomposite-0.4.5 + /nix/store/amlnqi4vvmpb9gjmyh1vr5hr2py12ss2-terraform-provider-docker-1.1.1-bin + /nix/store/ap4sr1n0wlgmybxbw3pvq8klh8snc3n8-sqlite-3.28.0 + /nix/store/apcaggi0q3vrb4ha1b07cjxiim2li5ly-gcc-7.4.0 + /nix/store/apjv9g35sklrab9lzz9r9rq7lnczv2wy-terraform-provider-triton-0.5.1-bin + /nix/store/aqgl1dqd6lr7jr9knqsyyq09bm0ibw7s-python3.7-cffi-1.12.3 + /nix/store/aqp0mrdbhvkm8rl1z0p2rkqnz6pbclhq-libXfixes-5.0.3 + /nix/store/b2rwzjp56yzd4jg2csx568h2dgj1l3l2-perl5.28.2-Try-Tiny-0.30 + /nix/store/b2wy5p5bykcnkwz5q1w8qq4qfzr4arc7-python3.7-MarkupSafe-1.1.1 + /nix/store/b6pdz8g070kbf0rdavjz6rip7sx06r8h-libkrb5-1.17-dev + /nix/store/b77nn2r1c8cqpr9prh1ldwr3m6xdrkpa-openldap-2.4.47 + /nix/store/b7w6bpx5z0ncy35kqxvmpg4lwrnc8jf2-libxml2-2.9.9 + /nix/store/bd1hz6m8gh9m91hikjhq7aiq2khdkx2r-fontconfig-2.12.6 + /nix/store/bf26sh99bngrnpzrj7gyz0689b060vak-terraform-provider-skytap-0.9.0-bin + /nix/store/bh412cii40qpzc20pzd48dq45jv9xm5a-perl5.28.2-HTTP-Date-6.02 + /nix/store/bvsihhp4jv61hz6mc17mn1sar03k0i8d-terraform-provider-telefonicaopencloud-1.0.0-bin + /nix/store/bxps2h6axpqrjxcmib344a3immy3gvhd-readline-6.3p08 + /nix/store/byxya0m4656ylf5imvs2v9p2c1av1kjl-terraform-provider-matchbox-0.2.3-bin + /nix/store/c7jkmfjhl3jkgnkrhh021vrqry7zplc1-linux-headers-4.19.16 + /nix/store/caa29d4y2zip0ly9mcc7f4w94blw8k60-lz4-1.9.1 + /nix/store/cd09qv56inq4gwa89656r4n0lq0vgidw-libXi-1.7.9 + /nix/store/ch6pz5kfg0bd3sfyf1813cpskg7lidns-python3.7-urllib3-1.24.2 + /nix/store/ck0lifb2jgkmg6c7frz7fxqwz5fbdnxk-terraform-provider-ultradns-0.1.0-bin + /nix/store/cncad2f4lfxfxnwd9lfhjjd89x3anxqr-terraform-provider-yandex-0.5.2-bin + /nix/store/csajl6aq80s9v2xbkmlzgfxlilmbzff6-terraform-provider-influxdb-1.1.0-bin + /nix/store/d0wcd9mid6067i6va19lwiv29hln6n2j-python3.7-requests-2.21.0 + /nix/store/d7rh74cch3ybp9r239j5c2c1rb0kx3pa-terraform-provider-chef-0.1.0-bin + /nix/store/daizqdqrm7g4favv814hnijmqhay8hs4-dbus-1.12.12 + /nix/store/di6rrbw1kbdrwxiymq91dgdvp2rvk1xv-dnsmasq-2.80 + /nix/store/drdzgwhnqjvq4g1aqsyz56c04k6dxnbi-bash-interactive-4.4-p23-doc + /nix/store/ds1prvgw3i3ic8c7axyrw4lwm3d0gqab-gcc-wrapper-7.4.0 + /nix/store/dvsw0fhfzqf4xg0q2idhs02rhwn4k8cv-libXrender-0.9.10 + /nix/store/dy437h3f5i500gv6znll974c87grzh3l-libXft-2.3.3 + /nix/store/dy5wi2sqnhbnlpvjr8a0z96id1mq243j-python3.7-six-1.12.0 + /nix/store/dz9dqcnz8v9cy54x5hax599zjwckp0kd-terraform-provider-gitlab-1.3.0-bin + /nix/store/f11valqiyik1ggdlnhg3ibwgrj1imidb-numactl-2.0.12 + /nix/store/f39sk2aim9xz7dzn7qvqh442xm58v77w-nfs-utils-2.3.3 + /nix/store/f3rbnn0jhm549mcp7k9ysjcq26j8fvyy-terraform-provider-segment-0.2.0-bin + /nix/store/f7zcmzqcavbj7bp1dlfk86f9bkqvk9p3-bridge-utils-1.5 + /nix/store/f7zh0d0n2dj4dcig903zd5jgb2cpaxf6-terraform-provider-ignition-1.0.1-bin + /nix/store/fhgiisqjpzwl8z4fchgc07avg1azmp0r-cyrus-sasl-2.1.27 + /nix/store/fky41n197z9n737kbih4j7ncnh7cngnz-perl5.28.2-Net-HTTP-6.18 + /nix/store/fq4765nh9p0si8mh9cnywsq48zr1qc27-terraform-provider-runscope-0.5.0-bin + /nix/store/fwvdxglj9asp4f90ihry29n2fm8a6i09-terraform-provider-aws-2.9.0-bin + /nix/store/g4qqgmrm254axgndybnpwg7s780bxy1a-numad-0.5 + /nix/store/g63hwfkd4yjncqh81ndn9vbmghdv41ng-terraform-provider-digitalocean-1.3.0 + /nix/store/gg469jh0m4dk4b0x6s44ziad69czbv22-python3.7-pycrypto-3.7.3 + /nix/store/gkpa27fykskx0dd52dca515gd91qhhgf-terraform-provider-local-1.2.2-bin + /nix/store/glrnpb3rkribnrjh5gzs24nmvl3m00cg-parted-3.2 + /nix/store/gn5cd1z252aip0rvds71g9mgfhh6i8p7-fontconfig-2.12.6-lib + /nix/store/gsl1dw8ycrdvlzczsl59mkz0qpbwcmz1-iptables-1.8.2 + /nix/store/gydzhj2y5j1ggbainbilvpxi5glw5hmf-terraform-provider-alicloud-1.41.0-bin + /nix/store/h02lb0p8krj1smsrid2n44ak058bbd82-expand-response-params + /nix/store/h40ib0qsa07b6ld1pv4x76xx2g7xgik6-terraform-provider-newrelic-1.5.0-bin + /nix/store/h8898ysg2s23k6palhxy9a5sbgrvvrcy-nfs-utils-2.3.3-lib + /nix/store/hbs2vrw1y8y1qz1hi71jaz0j3pl95qfs-terraform-provider-helm-0.9.1-bin + /nix/store/hg863b95fxv9zlk008qjyf87qgyx58h1-libseccomp-2.4.1-lib + /nix/store/hl8lzq90qjhq0n710lm5n17lc9i80vsh-terraform-provider-oraclepaas-1.5.2-bin + /nix/store/hnvmacd16kzmwcsavzkssrqj2kiryy2p-perl5.28.2-WWW-RobotRules-6.02 + /nix/store/hpmni5y805q7a07q9sn3nwjk4i2m2jl5-libkrb5-1.17 + /nix/store/hps5ziw9zq6mcjh9b7naaxawnqymws4m-jasper-2.0.16 + /nix/store/hy2xn2jxm4wp7j86p08m9xdpxncskdgv-terraform-provider-pagerduty-1.2.1-bin + /nix/store/i10riha5s5dgafznk3gwn36fyr3cpxb4-libXinerama-1.1.4 + /nix/store/i3kh8yq4kgkfn234pnwxnvxbrcgcckc8-curl-7.64.1-dev + /nix/store/i652w9gqbmc6k48lz3b02ncv7hpgc7nv-perl5.28.2-HTTP-Message-6.18 + /nix/store/i8859i082xqnrhzg7h6gz2ylc5wbw5pa-libnl-3.4.0 + /nix/store/id798ngchr83gc0mmqd3zlviljshjhvb-terraform-provider-nomad-1.3.0-bin + /nix/store/im1940h7b6pjlnh38q6lasdn8iybsv4v-python3.7-jmespath-0.9.4 + /nix/store/in7wgxanbdycb9wpq1j29928gllc0ap6-terraform-provider-profitbricks-1.4.4-bin + /nix/store/irqcza91k5smn6f4dyvqzw0zjn50d58f-libssh2-1.8.2 + /nix/store/isdbs6d2jk75kj0qk4s3prwlwcgkgalf-tf-plugin-env + /nix/store/ixycmxkr0wrz3gfxrnrdgcsk4gcyirpv-terraform-provider-azurestack-0.6.0-bin + /nix/store/j1px1l6vk39i3chghlwy9222jcjdfdq0-libcap-ng-0.7.9 + /nix/store/jf6lfawjvwr6ggnd4lhc5w4dp9v3kgh4-libXdmcp-1.1.3 + /nix/store/jjld4xam968mz645xh4g7i5zrnhsfyp9-terraform-provider-cobbler-1.0.1-bin + /nix/store/jn0bddfc3fzhnf5ns4s2khhzclswvzb2-libpcap-1.9.0 + /nix/store/jsg4bi31drwy614hdkkwf32m4wz3im6g-libassuan-2.5.3 + /nix/store/jszvy5lyyjbfi7mjr7s9bnbq9cyq858v-terraform-provider-external-1.1.2-bin + /nix/store/jvfc6r03a95aignzbfg987kspa04s0md-perl5.28.2-HTTP-Daemon-6.01 + /nix/store/jwwkky1pn1fw3yiaxmc5k3brb0rqlwvq-graphite2-1.3.6 + /nix/store/k1xlz5zy7rm2a428byz850c1igc2j1z8-readline-7.0p5 + /nix/store/k3qhzd0x8a1z6h5kyifnv3axbfs7fy66-terraform-provider-statuscake-0.2.0-bin + /nix/store/k4v5havnf7pmcv40xadh8mb7b0nbcgxz-libglvnd-1.0.0 + /nix/store/k59h7bs9307y7rb0z70vasvj8hd5pcn5-perl5.28.2-HTML-Tagset-3.20 + /nix/store/k5vljkz5p59nrh50vx5k2790ksqcxjpc-terraform-provider-dme-0.1.0-bin + /nix/store/k7g175rls2pk34m23wqhplv8mbnsc0lc-pixman-0.38.4 + /nix/store/kccb2k5hdjhdyxbxsri9lwwc4z1pvx6z-cups-2.2.11-lib + /nix/store/kds827ryxx16rwhrsdn9wnr2pxf5qaxm-terraform-provider-google-2.6.0-bin + /nix/store/kh18cbdb9f79gl58axwr8qq6c7bd0bl0-terraform-provider-acme-1.1.2-bin + /nix/store/kh2h5rnmm4gvjm8z7w2y511h15i7nhk9-gnum4-1.4.18 + /nix/store/kmscm0qm9j480wpd1yh42b1g0zc6qbmv-nghttp2-1.38.0-lib + /nix/store/kqmg1xxd3vi37bqh7gdvi61bkp7wb9hi-terraform-provider-circonus-0.2.0-bin + /nix/store/kvjcl6plvmkm6i2lzd7wrkbiis3b4vhg-terraform-provider-mysql-1.5.1-bin + /nix/store/kvy2sz5lvi89lnh4rmw1df4jsnhqf1ay-libnftnl-1.1.2 + /nix/store/l6ns1zcd18j9708y3agxgi0kihs4zc7i-terraform-provider-softlayer-0.0.1-bin + /nix/store/l8zqmzg19i62iz4argyjjr071rid3q9b-terraform-provider-cloudflare-1.13.0-bin + /nix/store/l9821zngvlh8bd6mlyzvi1mc754dyhjz-terraform-provider-libvirt-0.5.1-bin + /nix/store/lgrhsbfmpf1cjbpig8llxfrfb6xhz7xv-terraform-provider-scaleway-1.9.2-bin + /nix/store/lnzh3cjjcbafh6wsivw10wl60g7xplxj-libXrandr-1.5.2 + /nix/store/lr9yhdbn8a3la69j56cz0vi1qva973dv-kbd-2.0.4 + /nix/store/lswf09qbkkrqd0rgzaqyrkr44lf78y9x-libXext-1.3.4 + /nix/store/lvqp39d4hx776nkw3a0qfnvvjmnj49hc-procps-3.3.15 + /nix/store/m2n4drah6566qlccaabjhnnl4slql3cd-python3.7-pysocks-1.6.8 + /nix/store/m65jki67b02la5k5r9vgddcp13l32lw5-python3.7-httplib2-0.12.3 + /nix/store/m6hisb1d7q1c4z0s3icax40gynz4f8fl-gmp-6.1.2 + /nix/store/mmjbmvw64yl2756y1zvsxk0ic0nhzq2a-libnfnetlink-1.0.1 + /nix/store/mn0nzy294n07x1b92m9n0rwrv7z1441m-libunistring-0.9.10 + /nix/store/mp1hzpgp3sa6xac8dc7rldh5rab8lk2w-perl5.28.2-Test-Needs-0.002006 + /nix/store/mpnl3p6mzm71vci81r0h346jywm6863s-perl5.28.2-Encode-Locale-1.05 + /nix/store/mrclkdxryhjd6i36hlad6fwahjd14fmg-libpng-apng-1.6.37 + /nix/store/msa690459q4n9fiq125gsfambbd62qb4-libdaemon-0.14 + /nix/store/msfkr5yqdxjx5cm24pvn3q1552rsjn8h-libev-4.25 + /nix/store/mvmjdim7dn589inb8dsjxap08h4ip4h5-terraform-provider-huaweicloud-1.4.0-bin + /nix/store/n1dcmv0ii513dhlnllc790vfn8i9j9lj-python3.7-Jinja2-2.10.1 + /nix/store/n1mrfbzlh3cjm9mfyrp48pybl3sg4717-terraform-provider-vsphere-1.10.0-bin + /nix/store/n1y9i0bv0sg8n8759zd6smr2zjyn8jf3-python3.7-paramiko-2.4.2 + /nix/store/n2mzl8vljdksdqybihdy9mm5v7hm19q5-python3.7-idna-2.8 + /nix/store/n3rakrhhvi3bb0ffnjs51drmy157p51q-terraform-provider-vcd-2.1.0-bin + /nix/store/n6z00dm6a5fdv935v8bv59909ra51xli-npth-1.6 + /nix/store/n7wdfylfi5wnrjdg4im9v2q9gnl99mmb-terraform-provider-archive-1.2.2-bin + /nix/store/nc8x0pwchcc9xiv1nsj9idvpnfvkhh8p-terraform-provider-datadog-1.9.0-bin + /nix/store/ndbpc44lv43k7jnb0ip1qwk8f0slx685-bash-interactive-4.4-p23-dev + /nix/store/ndjjyr4rqibzkgs8w55bx2idhnckh39p-libidn2-2.1.1a + /nix/store/nfn8wgiqf418y444fch4bpf2ay6ca55i-curl-7.64.1-man + /nix/store/ni4nc256xs4f5hmhlhybxl8k40fwi5m3-libedit-20190324-3.1 + /nix/store/np4ikymr5fq5iknjfrwrgqmcsid4dmw9-terraform-provider-consul-2.3.0-bin + /nix/store/nq2x9w3rjd5l2yvlv328i19ljar8bdab-libidn-1.35 + /nix/store/nvyhmkghwxh5f1wiid27vzxa0ddx929p-python3.7-packaging-19.0 + /nix/store/nwhvl00i2wa4ms26lszk36vwir90jd3x-libvirt-4.10.0 + /nix/store/p00q64lbln1z9kfgpd2r6qhk0kc7i7w7-terraform-provider-oneandone-1.3.0-bin + /nix/store/p19dhs366b9zbbhs61xfw7d77sk9mkjr-terraform-provider-cloudscale-1.1.0-bin + /nix/store/p8s6295x84d594sxvzml8rsxqjdghmc5-cairo-1.16.0 + /nix/store/p9rjwvja55djz5g2qxyc9wzcpmska0ql-terraform-provider-dyn-1.1.0-bin + /nix/store/pb6r7dllpfw5cbhpmv2v2kms9a57r4v5-terraform-provider-tls-2.0.1-bin + /nix/store/pfd5p3pyfrkwxh42j491kkqgl8n9aa67-perl5.28.2-TermReadKey-2.38 + /nix/store/pk6r8sd18gmxns8r73qi2yrmzf4f4cp0-terraform-provider-arukas-1.1.0-bin + /nix/store/pknq6p5h43zm4r0dgjnfywql04hdv3js-atk-2.32.0 + /nix/store/pvsfn6d0byl3hfwnyfg21yivyj8iff8s-terraform-provider-opentelekomcloud-1.8.0-bin + /nix/store/pvzbhdzqm4i20v3flr5mf7yfs7n2lrvg-python3.7-dnspython-1.16.0 + /nix/store/pwkjsnbgb4mw0x5r5zh8s7c9wqryhmbl-dmidecode-3.2 + /nix/store/pwnppsfjfxibhwhf598l7mx31i8154j9-bison-3.3.2 + /nix/store/q0ndxs1vqdy5r749h5hhhbixgyf5yasx-terraform-provider-opsgenie-0.1.0-bin + /nix/store/qbwcp86aslamyhhmf2xx0l5d17dyg2jh-libmnl-1.0.4 + /nix/store/qd4j58ykdkg9yvy8kvgh0i00gacy0ldm-perl-5.28.2 + /nix/store/qgr66z24rfbb8cc965rr2sklh38p083n-git-crypt-0.6.0 + /nix/store/qhad1pgmn3z406pgk345281xb5zjqrkm-libelf-0.8.13 + /nix/store/qpj9rsal85rc94pizrkwb3c5nkivlfcl-p11-kit-0.23.14 + /nix/store/qq59cnpcbnp4p054ipbs54fv946r4qr8-python3.7-boto-2.49.0 + /nix/store/r3x6y48q13qwl9x1wwz37002b7fhyidv-python3.7-asn1crypto-0.24.0 + /nix/store/r54ql4g0hcxzp15sfjiagd1dmxh4s8n6-python3.7-bcrypt-3.1.6 + /nix/store/r6mrgd9k1jzzqrhphrg1qgxvgvbka7p8-pcre2-10.33 + /nix/store/r7lhx3aqyils26h7wbxbgf376c0n4ab5-libssh2-1.8.2-dev + /nix/store/rcn9d2q4mpapxf4qd54hkqz32ljhv0rw-util-linux-2.33.2 + /nix/store/rf104cwz7kaa51s49n4c2aar0jrrj8px-nghttp2-1.38.0-dev + /nix/store/ricz15lpkjrasc5cpzp6l60iwlc87wv3-avahi-0.7 + /nix/store/rj8xd9ajm3wqjz1vfkignlp011fss53q-bzip2-1.0.6.0.1-bin + /nix/store/rk78bh2s5yjpmgdhzqlf1hnj6ij0h20n-terraform-provider-tfe-0.8.2-bin + /nix/store/rl2z4bb7wkfp0g12ccqffh287qal1109-kmod-26 + /nix/store/rs21a235ix9v8y4hgazkzi6g1x5dqf7v-terraform-provider-netlify-0.1.0-bin + /nix/store/ryinn9xa3g8bn55nj1h54ypnlp9naq6i-stdenv-linux + /nix/store/rzfzb501miszas14xq6cr3c04m8kkdrb-terraform-0.11.14-bin + /nix/store/s3m5z3wxm94c0bfyjxmqf6i0gf1bpx90-libaio-0.3.110 + /nix/store/s5f3vpmig33nk4zyk228q55wdydd3pc2-python3-3.7.3 + /nix/store/s7p4iij8p4hi6bmc2bf3flyf6wa6yzrj-terraform-provider-dns-2.1.1-bin + /nix/store/s7rqxrfb631i53dfl90gac35095jyypq-util-linux-2.33.2-bin + /nix/store/scdscan356g79qb7cf776gy7src22zbl-terraform-provider-grafana-1.3.0-bin + /nix/store/sfrh0r54ykfzv62h17gi8hm6778j7k0l-libyaml-0.2.1 + /nix/store/shf6d1928fzxcaz6zh0bhcqv3xhvxhjd-terraform-provider-linode-1.6.0-bin + /nix/store/skkpbcqavjd8q0zmd94js6nz7pgbvpfl-mirrors-list + /nix/store/sm4yylq92rip64wdk3iniy91w48a90ia-openssl-1.0.2r + /nix/store/sn2cydjzikl3rws2nfa7pdvayb45brrd-terraform-provider-logicmonitor-1.2.0-bin + /nix/store/ssr1y1736h7c6p8vs76iyxwg5h889x7d-terraform-provider-dnsimple-0.1.0-bin + /nix/store/sw6n4yz49cz5vm4ggpk2l5j1vngac8j2-terraform-provider-secret-1.0.0-bin + /nix/store/sw6vlm5g6r6sivlncz7vh8ps7v7r22aa-terraform-provider-brightbox-1.1.2-bin + /nix/store/v31bag67wm17wkdg7zr9yi62c5028y59-libXau-1.0.9 + /nix/store/v5q3cnkjfy8rfacsjqn1nza93mbczgd5-gdk-pixbuf-2.38.1 + /nix/store/v69ld4vcgkr4i4giv1nzl4kax9zx1fpa-python3.7-pyasn1-0.4.5 + /nix/store/vbp6wnr2gyj50nabxgclkbqblmnwcnbg-terraform-provider-azuread-0.3.1-bin + /nix/store/vckbx0p1isjvmgjh7ppni3h87imazbzb-libcap-2.27-lib + /nix/store/vin4cp4m5af1mxkb2jqqi8xkf98ca2sv-python3.7-ansible-2.7.9 + /nix/store/vkdh5ibsmzj6p53krnqqz1pv620f42r0-terraform-provider-logentries-1.0.0-bin + /nix/store/vqvmd2r9pf9f74jqipbhrn7wksiiy1jf-pcsclite-1.8.25-bin + /nix/store/vrrs5p13mykyniglgfdsn8xii9b7s850-terraform-provider-powerdns-0.1.0-bin + /nix/store/vzjwz11r1yy02xv07vx2577pqizzx83n-nettle-3.4.1 + /nix/store/vzs0x1kaliybgk7yr9lrf6ad4x5v1k9y-libjpeg-turbo-2.0.2 + /nix/store/wbbwikfkc7fbida822a5z9b4xmsnwm3d-python3.7-chardet-3.0.4 + /nix/store/wccns8l8bav11z3xlhasmnkz383q1k9p-libnetfilter_conntrack-1.0.7 + /nix/store/wf5nv1gzrx378icqmjgwl2isg7s8ly80-lvm2-2.03.01 + /nix/store/wh8pg14c3ykmmnd342llbzjigahc54dw-terraform-provider-fastly-0.6.0-bin + /nix/store/wi2mn48l130r7wafvj757rvzfkla59if-pm-utils-1.4.1 + /nix/store/wi4jm555w0rc1daiy2sz9iwrpk6cb2d8-terraform-provider-ibm-0.11.1-bin + /nix/store/wkby8pdmg4vpvbl0d85gynh103k2h1ky-nghttp2-1.38.0 + /nix/store/wv8y2h4w7lxi9x6k8pzh3dxy7i4csfbm-terraform-provider-digitalocean-1.3.0-bin + /nix/store/wvccp35z40fj1v8xbz6czrm4bbiaqd45-perl5.28.2-HTTP-Negotiate-6.01 + /nix/store/wxdxzvaj3mcmf3i9yqwinpcb26iz3pas-perl5.28.2-Test-Fatal-0.014 + /nix/store/wy2alf2g85wym6i9n279d7a9nianx3is-curl-7.64.1-bin + /nix/store/wym0bhrfdx5ndfpx6y39c6j3pf2n6wak-libksba-1.3.5 + /nix/store/x1idgjd7vs75nj9s3krimbfjvh27n06d-terraform-provider-librato-0.1.0-bin + /nix/store/x3rijvjjrch1fjs60lrw9xb7pidp817f-gtk+-2.24.32 + /nix/store/x6zlzsjsd7m9mjkmxlp15vcay58g6a04-python3.7-pycryptodome-3.7.3 + /nix/store/x987jpq3yswa2aj51d63vwszfdm3r8ld-terraform-provider-atlas-0.1.1-bin + /nix/store/xav1lh5jlgrz7amaz7b5ghrz0c273lji-python3.7-pyOpenSSL-19.0.0 + /nix/store/xbcc8v50g7h7wqh892q7k22wb1ky8cg7-libevent-2.1.8 + /nix/store/xbwr4gb4zgjmcrbx82zlcp1jfgcz75ya-python3.7-cryptography-2.6.1-dev + /nix/store/xdhh337yhl93x33vzd9davinrbr9x9iz-libmicrohttpd-0.9.63 + /nix/store/xfgg45by0j5hxi1kdwh8x2pkfd67wwzd-nss-cacert-3.42.1 + /nix/store/xkv7la24vsyn9n23wc1izcmmp7aifzb3-terraform-provider-packet-2.1.0-bin + /nix/store/xmy3chnan9iiag9apm7dd825cmlkiiij-libusb-1.0.22 + /nix/store/xn8xmzxjx7i8wwfvbiwfgmv9mn4n45dk-terraform-provider-icinga2-0.2.0-bin + /nix/store/xs2k8driha83f9k017bkgch8lcl4z7w0-python3.7-ply-3.11 + /nix/store/xv9pfis6ixrsv7z1jrmgagi25qljvg0d-python3.7-pynacl-1.3.0 + /nix/store/xxwqa4rwfi97az8a6dl6vhqiyjvmnm9v-libsecret-0.18.8 + /nix/store/xyx7dm2b6ylcdp38pfwcip9ssx0fdzga-libtool-2.4.6-lib + /nix/store/xzzf5h2f02bwjcph28vly4wklp13wb0g-perl5.28.2-HTML-Parser-3.72 + /nix/store/y1v3g4m9bmmmvmw4z84m5fpmdy42lbr4-terraform-provider-selectel-2.1.0-bin + /nix/store/y20bvdwb0s95wa4gzhkkxd1xcc4c8whx-terraform-provider-postgresql-0.3.0-bin + /nix/store/y3x0fvlz4a30iajw3vd1rkg45vl3k15c-pcsclite-1.8.25 + /nix/store/y48is3y65apgyjwlwiyzjay1dw19l19a-dns-root-data-2019-01-11 + /nix/store/y5gfmqjp68h4fqq8z4p219pimm7ws49j-python3.7-cffi-1.12.3-dev + /nix/store/ybj0i3axzbkb4n4c4a5gz5lr3z70v7h3-bash-interactive-4.4-p23-man + /nix/store/ybsmcpfglj9fm5kjxyykbnwfjmhxrwfv-terraform-provider-google-beta-2.6.0-bin + /nix/store/yf4i32dx953p2dv2agfdyxdwg6ba0l61-python3.7-setuptools-41.0.1 + /nix/store/yfmvcf5cslq4f9kv0vlmxksdgvick22d-libgpg-error-1.36 + /nix/store/yh1w64xprl0rxmj8nvxpmikp6x3qvgdb-libffi-3.2.1 + /nix/store/yi01h84z709wf02id8hsb170z53wvk7r-glibc-2.27-dev + /nix/store/yik0kf8fgxjj1rj3s4bsrx1y6smz8rhx-cdrtools-3.02a06 + /nix/store/yjdcrd1lkzp8c7cawcpy40c4p3ngaw12-terraform-provider-rightscale-1.3.0-bin + /nix/store/ymn53mhfkkhs26qw708yv7bd7jmbp636-terraform-provider-nsxt-1.1.0-bin + /nix/store/yn7smb316b6d99wjw2v9fgxzbrqnq9jm-terraform-provider-gandi-1.0.0-bin + /nix/store/yr6qv6j9qrc03gl7bknw6p1fx1pzk0l9-terraform-provider-hcloud-1.9.0-bin + /nix/store/yrmrvha03pvdyi9ww2bi6xjpk5930sf8-glib-2.60.1 + /nix/store/yvqaj61p81kd4bq3dyw64idqj61rwpka-terraform-provider-hedvig-1.0.3-bin + /nix/store/z0sqda5bg0l4p0851nw05h7hii0jj1kr-python3.7-PyYAML-5.1 + /nix/store/z2darh83lb4rmsfnnyjc0hll51fyvj49-libSM-1.2.3 + /nix/store/z4vgv1w5cmd6p90grfgr1k6m87fydr3g-terraform-provider-ovh-0.3.0-bin + /nix/store/z68464p6aafah8b8xcybkwyhmqdf0jgx-gnupg-2.2.15 + /nix/store/z7jh25by0vv378gacjd01idi52dj688h-libtiff-4.0.10 + /nix/store/z7n5a3kwjylsgnc1d271j4dn41m6shpz-libtirpc-1.1.4 + /nix/store/z9fv0x6vwlx3xkcac3lg6v8g0figkx39-ncurses-6.1-20190112 + /nix/store/za8dm1xsfslzdk4j434w869bngrygici-perl5.28.2-URI-1.76 + /nix/store/zaq3w03j96qqhzpcfs9yacwa98sdsmiv-terraform-provider-oci-3.24.1-bin + /nix/store/zckvgwnyah69mmwn0g5vr4y85rmzwld9-libsodium-1.0.17 + /nix/store/zdfchj49phsp0sahcvdfp8ipc0chakg3-terraform-provider-rabbitmq-1.0.0-bin + /nix/store/zh1q7yvyaxlcmj3n6g0rrdaq0v73pp90-linux-pam-1.3.1 + /nix/store/znjkmjz0pgckxxzq0m9d17isnsd9s03q-cracklib-2.9.7 + /nix/store/zxdnpb673bf27hcqzvssv5629m4x5bjv-freetype-2.10.0 +copying path '/nix/store/drdzgwhnqjvq4g1aqsyz56c04k6dxnbi-bash-interactive-4.4-p23-doc' from 'https://cache.nixos.org'... +copying path '/nix/store/a2rr0irv2ssvvphvafgrxy4di0pkkagn-audit-2.8.5' from 'https://cache.nixos.org'... +copying path '/nix/store/8xy69pkisipvdmrpm1nmbi6qa2c6lhn0-bash-interactive-4.4-p23-info' from 'https://cache.nixos.org'... +copying path '/nix/store/ybj0i3axzbkb4n4c4a5gz5lr3z70v7h3-bash-interactive-4.4-p23-man' from 'https://cache.nixos.org'... +copying path '/nix/store/2dfjlvp38xzkyylwpavnh61azi0d168b-binutils-2.31.1' from 'https://cache.nixos.org'... +copying path '/nix/store/f7zcmzqcavbj7bp1dlfk86f9bkqvk9p3-bridge-utils-1.5' from 'https://cache.nixos.org'... +copying path '/nix/store/206dvjl6595dk40dli12ziv393ww54wl-bzip2-1.0.6.0.1' from 'https://cache.nixos.org'... +copying path '/nix/store/8aylwgi9nb6hsgz6620fzsllbc7h2rx1-c-ares-1.15.0' from 'https://cache.nixos.org'... +copying path '/nix/store/rj8xd9ajm3wqjz1vfkignlp011fss53q-bzip2-1.0.6.0.1-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/znjkmjz0pgckxxzq0m9d17isnsd9s03q-cracklib-2.9.7' from 'https://cache.nixos.org'... +copying path '/nix/store/nfn8wgiqf418y444fch4bpf2ay6ca55i-curl-7.64.1-man' from 'https://cache.nixos.org'... +copying path '/nix/store/6whclwjzwg46s0dkxwk1xz8cdcxnkd3y-db-4.8.30' from 'https://cache.nixos.org'... +copying path '/nix/store/2k46270d0h3gqj1c0wgx8prnj51jqryd-db-5.3.28' from 'https://cache.nixos.org'... +copying path '/nix/store/8gn2b5vvlazg608cj1y5l4igp9rckmnq-dejavu-fonts-minimal-2.37' from 'https://cache.nixos.org'... +copying path '/nix/store/pwkjsnbgb4mw0x5r5zh8s7c9wqryhmbl-dmidecode-3.2' from 'https://cache.nixos.org'... +copying path '/nix/store/y48is3y65apgyjwlwiyzjay1dw19l19a-dns-root-data-2019-01-11' from 'https://cache.nixos.org'... +copying path '/nix/store/8dl5c7n4555lr6qipki2424652gf8az8-ebtables-2.0.10-4' from 'https://cache.nixos.org'... +copying path '/nix/store/h02lb0p8krj1smsrid2n44ak058bbd82-expand-response-params' from 'https://cache.nixos.org'... +copying path '/nix/store/4jw2677fvb11aj1bal9a2iksqz0mk80m-expat-2.2.6' from 'https://cache.nixos.org'... +copying path '/nix/store/bd1hz6m8gh9m91hikjhq7aiq2khdkx2r-fontconfig-2.12.6' from 'https://cache.nixos.org'... +copying path '/nix/store/8ndwixznskf4zbf6h96ww4j8ap4j8067-fribidi-1.0.5' from 'https://cache.nixos.org'... +copying path '/nix/store/920nr51iw7qaplqjlqrlnql9g5ljq3vp-gdbm-1.18.1' from 'https://cache.nixos.org'... +copying path '/nix/store/m6hisb1d7q1c4z0s3icax40gynz4f8fl-gmp-6.1.2' from 'https://cache.nixos.org'... +copying path '/nix/store/kh2h5rnmm4gvjm8z7w2y511h15i7nhk9-gnum4-1.4.18' from 'https://cache.nixos.org'... +copying path '/nix/store/6554dpyahvcs49dmv434aky6bfkmqb30-gnumake-4.2.1' from 'https://cache.nixos.org'... +copying path '/nix/store/pwnppsfjfxibhwhf598l7mx31i8154j9-bison-3.3.2' from 'https://cache.nixos.org'... +copying path '/nix/store/jwwkky1pn1fw3yiaxmc5k3brb0rqlwvq-graphite2-1.3.6' from 'https://cache.nixos.org'... +copying path '/nix/store/6mz512j183wj7qas2qm6zkrks5k8rh00-gettext-0.19.8.1' from 'https://cache.nixos.org'... +copying path '/nix/store/4i1mw6av3d6pr9bqggb4hnv6cykbrhhi-kexec-tools-2.0.19' from 'https://cache.nixos.org'... +copying path '/nix/store/93rldbdly90q16lzk7hzilhk4qkdpqfq-keyutils-1.6-lib' from 'https://cache.nixos.org'... +copying path '/nix/store/rl2z4bb7wkfp0g12ccqffh287qal1109-kmod-26' from 'https://cache.nixos.org'... +copying path '/nix/store/4qq5hh1r6sqb0kpxc305rb468s45j4aw-libICE-1.0.9' from 'https://cache.nixos.org'... +copying path '/nix/store/v31bag67wm17wkdg7zr9yi62c5028y59-libXau-1.0.9' from 'https://cache.nixos.org'... +copying path '/nix/store/jf6lfawjvwr6ggnd4lhc5w4dp9v3kgh4-libXdmcp-1.1.3' from 'https://cache.nixos.org'... +copying path '/nix/store/s3m5z3wxm94c0bfyjxmqf6i0gf1bpx90-libaio-0.3.110' from 'https://cache.nixos.org'... +copying path '/nix/store/6b2jabk1scwhhk9bz7wjzycvmkiw419d-libapparmor-2.13.1' from 'https://cache.nixos.org'... +copying path '/nix/store/vckbx0p1isjvmgjh7ppni3h87imazbzb-libcap-2.27-lib' from 'https://cache.nixos.org'... +copying path '/nix/store/j1px1l6vk39i3chghlwy9222jcjdfdq0-libcap-ng-0.7.9' from 'https://cache.nixos.org'... +copying path '/nix/store/yik0kf8fgxjj1rj3s4bsrx1y6smz8rhx-cdrtools-3.02a06' from 'https://cache.nixos.org'... +copying path '/nix/store/msa690459q4n9fiq125gsfambbd62qb4-libdaemon-0.14' from 'https://cache.nixos.org'... +copying path '/nix/store/qhad1pgmn3z406pgk345281xb5zjqrkm-libelf-0.8.13' from 'https://cache.nixos.org'... +copying path '/nix/store/msfkr5yqdxjx5cm24pvn3q1552rsjn8h-libev-4.25' from 'https://cache.nixos.org'... +copying path '/nix/store/yh1w64xprl0rxmj8nvxpmikp6x3qvgdb-libffi-3.2.1' from 'https://cache.nixos.org'... +copying path '/nix/store/yfmvcf5cslq4f9kv0vlmxksdgvick22d-libgpg-error-1.36' from 'https://cache.nixos.org'... +copying path '/nix/store/0jl2dhydfh3jbfpkgkrixisqkhj12d4y-libffi-3.2.1-dev' from 'https://cache.nixos.org'... +copying path '/nix/store/jsg4bi31drwy614hdkkwf32m4wz3im6g-libassuan-2.5.3' from 'https://cache.nixos.org'... +copying path '/nix/store/94nrq9paz335s155x9za8n7kb0q3y211-libgcrypt-1.8.4' from 'https://cache.nixos.org'... +copying path '/nix/store/nq2x9w3rjd5l2yvlv328i19ljar8bdab-libidn-1.35' from 'https://cache.nixos.org'... +copying path '/nix/store/vzs0x1kaliybgk7yr9lrf6ad4x5v1k9y-libjpeg-turbo-2.0.2' from 'https://cache.nixos.org'... +copying path '/nix/store/hpmni5y805q7a07q9sn3nwjk4i2m2jl5-libkrb5-1.17' from 'https://cache.nixos.org'... +copying path '/nix/store/hps5ziw9zq6mcjh9b7naaxawnqymws4m-jasper-2.0.16' from 'https://cache.nixos.org'... +copying path '/nix/store/b6pdz8g070kbf0rdavjz6rip7sx06r8h-libkrb5-1.17-dev' from 'https://cache.nixos.org'... +copying path '/nix/store/wym0bhrfdx5ndfpx6y39c6j3pf2n6wak-libksba-1.3.5' from 'https://cache.nixos.org'... +copying path '/nix/store/qbwcp86aslamyhhmf2xx0l5d17dyg2jh-libmnl-1.0.4' from 'https://cache.nixos.org'... +copying path '/nix/store/mmjbmvw64yl2756y1zvsxk0ic0nhzq2a-libnfnetlink-1.0.1' from 'https://cache.nixos.org'... +copying path '/nix/store/kvy2sz5lvi89lnh4rmw1df4jsnhqf1ay-libnftnl-1.1.2' from 'https://cache.nixos.org'... +copying path '/nix/store/wccns8l8bav11z3xlhasmnkz383q1k9p-libnetfilter_conntrack-1.0.7' from 'https://cache.nixos.org'... +copying path '/nix/store/i8859i082xqnrhzg7h6gz2ylc5wbw5pa-libnl-3.4.0' from 'https://cache.nixos.org'... +copying path '/nix/store/jn0bddfc3fzhnf5ns4s2khhzclswvzb2-libpcap-1.9.0' from 'https://cache.nixos.org'... +copying path '/nix/store/7b7nbb0w2iwskwhzjhfwrif631h4smia-libpciaccess-0.14' from 'https://cache.nixos.org'... +copying path '/nix/store/gsl1dw8ycrdvlzczsl59mkz0qpbwcmz1-iptables-1.8.2' from 'https://cache.nixos.org'... +copying path '/nix/store/mrclkdxryhjd6i36hlad6fwahjd14fmg-libpng-apng-1.6.37' from 'https://cache.nixos.org'... +copying path '/nix/store/4z62pandn85xhcc5vazmi29cs2yps47b-iproute2-5.0.0' from 'https://cache.nixos.org'... +copying path '/nix/store/zxdnpb673bf27hcqzvssv5629m4x5bjv-freetype-2.10.0' from 'https://cache.nixos.org'... +copying path '/nix/store/hg863b95fxv9zlk008qjyf87qgyx58h1-libseccomp-2.4.1-lib' from 'https://cache.nixos.org'... +copying path '/nix/store/gn5cd1z252aip0rvds71g9mgfhh6i8p7-fontconfig-2.12.6-lib' from 'https://cache.nixos.org'... +copying path '/nix/store/zckvgwnyah69mmwn0g5vr4y85rmzwld9-libsodium-1.0.17' from 'https://cache.nixos.org'... +copying path '/nix/store/5l3967kll8m6s66zprzwb2p6vf2mh5yd-libtasn1-4.13' from 'https://cache.nixos.org'... +copying path '/nix/store/z7jh25by0vv378gacjd01idi52dj688h-libtiff-4.0.10' from 'https://cache.nixos.org'... +copying path '/nix/store/z7n5a3kwjylsgnc1d271j4dn41m6shpz-libtirpc-1.1.4' from 'https://cache.nixos.org'... +copying path '/nix/store/xyx7dm2b6ylcdp38pfwcip9ssx0fdzga-libtool-2.4.6-lib' from 'https://cache.nixos.org'... +copying path '/nix/store/mn0nzy294n07x1b92m9n0rwrv7z1441m-libunistring-0.9.10' from 'https://cache.nixos.org'... +copying path '/nix/store/9hysgvp7qrfcil4b5qhwdq2vm9hism13-libxcb-1.13.1' from 'https://cache.nixos.org'... +copying path '/nix/store/ndjjyr4rqibzkgs8w55bx2idhnckh39p-libidn2-2.1.1a' from 'https://cache.nixos.org'... +copying path '/nix/store/69vq0a9sqynmz335apm8zgyjdmq34s5j-libX11-1.6.7' from 'https://cache.nixos.org'... +copying path '/nix/store/b7w6bpx5z0ncy35kqxvmpg4lwrnc8jf2-libxml2-2.9.9' from 'https://cache.nixos.org'... +copying path '/nix/store/ag3fp30cz58ijm2yyy5adp1f3kw814b5-libXcomposite-0.4.5' from 'https://cache.nixos.org'... +copying path '/nix/store/lswf09qbkkrqd0rgzaqyrkr44lf78y9x-libXext-1.3.4' from 'https://cache.nixos.org'... +copying path '/nix/store/aqp0mrdbhvkm8rl1z0p2rkqnz6pbclhq-libXfixes-5.0.3' from 'https://cache.nixos.org'... +copying path '/nix/store/cd09qv56inq4gwa89656r4n0lq0vgidw-libXi-1.7.9' from 'https://cache.nixos.org'... +copying path '/nix/store/i10riha5s5dgafznk3gwn36fyr3cpxb4-libXinerama-1.1.4' from 'https://cache.nixos.org'... +copying path '/nix/store/dvsw0fhfzqf4xg0q2idhs02rhwn4k8cv-libXrender-0.9.10' from 'https://cache.nixos.org'... +copying path '/nix/store/k4v5havnf7pmcv40xadh8mb7b0nbcgxz-libglvnd-1.0.0' from 'https://cache.nixos.org'... +copying path '/nix/store/3s4fr71ykyw54kyyqavd0ba42klg0bhf-libXcursor-1.2.0' from 'https://cache.nixos.org'... +copying path '/nix/store/dy437h3f5i500gv6znll974c87grzh3l-libXft-2.3.3' from 'https://cache.nixos.org'... +copying path '/nix/store/lnzh3cjjcbafh6wsivw10wl60g7xplxj-libXrandr-1.5.2' from 'https://cache.nixos.org'... +copying path '/nix/store/sfrh0r54ykfzv62h17gi8hm6778j7k0l-libyaml-0.2.1' from 'https://cache.nixos.org'... +copying path '/nix/store/c7jkmfjhl3jkgnkrhh021vrqry7zplc1-linux-headers-4.19.16' from 'https://cache.nixos.org'... +copying path '/nix/store/zh1q7yvyaxlcmj3n6g0rrdaq0v73pp90-linux-pam-1.3.1' from 'https://cache.nixos.org'... +copying path '/nix/store/yi01h84z709wf02id8hsb170z53wvk7r-glibc-2.27-dev' from 'https://cache.nixos.org'... +copying path '/nix/store/lr9yhdbn8a3la69j56cz0vi1qva973dv-kbd-2.0.4' from 'https://cache.nixos.org'... +copying path '/nix/store/5qnlfx9qncn0fcw6mbfj6j58pz0cv0p3-binutils-wrapper-2.31.1' from 'https://cache.nixos.org'... +copying path '/nix/store/apcaggi0q3vrb4ha1b07cjxiim2li5ly-gcc-7.4.0' from 'https://cache.nixos.org'... +copying path '/nix/store/caa29d4y2zip0ly9mcc7f4w94blw8k60-lz4-1.9.1' from 'https://cache.nixos.org'... +copying path '/nix/store/ds1prvgw3i3ic8c7axyrw4lwm3d0gqab-gcc-wrapper-7.4.0' from 'https://cache.nixos.org'... +copying path '/nix/store/7mmn8ri08z48vfj69c2h66f3g349ilq1-mailcap-2.1.48' from 'https://cache.nixos.org'... +copying path '/nix/store/skkpbcqavjd8q0zmd94js6nz7pgbvpfl-mirrors-list' from 'https://cache.nixos.org'... +copying path '/nix/store/z9fv0x6vwlx3xkcac3lg6v8g0figkx39-ncurses-6.1-20190112' from 'https://cache.nixos.org'... +copying path '/nix/store/vzjwz11r1yy02xv07vx2577pqizzx83n-nettle-3.4.1' from 'https://cache.nixos.org'... +copying path '/nix/store/ni4nc256xs4f5hmhlhybxl8k40fwi5m3-libedit-20190324-3.1' from 'https://cache.nixos.org'... +copying path '/nix/store/h8898ysg2s23k6palhxy9a5sbgrvvrcy-nfs-utils-2.3.3-lib' from 'https://cache.nixos.org'... +copying path '/nix/store/wkby8pdmg4vpvbl0d85gynh103k2h1ky-nghttp2-1.38.0' from 'https://cache.nixos.org'... +copying path '/nix/store/kmscm0qm9j480wpd1yh42b1g0zc6qbmv-nghttp2-1.38.0-lib' from 'https://cache.nixos.org'... +copying path '/nix/store/n6z00dm6a5fdv935v8bv59909ra51xli-npth-1.6' from 'https://cache.nixos.org'... +copying path '/nix/store/xfgg45by0j5hxi1kdwh8x2pkfd67wwzd-nss-cacert-3.42.1' from 'https://cache.nixos.org'... +copying path '/nix/store/f11valqiyik1ggdlnhg3ibwgrj1imidb-numactl-2.0.12' from 'https://cache.nixos.org'... +copying path '/nix/store/g4qqgmrm254axgndybnpwg7s780bxy1a-numad-0.5' from 'https://cache.nixos.org'... +copying path '/nix/store/sm4yylq92rip64wdk3iniy91w48a90ia-openssl-1.0.2r' from 'https://cache.nixos.org'... +copying path '/nix/store/qpj9rsal85rc94pizrkwb3c5nkivlfcl-p11-kit-0.23.14' from 'https://cache.nixos.org'... +copying path '/nix/store/fhgiisqjpzwl8z4fchgc07avg1azmp0r-cyrus-sasl-2.1.27' from 'https://cache.nixos.org'... +copying path '/nix/store/xbcc8v50g7h7wqh892q7k22wb1ky8cg7-libevent-2.1.8' from 'https://cache.nixos.org'... +copying path '/nix/store/irqcza91k5smn6f4dyvqzw0zjn50d58f-libssh2-1.8.2' from 'https://cache.nixos.org'... +copying path '/nix/store/18rr3rg32imsnfyx6zb6s8lc8qpkdr74-nghttp2-1.38.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/5pjazw71xk4kysxrzacgjl4iai691k25-curl-7.64.1' from 'https://cache.nixos.org'... +copying path '/nix/store/r7lhx3aqyils26h7wbxbgf376c0n4ab5-libssh2-1.8.2-dev' from 'https://cache.nixos.org'... +copying path '/nix/store/wy2alf2g85wym6i9n279d7a9nianx3is-curl-7.64.1-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/rf104cwz7kaa51s49n4c2aar0jrrj8px-nghttp2-1.38.0-dev' from 'https://cache.nixos.org'... +copying path '/nix/store/b77nn2r1c8cqpr9prh1ldwr3m6xdrkpa-openldap-2.4.47' from 'https://cache.nixos.org'... +copying path '/nix/store/2ar3zk5fjr34ys2dqnsfbb678x6fdlj4-openssh-7.9p1' from 'https://cache.nixos.org'... +copying path '/nix/store/8yy3wngrdcpmjyw2ryas1y4wwhbd1356-patch-2.7.6' from 'https://cache.nixos.org'... +copying path '/nix/store/61shjilahl0d237fg9b3z3chza2lgms4-patchelf-0.9' from 'https://cache.nixos.org'... +copying path '/nix/store/r6mrgd9k1jzzqrhphrg1qgxvgvbka7p8-pcre2-10.33' from 'https://cache.nixos.org'... +copying path '/nix/store/qd4j58ykdkg9yvy8kvgh0i00gacy0ldm-perl-5.28.2' from 'https://cache.nixos.org'... +copying path '/nix/store/mpnl3p6mzm71vci81r0h346jywm6863s-perl5.28.2-Encode-Locale-1.05' from 'https://cache.nixos.org'... +copying path '/nix/store/85hh7apv9n3gganpnnq36zvlwm126mdh-openssl-1.0.2r-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/k59h7bs9307y7rb0z70vasvj8hd5pcn5-perl5.28.2-HTML-Tagset-3.20' from 'https://cache.nixos.org'... +copying path '/nix/store/3mgn9jnjzj1rgxclbixk5xa0kkx9xpw3-openssl-1.0.2r-dev' from 'https://cache.nixos.org'... +copying path '/nix/store/xzzf5h2f02bwjcph28vly4wklp13wb0g-perl5.28.2-HTML-Parser-3.72' from 'https://cache.nixos.org'... +copying path '/nix/store/bh412cii40qpzc20pzd48dq45jv9xm5a-perl5.28.2-HTTP-Date-6.02' from 'https://cache.nixos.org'... +copying path '/nix/store/5x1551gw825apcsnwx8gzfnmiapbz8yl-perl5.28.2-IO-HTML-1.001' from 'https://cache.nixos.org'... +copying path '/nix/store/0jwyd55g8nfhm25a0bh1j1by6afdriic-perl5.28.2-File-Listing-6.04' from 'https://cache.nixos.org'... +copying path '/nix/store/50rywa1m6asdz1y78a6dpa0xf98vm01v-perl5.28.2-LWP-MediaTypes-6.04' from 'https://cache.nixos.org'... +copying path '/nix/store/pfd5p3pyfrkwxh42j491kkqgl8n9aa67-perl5.28.2-TermReadKey-2.38' from 'https://cache.nixos.org'... +copying path '/nix/store/mp1hzpgp3sa6xac8dc7rldh5rab8lk2w-perl5.28.2-Test-Needs-0.002006' from 'https://cache.nixos.org'... +copying path '/nix/store/8vqr6vbvyzlpsk6q0mnj93sf5j1wr5qa-perl5.28.2-Test-RequiresInternet-0.05' from 'https://cache.nixos.org'... +copying path '/nix/store/b2rwzjp56yzd4jg2csx568h2dgj1l3l2-perl5.28.2-Try-Tiny-0.30' from 'https://cache.nixos.org'... +copying path '/nix/store/za8dm1xsfslzdk4j434w869bngrygici-perl5.28.2-URI-1.76' from 'https://cache.nixos.org'... +copying path '/nix/store/wxdxzvaj3mcmf3i9yqwinpcb26iz3pas-perl5.28.2-Test-Fatal-0.014' from 'https://cache.nixos.org'... +copying path '/nix/store/i652w9gqbmc6k48lz3b02ncv7hpgc7nv-perl5.28.2-HTTP-Message-6.18' from 'https://cache.nixos.org'... +copying path '/nix/store/fky41n197z9n737kbih4j7ncnh7cngnz-perl5.28.2-Net-HTTP-6.18' from 'https://cache.nixos.org'... +copying path '/nix/store/7gwvcm8dc24vnphbx85q1afaxhfhac28-perl5.28.2-HTTP-Cookies-6.04' from 'https://cache.nixos.org'... +copying path '/nix/store/jvfc6r03a95aignzbfg987kspa04s0md-perl5.28.2-HTTP-Daemon-6.01' from 'https://cache.nixos.org'... +copying path '/nix/store/wvccp35z40fj1v8xbz6czrm4bbiaqd45-perl5.28.2-HTTP-Negotiate-6.01' from 'https://cache.nixos.org'... +copying path '/nix/store/hnvmacd16kzmwcsavzkssrqj2kiryy2p-perl5.28.2-WWW-RobotRules-6.02' from 'https://cache.nixos.org'... +copying path '/nix/store/k7g175rls2pk34m23wqhplv8mbnsc0lc-pixman-0.38.4' from 'https://cache.nixos.org'... +copying path '/nix/store/91a8wnca647kfw67sk0iykdbyjpr8430-perl5.28.2-libwww-perl-6.38' from 'https://cache.nixos.org'... +copying path '/nix/store/xav1lh5jlgrz7amaz7b5ghrz0c273lji-python3.7-pyOpenSSL-19.0.0' from 'https://cache.nixos.org'... +copying path '/nix/store/43i41p1n1sxssmqpf9jp5x4gcy6r2fl6-git-2.21.0' from 'https://cache.nixos.org'... +copying path '/nix/store/bxps2h6axpqrjxcmib344a3immy3gvhd-readline-6.3p08' from 'https://cache.nixos.org'... +copying path '/nix/store/k1xlz5zy7rm2a428byz850c1igc2j1z8-readline-7.0p5' from 'https://cache.nixos.org'... +copying path '/nix/store/2xhsrw4ws6kc4x3983wdwwlnim27c6iz-shadow-4.6' from 'https://cache.nixos.org'... +copying path '/nix/store/715lcljfyp8grxlmaf51pn0n3ml3dwgg-bash-interactive-4.4-p23' from 'https://cache.nixos.org'... +copying path '/nix/store/ap4sr1n0wlgmybxbw3pvq8klh8snc3n8-sqlite-3.28.0' from 'https://cache.nixos.org'... +copying path '/nix/store/ndbpc44lv43k7jnb0ip1qwk8f0slx685-bash-interactive-4.4-p23-dev' from 'https://cache.nixos.org'... +copying path '/nix/store/3xzkc4wyadr3vrva2q320axjr6cyb43n-python-2.7.16' from 'https://cache.nixos.org'... +copying path '/nix/store/s5f3vpmig33nk4zyk228q55wdydd3pc2-python3-3.7.3' from 'https://cache.nixos.org'... +copying path '/nix/store/0w6l8kh3d30kg3nxc8xyi84gmrfxjnns-git-2.21.0' from 'https://cache.nixos.org'... +copying path '/nix/store/aqgl1dqd6lr7jr9knqsyyq09bm0ibw7s-python3.7-cffi-1.12.3' from 'https://cache.nixos.org'... +copying path '/nix/store/9fvjgcjn1d0c9476qlr05psvwljwzq59-python3.7-cryptography-2.6.1' from 'https://cache.nixos.org'... +copying path '/nix/store/yf4i32dx953p2dv2agfdyxdwg6ba0l61-python3.7-setuptools-41.0.1' from 'https://cache.nixos.org'... +copying path '/nix/store/7crry947d1xvp1f15c6q089l0gcy5hpc-stdenv-linux' from 'https://cache.nixos.org'... +copying path '/nix/store/b2wy5p5bykcnkwz5q1w8qq4qfzr4arc7-python3.7-MarkupSafe-1.1.1' from 'https://cache.nixos.org'... +copying path '/nix/store/z0sqda5bg0l4p0851nw05h7hii0jj1kr-python3.7-PyYAML-5.1' from 'https://cache.nixos.org'... +copying path '/nix/store/n1dcmv0ii513dhlnllc790vfn8i9j9lj-python3.7-Jinja2-2.10.1' from 'https://cache.nixos.org'... +copying path '/nix/store/r3x6y48q13qwl9x1wwz37002b7fhyidv-python3.7-asn1crypto-0.24.0' from 'https://cache.nixos.org'... +copying path '/nix/store/02nzlzdw0kiici9368jp5s84cpbqxkva-python3.7-certifi-2018.11.29' from 'https://cache.nixos.org'... +copying path '/nix/store/wbbwikfkc7fbida822a5z9b4xmsnwm3d-python3.7-chardet-3.0.4' from 'https://cache.nixos.org'... +copying path '/nix/store/pvzbhdzqm4i20v3flr5mf7yfs7n2lrvg-python3.7-dnspython-1.16.0' from 'https://cache.nixos.org'... +copying path '/nix/store/m65jki67b02la5k5r9vgddcp13l32lw5-python3.7-httplib2-0.12.3' from 'https://cache.nixos.org'... +copying path '/nix/store/n2mzl8vljdksdqybihdy9mm5v7hm19q5-python3.7-idna-2.8' from 'https://cache.nixos.org'... +copying path '/nix/store/31l04a1yxxdbdpzdp8mpfk96rhj3bg2c-python3.7-netaddr-0.7.19' from 'https://cache.nixos.org'... +copying path '/nix/store/xs2k8driha83f9k017bkgch8lcl4z7w0-python3.7-ply-3.11' from 'https://cache.nixos.org'... +copying path '/nix/store/v69ld4vcgkr4i4giv1nzl4kax9zx1fpa-python3.7-pyasn1-0.4.5' from 'https://cache.nixos.org'... +copying path '/nix/store/im1940h7b6pjlnh38q6lasdn8iybsv4v-python3.7-jmespath-0.9.4' from 'https://cache.nixos.org'... +copying path '/nix/store/9xb22l3577nznvd6dqqis6ixgmwq9ygh-python3.7-pycparser-2.19' from 'https://cache.nixos.org'... +copying path '/nix/store/x6zlzsjsd7m9mjkmxlp15vcay58g6a04-python3.7-pycryptodome-3.7.3' from 'https://cache.nixos.org'... +copying path '/nix/store/y5gfmqjp68h4fqq8z4p219pimm7ws49j-python3.7-cffi-1.12.3-dev' from 'https://cache.nixos.org'... +copying path '/nix/store/gg469jh0m4dk4b0x6s44ziad69czbv22-python3.7-pycrypto-3.7.3' from 'https://cache.nixos.org'... +copying path '/nix/store/5ydkc9jcaaxlz58dr7gvyhi3gcmafsfy-python3.7-pyparsing-2.3.1' from 'https://cache.nixos.org'... +copying path '/nix/store/m2n4drah6566qlccaabjhnnl4slql3cd-python3.7-pysocks-1.6.8' from 'https://cache.nixos.org'... +copying path '/nix/store/dy5wi2sqnhbnlpvjr8a0z96id1mq243j-python3.7-six-1.12.0' from 'https://cache.nixos.org'... +copying path '/nix/store/ryinn9xa3g8bn55nj1h54ypnlp9naq6i-stdenv-linux' from 'https://cache.nixos.org'... +copying path '/nix/store/r54ql4g0hcxzp15sfjiagd1dmxh4s8n6-python3.7-bcrypt-3.1.6' from 'https://cache.nixos.org'... +copying path '/nix/store/9pa3p1rqhnvlrngaqsx09766cl1j6zf3-python3.7-httpretty-0.9.6' from 'https://cache.nixos.org'... +copying path '/nix/store/nvyhmkghwxh5f1wiid27vzxa0ddx929p-python3.7-packaging-19.0' from 'https://cache.nixos.org'... +copying path '/nix/store/xv9pfis6ixrsv7z1jrmgagi25qljvg0d-python3.7-pynacl-1.3.0' from 'https://cache.nixos.org'... +copying path '/nix/store/xbwr4gb4zgjmcrbx82zlcp1jfgcz75ya-python3.7-cryptography-2.6.1-dev' from 'https://cache.nixos.org'... +copying path '/nix/store/1kz91g5mfj271lj5kxz2m1axcs2yqafy-thin-provisioning-tools-0.7.6' from 'https://cache.nixos.org'... +copying path '/nix/store/n1y9i0bv0sg8n8759zd6smr2zjyn8jf3-python3.7-paramiko-2.4.2' from 'https://cache.nixos.org'... +copying path '/nix/store/7256h1y98mmzsckwk2x7i3v3cxmvgrmq-python3.7-pyOpenSSL-19.0.0-dev' from 'https://cache.nixos.org'... +copying path '/nix/store/20wmykp8fj2izxdj8lic8ggcfpdid5ka-tzdata-2019a' from 'https://cache.nixos.org'... +copying path '/nix/store/ch6pz5kfg0bd3sfyf1813cpskg7lidns-python3.7-urllib3-1.24.2' from 'https://cache.nixos.org'... +copying path '/nix/store/749qksf79hvn0aprcznd9bwfv550qwh3-go-1.12.1' from 'https://cache.nixos.org'... +copying path '/nix/store/d0wcd9mid6067i6va19lwiv29hln6n2j-python3.7-requests-2.21.0' from 'https://cache.nixos.org'... +copying path '/nix/store/rzfzb501miszas14xq6cr3c04m8kkdrb-terraform-0.11.14-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/qq59cnpcbnp4p054ipbs54fv946r4qr8-python3.7-boto-2.49.0' from 'https://cache.nixos.org'... +copying path '/nix/store/479dvd7q6c18l3jl2myhfxmfsjbqjjch-python3.7-dopy-2016-01-04' from 'https://cache.nixos.org'... +copying path '/nix/store/kh18cbdb9f79gl58axwr8qq6c7bd0bl0-terraform-provider-acme-1.1.2-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/vin4cp4m5af1mxkb2jqqi8xkf98ca2sv-python3.7-ansible-2.7.9' from 'https://cache.nixos.org'... +copying path '/nix/store/gydzhj2y5j1ggbainbilvpxi5glw5hmf-terraform-provider-alicloud-1.41.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/0rc1jyfbxwffmsphyv2pfnxd6smysc1l-terraform-provider-ansible-0.0.4-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/n7wdfylfi5wnrjdg4im9v2q9gnl99mmb-terraform-provider-archive-1.2.2-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/pk6r8sd18gmxns8r73qi2yrmzf4f4cp0-terraform-provider-arukas-1.1.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/x987jpq3yswa2aj51d63vwszfdm3r8ld-terraform-provider-atlas-0.1.1-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/fwvdxglj9asp4f90ihry29n2fm8a6i09-terraform-provider-aws-2.9.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/vbp6wnr2gyj50nabxgclkbqblmnwcnbg-terraform-provider-azuread-0.3.1-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/7mqpbfs391s9hbnfzkpgw3inj8mkldr8-terraform-provider-azurerm-1.27.1-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/ixycmxkr0wrz3gfxrnrdgcsk4gcyirpv-terraform-provider-azurestack-0.6.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/1hml3hx7qlbkv139khazb24jh69nngcd-terraform-provider-bigip-0.12.2-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/63k736kr346ncpzv5yiqiyyyiqpa2h8m-terraform-provider-bitbucket-1.0.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/sw6vlm5g6r6sivlncz7vh8ps7v7r22aa-terraform-provider-brightbox-1.1.2-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/d7rh74cch3ybp9r239j5c2c1rb0kx3pa-terraform-provider-chef-0.1.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/kqmg1xxd3vi37bqh7gdvi61bkp7wb9hi-terraform-provider-circonus-0.2.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/7k3hvg4sfpr6y2bg8b7x9mkb0d2p3scr-terraform-provider-clc-0.1.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/l8zqmzg19i62iz4argyjjr071rid3q9b-terraform-provider-cloudflare-1.13.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/p19dhs366b9zbbhs61xfw7d77sk9mkjr-terraform-provider-cloudscale-1.1.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/0z8i6sq8mg138qnifr1z37y780xkk8hf-terraform-provider-cloudstack-0.2.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/jjld4xam968mz645xh4g7i5zrnhsfyp9-terraform-provider-cobbler-1.0.1-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/np4ikymr5fq5iknjfrwrgqmcsid4dmw9-terraform-provider-consul-2.3.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/nc8x0pwchcc9xiv1nsj9idvpnfvkhh8p-terraform-provider-datadog-1.9.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/wv8y2h4w7lxi9x6k8pzh3dxy7i4csfbm-terraform-provider-digitalocean-1.3.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/k5vljkz5p59nrh50vx5k2790ksqcxjpc-terraform-provider-dme-0.1.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/g63hwfkd4yjncqh81ndn9vbmghdv41ng-terraform-provider-digitalocean-1.3.0' from 'https://cache.nixos.org'... +copying path '/nix/store/s7p4iij8p4hi6bmc2bf3flyf6wa6yzrj-terraform-provider-dns-2.1.1-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/ssr1y1736h7c6p8vs76iyxwg5h889x7d-terraform-provider-dnsimple-0.1.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/amlnqi4vvmpb9gjmyh1vr5hr2py12ss2-terraform-provider-docker-1.1.1-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/p9rjwvja55djz5g2qxyc9wzcpmska0ql-terraform-provider-dyn-1.1.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/8z5vplmgshflm7yixhp8q7hy11xxxd8a-terraform-provider-elasticsearch-0.6.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/jszvy5lyyjbfi7mjr7s9bnbq9cyq858v-terraform-provider-external-1.1.2-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/wh8pg14c3ykmmnd342llbzjigahc54dw-terraform-provider-fastly-0.6.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/0bdf1xn7p6xzk008yr6cahq3wjlvah5g-terraform-provider-flexibleengine-1.5.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/yn7smb316b6d99wjw2v9fgxzbrqnq9jm-terraform-provider-gandi-1.0.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/95rd64vii9j6h31fcr9lba8m8940zfpj-terraform-provider-github-2.0.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/dz9dqcnz8v9cy54x5hax599zjwckp0kd-terraform-provider-gitlab-1.3.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/kds827ryxx16rwhrsdn9wnr2pxf5qaxm-terraform-provider-google-2.6.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/ybsmcpfglj9fm5kjxyykbnwfjmhxrwfv-terraform-provider-google-beta-2.6.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/scdscan356g79qb7cf776gy7src22zbl-terraform-provider-grafana-1.3.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/yr6qv6j9qrc03gl7bknw6p1fx1pzk0l9-terraform-provider-hcloud-1.9.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/yvqaj61p81kd4bq3dyw64idqj61rwpka-terraform-provider-hedvig-1.0.3-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/hbs2vrw1y8y1qz1hi71jaz0j3pl95qfs-terraform-provider-helm-0.9.1-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/5nkxpwdgpxs97yqh2fxz9y0rm80rc280-terraform-provider-heroku-1.9.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/2yy3pv77rwbxk7b2mpysmiqdzhmgmphg-terraform-provider-http-1.1.1-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/mvmjdim7dn589inb8dsjxap08h4ip4h5-terraform-provider-huaweicloud-1.4.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/wi4jm555w0rc1daiy2sz9iwrpk6cb2d8-terraform-provider-ibm-0.11.1-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/xn8xmzxjx7i8wwfvbiwfgmv9mn4n45dk-terraform-provider-icinga2-0.2.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/f7zh0d0n2dj4dcig903zd5jgb2cpaxf6-terraform-provider-ignition-1.0.1-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/csajl6aq80s9v2xbkmlzgfxlilmbzff6-terraform-provider-influxdb-1.1.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/5k67y2lglsnswrya21z51d4h87a081k5-terraform-provider-kubernetes-1.6.2-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/x1idgjd7vs75nj9s3krimbfjvh27n06d-terraform-provider-librato-0.1.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/shf6d1928fzxcaz6zh0bhcqv3xhvxhjd-terraform-provider-linode-1.6.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/gkpa27fykskx0dd52dca515gd91qhhgf-terraform-provider-local-1.2.2-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/vkdh5ibsmzj6p53krnqqz1pv620f42r0-terraform-provider-logentries-1.0.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/sn2cydjzikl3rws2nfa7pdvayb45brrd-terraform-provider-logicmonitor-1.2.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/3qjz5kfri8sa0dj1213rap75alpqsm2l-terraform-provider-mailgun-0.1.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/byxya0m4656ylf5imvs2v9p2c1av1kjl-terraform-provider-matchbox-0.2.3-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/kvjcl6plvmkm6i2lzd7wrkbiis3b4vhg-terraform-provider-mysql-1.5.1-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/rs21a235ix9v8y4hgazkzi6g1x5dqf7v-terraform-provider-netlify-0.1.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/h40ib0qsa07b6ld1pv4x76xx2g7xgik6-terraform-provider-newrelic-1.5.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/89wg3f6hk41gxm4n6cikj6r7gr2k7h8j-terraform-provider-nixos-0.0.1-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/id798ngchr83gc0mmqd3zlviljshjhvb-terraform-provider-nomad-1.3.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/8krs5vbid0ic6vvlvjvndvjb815q8hbd-terraform-provider-ns1-1.3.1-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/ymn53mhfkkhs26qw708yv7bd7jmbp636-terraform-provider-nsxt-1.1.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/5z3s6zbi98gh8cfliaplnmv15j568c46-terraform-provider-null-2.1.2-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/6brahzfjri338n3fggplfrsmf63mrwnx-terraform-provider-nutanix-1.0.1-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/zaq3w03j96qqhzpcfs9yacwa98sdsmiv-terraform-provider-oci-3.24.1-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/p00q64lbln1z9kfgpd2r6qhk0kc7i7w7-terraform-provider-oneandone-1.3.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/50wggbbr0wdg21hrvl4icwlppvk4464b-terraform-provider-opc-1.3.6-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/1wh5wgw6a3w91mk2avvn9ssw32nlw9kd-terraform-provider-openstack-1.18.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/pvsfn6d0byl3hfwnyfg21yivyj8iff8s-terraform-provider-opentelekomcloud-1.8.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/q0ndxs1vqdy5r749h5hhhbixgyf5yasx-terraform-provider-opsgenie-0.1.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/hl8lzq90qjhq0n710lm5n17lc9i80vsh-terraform-provider-oraclepaas-1.5.2-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/z4vgv1w5cmd6p90grfgr1k6m87fydr3g-terraform-provider-ovh-0.3.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/xkv7la24vsyn9n23wc1izcmmp7aifzb3-terraform-provider-packet-2.1.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/hy2xn2jxm4wp7j86p08m9xdpxncskdgv-terraform-provider-pagerduty-1.2.1-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/5lcz7p2xz1zp8iyd9yjmrg1kxw5yygnx-terraform-provider-panos-1.5.1-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/y20bvdwb0s95wa4gzhkkxd1xcc4c8whx-terraform-provider-postgresql-0.3.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/vrrs5p13mykyniglgfdsn8xii9b7s850-terraform-provider-powerdns-0.1.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/in7wgxanbdycb9wpq1j29928gllc0ap6-terraform-provider-profitbricks-1.4.4-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/zdfchj49phsp0sahcvdfp8ipc0chakg3-terraform-provider-rabbitmq-1.0.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/15fv1623h1vcn5z0nq42v5rgjirbp5r0-terraform-provider-rancher-1.2.1-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/7axz4xwz0vfrdgjyk59xg998bdqbvg5x-terraform-provider-random-2.1.2-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/yjdcrd1lkzp8c7cawcpy40c4p3ngaw12-terraform-provider-rightscale-1.3.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/2lh08897y86kxvyjdd1vlnkg8fz88nkd-terraform-provider-rundeck-0.1.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/fq4765nh9p0si8mh9cnywsq48zr1qc27-terraform-provider-runscope-0.5.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/lgrhsbfmpf1cjbpig8llxfrfb6xhz7xv-terraform-provider-scaleway-1.9.2-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/sw6n4yz49cz5vm4ggpk2l5j1vngac8j2-terraform-provider-secret-1.0.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/f3rbnn0jhm549mcp7k9ysjcq26j8fvyy-terraform-provider-segment-0.2.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/y1v3g4m9bmmmvmw4z84m5fpmdy42lbr4-terraform-provider-selectel-2.1.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/bf26sh99bngrnpzrj7gyz0689b060vak-terraform-provider-skytap-0.9.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/l6ns1zcd18j9708y3agxgi0kihs4zc7i-terraform-provider-softlayer-0.0.1-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/5ygnx64lyv5a8pnpmlj7bs8s2dz2hkxd-terraform-provider-spotinst-1.13.2-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/k3qhzd0x8a1z6h5kyifnv3axbfs7fy66-terraform-provider-statuscake-0.2.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/bvsihhp4jv61hz6mc17mn1sar03k0i8d-terraform-provider-telefonicaopencloud-1.0.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/9gjpg5lsdhgrhi805948c648nhn39l8z-terraform-provider-template-2.1.2-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/6hv1yfwyydyg2lzqcllwjb68xl4mrppw-terraform-provider-tencentcloud-1.5.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/7fd40sykaxj6dvya7mvif3f16wrqijr9-terraform-provider-terraform-1.0.2-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/rk78bh2s5yjpmgdhzqlf1hnj6ij0h20n-terraform-provider-tfe-0.8.2-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/pb6r7dllpfw5cbhpmv2v2kms9a57r4v5-terraform-provider-tls-2.0.1-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/apjv9g35sklrab9lzz9r9rq7lnczv2wy-terraform-provider-triton-0.5.1-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/5b0s7hhp52vq4psmicf8m8y2jr5jsiaz-terraform-provider-ucloud-1.6.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/ck0lifb2jgkmg6c7frz7fxqwz5fbdnxk-terraform-provider-ultradns-0.1.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/1fl7yd9chgswnabbsvva7xvg5ak1q44p-terraform-provider-vault-1.8.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/n3rakrhhvi3bb0ffnjs51drmy157p51q-terraform-provider-vcd-2.1.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/n1mrfbzlh3cjm9mfyrp48pybl3sg4717-terraform-provider-vsphere-1.10.0-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/cncad2f4lfxfxnwd9lfhjjd89x3anxqr-terraform-provider-yandex-0.5.2-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/2j9jm3jaxfn2g6wxak61wkhmrg6c4nn5-unbound-1.9.1-lib' from 'https://cache.nixos.org'... +copying path '/nix/store/01aggsi1ndjhnr93gcy8c4s1xbxab8dn-unzip-6.0' from 'https://cache.nixos.org'... +copying path '/nix/store/1dydqkwswavzkyvr1qr62zmx3nqpmpp4-gnutls-3.6.7' from 'https://cache.nixos.org'... +copying path '/nix/store/69msrhi85iay3cb7c3nksr0s8l0xpsc7-util-linux-2.33.2' from 'https://cache.nixos.org'... +copying path '/nix/store/xdhh337yhl93x33vzd9davinrbr9x9iz-libmicrohttpd-0.9.63' from 'https://cache.nixos.org'... +copying path '/nix/store/rcn9d2q4mpapxf4qd54hkqz32ljhv0rw-util-linux-2.33.2' from 'https://cache.nixos.org'... +copying path '/nix/store/50gfgyi2rxi4n25if8cqvlxlh5czl0wd-yajl-2.1.0' from 'https://cache.nixos.org'... +copying path '/nix/store/yrmrvha03pvdyi9ww2bi6xjpk5930sf8-glib-2.60.1' from 'https://cache.nixos.org'... +copying path '/nix/store/z2darh83lb4rmsfnnyjc0hll51fyvj49-libSM-1.2.3' from 'https://cache.nixos.org'... +copying path '/nix/store/pknq6p5h43zm4r0dgjnfywql04hdv3js-atk-2.32.0' from 'https://cache.nixos.org'... +copying path '/nix/store/p8s6295x84d594sxvzml8rsxqjdghmc5-cairo-1.16.0' from 'https://cache.nixos.org'... +copying path '/nix/store/v5q3cnkjfy8rfacsjqn1nza93mbczgd5-gdk-pixbuf-2.38.1' from 'https://cache.nixos.org'... +copying path '/nix/store/9yb9whkdgf3zyy85xac248kwq1wm6qd6-harfbuzz-2.3.1' from 'https://cache.nixos.org'... +copying path '/nix/store/xxwqa4rwfi97az8a6dl6vhqiyjvmnm9v-libsecret-0.18.8' from 'https://cache.nixos.org'... +copying path '/nix/store/3xq3w5fgz99rhp3rxfkbp0ahg37mgmly-pango-1.43.0' from 'https://cache.nixos.org'... +copying path '/nix/store/a185xh0jcx7il7hw2gfh0pmvrah3x67y-systemd-239.20190219-lib' from 'https://cache.nixos.org'... +copying path '/nix/store/s7rqxrfb631i53dfl90gac35095jyypq-util-linux-2.33.2-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/6hwdmzpspbnb7ix5z6m9h60jyy42kj90-dbus-1.12.12-lib' from 'https://cache.nixos.org'... +copying path '/nix/store/xmy3chnan9iiag9apm7dd825cmlkiiij-libusb-1.0.22' from 'https://cache.nixos.org'... +copying path '/nix/store/ricz15lpkjrasc5cpzp6l60iwlc87wv3-avahi-0.7' from 'https://cache.nixos.org'... +copying path '/nix/store/di6rrbw1kbdrwxiymq91dgdvp2rvk1xv-dnsmasq-2.80' from 'https://cache.nixos.org'... +copying path '/nix/store/kccb2k5hdjhdyxbxsri9lwwc4z1pvx6z-cups-2.2.11-lib' from 'https://cache.nixos.org'... +copying path '/nix/store/vqvmd2r9pf9f74jqipbhrn7wksiiy1jf-pcsclite-1.8.25-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/x3rijvjjrch1fjs60lrw9xb7pidp817f-gtk+-2.24.32' from 'https://cache.nixos.org'... +copying path '/nix/store/y3x0fvlz4a30iajw3vd1rkg45vl3k15c-pcsclite-1.8.25' from 'https://cache.nixos.org'... +copying path '/nix/store/a2cg0faxbwnicf41vwmw467jw7i9ix46-pinentry-1.1.0' from 'https://cache.nixos.org'... +copying path '/nix/store/lvqp39d4hx776nkw3a0qfnvvjmnj49hc-procps-3.3.15' from 'https://cache.nixos.org'... +copying path '/nix/store/z68464p6aafah8b8xcybkwyhmqdf0jgx-gnupg-2.2.15' from 'https://cache.nixos.org'... +copying path '/nix/store/6bvd29jny80ka8df9prr5hrl5yz7d98k-systemd-239.20190219' from 'https://cache.nixos.org'... +copying path '/nix/store/qgr66z24rfbb8cc965rr2sklh38p083n-git-crypt-0.6.0' from 'https://cache.nixos.org'... +copying path '/nix/store/daizqdqrm7g4favv814hnijmqhay8hs4-dbus-1.12.12' from 'https://cache.nixos.org'... +copying path '/nix/store/wf5nv1gzrx378icqmjgwl2isg7s8ly80-lvm2-2.03.01' from 'https://cache.nixos.org'... +copying path '/nix/store/97d3r4a7v1nal53x0gv17hrbbcp0rb21-util-linux-2.33.2-bin' from 'https://cache.nixos.org'... +copying path '/nix/store/glrnpb3rkribnrjh5gzs24nmvl3m00cg-parted-3.2' from 'https://cache.nixos.org'... +copying path '/nix/store/f39sk2aim9xz7dzn7qvqh442xm58v77w-nfs-utils-2.3.3' from 'https://cache.nixos.org'... +copying path '/nix/store/wi2mn48l130r7wafvj757rvzfkla59if-pm-utils-1.4.1' from 'https://cache.nixos.org'... +copying path '/nix/store/35mdgd1wc67g60azsrghzgn4fjhr5d2r-zfs-user-0.7.13-lib' from 'https://cache.nixos.org'... +copying path '/nix/store/9dk1gh07pwkvg62rns4k670h54bhfhgh-zlib-1.2.11-dev' from 'https://cache.nixos.org'... +copying path '/nix/store/8bxvyvd3ky0w5gk3k0lq2fmvj30fbzj8-zfs-user-0.7.13' from 'https://cache.nixos.org'... +copying path '/nix/store/i3kh8yq4kgkfn234pnwxnvxbrcgcckc8-curl-7.64.1-dev' from 'https://cache.nixos.org'... +copying path '/nix/store/nwhvl00i2wa4ms26lszk36vwir90jd3x-libvirt-4.10.0' from 'https://cache.nixos.org'... +building '/nix/store/as9r3n55czsdiq82iacs0hq12alxb2m0-remove-references-to.drv'... +copying path '/nix/store/l9821zngvlh8bd6mlyzvi1mc754dyhjz-terraform-provider-libvirt-0.5.1-bin' from 'https://cache.nixos.org'... +building '/nix/store/fdh1ahjdh3fgsz4qz386klsa9bsqil48-source.drv'... + +trying https://github.com/n3integration/terraform-godaddy/archive/v1.6.4.tar.gz + % Total % Received % Xferd Average Speed Time Time Time Current + Dload Upload Total Spent Left Speed +100 139 0 139 0 0 863 0 --:--:-- --:--:-- --:--:-- 858 +100 19326 0 19326 0 0 59282 0 --:--:-- --:--:-- --:--:-- 59282 +unpacking source archive /build/v1.6.4.tar.gz +copying path '/nix/store/isdbs6d2jk75kj0qk4s3prwlwcgkgalf-tf-plugin-env' from 'https://cache.nixos.org'... +building '/nix/store/x7r5kh20ajlnj6vw6fg649w0iypcg1ga-terraform-godaddy-1.6.4-go-modules.drv'... +unpacking sources +unpacking source archive /nix/store/m62ydk4wy6818sysfys0qz20cx5nzj7h-source +source root is source +patching sources +configuring +building +go: finding github.com/mitchellh/gox v0.4.0 +go: finding github.com/hashicorp/go-hclog v0.0.0-20181001195459-61d530d6c27f +go: finding github.com/hashicorp/go-getter v0.0.0-20181213035916-be39683deade +go: finding github.com/mitchellh/go-testing-interface v1.0.0 +go: finding github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af +go: finding github.com/agext/levenshtein v1.2.1 +go: finding github.com/apparentlymart/go-cidr v1.0.0 +go: finding github.com/bgentry/go-netrc v0.0.0-20140422174119-9fd32a8b3d3d +go: finding github.com/hashicorp/hil v0.0.0-20170627220502-fa9f258a9250 +go: finding github.com/posener/complete v1.2.1 +go: finding github.com/mitchellh/copystructure v1.0.0 +go: finding github.com/hashicorp/errwrap v1.0.0 +go: finding github.com/hashicorp/hcl2 v0.0.0-20181220012050-6631d7cd0a68 +go: finding github.com/hashicorp/go-version v1.0.0 +go: finding github.com/mitchellh/reflectwalk v1.0.0 +go: finding golang.org/x/net v0.0.0-20181220203305-927f97764cc3 +go: finding golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4 +go: finding golang.org/x/tools v0.0.0-20181221235234-d00ac6d27372 +go: finding github.com/golang/protobuf v1.2.0 +go: finding github.com/hashicorp/go-multierror v1.0.0 +go: finding github.com/mitchellh/go-homedir v1.0.0 +go: finding github.com/apparentlymart/go-textseg v1.0.0 +go: finding github.com/hashicorp/go-uuid v1.0.0 +go: finding github.com/satori/go.uuid v1.2.0 +go: finding github.com/mitchellh/cli v1.0.0 +go: finding github.com/mitchellh/mapstructure v1.1.2 +go: finding golang.org/x/lint v0.0.0-20181217174547-8f45f776aaf1 +go: finding golang.org/x/crypto v0.0.0-20181203042331-505ab145d0a9 +go: finding google.golang.org/appengine v1.4.0 +go: finding golang.org/x/net v0.0.0-20181114220301-adae6a3d119a +go: finding golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc +go: finding honnef.co/go/tools v0.0.0-20180920025451-e3ad64cb4ed3 +go: finding github.com/hashicorp/terraform v0.11.11 +go: finding github.com/aws/aws-sdk-go v1.15.78 +go: finding github.com/onsi/ginkgo v1.7.0 +go: finding github.com/kr/pty v1.1.3 +go: finding github.com/mattn/go-isatty v0.0.3 +go: finding github.com/onsi/gomega v1.4.3 +go: finding github.com/bgentry/speakeasy v0.1.0 +go: finding gopkg.in/yaml.v2 v2.2.2 +go: finding gopkg.in/yaml.v2 v2.2.1 +go: finding github.com/fatih/color v1.7.0 +go: finding github.com/zclconf/go-cty v0.0.0-20181129180422-88fbe721e0f8 +go: finding github.com/mitchellh/go-wordwrap v0.0.0-20150314170334-ad45545899c7 +go: finding golang.org/x/text v0.3.0 +go: finding github.com/sergi/go-diff v1.0.0 +go: finding github.com/armon/go-radix v1.0.0 +go: finding github.com/fsnotify/fsnotify v1.4.7 +go: finding golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e +go: finding golang.org/x/net v0.0.0-20181129055619-fae4c4e3ad76 +go: finding github.com/kr/pretty v0.1.0 +go: finding github.com/hashicorp/go-cleanhttp v0.5.0 +go: finding github.com/mattn/go-isatty v0.0.4 +go: finding gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 +go: finding github.com/go-test/deep v1.0.1 +go: finding cloud.google.com/go v0.34.0 +go: finding github.com/kylelemons/godebug v0.0.0-20170820004349-d65d576e9348 +go: finding howett.net/plist v0.0.0-20181124034731-591f970eefbb +go: finding github.com/hashicorp/errwrap v0.0.0-20180715044906-d6c0cd880357 +go: finding golang.org/x/crypto v0.0.0-20180816225734-aabede6cba87 +go: finding golang.org/x/net v0.0.0-20180724234803-3673e40ba225 +go: finding github.com/mattn/go-colorable v0.0.9 +go: finding github.com/hashicorp/yamux v0.0.0-20181012175058-2f1d1f20f75d +go: finding github.com/go-ini/ini v1.40.0 +go: finding github.com/mitchellh/iochan v1.0.0 +go: finding golang.org/x/sys v0.0.0-20181228144115-9a3f9b0469bb +go: finding github.com/jessevdk/go-flags v1.4.0 +go: finding github.com/posener/complete v1.1.1 +go: finding github.com/spf13/pflag v1.0.3 +go: finding github.com/stretchr/testify v1.2.2 +go: finding golang.org/x/oauth2 v0.0.0-20181203162652-d668ce993890 +go: finding github.com/davecgh/go-spew v1.1.1 +go: finding golang.org/x/net v0.0.0-20180906233101-161cd47e91fd +go: finding golang.org/x/sync v0.0.0-20181108010431-42b317875d0f +go: finding github.com/bsm/go-vlq v0.0.0-20150828105119-ec6e8d4f5f4e +go: finding gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 +go: finding github.com/mitchellh/go-wordwrap v1.0.0 +go: finding github.com/ulikunitz/xz v0.5.5 +go: finding github.com/hashicorp/hcl v1.0.0 +go: finding github.com/hashicorp/go-multierror v0.0.0-20180717150148-3d5d8f294aa0 +go: finding gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 +go: finding golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f +go: finding github.com/google/go-cmp v0.2.0 +go: finding github.com/golang/mock v1.2.0 +go: finding gopkg.in/fsnotify.v1 v1.4.7 +go: finding github.com/onsi/ginkgo v1.6.0 +go: finding github.com/golang/protobuf v1.1.0 +go: finding github.com/aws/aws-sdk-go v1.16.11 +go: finding github.com/hpcloud/tail v1.0.0 +go: finding google.golang.org/grpc v1.17.0 +go: finding github.com/blang/semver v3.5.1+incompatible +go: finding github.com/vmihailenco/msgpack v3.3.3+incompatible +go: finding github.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310 +go: finding golang.org/x/net v0.0.0-20180811021610-c39426892332 +go: finding github.com/zclconf/go-cty v0.0.0-20181218225846-4fe1e489ee06 +go: finding github.com/jmespath/go-jmespath v0.0.0-20160202185014-0b12d6b521d8 +go: finding github.com/hashicorp/go-plugin v0.0.0-20181212150838-f444068e8f5a +go: finding github.com/pmezard/go-difflib v1.0.0 +go: finding github.com/spf13/pflag v1.0.2 +go: finding github.com/hashicorp/go-safetemp v1.0.0 +go: finding github.com/vmihailenco/msgpack v4.0.1+incompatible +go: finding google.golang.org/genproto v0.0.0-20181221175505-bd9b4fb69e2f +go: finding golang.org/x/tools v0.0.0-20180828015842-6cd1fcedba52 +go: finding github.com/kr/text v0.1.0 +go: finding golang.org/x/net v0.0.0-20180826012351-8a410e7b638d +go: finding golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3 +go: finding github.com/apparentlymart/go-dump v0.0.0-20180507223929-23540a00eaa3 +go: finding github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b +go: finding github.com/golang/mock v1.1.1 +go: finding cloud.google.com/go v0.26.0 +go: finding github.com/oklog/run v1.0.0 +go: finding golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be +go: finding google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8 +go: finding github.com/hashicorp/go-hclog v0.0.0-20180709165350-ff2cf002a8dd +go: finding google.golang.org/grpc v1.14.0 +go: finding github.com/client9/misspell v0.3.4 +go: finding github.com/kr/pty v1.1.1 +go: finding google.golang.org/appengine v1.1.0 +go: finding honnef.co/go/tools v0.0.0-20180728063816-88497007e858 +go: finding golang.org/x/sys v0.0.0-20180830151530-49385e6e1522 +go: finding github.com/hashicorp/yamux v0.0.0-20180604194846-3520598351bb +go: finding github.com/kisielk/gotool v1.0.0 +go: finding github.com/mitchellh/go-testing-interface v0.0.0-20171004221916-a61a99592b77 +go: finding github.com/mitchellh/hashstructure v1.0.0 +go: finding golang.org/x/net v0.0.0-20181106065722-10aee1819953 +go: finding google.golang.org/grpc v1.16.0 +go: finding golang.org/x/lint v0.0.0-20180702182130-06c8688daad7 +go: finding github.com/golang/lint v0.0.0-20180702182130-06c8688daad7 +installing +hash mismatch in fixed-output derivation '/nix/store/q8y0mzjl78hfhazjgq2sc84i7dp9wnh0-terraform-godaddy-1.6.4-go-modules': + wanted: sha256:10n2dy7q9kk1ly58sw965n6qa8l0nffh8vyd1vslx0gdlyj25xxs + got: sha256:0p81wqw2n8vraxk20xwg717582ijwq2k7v5j3n13y4cd5bxd8hhz +cannot build derivation '/nix/store/w4ghinrmpq524k3617ikfc8i42aa0dbb-terraform-godaddy-1.6.4.drv': 1 dependencies couldn't be built +copying path '/nix/store/63gjp25l4cmdkl63zy0rcgmsvd2p2p34-terraform-0.11.14' from 'https://cache.nixos.org'... +error: build of '/nix/store/9drkn1qxkkcrz5g3413lpmbc2xysa582-terraform-0.11.14.drv', '/nix/store/w4ghinrmpq524k3617ikfc8i42aa0dbb-terraform-godaddy-1.6.4.drv' failed +---- diff --git a/src/content/en/pastebin/2019/12/29/raku-tuple-type.adoc b/src/content/en/pastebin/2019/12/29/raku-tuple-type.adoc new file mode 100644 index 0000000..50dd841 --- /dev/null +++ b/src/content/en/pastebin/2019/12/29/raku-tuple-type.adoc @@ -0,0 +1,28 @@ += Raku tuple type annotation +:categories: raku programming-languages + +[source,raku] +---- +# Single Str return value: this works +sub f1(Str $in --> Str) { + $in; +} + +# Tuple of Str as return value: this works +sub f2(Str $in) { + ($in, $in); +} + +# Tuple of Str as return value with type annotation: this doesn't works +sub f2(Str $in --> (Str, Str)) { + ($in, $in); +} +---- + +Error log is: + +[source,raku] +---- +===SORRY!=== Error while compiling /path/to/my/file +Malformed return value +---- diff --git a/src/content/en/pastebin/2020/01/04/guix-import-failure.adoc b/src/content/en/pastebin/2020/01/04/guix-import-failure.adoc new file mode 100644 index 0000000..5896645 --- /dev/null +++ b/src/content/en/pastebin/2020/01/04/guix-import-failure.adoc @@ -0,0 +1,35 @@ += Failure on Guix TeX Live importer +:categories: guix + +[source,sh] +---- +$ guix import texlive fontspec +redirection vers « https://ctan.org/xml/1.2/pkg/fontspec »... +Backtrace: + 11 (primitive-load "/home/andreh/.config/guix/current/bin/…") +In guix/ui.scm: + 1806:12 10 (run-guix-command _ . _) +In guix/scripts/import.scm: + 116:11 9 (guix-import . _) +In guix/scripts/import/texlive.scm: + 91:19 8 (guix-import-texlive . _) +In guix/memoization.scm: + 98:0 7 (_ #<hash-table 7fe80e6c1480 0/31> ("fontspec" "latex") _) +In unknown file: + 6 (_ #<procedure 7fe80e6e4de0 at guix/memoization.scm:17…> …) +In guix/store.scm: + 625:10 5 (call-with-store #<procedure 7fe80e714a60 at guix/impor…>) +In guix/import/texlive.scm: + 148:23 4 (_ #<store-connection 256.99 7fe811f3c960>) +In guix/utils.scm: + 664:8 3 (call-with-temporary-directory #<procedure 7fe80cac1b40…>) +In guix/svn-download.scm: + 160:14 2 (_ "/tmp/guix-directory.WtLohP") +In guix/build/svn.scm: + 39:2 1 (svn-fetch _ _ _ #:svn-command _ #:recursive? _ # _ # _) +In guix/build/utils.scm: + 652:6 0 (invoke _ . _) + +guix/build/utils.scm:652:6: In procedure invoke: +Throw to key `srfi-34' with args `(#<condition &invoke-error [program: "svn" arguments: ("export" "--non-interactive" "--trust-server-cert" "-r" "49435" "svn://www.tug.org/texlive/tags/texlive-2018.2/Master/texmf-dist/source/latex/fontspec" "/tmp/guix-directory.WtLohP") exit-status: 1 term-signal: #f stop-signal: #f] 7fe80d229c80>)'. +---- diff --git a/src/content/en/pastebin/2020/02/14/guix-shebang.adoc b/src/content/en/pastebin/2020/02/14/guix-shebang.adoc new file mode 100644 index 0000000..d415d36 --- /dev/null +++ b/src/content/en/pastebin/2020/02/14/guix-shebang.adoc @@ -0,0 +1,11 @@ += Guix shebang +:categories: guix + +[source,sh] +---- +#!/usr/bin/env -S guix environment --ad-hoc bash -- bash +set -Eeuo pipefail +cd "$(dirname "${BASH_SOURCE[0]}")" + +pwd +---- diff --git a/src/content/en/pastebin/2020/11/27/guix-build-local.adoc b/src/content/en/pastebin/2020/11/27/guix-build-local.adoc new file mode 100644 index 0000000..1a18d4b --- /dev/null +++ b/src/content/en/pastebin/2020/11/27/guix-build-local.adoc @@ -0,0 +1,51 @@ += Guix build local module +:categories: guix + +FIXED: rename `name` on line 9 of the first snippet, and use `"my-hello"` +instead of `"hello"`. + +''''' + +Inside a file named `build.scm`: + +[source,scheme] +---- +(define-module (build) + #:use-module (guix packages) + #:use-module (guix download) + #:use-module (guix build-system gnu) + #:use-module (guix licenses)) + +(define-public my-hello + (package + (name "hello") + (version "2.10") + (source (origin + (method url-fetch) + (uri (string-append "mirror://gnu/hello/hello-" version + ".tar.gz")) + (sha256 + (base32 + "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i")))) + (build-system gnu-build-system) + (synopsis "") + (description "") + (home-page "") + (license gpl3+))) +---- + +A plain build command didn't work: + +[source,sh] +---- +$ guix build -L. my-hello +guix build: error: my-hello : paquet inconnu +---- + +But with an eval expression it did: + +[source,sh] +---- +$ guix build -L. -e '(@ (build) my-hello)' +# works +---- diff --git a/src/content/en/pastebin/2020/12/15/guix-pack-fail.adoc b/src/content/en/pastebin/2020/12/15/guix-pack-fail.adoc new file mode 100644 index 0000000..3631fbc --- /dev/null +++ b/src/content/en/pastebin/2020/12/15/guix-pack-fail.adoc @@ -0,0 +1,86 @@ += Failure with relocatable Guix pack tarball +:categories: guix + +:post: https://guix.gnu.org/blog/2018/tarballs-the-ultimate-container-image-format/ + +FIXED: Use `GUIX_PROFILE= source etc/profile` + +''''' + +The example from the {post}[blog post] fails. + +[source,sh] +---- +$ tar xf `guix pack --relocatable -S /bin=bin -S /etc=etc guile gnutls guile-json` +$ source etc/profile +$ bin/guile -c '(use-modules (json))' +guile: warning: failed to install locale +Backtrace: +In ice-9/boot-9.scm: + 1736:10 13 (with-exception-handler _ _ #:unwind? _ # _) +In unknown file: + 12 (apply-smob/0 #<thunk 7f9d240ca740>) +In ice-9/boot-9.scm: + 718:2 11 (call-with-prompt ("prompt") #<procedure 7f9d240db740 ?> ?) +In ice-9/eval.scm: + 619:8 10 (_ #(#(#<directory (guile-user) 7f9d23d00f00>))) +In ice-9/command-line.scm: + 185:18 9 (_ #<input: string 7f9d23cfaa10>) +In unknown file: + 8 (eval (use-modules (json)) #<directory (guile-user) 7f9?>) +In ice-9/eval.scm: + 721:20 7 (primitive-eval (use-modules (json))) +In ice-9/psyntax.scm: + 1241:36 6 (expand-top-sequence ((use-modules (json))) _ _ #f _ _ _) + 1233:19 5 (parse _ (("placeholder" placeholder)) ((top) #(# # ?)) ?) + 285:10 4 (parse _ (("placeholder" placeholder)) (()) _ c&e (eval) ?) +In ice-9/boot-9.scm: + 3898:20 3 (process-use-modules _) + 222:17 2 (map1 (((json)))) + 3899:31 1 (_ ((json))) + 3300:6 0 (resolve-interface (json) #:select _ #:hide _ #:prefix _ ?) + +ice-9/boot-9.scm:3300:6: In procedure resolve-interface: +no code for module (json) +$ bin/guile -c '(use-modules (gnutls))' +guile: warning: failed to install locale +Backtrace: +In ice-9/boot-9.scm: + 1736:10 13 (with-exception-handler _ _ #:unwind? _ # _) +In unknown file: + 12 (apply-smob/0 #<thunk 7f7fe607a7c0>) +In ice-9/boot-9.scm: + 718:2 11 (call-with-prompt ("prompt") #<procedure 7f7fe6085940 ?> ?) +In ice-9/eval.scm: + 619:8 10 (_ #(#(#<directory (guile-user) 7f7fe5ca8f00>))) +In ice-9/command-line.scm: + 185:18 9 (_ #<input: string 7f7fe5ca2a10>) +In unknown file: + 8 (eval (use-modules (gnutls)) #<directory (guile-user) 7?>) +In ice-9/eval.scm: + 721:20 7 (primitive-eval (use-modules (gnutls))) +In ice-9/psyntax.scm: + 1241:36 6 (expand-top-sequence ((use-modules (gnutls))) _ _ #f _ _ ?) + 1233:19 5 (parse _ (("placeholder" placeholder)) ((top) #(# # ?)) ?) + 285:10 4 (parse _ (("placeholder" placeholder)) (()) _ c&e (eval) ?) +In ice-9/boot-9.scm: + 3898:20 3 (process-use-modules _) + 222:17 2 (map1 (((gnutls)))) + 3899:31 1 (_ ((gnutls))) + 3300:6 0 (resolve-interface (gnutls) #:select _ #:hide _ #:prefix ?) + +ice-9/boot-9.scm:3300:6: In procedure resolve-interface: +no code for module (gnutls) +---- + +My Guix version if fairly recent: + +[source,sh] +---- +$ guix describe +Génération 83 14 déc. 2020 00:28:16 (actuelle) + guix 41807eb + URL du dépôt : https://git.savannah.gnu.org/git/guix.git + branche: master + commit : 41807eb5329299b8c45cd49356a4ead01ce0d469 +---- diff --git a/src/content/en/pastebin/2021/04/03/naive-slugify-js.adoc b/src/content/en/pastebin/2021/04/03/naive-slugify-js.adoc new file mode 100644 index 0000000..184ce97 --- /dev/null +++ b/src/content/en/pastebin/2021/04/03/naive-slugify-js.adoc @@ -0,0 +1,28 @@ += JavaScript naive slugify +:updatedat: 2021-08-15 + +[source,javascript] +---- +const s = "Pézão: açaí, saci-pererê."; + +const slugify = s => + s + .toLowerCase() + .replaceAll(":", "") + .replaceAll(".", "") + .replaceAll(",", "") + .replaceAll("-", "") + .replaceAll("á", "a") + .replaceAll("ã", "a") + .replaceAll("à", "a") + .replaceAll("é", "e") + .replaceAll("ê", "e") + .replaceAll("í", "i") + .replaceAll("ó", "o") + .replaceAll("ô", "o") + .replaceAll("ú", "u") + .replaceAll("ü", "u") + .replaceAll("ç", "c"); + +console.log(slugify(s)); +---- diff --git a/src/content/en/pastebin/2021/06/08/reading-session-pt1.adoc b/src/content/en/pastebin/2021/06/08/reading-session-pt1.adoc new file mode 100644 index 0000000..d9f1f91 --- /dev/null +++ b/src/content/en/pastebin/2021/06/08/reading-session-pt1.adoc @@ -0,0 +1,66 @@ += Debit Reading Session - SICP solutions pt.1 + +[source,scheme] +---- +;; 1.41 +(define (double f) + (lambda (x) + (f (f x)))) + + +:;; 1.42 +(define (compose f g) + (lambda (x) + (f (g x)))) + + +;;; 1.43 +(define (repeated f n) + (if (= 1 n) + identity + (comp (repeated f (dec n))))) + + +;;; 2.27 +(define (map-tree node-fn leaf-fn tree) + (cond + ((null? tree) tree) + ((not (pair? tree)) (leaf-fn tree)) + (else + (node-fn + (cons (map-tree node-fn leaf-fn (car tree)) + (map-tree node-fn leaf-fn (cdr tree))))))) + +(define (map-nodes f tree) + (map-tree f identity tree)) + +(define (deep-reverse x) + (map-nodes reverse x)) + + +;;; 2.28 +(define (flatten tree) + (define (rec acc t) + (cond + ((null? t) acc) + ((not (pair? t)) (cons t acc)) + (else + (rec (rec (cdr t) acc) + (car t))))) + (rec nil tree)) + + +;;; 2.30 +(define (square-tree tree) + (map-leaves square tree)) + + +;;; 2.31 +(define square-tree map-leaves) ; ha! + + +;;; 2.32 +TODO +---- + +FYI: I just typed those in, I didn't yet test them yet. diff --git a/src/content/en/pastebin/2021/06/22/curl-wget.adoc b/src/content/en/pastebin/2021/06/22/curl-wget.adoc new file mode 100644 index 0000000..02a9744 --- /dev/null +++ b/src/content/en/pastebin/2021/06/22/curl-wget.adoc @@ -0,0 +1,92 @@ += cloc: curl and wget + +`curl`: + +[source,sh] +---- +$ pushd `mktemp -d` +/tmp/tmp.AZkwvk7azD ~/ +$ git clone git://github.com/curl/curl . +Clonage dans '.'... +remote: Enumerating objects: 167029, done. +remote: Counting objects: 100% (925/925), done. +remote: Compressing objects: 100% (372/372), done. +remote: Total 167029 (delta 590), reused 818 (delta 548), pack-reused 166104 +Réception d'objets: 100% (167029/167029), 75.63 Mio | 9.33 Mio/s, fait. +Résolution des deltas: 100% (131415/131415), fait. +$ cloc . + 3386 text files. + 3342 unique files. + 2084 files ignored. + +github.com/AlDanial/cloc v 1.90 T=1.34 s (973.7 files/s, 260104.4 lines/s) +------------------------------------------------------------------------------------ +Language files blank comment code +------------------------------------------------------------------------------------ +C 535 25645 36361 135318 +XML 23 21 20 45997 +m4 29 1526 1976 16972 +Perl 56 2611 4010 15411 +C/C++ Header 223 4178 10109 13794 +Markdown 53 2784 0 7038 +Visual Studio Solution 27 0 21 5049 +D 242 398 0 3549 +CMake 34 754 1288 3056 +DOS Batch 7 293 370 1554 +YAML 18 115 171 1493 +make 21 296 660 1440 +Bourne Shell 22 326 633 1136 +Pascal 2 228 0 634 +Python 4 196 221 628 +Visual Basic Script 1 30 60 341 +C++ 3 58 69 169 +Gencat NLS 1 2 0 130 +TNSDL 1 3 0 113 +Windows Resource File 2 17 47 110 +Bourne Again Shell 1 17 44 97 +Protocol Buffers 1 2 0 28 +diff 1 0 0 11 +Lisp 1 1 23 7 +TOML 1 0 0 3 +------------------------------------------------------------------------------------ +SUM: 1309 39501 56083 254078 +------------------------------------------------------------------------------------ +---- + +`wget`: + +[source,sh] +---- +$ pushd `mktemp -d` +/tmp/tmp.NX0udlJMiz ~/ +$ git clone git://git.savannah.gnu.org/wget.git . +Clonage dans '.'... +remote: Counting objects: 52248, done. +remote: Compressing objects: 100% (18430/18430), done. +remote: Total 52248 (delta 23879), reused 52248 (delta 23879) +Réception d'objets: 100% (52248/52248), 13.11 Mio | 6.18 Mio/s, fait. +Résolution des deltas: 100% (23879/23879), fait. +$ cloc . + 12210 text files. + 11629 unique files. + 11876 files ignored. + +github.com/AlDanial/cloc v 1.90 T=1.26 s (270.4 files/s, 61357.4 lines/s) +-------------------------------------------------------------------------------- +Language files blank comment code +-------------------------------------------------------------------------------- +C 53 6596 8955 34084 +Perl 106 1832 870 7415 +Python 105 1481 2374 5318 +C/C++ Header 43 704 1153 1486 +Bourne Shell 11 308 311 1278 +m4 3 172 183 940 +make 9 136 172 522 +YAML 3 27 13 515 +Bourne Again Shell 6 78 89 274 +Markdown 2 37 0 113 +lex 1 29 65 73 +-------------------------------------------------------------------------------- +SUM: 342 11400 14185 52018 +-------------------------------------------------------------------------------- +---- diff --git a/src/content/en/pastebin/2021/08/11/h1-spacing.adoc b/src/content/en/pastebin/2021/08/11/h1-spacing.adoc new file mode 100644 index 0000000..38dbb2e --- /dev/null +++ b/src/content/en/pastebin/2021/08/11/h1-spacing.adoc @@ -0,0 +1,75 @@ += Spaces around h1 tags +:updatedat: 2021-08-15 + +_EDIT_: Apparently, the behaviour below is consistent between Firefox and +Chromium for links, but not for `<h1>`. My conclusion is that the `<h1>` +behaviour is a Firefox quirk, but the `<a>` is expected. + +''''' + +The HTML below has selectable extra spaces after `<h1>` tags: + +[source,html] +---- +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="UTF-8" /> + <meta name="viewport" content="width=device-width, initial-scale=1" /> + <title>Spaces around h1 tags</title> + </head> + <body> + <main> + <h1> + With spaces around when selecting this heading + </h1> + <h1>Without spaces around</h1> + <p> + Is this expected behaviour? + </p> + </main> + </body> +</html> +---- + +The rendered output is: + +With spaces around when selecting this heading + +Without spaces around + +Is this expected behaviour? + +''''' + +The same with links: + +[source,html] +---- +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="UTF-8" /> + <meta name="viewport" content="width=device-width, initial-scale=1" /> + <title>Spaces after a tags</title> + </head> + <body> + <main> + <p> + <a href="#"> + With extra underlined space + </a> + </p> + <p> + <a href="#">Without extra underlined space</a> + </p> + </main> + </body> +</html> +---- + +The rendered output is: + +With extra underlined space after the link. + +Without extra underlined space after the link. diff --git a/src/content/en/pastebin/2021/09/02/sicp-3-19.adoc b/src/content/en/pastebin/2021/09/02/sicp-3-19.adoc new file mode 100644 index 0000000..166170f --- /dev/null +++ b/src/content/en/pastebin/2021/09/02/sicp-3-19.adoc @@ -0,0 +1,33 @@ += SICP exercise 3.19 +:categories: lisp programming-languages + +[source,scheme] +---- +(define (cycle? l) + (define (rec l x) + (cond + ((null? x) false) + ((eq? l x) true) + (true (rec l (cdr x))))) + (rec l (cdr l))) +---- + +Sample interactive session: + +[source,scheme] +---- +scheme@(guile-user)> (define true #t) +scheme@(guile-user)> (define false #f) +scheme@(guile-user)> +(define (cycle? l) + (define (rec l x) + (cond + ((null? x) false) + ((eq? l x) true) + (true (rec l (cdr x))))) + (rec l (cdr l))) +scheme@(guile-user)> (cycle? '(1 2 3)) +$9 = #f +scheme@(guile-user)> (cycle? (make-cycle '(1 2 3))) +$10 = #t +---- diff --git a/src/content/en/pastebin/2021/09/03/sicp-persistent-queue.adoc b/src/content/en/pastebin/2021/09/03/sicp-persistent-queue.adoc new file mode 100644 index 0000000..2b4a8a2 --- /dev/null +++ b/src/content/en/pastebin/2021/09/03/sicp-persistent-queue.adoc @@ -0,0 +1,76 @@ += SICP persistent amortized O(1) queue + +[source,scheme] +---- +(define (queue) + (cons '() + '())) + +(define (enqueue x q) + (cons (car q) + (cons x (cdr q)))) + +(define (flush q) + (cons (reverse (cdr q)) + '())) + +(define (dequeue q) + (if (null? (car q)) + (dequeue (flush q)) + (cons (caar q) + (cons (cdar q) + (cdr q))))) + +(define (empty? q) + (and (null? (car q)) + (null? (cdr q)))) + +(define (peek q) + (car (dequeue q))) + +(define (print-queue q) + (define (rec l leading-space?) + (when (not (null? l)) + (when leading-space? + (display " ")) + (display (car l)) + (rec (cdr l) #t))) + + (display "#q(") + (rec (car q) false) + (rec (reverse (cdr q)) (not (null? (car q)))) + (display ")") + (newline)) +---- + +Sample interactive session: + +[source,scheme] +---- +scheme@(guile-user)> (define true #t) +scheme@(guile-user)> (define false #f) +scheme@(guile-user)> (define q (queue)) +scheme@(guile-user)> (print-queue q) +#q() +scheme@(guile-user)> (print-queue (enqueue 'a q)) +#q(a) +scheme@(guile-user)> (print-queue q) +#q() +scheme@(guile-user)> (set! q (enqueue 'a q)) +scheme@(guile-user)> (print-queue q) +#q(a) +scheme@(guile-user)> (set! q (enqueue 'e (enqueue 'd (enqueue 'c (enqueue 'b q))))) +scheme@(guile-user)> (print-queue q) +#q(e d c b a) +scheme@(guile-user)> (peek q) +$28 = a +scheme@(guile-user)> (define ret (dequeue q)) +scheme@(guile-user)> (define value (car ret)) +scheme@(guile-user)> (set! q (cdr ret)) +scheme@(guile-user)> value +$29 = a +scheme@(guile-user)> (print-queue q) +#q(b c d e) +scheme@(guile-user)> (print-queue (cdr (dequeue (cdr (dequeue (enqueue 'g (enqueue 'f q))))))) +#q(d e f g) +---- diff --git a/src/content/en/pastebin/2022/07/14/git-cleanup.adoc b/src/content/en/pastebin/2022/07/14/git-cleanup.adoc new file mode 100644 index 0000000..b223f86 --- /dev/null +++ b/src/content/en/pastebin/2022/07/14/git-cleanup.adoc @@ -0,0 +1,59 @@ += git-cleanup command + +[source,sh] +---- +#!/bin/sh +set -eu + +usage() { + cat <<-'EOF' + Usage: + git cleanup + git cleanup -h + EOF +} + +help() { + cat <<-'EOF' + + Options: + -h, --help show this message + EOF +} + +for flag in "$@"; do + case "$flag" in + --) + break + ;; + --help) + usage + help + exit + ;; + *) + ;; + esac +done + +while getopts 'h' flag; do + case "$flag" in + h) + usage + help + exit + ;; + *) + usage >&2 + exit 2 + ;; + esac +done +shift $((OPTIND - 1)) + + + +git branch --merged | + grep -v -e '^\*' -e '^. main$' | + xargs git branch -d +---- diff --git a/src/content/en/pastebin/2023/07/22/funcallable-amop.adoc b/src/content/en/pastebin/2023/07/22/funcallable-amop.adoc new file mode 100644 index 0000000..47a8089 --- /dev/null +++ b/src/content/en/pastebin/2023/07/22/funcallable-amop.adoc @@ -0,0 +1,31 @@ += Funcallable AMOP + +Using `macrolet` to allow a `funcallable-standard-class` to be invoked without +using `funcall` directly, and let the macroexpansion do that instead: + +[source,lisp] +---- +#!/usr/bin/env li + +(asdf:load-system :closer-mop) + +(defclass constructor () + ((name :initarg :name :accessor constructor-name)) + (:metaclass closer-mop:funcallable-standard-class)) + +(defmethod initialize-instance :after ((c constructor) &key) + (with-slots (name) c + (closer-mop:set-funcallable-instance-function + c + (lambda (x) + (format t "~s: ~s - ~s~%" name :funcalled x))))) + +(let ((c (make-instance 'constructor :name "the-name"))) + (funcall c 1)) + +(let ((c (make-instance 'constructor :name "the-name"))) + (macrolet ((c (&body body) + `(funcall c ,@body))) + (funcall c 2) + (c 3))) +---- diff --git a/src/content/en/pastebin/categories.adoc b/src/content/en/pastebin/categories.adoc new file mode 100644 index 0000000..feb64ff --- /dev/null +++ b/src/content/en/pastebin/categories.adoc @@ -0,0 +1,2 @@ += Articles by category +:type: categories diff --git a/src/content/en/pastebin/index.adoc b/src/content/en/pastebin/index.adoc new file mode 100644 index 0000000..a5a8657 --- /dev/null +++ b/src/content/en/pastebin/index.adoc @@ -0,0 +1,2 @@ += Pastebins +:type: collection diff --git a/src/content/en/podcast/2020/12/19/test-entry.adoc b/src/content/en/podcast/2020/12/19/test-entry.adoc new file mode 100644 index 0000000..9e7eb00 --- /dev/null +++ b/src/content/en/podcast/2020/12/19/test-entry.adoc @@ -0,0 +1,102 @@ += A test entry +:categories: test + +audio: true + +After. + +A link to https://euandre.org[home]. + +Another link to home: https://euandre.org + +A code block: + +[source,sh] +---- +$ l +total 372K +drwxr-xr-x 23 andreh users 4,0K déc. 19 10:44 ./ +drwxr-xr-x 30 andreh users 4,0K déc. 14 17:28 ../ +-rw-r--r-- 1 andreh users 565 déc. 15 17:24 about.md +-rw-r--r-- 1 andreh users 330 déc. 19 10:04 aja.md +-rw-r--r-- 1 andreh users 125 déc. 19 10:04 aja-par-categorie.md +-rw-r--r-- 1 andreh users 496 déc. 19 10:04 a-propos.md +drwxr-xr-x 2 andreh users 4,0K déc. 16 04:36 _articles/ +-rw-r--r-- 1 andreh users 136 déc. 15 17:04 articles-by-category.md +-rw-r--r-- 1 andreh users 139 déc. 19 10:04 articles-par-categorie.md +-rw-r--r-- 1 andreh users 137 déc. 19 10:04 artigos-por-categoria.md +-rw-r--r-- 1 andreh users 140 déc. 19 10:04 artikoloj-lau-kategorio.md +-rw-r--r-- 1 andreh users 508 oct. 23 09:35 .build.yml +-rw-r--r-- 1 andreh users 9,0K déc. 19 10:09 _config.yml +-rw-r--r-- 1 andreh users 34K déc. 18 22:48 COPYING +-rw-r--r-- 1 andreh users 2,4K déc. 19 09:28 default.nix +-rw-r--r-- 1 andreh users 41 déc. 4 13:43 description +-rw-r--r-- 1 andreh users 134 déc. 19 10:04 diapositives.md +-rw-r--r-- 1 andreh users 139 déc. 19 10:04 diapositives-par-categorie.md +drwxr-xr-x 2 andreh users 4,0K nov. 15 20:01 en/ +drwxr-xr-x 2 andreh users 4,0K nov. 15 20:08 eo/ +-rw-r--r-- 1 andreh users 159 déc. 19 10:04 episodes-de-podcast-par-categorie.md +-rw-r--r-- 1 andreh users 159 déc. 19 10:04 episodios-do-podcast-por-categoria.md +-rw-r--r-- 1 andreh users 130 déc. 19 10:04 eslaides.md +-rw-r--r-- 1 andreh users 134 déc. 19 10:04 eslaides-por-categoria.md +-rw-r--r-- 1 andreh users 1,2K nov. 15 20:01 favicon.ico +drwxr-xr-x 2 andreh users 4,0K nov. 15 20:08 fr/ +-rw-r--r-- 1 andreh users 54 déc. 6 13:57 Gemfile +-rw-r--r-- 1 andreh users 1,6K déc. 6 13:57 Gemfile.lock +drwxr-xr-x 11 andreh users 4,0K déc. 19 10:44 .git/ +-rw-r--r-- 1 andreh users 75 oct. 16 07:05 .gitignore +-rw-r--r-- 1 andreh users 100 oct. 14 06:42 .gitmodules +-rw-r--r-- 1 andreh users 317 déc. 19 10:04 hea.md +-rw-r--r-- 1 andreh users 124 déc. 19 10:04 hea-por-categoria.md +-rw-r--r-- 1 andreh users 125 déc. 19 10:04 hml-lau-kategorio.md +-rw-r--r-- 1 andreh users 309 déc. 19 10:04 hml.md +-rw-r--r-- 1 andreh users 27 nov. 14 13:51 .ignore +drwxr-xr-x 2 andreh users 4,0K déc. 19 10:44 _includes/ +-rw-r--r-- 1 andreh users 123 nov. 15 20:01 index.md +drwxr-xr-x 3 andreh users 4,0K déc. 19 03:02 .jekyll-cache/ +drwxr-xr-x 2 andreh users 4,0K déc. 19 09:29 _layouts/ +drwxr-xr-x 5 andreh users 4,0K nov. 15 20:07 locale/ +-rw-r--r-- 1 andreh users 136 déc. 19 10:04 lumbildoj-lau-kategorio.md +-rw-r--r-- 1 andreh users 131 déc. 19 10:04 lumbildoj.md +drwxr-xr-x 2 andreh users 4,0K nov. 4 17:26 nix/ +drwxr-xr-x 2 andreh users 4,0K déc. 19 03:20 _pastebins/ +-rw-r--r-- 1 andreh users 139 déc. 15 17:08 pastebins-by-category.md +-rw-r--r-- 1 andreh users 140 nov. 15 20:01 pastebins.en.md +-rw-r--r-- 1 andreh users 140 déc. 19 10:04 pastebins.eo.md +-rw-r--r-- 1 andreh users 140 déc. 19 10:04 pastebins.fr.md +-rw-r--r-- 1 andreh users 142 déc. 19 10:04 pastebins-lau-kategorio.md +-rw-r--r-- 1 andreh users 142 déc. 19 10:04 pastebins-par-categorie.md +-rw-r--r-- 1 andreh users 141 déc. 19 10:04 pastebins-por-categoria.md +-rw-r--r-- 1 andreh users 140 déc. 19 10:04 pastebins.pt.md +drwxr-xr-x 2 andreh users 4,0K déc. 19 09:04 _plugins/ +-rw-r--r-- 1 andreh users 134 déc. 19 09:29 podcast.en.md +-rw-r--r-- 1 andreh users 152 déc. 19 09:29 podcast-episodes-by-category.md +-rw-r--r-- 1 andreh users 134 déc. 19 10:04 podcast.fr.md +-rw-r--r-- 1 andreh users 134 déc. 19 10:04 podcast.pt.md +drwxr-xr-x 2 andreh users 4,0K déc. 19 10:45 _podcasts/ +-rw-r--r-- 1 andreh users 151 déc. 19 10:04 podkastajoj-lau-kategorio.md +-rw-r--r-- 1 andreh users 135 déc. 19 10:04 podkasto.md +-rw-r--r-- 1 andreh users 471 déc. 19 10:04 pri.md +drwxr-xr-x 2 andreh users 4,0K nov. 15 20:08 pt/ +-rw-r--r-- 1 andreh users 3,8K nov. 15 20:01 public-key.txt +-rw-r--r-- 1 andreh users 58 oct. 23 09:54 README +drwxr-xr-x 3 andreh users 4,0K déc. 19 09:00 resources/ +lrwxrwxrwx 1 andreh users 54 déc. 19 10:44 result -> /nix/store/czw8d9gcim51k76ixcgmdpi9kpvysc9d-publish.sh/ +drwxr-xr-x 3 andreh users 4,0K déc. 19 10:35 scripts/ +-rw-r--r-- 1 andreh users 29 juil. 23 18:37 shell.nix +drwxr-xr-x 17 andreh users 4,0K déc. 19 10:35 _site/ +-rw-r--r-- 1 andreh users 3,9K déc. 19 10:29 site.json +-rw-r--r-- 1 andreh users 1,9K nov. 15 20:01 sitemap.xml +drwxr-xr-x 2 andreh users 4,0K déc. 16 04:36 _slides/ +-rw-r--r-- 1 andreh users 130 déc. 15 17:08 slides-by-category.md +-rw-r--r-- 1 andreh users 25 nov. 15 20:01 slides.css +-rw-r--r-- 1 andreh users 128 nov. 15 20:01 slides.md +-rw-r--r-- 1 andreh users 471 déc. 19 10:04 sobre.md +drwxr-xr-x 2 andreh users 4,0K déc. 19 09:04 static/ +-rw-r--r-- 1 andreh users 2,4K déc. 15 16:06 styles.css +-rwxr-xr-x 1 andreh users 265 nov. 6 10:16 tests.sh +-rw-r--r-- 1 andreh users 122 déc. 15 17:41 til-by-category.md +-rw-r--r-- 1 andreh users 265 nov. 15 20:01 til.md +drwxr-xr-x 2 andreh users 4,0K déc. 16 04:03 _tils/ +drwxr-xr-x 3 andreh users 4,0K oct. 10 09:20 vendor/ +---- diff --git a/src/content/en/podcast/2020/12/19/test-entry.flac b/src/content/en/podcast/2020/12/19/test-entry.flac Binary files differnew file mode 100644 index 0000000..786ab59 --- /dev/null +++ b/src/content/en/podcast/2020/12/19/test-entry.flac diff --git a/src/content/en/podcast/2020/12/19/test-entry.ogg b/src/content/en/podcast/2020/12/19/test-entry.ogg Binary files differnew file mode 100644 index 0000000..9bf5500 --- /dev/null +++ b/src/content/en/podcast/2020/12/19/test-entry.ogg diff --git a/src/content/en/podcast/categories.adoc b/src/content/en/podcast/categories.adoc new file mode 100644 index 0000000..feb64ff --- /dev/null +++ b/src/content/en/podcast/categories.adoc @@ -0,0 +1,2 @@ += Articles by category +:type: categories diff --git a/src/content/en/podcast/index.adoc b/src/content/en/podcast/index.adoc new file mode 100644 index 0000000..69f5cea --- /dev/null +++ b/src/content/en/podcast/index.adoc @@ -0,0 +1,2 @@ += Podcasts +:type: collection diff --git a/src/content/en/screencast/2021/02/07/autoqemu.adoc b/src/content/en/screencast/2021/02/07/autoqemu.adoc new file mode 100644 index 0000000..15bac9a --- /dev/null +++ b/src/content/en/screencast/2021/02/07/autoqemu.adoc @@ -0,0 +1,52 @@ += AutoQEMU - automate installation and SSH setup of ISO OS images +:updatedat: 2022-03-06 +:categories: first + +:empty: +:begriffs-tips: https://begriffs.com/posts/2020-08-31-portable-stable-software.html +:public-thread: https://talk.begriffs.com/pipermail/friends/2021-February/001263.html +:archived: https://euandre.org/static/attachments/autoqemu.tar.gz +:expect: https://core.tcl-lang.org/expect/index +:script: https://www.man7.org/linux/man-pages/man1/script.1.html +:klaatu: https://gnuworldorder.info/ + + +After reading begriffs "{begriffs-tips}[Tips for stable and portable software]", +the "Begriffs Buildfarm?" section caught my attention, as this is something I +would be interested in. + +After emailing the author, a {public-thread}[public thread] began on the +subject. + +As we discussed how it could be done, I decided to experiment with the idea of +automating the setup of virtual environments with QEMU. + +This screencast is a simple demo of automating the installation of Alpine Linux +3.12.3 standard x86_64 with +AutoQEMU{empty}footnote:autoqemu[ + The solution was a little too brittle to scale, and some distributions proved + to be particularly problematic. I've {archived}[archived] my progress if + you're interested in what I've done, and maybe wish to continue. +], which is nothing more than POSIX sh, {expect}[expect] scripts and Makefiles +glued together. + +As of this writing, I just worked on it for 2~3 days, so everything is still +pretty ad-hoc. + +The commands from the screencast +were{empty}footnote:script-command[ + Only now, writing again what I ran on the screencast I thought that I should + have tried something like {script}[script]. Maybe next time (thanks + {klaatu}[klaatu] for the tip!). +]: + +[source,sh] +---- +pushd `mktemp -d` +git clone https://euandre.org/git/autoqemu . +make +make install PREFIX=$HOME/.local +autoqemu ssh alpine +---- + +It assumes that `$HOME/.local/bin` is in `$PATH`. diff --git a/src/content/en/screencast/2021/02/07/autoqemu.tar.gz b/src/content/en/screencast/2021/02/07/autoqemu.tar.gz Binary files differnew file mode 100644 index 0000000..c2fa042 --- /dev/null +++ b/src/content/en/screencast/2021/02/07/autoqemu.tar.gz diff --git a/src/content/en/screencast/2021/02/07/autoqemu.webm b/src/content/en/screencast/2021/02/07/autoqemu.webm Binary files differnew file mode 100644 index 0000000..f553efb --- /dev/null +++ b/src/content/en/screencast/2021/02/07/autoqemu.webm diff --git a/src/content/en/screencast/categories.adoc b/src/content/en/screencast/categories.adoc new file mode 100644 index 0000000..feb64ff --- /dev/null +++ b/src/content/en/screencast/categories.adoc @@ -0,0 +1,2 @@ += Articles by category +:type: categories diff --git a/src/content/en/screencast/index.adoc b/src/content/en/screencast/index.adoc new file mode 100644 index 0000000..408b1d5 --- /dev/null +++ b/src/content/en/screencast/index.adoc @@ -0,0 +1,2 @@ += Screencasts +:type: collection diff --git a/src/content/en/slide/2020/10/19/feature-flags.adoc b/src/content/en/slide/2020/10/19/feature-flags.adoc new file mode 100644 index 0000000..1035903 --- /dev/null +++ b/src/content/en/slide/2020/10/19/feature-flags.adoc @@ -0,0 +1,10 @@ += Rollout, feature flag, experiment, operational toggle +:deck: feature-flags.pdf +:article: link:../../../../blog/2020/10/19/feature-flags.html + +Different use cases for backend, frontend and mobile. + +link:{deck}[See the PDF]. + +_The prose version of this presentation is the article +{article}[Feature flags: differences between backend, frontend and mobile]._ diff --git a/src/content/en/slide/2020/10/19/feature-flags.eslaides b/src/content/en/slide/2020/10/19/feature-flags.eslaides new file mode 100644 index 0000000..3392e3e --- /dev/null +++ b/src/content/en/slide/2020/10/19/feature-flags.eslaides @@ -0,0 +1,235 @@ += Rollout, feature flag, experiment, operational toggle + +Different use cases for backend, frontend and mobile + +<<< + +== "Feature flags" tend to come up when talking about continuous deployment + +<<< + +[.lead] +CI: continuous integration +[.lead] +CD: continuous delivery +[.lead] +CD: continuous deployment + +<<< + +== Types +. rollout +. feature flag +. experiment +. operational toggle + +// {favicon.svg} + +<<< + +== Rollout + +=== For *rolling out* a new version of software + +Short-lived using percentages + +// FIXME: links +* a new deployment of kubernetes +* new APK released to the Play Store + +<<< + +== Feature flag + +=== For turning a feature *on* or *off* + +Medium-lived using allow list, A/B test, percentage, app version, etc. + +* :new-chargeback-flow +* :new-debit-card-activation-screen + +<<< + +== Experiment + +=== For analysing behaviour + +Medium-lived using allow list and A/B test + +* :debit-withdrawal-test + +<<< + +== Operational toggle + +=== For disabling features in #crash-like situations + +Long-lived using percentage + +* :bank-barcode-payment +* :savings-bank-barcode-query-provider + +<<< + +[.lead] +We know know about the types +== But they have different relevance for backend, frontend and mobile + +<<< + +== backend +. rollout: k8s blue/green, canary and [.line-through]#common-rollout# common-xp +. feature flag: [.line-through]#common-rollout# common-xp and datasets +. experiment: common-xp +. operational toggle: [.line-through]#common-rollout# common-xp + +<<< + +== frontend +. rollout: CDN and page refreshes +. feature flag: percentages and maybe IPs (no :customer/id on the website) +. experiment: via dynamic backend control +. operational toggle: via dynamic backend control + +<<< + +== backend +. rollout: app stores +. feature flag: via dynamic backend control +. experiment: via dynamic backend control +. operational toggle: via dynamic backend control + +<<< + +[.lead] +Key differentiator is +== how much *control* we have over the environment + +<<< + +== backend +=== full control +// FIXME: emoji +// 🎉 + +<<< + +== frontend +=== partial control +We choose when to make a new version available + +<<< + +== mobile +=== very limited control +* app stores can restrict updates (worse for iOS) +* customers still have to download new versions + +<<< + +== Costs +* more complex code +* compatibility with old app versions +* nesting is exponential + +<<< + +== Benefits +* dynamicity + +<<< + +== weighting costs × benefits +=== The less control we have, the more we value dynamicity + +<<< + +== weighting costs × benefits +* backend: sometimes worth the cost +* frontend: almost always worth the cost +* mobile: *always* worth the cost + +<<< + +== Best practices + +<<< + +== dynamic content > feature flag +Always true for mobile, almost always for frontend + +<<< + +== Use :include-list for named groups +Always true for backend, frontend and mobile + +---- +{:rules + #{{:types :include-list + :content {:filename "debit-team-members.txt"}}}} + +---- + +<<< + +== Always use :app-version +Only for mobile + +---- +{:rules + #{{:types :app-version + :content {:min-version #{{:platform :android + :code 1000000} + {:platform :ios + :code 2000000}}}}}} + +---- + +<<< + +== Extend [.line-through]#common-rollout# common-xp if required + +That's how :include-list, :app-version, etc. were born + +<<< + +== Beware of many nested feature flags +True for backend, frontend and mobile + +<<< + +== Don't delete app-facing feature flags +True for mobile + +<<< + +== Include a feature flag on the whiteboarding phase + +<<< + +== Include deleting/retiring the feature flag at the end + +<<< + +== Avoid renaming a feature flag +Use :app-version with :min-version instead + +<<< + +== And most importantly... + +<<< + +== *Always* rely on a feature flag on the app +Never do a hotfix, avoid expedited releases at all costs + +<<< + +== References +// FIXME: links +. "Feature Toggles (aka Feature Flags)", by Pete Hodgson +. "Continuous integration vs. delivery vs. deployment", by Sten Pittet +. Accelerate, by N. Forsgren, J. Humble and G. Kim +. these slides: euandre.org/slide/ +. prose version of this presentation +. view source diff --git a/src/content/en/slide/2020/11/14/local-first-hype.adoc b/src/content/en/slide/2020/11/14/local-first-hype.adoc new file mode 100644 index 0000000..2aeaf3f --- /dev/null +++ b/src/content/en/slide/2020/11/14/local-first-hype.adoc @@ -0,0 +1,10 @@ += On local-first +:deck: local-first-hype.pdf +:article: link:../../../../blog/2020/11/14/local-first-review.html + +Beyond the CRDT silver bullet. + +link:{deck}[See the PDF]. + +_The prose version of this presentation is the article +{article}[Local-First Software: article review]._ diff --git a/src/content/en/slide/2020/11/14/local-first-hype.eslaides b/src/content/en/slide/2020/11/14/local-first-hype.eslaides new file mode 100644 index 0000000..8f44ad2 --- /dev/null +++ b/src/content/en/slide/2020/11/14/local-first-hype.eslaides @@ -0,0 +1,232 @@ += On "local-first" + +Beyond the CRDT silver bullet + +<<< + +== Part 1 +=== Exposition + +<<< + +== Target +... +* documents +* files +* personal data repositories +.. +Not: banking services, e-commerce, social networking, ride-sharing, etc. + +<<< + +... +== 7 ideals for local-first software + +<<< + +... +== 1 - no spinners: your work at your fingertips + +<<< + +... +== 2 - your work is not trapped on one device + +<<< + +... +== 3 - the network is optional + +<<< + +... +== 4 - seamless collaboration with your colleagues + +<<< + +... +== 5 - the long now + +<<< + +... +== 6 - security and privacy by default + +<<< + +... +== 7 - you retain ultimate ownership and control + +<<< + +== Towards a better future +.... +[.lead] +CRDTs (Conflict-free Replicated Data Types) as a Foundational Technology + +<<< + +== Use case +---- +# in node A and node B +s = "Hello, World" + +# in node A +s = "Hello, Alice" + +# in node B +s = "Hello, Bob" + +---- +How to reconcile those? +* Hello, ABloibce +* Hello, AliceBob +* Hello, BobAlice +* Hello, Alice +\... + +<<< + +== Existing CRDTs differ +* performance +* storage +* compression +* metadata overhead + +<<< + +... +== Hint towards the "automerge" CRDT + +<<< + +...... +[.lead] +@show comparison table, page 9 + +<<< + +== Part 2 +=== Critique + +<<< + +== Software license +... +[.lead] +@"In our opinion, maintaining control and ownership of data does not mean that the software must necessarily be open source." + +<<< + +== Example 1 - intentional restriction +---- +#!/bin/sh +TODAY="$(date +%s)" +LICENSE_EXPIRATION="$(date -d 2020-10-27 +%s)" +if [ "$TODAY" -ge "$LICENSE_EXPIRATION" ]; then + echo 'License expired!' + exit 1 +fi +echo $((2 + 2)) +---- +---- +# today +$ ./useful-adder.sh +4 + +# tomorrow +$ ./useful-adder.sh +License expired! +---- + +<<< + +== Example 2 - unintentional restriction +---- +# today +$ useful-program +# ... useful output ... + +# tomorrow, with more data +$ useful-program +ERROR: Panic! Stack overflow! + +---- + +<<< + +.. +== local-first *requires* free software + +Otherwise "The Long Now" (ideal nº5) is lost + +<<< + +== Denial of existing solutions +.. +[.lead] +@"In principle it is possible to collaborate without a repository service, e.g. by sending patch files by email, but the majority of Git users rely on GitHub." +Solution: either GitHub+CRDTs or git send-email + +<<< + +== Plain-text formats +[.lead] +@"Git is highly optimized for code and similar line-based text file" +It even pulls software to the plain text directtion, e.g.: +* delivery-templates +* common-core.protocols.config +Why not exploit that more? + +<<< + +== Ditching of web applications +.. +[.lead] +@"The architecture of web apps remains fundamentally server-centric +// FIXME: links +Disagree. Contrast PouchDB with Android Instant Apps + +<<< + +== Costs are underrated +* storage +* backups +* maintenance +Example: blog vs vlog + +<<< + +== Real-time collaboration a bit overrated +It is only possible on the presence of reliable, medium-quality network connection +.. +[.lead] +@"X also works when inside an elevator, subway or plane!" + +<<< + +== On CRDTs and developer experience +[.lead] +@"For an app developer, how does the use of a CRDT-based data layer compare to existing storage layers like a SQL database, a filesystem, or CoreData? Is a distributed system harder to write software for? +[.lead] +YES. +// FIXME: link +See "A Note on Distributed Computing" + +<<< + +== Conclusion +Why this is a "paper I love": it took offline-first and ran with it. +But a pinch of CRDT won't make the world local-first. +The tricky part is the end of the sentence: "in spite of the Cloud". + +<<< + +== References +// FIXME: links +. "Local-First Software: You Own Your Data, in spite of the Cloud", by M. Kleppmann, A. Wiggins, P. Van Hardenberg and M. F. McGranaghan +. The Morning Paper article +. "A Note on Distributed Compiting", by J. Waldo, G. Wyant, A. Wollrath and S. Kendall +. these slides: euandre.org/slide/ +. prose version of this presentation +. view source diff --git a/src/content/en/slide/categories.adoc b/src/content/en/slide/categories.adoc new file mode 100644 index 0000000..feb64ff --- /dev/null +++ b/src/content/en/slide/categories.adoc @@ -0,0 +1,2 @@ += Articles by category +:type: categories diff --git a/src/content/en/slide/index.adoc b/src/content/en/slide/index.adoc new file mode 100644 index 0000000..193c98a --- /dev/null +++ b/src/content/en/slide/index.adoc @@ -0,0 +1,2 @@ += Slides +:type: collection diff --git a/src/content/en/til/2020/08/12/filename-timestamp.adoc b/src/content/en/til/2020/08/12/filename-timestamp.adoc new file mode 100644 index 0000000..aa8d63b --- /dev/null +++ b/src/content/en/til/2020/08/12/filename-timestamp.adoc @@ -0,0 +1,31 @@ += Simple filename timestamp +:categories: shell + +When writing Jekyll posts or creating log files with dates on them, I usually +struggle with finding a direct way of accomplishing that. There's a simple +solution: `date -I`. + +[source,sh] +---- +./my-program.sh > my-program.$(date -I).log +cp post-template.md _posts/$(date -I)-post-slug.md +---- + +Using this built-in GNU/Linux tool allows you to `touch $(date -I).md` to +readily create a `2020-08-12.md` file. + +I always had to read `man date` or search the web over and over, and after doing +this repeatedly it became clear that both `date -I` and `date -Is` (`s` here +stands for seconds) are the thing that I'm looking for 95% of the time: + +[source,sh] +---- +# inside my-program.sh +echo "Program started at $(date -Is)" +# output is: +# Program started at 2020-08-12T09:04:58-03:00 +---- + +Both date formats are hierarchical, having the bigger time intervals to the +left. This means that you can easily sort them (and even tab-complete them) +with no extra effort or tool required. diff --git a/src/content/en/til/2020/08/13/code-jekyll.adoc b/src/content/en/til/2020/08/13/code-jekyll.adoc new file mode 100644 index 0000000..6bd90b0 --- /dev/null +++ b/src/content/en/til/2020/08/13/code-jekyll.adoc @@ -0,0 +1,159 @@ += Anchor headers and code lines in Jekyll + +:empty: +:jekyll: https://jekyllrb.com/ +:kramdown: https://kramdown.gettalong.org/ +:rouge: https://rouge.jneen.net/ +:jekyll-hook: https://jekyllrb.com/docs/plugins/hooks/ + +The default Jekyll toolbox ({jekyll}[Jekyll], {kramdown}[kramdown] and +{rouge}[rouge]) doesn't provide with a configuration option to add anchors to +headers and code blocks. + +The best way I found of doing this is by creating a simple Jekyll plugin, more +specifically, a {jekyll-hook}[Jekyll hook]. These allow you to jump in to the +Jekyll build and add a processing stage before of after Jekyll performs +something. + +All you have to do is add the code to `_plugins/my-jekyll-plugin-code.rb`, and +Jekyll knows to pick it up and call your code on the appropriate time. + +== Anchor on headers + +:jemoji: https://github.com/jekyll/jemoji +:jekyll-mentions: https://github.com/jekyll/jekyll-mentions +:html-regex: https://stackoverflow.com/questions/1732348/regex-match-open-tags-except-xhtml-self-contained-tags/1732454#1732454 + +Since I wanted to add anchors to headers in all documents, this Jekyll hook +works on `:documents` after they have been transformed into HTML, the +`:post_render` phase: + +[source,ruby] +---- +Jekyll::Hooks.register :documents, :post_render do |doc| + if doc.output_ext == ".html" + doc.output = + doc.output.gsub( + /<h([1-6])(.*?)id="([\w-]+)"(.*?)>(.*?)<\/h[1-6]>/, + '<a href="#\3"><h\1\2id="\3"\4>\5</h\1></a>' + ) + end +end +---- + +I've derived my implementations from two +"official"{empty}footnote:official[ + I don't know how official they are, I just assumed it because they live in the + same organization inside GitHub that Jekyll does. +] hooks, {jemoji}[jemoji] and {jekyll-mentions}[jekyll-mentions]. + +All I did was to wrap the header tag inside an `<a>`, and set the `href` of that +`<a>` to the existing id of the header. Before the hook the HTML looks like: + +[source,html] +---- +...some unmodified text... +<h2 id="my-header"> + My header +</h2> +...more unmodified text... +---- + +And after the hook should turn that into: + +[source,html] +---- +...some unmodified text... +<a href="#my-header"> + <h2 id="my-header"> + My header + </h2> +</a> +...more unmodified text... +---- + +The used regexp tries to match only h1-h6 tags, and keep the rest of the HTML +attributes untouched, since this isn't a general HTML parser, but the generated +HTML is somewhat under your control. Use at your own risk because +{html-regex}[you shouldn't parse HTML with regexps]. Also I used this strategy +in my environment, where no other plugins are installed. I haven't considered +how this approach may conflict with other Jekyll plugins. + +In the new anchor tag you can add your custom CSS class to style it as you wish. + +== Anchor on code blocks + +Adding anchors to code blocks needs a little bit of extra work, because line +numbers themselves don't have preexisting ids, so we need to generate them +without duplications between multiple code blocks in the same page. + +Similarly, this Jekyll hook also works on `:documents` in the `:post_render` +phase: + +[source,ruby] +---- +PREFIX = '<pre class="lineno">' +POSTFIX = '</pre>' +Jekyll::Hooks.register :documents, :post_render do |doc| + if doc.output_ext == ".html" + code_block_counter = 1 + doc.output = doc.output.gsub(/<pre class="lineno">[\n0-9]+<\/pre>/) do |match| + line_numbers = match + .gsub(/<pre class="lineno">([\n0-9]+)<\/pre>/, '\1') + .split("\n") + + anchored_line_numbers_array = line_numbers.map do |n| + id = "B#{code_block_counter}-L#{n}" + "<a id=\"#{id}\" href=\"##{id}\">#{n}</a>" + end + code_block_counter += 1 + + PREFIX + anchored_line_numbers_array.join("\n") + POSTFIX + end + end +end +---- + +This solution assumes the default Jekyll toolbox with code line numbers turned +on in `_config.yml`: + +[source,yaml] +---- +kramdown: + syntax_highlighter_opts: + span: + line_numbers: false + block: + line_numbers: true +---- + +The anchors go from B1-L1 to BN-LN, using the `code_block_counter` to track +which code block we're in and don't duplicate anchor ids. Before the hook the +HTML looks like: + +[source,html] +---- +...some unmodified text... +<pre class="lineno">1 +2 +3 +4 +5 +</pre> +...more unmodified text... +---- + +And after the hook should turn that into: + +[source,html] +---- +...some unmodified text... +<pre class="lineno"><a id="B1-L1" href="#B1-L1">1</a> +<a id="B1-L2" href="#B1-L2">2</a> +<a id="B1-L3" href="#B1-L3">3</a> +<a id="B1-L4" href="#B1-L4">4</a> +<a id="B1-L5" href="#B1-L5">5</a></pre> +...more unmodified text... +---- + +Happy writing :) diff --git a/src/content/en/til/2020/08/14/browse-git.adoc b/src/content/en/til/2020/08/14/browse-git.adoc new file mode 100644 index 0000000..6b3ff6d --- /dev/null +++ b/src/content/en/til/2020/08/14/browse-git.adoc @@ -0,0 +1,76 @@ += Browse a git repository at a specific commit +:categories: git + +I commonly use tools like `git log` together with `git show` when inspecting +past changes in a repository: + +[source,sh] +---- +git log +# search for a the commit I'm looking for +git show <my-commit> +# see the diff for the commit +---- + +But I also wanted to not only be able to look at the diff of a specific commit, +but to browse the whole repository at that specific commit. + +I used to accomplish it the "brute force" way: clone the whole repository in +another folder and checkout the commit there: + +[source,sh] +---- +git clone <original-repo> /tmp/tmp-repo-clone +cd /tmp-repo-clone +git checkout <my-commit> +---- + +But git itself allows we to specific the directory of the checkout by using the +`--work-tree` global git flag. This is what `man git` says about it: + +[source,text] +---- +--work-tree=<path> + Set the path to the working tree. It can be an absolute path or a path relative to the current working + directory. This can also be controlled by setting the GIT_WORK_TREE environment variable and the + core.worktree configuration variable (see core.worktree in git-config(1) for a more detailed + discussion). +---- + +So it allows us to set the desired path of the working tree. So if we want to +copy the contents of the current working tree into `copy/`: + +[source,sh] +---- +mkdir copy +git --work-tree=copy/ checkout . +---- + +After that `copy/` will contain a replica of the code in HEAD. But to checkout +a specific, we need some extra parameters: + +[source,sh] +---- +git --work-tree=<dir> checkout <my-commit> -- . +---- + +There's an extra `-- .` at the end, which initially looks like we're sending +Morse signals to git, but we're actually saying to `git-checkout` which sub +directory of `<my-commit>` we want to look at. Which means we can do something +like: + +[source,sh] +---- +git --work-tree=<dir> checkout <my-commit> -- src/ +---- + +And with that `<dir>` will only contain what was inside `src/` at `<commit>`. + +After any of those checkouts, you have to `git reset .` to reset your current +staging area back to what it was before the checkout. + +== References + +:so-link: https://stackoverflow.com/a/16493707 + +. {so-link}[GIT: Checkout to a specific folder] (StackOverflow) diff --git a/src/content/en/til/2020/08/16/git-search.adoc b/src/content/en/til/2020/08/16/git-search.adoc new file mode 100644 index 0000000..4113f3f --- /dev/null +++ b/src/content/en/til/2020/08/16/git-search.adoc @@ -0,0 +1,49 @@ += Search in git +:categories: git + +Here's a useful trio to know about to help you search things in git: + +. `git show <commit>` +. `git log --grep='<regexp>'` +. `git grep '<regexp>' [commit]` + +== 1. `git show <commit>` + +Show a specific commit and it's diff: + +[source,sh] +---- +git show +# shows the latest commit +git show <commit> +# shows an specific <commit> +git show v1.2 +# shows commit tagged with v1.2 +---- + +== 2. `git log --grep='<regexp>'` + +Search through the commit messages: + +[source,sh] +---- +git log --grep='refactor' +---- + +== 3. `git grep '<regexp>' [commit]` + +:browse-article: link:../14/browse-git.html + +Search content in git history: + +[source,sh] +---- +git grep 'TODO' +# search the repository for the "TODO" string +git grep 'TODO' $(git rev-list --all) +# search the whole history for "TODO" string +---- + +And if you find an occurrence of the regexp in a specific commit and you want to +browse the repository in that point in time, you can {browse-article}[use git +checkout for that]. diff --git a/src/content/en/til/2020/08/28/grep-online.adoc b/src/content/en/til/2020/08/28/grep-online.adoc new file mode 100644 index 0000000..77363ab --- /dev/null +++ b/src/content/en/til/2020/08/28/grep-online.adoc @@ -0,0 +1,133 @@ += Grep online repositories +:categories: git + +:cgit: https://git.zx2c4.com/cgit/ + +I often find interesting source code repositories online that I want to grep for +some pattern but I can't, because either: + +* the repository is on {cgit}[cgit] or a similar code repository that doesn't + allow search in files, or; +* the search function is really bad, and doesn't allow me to use regular + expressions for searching patterns in the code. + +Here's a simple script that allows you to overcome that problem easily: + +[source,sh] +---- +#!/usr/bin/env bash +set -eu + +end="\033[0m" +red="\033[0;31m" +red() { echo -e "${red}${1}${end}"; } + +usage() { + red "Missing argument $1.\n" + cat <<EOF +Usage: + $0 <REGEX_PATTERN> <REPOSITORY_URL> + + Arguments: + REGEX_PATTERN Regular expression that "git grep" can search + REPOSITORY_URL URL address that "git clone" can download the repository from + +Examples: + Searching "make get-git" in cgit repository: + git search 'make get-git' https://git.zx2c4.com/cgit/ + git search 'make get-git' https://git.zx2c4.com/cgit/ -- \$(git rev-list --all) +EOF + exit 2 +} + + +REGEX_PATTERN="${1:-}" +REPOSITORY_URL="${2:-}" +[[ -z "${REGEX_PATTERN}" ]] && usage 'REGEX_PATTERN' +[[ -z "${REPOSITORY_URL}" ]] && usage 'REPOSITORY_URL' + +mkdir -p /tmp/git-search +DIRNAME="$(echo "${REPOSITORY_URL%/}" | rev | cut -d/ -f1 | rev)" +if [[ ! -d "/tmp/git-search/${DIRNAME}" ]]; then + git clone "${REPOSITORY_URL}" "/tmp/git-search/${DIRNAME}" +fi +pushd "/tmp/git-search/${DIRNAME}" + +shift 3 || shift 2 # when "--" is missing +git grep "${REGEX_PATTERN}" "${@}" +---- + +It is a wrapper around `git grep` that downloads the repository when missing. +Save in a file called `git-search`, make the file executable and add it to your +path. + +Overview: + +* _lines 1~2_: ++ +Bash shebang and the `set -eu` options to exit on error or undefined +variables. + +* _lines 4~30_: ++ +Usage text to be printed when providing less arguments than expected. + +* _line 33_: ++ +Extract the repository name from the URL, removing trailing slashes. + +* _lines 34~37_: ++ +Download the repository when missing and go to the folder. + +* _line 39_: ++ +Make the variable `$@` contain the rest of the unused arguments. + +* _line 40_: ++ +Perform `git grep`, forwarding the remaining arguments from `$@`. + +Example output: + +[source,sh] +---- +$ git search 'make get-git' https://git.zx2c4.com/cgit/ +Clonage dans '/tmp/git-search/cgit'... +remote: Enumerating objects: 542, done. +remote: Counting objects: 100% (542/542), done. +remote: Compressing objects: 100% (101/101), done. +warning: object 51dd1eff1edc663674df9ab85d2786a40f7ae3a5: gitmodulesParse: could not parse gitmodules blob +remote: Total 7063 (delta 496), reused 446 (delta 441), pack-reused 6521 +Réception d'objets: 100% (7063/7063), 8.69 Mio | 5.39 Mio/s, fait. +Résolution des deltas: 100% (5047/5047), fait. +/tmp/git-search/cgit ~/dev/libre/songbooks/docs +README: $ make get-git + +$ git search 'make get-git' https://git.zx2c4.com/cgit/ +/tmp/git-search/cgit ~/dev/libre/songbooks/docs +README: $ make get-git +---- + +Subsequent greps on the same repository are faster because no download is +needed. + +When no argument is provided, it prints the usage text: + +[source,sh] +---- +$ git search +Missing argument REGEX_PATTERN. + +Usage: + /home/andreh/dev/libre/dotfiles/scripts/ad-hoc/git-search <REGEX_PATTERN> <REPOSITORY_URL> + + Arguments: + REGEX_PATTERN Regular expression that "git grep" can search + REPOSITORY_URL URL address that "git clone" can download the repository from + +Examples: + Searching "make get-git" in cgit repository: + git search 'make get-git' https://git.zx2c4.com/cgit/ + git search 'make get-git' https://git.zx2c4.com/cgit/ -- $(git rev-list --all) +---- diff --git a/src/content/en/til/2020/09/04/cli-email-fun-profit.adoc b/src/content/en/til/2020/09/04/cli-email-fun-profit.adoc new file mode 100644 index 0000000..1da1154 --- /dev/null +++ b/src/content/en/til/2020/09/04/cli-email-fun-profit.adoc @@ -0,0 +1,76 @@ += Send emails using the command line for fun and profit! + +:ssmtp: https://wiki.archlinux.org/index.php/SSMTP +:mailutils: https://mailutils.org/ + +Here are a few reasons why: + +. send yourself and other people notification of cronjobs, scripts runs, CI + jobs, _etc._ +. leverage the POSIX pipe `|`, and pipe emails away! +. because you can. + +Reason 3 is the fun part, reasons 1 and 2 are the profit part. + +First {ssmpt}[install and configure SSMTP] for using, say, Gmail as the email +server: + +[source,sh] +---- +# file /etc/ssmtp/ssmtp.conf +FromLineOverride=YES +MailHub=smtp.gmail.com:587 +UseSTARTTLS=YES +UseTLS=YES +rewriteDomain=gmail.com +root=username@gmail.com +AuthUser=username +AuthPass=password +---- + +Now install {mailutils}[GNU Mailutils] (`sudo apt-get install mailutils` or the +equivalent on your OS), and send yourself your first email: + +[source,sh] +---- +echo body | mail -aFrom:email@example.com email@example.com -s subject +---- + +And that's about it, you've got mail. Here are some more places where it might +be applicable: + +[source,sh] +---- +# report a backup cronjob, attaching logs +set -e + +finish() { + status=$? + if [[ $status = 0 ]]; then + STATUS="SUCCESS (status $status)" + else + STATUS="FAILURE (status $status)" + fi + + mail user@example.com \ + -s "Backup job report on $(hostname): ${STATUS}" \ + --content-type 'text/plain; charset=utf-8' \ + -A"$LOG_FILE" <<< 'The log report is in the attachment.' +} +trap finish EXIT + +do-long-backup-cmd-here +---- + +[source,sh] +---- +# share the output of a cmd with someone +some-program | mail someone@example.com -s "The weird logs that I was talking about" +---- + +...and so on. + +You may consider adding a `alias mail='mail -aFrom:email@example.com'` so you +don't keep re-entering the ``From:'' part. + +Send yourself some emails to see it working! diff --git a/src/content/en/til/2020/09/05/oldschool-pr.adoc b/src/content/en/til/2020/09/05/oldschool-pr.adoc new file mode 100644 index 0000000..392ec67 --- /dev/null +++ b/src/content/en/til/2020/09/05/oldschool-pr.adoc @@ -0,0 +1,110 @@ += Pull requests with Git, the old school way +:categories: git + +:empty: +:cgit: https://git.zx2c4.com/cgit/ + +It might be news to you, as it was to me, that "pull requests" that you can +create on a Git hosting provider’s web +UI{empty}footnote:pr-webui[ + And maybe even using the Git hosting provider’s API from the command line! +] like GitLab/Bitbucket/GitHub actually comes from Git itself: +`git request-pull`. + +At the very core, they accomplish the same thing: both the original and the web +UI ones are ways for you to request the project maintainers to pull in your +changes from your fork. It’s like saying: "hi there, I did some changes on my +clone of the repository, what do you think about bringing those in?". + +The only difference is that you’re working with only Git itself, so you’re not +tied to any Git hosting provider: you can send pull requests across them +transparently! You could even use your own {cgit}[cgit] installation. No need +to be locked in by any of them, putting the "D" back in "DVCS": it’s a +*distributed* version control system. + +== `git request-pull` introduction + +Here’s the raw output of a `git request-pull`: + +[source,sh] +---- +$ git request-pull HEAD public-origin +The following changes since commit 302c9f2f035c0360acd4e13142428c100a10d43f: + + db post: Add link to email exchange (2020-09-03 21:23:55 -0300) + +are available in the Git repository at: + + https://euandre.org/git/euandre.org/ + +for you to fetch changes up to 524c646cdac4153e54f2163e280176adbc4873fa: + + db post: better pinpoint sqlite unsuitability (2020-09-03 22:08:56 -0300) + +---------------------------------------------------------------- +EuAndreh (1): + db post: better pinpoint sqlite unsuitability + + _posts/2020-08-31-the-database-i-wish-i-had.md | 12 ++++++------ + 1 file changed, 6 insertions(+), 6 deletions(-) +---- + +That very first line is saying: "create me a pull request with only a single +commit, defined by `HEAD`, and use the URL defined by `public-origin`". + +Here’s a pitfall: you may try using your `origin` remote at first where I put +`public-origin`, but that is many times pointing to something like +`git@example.com`, or `git.example.com:repo.git` (check that with +`git remote -v | grep origin`). On both cases those are addresses available for +interaction via SSH, and it would be better if your pull requests used an +address ready for public consumption. + +A simple solution for that is for you to add the `public-origin` alias as the +HTTPS alternative to the SSH version: + +[source,sh] +---- +$ git remote add public-origin https://example.com/user/repo +---- + +Every Git hosting provider exposes repositories via HTTPS. + +Experiment it yourself, and get acquainted with the CLI. + +== Delivering decentralized pull requests + +:cli-email: link:../04/cli-email-fun-profit.html + +Now that you can create the content of a pull request, you can just +{cli-email}[deliver it] to the interested parties email: + +[source,sh] +---- +# send a PR with your last commit to the author's email +git request-pull HEAD public-origin | mail author@example.com -s "PR: Add thing to repo" + +# send a PR with your last 5 commits to the project's mailing +# list, including the patch +git request-pull -p HEAD~5 public-origin | \ + mail list@example.com -s "PR: Add another thing to repo" + +# send every commit that is new in "other-branch" +git request-pull master public-origin other-branch | \ + mail list@example.com -s 'PR: All commits from my "other-brach"' +---- + +== Conclusion + +:distgit: https://drewdevault.com/2018/07/23/Git-is-already-distributed.html + +In practice, I’ve never used or seen anyone use pull requests this way: +everybody is just {distgit}[sending patches via email]. + +If you stop to think about this model, the problem of "Git hosting providers +becoming too centralized" is a non-issue, and "Git federation" proposals are a +less attractive as they may sound initially. + +Using Git this way is not scary or so weird as the first impression may suggest. +It is actually how Git was designed to be used. + +Check `git help request-pull` for more info. diff --git a/src/content/en/til/2020/10/11/search-git-history.adoc b/src/content/en/til/2020/10/11/search-git-history.adoc new file mode 100644 index 0000000..696368c --- /dev/null +++ b/src/content/en/til/2020/10/11/search-git-history.adoc @@ -0,0 +1,29 @@ += Search changes to a filename pattern in Git history +:categories: git + +:yet: link:../../08/14/browse-git.html +:another: link:../../08/16/git-search.html +:entry: link:../../08/28/grep-online.html + +This is {yet}[yet] {another}[another] {entry}["search in Git"] TIL entry. You +could say that Git has a unintuitive CLI, or that is it very powerful. + +I wanted to search for an old file that I new that was in the history of the +repository, but was deleted some time ago. So I didn't really remember the +name, only bits of it. + +I immediately went to the list of TILs I had written on searching in Git, but it +wasn't readily obvious how to do it, so here it goes: + +[source,sh] +---- +git log -- *pattern* +---- + +You could add globs before the pattern to match things on any directory, and add +our `-p` friend to promptly see the diffs: + +[source,sh] +---- +git log -p -- **/*pattern* +---- diff --git a/src/content/en/til/2020/11/08/find-broken-symlink.adoc b/src/content/en/til/2020/11/08/find-broken-symlink.adoc new file mode 100644 index 0000000..624d24a --- /dev/null +++ b/src/content/en/til/2020/11/08/find-broken-symlink.adoc @@ -0,0 +1,25 @@ += Find broken symlinks with "find" +:categories: shell + +:annex: https://git-annex.branchable.com/ +:annex-wanted: https://git-annex.branchable.com/git-annex-wanted/ + +The `find` command knows how to show broken symlinks: + +[source,sh] +---- +find . -xtype l +---- + +This was useful to me when combined with {annex}[Git Annex]. Its +{annex-wanted}[`wanted`] option allows you to have a "sparse" checkout of the +content, and save space by not having to copy every annexed file locally: + +[source,sh] +---- +git annex wanted . 'exclude=Music/* and exclude=Videos/*' +---- + +You can `find` any broken symlinks outside those directories by querying with +Git Annex itself, but `find . -xtype l` works on other places too, where broken +symlinks might be a problem. diff --git a/src/content/en/til/2020/11/12/diy-nix-bash-ci.adoc b/src/content/en/til/2020/11/12/diy-nix-bash-ci.adoc new file mode 100644 index 0000000..97ace30 --- /dev/null +++ b/src/content/en/til/2020/11/12/diy-nix-bash-ci.adoc @@ -0,0 +1,63 @@ += DIY bare bones CI server with Bash and Nix +:categories: ci +:sort: 2 + +:post-receive: https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks +:example-project: https://euandreh.xyz/remembering/ci.html + +With a server with Nix installed (no need for NixOS), you can leverage its build +isolation for running CI jobs by adding a {post-receive}[post-receive] Git hook +to the server. + +In most of my project I like to keep a `test` attribute which runs the test with +`nix-build -A test`. This way, a post-receive hook could look like: + +[source,sh] +---- +#!/usr/bin/env bash +set -Eeuo pipefail +set -x + +LOGS_DIR="/data/static/ci-logs/libedn" +mkdir -p "$LOGS_DIR" +LOGFILE="${LOGS_DIR}/$(date -Is)-$(git rev-parse master).log" +exec &> >(tee -a "${LOGFILE}") + +unset GIT_DIR +CLONE="$(mktemp -d)" +git clone . "$CLONE" +pushd "$CLONE" + +finish() { + printf "\n\n>>> exit status was %s\n" "$?" +} +trap finish EXIT + +nix-build -A test +---- + +We initially (lines #5 to #8) create a log file, named after _when_ the run is +running and for _which_ commit it is running for. The `exec` and `tee` combo +allows the output of the script to go both to `stdout` _and_ the log file. This +makes the logs output show up when you do a `git push`. + +Lines #10 to #13 create a fresh clone of the repository and line #20 runs the +test command. + +After using a similar post-receive hook for a while, I now even generate a +simple HTML file to make the logs available ({example-project}[example project]) +through the browser. + +== Upsides + +No vendor lock-in, as all you need is a server with Nix installed. + +And if you pin the Nixpkgs version you're using, this very simple setup yields +extremely sandboxed runs on a very hermetic environment. + +== Downsides + +Besides the many missing shiny features of this very simplistic CI, `nix-build` +can be very resource intensive. Specifically, it consumes too much memory. So +if it has to download too many things, or the build closure gets too big, the +server might very well run out of memory. diff --git a/src/content/en/til/2020/11/12/git-bisect-automation.adoc b/src/content/en/til/2020/11/12/git-bisect-automation.adoc new file mode 100644 index 0000000..dff8737 --- /dev/null +++ b/src/content/en/til/2020/11/12/git-bisect-automation.adoc @@ -0,0 +1,25 @@ += Git bisect automation +:categories: git +:sort: 1 + +It is good to have an standardized way to run builds and tests on the repository +of a project, so that you can find when a bug was introduced by using +`git bisect run`. + +I've already been in the situation when a bug was introduced and I didn't know +how it even was occurring, and running Git bisect over hundreds of commits to +pinpoint the failing commit was very empowering: + +[source,sh] +---- +$ GOOD_COMMIT_SHA=e1fd0a817d192c5a5df72dd7422e36558fa78e46 +$ git bisect start HEAD $GOOD_COMMIT_SHA +$ git bisect run sn -c './build.sh && ./run-failing-case.sh' +---- + +Git will than do a binary search between the commits, and run the commands you +provide it with to find the failing commit. + +Instead of being afraid of doing a bisect, you should instead leverage it, and +make Git help you dig through the history of the repository to find the bad +code. diff --git a/src/content/en/til/2020/11/12/useful-bashvars.adoc b/src/content/en/til/2020/11/12/useful-bashvars.adoc new file mode 100644 index 0000000..fb148fb --- /dev/null +++ b/src/content/en/til/2020/11/12/useful-bashvars.adoc @@ -0,0 +1,61 @@ += Useful Bash variables +:categories: shell + +:bash: https://www.gnu.org/software/bash/ +:bash-bang-bang: https://www.gnu.org/software/bash/manual/bash.html#Event-Designators +:bash-dollar-underscore: https://www.gnu.org/software/bash/manual/bash.html#Special-Parameters + +{bash}[GNU Bash] has a few two letter variables that may be useful when typing +on the terminal. + +== `!!`: the text of the last command + +The {bash-bang-bang}[`!!` variable] refers to the previous command, and I find +useful when following chains for symlinks: + +[source,sh] +---- +$ which git +/run/current-system/sw/bin/git +$ readlink $(!!) +readlink $(which git) +/nix/store/5bgr1xpm4m0r72h9049jbbhagxdyrnyb-git-2.28.0/bin/git +---- + +It is also useful when you forget to prefix `sudo` to a command that requires +it: + +[source,sh] +---- +$ requires-sudo.sh +requires-sudo.sh: Permission denied +$ sudo !! +sudo ./requires-sudo.sh +# all good +---- + +Bash prints the command expansion before executing it, so it is better for you +to follow along what it is doing. + +== `$_`: most recent parameter + +The {bash-dollar-underscore}[`$_` variable] will give you the most recent +parameter you provided to a previous argument, which can save you typing +sometimes: + +[source,sh] +---- +# instead of... +$ mkdir -p a/b/c/d/ +$ cd a/b/c/d/ + +# ...you can: +$ mkdir -p a/b/c/d/ +$ cd $_ +---- + +== Conclusion + +I wouldn't use those in a script, as it would make the script terser to read, I +find those useful shortcut that are handy when writing at the interactive +terminal. diff --git a/src/content/en/til/2020/11/14/gpodder-media.adoc b/src/content/en/til/2020/11/14/gpodder-media.adoc new file mode 100644 index 0000000..f722f35 --- /dev/null +++ b/src/content/en/til/2020/11/14/gpodder-media.adoc @@ -0,0 +1,21 @@ += gPodder as a media subscription manager + +:re-discover: https://www.charlieharrington.com/unexpected-useless-and-urgent +:gpodder: https://gpodder.github.io/ + +As we {re-discover}[re-discover] the value of Atom/RSS feeds, most useful feed +clients I know of don't support media, specifically audio and video. + +{gpodder}[gPodder] does. + +It is mostly know as a desktop podcatcher. But the thing about podcasts is that +the feed is provided through an RSS/Atom feed. So you can just use gPodder as +your media feed client, where you have control of what you look at. + +I audio and video providers I know of offer an RSS/Atom view of their content, +so you can, say, treat any YouTube channel like a feed on its own. + +gPodder will then managed your feeds, watched/unwatched, queue downloads, etc. + +Being obvious now, it was a big finding for me. If it got you interested, I +recommend you giving gPodder a try. diff --git a/src/content/en/til/2020/11/30/git-notes-ci.adoc b/src/content/en/til/2020/11/30/git-notes-ci.adoc new file mode 100644 index 0000000..48a996b --- /dev/null +++ b/src/content/en/til/2020/11/30/git-notes-ci.adoc @@ -0,0 +1,112 @@ += Storing CI data on Git notes +:categories: git ci + +:git-notes: https://git-scm.com/docs/git-notes +:nix-bash-ci: link:../12/diy-nix-bash-ci.html +:cgit: https://git.zx2c4.com/cgit/ + +Extending the bare bones CI server I've {nix-bash-ci}[talked about before], +divoplade on Freenode suggested storing CI artifacts on {git-notes}[Git notes], +such as tarballs, binaries, logs, _etc_. + +I've written a small script that will put log files and CI job data on Git +notes, and make it visible on the porcelain log. It is a simple extension of +the previous article: + +[source,sh] +---- +#!/usr/bin/env bash +set -Eeuo pipefail +set -x + +PREFIX='/srv/ci/vps' +mkdir -p "$PREFIX" +read -r _ SHA _ # oldrev newrev refname +FILENAME="$(date -Is)-$SHA.log" +LOGFILE="$PREFIX/$FILENAME" +exec &> >(tee -a "$LOGFILE") + +echo "Starting CI job at: $(date -Is)" + +finish() { + STATUS="$?" + printf "\n\n>>> exit status was %s\n" "$STATUS" + echo "Finishing CI job at: $(date -Is)" + popd + NOTE=$(cat <<EOF +See CI logs with: + git notes --ref=refs/notes/ci-logs show $SHA + git notes --ref=refs/notes/ci-data show $SHA +EOF +) + git notes --ref=refs/notes/ci-data add -f -m "$STATUS $FILENAME" + git notes --ref=refs/notes/ci-logs add -f -F "$LOGFILE" + git notes add -f -m "$NOTE" + printf "\n\n>>> CI logs added as Git note." +} +trap finish EXIT + +unset GIT_DIR +CLONE="$(mktemp -d)" +git clone . "$CLONE" +pushd "$CLONE" +git config --global user.email git@euandre.org +git config --global user.name 'EuAndreh CI' + +./container make check site +./container make publish +---- + +The important part is in the `finish()` function: - #25 stores the exit status +and the generated filename separated by spaces; - #26 adds the log file in a +note using the `refs/notes/ci-logs` ref; - #27 it adds a note to the commit +saying how to see the logs. + +A commit now has an attached note, and shows it whenever you look at it: + +[source,diff] +---- +$ git show 87c57133abd8be5d7cc46afbf107f59b26066575 +commit 87c57133abd8be5d7cc46afbf107f59b26066575 +Author: EuAndreh <eu@euandre.org> +Date: Wed Feb 24 21:58:28 2021 -0300 + + vps/machines.scm: Change path to cronjob files + +Notes: + See CI logs with: + git notes --ref=refs/notes/ci-logs show 87c57133abd8be5d7cc46afbf107f59b26066575 + git notes --ref=refs/notes/ci-data show 87c57133abd8be5d7cc46afbf107f59b26066575 + +diff --git a/servers/vps/machines.scm b/servers/vps/machines.scm +index d1830ca..a4ccde7 100644 +--- a/servers/vps/machines.scm ++++ b/servers/vps/machines.scm +@@ -262,8 +262,8 @@ pki " mail-domain " key \"" (tls-priv-for mail-domain) "\"")) + (service mcron-service-type + (mcron-configuration + (jobs +- (list #~(job "30 1 * * 1" "guix gc -d") +- #~(job "30 0 * * *" "/var/lib/euandreh/backup.sh"))))) ++ (list #~(job "30 1 * * 1" "/opt/bin/gc.sh") ++ #~(job "30 0 * * *" "/opt/bin/backup.sh"))))) + (service dhcp-client-service-type) + #; + (service opensmtpd-service-type +---- + +Other tools such as {cgit}[cgit] will also show notes on the web interface: +https://euandre.org/git/servers/commit?id=87c57133abd8be5d7cc46afbf107f59b26066575. + +You can go even further: since cgit can serve raw blob directly, you can even +serve such artifacts (log files, release artifacts, binaries) from cgit itself: + +[source,sh] +---- +$ SHA="$(git notes --ref=refs/notes/ci-logs list 87c57133abd8be5d7cc46afbf107f59b26066575)" +$ echo "https://euandre.org/git/servers/blob?id=$SHA" +https://euandre.org/git/servers/blob?id=1707a97bae24e3864fe7943f8dda6d01c294fb5c +---- + +And like that you'll have cgit serving the artifacts for you: +https://euandre.org/git/servers/blob?id=1707a97bae24e3864fe7943f8dda6d01c294fb5c. diff --git a/src/content/en/til/2020/12/15/shellcheck-repo.adoc b/src/content/en/til/2020/12/15/shellcheck-repo.adoc new file mode 100644 index 0000000..387e793 --- /dev/null +++ b/src/content/en/til/2020/12/15/shellcheck-repo.adoc @@ -0,0 +1,165 @@ += Awk snippet: ShellCheck all scripts in a repository +:categories: shell +:updatedat: 2020-12-16 + +:awk-20-min: https://ferd.ca/awk-in-20-minutes.html +:shellcheck: https://www.shellcheck.net/ + +Inspired by Fred Herbert's "{awk-20-min}[Awk in 20 Minutes]", here's a problem I +just solved with a line of Awk: run ShellCheck in all scripts of a repository. + +In my repositories I usually have Bash and POSIX scripts, which I want to keep +tidy with {shellcheck}[ShellCheck]. Here's the first version of +`assert-shellcheck.sh`: + +[source,sh] +---- +#!/bin/sh -eux + +find . -type f -name '*.sh' -print0 | xargs -0 shellcheck +---- + +This is the type of script that I copy around to all repositories, and I want it +to be capable of working on any repository, without requiring a list of files to +run ShellCheck on. + +This first version worked fine, as all my scripts had the `.sh' ending. But I +recently added some scripts without any extension, so `assert-shellcheck.sh` +called for a second version. The first attempt was to try grepping the shebang +line: + +[source,sh] +---- +$ grep '^#!/' assert-shellcheck.sh +#!/usr/sh +---- + +Good, we have a grep pattern on the first try. Let's try to find all the +matching files: + +[source,sh] +---- +$ find . -type f | xargs grep -l '^#!/' +./TODOs.org +./.git/hooks/pre-commit.sample +./.git/hooks/pre-push.sample +./.git/hooks/pre-merge-commit.sample +./.git/hooks/fsmonitor-watchman.sample +./.git/hooks/pre-applypatch.sample +./.git/hooks/pre-push +./.git/hooks/prepare-commit-msg.sample +./.git/hooks/commit-msg.sample +./.git/hooks/post-update.sample +./.git/hooks/pre-receive.sample +./.git/hooks/applypatch-msg.sample +./.git/hooks/pre-rebase.sample +./.git/hooks/update.sample +./build-aux/with-guile-env.in +./build-aux/test-driver +./build-aux/missing +./build-aux/install-sh +./build-aux/install-sh~ +./bootstrap +./scripts/assert-todos.sh +./scripts/songbooks +./scripts/compile-readme.sh +./scripts/ci-build.sh +./scripts/generate-tasks-and-bugs.sh +./scripts/songbooks.in +./scripts/with-container.sh +./scripts/assert-shellcheck.sh +---- + +This approach has a problem, though: it includes files ignored by Git, such as +`builld-aux/install-sh~`, and even goes into the `.git/` directory and finds +sample hooks in `.git/hooks/*`. + +To list the files that Git is tracking we'll try `git ls-files`: + +[source,sh] +---- +$ git ls-files | xargs grep -l '^#!/' +TODOs.org +bootstrap +build-aux/with-guile-env.in +old/scripts/assert-docs-spelling.sh +old/scripts/build-site.sh +old/scripts/builder.bats.sh +scripts/assert-shellcheck.sh +scripts/assert-todos.sh +scripts/ci-build.sh +scripts/compile-readme.sh +scripts/generate-tasks-and-bugs.sh +scripts/songbooks.in +scripts/with-container.sh +---- + +It looks to be almost there, but the `TODOs.org` entry shows a flaw in it: grep +is looking for a +'^#!/'+ pattern on any part of the file. In my case, +`TODOs.org` had a snippet in the middle of the file where a line started with ++#!/bin/sh+. + +So what we actually want is to match the *first* line against the pattern. We +could loop through each file, get the first line with `head -n 1` and grep +against that, but this is starting to look messy. I bet there is another way of +doing it concisely... + +Let's try Awk. I need a way to select the line numbers to replace `head -n 1`, +and to stop processing the file if the pattern matches. A quick search points +me to using `FNR` for the former, and `{ nextline }` for the latter. Let's try +it: + +[source,sh] +---- +$ git ls-files | xargs awk 'FNR>1 { nextfile } /^#!\// { print FILENAME; nextfile }' +bootstrap +build-aux/with-guile-env.in +old/scripts/assert-docs-spelling.sh +old/scripts/build-site.sh +old/scripts/builder.bats.sh +scripts/assert-shellcheck.sh +scripts/assert-todos.sh +scripts/ci-build.sh +scripts/compile-readme.sh +scripts/generate-tasks-and-bugs.sh +scripts/songbooks.in +scripts/with-container.sh +---- + +Great! Only `TODOs.org` is missing, but the script is much better: instead of +matching against any part of the file that may have a shebang-like line, we only +look for the first. Let's put it back into the `assert-shellcheck.sh` file and +use `NULL` for separators to accommodate files with spaces in the name: + +[source,sh] +---- +#!/usr/sh -eux + +git ls-files -z | \ + xargs -0 awk 'FNR>1 { nextfile } /^#!\// { print FILENAME; nextfile }' | \ + xargs shellcheck +---- + +This is where I've stopped, but I imagine a likely improvement: match against +only +#!/bin/sh+ and +#!/usr/bin/env bash+ shebangs (the ones I use most), to +avoid running ShellCheck on Perl files, or other shebangs. + +Also when reviewing the text of this article, I found that `{ nextfile }` is a +GNU Awk extension. It would be an improvement if `assert-shellcheck.sh` relied +on the POSIX subset of Awk for working correctly. + +== _Update_ + +After publishing, I could remove `{ nextfile }` and even make the script +simpler: + +[source,sh] +---- +#!/usr/sh -eux + +git ls-files -z | \ + xargs -0 awk 'FNR==1 && /^#!\// { print FILENAME }' | \ + xargs shellcheck +---- + +Now both the shell and Awk usage are POSIX compatible. diff --git a/src/content/en/til/2020/12/29/svg.adoc b/src/content/en/til/2020/12/29/svg.adoc new file mode 100644 index 0000000..e0e75a5 --- /dev/null +++ b/src/content/en/til/2020/12/29/svg.adoc @@ -0,0 +1,125 @@ += SVG favicon +:updatedat: 2021-01-12 + +:favicon: link:../../../../../img/favicon.svg + +I've wanted to change this website's favicon from a plain `.ico` file to a +proper SVG. The problem I was trying to solve was to reuse the same image on +other places, such as avatars. + +Generating a PNG from the existing 16x16 icon was possible but bad: the final +image was blurry. Converting the `.ico` to an SVG was possible, but +sub-optimal: tools try to guess some vector paths, and the final SVG didn't +match the original. + +Instead I used a tool to draw the "vector pixels" as black squares, and after +getting the final result I manually cleaned-up the generated XML: + +[source,xml] +---- +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 16 16" width="16" height="16"> + <path d="M 0 8 L 1 8 L 1 9 L 0 9 L 0 8 Z" /> + <path d="M 0 13 L 1 13 L 1 14 L 0 14 L 0 13 Z" /> + <path d="M 1 8 L 2 8 L 2 9 L 1 9 L 1 8 Z" /> + <path d="M 1 13 L 2 13 L 2 14 L 1 14 L 1 13 Z" /> + <path d="M 2 8 L 3 8 L 3 9 L 2 9 L 2 8 Z" /> + <path d="M 2 13 L 3 13 L 3 14 L 2 14 L 2 13 Z" /> + <path d="M 3 8 L 4 8 L 4 9 L 3 9 L 3 8 Z" /> + <path d="M 3 13 L 4 13 L 4 14 L 3 14 L 3 13 Z" /> + <path d="M 4 7 L 5 7 L 5 8 L 4 8 L 4 7 Z" /> + <path d="M 4 8 L 5 8 L 5 9 L 4 9 L 4 8 Z" /> + <path d="M 4 13 L 5 13 L 5 14 L 4 14 L 4 13 Z" /> + <path d="M 5 6 L 6 6 L 6 7 L 5 7 L 5 6 Z" /> + <path d="M 5 7 L 6 7 L 6 8 L 5 8 L 5 7 Z" /> + <path d="M 5 13 L 6 13 L 6 14 L 5 14 L 5 13 Z" /> + <path d="M 6 5 L 7 5 L 7 6 L 6 6 L 6 5 Z" /> + <path d="M 6 6 L 7 6 L 7 7 L 6 7 L 6 6 Z" /> + <path d="M 6 14 L 7 14 L 7 15 L 6 15 L 6 14 Z" /> + <path d="M 7 1 L 8 1 L 8 2 L 7 2 L 7 1 Z" /> + <path d="M 7 14 L 8 14 L 8 15 L 7 15 L 7 14 Z" /> + <path d="M 7 15 L 8 15 L 8 16 L 7 16 L 7 15 Z" /> + <path d="M 7 2 L 8 2 L 8 3 L 7 3 L 7 2 Z" /> + <path d="M 7 3 L 8 3 L 8 4 L 7 4 L 7 3 Z" /> + <path d="M 7 4 L 8 4 L 8 5 L 7 5 L 7 4 Z" /> + <path d="M 7 5 L 8 5 L 8 6 L 7 6 L 7 5 Z" /> + <path d="M 8 1 L 9 1 L 9 2 L 8 2 L 8 1 Z" /> + <path d="M 8 15 L 9 15 L 9 16 L 8 16 L 8 15 Z" /> + <path d="M 9 1 L 10 1 L 10 2 L 9 2 L 9 1 Z" /> + <path d="M 9 2 L 10 2 L 10 3 L 9 3 L 9 2 Z" /> + <path d="M 9 6 L 10 6 L 10 7 L 9 7 L 9 6 Z" /> + <path d="M 9 15 L 10 15 L 10 16 L 9 16 L 9 15 Z" /> + <path d="M 10 2 L 11 2 L 11 3 L 10 3 L 10 2 Z" /> + <path d="M 10 3 L 11 3 L 11 4 L 10 4 L 10 3 Z" /> + <path d="M 10 4 L 11 4 L 11 5 L 10 5 L 10 4 Z" /> + <path d="M 10 5 L 11 5 L 11 6 L 10 6 L 10 5 Z" /> + <path d="M 10 6 L 11 6 L 11 7 L 10 7 L 10 6 Z" /> + <path d="M 11 6 L 12 6 L 12 7 L 11 7 L 11 6 Z" /> + <path d="M 11 8 L 12 8 L 12 9 L 11 9 L 11 8 Z" /> + <path d="M 10 15 L 11 15 L 11 16 L 10 16 L 10 15 Z" /> + <path d="M 11 10 L 12 10 L 12 11 L 11 11 L 11 10 Z" /> + <path d="M 11 12 L 12 12 L 12 13 L 11 13 L 11 12 Z" /> + <path d="M 11 14 L 12 14 L 12 15 L 11 15 L 11 14 Z" /> + <path d="M 11 15 L 12 15 L 12 16 L 11 16 L 11 15 Z" /> + <path d="M 12 6 L 13 6 L 13 7 L 12 7 L 12 6 Z" /> + <path d="M 12 8 L 13 8 L 13 9 L 12 9 L 12 8 Z" /> + <path d="M 12 10 L 13 10 L 13 11 L 12 11 L 12 10 Z" /> + <path d="M 12 12 L 13 12 L 13 13 L 12 13 L 12 12 Z" /> + <path d="M 12 14 L 13 14 L 13 15 L 12 15 L 12 14 Z" /> + <path d="M 13 6 L 14 6 L 14 7 L 13 7 L 13 6 Z" /> + <path d="M 13 8 L 14 8 L 14 9 L 13 9 L 13 8 Z" /> + <path d="M 13 10 L 14 10 L 14 11 L 13 11 L 13 10 Z" /> + <path d="M 13 12 L 14 12 L 14 13 L 13 13 L 13 12 Z" /> + <path d="M 13 13 L 14 13 L 14 14 L 13 14 L 13 13 Z" /> + <path d="M 13 14 L 14 14 L 14 15 L 13 15 L 13 14 Z" /> + <path d="M 14 7 L 15 7 L 15 8 L 14 8 L 14 7 Z" /> + <path d="M 14 8 L 15 8 L 15 9 L 14 9 L 14 8 Z" /> + <path d="M 14 9 L 15 9 L 15 10 L 14 10 L 14 9 Z" /> + <path d="M 14 10 L 15 10 L 15 11 L 14 11 L 14 10 Z" /> + <path d="M 14 11 L 15 11 L 15 12 L 14 12 L 14 11 Z" /> + <path d="M 14 12 L 15 12 L 15 13 L 14 13 L 14 12 Z" /> +</svg> +---- + +The good thing about this new favicon (at {favicon}[`/static/lord-favicon.svg`]) +is that a) it is simple enough that I feel comfortable editing it manually and +b) it is an SVG, which means I can generate any desired size. + +With the new favicon file, I now had to add to the templates' `<head>` a +`<link>` to this icon: + +[source,html] +---- +<head> + <meta charset="UTF-8" /> + <link rel="icon" type="image/svg+xml" href="/static/favicon.svg"> + ... +---- + +Still missing is a bitmap image for places that can't handle vector images. I +used Jekyll generator to create an PNG from the existing SVG: + +[source,ruby] +---- +module Jekyll + class FaviconGenerator < Generator + safe true + priority :high + + SIZE = 420 + + def generate(site) + svg = 'static/favicon.svg' + png = 'static/favicon.png' + unless File.exist? png then + puts "Missing '#{png}', generating..." + puts `inkscape -o #{png} -w #{SIZE} -h #{SIZE} #{svg}` + end + end + end +end +---- + +I had to increase the priority of the generator so that it would run before +other places that would use a `{% link /static/lord-favicon.png %}`, otherwise +the file would be considered missing. diff --git a/src/content/en/til/2021/01/12/curl-awk-emails.adoc b/src/content/en/til/2021/01/12/curl-awk-emails.adoc new file mode 100644 index 0000000..d432da2 --- /dev/null +++ b/src/content/en/til/2021/01/12/curl-awk-emails.adoc @@ -0,0 +1,148 @@ += Awk snippet: send email to multiple recipients with cURL + +:neomutt: https://neomutt.org/ +:found-out-article: https://blog.edmdesigner.com/send-email-from-linux-command-line/ +:curl: https://curl.se/ + +As I experiment with {neomutt}[Neomutt], I wanted to keep being able to enqueue +emails for sending later like my previous setup, so that I didn't rely on having +an internet connection. + +My requirements for the `sendmail` command were: + +. store the email in a file, and send it later; +. send from different addresses, using different SMTP servers. + +I couldn't find an MTA that could accomplish that, but I was able to quickly +write a solution. + +The first part was the easiest: store the email in a file: + +[source,sh] +---- +# ~/.config/mutt/muttrc: +set sendmail=~/bin/enqueue-email.sh + +# ~/bin/enqueue-email.sh: +#!/bin/sh -eu + +cat - > "$HOME/mbsync/my-queued-emails/$(date -Is)" +---- + +Now that I had the email file store locally, I needed a program to send the +email from the file, so that I could create a cronjob like: + +[source,sh] +---- +for f in ~/mbsync/my-queued-emails/*; do + ~/bin/dispatch-email.sh "$f" && rm "$f" +done +---- + +The `dispatch-email.sh` would have to look at the `From:` header and decide +which SMTP server to use. As I {found-out-article}[found out] that {curl}[curl] +supports SMTP and is able to send emails, this is what I ended up with: + +[source,sh] +---- +#!/bin/sh -eu + +F="$1" + +rcpt="$(awk ' + match($0, /^(To|Cc|Bcc): (.*)$/, m) { + split(m[2], tos, ",") + for (i in tos) { + print "--mail-rcpt " tos[i] + } + } +' "$F")" + +if grep -qE '^From: .*<addr@server1\.org>$' "$F"; then + curl \ + -s \ + --url smtp://smtp.server1.org:587 \ + --ssl-reqd \ + --mail-from addr@server1.org \ + $rcpt \ + --user 'addr@server1.org:my-long-and-secure-passphrase' \ + --upload-file "$F" +elif grep -qE '^From: .*<addr@server2\.org>$' "$F"; then + curl \ + -s \ + --url smtp://smtp.server2.org:587 \ + --ssl-reqd \ + --mail-from addr@server2.org \ + $rcpt \ + --user 'addr@server2.org:my-long-and-secure-passphrase' \ + --upload-file "$F" +else + echo 'Bad "From: " address' + exit 1 +fi +---- + +Most of curl flags used are self-explanatory, except for `$rcpt`. + +curl connects to the SMTP server, but doesn't set the recipient address by +looking at the message. My solution was to generate the curl flags, store them +in `$rcpt` and use it unquoted to leverage shell word splitting. + +To me, the most interesting part was building the `$rcpt` flags. My first +instinct was to try grep, but it couldn't print only matches in a regex. As I +started to turn towards sed, I envisioned needing something else to loop over +the sed output, and I then moved to Awk. + +In the short Awk snippet, 3 things were new to me: the `match(...)`, +`split(...)` and `for () {}`. The only other function I have ever used was +`gsub(...)`, but these new ones felt similar enough that I could almost guess +their behaviour and arguments. `match(...)` stores the matches of a regex on +the given array positionally, and `split(...)` stores the chunks in the given +array. + +I even did it incrementally: + +[source,sh] +---- +$ H='To: to@example.com, to2@example.com\nCc: cc@example.com, cc2@example.com\nBcc: bcc@example.com,bcc2@example.com\n' +$ printf "$H" | awk '/^To: .*$/ { print $0 }' +To: to@example.com, to2@example.com +$ printf "$H" | awk 'match($0, /^To: (.*)$/, m) { print m }' +awk: ligne de commande:1: (FILENAME=- FNR=1) fatal : tentative d'utilisation du tableau « m » dans un contexte scalaire +$ printf "$H" | awk 'match($0, /^To: (.*)$/, m) { print m[0] }' +To: to@example.com, to2@example.com +$ printf "$H" | awk 'match($0, /^To: (.*)$/, m) { print m[1] }' +to@example.com, to2@example.com +$ printf "$H" | awk 'match($0, /^To: (.*)$/, m) { split(m[1], tos, " "); print tos }' +awk: ligne de commande:1: (FILENAME=- FNR=1) fatal : tentative d'utilisation du tableau « tos » dans un contexte scalaire +$ printf "$H" | awk 'match($0, /^To: (.*)$/, m) { split(m[1], tos, " "); print tos[0] }' + +$ printf "$H" | awk 'match($0, /^To: (.*)$/, m) { split(m[1], tos, " "); print tos[1] }' +to@example.com, +$ printf "$H" | awk 'match($0, /^To: (.*)$/, m) { split(m[1], tos, " "); print tos[2] }' +to2@example.com +$ printf "$H" | awk 'match($0, /^To: (.*)$/, m) { split(m[1], tos, " "); print tos[3] }' +---- + +(This isn't the verbatim interactive session, but a cleaned version to make it +more readable.) + +At this point, I realized I needed a for loop over the `tos` array, and I moved +the Awk snippet into the `~/bin/dispatch-email.sh`. I liked the final thing: + +[source,awk] +---- +match($0, /^(To|Cc|Bcc): (.*)$/, m) { + split(m[2], tos, ",") + for (i in tos) { + print "--mail-rcpt " tos[i] + } +} +---- + +As I learn more about Awk, I feel that it is too undervalued, as many people +turn to Perl or other programming languages when Awk suffices. The advantage is +pretty clear: writing programs that run on any POSIX system, without extra +dependencies required. + +Coding to the standards is underrated. diff --git a/src/content/en/til/2021/01/17/posix-shebang.adoc b/src/content/en/til/2021/01/17/posix-shebang.adoc new file mode 100644 index 0000000..5cf0695 --- /dev/null +++ b/src/content/en/til/2021/01/17/posix-shebang.adoc @@ -0,0 +1,58 @@ += POSIX sh and shebangs + +:awk-1: link:../../../2020/12/15/shellcheck-repo.html +:awk-2: link:../12/curl-awk-emails.html + +As I {awk-1}[keep moving] {awk-2}[towards POSIX], I'm on the process of +migrating all my Bash scripts to POSIX sh. + +As I dropped `[[`, arrays and other Bashisms, I was left staring at the first +line of every script, wondering what to do: what is the POSIX sh equivalent of +`#!/usr/bin/env bash`? I already knew that POSIX says nothing about shebangs, +and that the portable way to call a POSIX sh script is `sh script.sh`, but +I didn't know what to do with that first line. + +What I had previously was: + +[source,sh] +---- +#!/usr/bin/env bash +set -Eeuo pipefail +cd "$(dirname "${BASH_SOURCE[0]}")" +---- + +Obviously, the `$BASH_SOURCE` would be gone, and I would have to adapt some of +my scripts to not rely on the script location. The `-E` and `-o pipefail` +options were also gone, and would be replaced by nothing. + +I converted all of them to: + +[source,sh] +---- +#!/bin/sh -eu +---- + +I moved the `-eu` options to the shebang line itself, striving for conciseness. +But as I changed callers from `./script.sh` to `sh script.sh`, things started to +fail. Some tests that should fail reported errors, but didn't return 1. + +My first reaction was to revert back to `./script.sh`, but the POSIX bug I +caught is a strong strain, and when I went back to it, I figured that the +callers were missing some flags. Specifically, `sh -eu script.sh`. + +Then it clicked: when running with `sh script.sh`, the shebang line with the sh +options is ignored, as it is a comment! + +Which means that the shebang most friendly with POSIX is: + +[source,sh] +---- +#!/bin/sh +set -eu +---- + +. when running via `./script.sh`, if the system has an executable at `/bin/sh`, + it will be used to run the script; +. when running via `sh script.sh`, the sh options aren't ignored as previously. + +TIL. diff --git a/src/content/en/til/2021/04/24/cl-generic-precedence.adoc b/src/content/en/til/2021/04/24/cl-generic-precedence.adoc new file mode 100644 index 0000000..541afb0 --- /dev/null +++ b/src/content/en/til/2021/04/24/cl-generic-precedence.adoc @@ -0,0 +1,149 @@ += Common Lisp argument precedence order parameterization of a generic function + +When CLOS dispatches a method, it picks the most specific method definition to +the argument list: + +[source,lisp] +---- + +* (defgeneric a-fn (x)) +#<STANDARD-GENERIC-FUNCTION A-FN (0) {5815ACB9}> + +* (defmethod a-fn (x) :default-method) +#<STANDARD-METHOD A-FN (T) {581DB535}> + +* (defmethod a-fn ((x number)) :a-number) +#<STANDARD-METHOD A-FN (NUMBER) {58241645}> + +* (defmethod a-fn ((x (eql 1))) :number-1) +#<STANDARD-METHOD A-FN ((EQL 1)) {582A7D75}> + +* (a-fn nil) +:DEFAULT-METHOD + +* (a-fn "1") +:DEFAULT-METHOD + +* (a-fn 0) +:A-NUMBER + +* (a-fn 1) +:NUMBER-1 +---- + +CLOS uses a similar logic when choosing the method from parent classes, when +multiple ones are available: + +[source,lisp] +---- +* (defclass class-a () ()) + +#<STANDARD-CLASS CLASS-A {583E0B25}> +* (defclass class-b () ()) + +#<STANDARD-CLASS CLASS-B {583E7F6D}> +* (defgeneric another-fn (obj)) + +#<STANDARD-GENERIC-FUNCTION ANOTHER-FN (0) {583DA749}> +* (defmethod another-fn ((obj class-a)) :class-a) +; Compiling LAMBDA (.PV-CELL. .NEXT-METHOD-CALL. OBJ): +; Compiling Top-Level Form: + +#<STANDARD-METHOD ANOTHER-FN (CLASS-A) {584523C5}> +* (defmethod another-fn ((obj class-b)) :class-b) +; Compiling LAMBDA (.PV-CELL. .NEXT-METHOD-CALL. OBJ): +; Compiling Top-Level Form: + +#<STANDARD-METHOD ANOTHER-FN (CLASS-B) {584B8895}> +---- + +Given the above definitions, when inheriting from `class-a` and `class-b`, the +order of inheritance matters: + +[source,lisp] +---- +* (defclass class-a-coming-first (class-a class-b) ()) +#<STANDARD-CLASS CLASS-A-COMING-FIRST {584BE6AD}> + +* (defclass class-b-coming-first (class-b class-a) ()) +#<STANDARD-CLASS CLASS-B-COMING-FIRST {584C744D}> + +* (another-fn (make-instance 'class-a-coming-first)) +:CLASS-A + +* (another-fn (make-instance 'class-b-coming-first)) +:CLASS-B +---- + +Combining the order of inheritance with generic functions with multiple +arguments, CLOS has to make a choice of how to pick a method given two competing +definitions, and its default strategy is prioritizing from left to right: + +[source,lisp] +---- +* (defgeneric yet-another-fn (obj1 obj2)) +#<STANDARD-GENERIC-FUNCTION YET-ANOTHER-FN (0) {584D9EC9}> + +* (defmethod yet-another-fn ((obj1 class-a) obj2) :first-arg-specialized) +#<STANDARD-METHOD YET-ANOTHER-FN (CLASS-A T) {5854269D}> + +* (defmethod yet-another-fn (obj1 (obj2 class-b)) :second-arg-specialized) +#<STANDARD-METHOD YET-ANOTHER-FN (T CLASS-B) {585AAAAD}> + +* (yet-another-fn (make-instance 'class-a) (make-instance 'class-b)) +:FIRST-ARG-SPECIALIZED +---- + +CLOS has to make a choice between the first and the second definition of +`yet-another-fn`, but its choice is just a heuristic. What if we want the +choice to be based on the second argument, instead of the first? + +For that, we use the `:argument-precedence-order` option when declaring a +generic function: + +[source,lisp] +---- +* (defgeneric yet-another-fn (obj1 obj2) (:argument-precedence-order obj2 obj1)) +#<STANDARD-GENERIC-FUNCTION YET-ANOTHER-FN (2) {584D9EC9}> + +* (yet-another-fn (make-instance 'class-a) (make-instance 'class-b)) +:SECOND-ARG-SPECIALIZED +---- + +I liked that the `:argument-precedence-order` option exists. We shouldn't have +to change the arguments from `(obj1 obj2)` to `(obj2 obj1)` just to make CLOS +pick the method that we want. We can configure its default behaviour if +desired, and keep the order of arguments however it best fits the generic +function. + +== Comparison with Clojure + +Clojure has an equivalent, when using `defmulti`. + +Since when declaring a multi-method with `defmulti` we must define the dispatch +function, Clojure uses it to pick the method definition. Since the dispatch +function is required, there is no need for a default behaviour, such as +left-to-right. + +== Conclusion + +Making the argument precedence order configurable for generic functions but not +for class definitions makes a lot of sense. + +When declaring a class, we can choose the precedence order, and that is about +it. But when defining a generic function, the order of arguments is more +important to the function semantics, and the argument precedence being +left-to-right is just the default behaviour. + +One shouldn't change the order of arguments of a generic function for the sake +of tailoring it to the CLOS priority ranking algorithm, but doing it for a class +definition is just fine. + +TIL. + +== References + +:clos-wiki: https://en.wikipedia.org/wiki/Object-Oriented_Programming_in_Common_Lisp + +. {clos-wiki}[Object-Oriented Programming in Common Lisp: A Programmer's Guide + to CLOS], by Sonja E. Keene diff --git a/src/content/en/til/2021/04/24/clojure-autocurry.adoc b/src/content/en/til/2021/04/24/clojure-autocurry.adoc new file mode 100644 index 0000000..a2c2835 --- /dev/null +++ b/src/content/en/til/2021/04/24/clojure-autocurry.adoc @@ -0,0 +1,135 @@ += Clojure auto curry +:sort: 1 +:updatedat: 2021-04-27 + +:defcurry-orig: https://lorettahe.github.io/clojure/2016/09/22/clojure-auto-curry + +Here's a simple macro defined by {defcurry-orig}[Loretta He] to create Clojure +functions that are curried on all arguments, relying on Clojure's multi-arity +support: + +[source,clojure] +---- +(defmacro defcurry + [name args & body] + (let [partials (map (fn [n] + `(~(subvec args 0 n) (partial ~name ~@(take n args)))) + (range 1 (count args)))] + `(defn ~name + (~args ~@body) + ~@partials))) +---- + +A naive `add` definition, alongside its usage and macroexpansion: + +[source,clojure] +---- +user=> (defcurry add + [a b c d e] + (+ 1 2 3 4 5)) +#'user/add + +user=> (add 1) +#object[clojure.core$partial$fn__5857 0x2c708440 "clojure.core$partial$fn__5857@2c708440"] + +user=> (add 1 2 3 4) +#object[clojure.core$partial$fn__5863 0xf4c0e4e "clojure.core$partial$fn__5863@f4c0e4e"] + +user=> ((add 1) 2 3 4 5) +15 + +user=> (((add 1) 2 3) 4 5) +15 + +user=> (use 'clojure.pprint) +nil + +user=> (pprint + (macroexpand + '(defcurry add + [a b c d e] + (+ 1 2 3 4 5)))) +(def + add + (clojure.core/fn + ([a b c d e] (+ 1 2 3 4 5)) + ([a] (clojure.core/partial add a)) + ([a b] (clojure.core/partial add a b)) + ([a b c] (clojure.core/partial add a b c)) + ([a b c d] (clojure.core/partial add a b c d)))) +nil +---- + +This simplistic `defcurry` definition doesn't support optional parameters, +multi-arity, `&` rest arguments, docstrings, etc., but it could certainly evolve +to do so. + +I like how `defcurry` is so short, and abdicates the responsability of doing the +multi-arity logic to Clojure's built-in multi-arity support. Simple and +elegant. + +Same Clojure as before, now with auto-currying via macros. + +== Comparison with Common Lisp + +My attempt at writing an equivalent for Common Lisp gives me: + +[source,lisp] +---- +(defun partial (fn &rest args) + (lambda (&rest args2) + (apply fn (append args args2)))) + +(defun curry-n (n func) + (cond ((< n 0) (error "Too many arguments")) + ((zerop n) (funcall func)) + (t (lambda (&rest rest) + (curry-n (- n (length rest)) + (apply #'partial func rest)))))) + +(defmacro defcurry (name args &body body) + `(defun ,name (&rest rest) + (let ((func (lambda ,args ,@body))) + (curry-n (- ,(length args) (length rest)) + (apply #'partial func rest))))) +---- + +Without built-in multi-arity support, we have to do more work, like tracking the +number of arguments consumed so far. We also have to write `#'partial` +ourselves. That is, without dependending on any library, sticking to ANSI +Common Lisp. + +The usage is pretty similar: + +[source,lisp] +---- +* (defcurry add (a b c d e) + (+ a b c d e)) +ADD + +* (add 1) +#<FUNCTION (LAMBDA (&REST REST) :IN CURRY-N) {100216419B}> + +* (funcall (add 1) 2 3 4) +#<FUNCTION (LAMBDA (&REST REST) :IN CURRY-N) {100216537B}> + +* (funcall (add 1) 2 3 4 5) +15 + +* (funcall (funcall (add 1) 2 3) 4 5) +15 + +* (macroexpand-1 + '(defcurry add (a b c d e) + (+ a b c d e))) +(DEFUN ADD (&REST REST) + (LET ((FUNC (LAMBDA (A B C D E) (+ A B C D E)))) + (CURRY-N (- 5 (LENGTH REST)) (APPLY #'PARTIAL FUNC REST)))) +T +---- + +This also require `funcall`s, since we return a `lambda` that doesn't live in +the function namespace. + +Like the Clojure one, it doesn't support optional parameters, `&rest` rest +arguments, docstrings, etc., but it also could evolve to do so. diff --git a/src/content/en/til/2021/04/24/scm-nif.adoc b/src/content/en/til/2021/04/24/scm-nif.adoc new file mode 100644 index 0000000..2ea8a6f --- /dev/null +++ b/src/content/en/til/2021/04/24/scm-nif.adoc @@ -0,0 +1,61 @@ += Three-way conditional for number signs on Lisp +:categories: lisp scheme common-lisp +:sort: 2 +:updatedat: 2021-08-14 + +:on-lisp: https://www.paulgraham.com/onlisptext.html +:sicp: https://mitpress.mit.edu/sites/default/files/sicp/index.html + +A useful macro from Paul Graham's {on-lisp}[On Lisp] book: + +[source,lisp] +---- +(defmacro nif (expr pos zero neg) + (let ((g (gensym))) + `(let ((,g ,expr)) + (cond ((plusp ,g) ,pos) + ((zerop ,g) ,zero) + (t ,neg))))) +---- + +After I looked at this macro, I started seeing opportunities to using it in many +places, and yet I didn't see anyone else using it. + +The latest example I can think of is section 1.3.3 of {sicp}[Structure and +Interpretation of Computer Programs], which I was reading recently: + +[source,scheme] +---- +(define (search f neg-point pos-point) + (let ((midpoint (average neg-point pos-point))) + (if (close-enough? neg-point post-point) + midpoint + (let ((test-value (f midpoint))) + (cond ((positive? test-value) + (search f neg-point midpoint)) + ((negative? test-value) + (search f midpoint pos-point)) + (else midpoint)))))) +---- + +Not that the book should introduce such macro this early, but I couldn't avoid +feeling bothered by not using the `nif` macro, which could even remove the need +for the intermediate `test-value` variable: + +[source,scheme] +---- +(define (search f neg-point pos-point) + (let ((midpoint (average neg-point pos-point))) + (if (close-enough? neg-point post-point) + midpoint + (nif (f midpoint) + (search f neg-point midpoint) + (midpoint) + (search f midpoint pos-point))))) +---- + +It also avoids `cond`'s extra clunky parentheses for grouping, which is +unnecessary but built-in. + +As a macro, I personally feel it tilts the balance towards expressivenes despite +its extra cognitive load toll. diff --git a/src/content/en/til/2021/07/23/git-tls-gpg.adoc b/src/content/en/til/2021/07/23/git-tls-gpg.adoc new file mode 100644 index 0000000..f198c2b --- /dev/null +++ b/src/content/en/til/2021/07/23/git-tls-gpg.adoc @@ -0,0 +1,45 @@ += GPG verification of Git repositories without TLS + +:empty: +:git-protocol: https://git-scm.com/book/en/v2/Git-on-the-Server-The-Protocols#_the_git_protocol +:remembering: https://euandreh.xyz/remembering/ + +For online Git repositories that use the [Git Protocol] for serving code, you +can can use GPG to handle authentication, if you have the committer's public +key. + +Here's how I'd verify that I've cloned an authentic version of +{remembering}[remembering]footnote:not-available[ + Funnily enough, not available anymore via the Git Protocol, now only with + HTTPS. +]: + +[source,sh] +---- +$ wget -qO- https://euandre.org/public.asc | gpg --import - +gpg: clef 81F90EC3CD356060 : « EuAndreh <eu@euandre.org> » n'est pas modifiée +gpg: Quantité totale traitée : 1 +gpg: non modifiées : 1 +$ pushd `mktemp -d` +$ git clone git://euandreh.xyz/remembering . +$ git verify-commit HEAD +gpg: Signature faite le dim. 27 juin 2021 16:50:21 -03 +gpg: avec la clef RSA 5BDAE9B8B2F6C6BCBB0D6CE581F90EC3CD356060 +gpg: Bonne signature de « EuAndreh <eu@euandre.org> » [ultime] +---- + +On the first line we import the public key (funnily enough, available via +HTTPS), and after cloning the code via the insecure `git://` protocol, we use +`git verify-commit` to check the signature. + +The verification is successful, and we can see that the public key from the +signature matches the fingerprint of the imported one. However +`git verify-commit` doesn't have an option to check which public key you want to +verify the commit against. Which means that if a MITM attack happens, the +attacker could very easily serve a malicious repository with signed commits, and +you'd have to verify the public key by yourself. That would need to happen for +subsequent fetches, too. + +Even though this is possible, it is not very convenient, and certainly very +brittle. Despite the fact that the Git Protocol is much faster, it being harder +to make secure is a big downside. diff --git a/src/content/en/til/2021/08/11/js-bigint-reviver.adoc b/src/content/en/til/2021/08/11/js-bigint-reviver.adoc new file mode 100644 index 0000000..98ee79b --- /dev/null +++ b/src/content/en/til/2021/08/11/js-bigint-reviver.adoc @@ -0,0 +1,89 @@ += Encoding and decoding JavaScript BigInt values with reviver +:updatedat: 2021-08-13 + +:reviver-fn: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/parse#using_the_reviver_parameter +:bigint: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/BigInt +:json-rfc: https://datatracker.ietf.org/doc/html/rfc8259 + +`JSON.parse()` accepts a second parameter: a {reviver-fn}[`reviver()` function]. +It is a function that can be used to transform the `JSON` values as they're +being parsed. + +As it turns out, when combined with JavaScript's {bigint}[`BigInt`] type, you +can parse and encode JavaScript `BigInt` numbers via JSON: + +[source,javascript] +---- +const bigIntReviver = (_, value) => + typeof value === "string" && value.match(/^-?[0-9]+n$/) + ? BigInt(value.slice(0, value.length - 1)) + : value; +---- + +I chose to interpret strings that contains only numbers and an ending `n` +suffix as `BigInt` values, similar to how JavaScript interprets `123` (a number) +differently from `123n` (a `bigint`); + +We do those checks before constructing the `BigInt` to avoid throwing needless +exceptions and catching them on the parsing function, as this could easily +become a bottleneck when parsing large JSON values. + +In order to do the full roundtrip, we now only need the `toJSON()` counterpart: + +[source,javascript] +---- +BigInt.prototype.toJSON = function() { + return this.toString() + "n"; +}; +---- + +With both `bigIntReviver` and `toJSON` defined, we can now successfully parse +and encode JavaScript objects with `BigInt` values transparently: + +[source,javascript] +---- +const s = `[ + null, + true, + false, + -1, + 3.14, + "a string", + { "a-number": "-123" }, + { "a-bigint": "-123n" } +]`; + +const parsed = JSON.parse(s, bigIntReviver); +const s2 = JSON.stringify(parsed); + +console.log(parsed); +console.log(s2); + +console.log(typeof parsed[6]["a-number"]) +console.log(typeof parsed[7]["a-bigint"]) +---- + +The output of the above is: + +[source,javascript] +---- +[ + null, + true, + false, + -1, + 3.14, + 'a string', + { 'a-number': '-123' }, + { 'a-bigint': -123n } +] +[null,true,false,-1,3.14,"a string",{"a-number":"-123"},{"a-bigint":"-123n"}] +string +bigint +---- + +If you're on a web browser, you can probably try copying and pasting the above +code on the console right now, as is. + +Even though {json-rfc}[`JSON`] doesn't include `BigInt` number, encoding and +decoding them as strings is quite trivial on JavaScript. diff --git a/src/content/en/til/2026/09/05/multiline-sh-comments.adoc b/src/content/en/til/2026/09/05/multiline-sh-comments.adoc new file mode 100644 index 0000000..13fef6d --- /dev/null +++ b/src/content/en/til/2026/09/05/multiline-sh-comments.adoc @@ -0,0 +1,66 @@ += Multiline comments in POSIX sh +:categories: sh + +:posixsh: https://pubs.opengroup.org/onlinepubs/9799919799/utilities/V3_chap02.html#tag_19_07_04 + + +I realized one can use heredocs for multiline comments in sh: + +[source,sh] +---- +cat <<'COMMENT' > /dev/null +if wip; then + unfinished and broken syntax > +fii +COMMENT +---- + + +...instead of: + +[source,sh] +---- +# if wip; then +# unfinished and broken syntax > +# fii +---- + + +What is going on: the {posixsh}[language defines] `<<` as a redirect of the +"here-document" type. After `<<` you can put anything, and sh will use that as +a token to look for on the following lines: + +[source,sh] +---- +cat <<'bleh' +... +bleh +---- + +Make sure to include the `'quotes'` around the word, otherwise anything with a +`$` would get replaced just like a `"double-quoted"` string. + +"bleh" works fine, like any word would. + +But that example left as-is would print the string to stdout. To avoid that, we +shove the contents to `/dev/null`. So it isn't technically a comment, its more +like a constant string that gets emitted and discarded. + +Sure its hacky, but at this point what in sh isn't? It even supports nesting! + +[source,sh] +---- +cat <<'COMMENT1' > /dev/null + +cat <<'COMMENT' > /dev/null + +# ... code ... + +COMMENT + +COMMENT1 +---- + + +I've never committed any code with this, just used while debugging sh code. +Make sure to handle with care diff --git a/src/content/en/til/categories.adoc b/src/content/en/til/categories.adoc new file mode 100644 index 0000000..feb64ff --- /dev/null +++ b/src/content/en/til/categories.adoc @@ -0,0 +1,2 @@ += Articles by category +:type: categories diff --git a/src/content/en/til/index.adoc b/src/content/en/til/index.adoc new file mode 100644 index 0000000..2d100d0 --- /dev/null +++ b/src/content/en/til/index.adoc @@ -0,0 +1,8 @@ += Today I Learned +:type: collection + +:anna-e-so: https://til.flourishing.stream/ + +**T**oday **I** **L**earned: small entries of useful knowledge. + +Shameless rip-off of {anna-e-so}[Anna e só]. diff --git a/src/content/favicon.ico b/src/content/favicon.ico Binary files differnew file mode 100644 index 0000000..b35a439 --- /dev/null +++ b/src/content/favicon.ico diff --git a/src/content/favicon.png b/src/content/favicon.png Binary files differnew file mode 100644 index 0000000..bab901f --- /dev/null +++ b/src/content/favicon.png diff --git a/src/content/git b/src/content/git new file mode 120000 index 0000000..dc1dc0c --- /dev/null +++ b/src/content/git @@ -0,0 +1 @@ +/dev/null
\ No newline at end of file diff --git a/src/content/img/atom.svg b/src/content/img/atom.svg new file mode 100644 index 0000000..23ab291 --- /dev/null +++ b/src/content/img/atom.svg @@ -0,0 +1,6 @@ +<?xml version="1.0" encoding="utf-8"?> +<svg width="22" height="22" viewBox="0 0 1792 1792" xmlns="http://www.w3.org/2000/svg"> + <path + fill="#EA990E" + d="M576 1344q0 80-56 136t-136 56-136-56-56-136 56-136 136-56 136 56 56 136zm512 123q2 28-17 48-18 21-47 21h-135q-25 0-43-16.5t-20-41.5q-22-229-184.5-391.5t-391.5-184.5q-25-2-41.5-20t-16.5-43v-135q0-29 21-47 17-17 43-17h5q160 13 306 80.5t259 181.5q114 113 181.5 259t80.5 306zm512 2q2 27-18 47-18 20-46 20h-143q-26 0-44.5-17.5t-19.5-42.5q-12-215-101-408.5t-231.5-336-336-231.5-408.5-102q-25-1-42.5-19.5t-17.5-43.5v-143q0-28 20-46 18-18 44-18h3q262 13 501.5 120t425.5 294q187 186 294 425.5t120 501.5z" /> +</svg> diff --git a/src/content/img/envelope.svg b/src/content/img/envelope.svg new file mode 100644 index 0000000..aba3a80 --- /dev/null +++ b/src/content/img/envelope.svg @@ -0,0 +1,11 @@ +<?xml version="1.0" encoding="utf-8"?> +<svg width="22" height="22" viewBox="0 0 1792 1792" xmlns="http://www.w3.org/2000/svg"> + <style> + svg { fill: black } + @media (prefers-color-scheme: dark) { + svg { fill: white } + } + </style> + <path + d="M1664 1504v-768q-32 36-69 66-268 206-426 338-51 43-83 67t-86.5 48.5-102.5 24.5h-2q-48 0-102.5-24.5t-86.5-48.5-83-67q-158-132-426-338-37-30-69-66v768q0 13 9.5 22.5t22.5 9.5h1472q13 0 22.5-9.5t9.5-22.5zm0-1051v-24.5l-.5-13-3-12.5-5.5-9-9-7.5-14-2.5h-1472q-13 0-22.5 9.5t-9.5 22.5q0 168 147 284 193 152 401 317 6 5 35 29.5t46 37.5 44.5 31.5 50.5 27.5 43 9h2q20 0 43-9t50.5-27.5 44.5-31.5 46-37.5 35-29.5q208-165 401-317 54-43 100.5-115.5t46.5-131.5zm128-37v1088q0 66-47 113t-113 47h-1472q-66 0-113-47t-47-113v-1088q0-66 47-113t113-47h1472q66 0 113 47t47 113z" /> +</svg> diff --git a/src/content/img/favicon.svg b/src/content/img/favicon.svg new file mode 100644 index 0000000..1916d84 --- /dev/null +++ b/src/content/img/favicon.svg @@ -0,0 +1,92 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 16 16" width="16" height="16"> + <style> + svg { + fill: #1f6f5b; + } + @media (prefers-color-scheme: dark) { + fill: 5cc2a2; + } + </style> + <path d="M 0 0 L 1 0 L 1 1 L 0 1 L 0 0 Z" /> + <path d="M 0 15 L 1 15 L 1 16 L 0 16 L 0 15 Z" /> + <path d="M 15 0 L 16 0 L 16 1 L 15 1 L 15 0 Z" /> + <path d="M 1 0 L 2 0 L 2 1 L 1 1 L 1 0 Z" /> + <path d="M 1 15 L 2 15 L 2 16 L 1 16 L 1 15 Z" /> + <path d="M 0 1 L 1 1 L 1 2 L 0 2 L 0 1 Z" /> + <path d="M 15 1 L 16 1 L 16 2 L 15 2 L 15 1 Z" /> + <path d="M 2 0 L 3 0 L 3 1 L 2 1 L 2 0 Z" /> + <path d="M 2 15 L 3 15 L 3 16 L 2 16 L 2 15 Z" /> + <path d="M 0 2 L 1 2 L 1 3 L 0 3 L 0 2 Z" /> + <path d="M 15 2 L 16 2 L 16 3 L 15 3 L 15 2 Z" /> + <path d="M 3 0 L 4 0 L 4 1 L 3 1 L 3 0 Z" /> + <path d="M 3 15 L 4 15 L 4 16 L 3 16 L 3 15 Z" /> + <path d="M 0 3 L 1 3 L 1 4 L 0 4 L 0 3 Z" /> + <path d="M 15 3 L 16 3 L 16 4 L 15 4 L 15 3 Z" /> + <path d="M 4 0 L 5 0 L 5 1 L 4 1 L 4 0 Z" /> + <path d="M 4 15 L 5 15 L 5 16 L 4 16 L 4 15 Z" /> + <path d="M 0 4 L 1 4 L 1 5 L 0 5 L 0 4 Z" /> + <path d="M 15 4 L 16 4 L 16 5 L 15 5 L 15 4 Z" /> + <path d="M 5 0 L 6 0 L 6 1 L 5 1 L 5 0 Z" /> + <path d="M 5 15 L 6 15 L 6 16 L 5 16 L 5 15 Z" /> + <path d="M 0 5 L 1 5 L 1 6 L 0 6 L 0 5 Z" /> + <path d="M 15 5 L 16 5 L 16 6 L 15 6 L 15 5 Z" /> + <path d="M 6 0 L 7 0 L 7 1 L 6 1 L 6 0 Z" /> + <path d="M 6 15 L 7 15 L 7 16 L 6 16 L 6 15 Z" /> + <path d="M 0 6 L 1 6 L 1 7 L 0 7 L 0 6 Z" /> + <path d="M 15 6 L 16 6 L 16 7 L 15 7 L 15 6 Z" /> + <path d="M 7 0 L 8 0 L 8 1 L 7 1 L 7 0 Z" /> + <path d="M 7 15 L 8 15 L 8 16 L 7 16 L 7 15 Z" /> + <path d="M 0 7 L 1 7 L 1 8 L 0 8 L 0 7 Z" /> + <path d="M 15 7 L 16 7 L 16 8 L 15 8 L 15 7 Z" /> + <path d="M 8 0 L 9 0 L 9 1 L 8 1 L 8 0 Z" /> + <path d="M 8 15 L 9 15 L 9 16 L 8 16 L 8 15 Z" /> + <path d="M 0 8 L 1 8 L 1 9 L 0 9 L 0 8 Z" /> + <path d="M 15 8 L 16 8 L 16 9 L 15 9 L 15 8 Z" /> + <path d="M 9 0 L 10 0 L 10 1 L 9 1 L 9 0 Z" /> + <path d="M 9 15 L 10 15 L 10 16 L 9 16 L 9 15 Z" /> + <path d="M 0 9 L 1 9 L 1 10 L 0 10 L 0 9 Z" /> + <path d="M 15 9 L 16 9 L 16 10 L 15 10 L 15 9 Z" /> + <path d="M 10 0 L 11 0 L 11 1 L 10 1 L 10 0 Z" /> + <path d="M 10 15 L 11 15 L 11 16 L 10 16 L 10 15 Z" /> + <path d="M 0 10 L 1 10 L 1 11 L 0 11 L 0 10 Z" /> + <path d="M 15 10 L 16 10 L 16 11 L 15 11 L 15 10 Z" /> + <path d="M 11 0 L 12 0 L 12 1 L 11 1 L 11 0 Z" /> + <path d="M 11 15 L 12 15 L 12 16 L 11 16 L 11 15 Z" /> + <path d="M 0 11 L 1 11 L 1 12 L 0 12 L 0 11 Z" /> + <path d="M 15 11 L 16 11 L 16 12 L 15 12 L 15 11 Z" /> + <path d="M 12 0 L 13 0 L 13 1 L 12 1 L 12 0 Z" /> + <path d="M 12 15 L 13 15 L 13 16 L 12 16 L 12 15 Z" /> + <path d="M 0 12 L 1 12 L 1 13 L 0 13 L 0 12 Z" /> + <path d="M 15 12 L 16 12 L 16 13 L 15 13 L 15 12 Z" /> + <path d="M 13 0 L 14 0 L 14 1 L 13 1 L 13 0 Z" /> + <path d="M 13 15 L 14 15 L 14 16 L 13 16 L 13 15 Z" /> + <path d="M 0 13 L 1 13 L 1 14 L 0 14 L 0 13 Z" /> + <path d="M 15 13 L 16 13 L 16 14 L 15 14 L 15 13 Z" /> + <path d="M 14 0 L 15 0 L 15 1 L 14 1 L 14 0 Z" /> + <path d="M 14 15 L 15 15 L 15 16 L 14 16 L 14 15 Z" /> + <path d="M 0 14 L 1 14 L 1 15 L 0 15 L 0 14 Z" /> + <path d="M 15 14 L 16 14 L 16 15 L 15 15 L 15 14 Z" /> + <path d="M 15 15 L 16 15 L 16 16 L 15 16 L 15 15 Z" /> + <path d="M 4 7 L 5 7 L 5 8 L 4 8 L 4 7 Z" /> + <path d="M 5 7 L 6 7 L 6 8 L 5 8 L 5 7 Z" /> + <path d="M 3 8 L 4 8 L 4 9 L 3 9 L 3 8 Z" /> + <path d="M 6 8 L 7 8 L 7 9 L 6 9 L 6 8 Z" /> + <path d="M 3 9 L 4 9 L 4 10 L 3 10 L 3 9 Z" /> + <path d="M 4 9 L 5 9 L 5 10 L 4 10 L 4 9 Z" /> + <path d="M 5 9 L 6 9 L 6 10 L 5 10 L 5 9 Z" /> + <path d="M 3 10 L 4 10 L 4 11 L 3 11 L 3 10 Z" /> + <path d="M 4 11 L 5 11 L 5 12 L 4 12 L 4 11 Z" /> + <path d="M 5 11 L 6 11 L 6 12 L 5 12 L 5 11 Z" /> + <path d="M 8 7 L 9 7 L 9 8 L 8 8 L 8 7 Z" /> + <path d="M 11 7 L 12 7 L 12 8 L 11 8 L 11 7 Z" /> + <path d="M 8 8 L 9 8 L 9 9 L 8 9 L 8 8 Z" /> + <path d="M 11 8 L 12 8 L 12 9 L 11 9 L 11 8 Z" /> + <path d="M 8 9 L 9 9 L 9 10 L 8 10 L 8 9 Z" /> + <path d="M 11 9 L 12 9 L 12 10 L 11 10 L 11 9 Z" /> + <path d="M 8 10 L 9 10 L 9 11 L 8 11 L 8 10 Z" /> + <path d="M 11 10 L 12 10 L 12 11 L 11 11 L 11 10 Z" /> + <path d="M 9 11 L 10 11 L 10 12 L 9 12 L 9 11 Z" /> + <path d="M 10 11 L 11 11 L 11 12 L 10 12 L 10 11 Z" /> + <path d="M 11 11 L 12 11 L 12 12 L 11 12 L 11 11 Z" /> +</svg> diff --git a/src/content/img/link.svg b/src/content/img/link.svg new file mode 100644 index 0000000..7a0aa14 --- /dev/null +++ b/src/content/img/link.svg @@ -0,0 +1,16 @@ +<?xml version="1.0" encoding="utf-8"?> +<svg width="22" height="22" viewBox="0 0 16 16" xmlns="http://www.w3.org/2000/svg"> + <style> + svg { + fill: black; + } + @media (prefers-color-scheme: dark) { + svg { + fill: white; + } + } + </style> + <path + fill-rule="evenodd" + d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z" /> +</svg> diff --git a/src/content/img/lock.svg b/src/content/img/lock.svg new file mode 100644 index 0000000..faad538 --- /dev/null +++ b/src/content/img/lock.svg @@ -0,0 +1,15 @@ +<?xml version="1.0" encoding="utf-8"?> +<svg width="22" height="22" viewBox="0 0 1792 1792" xmlns="http://www.w3.org/2000/svg"> + <style> + svg { + fill: black; + } + @media (prefers-color-scheme: dark) { + svg { + fill: white; + } + } + </style> + <path + d="M640 768h512v-192q0-106-75-181t-181-75-181 75-75 181v192zm832 96v576q0 40-28 68t-68 28h-960q-40 0-68-28t-28-68v-576q0-40 28-68t68-28h32v-192q0-184 132-316t316-132 316 132 132 316v192h32q40 0 68 28t28 68z" /> +</svg> diff --git a/src/content/img/logo.svg b/src/content/img/logo.svg new file mode 100644 index 0000000..662e0d5 --- /dev/null +++ b/src/content/img/logo.svg @@ -0,0 +1,92 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 16 16" width="16" height="16"> + <style> + svg { fill: black } + .a { fill: #1F6F5B } + @media (prefers-color-scheme: dark) { + svg { fill: white } + .a { fill: #5CC2A2 } + } + </style> + <path d="M 0 0 L 1 0 L 1 1 L 0 1 L 0 0 Z" /> + <path d="M 0 15 L 1 15 L 1 16 L 0 16 L 0 15 Z" /> + <path d="M 15 0 L 16 0 L 16 1 L 15 1 L 15 0 Z" /> + <path d="M 1 0 L 2 0 L 2 1 L 1 1 L 1 0 Z" /> + <path d="M 1 15 L 2 15 L 2 16 L 1 16 L 1 15 Z" /> + <path d="M 0 1 L 1 1 L 1 2 L 0 2 L 0 1 Z" /> + <path d="M 15 1 L 16 1 L 16 2 L 15 2 L 15 1 Z" /> + <path d="M 2 0 L 3 0 L 3 1 L 2 1 L 2 0 Z" /> + <path d="M 2 15 L 3 15 L 3 16 L 2 16 L 2 15 Z" /> + <path d="M 0 2 L 1 2 L 1 3 L 0 3 L 0 2 Z" /> + <path d="M 15 2 L 16 2 L 16 3 L 15 3 L 15 2 Z" /> + <path d="M 3 0 L 4 0 L 4 1 L 3 1 L 3 0 Z" /> + <path d="M 3 15 L 4 15 L 4 16 L 3 16 L 3 15 Z" /> + <path d="M 0 3 L 1 3 L 1 4 L 0 4 L 0 3 Z" /> + <path d="M 15 3 L 16 3 L 16 4 L 15 4 L 15 3 Z" /> + <path d="M 4 0 L 5 0 L 5 1 L 4 1 L 4 0 Z" /> + <path d="M 4 15 L 5 15 L 5 16 L 4 16 L 4 15 Z" /> + <path d="M 0 4 L 1 4 L 1 5 L 0 5 L 0 4 Z" /> + <path d="M 15 4 L 16 4 L 16 5 L 15 5 L 15 4 Z" /> + <path d="M 5 0 L 6 0 L 6 1 L 5 1 L 5 0 Z" /> + <path d="M 5 15 L 6 15 L 6 16 L 5 16 L 5 15 Z" /> + <path d="M 0 5 L 1 5 L 1 6 L 0 6 L 0 5 Z" /> + <path d="M 15 5 L 16 5 L 16 6 L 15 6 L 15 5 Z" /> + <path d="M 6 0 L 7 0 L 7 1 L 6 1 L 6 0 Z" /> + <path d="M 6 15 L 7 15 L 7 16 L 6 16 L 6 15 Z" /> + <path d="M 0 6 L 1 6 L 1 7 L 0 7 L 0 6 Z" /> + <path d="M 15 6 L 16 6 L 16 7 L 15 7 L 15 6 Z" /> + <path d="M 7 0 L 8 0 L 8 1 L 7 1 L 7 0 Z" /> + <path d="M 7 15 L 8 15 L 8 16 L 7 16 L 7 15 Z" /> + <path d="M 0 7 L 1 7 L 1 8 L 0 8 L 0 7 Z" /> + <path d="M 15 7 L 16 7 L 16 8 L 15 8 L 15 7 Z" /> + <path d="M 8 0 L 9 0 L 9 1 L 8 1 L 8 0 Z" /> + <path d="M 8 15 L 9 15 L 9 16 L 8 16 L 8 15 Z" /> + <path d="M 0 8 L 1 8 L 1 9 L 0 9 L 0 8 Z" /> + <path d="M 15 8 L 16 8 L 16 9 L 15 9 L 15 8 Z" /> + <path d="M 9 0 L 10 0 L 10 1 L 9 1 L 9 0 Z" /> + <path d="M 9 15 L 10 15 L 10 16 L 9 16 L 9 15 Z" /> + <path d="M 0 9 L 1 9 L 1 10 L 0 10 L 0 9 Z" /> + <path d="M 15 9 L 16 9 L 16 10 L 15 10 L 15 9 Z" /> + <path d="M 10 0 L 11 0 L 11 1 L 10 1 L 10 0 Z" /> + <path d="M 10 15 L 11 15 L 11 16 L 10 16 L 10 15 Z" /> + <path d="M 0 10 L 1 10 L 1 11 L 0 11 L 0 10 Z" /> + <path d="M 15 10 L 16 10 L 16 11 L 15 11 L 15 10 Z" /> + <path d="M 11 0 L 12 0 L 12 1 L 11 1 L 11 0 Z" /> + <path d="M 11 15 L 12 15 L 12 16 L 11 16 L 11 15 Z" /> + <path d="M 0 11 L 1 11 L 1 12 L 0 12 L 0 11 Z" /> + <path d="M 15 11 L 16 11 L 16 12 L 15 12 L 15 11 Z" /> + <path d="M 12 0 L 13 0 L 13 1 L 12 1 L 12 0 Z" /> + <path d="M 12 15 L 13 15 L 13 16 L 12 16 L 12 15 Z" /> + <path d="M 0 12 L 1 12 L 1 13 L 0 13 L 0 12 Z" /> + <path d="M 15 12 L 16 12 L 16 13 L 15 13 L 15 12 Z" /> + <path d="M 13 0 L 14 0 L 14 1 L 13 1 L 13 0 Z" /> + <path d="M 13 15 L 14 15 L 14 16 L 13 16 L 13 15 Z" /> + <path d="M 0 13 L 1 13 L 1 14 L 0 14 L 0 13 Z" /> + <path d="M 15 13 L 16 13 L 16 14 L 15 14 L 15 13 Z" /> + <path d="M 14 0 L 15 0 L 15 1 L 14 1 L 14 0 Z" /> + <path d="M 14 15 L 15 15 L 15 16 L 14 16 L 14 15 Z" /> + <path d="M 0 14 L 1 14 L 1 15 L 0 15 L 0 14 Z" /> + <path d="M 15 14 L 16 14 L 16 15 L 15 15 L 15 14 Z" /> + <path d="M 15 15 L 16 15 L 16 16 L 15 16 L 15 15 Z" /> + <path class="a" d="M 4 7 L 5 7 L 5 8 L 4 8 L 4 7 Z" /> + <path class="a" d="M 5 7 L 6 7 L 6 8 L 5 8 L 5 7 Z" /> + <path class="a" d="M 3 8 L 4 8 L 4 9 L 3 9 L 3 8 Z" /> + <path class="a" d="M 6 8 L 7 8 L 7 9 L 6 9 L 6 8 Z" /> + <path class="a" d="M 3 9 L 4 9 L 4 10 L 3 10 L 3 9 Z" /> + <path class="a" d="M 4 9 L 5 9 L 5 10 L 4 10 L 4 9 Z" /> + <path class="a" d="M 5 9 L 6 9 L 6 10 L 5 10 L 5 9 Z" /> + <path class="a" d="M 3 10 L 4 10 L 4 11 L 3 11 L 3 10 Z" /> + <path class="a" d="M 4 11 L 5 11 L 5 12 L 4 12 L 4 11 Z" /> + <path class="a" d="M 5 11 L 6 11 L 6 12 L 5 12 L 5 11 Z" /> + <path class="a" d="M 8 7 L 9 7 L 9 8 L 8 8 L 8 7 Z" /> + <path class="a" d="M 11 7 L 12 7 L 12 8 L 11 8 L 11 7 Z" /> + <path class="a" d="M 8 8 L 9 8 L 9 9 L 8 9 L 8 8 Z" /> + <path class="a" d="M 11 8 L 12 8 L 12 9 L 11 9 L 11 8 Z" /> + <path class="a" d="M 8 9 L 9 9 L 9 10 L 8 10 L 8 9 Z" /> + <path class="a" d="M 11 9 L 12 9 L 12 10 L 11 10 L 11 9 Z" /> + <path class="a" d="M 8 10 L 9 10 L 9 11 L 8 11 L 8 10 Z" /> + <path class="a" d="M 11 10 L 12 10 L 12 11 L 11 11 L 11 10 Z" /> + <path class="a" d="M 9 11 L 10 11 L 10 12 L 9 12 L 9 11 Z" /> + <path class="a" d="M 10 11 L 11 11 L 11 12 L 10 12 L 10 11 Z" /> + <path class="a" d="M 11 11 L 12 11 L 12 12 L 11 12 L 11 11 Z" /> +</svg> diff --git a/src/content/img/monogram-filled-512.png b/src/content/img/monogram-filled-512.png Binary files differnew file mode 100644 index 0000000..0b8e36f --- /dev/null +++ b/src/content/img/monogram-filled-512.png diff --git a/src/content/img/monogram-filled.svg b/src/content/img/monogram-filled.svg new file mode 100644 index 0000000..b7781b6 --- /dev/null +++ b/src/content/img/monogram-filled.svg @@ -0,0 +1,28 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 16 16" width="16" height="16"> + <style> + svg { fill: #F3F5F3 } + </style> + <rect width="16" height="16" fill="#1F6F5B" /> + <path class="a" d="M 4 7 L 5 7 L 5 8 L 4 8 L 4 7 Z" /> + <path class="a" d="M 5 7 L 6 7 L 6 8 L 5 8 L 5 7 Z" /> + <path class="a" d="M 3 8 L 4 8 L 4 9 L 3 9 L 3 8 Z" /> + <path class="a" d="M 6 8 L 7 8 L 7 9 L 6 9 L 6 8 Z" /> + <path class="a" d="M 3 9 L 4 9 L 4 10 L 3 10 L 3 9 Z" /> + <path class="a" d="M 4 9 L 5 9 L 5 10 L 4 10 L 4 9 Z" /> + <path class="a" d="M 5 9 L 6 9 L 6 10 L 5 10 L 5 9 Z" /> + <path class="a" d="M 3 10 L 4 10 L 4 11 L 3 11 L 3 10 Z" /> + <path class="a" d="M 4 11 L 5 11 L 5 12 L 4 12 L 4 11 Z" /> + <path class="a" d="M 5 11 L 6 11 L 6 12 L 5 12 L 5 11 Z" /> + <path class="a" d="M 8 7 L 9 7 L 9 8 L 8 8 L 8 7 Z" /> + <path class="a" d="M 11 7 L 12 7 L 12 8 L 11 8 L 11 7 Z" /> + <path class="a" d="M 8 8 L 9 8 L 9 9 L 8 9 L 8 8 Z" /> + <path class="a" d="M 11 8 L 12 8 L 12 9 L 11 9 L 11 8 Z" /> + <path class="a" d="M 8 9 L 9 9 L 9 10 L 8 10 L 8 9 Z" /> + <path class="a" d="M 11 9 L 12 9 L 12 10 L 11 10 L 11 9 Z" /> + <path class="a" d="M 8 10 L 9 10 L 9 11 L 8 11 L 8 10 Z" /> + <path class="a" d="M 11 10 L 12 10 L 12 11 L 11 11 L 11 10 Z" /> + <path class="a" d="M 9 11 L 10 11 L 10 12 L 9 12 L 9 11 Z" /> + <path class="a" d="M 10 11 L 11 11 L 11 12 L 10 12 L 10 11 Z" /> + <path class="a" d="M 11 11 L 12 11 L 12 12 L 11 12 L 11 11 Z" /> +</svg> diff --git a/src/content/img/thumb.svg b/src/content/img/thumb.svg new file mode 100644 index 0000000..ed73c18 --- /dev/null +++ b/src/content/img/thumb.svg @@ -0,0 +1,68 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 16 16" width="16" height="16"> + <style> + svg { fill: #1F6F5B } + @media (prefers-color-scheme: dark) { + svg { fill: #5CC2A2 } + } + </style> + <path class="a" d="M 0 8 L 1 8 L 1 9 L 0 9 L 0 8 Z" /> + <path class="a" d="M 0 13 L 1 13 L 1 14 L 0 14 L 0 13 Z" /> + <path class="a" d="M 1 8 L 2 8 L 2 9 L 1 9 L 1 8 Z" /> + <path class="a" d="M 1 13 L 2 13 L 2 14 L 1 14 L 1 13 Z" /> + <path class="a" d="M 2 8 L 3 8 L 3 9 L 2 9 L 2 8 Z" /> + <path class="a" d="M 2 13 L 3 13 L 3 14 L 2 14 L 2 13 Z" /> + <path class="a" d="M 3 8 L 4 8 L 4 9 L 3 9 L 3 8 Z" /> + <path class="a" d="M 3 13 L 4 13 L 4 14 L 3 14 L 3 13 Z" /> + <path class="a" d="M 4 7 L 5 7 L 5 8 L 4 8 L 4 7 Z" /> + <path class="a" d="M 4 8 L 5 8 L 5 9 L 4 9 L 4 8 Z" /> + <path class="a" d="M 4 13 L 5 13 L 5 14 L 4 14 L 4 13 Z" /> + <path class="a" d="M 5 6 L 6 6 L 6 7 L 5 7 L 5 6 Z" /> + <path class="a" d="M 5 7 L 6 7 L 6 8 L 5 8 L 5 7 Z" /> + <path class="a" d="M 5 13 L 6 13 L 6 14 L 5 14 L 5 13 Z" /> + <path class="a" d="M 6 5 L 7 5 L 7 6 L 6 6 L 6 5 Z" /> + <path class="a" d="M 6 6 L 7 6 L 7 7 L 6 7 L 6 6 Z" /> + <path class="a" d="M 6 14 L 7 14 L 7 15 L 6 15 L 6 14 Z" /> + <path class="a" d="M 7 1 L 8 1 L 8 2 L 7 2 L 7 1 Z" /> + <path class="a" d="M 7 14 L 8 14 L 8 15 L 7 15 L 7 14 Z" /> + <path class="a" d="M 7 15 L 8 15 L 8 16 L 7 16 L 7 15 Z" /> + <path class="a" d="M 7 2 L 8 2 L 8 3 L 7 3 L 7 2 Z" /> + <path class="a" d="M 7 3 L 8 3 L 8 4 L 7 4 L 7 3 Z" /> + <path class="a" d="M 7 4 L 8 4 L 8 5 L 7 5 L 7 4 Z" /> + <path class="a" d="M 7 5 L 8 5 L 8 6 L 7 6 L 7 5 Z" /> + <path class="a" d="M 8 1 L 9 1 L 9 2 L 8 2 L 8 1 Z" /> + <path class="a" d="M 8 15 L 9 15 L 9 16 L 8 16 L 8 15 Z" /> + <path class="a" d="M 9 1 L 10 1 L 10 2 L 9 2 L 9 1 Z" /> + <path class="a" d="M 9 2 L 10 2 L 10 3 L 9 3 L 9 2 Z" /> + <path class="a" d="M 9 6 L 10 6 L 10 7 L 9 7 L 9 6 Z" /> + <path class="a" d="M 9 15 L 10 15 L 10 16 L 9 16 L 9 15 Z" /> + <path class="a" d="M 10 2 L 11 2 L 11 3 L 10 3 L 10 2 Z" /> + <path class="a" d="M 10 3 L 11 3 L 11 4 L 10 4 L 10 3 Z" /> + <path class="a" d="M 10 4 L 11 4 L 11 5 L 10 5 L 10 4 Z" /> + <path class="a" d="M 10 5 L 11 5 L 11 6 L 10 6 L 10 5 Z" /> + <path class="a" d="M 10 6 L 11 6 L 11 7 L 10 7 L 10 6 Z" /> + <path class="a" d="M 11 6 L 12 6 L 12 7 L 11 7 L 11 6 Z" /> + <path class="a" d="M 11 8 L 12 8 L 12 9 L 11 9 L 11 8 Z" /> + <path class="a" d="M 10 15 L 11 15 L 11 16 L 10 16 L 10 15 Z" /> + <path class="a" d="M 11 10 L 12 10 L 12 11 L 11 11 L 11 10 Z" /> + <path class="a" d="M 11 12 L 12 12 L 12 13 L 11 13 L 11 12 Z" /> + <path class="a" d="M 11 14 L 12 14 L 12 15 L 11 15 L 11 14 Z" /> + <path class="a" d="M 11 15 L 12 15 L 12 16 L 11 16 L 11 15 Z" /> + <path class="a" d="M 12 6 L 13 6 L 13 7 L 12 7 L 12 6 Z" /> + <path class="a" d="M 12 8 L 13 8 L 13 9 L 12 9 L 12 8 Z" /> + <path class="a" d="M 12 10 L 13 10 L 13 11 L 12 11 L 12 10 Z" /> + <path class="a" d="M 12 12 L 13 12 L 13 13 L 12 13 L 12 12 Z" /> + <path class="a" d="M 12 14 L 13 14 L 13 15 L 12 15 L 12 14 Z" /> + <path class="a" d="M 13 6 L 14 6 L 14 7 L 13 7 L 13 6 Z" /> + <path class="a" d="M 13 8 L 14 8 L 14 9 L 13 9 L 13 8 Z" /> + <path class="a" d="M 13 10 L 14 10 L 14 11 L 13 11 L 13 10 Z" /> + <path class="a" d="M 13 12 L 14 12 L 14 13 L 13 13 L 13 12 Z" /> + <path class="a" d="M 13 13 L 14 13 L 14 14 L 13 14 L 13 13 Z" /> + <path class="a" d="M 13 14 L 14 14 L 14 15 L 13 15 L 13 14 Z" /> + <path class="a" d="M 14 7 L 15 7 L 15 8 L 14 8 L 14 7 Z" /> + <path class="a" d="M 14 8 L 15 8 L 15 9 L 14 9 L 14 8 Z" /> + <path class="a" d="M 14 9 L 15 9 L 15 10 L 14 10 L 14 9 Z" /> + <path class="a" d="M 14 10 L 15 10 L 15 11 L 14 11 L 14 10 Z" /> + <path class="a" d="M 14 11 L 15 11 L 15 12 L 14 12 L 14 11 Z" /> + <path class="a" d="M 14 12 L 15 12 L 15 13 L 14 13 L 14 12 Z" /> +</svg> diff --git a/src/content/img/wordmark-filled-1500x500.png b/src/content/img/wordmark-filled-1500x500.png Binary files differnew file mode 100644 index 0000000..5102180 --- /dev/null +++ b/src/content/img/wordmark-filled-1500x500.png diff --git a/src/content/img/wordmark-filled-1584x396.png b/src/content/img/wordmark-filled-1584x396.png Binary files differnew file mode 100644 index 0000000..e62ea6c --- /dev/null +++ b/src/content/img/wordmark-filled-1584x396.png diff --git a/src/content/img/wordmark-filled.svg b/src/content/img/wordmark-filled.svg new file mode 100644 index 0000000..6082ca4 --- /dev/null +++ b/src/content/img/wordmark-filled.svg @@ -0,0 +1 @@ +<svg xmlns="http://www.w3.org/2000/svg" width="1500" height="500" viewBox="0 0 1500 500"><rect width="1500" height="500" fill="#1F6F5B"/><g transform="translate(382,178) scale(16)" shape-rendering="crispEdges"><path fill="#F3F5F3" d="M 2 3 L 3 3 L 3 4 L 2 4 L 2 3 Z"/><path fill="#F3F5F3" d="M 3 3 L 4 3 L 4 4 L 3 4 L 3 3 Z"/><path fill="#F3F5F3" d="M 1 4 L 2 4 L 2 5 L 1 5 L 1 4 Z"/><path fill="#F3F5F3" d="M 4 4 L 5 4 L 5 5 L 4 5 L 4 4 Z"/><path fill="#F3F5F3" d="M 1 5 L 2 5 L 2 6 L 1 6 L 1 5 Z"/><path fill="#F3F5F3" d="M 2 5 L 3 5 L 3 6 L 2 6 L 2 5 Z"/><path fill="#F3F5F3" d="M 3 5 L 4 5 L 4 6 L 3 6 L 3 5 Z"/><path fill="#F3F5F3" d="M 1 6 L 2 6 L 2 7 L 1 7 L 1 6 Z"/><path fill="#F3F5F3" d="M 2 7 L 3 7 L 3 8 L 2 8 L 2 7 Z"/><path fill="#F3F5F3" d="M 3 7 L 4 7 L 4 8 L 3 8 L 3 7 Z"/><path fill="#F3F5F3" d="M 6 3 L 7 3 L 7 4 L 6 4 L 6 3 Z"/><path fill="#F3F5F3" d="M 9 3 L 10 3 L 10 4 L 9 4 L 9 3 Z"/><path fill="#F3F5F3" d="M 6 4 L 7 4 L 7 5 L 6 5 L 6 4 Z"/><path fill="#F3F5F3" d="M 9 4 L 10 4 L 10 5 L 9 5 L 9 4 Z"/><path fill="#F3F5F3" d="M 6 5 L 7 5 L 7 6 L 6 6 L 6 5 Z"/><path fill="#F3F5F3" d="M 9 5 L 10 5 L 10 6 L 9 6 L 9 5 Z"/><path fill="#F3F5F3" d="M 6 6 L 7 6 L 7 7 L 6 7 L 6 6 Z"/><path fill="#F3F5F3" d="M 9 6 L 10 6 L 10 7 L 9 7 L 9 6 Z"/><path fill="#F3F5F3" d="M 7 7 L 8 7 L 8 8 L 7 8 L 7 7 Z"/><path fill="#F3F5F3" d="M 8 7 L 9 7 L 9 8 L 8 8 L 8 7 Z"/><path fill="#F3F5F3" d="M 9 7 L 10 7 L 10 8 L 9 8 L 9 7 Z"/><path fill="#F3F5F3" d="M 12 3 L 13 3 L 13 4 L 12 4 L 12 3 Z"/><path fill="#F3F5F3" d="M 13 3 L 14 3 L 14 4 L 13 4 L 13 3 Z"/><path fill="#F3F5F3" d="M 14 4 L 15 4 L 15 5 L 14 5 L 14 4 Z"/><path fill="#F3F5F3" d="M 12 5 L 13 5 L 13 6 L 12 6 L 12 5 Z"/><path fill="#F3F5F3" d="M 13 5 L 14 5 L 14 6 L 13 6 L 13 5 Z"/><path fill="#F3F5F3" d="M 14 5 L 15 5 L 15 6 L 14 6 L 14 5 Z"/><path fill="#F3F5F3" d="M 11 6 L 12 6 L 12 7 L 11 7 L 11 6 Z"/><path fill="#F3F5F3" d="M 14 6 L 15 6 L 15 7 L 14 7 L 14 6 Z"/><path fill="#F3F5F3" d="M 12 7 L 13 7 L 13 8 L 12 8 L 12 7 Z"/><path fill="#F3F5F3" d="M 13 7 L 14 7 L 14 8 L 13 8 L 13 7 Z"/><path fill="#F3F5F3" d="M 14 7 L 15 7 L 15 8 L 14 8 L 14 7 Z"/><path fill="#F3F5F3" d="M 16 3 L 17 3 L 17 4 L 16 4 L 16 3 Z"/><path fill="#F3F5F3" d="M 17 3 L 18 3 L 18 4 L 17 4 L 17 3 Z"/><path fill="#F3F5F3" d="M 18 3 L 19 3 L 19 4 L 18 4 L 18 3 Z"/><path fill="#F3F5F3" d="M 16 4 L 17 4 L 17 5 L 16 5 L 16 4 Z"/><path fill="#F3F5F3" d="M 19 4 L 20 4 L 20 5 L 19 5 L 19 4 Z"/><path fill="#F3F5F3" d="M 16 5 L 17 5 L 17 6 L 16 6 L 16 5 Z"/><path fill="#F3F5F3" d="M 19 5 L 20 5 L 20 6 L 19 6 L 19 5 Z"/><path fill="#F3F5F3" d="M 16 6 L 17 6 L 17 7 L 16 7 L 16 6 Z"/><path fill="#F3F5F3" d="M 19 6 L 20 6 L 20 7 L 19 7 L 19 6 Z"/><path fill="#F3F5F3" d="M 16 7 L 17 7 L 17 8 L 16 8 L 16 7 Z"/><path fill="#F3F5F3" d="M 19 7 L 20 7 L 20 8 L 19 8 L 19 7 Z"/><path fill="#F3F5F3" d="M 24 1 L 25 1 L 25 2 L 24 2 L 24 1 Z"/><path fill="#F3F5F3" d="M 24 2 L 25 2 L 25 3 L 24 3 L 24 2 Z"/><path fill="#F3F5F3" d="M 22 3 L 23 3 L 23 4 L 22 4 L 22 3 Z"/><path fill="#F3F5F3" d="M 23 3 L 24 3 L 24 4 L 23 4 L 23 3 Z"/><path fill="#F3F5F3" d="M 24 3 L 25 3 L 25 4 L 24 4 L 24 3 Z"/><path fill="#F3F5F3" d="M 21 4 L 22 4 L 22 5 L 21 5 L 21 4 Z"/><path fill="#F3F5F3" d="M 24 4 L 25 4 L 25 5 L 24 5 L 24 4 Z"/><path fill="#F3F5F3" d="M 21 5 L 22 5 L 22 6 L 21 6 L 21 5 Z"/><path fill="#F3F5F3" d="M 24 5 L 25 5 L 25 6 L 24 6 L 24 5 Z"/><path fill="#F3F5F3" d="M 21 6 L 22 6 L 22 7 L 21 7 L 21 6 Z"/><path fill="#F3F5F3" d="M 24 6 L 25 6 L 25 7 L 24 7 L 24 6 Z"/><path fill="#F3F5F3" d="M 22 7 L 23 7 L 23 8 L 22 8 L 22 7 Z"/><path fill="#F3F5F3" d="M 23 7 L 24 7 L 24 8 L 23 8 L 23 7 Z"/><path fill="#F3F5F3" d="M 24 7 L 25 7 L 25 8 L 24 8 L 24 7 Z"/><path fill="#F3F5F3" d="M 26 3 L 27 3 L 27 4 L 26 4 L 26 3 Z"/><path fill="#F3F5F3" d="M 28 3 L 29 3 L 29 4 L 28 4 L 28 3 Z"/><path fill="#F3F5F3" d="M 29 3 L 30 3 L 30 4 L 29 4 L 29 3 Z"/><path fill="#F3F5F3" d="M 26 4 L 27 4 L 27 5 L 26 5 L 26 4 Z"/><path fill="#F3F5F3" d="M 27 4 L 28 4 L 28 5 L 27 5 L 27 4 Z"/><path fill="#F3F5F3" d="M 26 5 L 27 5 L 27 6 L 26 6 L 26 5 Z"/><path fill="#F3F5F3" d="M 26 6 L 27 6 L 27 7 L 26 7 L 26 6 Z"/><path fill="#F3F5F3" d="M 26 7 L 27 7 L 27 8 L 26 8 L 26 7 Z"/><path fill="#F3F5F3" d="M 32 3 L 33 3 L 33 4 L 32 4 L 32 3 Z"/><path fill="#F3F5F3" d="M 33 3 L 34 3 L 34 4 L 33 4 L 33 3 Z"/><path fill="#F3F5F3" d="M 31 4 L 32 4 L 32 5 L 31 5 L 31 4 Z"/><path fill="#F3F5F3" d="M 34 4 L 35 4 L 35 5 L 34 5 L 34 4 Z"/><path fill="#F3F5F3" d="M 31 5 L 32 5 L 32 6 L 31 6 L 31 5 Z"/><path fill="#F3F5F3" d="M 32 5 L 33 5 L 33 6 L 32 6 L 32 5 Z"/><path fill="#F3F5F3" d="M 33 5 L 34 5 L 34 6 L 33 6 L 33 5 Z"/><path fill="#F3F5F3" d="M 31 6 L 32 6 L 32 7 L 31 7 L 31 6 Z"/><path fill="#F3F5F3" d="M 32 7 L 33 7 L 33 8 L 32 8 L 32 7 Z"/><path fill="#F3F5F3" d="M 33 7 L 34 7 L 34 8 L 33 8 L 33 7 Z"/><path fill="#F3F5F3" d="M 36 1 L 37 1 L 37 2 L 36 2 L 36 1 Z"/><path fill="#F3F5F3" d="M 36 2 L 37 2 L 37 3 L 36 3 L 36 2 Z"/><path fill="#F3F5F3" d="M 36 3 L 37 3 L 37 4 L 36 4 L 36 3 Z"/><path fill="#F3F5F3" d="M 37 3 L 38 3 L 38 4 L 37 4 L 37 3 Z"/><path fill="#F3F5F3" d="M 38 3 L 39 3 L 39 4 L 38 4 L 38 3 Z"/><path fill="#F3F5F3" d="M 36 4 L 37 4 L 37 5 L 36 5 L 36 4 Z"/><path fill="#F3F5F3" d="M 39 4 L 40 4 L 40 5 L 39 5 L 39 4 Z"/><path fill="#F3F5F3" d="M 36 5 L 37 5 L 37 6 L 36 6 L 36 5 Z"/><path fill="#F3F5F3" d="M 39 5 L 40 5 L 40 6 L 39 6 L 39 5 Z"/><path fill="#F3F5F3" d="M 36 6 L 37 6 L 37 7 L 36 7 L 36 6 Z"/><path fill="#F3F5F3" d="M 39 6 L 40 6 L 40 7 L 39 7 L 39 6 Z"/><path fill="#F3F5F3" d="M 36 7 L 37 7 L 37 8 L 36 8 L 36 7 Z"/><path fill="#F3F5F3" d="M 39 7 L 40 7 L 40 8 L 39 8 L 39 7 Z"/><path fill="#F3F5F3" d="M 42 7 L 43 7 L 43 8 L 42 8 L 42 7 Z"/><path fill="#F3F5F3" d="M 43 7 L 44 7 L 44 8 L 43 8 L 43 7 Z"/><path fill="#F3F5F3" d="M 44 7 L 45 7 L 45 8 L 44 8 L 44 7 Z"/></g></svg>
\ No newline at end of file diff --git a/src/content/img/wordmark.svg b/src/content/img/wordmark.svg new file mode 100644 index 0000000..2afbb4d --- /dev/null +++ b/src/content/img/wordmark.svg @@ -0,0 +1,102 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 46 9" width="46" height="9"> + <style> + svg { fill: black } + .a { fill: #1F6F5B } + @media (prefers-color-scheme: dark) { + svg { fill: white } + .a { fill: #5CC2A2 } + } + </style> + <path class="a" d="M 2 3 L 3 3 L 3 4 L 2 4 L 2 3 Z" /> + <path class="a" d="M 3 3 L 4 3 L 4 4 L 3 4 L 3 3 Z" /> + <path class="a" d="M 1 4 L 2 4 L 2 5 L 1 5 L 1 4 Z" /> + <path class="a" d="M 4 4 L 5 4 L 5 5 L 4 5 L 4 4 Z" /> + <path class="a" d="M 1 5 L 2 5 L 2 6 L 1 6 L 1 5 Z" /> + <path class="a" d="M 2 5 L 3 5 L 3 6 L 2 6 L 2 5 Z" /> + <path class="a" d="M 3 5 L 4 5 L 4 6 L 3 6 L 3 5 Z" /> + <path class="a" d="M 1 6 L 2 6 L 2 7 L 1 7 L 1 6 Z" /> + <path class="a" d="M 2 7 L 3 7 L 3 8 L 2 8 L 2 7 Z" /> + <path class="a" d="M 3 7 L 4 7 L 4 8 L 3 8 L 3 7 Z" /> + <path class="a" d="M 6 3 L 7 3 L 7 4 L 6 4 L 6 3 Z" /> + <path class="a" d="M 9 3 L 10 3 L 10 4 L 9 4 L 9 3 Z" /> + <path class="a" d="M 6 4 L 7 4 L 7 5 L 6 5 L 6 4 Z" /> + <path class="a" d="M 9 4 L 10 4 L 10 5 L 9 5 L 9 4 Z" /> + <path class="a" d="M 6 5 L 7 5 L 7 6 L 6 6 L 6 5 Z" /> + <path class="a" d="M 9 5 L 10 5 L 10 6 L 9 6 L 9 5 Z" /> + <path class="a" d="M 6 6 L 7 6 L 7 7 L 6 7 L 6 6 Z" /> + <path class="a" d="M 9 6 L 10 6 L 10 7 L 9 7 L 9 6 Z" /> + <path class="a" d="M 7 7 L 8 7 L 8 8 L 7 8 L 7 7 Z" /> + <path class="a" d="M 8 7 L 9 7 L 9 8 L 8 8 L 8 7 Z" /> + <path class="a" d="M 9 7 L 10 7 L 10 8 L 9 8 L 9 7 Z" /> + <path d="M 12 3 L 13 3 L 13 4 L 12 4 L 12 3 Z" /> + <path d="M 13 3 L 14 3 L 14 4 L 13 4 L 13 3 Z" /> + <path d="M 14 4 L 15 4 L 15 5 L 14 5 L 14 4 Z" /> + <path d="M 12 5 L 13 5 L 13 6 L 12 6 L 12 5 Z" /> + <path d="M 13 5 L 14 5 L 14 6 L 13 6 L 13 5 Z" /> + <path d="M 14 5 L 15 5 L 15 6 L 14 6 L 14 5 Z" /> + <path d="M 11 6 L 12 6 L 12 7 L 11 7 L 11 6 Z" /> + <path d="M 14 6 L 15 6 L 15 7 L 14 7 L 14 6 Z" /> + <path d="M 12 7 L 13 7 L 13 8 L 12 8 L 12 7 Z" /> + <path d="M 13 7 L 14 7 L 14 8 L 13 8 L 13 7 Z" /> + <path d="M 14 7 L 15 7 L 15 8 L 14 8 L 14 7 Z" /> + <path d="M 16 3 L 17 3 L 17 4 L 16 4 L 16 3 Z" /> + <path d="M 17 3 L 18 3 L 18 4 L 17 4 L 17 3 Z" /> + <path d="M 18 3 L 19 3 L 19 4 L 18 4 L 18 3 Z" /> + <path d="M 16 4 L 17 4 L 17 5 L 16 5 L 16 4 Z" /> + <path d="M 19 4 L 20 4 L 20 5 L 19 5 L 19 4 Z" /> + <path d="M 16 5 L 17 5 L 17 6 L 16 6 L 16 5 Z" /> + <path d="M 19 5 L 20 5 L 20 6 L 19 6 L 19 5 Z" /> + <path d="M 16 6 L 17 6 L 17 7 L 16 7 L 16 6 Z" /> + <path d="M 19 6 L 20 6 L 20 7 L 19 7 L 19 6 Z" /> + <path d="M 16 7 L 17 7 L 17 8 L 16 8 L 16 7 Z" /> + <path d="M 19 7 L 20 7 L 20 8 L 19 8 L 19 7 Z" /> + <path d="M 24 1 L 25 1 L 25 2 L 24 2 L 24 1 Z" /> + <path d="M 24 2 L 25 2 L 25 3 L 24 3 L 24 2 Z" /> + <path d="M 22 3 L 23 3 L 23 4 L 22 4 L 22 3 Z" /> + <path d="M 23 3 L 24 3 L 24 4 L 23 4 L 23 3 Z" /> + <path d="M 24 3 L 25 3 L 25 4 L 24 4 L 24 3 Z" /> + <path d="M 21 4 L 22 4 L 22 5 L 21 5 L 21 4 Z" /> + <path d="M 24 4 L 25 4 L 25 5 L 24 5 L 24 4 Z" /> + <path d="M 21 5 L 22 5 L 22 6 L 21 6 L 21 5 Z" /> + <path d="M 24 5 L 25 5 L 25 6 L 24 6 L 24 5 Z" /> + <path d="M 21 6 L 22 6 L 22 7 L 21 7 L 21 6 Z" /> + <path d="M 24 6 L 25 6 L 25 7 L 24 7 L 24 6 Z" /> + <path d="M 22 7 L 23 7 L 23 8 L 22 8 L 22 7 Z" /> + <path d="M 23 7 L 24 7 L 24 8 L 23 8 L 23 7 Z" /> + <path d="M 24 7 L 25 7 L 25 8 L 24 8 L 24 7 Z" /> + <path d="M 26 3 L 27 3 L 27 4 L 26 4 L 26 3 Z" /> + <path d="M 28 3 L 29 3 L 29 4 L 28 4 L 28 3 Z" /> + <path d="M 29 3 L 30 3 L 30 4 L 29 4 L 29 3 Z" /> + <path d="M 26 4 L 27 4 L 27 5 L 26 5 L 26 4 Z" /> + <path d="M 27 4 L 28 4 L 28 5 L 27 5 L 27 4 Z" /> + <path d="M 26 5 L 27 5 L 27 6 L 26 6 L 26 5 Z" /> + <path d="M 26 6 L 27 6 L 27 7 L 26 7 L 26 6 Z" /> + <path d="M 26 7 L 27 7 L 27 8 L 26 8 L 26 7 Z" /> + <path d="M 32 3 L 33 3 L 33 4 L 32 4 L 32 3 Z" /> + <path d="M 33 3 L 34 3 L 34 4 L 33 4 L 33 3 Z" /> + <path d="M 31 4 L 32 4 L 32 5 L 31 5 L 31 4 Z" /> + <path d="M 34 4 L 35 4 L 35 5 L 34 5 L 34 4 Z" /> + <path d="M 31 5 L 32 5 L 32 6 L 31 6 L 31 5 Z" /> + <path d="M 32 5 L 33 5 L 33 6 L 32 6 L 32 5 Z" /> + <path d="M 33 5 L 34 5 L 34 6 L 33 6 L 33 5 Z" /> + <path d="M 31 6 L 32 6 L 32 7 L 31 7 L 31 6 Z" /> + <path d="M 32 7 L 33 7 L 33 8 L 32 8 L 32 7 Z" /> + <path d="M 33 7 L 34 7 L 34 8 L 33 8 L 33 7 Z" /> + <path d="M 36 1 L 37 1 L 37 2 L 36 2 L 36 1 Z" /> + <path d="M 36 2 L 37 2 L 37 3 L 36 3 L 36 2 Z" /> + <path d="M 36 3 L 37 3 L 37 4 L 36 4 L 36 3 Z" /> + <path d="M 37 3 L 38 3 L 38 4 L 37 4 L 37 3 Z" /> + <path d="M 38 3 L 39 3 L 39 4 L 38 4 L 38 3 Z" /> + <path d="M 36 4 L 37 4 L 37 5 L 36 5 L 36 4 Z" /> + <path d="M 39 4 L 40 4 L 40 5 L 39 5 L 39 4 Z" /> + <path d="M 36 5 L 37 5 L 37 6 L 36 6 L 36 5 Z" /> + <path d="M 39 5 L 40 5 L 40 6 L 39 6 L 39 5 Z" /> + <path d="M 36 6 L 37 6 L 37 7 L 36 7 L 36 6 Z" /> + <path d="M 39 6 L 40 6 L 40 7 L 39 7 L 39 6 Z" /> + <path d="M 36 7 L 37 7 L 37 8 L 36 8 L 36 7 Z" /> + <path d="M 39 7 L 40 7 L 40 8 L 39 8 L 39 7 Z" /> + <path d="M 42 7 L 43 7 L 43 8 L 42 8 L 42 7 Z" /> + <path d="M 43 7 L 44 7 L 44 8 L 43 8 L 43 7 Z" /> + <path d="M 44 7 L 45 7 L 45 8 L 44 8 L 44 7 Z" /> +</svg> diff --git a/src/content/l/multiline-sh b/src/content/l/multiline-sh new file mode 120000 index 0000000..77800c7 --- /dev/null +++ b/src/content/l/multiline-sh @@ -0,0 +1 @@ +../en/til/2026/09/05/multiline-sh-comments.html
\ No newline at end of file diff --git a/src/content/live b/src/content/live new file mode 120000 index 0000000..dc1dc0c --- /dev/null +++ b/src/content/live @@ -0,0 +1 @@ +/dev/null
\ No newline at end of file diff --git a/src/content/public.asc b/src/content/public.asc deleted file mode 100644 index 533b54c..0000000 --- a/src/content/public.asc +++ /dev/null @@ -1,86 +0,0 @@ ------BEGIN PGP PUBLIC KEY BLOCK----- - -mQINBFjVvh4BEADIlHUiO6IfkhcNm3J7ilXERgimvKuFNyLIUPZlDcESC1ORrv4y -9slMDA5uojXctuLRC7nNdynLP+eFFfVUQ+hUXcV24AzyOE0CYo5c4PQA5TLe2AUC -E9YqqfQF4XuNddY+UpcG47MuVDR+6SHkFkF29ATzpmShJj41lc7a9CdRib+62Wpe -h7WJOFj/YoxMCBBzic4tiFNgoYobu+lLxyA4T2kCmxEaiZzc6eXBDDgJ0STL4+S8 -avpglaQ+mb5gHbH0yOtuwDG3sWyHKf7LSRVtzWvOqaGmRUmmDsSPjb5vQqvT8EMq -UfqFFZhScLalthF3PhG0SLXPvoCoRm2aLkN+O3sv057RqaN8E39223mmz6EMXmLk -H/U5qk2SUl3dx86dIQcB+2WUVu5zuFyfR1g6tD+DcqzxGc9XB7Gz/0TTDf3OimHb -rp1x5i/04198ocRZT3MzXx8H25tLMS/rHmE87YdgPhMTWheSUevyhoGNHfAOcDwX -P2oGzELXbLqHxtjENMEw2E996KrSmpcz7WOqIl3PHS1J6eRZoYQesXE+SZTeIiYb -wD0kkZGYhBZbtLC4VWIuU2T3AL/2hF6aUh1tj1B6vcV0i3HpIHNbvPAF/I0NUhhc -Gxwwi+ggG/MBHBbxkq7LvG5DfDbav0ZoZaov5dyhtX0CBWjVYATvjRfeAwARAQAB -tBlFdUFuZHJlaCA8ZXVAZXVhbmRyZS5vcmc+iQI5BBMBCAAjBQJY1b4eAhsDBwsJ -CAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQgfkOw801YGCWzg//QtDpwgbDY9uC -Y9a/RgUsbqGAYzSInsbyDCXrAAhWGzkDMLPeFp03Sw9QyCDe0wWu8L2H4hV/FN58 -+4G6353ISwkqsf9R+P9lQs/5dwG7lp5/Gez8bZK3y7zFrdtVwcOCb4De+9fhPsgP -9pRU8dHpLNo8Ui9IzbiYla7aGxXQdkXU2cvOuEoiuFgvcWU1KWNOWrjImATcC8EF -8VaEaZYGRXz8lML8KgsAUxrjFkk6tqxrMlOLTjY0BuzcYZpt5XLZ2NuSIDYBoSib -uBQ1H7DLGa+r0hnNjVEBmMOvFA1hbWa33h1AyYjYhoeVlBYpoHuDosEFqkwZ+otz -zvImaRAOOFX1IehifTGEFie3imuOHdVuRjXb8SGu8Cgeby0T096A/vf+L1S35nc2 -mdRCUE/SIURW6hfH7uT6KqpokU86vozKmNzIcV3zhAXJ9UYwQqZgg2H3DOcTtZyE -jVBl2glspoclsfR20T+g+qPqNDAgoDbC71fEAbUTACQau162utpHiabog7e7vyhI -go5xdjxA8xb3Jtn39pYzbg75ArZqPbxHNZ38m00EBtC5EkD4DFh0cpQ2peuZIh1k -c5bragCt8o6cV9t4jaq+TtVv4PrFEPqEd+w1FqqwabBq3xSsIgKg2X5rXQkktymB -un+oN41wofuTZIoGNt8nnGb+skFBxgyJAlYEEwEKAEACGwMHCwkIBwMCAQYVCAIJ -CgsEFgIDAQIeAQIXgBYhBFva6biy9sa8uw1s5YH5DsPNNWBgBQJi00VjBQkNv+5F -AAoJEIH5DsPNNWBgy9IP/A8ERtFP3B5BDfIb4BUyw9AvWPAMyNfuKiXVcfrn/CGn -D+x0dx5doGcIXskTWGEow1/6sFSheYk728wO3pp+DUaDp+2rVwO2AsKBEjBptk9i -b9YJ4fl4rYtltscLHBGflrQ6C8jIwBqt72Ots+F7IEXy1NcskS/jU6DUzLPDmOog -doM5IHD/2Fekmq8QVvyryH0nT5YxaJ/qRgOr1NTnnmgTcZHO7l21gJNvWo1QJLME -lz5xNXRN/rFl5xQ3NxqVh9hwDwp/k5lXW0dxJCpmjbNKG2hNsTYrjTFrG6mSaER5 -0rdzGzQVWavyR+PDY5KRRKupYY4P5luLFy9zCdBr+ZBDTHmLfRcwXubLOSmq+gUO -8LievpDZITHtgtWGIhWWqA80gOoqWRfAO+cpDpCqWIa+KoZyaxd19WXUqHEBr6Y9 -ZcyCCenM/+WsfmySNqAo6HGVoehewMVSRI6GObS9bdDDJTa3QySQGjdRyAn3uavo -JwjpXfy09Kirji2x9G85OzOdXDNUrMqu0nB4AFxOU0SLhg0YpRJCig/2uuYRhRMe -gLFM52AGxk1LfK9Pjrr2V029eRclD8SwC/F51YFP6CKGMyYHJWuaBJL1HXr/fzDD -sLq4K1TZN/8TpYRA6t8B1mY/57KVsv2naWprmVv7q2eNU17nriLQiYYqfybcVGwn -uQINBFjVvh4BEADzt2iKa1gSksHtTFkPQ5ULqUF2sHDClr3ykbLq/AxgSCON58eP -A9SKQy2O+qDpojHAN1UULJgHEn34afzMkBzjxcJXMRgaTV2M+1trjwx/VluD9OKX -wmnhmSdvCIP7Z0qdhU78maLq10UG1vVwej3kVlxsf4Eu2ZA+NeIr7Tj0DERqEDQo -DRtNPVEy3h1xoYruy/VjNDi1CI3yFkM6HW1CgRA50rI7GDtvOuitZy+9Lpqs0mWq -vdApWZxoQwslFcziNd+ZVaQjgO6LSnkDttRkAOblFiD710OQy3/Yo97i7bqsKrnZ -qQMRUk0n12VXY9I94c7ELfViVqGk123ELtTViiIz5BT5iQRkJj1GiizTgGY6cfsj -kwWwvabpmWYdyQ85sYoVuNAPz3yDaLdtStWRNHWi4+UHC03J2BiBgIrQbuXoNGuc -j0b1fsntdntaBoZgFygwW6kXUjHLeEfnrGX3C2X49zg0rBTvEzdZwr2K0xgc2z26 -1EEf5ObmOGRt27K1fwrCxKHbKTscReHv78S4v3uN/9LvHfvIEaBoYHqMCcxy7Aii -dk+02dNDO/jZDnTAJH2NWhyB+PJvrlnK34zHhUMVH0i5nUjaCDL/n07Vd2sbE5qW -ivE2MWeayVKRGPci80tEGA1i42FJzGiA1uZrxXNImnsyxQyS8cr9iKoTIQARAQAB -iQIfBBgBCAAJBQJY1b4eAhsMAAoJEIH5DsPNNWBg+bYQALJyD1nyuz8+vl8rqj7K -Z9aRSW+XeG/wz6xrAqdY3OVvHwXYw33pgOmhNhfMUgP/Uy5OsxZdjIO7NzyKa2H9 -JoVSsAs/eLQDOQCcwXruBND6zuxt99kZh6o/Xp4lII9vuLafKner+fWluFHhOy/w -E3Q3VwCbC9npbmzweEl9Q83R7IxbEhtFF5HV0wKVRzW/GX7iWADoHpkAAQ2sUnQp -HhE1wOrdPm0dD9BEbTRQHekUiIQ8cFoORyWbJBwbflY64ioaFjyM+Ji49pNMykie -LzQFW1UYyhkXJeTvv93ym4XyMi2mhsOzna7mG1bonKvbKj6qaXb7gFHUXHh/ARuu -6CNARzBh6BTp+7c1brthGjT/L8CxrAeW2oE5wVIRuk8mdKiFoK3BuXc1P+vsnp36 -ioOQ0y+KPcp+PSbw6oDp7hTHztcW/3EoAgyHneWCmtYYi6RmVptTNpeeyHwqRP/O -elCN1cw9zopofVQhnxDEUgzVPrWWaE7UR6vrHbzlXvWMeGTYtmdmo/9xkYbQzZW7 -y90QLUGyDwQ+KeCG29W3EhygGy3myVQbRaXywgzzO2YvovjATDa7wZQrXNoVE7J9 -uLonNtRlyRlTAfFP6hCLDXwuE6WRHXhdu7aFKbq0LQGFv5hY4wPUp8vnUtGYT/wo -qqSkuSYhzNvmuKBIHPs6YD8duQINBGC7n68BEADnUv7iWOejQNa3fZ6v4lkHT6qF -Rp2+NuzIpFJ2Vy7eP58XZoiz6HJPcCU8Hf95JXwaXEwS4S7mXdw1x60hd8JIe058 -Ek6MZSSVQmlLfocGsAYj1wTrLmnQ8+PV0IeQlNj1aytBI1fL+v3IPt+JdLt6b+g3 -vwcEUU9efzxx2E0KZ5GIpb2meiCQ6ha+tcd7XqegB53eQj/h/coE2zLJodpaJ3xb -j894pE/OJCNC0+4d0Sv7oHhY7QoLYldTQbSgPyhyfl4iZpJf6OEPZxK2cJaB+cbe -oBB6aGNyU+CIJToM+uAJJ7H7EpvxfcnfJQ1PuY5szTdvFbW820euiUEKEW69mW4u -aFNPSc6D4Z8tZ5hXQIqBD40irULhF0CYNkIILmyNV/KJIZ5HkbQ1q+UrCFHJyvuH -/3aCTjj9OSfE7xHPQ3xd3Xw8vvj0Mjie09xFbbcklBTw5WRzH7cw8c+Q0O69kZZ8 -b+ykcdzWTeZeWNdnzptNqnMjfheig90rUIJ7DN0c+53jCUcGpWJxJhcYF9Uk1RNH -mSE5+VzK1y+20t0grVFX90nApm4Tl35QPrX7Qxp9C81cWiUB8xCAE6jYrmd4x+P/ -3wSQfc1Xg0Eg3QjJB+6JD7cbyDJpzDR3ja+CLZCAr9I0B4rDKD2d6et/z67iXPnZ -UWMyZ8RVVZPFbBMOTwARAQABiQI8BBgBCAAmFiEEW9rpuLL2xry7DWzlgfkOw801 -YGAFAmC7n68CGyAFCQPCZwAACgkQgfkOw801YGAS7hAAvAEKKdNj8NK8STfehHIH -QYxdotNHJc3b0rUa/Kzb9ELTvYgheHH6Dq26c/YSoApJxUrgUVDSJwAJV4T9JqPX -rfCfhyzfdxocXVAWH01dhWWxCOh/S/gLB/r2CvymbFbNGY6y8vyxG8TahGYZQJEE -ynUtw+S1sfrbqc8EMGmnw67z/hK3JIcfNrNxvt7FXo1HHcNEMRiah2NtwO9sumEK -041y7v2efGS4z1i5FIarf/2HtIgIGs77B0G54o4IhgzJzUEYWlHumXKMsETNT3zI -9uukR16RRkwxqOj6fOD9qNvnM1Tzf9T5DClrS5klz448qlpWWiUDABmyBMDqGKWS -vr6oi24iemJ4LoAUws1tPCE5WukFKr69UQ9Ab4DuSWwPbQ51RUjMJPeqdV53GnjU -H6gNBKqxlC0ccuwY3V2kDb8lc46pyN7rqLVZ0IENZ0PFHmfvH+rPkybEjRBqFbhf -nkDPnHuXSPhsCGPk45OQxnqqCf4QFqyOTG3slc6yk/N4Bz0IVNOFq5sewISGeolb -4uOF951f5gA2cUy5FXu8Hf8vkdJuB70nHtJLNijloPbAQFq9SuVpvAOlSFLB2wiy -VgSGXzb4jfIEJidZlsveHDkg/LTzrkHu+f1Qj5thHXN7ARPWvZp1eNFSA6iV7Sho -LsPdAc9FGcUNEy+/AlLpM1Y= -=2ZCp ------END PGP PUBLIC KEY BLOCK----- diff --git a/src/content/public.asc.txt b/src/content/public.asc.txt new file mode 100644 index 0000000..2519d0f --- /dev/null +++ b/src/content/public.asc.txt @@ -0,0 +1,99 @@ +-----BEGIN PGP PUBLIC KEY BLOCK----- + +mQINBFjVvh4BEADIlHUiO6IfkhcNm3J7ilXERgimvKuFNyLIUPZlDcESC1ORrv4y +9slMDA5uojXctuLRC7nNdynLP+eFFfVUQ+hUXcV24AzyOE0CYo5c4PQA5TLe2AUC +E9YqqfQF4XuNddY+UpcG47MuVDR+6SHkFkF29ATzpmShJj41lc7a9CdRib+62Wpe +h7WJOFj/YoxMCBBzic4tiFNgoYobu+lLxyA4T2kCmxEaiZzc6eXBDDgJ0STL4+S8 +avpglaQ+mb5gHbH0yOtuwDG3sWyHKf7LSRVtzWvOqaGmRUmmDsSPjb5vQqvT8EMq +UfqFFZhScLalthF3PhG0SLXPvoCoRm2aLkN+O3sv057RqaN8E39223mmz6EMXmLk +H/U5qk2SUl3dx86dIQcB+2WUVu5zuFyfR1g6tD+DcqzxGc9XB7Gz/0TTDf3OimHb +rp1x5i/04198ocRZT3MzXx8H25tLMS/rHmE87YdgPhMTWheSUevyhoGNHfAOcDwX +P2oGzELXbLqHxtjENMEw2E996KrSmpcz7WOqIl3PHS1J6eRZoYQesXE+SZTeIiYb +wD0kkZGYhBZbtLC4VWIuU2T3AL/2hF6aUh1tj1B6vcV0i3HpIHNbvPAF/I0NUhhc +Gxwwi+ggG/MBHBbxkq7LvG5DfDbav0ZoZaov5dyhtX0CBWjVYATvjRfeAwARAQAB +tCFwYXBvLmltIFN1cHBvcnQgPHN1cHBvcnRAcGFwby5pbT6JAlcEEwEIAEECGwMF +CwkIBwICIgIGFQoJCAsCBBYCAwECHgcCF4AWIQRb2um4svbGvLsNbOWB+Q7DzTVg +YAUCamFh6AUJFU4KygAKCRCB+Q7DzTVgYLrhEACKa/uQt+BCDJIFUHKQ36mKzJR5 +ubKym2HRtsbC2L+aVW8pWR7vk3VhqOWV9PLL4hOPJpKWMvxZEzpfYWvbFgjK3Uet +b2iBTOOU9YJ5eyO6kCcSKHWMUWegk23RGCCyxLfeTz9pf4QSGwCjkLiWquPOiT3o +RHCpOkWZg3aBlsxIozmdi/Dv0zNYNfpu6N7NRMfQDv8IYEq66VkpuPulQmSu+SAE +ja7/Z7ed4/mQxC6gN5EIULrmI42h5LnyCTENNVN8vSZMNvjTeXlU53AHEGChaA9l ++/YxMYKk3RS+CS+1hQWjT2vctZ4+7kVfaZf8rLwcSwHhhmEdgxKCH1stP4vCbtVo +xOvQNPwLi+5rSve3XlPEwXspteN9Oja8/xPG5qhtHz2dudg77mhL5+Yo5iemiA5A +6FLtpDKW//cS8vkW1B6RBsq1vwzXdbvF9ePzOf8IO/8vZTBEEv/Taf9SW4pSWE3J +gzYGjiJwWoZZnfpDus4BVw4No6bfbQ/XGFgXu6q66R5KQRxrhPMOQ1EU9OB6hm4z +g2V1MALnVFuBWG1iWp+PTH+mlWS/wnXdsKsImOVMDG2bW31mO9diR4qVu7IMZweQ +FUo6R6e6gcmLW8z/Lad+eyp8srCWed4N+TxVeDG1bECBIF++gjGnHH7F0QXuxIql +7OZGV01whHmiItNfc7QZRXVBbmRyZWggPGV1QGV1YW5kcmUub3JnPokCOQQTAQgA +IwUCWNW+HgIbAwcLCQgHAwIBBhUIAgkKCwQWAgMBAh4BAheAAAoJEIH5DsPNNWBg +ls4P/0LQ6cIGw2PbgmPWv0YFLG6hgGM0iJ7G8gwl6wAIVhs5AzCz3hadN0sPUMgg +3tMFrvC9h+IVfxTefPuBut+dyEsJKrH/Ufj/ZULP+XcBu5aefxns/G2St8u8xa3b +VcHDgm+A3vvX4T7ID/aUVPHR6SzaPFIvSM24mJWu2hsV0HZF1NnLzrhKIrhYL3Fl +NSljTlq4yJgE3AvBBfFWhGmWBkV8/JTC/CoLAFMa4xZJOrasazJTi042NAbs3GGa +beVy2djbkiA2AaEom7gUNR+wyxmvq9IZzY1RAZjDrxQNYW1mt94dQMmI2IaHlZQW +KaB7g6LBBapMGfqLc87yJmkQDjhV9SHoYn0xhBYnt4prjh3VbkY12/EhrvAoHm8t +E9PegP73/i9Ut+Z3NpnUQlBP0iFEVuoXx+7k+iqqaJFPOr6MypjcyHFd84QFyfVG +MEKmYINh9wznE7WchI1QZdoJbKaHJbH0dtE/oPqj6jQwIKA2wu9XxAG1EwAkGrte +trraR4mm6IO3u78oSIKOcXY8QPMW9ybZ9/aWM24O+QK2aj28RzWd/JtNBAbQuRJA ++AxYdHKUNqXrmSIdZHOW62oArfKOnFfbeI2qvk7Vb+D6xRD6hHfsNRaqsGmwat8U +rCICoNl+a10JJLcpgbp/qDeNcKH7k2SKBjbfJ5xm/rJBQcYMiQJWBBMBCgBAAhsD +BwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AWIQRb2um4svbGvLsNbOWB+Q7DzTVg +YAUCamFh6AUJFU4KygAKCRCB+Q7DzTVgYJjSD/9rgUEuF/1x+veK/nclBVHt0fyd +ze+ESqZH+sZx2U564AqP6yehjoz+cervDwmER1ja2sFyCddXS3CUCwr7B0vVMc2R +MH5yaQ7vtnNV0xAGKs7rPiY/KWfZRoif+2dU9l6v/n4zZTqbC/JY8iuuF8oyC0MH +o8sKnsmPJiLg0he2PChThjbmVThHhCA6aM35q4zeq3NYLdRECzbySerqcZxONPFv +9jIYTN6eXdq4Bm/qf835/r9gTtvrodn3Fu7XRINpxxmGTJ2cCbZNBPQbvTd9FsSO +Sc1I1K6uO81x3xoPSmxH0QIhaJmIfvvatHGexbaqipOYDQA904wzFGrNdNm9/oXc +CAA61g/URdUtrgnFPe87LydFkhfym5OQSHwywO9OxlHCSKaprziUIq+UuQ1t5ka7 ++DrdHj6GA6F/aeOnP9J0wZJY0ESElzhU7HEcObUstDeabI27AhmrwidJmjwYj4i+ +Z+lNFiRM+3dBAxlg6d4V0x4M2yG2lrHdFoMxUT2aKYjgpViPjtvtxQ0q5MOxusg4 +vPHE3sT3UGggezUicf1A5fKhNxN9gE+ImyvxDqZBNN+z1ClVHqGBJt9slaCSRZwi +tVzf8bu36uZquQzkNrzJgNEwVubG6o2ec7bVxDnvCFPACWY4hbCC80E+1F/hs7I8 +8V9mqXIBAIGz/PyLLLkCDQRY1b4eARAA87doimtYEpLB7UxZD0OVC6lBdrBwwpa9 +8pGy6vwMYEgjjefHjwPUikMtjvqg6aIxwDdVFCyYBxJ9+Gn8zJAc48XCVzEYGk1d +jPtba48Mf1Zbg/Til8Jp4ZknbwiD+2dKnYVO/Jmi6tdFBtb1cHo95FZcbH+BLtmQ +PjXiK+049AxEahA0KA0bTT1RMt4dcaGK7sv1YzQ4tQiN8hZDOh1tQoEQOdKyOxg7 +bzrorWcvvS6arNJlqr3QKVmcaEMLJRXM4jXfmVWkI4Dui0p5A7bUZADm5RYg+9dD +kMt/2KPe4u26rCq52akDEVJNJ9dlV2PSPeHOxC31YlahpNdtxC7U1YoiM+QU+YkE +ZCY9Roos04BmOnH7I5MFsL2m6ZlmHckPObGKFbjQD898g2i3bUrVkTR1ouPlBwtN +ydgYgYCK0G7l6DRrnI9G9X7J7XZ7WgaGYBcoMFupF1Ixy3hH56xl9wtl+Pc4NKwU +7xM3WcK9itMYHNs9utRBH+Tm5jhkbduytX8KwsSh2yk7HEXh7+/EuL97jf/S7x37 +yBGgaGB6jAnMcuwIonZPtNnTQzv42Q50wCR9jVocgfjyb65Zyt+Mx4VDFR9IuZ1I +2ggy/59O1XdrGxOalorxNjFnmslSkRj3IvNLRBgNYuNhScxogNbma8VzSJp7MsUM +kvHK/YiqEyEAEQEAAYkCHwQYAQgACQUCWNW+HgIbDAAKCRCB+Q7DzTVgYPm2EACy +cg9Z8rs/Pr5fK6o+ymfWkUlvl3hv8M+sawKnWNzlbx8F2MN96YDpoTYXzFID/1Mu +TrMWXYyDuzc8imth/SaFUrALP3i0AzkAnMF67gTQ+s7sbffZGYeqP16eJSCPb7i2 +nyp3q/n1pbhR4Tsv8BN0N1cAmwvZ6W5s8HhJfUPN0eyMWxIbRReR1dMClUc1vxl+ +4lgA6B6ZAAENrFJ0KR4RNcDq3T5tHQ/QRG00UB3pFIiEPHBaDkclmyQcG35WOuIq +GhY8jPiYuPaTTMpIni80BVtVGMoZFyXk77/d8puF8jItpobDs52u5htW6Jyr2yo+ +qml2+4BR1Fx4fwEbrugjQEcwYegU6fu3NW67YRo0/y/AsawHltqBOcFSEbpPJnSo +haCtwbl3NT/r7J6d+oqDkNMvij3Kfj0m8OqA6e4Ux87XFv9xKAIMh53lgprWGIuk +ZlabUzaXnsh8KkT/znpQjdXMPc6KaH1UIZ8QxFIM1T61lmhO1Eer6x285V71jHhk +2LZnZqP/cZGG0M2Vu8vdEC1Bsg8EPinghtvVtxIcoBst5slUG0Wl8sIM8ztmL6L4 +wEw2u8GUK1zaFROyfbi6JzbUZckZUwHxT+oQiw18LhOlkR14Xbu2hSm6tC0Bhb+Y +WOMD1KfL51LRmE/8KKqkpLkmIczb5rigSBz7OmA/HbkCDQRgu5+vARAA51L+4ljn +o0DWt32er+JZB0+qhUadvjbsyKRSdlcu3j+fF2aIs+hyT3AlPB3/eSV8GlxMEuEu +5l3cNcetIXfCSHtOfBJOjGUklUJpS36HBrAGI9cE6y5p0PPj1dCHkJTY9WsrQSNX +y/r9yD7fiXS7em/oN78HBFFPXn88cdhNCmeRiKW9pnogkOoWvrXHe16noAed3kI/ +4f3KBNsyyaHaWid8W4/PeKRPziQjQtPuHdEr+6B4WO0KC2JXU0G0oD8ocn5eImaS +X+jhD2cStnCWgfnG3qAQemhjclPgiCU6DPrgCSex+xKb8X3J3yUNT7mObM03bxW1 +vNtHrolBChFuvZluLmhTT0nOg+GfLWeYV0CKgQ+NIq1C4RdAmDZCCC5sjVfyiSGe +R5G0NavlKwhRycr7h/92gk44/TknxO8Rz0N8Xd18PL749DI4ntPcRW23JJQU8OVk +cx+3MPHPkNDuvZGWfG/spHHc1k3mXljXZ86bTapzI34XooPdK1CCewzdHPud4wlH +BqVicSYXGBfVJNUTR5khOflcytcvttLdIK1RV/dJwKZuE5d+UD61+0MafQvNXFol +AfMQgBOo2K5neMfj/98EkH3NV4NBIN0IyQfuiQ+3G8gyacw0d42vgi2QgK/SNAeK +wyg9nenrf8+u4lz52VFjMmfEVVWTxWwTDk8AEQEAAYkCPAQYAQgAJgIbIBYhBFva +6biy9sa8uw1s5YH5DsPNNWBgBQJk/eCDBQkIBKfUAAoJEIH5DsPNNWBgXdUP/27B +0vAdg9zwhOvRXTgYRtAZjsPPGegc9o8ACxt74aKOYGYICo0Xnru/M4fXVS4LLRf1 +c+iuf9NxonWTJUnORts7i9oM2CDWs3HT3yc06GPwfvHTPUrCFwiOWroD06op1SgT +I2stjSPO4qK1RrNAnFFZgh9d/Z7NvDiqwlt1XbJB3ZcZJHOSZnhxlOfedPlgqxLw +z7qztTFm9l4Y816I80CG3/dVDTlqtKQfJDk7RfyTavxLSPR4TcNaYWoqjGnzb48C +QXyreSrL6OGdReoBuNw8eQSxohBWI3PKBn/G+4HvzlvDFa745T9B4yTnvVJh2SF0 +Cqbg5FcRsHpTdL2qo9e1MleeBkGZnxWFKWZWcJoczD1pxjZGIFNI1aMxoSMB0yA6 +T0PAbuproAk2TXn8NDzsFJP6ERM+3c1ARNiwZ8wCjiNChxX5cEh4ciNltkQ54ZF/ +7SIQWBxcJ8IwHuckIwr30Om0lhOiuPbZWrpLX0/dBCU4qdwOVaQdaRopJU7E2XIW +rN/8i+o7WAE9xdIdVHZg0H+WPWa/lOs5yfsSXES3Y7wxOR8chW038/y82TECk7LP ++dZyFKJ26VSvqPHxMqmji/UJjOB+K5xaaCflnzsKzsaEf5fRCIrCp6ldIXh0/x0s +pqsEJPXPOb40oBnNzLOSv9X++wKq+oXWEX8OSqhr +=mG9H +-----END PGP PUBLIC KEY BLOCK----- diff --git a/src/content/s b/src/content/s new file mode 120000 index 0000000..dc1dc0c --- /dev/null +++ b/src/content/s @@ -0,0 +1 @@ +/dev/null
\ No newline at end of file diff --git a/src/content/security.txt b/src/content/security.txt deleted file mode 120000 index abdf74b..0000000 --- a/src/content/security.txt +++ /dev/null @@ -1 +0,0 @@ -.well-known/security.txt
\ No newline at end of file diff --git a/src/content/style.css b/src/content/style.css index 653092f..6c69152 100644 --- a/src/content/style.css +++ b/src/content/style.css @@ -1,12 +1,83 @@ +:root { + --color-fg: #171D1A; + --color-bg: #F3F5F3; + --color-2nd-fg: #4A5651; + --color-2nd-bg: #E9EDEA; + --color-3rd-fg: #6F7B76; + --color-line: #CBD3CE; + --color-line-soft: #DDE3DF; + --color-accent: #1F6F5B; + --color-accent-soft: #D8E9E2; + --color-ochre: #9A6B1F; + --color-code-bg: #E6EBE8; + --color-pre-border: #CBD3CE; + + --font-body: sans-serif; + --font-display: serif; + --font-mono: monospace; + + color: var(--color-fg); + background-color: var(--color-bg); +} + +@media (prefers-color-scheme: dark) { + :root { + --color-fg: #E4EAE7; + --color-bg: #121716; + --color-2nd-fg: #A7B3AE; + --color-2nd-bg: #19201E; + --color-3rd-fg: #7E8A85; + --color-line: #2C3733; + --color-line-soft: #222B28; + --color-accent: #5CC2A2; + --color-accent-soft: #1B3A31; + --color-ochre: #D9A85A; + --color-code-bg: #1C2522; + --color-pre-border: #2C3733; + } +} + +a { + color: var(--color-accent); +} + +a:visited { + color: var(--color-accent); +} + + /* General declarations */ body { margin: 0px auto; padding: 1%; - max-width: 750px; + max-width: 68ch; + font-family: var(--font-body); + font-size: 17px; + line-height: 1.6; } -.simple-icon { +h1, h2, h3, h4, h5, h6 { + font-family: var(--font-display); + font-weight: 500; + line-height: 1.2; + text-wrap: balance; +} + +h1 { font-size: 39px; } +h2 { font-size: 31px; } +h3 { font-size: 25px; } +h4, h5, h6 { font-size: 20px; } + +code, pre, .mono { + font-family: var(--font-mono); +} + +code { + background-color: var(--color-code-bg); +} + +.icon { width: 22px; height: 22px; vertical-align: middle; @@ -16,80 +87,144 @@ body { /* Navigation header */ header { - border-bottom: 2px solid black; margin-bottom: 30px; padding: 12px 0px 12px 0px; } -nav .nav-row { +nav, nav ol { display: flex; flex-direction: row; - justify-content: space-between; flex-wrap: wrap; + align-items: center; } +nav ol { + list-style: none; + margin: 0px; + padding: 0px; +} + + nav a { - color: maroon; font-size: 18px; margin: 12px; text-decoration: none; } -nav ul, nav li { - display: inline; +nav a img { + width: 32px; + height: 32px; +} + +nav ol.languages { + margin-left: auto; } -nav ul li a { - color: black; +nav ol.languages a, nav ol.languages li { + font-family: var(--font-mono); font-size: 14px; margin: 6px; } +nav ol.languages li { + color: var(--color-fg); +} + +nav ol.languages a, +nav ol.languages a:visited { + color: var(--color-2nd-fg); +} + +nav ol.languages a:hover { + color: var(--color-fg); +} + +#tagline { + margin: 0px 12px 6px 12px; + font-size: 14px; + color: var(--color-2nd-fg); +} + + +/* A published recording */ + +#media video, #media audio { + width: 100%; + max-width: 100%; +} + +#media-links { + font-family: var(--font-mono); + font-size: 14px; + color: var(--color-2nd-fg); +} + /* Article bodies */ -div.header { - color: #555; +#published-at, #updated-at { + color: var(--color-2nd-fg); font-size: 14px; font-style: italic; } blockquote { font-style: italic; - color: dimgrey; + color: var(--color-2nd-fg); padding-left: 10px; - border-left: 3px solid #ccc; + border-left: 3px solid var(--color-2nd-fg); } -ul.no-style { +.collection-list { list-style-type: none; } -ul.no-style li { +.collection-list li { margin: 20px 0px; } /* Footer */ -footer { - border-top: solid 2px black; +body > footer { font-size: 14px; - margin-top: 30px; padding: 12px 0px 12px 0px; } -footer li { +body > footer li { list-style-type: none; margin-top: 10px; } -footer li a { +body > footer li a { margin-left: 5px; } -div.post-footer, div.footnotes { - border-top: 0.5px solid #555; +.content-footer, .page-footer, #footnotes { + border-top: 0.5px solid var(--color-2nd-fg); +} + + + +table.tableblock { + border-collapse: collapse; + margin: 16px 0px; +} + +table.tableblock td, table.tableblock th { + text-align: left; + vertical-align: top; + padding: 6px 24px 6px 0px; + border-bottom: 1px solid var(--color-line-soft); +} + +table.tableblock th { + color: var(--color-2nd-fg); + font-size: 14px; +} + +table.tableblock p { + margin: 0px; } @@ -101,14 +236,16 @@ pre.lineno { margin-right: 3px; padding-right: 3px; border-right: 1px solid; - border-color: hsla(0, 0%, 0%, 0.3); + border-color: var(--color-pre-border); text-align: right; user-select: none; } -pre.highlight { - border: 1px solid #ccc; +.highlight, .listingblock .content { + border: 1px solid; + border-color: var(--color-pre-border); border-radius: 10px; + background-color: var(--color-code-bg); } pre { @@ -119,7 +256,8 @@ pre { /* Code block anchors */ a.code-line-anchor { - color: black; + color: var(--color-fg); + user-select: none; text-decoration: none; } @@ -131,7 +269,7 @@ a.code-line-anchor:hover { /* Header anchor */ div.header-anchor { - color: black; + color: var(--color-fg); text-decoration: none; display: block; margin-bottom: 15px; @@ -158,8 +296,83 @@ div.header-anchor:hover img { /* Plaintext code block links */ -div.plaintext-link { +.plaintext { margin: auto auto 0 auto; text-align: right; - font-family: monospace; + font-family: var(--font-mono); +} + +a.anchor { + background: transparent url(img/link.svg) center right no-repeat; + padding-right: 22px; + margin-left: 10px; + visibility: hidden; +} + +h2:hover a.anchor { + visibility: visible; +} + + +/* Fix asciidoc shortcoming */ + +.line-through { + text-decoration: line-through; +} + + +#comments .message { + background-color: var(--color-2nd-bg); + border: 1px solid; + border-color: var(--color-pre-border); + border-radius: 10px; + padding: 0.5rem 1rem; + margin-bottom: 1rem; + margin-left: 9rem; +} + +#comments .message[data-depth="0"] { margin-left: 0; } +#comments .message[data-depth="1"] { margin-left: 1rem; } +#comments .message[data-depth="2"] { margin-left: 2rem; } +#comments .message[data-depth="3"] { margin-left: 3rem; } +#comments .message[data-depth="4"] { margin-left: 4rem; } +#comments .message[data-depth="5"] { margin-left: 5rem; } +#comments .message[data-depth="6"] { margin-left: 6rem; } +#comments .message[data-depth="7"] { margin-left: 7rem; } +#comments .message[data-depth="8"] { margin-left: 8rem; } + +#comments .message-header { + display: flex; + flex-wrap: wrap; + align-items: baseline; + gap: 0 1rem; + font-size: 0.9em; +} + +#comments .from { + font-weight: bold; +} + +#comments .date, #comments .reply, #comments .handle { + color: var(--color-2nd-fg); +} + +#comments .handle { + font-weight: normal; +} + +#comments .message-body { + background-color: transparent; + border: none; + overflow-wrap: anywhere; + font-size: 0.9em; +} + +#comments pre.message-body { white-space: pre-wrap; } +#comments div.message-body p { margin: 0.5em 0; } +#comments div.message-body p:first-child { margin-top: 0; } +#comments div.message-body p:last-child { margin-bottom: 0; } + +@media (max-width: 40rem) { + #comments .message[data-depth] { margin-left: 0; } } diff --git a/src/i18n.txt b/src/i18n.txt new file mode 100644 index 0000000..8a5f029 --- /dev/null +++ b/src/i18n.txt @@ -0,0 +1,52 @@ +# nav + +en/blog/ Blog + +en/til/ TIL + +en/screencast/ Screencasts + +en/pastebin/ Pastebins + +en/slide/ Slides + +en/about.html About + +# names + +categories categories + +category category + +feed Atom feed + +feedicon_alt The Atom feed icon + +tagline Robust software, for everybody. + +prev Previous: + +next Next: + +comment Comment + +about_page about this page + +discussions view discussions + +source view source + +# languages + +de Deutsch + +en English + +eo Esperanto + +es Español + +fr Français + +pt Português + diff --git a/src/symlinks.txt b/src/symlinks.txt new file mode 100644 index 0000000..27c9375 --- /dev/null +++ b/src/symlinks.txt @@ -0,0 +1,346 @@ +# Paths that existed as I changed my mind on where to put them +en/favicon.ico favicon.ico +favicon/favicon.ico favicon.ico +static/favicon.svg img/favicon.svg +css/styles.css style.css +static/style.css style.css +public-key.txt public.asc.txt +atom.xml en/blog/feed.xml +feed.xml en/blog/feed.xml +rss.xml en/blog/feed.xml +feed.atom en/blog/feed.xml +feed.en.atom en/blog/feed.xml +feed.til.en.atom en/til/feed.xml + +# Some I think existed, and where the logs show a few hundred missing hits +images/atom.svg img/atom.svg +images/link.svg img/link.svg +images/lock.svg img/lock.svg +images/envelope.svg img/envelope.svg + +# "Official" redirections +index.html en/index.html +security.txt .well-known/security.txt + +# Current published pages +static/attachments/autoqemu.tar.gz en/screencast/2021/02/07/autoqemu.tar.gz +static/attachments/cargo2nix-demo.tar.gz en/blog/2020/10/05/cargo2nix-demo.tar.gz +static/attachments/cargo2nix.tar.gz en/blog/2020/10/05/cargo2nix.tar.gz +static/attachments/fallible.tar.gz en/blog/2021/02/17/fallible.tar.gz +static/attachments/swift2nix-demo.tar.gz en/blog/2020/10/05/swift2nix-demo.tar.gz +static/attachments/swift2nix.tar.gz en/blog/2020/10/05/swift2nix.tar.gz +static/atom.svg img/atom.svg +static/envelope.svg img/envelope.svg +static/link.svg img/link.svg +static/lock.svg img/lock.svg +static/lord-favicon.ico favicon.ico +static/lord-favicon.png favicon.png +static/lord-favicon.svg img/favicon.svg + +resources/podcasts/2020-12-19-a-test-entry.flac en/podcast/2020/12/19/test-entry.flac +resources/podcasts/2020-12-19-a-test-entry.flac.torrent en/podcast/2020/12/19/test-entry.flac.torrent +resources/podcasts/2020-12-19-a-test-entry.ogg en/podcast/2020/12/19/test-entry.ogg +resources/screencasts/2021-02-07-autoqemu-automate-installation-and-ssh-setup-of-iso-os-images.webm en/screencast/2021/02/07/autoqemu.webm +resources/screencasts/2021-02-07-autoqemu-automate-installation-and-ssh-setup-of-iso-os-images.webm.torrent en/screencast/2021/02/07/autoqemu.webm.torrent + + +til.html en/til/index.html +podcast.en.html en/podcast/index.html +screencasts.en.html en/screencast/index.html +pastebins.en.html en/pastebin/index.html +about.html en/about.html +articles-by-category.html en/blog/categories.html +til-by-category.html en/til/categories.html +podcast-episodes-by-category.html en/podcast/categories.html +screencasts-by-category.html en/screencast/categories.html +pastebins-by-category.html en/pastebin/categories.html + +feed.articles.en.atom en/blog/feed.xml +feed.blog.en.atom en/blog/feed.xml +feed.posts.en.atom en/blog/feed.xml +feed.tils.en.atom en/til/feed.xml +feed.podcasts.en.atom en/podcast/feed.xml +feed.screencasts.en.atom en/screencast/feed.xml +feed.pastebins.en.atom en/pastebin/feed.xml + +feed.articles-by-category.en.article-review.atom en/blog/feed.article-review.xml +feed.articles-by-category.en.mediator.atom en/blog/feed.mediator.xml +feed.articles-by-category.en.presentation.atom en/blog/feed.presentation.xml +feed.articles-by-category.en.video-review.atom en/blog/feed.video-review.xml +feed.pastebins-by-category.en.guix.atom en/pastebin/feed.guix.xml +feed.pastebins-by-category.en.nix.atom en/pastebin/feed.nix.xml +feed.tils-by-category.en.ci.atom en/til/feed.ci.xml +feed.tils-by-category.en.git.atom en/til/feed.git.xml +feed.tils-by-category.en.shell.atom en/til/feed.shell.xml + + +# Legacy article permalinks and their code sidecars, from the +# pre-rewrite URL scheme. Sidecars are aligned in document +# order, not by index: three articles gained or lost a block +# in the migration, so the numbering drifts. +2021/04/29/a-relational-model-of-data-for-large-shared-data-banks-article-review.html en/blog/2021/04/29/relational-review.html +2021/04/29/a-relational-model-of-data-for-large-shared-data-banks-article-review.html.0.txt en/blog/2021/04/29/relational-review.html.0.txt +2021/04/29/a-relational-model-of-data-for-large-shared-data-banks-article-review.html.1.txt en/blog/2021/04/29/relational-review.html.1.txt +2021/04/29/a-relational-model-of-data-for-large-shared-data-banks-article-review.html.2.txt en/blog/2021/04/29/relational-review.html.2.txt +podcast/2020/12/19/a-test-entry.html en/podcast/2020/12/19/test-entry.html +podcast/2020/12/19/a-test-entry.html.0.txt en/podcast/2020/12/19/test-entry.html.0.txt +2021/02/17/ann-fallible-fault-injection-library-for-stress-testing-failure-scenarios.html en/blog/2021/02/17/fallible.html +2021/02/17/ann-fallible-fault-injection-library-for-stress-testing-failure-scenarios.html.0.txt en/blog/2021/02/17/fallible.html.0.txt +2021/02/17/ann-fallible-fault-injection-library-for-stress-testing-failure-scenarios.html.1.txt en/blog/2021/02/17/fallible.html.1.txt +2021/02/17/ann-fallible-fault-injection-library-for-stress-testing-failure-scenarios.html.2.txt en/blog/2021/02/17/fallible.html.2.txt +2021/02/17/ann-fallible-fault-injection-library-for-stress-testing-failure-scenarios.html.3.txt en/blog/2021/02/17/fallible.html.3.txt +2021/02/17/ann-fallible-fault-injection-library-for-stress-testing-failure-scenarios.html.4.txt en/blog/2021/02/17/fallible.html.4.txt +2021/02/17/ann-fallible-fault-injection-library-for-stress-testing-failure-scenarios.html.5.txt en/blog/2021/02/17/fallible.html.5.txt +2021/02/17/ann-fallible-fault-injection-library-for-stress-testing-failure-scenarios.html.6.txt en/blog/2021/02/17/fallible.html.6.txt +2021/01/26/ann-remembering-add-memory-to-dmenu-fzf-and-similar-tools.html en/blog/2021/01/26/remembering-ann.html +2021/01/26/ann-remembering-add-memory-to-dmenu-fzf-and-similar-tools.html.0.txt en/blog/2021/01/26/remembering-ann.html.0.txt +2021/01/26/ann-remembering-add-memory-to-dmenu-fzf-and-similar-tools.html.1.txt en/blog/2021/01/26/remembering-ann.html.1.txt +2021/01/26/ann-remembering-add-memory-to-dmenu-fzf-and-similar-tools.html.2.txt en/blog/2021/01/26/remembering-ann.html.2.txt +2021/01/26/ann-remembering-add-memory-to-dmenu-fzf-and-similar-tools.html.3.txt en/blog/2021/01/26/remembering-ann.html.4.txt +2021/01/26/ann-remembering-add-memory-to-dmenu-fzf-and-similar-tools.html.4.txt en/blog/2021/01/26/remembering-ann.html.5.txt +2021/01/26/ann-remembering-add-memory-to-dmenu-fzf-and-similar-tools.html.5.txt en/blog/2021/01/26/remembering-ann.html.6.txt +2021/01/26/ann-remembering-add-memory-to-dmenu-fzf-and-similar-tools.html.6.txt en/blog/2021/01/26/remembering-ann.html.7.txt +2021/01/26/ann-remembering-add-memory-to-dmenu-fzf-and-similar-tools.html.7.txt en/blog/2021/01/26/remembering-ann.html.8.txt +til/2020/08/13/anchor-headers-and-code-lines-in-jekyll.html en/til/2020/08/13/code-jekyll.html +til/2020/08/13/anchor-headers-and-code-lines-in-jekyll.html.0.txt en/til/2020/08/13/code-jekyll.html.0.txt +til/2020/08/13/anchor-headers-and-code-lines-in-jekyll.html.1.txt en/til/2020/08/13/code-jekyll.html.1.txt +til/2020/08/13/anchor-headers-and-code-lines-in-jekyll.html.2.txt en/til/2020/08/13/code-jekyll.html.2.txt +til/2020/08/13/anchor-headers-and-code-lines-in-jekyll.html.3.txt en/til/2020/08/13/code-jekyll.html.3.txt +til/2020/08/13/anchor-headers-and-code-lines-in-jekyll.html.4.txt en/til/2020/08/13/code-jekyll.html.4.txt +til/2020/08/13/anchor-headers-and-code-lines-in-jekyll.html.5.txt en/til/2020/08/13/code-jekyll.html.5.txt +til/2020/08/13/anchor-headers-and-code-lines-in-jekyll.html.6.txt en/til/2020/08/13/code-jekyll.html.6.txt +screencast/2021/02/07/autoqemu-automate-installation-and-ssh-setup-of-iso-os-images.html en/screencast/2021/02/07/autoqemu.html +screencast/2021/02/07/autoqemu-automate-installation-and-ssh-setup-of-iso-os-images.html.0.txt en/screencast/2021/02/07/autoqemu.html.0.txt +til/2020/12/15/awk-snippet-shellcheck-all-scripts-in-a-repository.html en/til/2020/12/15/shellcheck-repo.html +til/2020/12/15/awk-snippet-shellcheck-all-scripts-in-a-repository.html.0.txt en/til/2020/12/15/shellcheck-repo.html.0.txt +til/2020/12/15/awk-snippet-shellcheck-all-scripts-in-a-repository.html.1.txt en/til/2020/12/15/shellcheck-repo.html.1.txt +til/2020/12/15/awk-snippet-shellcheck-all-scripts-in-a-repository.html.2.txt en/til/2020/12/15/shellcheck-repo.html.2.txt +til/2020/12/15/awk-snippet-shellcheck-all-scripts-in-a-repository.html.3.txt en/til/2020/12/15/shellcheck-repo.html.3.txt +til/2020/12/15/awk-snippet-shellcheck-all-scripts-in-a-repository.html.4.txt en/til/2020/12/15/shellcheck-repo.html.4.txt +til/2020/12/15/awk-snippet-shellcheck-all-scripts-in-a-repository.html.5.txt en/til/2020/12/15/shellcheck-repo.html.5.txt +til/2020/12/15/awk-snippet-shellcheck-all-scripts-in-a-repository.html.6.txt en/til/2020/12/15/shellcheck-repo.html.6.txt +til/2021/01/12/awk-snippet-send-email-to-multiple-recipients-with-curl.html en/til/2021/01/12/curl-awk-emails.html +til/2021/01/12/awk-snippet-send-email-to-multiple-recipients-with-curl.html.0.txt en/til/2021/01/12/curl-awk-emails.html.0.txt +til/2021/01/12/awk-snippet-send-email-to-multiple-recipients-with-curl.html.1.txt en/til/2021/01/12/curl-awk-emails.html.1.txt +til/2021/01/12/awk-snippet-send-email-to-multiple-recipients-with-curl.html.2.txt en/til/2021/01/12/curl-awk-emails.html.2.txt +til/2021/01/12/awk-snippet-send-email-to-multiple-recipients-with-curl.html.3.txt en/til/2021/01/12/curl-awk-emails.html.3.txt +til/2021/01/12/awk-snippet-send-email-to-multiple-recipients-with-curl.html.4.txt en/til/2021/01/12/curl-awk-emails.html.4.txt +til/2020/08/14/browse-a-git-repository-at-a-specific-commit.html en/til/2020/08/14/browse-git.html +til/2020/08/14/browse-a-git-repository-at-a-specific-commit.html.0.txt en/til/2020/08/14/browse-git.html.0.txt +til/2020/08/14/browse-a-git-repository-at-a-specific-commit.html.1.txt en/til/2020/08/14/browse-git.html.1.txt +til/2020/08/14/browse-a-git-repository-at-a-specific-commit.html.2.txt en/til/2020/08/14/browse-git.html.3.txt +til/2020/08/14/browse-a-git-repository-at-a-specific-commit.html.3.txt en/til/2020/08/14/browse-git.html.4.txt +til/2020/08/14/browse-a-git-repository-at-a-specific-commit.html.4.txt en/til/2020/08/14/browse-git.html.5.txt +til/2021/04/24/clojure-auto-curry.html en/til/2021/04/24/clojure-autocurry.html +til/2021/04/24/clojure-auto-curry.html.0.txt en/til/2021/04/24/clojure-autocurry.html.0.txt +til/2021/04/24/clojure-auto-curry.html.1.txt en/til/2021/04/24/clojure-autocurry.html.1.txt +til/2021/04/24/clojure-auto-curry.html.2.txt en/til/2021/04/24/clojure-autocurry.html.2.txt +til/2021/04/24/clojure-auto-curry.html.3.txt en/til/2021/04/24/clojure-autocurry.html.3.txt +til/2021/04/24/common-lisp-argument-precedence-order-parameterization-of-a-generic-function.html en/til/2021/04/24/cl-generic-precedence.html +til/2021/04/24/common-lisp-argument-precedence-order-parameterization-of-a-generic-function.html.0.txt en/til/2021/04/24/cl-generic-precedence.html.0.txt +til/2021/04/24/common-lisp-argument-precedence-order-parameterization-of-a-generic-function.html.1.txt en/til/2021/04/24/cl-generic-precedence.html.1.txt +til/2021/04/24/common-lisp-argument-precedence-order-parameterization-of-a-generic-function.html.2.txt en/til/2021/04/24/cl-generic-precedence.html.2.txt +til/2021/04/24/common-lisp-argument-precedence-order-parameterization-of-a-generic-function.html.3.txt en/til/2021/04/24/cl-generic-precedence.html.3.txt +til/2021/04/24/common-lisp-argument-precedence-order-parameterization-of-a-generic-function.html.4.txt en/til/2021/04/24/cl-generic-precedence.html.4.txt +2020/11/07/diy-an-offline-bug-tracker-with-text-files-git-and-email.html en/blog/2020/11/07/diy-bugs.html +til/2020/11/12/diy-bare-bones-ci-server-with-bash-and-nix.html en/til/2020/11/12/diy-nix-bash-ci.html +til/2020/11/12/diy-bare-bones-ci-server-with-bash-and-nix.html.0.txt en/til/2020/11/12/diy-nix-bash-ci.html.0.txt +pastebin/2021/06/08/debit-reading-session-sicp-solutions-pt-1.html en/pastebin/2021/06/08/reading-session-pt1.html +pastebin/2021/06/08/debit-reading-session-sicp-solutions-pt-1.html.0.txt en/pastebin/2021/06/08/reading-session-pt1.html.0.txt +2020/11/12/durable-persistent-trees-and-parser-combinators-building-a-database.html en/blog/2020/11/12/database-parsers-trees.html +2020/11/12/durable-persistent-trees-and-parser-combinators-building-a-database.html.0.txt en/blog/2020/11/12/database-parsers-trees.html.0.txt +2020/11/12/durable-persistent-trees-and-parser-combinators-building-a-database.html.1.txt en/blog/2020/11/12/database-parsers-trees.html.1.txt +til/2021/08/11/encoding-and-decoding-javascript-bigint-values-with-reviver.html en/til/2021/08/11/js-bigint-reviver.html +til/2021/08/11/encoding-and-decoding-javascript-bigint-values-with-reviver.html.0.txt en/til/2021/08/11/js-bigint-reviver.html.0.txt +til/2021/08/11/encoding-and-decoding-javascript-bigint-values-with-reviver.html.1.txt en/til/2021/08/11/js-bigint-reviver.html.1.txt +til/2021/08/11/encoding-and-decoding-javascript-bigint-values-with-reviver.html.2.txt en/til/2021/08/11/js-bigint-reviver.html.2.txt +til/2021/08/11/encoding-and-decoding-javascript-bigint-values-with-reviver.html.3.txt en/til/2021/08/11/js-bigint-reviver.html.3.txt +pastebin/2020/01/04/failure-on-guix-tex-live-importer.html en/pastebin/2020/01/04/guix-import-failure.html +pastebin/2020/01/04/failure-on-guix-tex-live-importer.html.0.txt en/pastebin/2020/01/04/guix-import-failure.html.0.txt +pastebin/2020/12/15/failure-with-relocatable-guix-pack-tarball.html en/pastebin/2020/12/15/guix-pack-fail.html +pastebin/2020/12/15/failure-with-relocatable-guix-pack-tarball.html.0.txt en/pastebin/2020/12/15/guix-pack-fail.html.0.txt +pastebin/2020/12/15/failure-with-relocatable-guix-pack-tarball.html.1.txt en/pastebin/2020/12/15/guix-pack-fail.html.1.txt +2020/10/19/feature-flags-differences-between-backend-frontend-and-mobile.html en/blog/2020/10/19/feature-flags.html +2020/10/19/feature-flags-differences-between-backend-frontend-and-mobile.html.0.txt en/blog/2020/10/19/feature-flags.html.0.txt +2020/10/19/feature-flags-differences-between-backend-frontend-and-mobile.html.1.txt en/blog/2020/10/19/feature-flags.html.1.txt +til/2020/11/08/find-broken-symlinks-with-find.html en/til/2020/11/08/find-broken-symlink.html +til/2020/11/08/find-broken-symlinks-with-find.html.0.txt en/til/2020/11/08/find-broken-symlink.html.0.txt +til/2020/11/08/find-broken-symlinks-with-find.html.1.txt en/til/2020/11/08/find-broken-symlink.html.1.txt +pastebin/2023/07/22/funcallable-amop.html en/pastebin/2023/07/22/funcallable-amop.html +pastebin/2023/07/22/funcallable-amop.html.0.txt en/pastebin/2023/07/22/funcallable-amop.html.0.txt +pastebin/2018/07/13/gnu-guix-systemd-daemon-for-nixos.html en/pastebin/2018/07/13/guix-nixos-systemd.html +pastebin/2018/07/13/gnu-guix-systemd-daemon-for-nixos.html.0.txt en/pastebin/2018/07/13/guix-nixos-systemd.html.0.txt +til/2021/07/23/gpg-verification-of-git-repositories-without-tls.html en/til/2021/07/23/git-tls-gpg.html +til/2021/07/23/gpg-verification-of-git-repositories-without-tls.html.0.txt en/til/2021/07/23/git-tls-gpg.html.0.txt +til/2020/11/12/git-bisect-automation.html en/til/2020/11/12/git-bisect-automation.html +til/2020/11/12/git-bisect-automation.html.0.txt en/til/2020/11/12/git-bisect-automation.html.0.txt +til/2020/08/28/grep-online-repositories.html en/til/2020/08/28/grep-online.html +til/2020/08/28/grep-online-repositories.html.0.txt en/til/2020/08/28/grep-online.html.0.txt +til/2020/08/28/grep-online-repositories.html.1.txt en/til/2020/08/28/grep-online.html.1.txt +til/2020/08/28/grep-online-repositories.html.2.txt en/til/2020/08/28/grep-online.html.2.txt +pastebin/2020/11/27/guix-build-local-module.html en/pastebin/2020/11/27/guix-build-local.html +pastebin/2020/11/27/guix-build-local-module.html.0.txt en/pastebin/2020/11/27/guix-build-local.html.0.txt +pastebin/2020/11/27/guix-build-local-module.html.1.txt en/pastebin/2020/11/27/guix-build-local.html.1.txt +pastebin/2020/11/27/guix-build-local-module.html.2.txt en/pastebin/2020/11/27/guix-build-local.html.2.txt +pastebin/2018/07/13/guix-builder-user-creation-commands.html en/pastebin/2018/07/13/guixbuilder.html +pastebin/2018/07/13/guix-builder-user-creation-commands.html.0.txt en/pastebin/2018/07/13/guixbuilder.html.0.txt +2020/08/10/guix-inside-sourcehut-builds-sr-ht-ci.html en/blog/2020/08/10/guix-srht.html +2020/08/10/guix-inside-sourcehut-builds-sr-ht-ci.html.0.txt en/blog/2020/08/10/guix-srht.html.0.txt +2020/08/10/guix-inside-sourcehut-builds-sr-ht-ci.html.1.txt en/blog/2020/08/10/guix-srht.html.1.txt +2020/08/10/guix-inside-sourcehut-builds-sr-ht-ci.html.2.txt en/blog/2020/08/10/guix-srht.html.2.txt +pastebin/2020/02/14/guix-shebang.html en/pastebin/2020/02/14/guix-shebang.html +pastebin/2020/02/14/guix-shebang.html.0.txt en/pastebin/2020/02/14/guix-shebang.html.0.txt +pastebin/2018/07/13/guix-users-in-nixos-system-configuration.html en/pastebin/2018/07/13/guixbuilder-nixos.html +pastebin/2018/07/13/guix-users-in-nixos-system-configuration.html.0.txt en/pastebin/2018/07/13/guixbuilder-nixos.html.0.txt +2020/10/20/how-not-to-interview-engineers.html en/blog/2020/10/20/wrong-interviewing.html +pastebin/2019/06/08/inconsistent-hash-of-buildgomodule.html en/pastebin/2019/06/08/inconsistent-hash.html +pastebin/2019/06/08/inconsistent-hash-of-buildgomodule.html.0.txt en/pastebin/2019/06/08/inconsistent-hash.html.0.txt +pastebin/2019/06/08/inconsistent-hash-of-buildgomodule.html.1.txt en/pastebin/2019/06/08/inconsistent-hash.html.1.txt +pastebin/2019/06/08/inconsistent-hash-of-buildgomodule.html.2.txt en/pastebin/2019/06/08/inconsistent-hash.html.2.txt +pastebin/2021/04/03/javascript-naive-slugify.html en/pastebin/2021/04/03/naive-slugify-js.html +pastebin/2021/04/03/javascript-naive-slugify.html.0.txt en/pastebin/2021/04/03/naive-slugify-js.html.0.txt +pastebin/2018/07/25/nix-exps.html en/pastebin/2018/07/25/nix-exps.html +pastebin/2018/07/25/nix-exps.html.0.txt en/pastebin/2018/07/25/nix-exps.html.0.txt +pastebin/2018/07/11/nix-pinning.html en/pastebin/2018/07/11/nix-pinning.html +pastebin/2018/07/11/nix-pinning.html.0.txt en/pastebin/2018/07/11/nix-pinning.html.0.txt +pastebin/2018/07/13/nix-string-padding.html en/pastebin/2018/07/13/nix-strpad.html +pastebin/2018/07/13/nix-string-padding.html.0.txt en/pastebin/2018/07/13/nix-strpad.html.0.txt +til/2021/01/17/posix-sh-and-shebangs.html en/til/2021/01/17/posix-shebang.html +til/2021/01/17/posix-sh-and-shebangs.html.0.txt en/til/2021/01/17/posix-shebang.html.0.txt +til/2021/01/17/posix-sh-and-shebangs.html.1.txt en/til/2021/01/17/posix-shebang.html.1.txt +til/2021/01/17/posix-sh-and-shebangs.html.2.txt en/til/2021/01/17/posix-shebang.html.2.txt +til/2020/09/05/pull-requests-with-git-the-old-school-way.html en/til/2020/09/05/oldschool-pr.html +til/2020/09/05/pull-requests-with-git-the-old-school-way.html.0.txt en/til/2020/09/05/oldschool-pr.html.0.txt +til/2020/09/05/pull-requests-with-git-the-old-school-way.html.1.txt en/til/2020/09/05/oldschool-pr.html.1.txt +til/2020/09/05/pull-requests-with-git-the-old-school-way.html.2.txt en/til/2020/09/05/oldschool-pr.html.2.txt +pastebin/2016/04/05/rpn-macro-setup.html en/pastebin/2016/04/05/rpn.html +pastebin/2016/04/05/rpn-macro-setup.html.0.txt en/pastebin/2016/04/05/rpn.html.0.txt +pastebin/2019/12/29/raku-tuple-type-annotation.html en/pastebin/2019/12/29/raku-tuple-type.html +pastebin/2019/12/29/raku-tuple-type-annotation.html.0.txt en/pastebin/2019/12/29/raku-tuple-type.html.0.txt +pastebin/2019/12/29/raku-tuple-type-annotation.html.1.txt en/pastebin/2019/12/29/raku-tuple-type.html.1.txt +2018/07/17/running-guix-on-nixos.html en/blog/2018/07/17/guix-nixos.html +2018/07/17/running-guix-on-nixos.html.0.txt en/blog/2018/07/17/guix-nixos.html.0.txt +2018/07/17/running-guix-on-nixos.html.1.txt en/blog/2018/07/17/guix-nixos.html.1.txt +2018/07/17/running-guix-on-nixos.html.2.txt en/blog/2018/07/17/guix-nixos.html.2.txt +2018/07/17/running-guix-on-nixos.html.3.txt en/blog/2018/07/17/guix-nixos.html.3.txt +2018/07/17/running-guix-on-nixos.html.4.txt en/blog/2018/07/17/guix-nixos.html.4.txt +2018/07/17/running-guix-on-nixos.html.5.txt en/blog/2018/07/17/guix-nixos.html.5.txt +pastebin/2021/09/02/sicp-exercise-3-19.html en/pastebin/2021/09/02/sicp-3-19.html +pastebin/2021/09/02/sicp-exercise-3-19.html.0.txt en/pastebin/2021/09/02/sicp-3-19.html.0.txt +pastebin/2021/09/02/sicp-exercise-3-19.html.1.txt en/pastebin/2021/09/02/sicp-3-19.html.1.txt +pastebin/2021/09/03/sicp-persistent-amortized-o1-queue.html en/pastebin/2021/09/03/sicp-persistent-queue.html +pastebin/2021/09/03/sicp-persistent-amortized-o1-queue.html.0.txt en/pastebin/2021/09/03/sicp-persistent-queue.html.0.txt +pastebin/2021/09/03/sicp-persistent-amortized-o1-queue.html.1.txt en/pastebin/2021/09/03/sicp-persistent-queue.html.1.txt +til/2020/12/29/svg-favicon.html en/til/2020/12/29/svg.html +til/2020/12/29/svg-favicon.html.0.txt en/til/2020/12/29/svg.html.0.txt +til/2020/12/29/svg-favicon.html.1.txt en/til/2020/12/29/svg.html.1.txt +til/2020/12/29/svg-favicon.html.2.txt en/til/2020/12/29/svg.html.2.txt +til/2020/10/11/search-changes-to-a-filename-pattern-in-git-history.html en/til/2020/10/11/search-git-history.html +til/2020/10/11/search-changes-to-a-filename-pattern-in-git-history.html.0.txt en/til/2020/10/11/search-git-history.html.0.txt +til/2020/10/11/search-changes-to-a-filename-pattern-in-git-history.html.1.txt en/til/2020/10/11/search-git-history.html.1.txt +til/2020/08/16/search-in-git.html en/til/2020/08/16/git-search.html +til/2020/08/16/search-in-git.html.0.txt en/til/2020/08/16/git-search.html.0.txt +til/2020/08/16/search-in-git.html.1.txt en/til/2020/08/16/git-search.html.1.txt +til/2020/08/16/search-in-git.html.2.txt en/til/2020/08/16/git-search.html.2.txt +til/2020/09/04/send-emails-using-the-command-line-for-fun-and-profit.html en/til/2020/09/04/cli-email-fun-profit.html +til/2020/09/04/send-emails-using-the-command-line-for-fun-and-profit.html.0.txt en/til/2020/09/04/cli-email-fun-profit.html.0.txt +til/2020/09/04/send-emails-using-the-command-line-for-fun-and-profit.html.1.txt en/til/2020/09/04/cli-email-fun-profit.html.1.txt +til/2020/09/04/send-emails-using-the-command-line-for-fun-and-profit.html.2.txt en/til/2020/09/04/cli-email-fun-profit.html.2.txt +til/2020/09/04/send-emails-using-the-command-line-for-fun-and-profit.html.3.txt en/til/2020/09/04/cli-email-fun-profit.html.3.txt +til/2020/08/12/simple-filename-timestamp.html en/til/2020/08/12/filename-timestamp.html +til/2020/08/12/simple-filename-timestamp.html.0.txt en/til/2020/08/12/filename-timestamp.html.0.txt +til/2020/08/12/simple-filename-timestamp.html.1.txt en/til/2020/08/12/filename-timestamp.html.1.txt +pastebin/2021/08/11/spaces-around-h1-tags.html en/pastebin/2021/08/11/h1-spacing.html +pastebin/2021/08/11/spaces-around-h1-tags.html.0.txt en/pastebin/2021/08/11/h1-spacing.html.0.txt +pastebin/2021/08/11/spaces-around-h1-tags.html.1.txt en/pastebin/2021/08/11/h1-spacing.html.1.txt +til/2020/11/30/storing-ci-data-on-git-notes.html en/til/2020/11/30/git-notes-ci.html +til/2020/11/30/storing-ci-data-on-git-notes.html.0.txt en/til/2020/11/30/git-notes-ci.html.0.txt +til/2020/11/30/storing-ci-data-on-git-notes.html.1.txt en/til/2020/11/30/git-notes-ci.html.1.txt +til/2020/11/30/storing-ci-data-on-git-notes.html.2.txt en/til/2020/11/30/git-notes-ci.html.2.txt +2020/11/08/the-next-paradigm-shift-in-programming-video-review.html en/blog/2020/11/08/paradigm-shift-review.html +2020/08/31/the-database-i-wish-i-had.html en/blog/2020/08/31/database-i-wish-i-had.html +til/2021/04/24/three-way-conditional-for-number-signs-on-lisp.html en/til/2021/04/24/scm-nif.html +til/2021/04/24/three-way-conditional-for-number-signs-on-lisp.html.0.txt en/til/2021/04/24/scm-nif.html.0.txt +til/2021/04/24/three-way-conditional-for-number-signs-on-lisp.html.1.txt en/til/2021/04/24/scm-nif.html.1.txt +til/2021/04/24/three-way-conditional-for-number-signs-on-lisp.html.2.txt en/til/2021/04/24/scm-nif.html.2.txt +til/2020/11/12/useful-bash-variables.html en/til/2020/11/12/useful-bashvars.html +til/2020/11/12/useful-bash-variables.html.0.txt en/til/2020/11/12/useful-bashvars.html.0.txt +til/2020/11/12/useful-bash-variables.html.1.txt en/til/2020/11/12/useful-bashvars.html.1.txt +til/2020/11/12/useful-bash-variables.html.2.txt en/til/2020/11/12/useful-bashvars.html.2.txt +2018/12/21/using-youtube-dl-to-manage-youtube-subscriptions.html en/blog/2018/12/21/ytdl-subs.html +2018/12/21/using-youtube-dl-to-manage-youtube-subscriptions.html.0.txt en/blog/2018/12/21/ytdl-subs.html.0.txt +2018/12/21/using-youtube-dl-to-manage-youtube-subscriptions.html.1.txt en/blog/2018/12/21/ytdl-subs.html.1.txt +2018/12/21/using-youtube-dl-to-manage-youtube-subscriptions.html.2.txt en/blog/2018/12/21/ytdl-subs.html.2.txt +2018/12/21/using-youtube-dl-to-manage-youtube-subscriptions.html.3.txt en/blog/2018/12/21/ytdl-subs.html.3.txt +2018/12/21/using-youtube-dl-to-manage-youtube-subscriptions.html.4.txt en/blog/2018/12/21/ytdl-subs.html.4.txt +2019/06/02/using-nixos-as-an-stateless-workstation.html en/blog/2019/06/02/nixos-stateless-workstation.html +2019/06/02/using-nixos-as-an-stateless-workstation.html.0.txt en/blog/2019/06/02/nixos-stateless-workstation.html.0.txt +2018/08/01/verifying-npm-ci-reproducibility.html en/blog/2018/08/01/npm-ci-reproducibility.html +2018/08/01/verifying-npm-ci-reproducibility.html.0.txt en/blog/2018/08/01/npm-ci-reproducibility.html.0.txt +2018/08/01/verifying-npm-ci-reproducibility.html.1.txt en/blog/2018/08/01/npm-ci-reproducibility.html.1.txt +2018/08/01/verifying-npm-ci-reproducibility.html.2.txt en/blog/2018/08/01/npm-ci-reproducibility.html.2.txt +2020/10/05/cargo2nix-dramatically-simpler-rust-in-nix.html en/blog/2020/10/05/cargo2nix.html +2020/10/05/cargo2nix-dramatically-simpler-rust-in-nix.html.0.txt en/blog/2020/10/05/cargo2nix.html.0.txt +2020/10/05/cargo2nix-dramatically-simpler-rust-in-nix.html.1.txt en/blog/2020/10/05/cargo2nix.html.1.txt +til/2020/11/14/gpodder-as-a-media-subscription-manager.html en/til/2020/11/14/gpodder-media.html +pastebin/2022/07/14/git-cleanup-command.html en/pastebin/2022/07/14/git-cleanup.html +pastebin/2022/07/14/git-cleanup-command.html.0.txt en/pastebin/2022/07/14/git-cleanup.html.0.txt +pastebin/2018/07/25/nix-show-derivation-sample-output.html en/pastebin/2018/07/25/nix-showdrv.html +pastebin/2018/07/25/nix-show-derivation-sample-output.html.0.txt en/pastebin/2018/07/25/nix-showdrv.html.0.txt +2020/10/05/swift2nix-run-swift-inside-nix-builds.html en/blog/2020/10/05/swift2nix.html +2020/10/05/swift2nix-run-swift-inside-nix-builds.html.0.txt en/blog/2020/10/05/swift2nix.html.0.txt +2020/10/05/swift2nix-run-swift-inside-nix-builds.html.1.txt en/blog/2020/10/05/swift2nix.html.1.txt +2020/10/05/swift2nix-run-swift-inside-nix-builds.html.2.txt en/blog/2020/10/05/swift2nix.html.2.txt +2020/11/14/local-first-software-you-own-your-data-in-spite-of-the-cloud-article-review.html en/blog/2020/11/14/local-first-review.html +2020/11/14/local-first-software-you-own-your-data-in-spite-of-the-cloud-article-review.html.0.txt en/blog/2020/11/14/local-first-review.html.0.txt +2020/11/14/local-first-software-you-own-your-data-in-spite-of-the-cloud-article-review.html.1.txt en/blog/2020/11/14/local-first-review.html.1.txt +2020/11/14/local-first-software-you-own-your-data-in-spite-of-the-cloud-article-review.html.2.txt en/blog/2020/11/14/local-first-review.html.2.txt +pastebin/2021/06/22/cloc-curl-and-wget.html en/pastebin/2021/06/22/curl-wget.html +pastebin/2021/06/22/cloc-curl-and-wget.html.0.txt en/pastebin/2021/06/22/curl-wget.html.0.txt +pastebin/2021/06/22/cloc-curl-and-wget.html.1.txt en/pastebin/2021/06/22/curl-wget.html.1.txt +til/2020/08/12/nome-de-arquivo-com-timestamp-simplificado.html pt/hea/2020/08/12/arquivo-datado.html +til/2020/08/12/nome-de-arquivo-com-timestamp-simplificado.html.0.txt pt/hea/2020/08/12/arquivo-datado.html.0.txt +til/2020/08/12/nome-de-arquivo-com-timestamp-simplificado.html.1.txt pt/hea/2020/08/12/arquivo-datado.html.1.txt + +# Asset spellings the old site used +styles.css style.css +public.asc public.asc.txt + + +# Non-English pages at their pre-rewrite slugs +sobre.html pt/sobre.html +a-propos.html fr/a-propos.html +pri.html eo/pri.html +hea.html pt/hea/index.html +hea-por-categoria.html pt/hea/categorias.html + +# Non-English feeds +feed.tils.pt.atom pt/hea/feed.xml +feed.tils-by-category.pt.shell.atom pt/hea/feed.shell.xml + +# Media at their pre-rewrite paths +podcast/2020/12/19/a-test-entry.ogg en/podcast/2020/12/19/test-entry.ogg +podcast/2020/12/19/a-test-entry.ogg.torrent en/podcast/2020/12/19/test-entry.ogg.torrent +screencast/2021/02/07/autoqemu-automate-installation-and-ssh-setup-of-iso-os-images.webm en/screencast/2021/02/07/autoqemu.webm +screencast/2021/02/07/autoqemu-automate-installation-and-ssh-setup-of-iso-os-images.webm.torrent en/screencast/2021/02/07/autoqemu.webm.torrent + +# Pre-compressed twins the old site served + +# Slide listings at their pre-rewrite slugs. The decks themselves +# are PDFs; their pages link to them. +slides.html en/slide/index.html +slides-by-category.html en/slide/categories.html +slides/2020/10/19/rollout-feature-flag-experiment-operational-toggle.html en/slide/2020/10/19/feature-flags.html +slides/2020/11/14/on-local-first-beyond-the-crdt-silver-bullet.html en/slide/2020/11/14/local-first-hype.html +feed.slides.en.atom en/slide/feed.xml +2019/06/02/stateless-os.html en/blog/2019/06/02/nixos-stateless-workstation.html +2019/06/02/stateless-os.html.0.txt en/blog/2019/06/02/nixos-stateless-workstation.html.0.txt +pastebin/raku-tuple-type-annotation.html en/pastebin/2019/12/29/raku-tuple-type.html +pastebin/raku-tuple-type-annotation.html.0.txt en/pastebin/2019/12/29/raku-tuple-type.html.0.txt +pastebin/raku-tuple-type-annotation.html.1.txt en/pastebin/2019/12/29/raku-tuple-type.html.1.txt +pastebin/inconsistent-hash-of-buildgomodule.html en/pastebin/2019/06/08/inconsistent-hash.html +pastebin/inconsistent-hash-of-buildgomodule.html.0.txt en/pastebin/2019/06/08/inconsistent-hash.html.0.txt +pastebin/inconsistent-hash-of-buildgomodule.html.1.txt en/pastebin/2019/06/08/inconsistent-hash.html.1.txt +pastebin/inconsistent-hash-of-buildgomodule.html.2.txt en/pastebin/2019/06/08/inconsistent-hash.html.2.txt +2021/01/26/ann-remembering-add-memory-to-dmenu-fzf-and-similar-tools.html.8.txt en/blog/2021/01/26/remembering-ann.html.8.txt +til/2020/08/14/browse-a-git-repository-at-a-specific-commit.html.5.txt en/til/2020/08/14/browse-git.html.5.txt |
