summaryrefslogtreecommitdiff
path: root/src/content/tils/2020/12/15
diff options
context:
space:
mode:
authorEuAndreh <eu@euandre.org>2025-03-31 21:51:40 -0300
committerEuAndreh <eu@euandre.org>2025-03-31 21:51:40 -0300
commit570ec471d1605318aeefb030cd78682ae442235b (patch)
tree51e17eabe37c6689f8799b55e6875c3480329a2c /src/content/tils/2020/12/15
parentMakefile, mkdeps.sh: Derive index.html and feed.xml from more static "sortdat... (diff)
downloadeuandre.org-570ec471d1605318aeefb030cd78682ae442235b.tar.gz
euandre.org-570ec471d1605318aeefb030cd78682ae442235b.tar.xz
src/content/: Update all files left to asciidoc
Diffstat (limited to 'src/content/tils/2020/12/15')
-rw-r--r--src/content/tils/2020/12/15/shellcheck-repo.adoc90
1 files changed, 40 insertions, 50 deletions
diff --git a/src/content/tils/2020/12/15/shellcheck-repo.adoc b/src/content/tils/2020/12/15/shellcheck-repo.adoc
index 71d10a3..960812e 100644
--- a/src/content/tils/2020/12/15/shellcheck-repo.adoc
+++ b/src/content/tils/2020/12/15/shellcheck-repo.adoc
@@ -1,52 +1,41 @@
----
+= Awk snippet: ShellCheck all scripts in a repository
-title: 'Awk snippet: ShellCheck all scripts in a repository'
+:awk-20-min: https://ferd.ca/awk-in-20-minutes.html
+:shellcheck: https://www.shellcheck.net/
-date: 2020-12-15
-
-updated_at: 2020-12-16
-
-layout: post
-
-lang: en
-
-ref: awk-snippet-shellcheck-all-scripts-in-a-repository
-
-eu_categories: shell
-
----
-
-Inspired by Fred Herbert's "[Awk in 20 Minutes][awk-20min]", here's a problem I
+Inspired by Fred Herbert's "{awk-20-min}[Awk in 20 Minutes]", here's a problem I
just solved with a line of Awk: run ShellCheck in all scripts of a repository.
In my repositories I usually have Bash and POSIX scripts, which I want to keep
-tidy with [ShellCheck][shellcheck]. Here's the first version of
+tidy with {shellcheck}[ShellCheck]. Here's the first version of
`assert-shellcheck.sh`:
-```shell
+[source,shell]
+----
#!/bin/sh -eux
find . -type f -name '*.sh' -print0 | xargs -0 shellcheck
-```
+----
This is the type of script that I copy around to all repositories, and I want it
to be capable of working on any repository, without requiring a list of files to
run ShellCheck on.
-This first version worked fine, as all my scripts had the '.sh' ending. But I
+This first version worked fine, as all my scripts had the `.sh' ending. But I
recently added some scripts without any extension, so `assert-shellcheck.sh`
-called for a second version. The first attempt was to try grepping the shebang
-line:
+called for a second version. The first attempt was to try grepping the shebang line:
-```shell
+[source,shell]
+----
$ grep '^#!/' assert-shellcheck.sh
#!/usr/sh
-```
+----
-Good, we have a grep pattern on the first try. Let's try to find all the
+Good, we have a grep pattern on the first try. Let's try to find all the
matching files:
-```shell
+[source,shell]
+----
$ find . -type f | xargs grep -l '^#!/'
./TODOs.org
./.git/hooks/pre-commit.sample
@@ -76,7 +65,7 @@ $ find . -type f | xargs grep -l '^#!/'
./scripts/songbooks.in
./scripts/with-container.sh
./scripts/assert-shellcheck.sh
-```
+----
This approach has a problem, though: it includes files ignored by Git, such as
`builld-aux/install-sh~`, and even goes into the `.git/` directory and finds
@@ -84,7 +73,8 @@ sample hooks in `.git/hooks/*`.
To list the files that Git is tracking we'll try `git ls-files`:
-```shell
+[source,shell]
+----
$ git ls-files | xargs grep -l '^#!/'
TODOs.org
bootstrap
@@ -99,23 +89,25 @@ scripts/compile-readme.sh
scripts/generate-tasks-and-bugs.sh
scripts/songbooks.in
scripts/with-container.sh
-```
+----
It looks to be almost there, but the `TODOs.org` entry shows a flaw in it: grep
-is looking for a `'^#!/'` pattern on any part of the file. In my case,
+is looking for a +'^#!/'+ pattern on any part of the file. In my case,
`TODOs.org` had a snippet in the middle of the file where a line started with
-`#!/bin/sh`.
++#!/bin/sh+.
-So what we actually want is to match the **first** line against the pattern. We
+So what we actually want is to match the *first* line against the pattern. We
could loop through each file, get the first line with `head -n 1` and grep
-against that, but this is starting to look messy. I bet there is another way of
+against that, but this is starting to look messy. I bet there is another way of
doing it concisely...
-Let's try Awk. I need a way to select the line numbers to replace `head -n 1`,
-and to stop processing the file if the pattern matches. A quick search points me
-to using `FNR` for the former, and `{ nextline }` for the latter. Let's try it:
+Let's try Awk. I need a way to select the line numbers to replace `head -n 1`,
+and to stop processing the file if the pattern matches. A quick search points
+me to using `FNR` for the former, and `{ nextline }` for the latter. Let's try
+it:
-```shell
+[source,shell]
+----
$ git ls-files | xargs awk 'FNR>1 { nextfile } /^#!\// { print FILENAME; nextfile }'
bootstrap
build-aux/with-guile-env.in
@@ -129,43 +121,41 @@ scripts/compile-readme.sh
scripts/generate-tasks-and-bugs.sh
scripts/songbooks.in
scripts/with-container.sh
-```
+----
Great! Only `TODOs.org` is missing, but the script is much better: instead of
matching against any part of the file that may have a shebang-like line, we only
-look for the first. Let's put it back into the `assert-shellcheck.sh` file and
+look for the first. Let's put it back into the `assert-shellcheck.sh` file and
use `NULL` for separators to accommodate files with spaces in the name:
-```
+....
#!/usr/sh -eux
git ls-files -z | \
xargs -0 awk 'FNR>1 { nextfile } /^#!\// { print FILENAME; nextfile }' | \
xargs shellcheck
-```
+....
This is where I've stopped, but I imagine a likely improvement: match against
-only `#!/bin/sh` and `#!/usr/bin/env bash` shebangs (the ones I use most), to
+only +#!/bin/sh+ and +#!/usr/bin/env bash+ shebangs (the ones I use most), to
avoid running ShellCheck on Perl files, or other shebangs.
Also when reviewing the text of this article, I found that `{ nextfile }` is a
-GNU Awk extension. It would be an improvement if `assert-shellcheck.sh` relied
+GNU Awk extension. It would be an improvement if `assert-shellcheck.sh` relied
on the POSIX subset of Awk for working correctly.
-## *Update*
+== _Update_
After publishing, I could remove `{ nextfile }` and even make the script
simpler:
-```shell
+[source,shell]
+----
#!/usr/sh -eux
git ls-files -z | \
xargs -0 awk 'FNR==1 && /^#!\// { print FILENAME }' | \
xargs shellcheck
-```
+----
Now both the shell and Awk usage are POSIX compatible.
-
-[awk-20min]: https://ferd.ca/awk-in-20-minutes.html
-[shellcheck]: https://www.shellcheck.net/