diff options
author | EuAndreh <eu@euandre.org> | 2025-03-31 21:51:40 -0300 |
---|---|---|
committer | EuAndreh <eu@euandre.org> | 2025-03-31 21:51:40 -0300 |
commit | 570ec471d1605318aeefb030cd78682ae442235b (patch) | |
tree | 51e17eabe37c6689f8799b55e6875c3480329a2c /src/content/tils/2020/12/15 | |
parent | Makefile, mkdeps.sh: Derive index.html and feed.xml from more static "sortdat... (diff) | |
download | euandre.org-570ec471d1605318aeefb030cd78682ae442235b.tar.gz euandre.org-570ec471d1605318aeefb030cd78682ae442235b.tar.xz |
src/content/: Update all files left to asciidoc
Diffstat (limited to 'src/content/tils/2020/12/15')
-rw-r--r-- | src/content/tils/2020/12/15/shellcheck-repo.adoc | 90 |
1 files changed, 40 insertions, 50 deletions
diff --git a/src/content/tils/2020/12/15/shellcheck-repo.adoc b/src/content/tils/2020/12/15/shellcheck-repo.adoc index 71d10a3..960812e 100644 --- a/src/content/tils/2020/12/15/shellcheck-repo.adoc +++ b/src/content/tils/2020/12/15/shellcheck-repo.adoc @@ -1,52 +1,41 @@ ---- += Awk snippet: ShellCheck all scripts in a repository -title: 'Awk snippet: ShellCheck all scripts in a repository' +:awk-20-min: https://ferd.ca/awk-in-20-minutes.html +:shellcheck: https://www.shellcheck.net/ -date: 2020-12-15 - -updated_at: 2020-12-16 - -layout: post - -lang: en - -ref: awk-snippet-shellcheck-all-scripts-in-a-repository - -eu_categories: shell - ---- - -Inspired by Fred Herbert's "[Awk in 20 Minutes][awk-20min]", here's a problem I +Inspired by Fred Herbert's "{awk-20-min}[Awk in 20 Minutes]", here's a problem I just solved with a line of Awk: run ShellCheck in all scripts of a repository. In my repositories I usually have Bash and POSIX scripts, which I want to keep -tidy with [ShellCheck][shellcheck]. Here's the first version of +tidy with {shellcheck}[ShellCheck]. Here's the first version of `assert-shellcheck.sh`: -```shell +[source,shell] +---- #!/bin/sh -eux find . -type f -name '*.sh' -print0 | xargs -0 shellcheck -``` +---- This is the type of script that I copy around to all repositories, and I want it to be capable of working on any repository, without requiring a list of files to run ShellCheck on. -This first version worked fine, as all my scripts had the '.sh' ending. But I +This first version worked fine, as all my scripts had the `.sh' ending. But I recently added some scripts without any extension, so `assert-shellcheck.sh` -called for a second version. The first attempt was to try grepping the shebang -line: +called for a second version. The first attempt was to try grepping the shebang line: -```shell +[source,shell] +---- $ grep '^#!/' assert-shellcheck.sh #!/usr/sh -``` +---- -Good, we have a grep pattern on the first try. Let's try to find all the +Good, we have a grep pattern on the first try. Let's try to find all the matching files: -```shell +[source,shell] +---- $ find . -type f | xargs grep -l '^#!/' ./TODOs.org ./.git/hooks/pre-commit.sample @@ -76,7 +65,7 @@ $ find . -type f | xargs grep -l '^#!/' ./scripts/songbooks.in ./scripts/with-container.sh ./scripts/assert-shellcheck.sh -``` +---- This approach has a problem, though: it includes files ignored by Git, such as `builld-aux/install-sh~`, and even goes into the `.git/` directory and finds @@ -84,7 +73,8 @@ sample hooks in `.git/hooks/*`. To list the files that Git is tracking we'll try `git ls-files`: -```shell +[source,shell] +---- $ git ls-files | xargs grep -l '^#!/' TODOs.org bootstrap @@ -99,23 +89,25 @@ scripts/compile-readme.sh scripts/generate-tasks-and-bugs.sh scripts/songbooks.in scripts/with-container.sh -``` +---- It looks to be almost there, but the `TODOs.org` entry shows a flaw in it: grep -is looking for a `'^#!/'` pattern on any part of the file. In my case, +is looking for a +'^#!/'+ pattern on any part of the file. In my case, `TODOs.org` had a snippet in the middle of the file where a line started with -`#!/bin/sh`. ++#!/bin/sh+. -So what we actually want is to match the **first** line against the pattern. We +So what we actually want is to match the *first* line against the pattern. We could loop through each file, get the first line with `head -n 1` and grep -against that, but this is starting to look messy. I bet there is another way of +against that, but this is starting to look messy. I bet there is another way of doing it concisely... -Let's try Awk. I need a way to select the line numbers to replace `head -n 1`, -and to stop processing the file if the pattern matches. A quick search points me -to using `FNR` for the former, and `{ nextline }` for the latter. Let's try it: +Let's try Awk. I need a way to select the line numbers to replace `head -n 1`, +and to stop processing the file if the pattern matches. A quick search points +me to using `FNR` for the former, and `{ nextline }` for the latter. Let's try +it: -```shell +[source,shell] +---- $ git ls-files | xargs awk 'FNR>1 { nextfile } /^#!\// { print FILENAME; nextfile }' bootstrap build-aux/with-guile-env.in @@ -129,43 +121,41 @@ scripts/compile-readme.sh scripts/generate-tasks-and-bugs.sh scripts/songbooks.in scripts/with-container.sh -``` +---- Great! Only `TODOs.org` is missing, but the script is much better: instead of matching against any part of the file that may have a shebang-like line, we only -look for the first. Let's put it back into the `assert-shellcheck.sh` file and +look for the first. Let's put it back into the `assert-shellcheck.sh` file and use `NULL` for separators to accommodate files with spaces in the name: -``` +.... #!/usr/sh -eux git ls-files -z | \ xargs -0 awk 'FNR>1 { nextfile } /^#!\// { print FILENAME; nextfile }' | \ xargs shellcheck -``` +.... This is where I've stopped, but I imagine a likely improvement: match against -only `#!/bin/sh` and `#!/usr/bin/env bash` shebangs (the ones I use most), to +only +#!/bin/sh+ and +#!/usr/bin/env bash+ shebangs (the ones I use most), to avoid running ShellCheck on Perl files, or other shebangs. Also when reviewing the text of this article, I found that `{ nextfile }` is a -GNU Awk extension. It would be an improvement if `assert-shellcheck.sh` relied +GNU Awk extension. It would be an improvement if `assert-shellcheck.sh` relied on the POSIX subset of Awk for working correctly. -## *Update* +== _Update_ After publishing, I could remove `{ nextfile }` and even make the script simpler: -```shell +[source,shell] +---- #!/usr/sh -eux git ls-files -z | \ xargs -0 awk 'FNR==1 && /^#!\// { print FILENAME }' | \ xargs shellcheck -``` +---- Now both the shell and Awk usage are POSIX compatible. - -[awk-20min]: https://ferd.ca/awk-in-20-minutes.html -[shellcheck]: https://www.shellcheck.net/ |