Awk snippet: ShellCheck all scripts in a repository

Inspired by Fred Herbert’s "Awk in 20 Minutes", here’s a problem I just solved with a line of Awk: run ShellCheck in all scripts of a repository.

In my repositories I usually have Bash and POSIX scripts, which I want to keep tidy with ShellCheck. Here’s the first version of assert-shellcheck.sh:

This is the type of script that I copy around to all repositories, and I want it to be capable of working on any repository, without requiring a list of files to run ShellCheck on.

This first version worked fine, as all my scripts had the .sh' ending. But I recently added some scripts without any extension, so `assert-shellcheck.sh called for a second version. The first attempt was to try grepping the shebang line:

Good, we have a grep pattern on the first try. Let’s try to find all the matching files:

This approach has a problem, though: it includes files ignored by Git, such as builld-aux/install-sh~, and even goes into the .git/ directory and finds sample hooks in .git/hooks/*.

To list the files that Git is tracking we’ll try git ls-files:

It looks to be almost there, but the TODOs.org entry shows a flaw in it: grep is looking for a '^#!/' pattern on any part of the file. In my case, TODOs.org had a snippet in the middle of the file where a line started with #!/bin/sh.

So what we actually want is to match the first line against the pattern. We could loop through each file, get the first line with head -n 1 and grep against that, but this is starting to look messy. I bet there is another way of doing it concisely…​

Let’s try Awk. I need a way to select the line numbers to replace head -n 1, and to stop processing the file if the pattern matches. A quick search points me to using FNR for the former, and { nextline } for the latter. Let’s try it:

Great! Only TODOs.org is missing, but the script is much better: instead of matching against any part of the file that may have a shebang-like line, we only look for the first. Let’s put it back into the assert-shellcheck.sh file and use NULL for separators to accommodate files with spaces in the name:

This is where I’ve stopped, but I imagine a likely improvement: match against only #!/bin/sh and #!/usr/bin/env bash shebangs (the ones I use most), to avoid running ShellCheck on Perl files, or other shebangs.

Also when reviewing the text of this article, I found that { nextfile } is a GNU Awk extension. It would be an improvement if assert-shellcheck.sh relied on the POSIX subset of Awk for working correctly.