3 Jan
2023
3 Jan
'23
11:26 p.m.
On Tue, Jan 3, 2023 at 10:09 AM Douglas McIlroy
<douglas.mcilroy(a)dartmouth.edu> wrote:
The forgotten compilers were typically used to supply
glue
to paste major tools together. The nature of that glue---often
simple data reformatting--inspired tools like sed and awk.
Each use of a tool became a process that saved many minutes
of work that would in a scriptless world be guided by hand,
boringly and unreliably.
This, I think, begs the question I raised earlier: what precisely is
"the shell"? The command interpreter by itself isn't that interesting
without a supporting toolset. And now we get into the crux or what it
means to be "portable": what is the minimum set of such tools that
allow us to write portable shell scripts?
Again I ask, am I allowed to use `nawk`? How about '#!' lines? xargs?
cut? paste?
Perhaps that's the benefit of experience: over time, one develops a
sense of what works across the broadest range of systems that one will
likely encounter. But as Warner pointed out, we can be thankful we're
not using Eunice.
A few years ago, I was having lunch with some folks from the Go team
and one of them remarked, "you shouldn't write a shell script that's
longer than about 10 lines. Once you do, it's time to rewrite it in a
real programming language." I was a bit taken aback, but they had a
point. I'll note that Go standardized on using bash everywhere.
Yet glue processes typically did only microseconds of
"real" work. In the name of efficiency, the operations began
to be incorporated directly into the shell. The first
inklings of this can be seen in "echo" and various forms
of variable-substitution making their way into the v7
shell. The phenomenon proliferated into putting what were
typically canned sed one-liners (but not sed itself) into
the shell.
Perhaps I've told this story, but 10 or 12 years ago at Google a
colleague was writing a script to find the location of some resource
or other. I no longer recall what, exactly, but he had written this
massively over-structured thing: shell functions that ran one or two
commands, heavily commented, and he was trying to `return` the exit
status from those commands as the result from the function. It wasn't
working because, well, that's just not how such things work. I sat
down, and rewrote it into about 10 lines; the salient part was that
there was a conditional that determined whether the system it was
running on had a 64-bit or 32-bit userland and ran a handful of
commands depending. I showed him the revised script and he was
uncomfortable; after a while he kind of gave me a look and said,
"yeah, I just don't think people conceptualize of problems that way
anymore." Meaning, it was far more natural for him and others to just
write a Python program instead of a simple script. To him, the script
wasn't simple, but the Python program was.
That has stuck with me for a long, long time. Anyway, we checked the
script in and it worked; it's probably still running, though I suspect
most, if not all, 32-bit systems are gone from Google so perhaps it
was long-ago recycled. I have no idea anymore, but it really gave me
pause: had everything I'd learned about Unix in the prior 20 years
become obsolete? Was I an anachronism amongst my colleagues? Were they
right, and I wrong about how to approach problems?
Lots of specializations crowded out universality. A
side
effect was an explosion of knowledge required to write
or understand code. Such is the tragedy of "forgetting
compilers".
Something I've noticed is that lots of people try to increase
complexity to solve problems, and it rarely occurs to them to
eliminate complexity to solve problems. Sometimes the reasons for this
are good; most of the time they are not.
- Dan C.