> The shell, awk, sed, etc. had arrived at more or less fully formed
> versions by 1980. Perl (and TCL) did not appear until the very end of
> the 1980’s. What filled the gap in that decade, if anything?
Nothing. It was the need for filling the gap that engendered Perl.
ISTR that lwall said his specific problem with using awk was its inability to read from files other than those mentioned on the command line (and them only sequentially). Old awk did not have getline. There were probably other itches he wanted to scratch, but that one was a deal-breaker (or -maker).
Remember that Perl 1 through Perl 3 were around durinig the 80s; Perl 4
sort of settled in for a longer time.
Perl 2 was the faster (though still very slow) regex engine; Perl 3 was binary I/O; Perl 4 was the Camel Book, which is why the version number stayed at 4.xxx until something in the Camel Book broke.
"New" awk didn't get out of the labs until 1987/1988, and that was only
via SVR3.2 and/or the AT&T Toolchest; it wasn't open source, nor
did it provide access to many of the needed system calls in the way
that Perl did.
The project I worked on in 1999-2005 was about half Perl and half shell+sed+awk+etc.... I was able to do that because I had worked on an earlier project that was by fiat all Perl 4. I don't use Perl any more, but I still use the shell toolkit constantly.
I have a back burner project to allow convenient manual data file transformation. It grew out of my dissatisfaction with using the m,n! command in various editors: you can undo, but it's slow, and you have no access to the full undo tree except in vim. My proposed 'mung' command doesn't actually edit at the micro level, but it lets you use | to create a new state of the tree, and each state is stored in a file (disk is cheap). The tree is persistent. When you have the file the way you want it, you write it out and, if you want, generate a shell script that replicates what you just did so it's repeatable.