30 Mar
2025
30 Mar
'25
12:48 p.m.
"The requirement that awk add a trailing
<newline> to the program argument
text is to simplify the grammar, making it match a text file in form."
This should no more be a *requirement* for awk than globbing should have
been a requirement for MS-DOS apps. A widespread principle deserves a
widespread answer. If it is a requirement on awk, then for interoperability
it should be made a requirement on all programs that handle text files,
especially editors.
The way to do that, of course, would be to redefine text file to allow a
non-newline as the last character. Ugh.
Not warning perpetuates travesties like "awk END{print NR}' " giving a
different
answer than "wc -l".
I agree that awk does the kind thing by supplying the final newline. But
it should recognize that this is non-standard behavior and warn in the
interest of discouraging the proliferation of garbage.
Postel's so-called "robustness principle" is in play here. "Be
conservative
in what you send, be liberal in what you accept" would better read,
"Send conservatively; receive amply but grudgingly".
Doug
30 Mar
30 Mar
2 p.m.
New subject: non-Posix input files
Doug,
It turns out there wasn't enough context and we both got
confused. The "requirement" is that
awk '{ print }' ...
look to the lexer / grammer as if it had been "{ print }\n".
There is no requirement that input "text" files behave as if a
final newline is always there.
Thanks,
Arnold
Douglas McIlroy <douglas.mcilroy(a)dartmouth.edu> wrote:
"The
requirement that awk add a trailing <newline> to the program argument
text is to simplify the grammar, making it match a text file in form."
This should no more be a *requirement* for awk than globbing should have
been a requirement for MS-DOS apps. A widespread principle deserves a
widespread answer. If it is a requirement on awk, then for interoperability
it should be made a requirement on all programs that handle text files,
especially editors.
The way to do that, of course, would be to redefine text file to allow a
non-newline as the last character. Ugh.
Not warning perpetuates travesties like "awk END{print NR}' " giving a
different
answer than "wc -l".
I agree that awk does the kind thing by supplying the final newline. But
it should recognize that this is non-standard behavior and warn in the
interest of discouraging the proliferation of garbage.
Postel's so-called "robustness principle" is in play here. "Be
conservative
in what you send, be liberal in what you accept" would better read,
"Send conservatively; receive amply but grudgingly".
Doug 
9:45 p.m.
New subject: non-Posix input files
On 3/30/25 10:00 AM, arnold(a)skeeve.com wrote:
Doug,
It turns out there wasn't enough context and we both got
confused. The "requirement" is that
awk '{ print }' ...
look to the lexer / grammer as if it had been "{ print }\n".
There is no requirement that input "text" files behave as if a
final newline is always there.
There's a similar situation with `sh -c', since sh can treat the end of the
string as a command terminator. The rationale notes it:
"Earlier versions of this standard required that input files to the shell
be text files except that line lengths were unlimited. However, that was
overly restrictive in relation to the fact that shells can parse a script
without a trailing newline[.]"
Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU chet(a)case.edu http://tiswww.cwru.edu/~chet/
2:23 p.m.
New subject: the case of the missing new line, Re: non-Posix input files
It appears that Douglas McIlroy <douglas.mcilroy(a)dartmouth.edu> said:
Postel's so-called "robustness
principle" is in play here. "Be conservative
in what you send, be liberal in what you accept" would better read,
"Send conservatively; receive amply but grudgingly".
I'm reasonably sure he meant it more narrowly than that. "Be liberal" only
applied when the spec was ambiguous, as was often the case in the early Internet
when new corner cases popped up every day. When the spec clearly said to do
something he had no problem rejecting inputs that were wrong.
I haven't been writing code to handle byte stream text files as long as you have
but there's an unfixable problem since Unix lets you put a stream of bytes
not ending with \n into a file and then try to treat it as text. Whatever you
do will be wrong for some uses.
Perhaps we can segue to the argument about semicolons in Algol and Pascal vs.
semicolons in PL/I and C which is more or less the same issue.
R's,
John
PS: If only the Unix file system had records with two-byte length prefixes, or
better fixed length 80 byte records, you know, like real computers do.
3:35 p.m.
New subject: the case of the missing new line, Re: non-Posix input files
On 3/30/25 10:23 AM, John Levine wrote:
PS: If only the Unix file system had records with
two-byte length prefixes, or
better fixed length 80 byte records, you know, like real computers do.
Luxury.
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU chet(a)case.edu http://tiswww.cwru.edu/~chet/
2
days inactive
2
days old
4 comments
4 participants
participants (4)
-
arnold＠skeeve.com -
Chet Ramey -
Douglas McIlroy -
John Levine