27 Mar
2025
27 Mar
'25
5:35 p.m.
> info groff gives semantics for including nonempty
files that don't end
> with newline. Such files violate the Posix definition of text file.
>
> Although groff is certainly justified in providing semantics for
> non-Posix text, I suggest that it should warn when it does so.
That's true but I'm hesitant to put groff in
the business of wagging its
finger at users feeding it non-strictly-conforming text files when doing
so doesn't cause it any problems.
Causing groff problems is an odd criterion. The fact that groff will paste
files together unless the first happens to end in a newline is a sign of
groff 's internals, not of the underlying problem.
A newline missing at the end of a file is typically a symptom of either the
incaution of some other program (perhaps an editor) or of a file having
been unexpectedly truncated (as by a program abort). The latter cause
is common enough to justify warning always, not just about cases that
are inconvenient to groff.
Groff is what it is, but if the treatment of absent final newlines were up
for grabs, I'd argue for the more common solution: in all cases insert
a newline and warn.
Doug
27 Mar
27 Mar
6:45 p.m.
New subject: non-Posix input files
Branden Robin wrote:
> info groff gives semantics for including nonempty
files that don't end
>> with newline. Such files violate the Posix definition of text file.
Not so fast. POSIX does not define a text file (that's an ANSI C-ism -
thank you DOS -- more in minute).
Even in the original POSIX definition we were very careful to >>never<< do
that abomination. What the current POSIX specs says is:
3.164 File
An object that can be written to, or read from, or both. A file has certain
attributes, including access permissions and type. File types include regular
file, character special file, block special file, FIFO special file,
symbolic link, socket, and directory. Other types of files may be supported
by the implementation.
....
3.317 Regular File
A file that is a randomly accessible sequence of bytes, with no further
structure imposed by the system.
In fact, nowhere in the term "text file" defined (or used — search for it)
in the POSIX standard. The problem is that at the same time, ANSI C was
being defined in the early 1980s The DOS weenies wanted to support the
silly two characters line terminations and old notion of a structured "text
file" which PC/MS-DOS had inherited from the DEC and IBM OS's of 1960s and
what IBM has called "access methods" in the old days (in UNIX everything is
just a stream of bytes thank you or in Multics, everything is segment).
The UNIX folks on the C committee were very much against adding the support
— which Lattice C for MS-DOS had added at the time but happened to be the
most popular C compiler for the DOS target and the number of user of the
compiler was growing extremely rapidly. They were also pushing for things
like the near and far keywords. It was quite a fight in the C community. (It
was one of the additions that caused Dennis to quip: *"**When I read
commentary about suggestions for where C should go, I often think back and
give thanks that it wasn't developed under the advice of a worldwide
crowd."*
Anyway, the UNIX folk had to decide which eveils the DOS folks were pushes
they could live with and a compromise was the creation of the new "rb"
"wb"
"ab" crud in the fopen(3) call since we you use "r", "w",
"b" as because
POSIX allows UNIX to open a what ANSI C called a "text file" since there no
such thing under POSIX.
But in the ANSI C spec (which, remember, is >>only<< for C source code), it
says:
5.2.1 Character sets
.... The representation of each member of the source and execution basic
character sets shall fit in a byte.
In both the source and execution basic character sets, the value of each
character after 0 in the above
list of decimal digits shall be one greater than the value of the previous. In
source files, there shall
be some way of indicating the end of each line of text; this International
Standard treats such an
end-of-line indicator as if it were a single new-line character. In the
basic execution character set,
there shall be control characters representing alert, backspace, carriage
return, and new line. If any
other characters are encountered in a source file (except in an identifier,
a character constant, a string
literal, a header name, a comment, or a preprocessing token that is never
converted to a token), the
behavior is undefined.
The key point in the highlighted text in ANSI C is that it does not *require
it*. A problem in the ANSI C standard is that does >>use<< the term
"text file" although ANSI C never really defines what one is (and probably
could not because of the fight - UNIX would have said -- nope -- no
structure required. ANSI C does say in:
7.21.2 Streams
...
Environmental limits
An implementation shall support text files with lines containing at least
254 characters, including
the terminating new-line character. The value of the macro BUFSIZ shall be
at least 256.
The whole compromise was that you did not have to use termination of type.
If the concept of a 'textfile" is local to the implementation. UNIX folks
get to keep doing things as they did before, so if you used fopen with "r"
or "w" its up to the program to worry about the format of the file "access
modes" from the 1960s were not needed.
ᐧ
6:52 p.m.
New subject: non-Posix input files
On 3/27/25 2:45 PM, Clem Cole wrote:
Branden Robin wrote:
It does.
"3.387 Text File
A file that contains characters organized into zero or more lines. The
lines do not contain NUL characters and none can exceed {LINE_MAX} bytes in
length, including the <newline> character. Although POSIX.1-2024 does not
distinguish between text files and binary files (see the ISO C standard),
many utilities only produce predictable or meaningful output when operating
on text files. The standard utilities that have such restrictions always
specify "text files" in their STDIN or INPUT FILES sections."
https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap03.html#ta…
A line is defined as:
"3.185 Line
A sequence of zero or more non-<newline> characters plus a terminating
<newline> character."
https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap03.html#ta…
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU chet(a)case.edu http://tiswww.cwru.edu/~chet/
> info groff gives semantics for including
nonempty files that don't end
> with newline. Such files violate the Posix definition of text file.
Not so fast. POSIX does not define a text file (that's an ANSI C-ism -
thank you DOS -- more in minute).
7 p.m.
New subject: non-Posix input files
Argh -- I standard corrected. We worked hard at the beginning to keep that
crap out -- sigh.
But at least is does says: POSIX.1-2024 *does not **distinguish between
text files and binary files* (see the ISO C standard)
So my point is the same: don't blame POSIX- this is an ANSI C issue, and
utilities really should not care.
ᐧ
On Thu, Mar 27, 2025 at 2:53 PM Chet Ramey <chet.ramey(a)case.edu> wrote:
On 3/27/25 2:45 PM, Clem Cole wrote:
Branden Robin wrote:
>> info groff gives semantics for including nonempty files that
don't end
>> with newline. Such files violate
the Posix definition of text
file.
Not so fast. POSIX does not define a text file (that's an ANSI C-ism -
thank you DOS -- more in minute).
It does.
"3.387 Text File
A file that contains characters organized into zero or more lines. The
lines do not contain NUL characters and none can exceed {LINE_MAX} bytes in
length, including the <newline> character. Although POSIX.1-2024 does not
distinguish between text files and binary files (see the ISO C standard),
many utilities only produce predictable or meaningful output when
operating
on text files. The standard utilities that have such restrictions always
specify "text files" in their STDIN or INPUT FILES sections."
https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap03.html#ta…
A line is defined as:
"3.185 Line
A sequence of zero or more non-<newline> characters plus a terminating
<newline> character."
https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap03.html#ta…
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU chet(a)case.edu http://tiswww.cwru.edu/~chet/
7:05 p.m.
New subject: non-Posix input files
On 3/27/25 3:00 PM, Clem Cole wrote:
Argh -- I standard corrected. We worked hard at the
beginning to keep that
crap out -- sigh.
But at least is does says: POSIX.1-2024 /_does not _//_distinguish between
text files and binary files_/ (see the ISO C standard)
It also says "The standard utilities that have such restrictions always
specify "text files" in their STDIN or INPUT FILES sections," so you
can't
avoid it.
awk is one such utility (sh is not). This is an application requirement, so
awk is required to add a newline at the end of a file that does not have
one.
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU chet(a)case.edu http://tiswww.cwru.edu/~chet/
7:35 p.m.
New subject: non-Posix input files
Chet - as I said, we tried so hard to keep that kind of crap out. Dennis
was right. FWIW: with UNIX (POSIX), input will end if an EOF and the ANSI
C untilies will obey it or a newline, so you can write the code to work
fine either way. But that's a choice of implementation/what subroutines -
how you think about the data.
I'll accept that that is what the words say in the >>awk<< specification
document, but as one of the original authors of the first UNIX standard and
the later POSIX standard I can say we tried hard to mak sure we got it
right and follow the idea: A regular file has no structure and never to
allow the standard to impose it. I think the core standard still says
that, and the basic idea is unchanged. The actual structure of the input
file is an application idea, not a UNIX/POSIX defined idea.
The issue here is the term POSIX. Do you mean it to be the kernel (.1)
and if a >>specific<< application with .2 (the C compiler itself, awk, ed)
which might put structure onto the file and that's fine. The >>OS<<
does
not set the structure — it is done by something else.
I understand having the application do it; I wish it did not. Many
applications (even text editors) can (and have) been written without
needing one specific structure, which is my point. I also accept that the
folks who took over the standard in the name of "progress" changed
(relaxed) much of what we worked so hard to avoid, knowing there were
dragons - particularly WRT to textual information. We really did not want
to repeat the errors of the 1960s. *i.e.*, as George Santayana originally
wrote, *“Those who cannot remember the past are condemned to repeat it.” *
ᐧ
On Thu, Mar 27, 2025 at 3:05 PM Chet Ramey <chet.ramey(a)case.edu> wrote:
On 3/27/25 3:00 PM, Clem Cole wrote:
Argh -- I standard corrected. We worked hard at
the beginning to keep
that
crap out -- sigh.
But at least is does says: POSIX.1-2024 /_does not _//_distinguish
between
text files and binary files_/ (see the ISO C
standard)
It also says "The standard utilities that have such restrictions always
specify "text files" in their STDIN or INPUT FILES sections," so you
can't
avoid it.
awk is one such utility (sh is not). This is an application requirement, so
awk is required to add a newline at the end of a file that does not have
one.
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU chet(a)case.edu http://tiswww.cwru.edu/~chet/
7:50 p.m.
New subject: non-Posix input files
It seems my reply to Clem went astray. POSIX Part 2, Shell and
Utilities, is very clear:
2..2.2.181 text file: A File that contains characters organized into
one or more lines.
The lines shall not contain NUL characters ...
2.2.2.95 line: A sequence of zero or more non-<newline> characters
plus a terminating <newline> character,
Oddly--and in my opinion wrongly--the standard excludes empty files.
It would be a shock
if an editor or groff refused to process an empty file and thereby
broke Kernighan's law, "Do
nothing gracefully".
Doug
On Thu, Mar 27, 2025 at 3:35 PM Clem Cole <clemc(a)ccc.com> wrote:
Chet - as I said, we tried so hard to keep that kind of crap out. Dennis was right.
FWIW: with UNIX (POSIX), input will end if an EOF and the ANSI C untilies will obey it or
a newline, so you can write the code to work fine either way. But that's a choice of
implementation/what subroutines - how you think about the data.
I'll accept that that is what the words say in the >>awk<< specification
document, but as one of the original authors of the first UNIX standard and the later
POSIX standard I can say we tried hard to mak sure we got it right and follow the idea: A
regular file has no structure and never to allow the standard to impose it. I think the
core standard still says that, and the basic idea is unchanged. The actual structure of
the input file is an application idea, not a UNIX/POSIX defined idea.
The issue here is the term POSIX. Do you mean it to be the kernel (.1) and if a
>>specific<< application with .2 (the C compiler itself, awk, ed) which might
put structure onto the file and that's fine. The >>OS<< does not set
the structure — it is done by something else.
I understand having the application do it; I wish it did not. Many applications (even
text editors) can (and have) been written without needing one specific structure, which is
my point. I also accept that the folks who took over the standard in the name of
"progress" changed (relaxed) much of what we worked so hard to avoid, knowing
there were dragons - particularly WRT to textual information. We really did not want to
repeat the errors of the 1960s. i.e., as George Santayana originally wrote, “Those who
cannot remember the past are condemned to repeat it.”
ᐧ
On Thu, Mar 27, 2025 at 3:05 PM Chet Ramey <chet.ramey(a)case.edu> wrote:
>
> On 3/27/25 3:00 PM, Clem Cole wrote:
> > Argh -- I standard corrected. We worked hard at the beginning to keep that
> > crap out -- sigh.
> >
> > But at least is does says: POSIX.1-2024 /_does not _//_distinguish between
> > text files and binary files_/ (see the ISO C standard)
>
> It also says "The standard utilities that have such restrictions always
> specify "text files" in their STDIN or INPUT FILES sections," so you
can't
> avoid it.
>
> awk is one such utility (sh is not). This is an application requirement, so
> awk is required to add a newline at the end of a file that does not have
> one.
>
> --
> ``The lyf so short, the craft so long to lerne.'' - Chaucer
> ``Ars longa, vita brevis'' - Hippocrates
> Chet Ramey, UTech, CWRU chet(a)case.edu http://tiswww.cwru.edu/~chet/
8:08 p.m.
New subject: non-Posix input files
Hi Doug,
At 2025-03-27T15:50:50-0400, Douglas McIlroy wrote:
It seems my reply to Clem went astray. POSIX Part 2,
Shell and
Utilities, is very clear:
2..2.2.181 text file: A File that contains characters organized into
one or more lines.
The lines shall not contain NUL characters ...
2.2.2.95 line: A sequence of zero or more non-<newline> characters
plus a terminating <newline> character,
Oddly--and in my opinion wrongly--the standard excludes empty files.
It would be a shock if an editor or groff refused to process an empty
file and thereby broke Kernighan's law, "Do nothing gracefully".
I think I can claim that GNU troff has long upheld that principle.
$ printf '' | ~/groff-HEAD/bin/groff | grep . || echo NO OUTPUT
NO OUTPUT
$ printf '' | ~/groff-1.23.0/bin/groff | grep . || echo NO OUTPUT
NO OUTPUT
$ printf '' | ~/groff-1.22.4/bin/groff | grep . || echo NO OUTPUT
NO OUTPUT
$ printf '' | ~/groff-1.22.3/bin/groff | grep . || echo NO OUTPUT
NO OUTPUT
That carries us back through over 10 years of groff development. I'd
bet it's been the case since 1991. Maybe earlier.
Regards,
Branden
8:02 p.m.
New subject: non-Posix input files
[CC updated]
Hi folks,
This thread started on the groff@gnu list, so I'm looping it back in.
https://lists.gnu.org/archive/html/groff/2025-02/msg00088.html
https://lists.gnu.org/archive/html/groff/2025-03/msg00070.html
The formatter, GNU troff:
* catenates its input into a single stream, disregarding any file
"boundaries" except the first and the last;
* does not generally require the input to end with a newline;
* throws a warning diagnostic if the input ends with the *roff escape
character (and ignores that character);
* warns about and discards NUL characters from the input (along with
several others that are invalid as input);
* always terminates its _output_ files (whether in Kernighan's "trout"
format [originally seen in CSTR #97], or "approximate" [`troff -a`]
format) with newlines (ASCII LF);
* and writes only printable ASCII characters and the space in
"trout"/"grout" or approximate outputs, with one esoteric
exception.[1]
I'd like to know what, specifically, this program should be doing
differently.
Regards,
Branden
[1] The `cf` request permits the copying of abritrary bytes from the
input file named by it to "grout" output. In the forthcoming
_groff_ 1.24.0, the formatter handles this request only in the
non-default "unsafe mode" enabled by the `-U` command-line option.
Quoting groff's Texinfo manual:
-- Request: .trf ["]file
-- Request: .cf ["]file
Copy FILE as "throughput" to GNU 'troff' output. Each line of
FILE
is output as if preceded by '\!', but is not interpreted by the
formatter. If FILE does not end with a newline, 'trf' appends one.
Both requests break the line before reading FILE, unless invoked
with the no-break control character.
When used in a diversion, these requests embed a node (*note Gtroff
Internals::) in it that, when interpolated, causes the contents of
FILE to be copied to the output. In AT&T 'troff', 'cf' does
so
even if a diversion is in use; this behavior is so anomalous that
it must be considered a bug.
'cf' copies the contents of FILE completely unprocessed; it is
therefore an error to use this request in safer mode, which is the
default. Invoke GNU 'troff' or a front end with the '-U' option
to
enable unsafe mode.
'trf' discards invalid input characters; recall *note
Identifiers::.
For 'cf', within a diversion, "completely unprocessed" means
that
each line of a file to be inserted is handled as if it were
preceded by '\!\\!'.
*Caution:* Use of these requests can put syntactically invalid
content into the formatter's output, which 'groff''s output
drivers
then fail to process. One application is to pass instructions to a
postprocessor that interprets FILE's contents and filters it out
before sending it to the output device.
8:09 p.m.
New subject: non-Posix input files
On 3/27/25 3:35 PM, Clem Cole wrote:
Chet - as I said, we tried so hard to keep that kind
of crap out.
I believe you!
Even the first published version of 1003.2, in 1992, says
"The definition of text file has caused controversy." and discusses how
the definition seemingly outlaws empty files.
I'll accept that that is what the words say in
the >>awk<< specification
document, but as one of the original authors of the first UNIX standard and
the later POSIX standard I can say we tried hard to mak sure we got it
right and follow the idea: A regular file has no structure and never to
allow the standard to impose it.
Sure. I think the point is that there are a bunch of utilities, like awk,
bc, ed, sort, and so on, whose input is required to be a text file as
defined by the standard. troff isn't in the standard, so Branden can do
what he likes, and if he wants to accept text files as the standard defines
them, then ok.
The issue here is the term POSIX. Do you mean it to
be the kernel (.1)
and if a >>specific<< application with .2 (the C compiler itself, awk, ed)
There is no distinction like that any more. POSIX is all of it: shell,
utilities, library functions, system calls. The standard guarantees that
where an input file is required to be a text file it is explicitly
mentioned, and it does that.
I understand having the application do it; I wish it
did not.
Sorry, jargon. The application is the thing that calls awk; awk is a
utility (expressive shrug).
So whatever calls awk is responsible for making sure the script and any
input files are text files. awk can add a newline to the end of a script
file that doesn't have one, so the grammar works right. It's also entitled
to do whatever it wants, including throw an error, if the input file isn't
a text file.
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU chet(a)case.edu http://tiswww.cwru.edu/~chet/
28 Mar
28 Mar
12:19 a.m.
New subject: non-Posix input files
What is a "character" according to POSIX? I suspect it's just a byte,
which
is a long-outdated definition, the char keyword notwithstanding.
-rob
12:37 a.m.
New subject: non-Posix input files
POSIX.1 is clear - a regular file is a sequence of bytes and that is the
only requirement. See my previous email where I give the section number
which is very crisp.
Sent from a handheld expect more typos than usual
On Thu, Mar 27, 2025 at 8:19 PM Rob Pike <robpike(a)gmail.com> wrote:
What is a "character" according to POSIX? I
suspect it's just a byte,
which is a long-outdated definition, the char keyword notwithstanding.
-rob
12:38 a.m.
New subject: non-Posix input files
I should have added that .1 does not talk about characters. That was left
to ANSI C
Sent from a handheld expect more typos than usual
On Thu, Mar 27, 2025 at 8:37 PM Clem Cole <clemc(a)ccc.com> wrote:
POSIX.1 is clear - a regular file is a sequence of
bytes and that is the
only requirement. See my previous email where I give the section number
which is very crisp.
Sent from a handheld expect more typos than usual
On Thu, Mar 27, 2025 at 8:19 PM Rob Pike <robpike(a)gmail.com> wrote:
> What is a "character" according to POSIX? I suspect it's just a byte,
> which is a long-outdated definition, the char keyword notwithstanding.
>
> -rob
>
>
1:02 a.m.
New subject: non-Posix input files
On 3/27/25 8:19 PM, Rob Pike wrote:
What is a "character" according to POSIX? I
suspect it's just a byte, which
is a long-outdated definition, the char keyword notwithstanding.
"3.58 Character
A sequence of one or more bytes representing a member of a character set.
Note:
This term corresponds to the ISO C standard term multi-byte character,
where a single-byte character is a special case of a multi-byte character.
Unlike the usage in the ISO C standard, character here has no necessary
relationship with storage space, and byte is used when storage space is
discussed.
See the definition of the portable character set in 6.1 Portable
Character Set for a further explanation of the graphical representations of
(abstract) characters, as opposed to character encodings."
https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap03.html#ta…
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU chet(a)case.edu http://tiswww.cwru.edu/~chet/
7:22 a.m.
New subject: non-Posix input files
Chet Ramey via TUHS <tuhs(a)tuhs.org> wrote:
It also says "The standard utilities that have
such restrictions always
specify "text files" in their STDIN or INPUT FILES sections," so you
can't
avoid it.
awk is one such utility (sh is not). This is an application requirement, so
awk is required to add a newline at the end of a file that does not have
one.
No awk I know of actually does that. Instead, they all treat end of file
as terminating the current input record:
$ for awk in gawk nawk mawk mksawk "busybox awk" goawk
do echo == $awk
echo -n ab:cd | $awk -v RS=: '{ print ">" $0 "<" }'
done
== gawk
ab<
cd<
== nawk
ab<
cd<
== mawk
ab<
cd<
== mksawk
ab<
cd<
== busybox awk
ab<
cd<
== goawk
ab<
cd<
If a newline were being added, the output would have been:
ab<
cd
<
Nor is this something that I'm about to change, or suggest to any other
awk author to change.
My two cents,
Arnold
12:25 p.m.
New subject: non-Posix input files
On 3/28/25 3:22 AM, arnold(a)skeeve.com wrote:
Chet Ramey via TUHS <tuhs(a)tuhs.org> wrote:
Sorry, I should have been more clear. This requirement is for scripts:
"The requirement that awk add a trailing <newline> to the program argument
text is to simplify the grammar, making it match a text file in form."
which I imagine all awk implementations do.
I think awk does the reasonable thing with an input file that doesn't end
in a newline.
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU chet(a)case.edu http://tiswww.cwru.edu/~chet/
It also says "The standard utilities that
have such restrictions always
specify "text files" in their STDIN or INPUT FILES sections," so you
can't
avoid it.
awk is one such utility (sh is not). This is an application requirement, so
awk is required to add a newline at the end of a file that does not have
one.
No awk I know of actually does that. Instead, they all treat end of file
as terminating the current input record:
30 Mar
30 Mar
9:13 a.m.
New subject: non-Posix input files
Chet Ramey <chet.ramey(a)case.edu> wrote:
Sorry, I should have been more clear. This requirement
is for scripts:
"The requirement that awk add a trailing <newline> to the program argument
text is to simplify the grammar, making it match a text file in form."
which I imagine all awk implementations do.
Ah. Indeed, gawk does this. I don't remember what goes in The One
True Awk.
Thanks,
Arnold
27 Mar
27 Mar
6:53 p.m.
New subject: non-Posix input files
I think you're looking at the wrong Posix document. Posix Shell and
Utilities (at least my ancient copy) says
2.2.2.181 Text file: A file that contains one or more lines.
The Lines shall not contain NUL characters ...
Doug
On Thu, Mar 27, 2025 at 2:45 PM Clem Cole <clemc(a)ccc.com> wrote:
Branden Robin wrote:
>> info groff gives semantics for including
nonempty files that don't end
>> with newline. Such files violate the Posix definition of text file.
Not so fast. POSIX does not define a text file (that's an ANSI C-ism - thank you
DOS -- more in minute).
Even in the original POSIX definition we were very careful to >>never<< do
that abomination. What the current POSIX specs says is:
3.164 File
An object that can be written to, or read from, or both. A file has certain attributes,
including access permissions and type. File types include regular file, character special
file, block special file, FIFO special file, symbolic link, socket, and directory. Other
types of files may be supported by the implementation.
....
3.317 Regular File
A file that is a randomly accessible sequence of bytes, with no further structure imposed
by the system.
In fact, nowhere in the term "text file" defined (or used — search for it) in
the POSIX standard. The problem is that at the same time, ANSI C was being defined in
the early 1980s The DOS weenies wanted to support the silly two characters line
terminations and old notion of a structured "text file" which PC/MS-DOS had
inherited from the DEC and IBM OS's of 1960s and what IBM has called "access
methods" in the old days (in UNIX everything is just a stream of bytes thank you or
in Multics, everything is segment). The UNIX folks on the C committee were very much
against adding the support — which Lattice C for MS-DOS had added at the time but happened
to be the most popular C compiler for the DOS target and the number of user of the
compiler was growing extremely rapidly. They were also pushing for things like the near
and far keywords. It was quite a fight in the C community. (It was one of the additions
that caused Dennis to quip: "When I read commentary about suggestions for where C
should go, I often think back and give thanks that it wasn't developed under the
advice of a worldwide crowd."
Anyway, the UNIX folk had to decide which eveils the DOS folks were pushes they could
live with and a compromise was the creation of the new "rb" "wb"
"ab" crud in the fopen(3) call since we you use "r", "w",
"b" as because POSIX allows UNIX to open a what ANSI C called a "text
file" since there no such thing under POSIX.
But in the ANSI C spec (which, remember, is >>only<< for C source code), it
says:
5.2.1 Character sets
.... The representation of each member of the source and execution basic character sets
shall fit in a byte.
In both the source and execution basic character sets, the value of each character after
0 in the above
list of decimal digits shall be one greater than the value of the previous. In source
files, there shall
be some way of indicating the end of each line of text; this International Standard
treats such an
end-of-line indicator as if it were a single new-line character. In the basic execution
character set,
there shall be control characters representing alert, backspace, carriage return, and new
line. If any
other characters are encountered in a source file (except in an identifier, a character
constant, a string
literal, a header name, a comment, or a preprocessing token that is never converted to a
token), the
behavior is undefined.
The key point in the highlighted text in ANSI C is that it does not require it. A
problem in the ANSI C standard is that does >>use<< the term "text
file" although ANSI C never really defines what one is (and probably could not
because of the fight - UNIX would have said -- nope -- no structure required. ANSI C does
say in:
7.21.2 Streams
...
Environmental limits
An implementation shall support text files with lines containing at least 254 characters,
including
the terminating new-line character. The value of the macro BUFSIZ shall be at least 256.
The whole compromise was that you did not have to use termination of type. If the concept
of a 'textfile" is local to the implementation. UNIX folks get to keep doing
things as they did before, so if you used fopen with "r" or "w" its up
to the program to worry about the format of the file "access modes" from the
1960s were not needed.
ᐧ
2
days inactive
5
days old
17 comments
6 participants
participants (6)
-
arnold＠skeeve.com -
Chet Ramey -
Clem Cole -
Douglas McIlroy -
G. Branden Robinson -
Rob Pike