8 Mar
2023
8 Mar
'23
2:10 p.m.
In PWB1, support for 'huge' files appears to have been removed. If one
compares bmap() in PWB1'S subr.c with V6's, the "'huge' fetch of
double
indirect block" code is gone. I guess PWB didn't need very large (>
8*256*512
= 1,048,576 bytes) files? I'm not sure what the _benefits_ of removing it
were, though - unless PWB was generating lots of files of between 7*256*512
and 8*256*512 bytes in length, and they wanted to avoid the overhead of the
double-indirect block? (The savings in code space are derisory - unlike in
LSX/MINI-UNIX.) Anyone know?
Noel
9 Mar
9 Mar
9:21 a.m.
I have not seen the UNIX kernel source code in quite a while, but as I
recall the double indirect block algorithm did not kick in until the file
exceeded a certain threshold. So it would not make sense to remove the code
for performance reasons.
Perhaps this is more likely due to the use of larger logical block sizes....
Is the code physically removed or IFDEF'd out for conditional compilation?
Perhaps someone decided that programmers would never need to test code on
large files..
On Wed, Mar 8, 2023, 8:10 AM Noel Chiappa <jnc(a)mercury.lcs.mit.edu> wrote:
In PWB1, support for 'huge' files appears to
have been removed. If one
compares bmap() in PWB1'S subr.c with V6's, the "'huge' fetch of
double
indirect block" code is gone. I guess PWB didn't need very large (>
8*256*512
= 1,048,576 bytes) files? I'm not sure what the _benefits_ of removing it
were, though - unless PWB was generating lots of files of between 7*256*512
and 8*256*512 bytes in length, and they wanted to avoid the overhead of the
double-indirect block? (The savings in code space are derisory - unlike in
LSX/MINI-UNIX.) Anyone know?
Noel
1:40 p.m.
removed in PWB/UNIX 1.0.
ifdef'd out in the Harvard/Radcliffe Student Time-sharing System (HRSTS)
parts found in tuhs/Applications/Usenix_77. h/distrib.note includes:
'4. "Huge" files are not supported by our modifications. This is not
necessarily a hard restriction, however we early on decided
we wanted no more than one level of indirection all the way up to 1 megabyte,
and had no need for larger files. The incompatibilities may be minimal,
but we have not even bothered to seek them out.'
On Thu, Mar 09, 2023 at 03:21:19AM -0500, Kenneth Goodwin wrote:
I have not seen the UNIX kernel source code in quite a
while, but as I
recall the double indirect block algorithm did not kick in until the file
exceeded a certain threshold. So it would not make sense to remove the code
for performance reasons.
Perhaps this is more likely due to the use of larger logical block sizes....
Is the code physically removed or IFDEF'd out for conditional compilation?
Perhaps someone decided that programmers would never need to test code on
large files..
On Wed, Mar 8, 2023, 8:10 AM Noel Chiappa <jnc(a)mercury.lcs.mit.edu> wrote:
> In PWB1, support for 'huge' files appears to have been removed. If one
> compares bmap() in PWB1'S subr.c with V6's, the "'huge'
fetch of double
> indirect block" code is gone. I guess PWB didn't need very large (>
> 8*256*512
> = 1,048,576 bytes) files? I'm not sure what the _benefits_ of removing it
> were, though - unless PWB was generating lots of files of between 7*256*512
> and 8*256*512 bytes in length, and they wanted to avoid the overhead of the
> double-indirect block? (The savings in code space are derisory - unlike in
> LSX/MINI-UNIX.) Anyone know?
>
> Noel
>
4:07 p.m.
In the version 6 file system, If the file was less than 4K, the four
blocks in the inode refer directly to the disk blocks containing the
file data.
After thef ile grows past that, the large bit is set in the inode and
these blocks contain the single indirect blocks, referencing 256 data
blocks each, until you used up seven of these.
The last one was the double indirect block that pointed to other
indirect blocks. The file size was stored in 24 bits, and the block
numbers were 16 bit.
As Noel points out, PWB uses the same file system as V6 but with the
HUGE (last double indirect) code commented out. I have no idea why and
I never noticed it before. While we used a lot of stuff from the PWB
distribution (notably SCCS) we had no real interest in using the PWB’s
kernel because our V6 kernel had diverged sharply from the Bell
versions.
Version 7 came around with a different filesystem layout. File sizes
were 32 bits, and the disk block indexes were 24 bits. The latter was
stored as a series of bytes on the disk but expanded into longs in
memeory.
There was no large bit anymore. The first 8 blocks addresses in the
inode were always single indirect, the next two were single indirect,
and the last one was double indirect. (If my memory serves me right)
At BRL, we modified our very-hacked V6 kernel to have the ability to
mount either V6 or V7 filesystems.
We never ran V5, but I think V5 is essentially the V6 file system
without the HUGE support again.
-Ron
4:13 p.m.
That should read eight blocks in the inode not four in the first
sentence (and up to eight later).
------ Original Message ------
From "Ron Natalie" <ron(a)ronnatalie.com>
To "Kenneth Goodwin" <kennethgoodwin56(a)gmail.com>; "Noel
Chiappa"
<jnc(a)mercury.lcs.mit.edu>
Cc "The Eunuchs Hysterical Society" <tuhs(a)tuhs.org>
Date 3/9/2023 10:07:17 AM
Subject [TUHS] Re: 'Huge' file support removed from PWB1
In the version 6 file system, If the file was less than
4K, the four
blocks in the inode refer directly to the disk blocks containing the
file data.
After thef ile grows past that, the large bit is set in the inode and
these blocks contain the single indirect blocks, referencing 256 data
blocks each, until you used up seven of these.
The last one was the double indirect block that pointed to other
indirect blocks. The file size was stored in 24 bits, and the block
numbers were 16 bit.
As Noel points out, PWB uses the same file system as V6 but with the
HUGE (last double indirect) code commented out. I have no idea why and
I never noticed it before. While we used a lot of stuff from the PWB
distribution (notably SCCS) we had no real interest in using the PWB’s
kernel because our V6 kernel had diverged sharply from the Bell
versions.
Version 7 came around with a different filesystem layout. File sizes
were 32 bits, and the disk block indexes were 24 bits. The latter was
stored as a series of bytes on the disk but expanded into longs in
memeory.
There was no large bit anymore. The first 8 blocks addresses in the
inode were always single indirect, the next two were single indirect,
and the last one was double indirect. (If my memory serves me right)
At BRL, we modified our very-hacked V6 kernel to have the ability to
mount either V6 or V7 filesystems.
We never ran V5, but I think V5 is essentially the V6 file system
without the HUGE support again.
-Ron
6:35 p.m.
"Ron Natalie" <ron(a)ronnatalie.com> wrote:
Version 7 came around with a different filesystem
layout. File sizes
were 32 bits, and the disk block indexes were 24 bits. The latter was
stored as a series of bytes on the disk but expanded into longs in
memeory.
There was no large bit anymore. The first 8 blocks addresses in the
inode were always single indirect, the next two were single indirect,
and the last one was double indirect. (If my memory serves me right)
Wasn't there a triple-indirect block as the very last one?
Arnold
6:01 p.m.
Noel Chiappa <jnc(a)mercury.lcs.mit.edu> writes:
In PWB1, support for 'huge' files appears to
have been removed.
They did make various changes to the system to protect against resource
exhaustion in what was a genuine production multi-user facility, and
might possibly have made such a change to ensure that a runaway process
couldn't write a file more than a megabyte in size.
It's not mentioned in the PWB article in the special UNIX issue of the
Bell System Technical Journal, though, and that article does describe
various changes they'd made, explaining how and why.
However: PWB grew out of Research, side by side with it, evaluating and
adopting changes along the way. They aimed to stay mostly in synch,
but chose to keep and maintain a number of differences. Maybe they
started their work before the "huge" file support was introduced, and
then simply did not include that change? It's a backward incompatible
change (at least if you have at least one file that already uses the
last indirect block of its inode), and they might have decided to cross
that bridge if and when they ever got to it.
-tih
--
Most people who graduate with CS degrees don't understand the significance
of Lisp. Lisp is the most important idea in computer science. --Alan Kay
10 Mar
10 Mar
12:25 a.m.
I talked with John Mashey this PM and asked him about the huge file stuff
and PWB1.
First, he did not remember that they had done that, but ...
He then emphasized that the PWB team was running the first (and for a long
time the largest) "computer center" using UNIX in the Bell System in
Piscataway - supporting over 1000 programmers as a front end to the
mainframes. His memory is that they mostly used 88 Mbyte RP04s [remember,
at the time, RK05s were only 2.5M]. Needing to support a megabyte or
larger file would have been relatively rare for them. He also pointed out
they needed to run their system on systems as small as 48K byte 11/40s to
as large as 1M 11/70s.
As he said, his team tried to track Dennis and Ken's Research
system closely since the PWB team was not primarily in the system
development/enhancement business -* i.e.* they tied to keep what would
become PWB1 and whatever was running at Research as similar as they could
[he told some stories about running to MH to get Dennis' latest compiler
binary because the incremental development scheme dmr used would make
someone like PWB that took snapshots at different times, end up with a
compiler that could not compile itself --#1#]. But the PWB group >> was <<
interested in making the system they were running as reliable as possible
and, more importantly, being as graceful as possible when different
resources were exhausted.
He said it would have made sense for them to remove the "huge file" support
to help to defend against running out of space and the system having issues
[V6 was extremely ungrateful about that condition]. So limiting
file size became a simple quota, if you will. A runaway program was less
likely to take the system out. So the process would die because the file
got too large (or, to use the IBM shop term in those days -- ABEND).
#1# He also regaled a bit about Ken's infamous Turning Award hack and
discovering symbols in the compiler binary they could not understand ;-)
622
days inactive
623
days old
7 comments
7 participants
participants (7)
-
arnold＠skeeve.com -
Clem Cole -
jnc＠mercury.lcs.mit.edu -
Jonathan Gray -
Kenneth Goodwin -
Ron Natalie -
Tom Ivar Helbekkmo