I see, no I had not realized that code is still in use, I would have
thought it had been replaced by a whole lot of POSIX bloat. Admittedly the
2.11BSD ctime/asctime/localtime/timezone stuff is simplistic and doesn't
address complicated cases but it's good enough.
However I have to resist the temptation to improve or update stuff in
2.11BSD, I went down that path many times (with the Makefiles project for
instance) and because everything is interdependent you always introduce
more problems and get deeper and deeper enmeshed. In order to stay in
control I only fix essentials and apply a rule of minimal change, period.
This applies until I have a baseline that builds exactly the same binary
system image as the native build. Then I might proactively improve parts of
the system but I will not do it reactively if you follow.
As I see it the zic behaviour is not a bug since time_t is 32 bits on
2.11BSD and has no unreasonable values, and localtime() is BSD not POSIX
compliant and is not allowed to return NULL.
cheers, Nick
On 14/01/2017 6:53 AM, "Random832" <random832(a)fastmail.com> wrote:
On Fri, Jan 13, 2017, at 12:57, Nick Downing wrote:
> I then ended up doing a fair bit of re-engineering, how this came
> about was that I had to port the timezone compiler (zic) to run on the
> Linux cross compilation host, since the goal is eventually to build a
> SIMH-bootable disk (filesystem) with everything on it. This was a bit
> involved, it crashed initially and it turned out it was doing
> localtime() on really small and large values to try to figure out the
> range of years the system could handle. On the Linux system this
> returns NULL for unreasonable time_t values which POSIX allows it to
> do. Hence the crash. It wasn't too obvious how to port this code. (But
> whatever I did, it had to produce the exact same timezone files as a
> native build).
You know that the timezone file format that it uses is still in use
today, right? There's extra data at the end in modern ones for 64-bit
data, but the format itself is cross-platform, with defined field widths
and big-endian byte order.
What do you get when you compare the native built timezone files with
one from your linux host's own zic? It *should* only differ by the
version number in the header [first five bytes "TZif2" vs "TZif"] and
the 64-bit section, if you're giving it the same input files. And I bet
you could take the current version of the code from IANA and, if it
matters to you, remove the parts that output the 64-bit data. If nothing
else, looking at the modern code and the version in 2.11BSD side-by-side
will let you backport bug fixes.
(Note: Technically, the version present in most Linux systems is a fork
maintained with glibc rather than the main version of the code from
IANA)
So I got a fair bit further advanced on my 2.11BSD cross compiler
project, at the moment it can make a respectable unix tree (/bin /usr
etc) with a unix kernel and most software in it. I unpacked the
resulting tarball and chrooted into it on a running SIMH system and it
worked OK, was a bit rough around the edges (missing a few necessary
files in /usr/share and that kind of thing) but did not crash. I
haven't tested the kernel recently but last time I tested it, it
booted, and the checksys output is OK.
I then ended up doing a fair bit of re-engineering, how this came
about was that I had to port the timezone compiler (zic) to run on the
Linux cross compilation host, since the goal is eventually to build a
SIMH-bootable disk (filesystem) with everything on it. This was a bit
involved, it crashed initially and it turned out it was doing
localtime() on really small and large values to try to figure out the
range of years the system could handle. On the Linux system this
returns NULL for unreasonable time_t values which POSIX allows it to
do. Hence the crash. It wasn't too obvious how to port this code. (But
whatever I did, it had to produce the exact same timezone files as a
native build).
So what I ended up doing was to port a tiny subset of 2.11BSD libc to
Linux, including its types. I copied the ctime.c module and prefixed
everything with "cross_" so there was "cross_time_t" and so forth, and
"#include <time.h>" became "#include <cross/time.h>", in turn this
depends on "#include <cross/sys/types.h>" and so on. That way, the
original logic worked unchanged.
I decided to also redo the cross compilation tools (as, cc, ld, nm,
ranlib and so on) using the same approach, since it was conceptually
elegant. This involved making e.g. "cross_off_t" and "struct
cross_exec" available by "#include <cross/a.out.h>", and obviously the
scheme extends to whatever libc functions we want to use. In
particular we use floating point, and I plan to make a "cross_atof()"
for the C compiler's PDP-11-formatted floating-point constant
handling, etc. (This side of things, like the cross tools, was
working, but was not terribly elegant before).
So then I got to thinking, actually this is an incredibly powerful
approach. Instead of just going at it piecemeal, would it not be
easier just to port the entire thing across? To give an example what I
mean, the linker contains code like this:
if (nund==0)
printf("Undefined:\n");
nund++;
printf("%.*s\n", NNAMESIZE, sp->n_name);
It is printing n_name from a fixed-size char array, so to save the
cost of doing a strncpy they have used that "%.*s" syntax which tells
printf not to go past the end of the char array. But this isn't
supported in Linux. I keep noticing little problems like this
(actually I switched off "-Wformat" which was possibly a bad idea). So
with my latest plan this will actually run the 2.11BSD printf()
instead of the Linux printf(), and the 2.11BSD stdio (fixing various
other breakage that occured because BUFSIZ isn't 512 on the Linux
system), and so on. What I will do is, provide a low level syscalls
module like cross_open(), cross_read(), cross_write() and so on, which
just redirect the request into the normal Linux system calls, while
adjusting for the fact that size_t is 16 bits and so forth. This will
be really great.
In case it sounds like this is over-engineering, well bear in mind
that one knotty problem I hadn't yet tackled is the standalone
utilities, for instance the 2.11BSD tape distribution contains a
standalone "restor" program which is essentially a subset of the
kernel, including its device drivers, packaged with the normal
"restor" utility into one executable that can be loaded off the tape.
It was quite important to me that I get this ported over to Linux, so
that I can produce filesystems, etc, at the Linux level, all ready to
boot when I attach them to SIMH. But it was going to be hugely
challenging, since compiling any program that includes more than the
most basic kernel headers would have caused loads of conflicts with
Linux's kernel headers and system calls. So the new approach
completely fixes all that.
I did some experiments the last few days with a program that I created
called "xify". What it does is to read a C file, and to every
identifier it finds, including macro names, C-level identifiers,
include file names, etc, it prepends the sequence "x_". The logic is a
bit convoluted since it has to leave keywords alone and it has to
translate types so that "unsigned int" becomes "x_unsigned_int" which
I can define with a typedef, and so on. Ancient C constructs like
"register i;" were rather problematic, but I have got a satisfactory
prototype system now.
I also decided to focus on 4.3BSD rather than 2.11BSD, since by this
stage I know the internals and the build system extremely intimately,
and I'm aware of quite a lot of inconsistencies which will be a lot of
work to tidy up, basically things that had been hurriedly ported from
4.3BSD while trying not to change the corresponding 2.8~2.10BSD code
too much. Also in the build system there are quite a few different
ways of implementing "make depend" for example, and this annoys me, I
did have some ambitious projects to tidy it all up but it's too
difficult. So a fresh start is good, and I am satisfied with the
2.11BSD project up to this moment.
So what will happen next is basically once I have "-lx_c" (the "cross"
version of the 4.3BSD C library) including the "xified" versions of
the kernel headers, then I will try to get the 4.3BSD kernel running
on top of Linux, it will be a bit like User-Mode Linux. It will use
simulated network devices like libpcap, or basically just whatever
SIMH uses, since I can easily drop in the relevant SIMH code and then
connect it up using the 4.3BSD kernel's devtab. The standalone
utilities like "restor" should then "just work". The cross toolchain
should also "just work" apart from the floating point issue, since it
was previously targeting the VAX which is little-endian, and the
wordsize issues and the library issues are taken care of by "xifying".
Very nice.
The "xifying" stuff is in a new repository 43bsd.git at my bitbucket
(user nick_d2).
cheers, Nick
> From: Paul Ruizendaal
>> On 12 Jan 2017, at 4:54 , Clem Cole wrote:
>> The specifications for what would become IP and TCP were kicking around
>> ... in the late 1970s.
The whole works actually started considerably earlier than that; the roots go
back to 1972, with the formation of the International Packet Network Working
Group - although that group went defunct before TCP/IP itself was developed
under DARPA's lead.
I don't recall the early history well, in detail - there's a long draft
article by Ronda Hauben which goes into it in detail, and there's also "INWG
and the Conception of the Internet: An Eyewitness Account" by Alexander
McKenzie which covers it too.
By 1977 the DARPA-led effort had produced several working prototype
implementations, and TCP/IP (originally there was only TCP, without a separate
data packet carriage layer) were up to version 3.
> My understanding is that all RFC's and IEN's were available to all legit
> users of the Arpanet.
Yes and no. The earliest distribution mechanism (for the initial NCP/ARPANet
work) was hardcopy (you can't distribute things over the 'net before you have
it working :-), and in fact until a recent effort to put them all online, not
all RFC's were available in machine-readable form. (I think some IEN's still
aren't.) So for many of them, if you wanted a copy, you had to have someone at
ISI make a photocopy (although I think they stocked them early on) and
physically mail it to you!
> Apparently nobody had the idea to put all RFC's in a directory and give
> FTP access to it.
I honestly don't recall when that happened; it does seem obvious in
retrospect! Most of us were creating document in online text systems, and it
would have been trivial to make them available in machine-readable form. Old
habits die hard, I guess... :-)
> I think I should put a question out about this, over on the internet
> history mailing list.
Yes, good idea.
> As an aside: IMHO, conceptually the difference between NCP and TCP
> wasn't all that big.
Depends. Yes, the service provided to the _clients_ was very similar (which
can be seen in how similar the NCP and TCP versions of thing like TELNET, FTP,
etc were), but internally, they are very different.
> In my current understanding the big difference that was NCP assumes
> in-order, reliable delivery of packets ... and that TCP allows for
> unreliable links.
Yes, that's pretty accurate (but it does mean that there are _a lot_ of
differences internally - re-transmissions, etc). One other important
difference is that there's no flow control in the underlying network
(something that took years to understand and deal with properly).
> yes, these concepts were kicking around for over a decade in academia
> before BSD.
TCP/IP was the product of a large, well-organized, DARPA-funded and -led
effort which involved industry, academic and government players (the first
two, for the most part, DARPA-funded). So I wouldn't really call it an
'academic' project.
Noel
Thanks Warren for saving me to sort out the confusion. I am sorry I started it in the first place.
On Tue, Jan 10, 2017 at 11:20 AM, Joerg Schilling <schily(a)schily.net <mailto:schily@schily.net>> wrote:
> …. Note that
> this list is very similar to that in the early part of his book on System V
> internals.
Having having just removed the dust off my old copy of TMGE, it is interesting that the list I wrote here is very similar to what I wrote back in 1993. Just goes to show, Alzheimer’s hasn’t got me yet ;)
Ok, the story so far. Berny wrote:
Here's the breakdown of SVR4 kernel lineage as I recall it. ...
From SunOS:
vnodes
VFS
Dan Cross wrote:
> VFSSW <=== NO, this is from SunOS-4
Surely Berny meant the file system switch here, which could have come
from early system V, but originated in research Unix (8th edition?).
Joerg Schilling wrote:
It is rather a part of the VFS interface that has first been completed
with SunOS-3.0 in late 1985.
And this is where the confusion starts. Does "It" refer to FSS or VFS?
I've just looked through some sources. The file system switch was in SysVR3:
uts/3b2/sys/mount.h:
/* Flag bits passed to the mount system call */
#define MS_RDONLY 0x1 /* read only bit */
#define MS_FSS 0x2 /* FSS (4-argument) mount */
VFS was in SunOS 4.1.4:
sys/sys/vfs.h:
struct vfssw {
char *vsw_name; /* type name string */
struct vfsops *vsw_ops; /* filesystem operations vector */
};
And VFS is in SysVR4:
uts/i386/sys/vfs.h:
typedef struct vfssw {
char *vsw_name; /* type name string */
int (*vsw_init)(); /* init routine */
struct vfsops *vsw_vfsops; /* filesystem operations vector */
long vsw_flag; /* flags */
} vfssw_t;
Interestingly, the "filesystem operations vector" comment also occurs
in FreeBSD 5.3, NetBSD-5.0.2 and OpenBSD-4.6. Look for vector here:
http://minnie.tuhs.org/cgi-bin/utree.pl?file=FreeBSD-5.3/sys/sys/mount.hhttp://minnie.tuhs.org/cgi-bin/utree.pl?file=NetBSD-5.0.2/sys/compat/sys/mo…http://minnie.tuhs.org/cgi-bin/utree.pl?file=OpenBSD-4.6/sys/sys/mount.h
Larry wrote:
System Vr3 had something called the file system switch which is what
Berny is talking about. SunOS had virtual file system layer (VFS) and
that would be one of things ported to SVr4.
which is consistent with everybody else.
So now that we have consistency, let's move on.
Cheers, Warren
I have been trolling these many threads lately of interest. So thought I should chip in.
"SVr4 was not based on SunOS, although it incorporated
many of the best features of SunOS 4.x”.
IMHO this statement is almost true (there were many great features from BSD too!).
SunOS 5.0 was ported from SVR4 in early 1991 and released as Solaris 2.0 in 1992 for desktop only.
Back in the late 80s, Sun and AT&T partnered development efforts so it’s no surprise that SunOS morphed into SVR4. Indeed it was Sun and AT&T who were the founding members of Unix International…with an aim to provide direction and unification of SVR4.
I remember when I went to work for Sun (much later in 2003), and found that the code base was remarkably similar to the SVR4 code (if not exact in many areas).
Here’s the breakdown of SVR4 kernel lineage as I recall it. I am pretty sure this is correct. But I am sure many of you will put me right if I am wrong ;)
From BSD:
TCP/IP
C Shell
Sockets
Process groups and job Control
Some signals
FFS in UFS guise
Multi groups/file ownership
Some system calls
COFF
From SunOS:
vnodes
VFS
VM
mmap
LWP and kernel threads
/proc
Dynamic linking extensions
NFS
RPC
XDR
From SVR3:
.so libs
revamped signals and trampoline code
VFSSW
RFS
STREAMS and TLI
IPC (Shared memory, Message queues, semaphores)
Additional features in SVR4 from USL:
new boot process.
ksh
real time extensions
Service access facility
Enhancements to STREAMS
ELF
> I wonder if >pdd ... was in any way any inspiration for /proc?
That may have been a bit too cryptic. "pdd" ('process directory directory')
was a top-level directory in the Multics filesystem which contained a
directory for each process active in the system; each directory contained data
(in segments - roughly, 'files', but Multics didn't have files because it was
a single-level store system) associated with the process, such as its kernel-
and user-mode (effectively - technically, ring-0 and ring-4) stacks, etc.
So if a process was sitting in a system call, you could go into the right
directory in >pdd and look at its kernel stack and see the sequence of
procedure calls (with arguments) that had led it to the point where it
blocked. Etc, etc.
Noel
>Date: Wed, 11 Jan 2017 02:16:26 +0000
>From: Steve Simon <steve(a)quintile.net>
>To: tuhs(a)tuhs.org
>Subject: [TUHS] Rje / sna networking
>Message-ID: <05DECAED-065A-4520-805A-D86CF1596A01(a)quintile.net>
>Content-Type: text/plain; charset=us-ascii
>Casual interest,
>Anyone ever used RJE from SYS-III - IBM mainframe remote job entry
>System? I started on Edition 7 on an interdata so I am (pretty much) too young
>for that era, unless I am fooling myself.
>-Steve
In the 90sh DEC in Europe had a number of products on top of SCO UNIX
3.2V4.2 calling DECadvantage (from the German part of former Philips
Information Systems). Included were an SNA environment with 3270
display/print, 3770/RJE, and APPC. I've used RJE for downloading daily
reports in one of the banks here in Thailand. Long time ago though.
Still have various sample scripts I put together that time.
> /proc came from research
Indeed it did.
> /proc was done by Roger at AT&T (maybe USL). I recall him telling me that
> he was not the original author though and that it came from PWB.
Roger Faulkner's /proc article, recently cited in tuhs, begins with
acknowledgment to Tom Killian, who originated /proc in research.
(That was Tom's spectacular debut when he switched from high-energy
physics, at Argonne IIRC, to CS at Bell Labs.)
doug
