The UNIX* Operating System

After three decades of use, the UNIX* (link to Unix-Trademark file) computer operating system is still regarded as the most powerful, versatile, and flexible operating system (OS) in the computer world.

Like another legendary creature whose name also ends in 'x,' UNIX rose from the ashes of a multi-organizational effort in the early 1960s to develop a dependable timesharing operating system.

The joint effort failed, but a few survivors from Bell Labs tried again, and what followed was a system that offers its users a work environment that has been described as "of unusual simplicity, power, and elegance...."

The system also fostered a distinctive approach to software design -- solving a problem by interconnecting a few off-the-shelf software programs, rather than by doing it by hand or writing special purpose programs or subsystems.

Today, ...(I'd like to list a few current uses of UNIX -- at a very high level. AT&T Labs claims on their web site UNIX is the basis for the Internbet today. That would be an interesting "grabber.")

It's development and evolution led to a new philosophy of computing, and it has been a never-ending source of both challenges and joy to programmers around the world.

Before Multics there was chaos, and afterwards, too

Computer systems didn't talk to each other in the early days of computing. Even the various computer lines made by the same company often needed interpreters. And forget any interoperability of systems by different vendors!

In addition, operating systems very often performed only limited tasks, and only on the machines for which they were written. If a business upgraded to a bigger, more powerful computer, the old operating system probably wouldn't work on the new computer, and often the company's data had to entered -- again -- into the new machine.

Programmers also talked wistfully then of computers "doing useful work," rather than performing impressive but not necessarily productive functions designed to demonstrate a programmer's skills.

To try to bring an end to the chaos that was plaguing the fledgling computer industry, a group of computer scientists from Bell Labs and GE in 1965 joined an effort underway at MIT on what was called the Multics (Multiplexed Information and Computing Service) mainframe timesharing system. The objective was to develop a convenient, interactive, useable computer system that could support many users.

Over time, hope was replaced by frustration as the group effort initially failed to produce an economically useful system. Bell Labs withdrew from the effort in 1969 but a small band of users at Bell Labs Computing Science Research Center in Murray Hill -- Ken Thompson, Dennis Ritchie, Doug McIlroy, and J. F. Ossanna -- continued to seek the Holy Grail.

From Multics to something else

Actually, Multics worked, and eventually became a product, but not initially on the scale its developers wanted. "Even though Multics could not then support many users, it could support us, albeit at exorbitant cost," Ritchie explained. "We didn't want to lose the pleasant niche we occupied. What we wanted to preserve was just not a good environment in which to do programming, but a system around which a fellowship could form."1(Link to UNIX - Sources file)

"During 1969, we began trying to find an alternative to Multics," Ritchie said. "Throughout 1969, we lobbied for a medium-scale machine for which we promised to write an operating system. Our proposals were never clearly and finally turned down, but they were never accepted, either.

"Eventually, we presented an exquisitely complicated proposal involving outright purchase, third-party lease, and equipment trade-in, all designed to minimize financial outlay," Ritchie said, but the proposal was rejected. "Rumor soon had it that Bill Baker, the vice president of Research, exclaimed 'Bell Labs just doesn't do business this way!'"

Ritchie conceeded, "Actually, it is perfectly obvious in retrospect -- and it should have been at the time -- that we were asking the Labs to spend too much money on too few people with two vague a plan." He also noted that buying a new machine might lead to another expensive Multics project, which management wanted to avoid, or developing another computer center, a responsibilty which Research wanted to avoid..

In the beginning: botched acronyms

The origins to UNIX can be traced back, somewhat fuzzily, to the early spring of 1969 during an informal discussion of just what then researchers wanted a computer operating system to do.. "Dennis, (Rudd) Canaday and myself were just discussing these ideas of the general nature, of keeping the files out of each other's hair, and the nitty-gritty of expanding, of the real implementation: where you put the block address ...", Thompson explained. 2 (Link to UNIX - Sources file)

At the end of the discussion, Canaday picked up the phone, dialed into a Bell Labs dictation service, and read in his notes. "The next day these notes came back," Thompson said, "and all the acronyms were butchered, like 'inode' and 'eyen,'" for (???What should they have been???)

Butchered or not, the notes became the basis for UNIX. Each researcher received a copy of the notes "...and they became the working document for the file system," he said.

The infamous PDP-7 comes to the rescue

While mulling over the problems of operating systems in 1969, Thompson in his spare time developed a computer game called "Space Travel." The game simulated the motion of the planets in the solar system. A player could cruise between the plants, enjoy the scenery, and even land the ship on the planets and moons.

The game, first written on Multics and then transliterated into Fortran for the GECOS operating system, ran on a GE 635 computer. The game's display was jerky and hard to control because the player had to type commands to control the ship -- the computer mouse hadn't been invented then. Also, it cost about $75 in CPU time on the big GE 635, a cost that hardly endeared it to management.

"It did not take long, therefore, for Thompson to find a little-used PDP-7 computer with an excellent display terminal," Ritchie explained. "He and I rewrote Space Travel to run on this machine." Their effort included a floating-point arithmetic package, the pointwise specification of the graphics characters for the display, and a de-bugging subsystem that continuously displayed the contents of typed-in locations in the corner of the screen.

"All this was written in assembly language for a cross-assembler that ran under GECOS and produced paper tapes to be carried to the PDP-7," Ritchie said. "Space Travel, though it made a very attractive game, served mainly as an introduction to the clumsy technology of preparing programs for the PDP-7."

"It was the natural candidate as the place to put the file system," Thompson said. "When we hacked out this design, this rough design of the file system on the dictation machine that day in Canaday's office, I went off and implemented it on the PDP-7."

The UNIX system begins to take shape

What Thompson had on the PDP-7 was a system, but not really an operating system. It needed some fleshing out.

So during the summer of 1969, Thompson began implementing the paper file system, which Ritchie liked to refer to as the "chalk file system," since it emerged from countless discussions at chalkboards in the Computing Science Research Center.

Thompson's wife, Bonnie, took their year-old son to visit Thompson's parents in California, so Thompson, who preferred then to working in the afternoon and evening hours, had few distractions. "I allocated a week each to the operating system, the shell, the editor, and the assembler to reproduce itself..."

Thompson first fleshed out the requirements for an operating system, in particular the notion of processes. Then he developed a small set of user-level utilities: the means to copy, print, delete and edit files. And he developed a command interpreter, or shell.
 

It looked like an operating system, almost


The initial implementation "was totally rewritten in a form that looked like an operating system," Thompson said, "with tools that were sort of known, you know, assembler, editor, shell." The system, "...if not maintaining itself, was right on the verge of maintaining itself, totally severing the GECOS connection."

He was referring to the fact that up to this point, all the programs were written using GECOS and transferred to the PDP-7 by paper tape. But once an assembler was completed, the system was able to support itself.

As Ritchie summed up the effort, "Although it was not until well into 1970 that Brian Kernighan suggested the name 'UNIX,' in a somewhat treacherous pun on 'Multics,' the operating system we know today was born."

C language follows NB, which follows B

The first version of UNIX was written in assembler language, but Thompson's intention was that it would be written in a high-level language.

Thompson first tried to use Fortran, but gave up after the first day. Then he wrote a very simple language he called B, which he got going on the PDP-7. It worked, but there were problems. First, because the implementation was interpreted, it was always going to be slow. Second, the basic notions of B, which was based on the word-oriented BCPL, just were not right for a byte-oriented machine like the PDP-7.

Ritchie started to add types to B, which for a while was called NB for "New B," and then started to write a compiler for it. "So that the first phase of C was really these two phases in short succession of, first, some language changes from B, really, adding the type structure without too much change in the syntax; and doing the compiler," Ritchie said.

"The second phase was slower," he said of rewriting UNIX in C. Thompson started in the summer of 1972 but had two problems: figuring out how to run the basic co-routines, that is, how to switch control from one process to another; and the difficulty in getting the proper data structure, since the original version of C did not have strutures.

"The combination of the things caused Ken to give up over the summer," Ritchie said. "Over the year, I added structures and probably made the complier code somewhat better -- better code -- and so over the next summer, that was when we made the concerted effort and actually did redo the whole operating system in C."

Connecting streams like a garden hose

Another innovation of UNIX was the development of pipes, which gave programmers the ability to string together a number of prcesses for a specific output.

Doug McIlroy, then a department head in the Computing Science Research Center, is credited for the concept of pipes (link to UNIX-Dartmouth file) at Bell Labs, and Thompson gets the credit for actually doing it.

McIlroy had been working on macros in the later 1950s, and was always theorizing to anyone who would listen about linking macros together to eliminate the need to make a series of discrete commands to obtain an end result.

"If you think about macros," McIlroy explained, "they mainly involve switching data streams. I mean, you're taking input and you suddenly come to a macro call, and that says, 'Stop taking input from here and go take it from there.'

"Somewhere at that time I talked of a macro as a 'switchyard for data streams,' and there's a paper hanging in Brian Kernighan's office still, which he dredged up from somewhere, where I talked about screwing together streams like a garden hose. So this idea had been banging around in my head for a long time."

Back to the chalkboards to work out the synactics

While Thompson and Ritchie were at the chalkboard sketching out a file system, McIlroy was at his own chalkboard trying to sketch out how to connect processes together and to work out a prefex notation language to do it.

It wasn't easy. "It's very easy to say 'cat into grep into...,' or 'who into cat into grep,'" McIlroy explained. "But there are all these side parameters that these commands have; they just don't have input and output arguments, but they have all these options."

"Synactically, it was not clear how to stick the options into this chain of things written in prefix notation, cat of grep of who [i.e. cat)grep(who))]," he said. "Synactic blinders: I didn't see how to do it."

"I'm going to do it,' and so he did

Although stymied, McIlroy didn't drop the idea. "And over a period from 1970 to 1972, I'd from time to time say, 'How about making something like this?', and I'd put up another proposal, another proposal, another proposal. And one day I came up with a syntax for the shell that went along with the piping, and Ken said, 'I'm going to do it!'"

"He was tired of hearing this  stuff," McIlroy explained."He didn't do exactly what I had proposed for the pipe system call. He invented a slightly better one that finally got changed once more to what we have today. He did use my clumsy syntax."

"Thompson saw that file arguments weren't going to fit with this scheme of things and he went in and changed all those programs in the same night. I don't know how...and the next morning we had this orgy of one-liners."

"He put pipes into UNIX, he put this notation into shell, all in one night," McElroy said.

Creating the tool box approach to programming

As technically neat as the accomplishment was, when Thompson created pipes, he also put something else into UNIX -- a philosophy.

As McIlroy described it, "the philosophy that everyone started to put forth" was 'Write programs that do one thing and do it well. Write programs to work together. Write programs that handle text streams, because that is a universal interface.'"

"All of these ideas, which add up to the tool approach, might have been there in unformed way prior to pipes, but they really came in afterwards."

Kernighan agreed. He noted that while input/output direction predates pipes, the development of pipes led to the concept of tools, software programs that would be in a "tool box," available when you need them.

He noted that pipes made software programs somewhat analogous to working with Roman numerals instead of Arabic numerals. "It's not that you can't do arithmetic," he said, "but it is a bear."

"I remember the preposterous syntax, the ">,>" or whatever the syntax that everyone came up with, and then all of a sudden there was the vertical bar ( | ) and just everything clicked at that point," he said. The bar was the syntax that made pipes work: who | cat | grep.

"And that's, I think, when we started to think consciously about tools, because then you could compose things together....compose them at the keyboard and get 'em right every time."

'What you think is going on, is going on'

Novices to the system could experiment, linking different commands together for what they thought should be the output. And very often their "pipes" worked the first time!

When Joe Condon, the owner of the PDP-7 that Thompson first used for UNIX, started using UNIX himself, he asked a co-worker how to do something. "'What do you think is the reasonable thing to do, Joe?'" he was asked in return.

"That was a very interesting clue to the philosophy of how UNIX worked," Condon later said. "The system worked in a way which is easy to understand. It wasn't a complex function, hidden in a bunch of rules and verbiage and what not."

"Cognitive engineering" is what Condon called it, "...that the black box should be simple enough such that when you form the model of what's going on in the black box, that's in fact what is going on in the black box."

The manual even warned of bugs

Most manuals are often an afterthought, something cobbled together after the product is made. The UNIX manual, in contrast, reflected the philosopy of UNIX in design and content. It even told you where the bugs were.

The manual style initially was set by Ritchie, but McIlroy soon took over its compilation as a love of labor. "The fact that there was a manual, that he (McIlroy) insisted on a high standard for the manual, meant that he insisted on a high standard for every one of the programs that was documented," explained Sandy Fraser.

Fraser, then a member of technical staff in the Computing Science Research Center, said that before a program got into the manual, it often had to be rewritten to meet the manual's standards. "And then add to all that, it's probably the first manual that ever had a section with bugs  in it. That's a level of honesty you don't find."

(Illustration of manual page AS(1) from Sys V, Rel 2.0 with BUGS highlighted)

Caption:  The manual for System V, R 2.0 warned that the as assembler command on a PDP-11 was susceptible to syntax errors.

"Cleaning something up so you can talk about it is quite typical of UNIX," McIlroy said. "Every time another edition of the manual would be made, there would be a flurry of activity. When you wrote down the 'uglies,' you'd say, 'We can't put this in print,' and you'd take out a feature or put features in to make them easier to talk about."

Fraser summed it up the approach of the UNIX developers: "I think the level of intellectual honesty that was present in that activity is rare."

Porting UNIX for its first commercial application

It soon became obvious that the PDP-7 machine, which the UNIX group didn't own, was becoming obsolete. In 1970, they proposed buying a PDP-11 for about $65,000, which arrived by the end of the summer. The system was so new no disk was available at the time, so the effort to port UNIX didn't begin until December.

(Insert pix now at http://cm.bell-labs.com/cm/cs/who/dmr/picture.html )

Caption:  Dennis Ritchie (standing) and Ken Thompson begin porting UNIX to the PDP-11 via two Teletype 33 terminals.

"During the protracted arrival of the hardware," Ritchie said, "the increasing usefulness of the PDP-7 UNIX made it appropriate to justify creating PDP-11 UNIX as a development tool, to be used in writing a more special-purpose system, text processing."

The first potential customer was Bell Labs Patent Department, which was evaluating a commercial system to prepare patent applications. In developing UNIX to support text processing, the Computing Science Research Center supported three Patent typists who spent the day busily typing, editing, and formatting patent applications, Ritchie said. "The experiment was trying, but successful."

"Not only did the Patent Department adopt UNIX, and thus become the first of many groups at the Laboratories to ratify our work, but we acquired sufficient credibility to convince our own management to acquire one of the first PDP 11/45 systems made."

"The rest," Ritchie said, "is history."

Sharing UNIX with the rest of the world

In 1976-77, Ken Thompson took a six-month sabbatical from Bell Labs to teach as a visiting professor at the Computer Science Department at the University of California-Berkeley. What he taught, of course, was the UNIX system. While there, he also developed much of what eventually became Version 6.

The system was an instant hit, and the wordspread quickly throughout the academic community.

When Thompson returned to Bell Labs, students and professors at Berkeley continued to enhance UNIX. Eventually, many of these enhancements were incorporated into what became known as Berkeley Software Distribution (BSD) Version 4.2, which many other universities also bought.

(???Is there more to be said about this period???)
(???Are there any photos from Berkeley we could use???)

Business gets the word

The germ of UNIX was planted at Berkeley. Businesses eventually learned of UNIX from their newly hired programmers who had used it in college.

Soon a new business opportunity developed -- writing programs to run on UNIX for commercial use. What made UNIX popular for business applications was its time-sharing, multitasking capability, permitting many peope to use the mini or mainframe; its portability across different vendor's machines; and its e-mail capability. E-mail has taken the business world by storm within the last 10 years, but UNIX users have been communicating with others all over the globe for the last quarter century or more.
 

At Berkeley, as elsewhere, computer scientists started "improving" UNIX, adding new features and applications, revising code, trying to push this versatile operating system to its limits. Over time, a number of "versions" of UNIX were floating around, including Bell Labs official versions.
 

Early versions of UNIX system

Version

Version 6

Year released

1975

Applications

Universities
Version 7 1978 Universities and commercial. The basis for System V.
System III 1981 Commercial
System V, Release 1 1983 Commercial
System V, Release 2 1984 Commercial, enhancements and performance improvements
Version 8 ??? Universities
Version 9 ??? Universities
Version 10 ??? Universities

(Am I missing some versions here?)

In addition, there are university versions, most notably UNIX-Berkeley, as well as implementations, such as Carnegie Mellon's MACH.

The UNIX wars

As the versions of UNIX grew in number, the UNIX System Group, organized to develop commercial applications of UNIX, agreed with another licensee to work together to pull the best from all versions into a unified UNIX. While many applauded this decision, a group of licensees expressed the fear that Bell Labs proposed partner would have a commercial advantage over them.

The concerned group formed a special interest group, the Open Systems Foundation (OSF), to lobby for an "open" UNIX within the UNIX community. Soon several large companies -- who at the time were promoting their own proprietary operating systems in competition to UNIX -- also joined the OSF.

In response, the then-parent company of Bell Labs, AT&T, and a second group of licensees formed their own group, UNIX International. The technical issues soon took second seat to what can be charitably described as competitive maneuverings, and the trade press dubbed the ensuing controversary the "UNIX wars."

When efforts failed to bring the two groups together, each group brought out its own version of an "open" UNIX. Media wags soon noted the dispute could be viewed two ways: positively, since the number of UNIX versions were now reduced to two; or negatively, since there now were two more versions of UNIX to add to the pile.

UNIX moves on

(I'm not sure I have the correct chronology of USG sale and trivestiture. I'm checking.)

When AT&T announced in 1995 that it was planning to divest itself of many of its equipment manufacturing operations that eventually become the core of the new Lucent Technologies, it also announced that it would sell its NCR subsidiary and get out of the computer business.

About the same time, AT&T decided to "divest" itself of UNIX as well, and sold the UNIX System Group to the Novell Corporation. Novell later sold the UNIX business to the Santa Cruz Organization, while UNIX standards and trademark issues are handled by the The Open Forum.

Meanwhile, the Bell Labs Computing Science Research Center continued their development efforts on UNIX, eventually developing Versions 8, 9 and 10.

Where UNIX stands today

The successes of UNIX are intertwined with C, the first general-purpose programming language to combine the efficiency of assembly language with high-level abstract expressiveness. Like UNIX, C programs can move essentially without change from machine to machine, eliminating the need for expensive, error-prone software  rewrites.

UNIX-based systems are sold today by a number of companies. The systems include the Solaris* system from Sun Microsystems, HPUX* from Hewlett-Packard, AIX ??? from IBM, Ultrix* from Compaq, and ??? from SGI. In addition there are many open-source, UNIX-compatible implementations, such as Microsoft's Xenix, Linux, 3BSD and NetBSD (???Names of companies that developed implementations???).

UNIX is the operating system of most large Internet servers, businesses and universities, and a major part of academic and industrial research in operating systems is based on UNIX. Most commercial software is written in C or C++, a direct descendant of C that was also developed at Bell Labs, or more recently Java, a C++ descendant developed at Sun Microsystems.

Much of the progress of computer hardware, software and networks during the last quarter century can be traced to the UNIX system. It embodies visionary ideas -- deliberate generality and openness -- that continue to be a strong force today. Many of its approaches and notations have influenced the entire span of subsequent operating systems.

"Thirty years after its creation, UNIX still remains a phenomenon," Ritchie marvelled.

###

* Product names are the registered marks of their respective companies.

.........