I believe the following can count as an open source operating system (though this won’t satisfy the latter day purists). From Per Brinch-Hansen’s “Monitors and Concurrent Pascal: a Personal History” (1993):
“At Caltech we prepared a distribution tape with the source text and portable code of the Solo system, including the Concurrent and Sequential Pascal compilers. The system reports were supplemented by implementation notes (Brinch Hansen 1976b).
By the spring of 1976 we had distributed the system to 75 companies and 100 universities in 21 countries: Australia, Austria, Belgium, Canada, Denmark, Finland, France, Germany, Great Britain, Holland, India, Ireland, Italy, Japan, Norway, South Africa, the Soviet Union, Spain, Sweden, Switzerland, and the United States.”
This retrospective paper is worth reading in this age where the quest of higher and higher performance has produced super fast but very complicated and insecure machines where even synchronization has become a tricky affair (see for example the recent three articles by Russ Cox on his research.swtch.com site).
Can’t resist quoting Charles Hayden’s (Solocomment from the paper:
“What was remarkable about [Concurrent Pascal] is that one could write experimental operating systems on a virtual machine without having to resort to machine registers, assembly language, etc. The development environment provided a way to do operating systems in a controlled way, on the “bare hardware” of a much nicer machine than any real computer. . .
I think the significance of the system was . . . that one could provide a protected environment for concurrent programming—a high-level language environment which could maintain the illusion that there was no “machine” level. It was remarkable that through compile time restrictions and virtual machine error checking, that you could understand the program behavior by looking at the Pascal, not at the machine’s registers and memory. It was remarkable that the machine could retain its integrity while programs were being developed, without hardware memory protection.”
Nowadays writing an os kernel is considered quite a major effort. IMHO there has been nothing new in this area from a programming perspective since the ‘70s and no guidance for h/w design which has become increasingly more complex and “magic” (as per Artur C Clarke’s definition).