6 Dec
2024
6 Dec
'24
3:50 a.m.
Some of the posts in this thread confuse pipes in UNIX shells (the |
symbol) with the pipe system call.
In the shell, how two processes can be connected with a pipe is very
constrained (only one unidirectional pipe). But the pipe system call can be
used to build much more elaborate connections. Back in 1980, when I was
still at Bell Labs, I wrote a shell called 2dsh ("two dimensional shell)
that had a more complex syntax. The memo I wrote, "2DSH—An experimental
shell for connecting processes with multiple data streams", wasn't
published externally, but exists as a Bell Labs memo. I found a reference
here:
https://scholar.google.com/scholar_lookup?title=2DSH%E2%80%94An+experimenta…
.
Here are two examples from that memo:
[image: image.png]
I stumbled across another paper from 2017 titled "Extending Unix Pipelines
to DAGs," which references my un-published Bell Labs memo. I haven't read
it since I don't subscribe to IEEE Transactions on Computers.
A while ago Doug McIlroy was kind enough to send me a scan of my memo, but
I don't think I'm allowed to publish it here. In that memo, I credit Doug
for coming with a very similar idea around the same time ("A Notation for
Arboreal Plumbing").
Marc Rochkind
On Thu, Dec 5, 2024 at 2:13 PM Dan Cross <crossd(a)gmail.com> wrote:
On Thu, Dec 5, 2024 at 3:56 PM
<arnold(a)skeeve.com> wrote:
And fanout is simply
... | tee >(pipeline1) >(pipeline2)
And indeed these things are pretty nifty, but don't they generate
trees, and not arbitrary dags? They don't quite capture the full
generality of Morrison-style networks since it doesn't seem like
there's a way to connect process substitution fan-out with fan-in; at
least, not conveniently.
It is, perhaps, notable that Go allows me to do this sort of thing
with channels and goroutines, but it has `select` built into the
language.
Funny, despite using Unix almost daily for over 30 years now, I don't
think I've ever felt limited by the power of pipelines. On the
contrary, I've lost count of the times I've felt limited on systems
that do Not support pipes.
- Dan C.
--
*My new email address is mrochkind(a)gmail.com <mrochkind(a)gmail.com>*
Chet Ramey via TUHS <tuhs(a)tuhs.org> wrote:
> On 12/5/24 10:19 AM, Dan Cross wrote:
>
> > Unix pipelines, on the other hand, tend to be used in a manner that
is
> > strictly linear, without the fan-out
and fan-in capabilities
described
> > by Morrison. Of course, nothing
prevents one from building a
> > Morrison-style "network" from Unix processes and pipes, though
it's
> > hard to see how that would work without something like `select`,
which
didn't yet exist in 1978. Regardless, Unix still
doesn't expose a
particularly convenient syntax for expressing these sorts of
constructions at the shell.
Process substitution is about as close as we can get, but most programs
still process their filename arguments one at a time, beginning to end.
The canonical process substitution example is
diff <(old-program-version) <(new-program-version)
to do simple regression testing. 