6 Dec
2024
6 Dec
'24
8:21 a.m.
On Thu, Dec 5, 2024 at 9:02 PM John Levine <johnl(a)taugh.com> wrote:
According to Dan Cross <crossd(a)gmail.com>:
And indeed these things are pretty nifty, but don't they generate
trees, and not arbitrary dags? They don't quite capture the full
generality of Morrison-style networks since it doesn't seem like
there's a way to connect process substitution fan-out with fan-in; at
least, not conveniently.
You can use mkfifo to make named pipes and then plumb them as wanted. It's
not particularly beautiful but it's as general as you could want.
That's worked since 4.4BSD which was quite a while ago.
Sure, but that's orthogonal since it's not really using shell syntax
to build the thing up in a canonical way. I mean, perhaps in some
literal sense it is (one types the commands into the interpreter...)
but nowhere near the expressiveness of `ls | grep foo | whatever`.
Fundamentally, the questions here are all about syntax. As I wrote in
my first email, the system calls are very general, and could be used
to build arbitrary directed graphs (not even limited to DAGs) of
processes connected by pipes in all sorts of configurations
(presumably those processes would be written to handle multiple input
and output streams), without having to resort to named pipes. One can
imagine some sort of DSL that describes such graphs of pipes and
processes (Bakul alluded to this), and an interpreter for that DSL
that creates the described set of pipes and processes, all
appropriately connected and running. One could even imagine such a
thing being used as part of a more traditional pipeline.
My claim is simply that extant shell syntax isn't really amenable to
this in a natural way. As Chet and Arnold pointed out, process
substitution as pioneered in ksh gets us a little closer, but it's not
quite as general (I believe any such graphs would have to be acyclic);
it's certainly not as syntactically pleasant.
There has been work along these lines; I was sent a reference off-list
to a paper by Spinellis and Fragkoulis about a DAG-oriented shell:
https://www.spinellis.gr/sw/dgsh/, which seems relevant.
- Dan C.
diff <(old-program-version)
<(new-program-version)
to do simple regression testing.
And fanout is simply
... | tee >(pipeline1) >(pipeline2)