4 Aug
2021
4 Aug
'21
11:48 p.m.
What do folks think about event-driven programming as a substitute for threads in UI and
process control settings?
I wrote the service processor code for the Sicortex Machines using libevent.a and I
thought it was very lightweight and fairly easy to think about. (This was a thing running
on ucLinux on a tiny 16 MB coldfire that managed the consoles and power supplies and temp
sensors and JTAG access and booting and so forth.)
Tk (IIRC) has a straightforward event driven model for UI interactions.
Meanwhile, the dropbox plugin for my Mac has 120 threads running. WTF?
This was triggered by the fork/spawn discussion.
-Larry
(started with Unix at V6 on an 11/34)
5 Aug
5 Aug
12:02 a.m.
A computer is a state machine. Threads are for people who can't program
state machines.
Alan Cox
On Wed, Aug 04, 2021 at 05:48:46PM -0400, Lawrence Stewart wrote:
What do folks think about event-driven programming as
a substitute for threads in UI and process control settings?
I wrote the service processor code for the Sicortex Machines using libevent.a and I
thought it was very lightweight and fairly easy to think about. (This was a thing running
on ucLinux on a tiny 16 MB coldfire that managed the consoles and power supplies and temp
sensors and JTAG access and booting and so forth.)
Tk (IIRC) has a straightforward event driven model for UI interactions.
Meanwhile, the dropbox plugin for my Mac has 120 threads running. WTF?
This was triggered by the fork/spawn discussion.
-Larry
(started with Unix at V6 on an 11/34)
--
---
Larry McVoy lm at mcvoy.com http://www.mcvoy.com/lm
12:41 a.m.
On Wed, Aug 4, 2021 at 6:02 PM Larry McVoy <lm(a)mcvoy.com> wrote:
A computer is a state machine. Threads are for
people who can't
program
state machines.
Alan Cox
Orly? Try embedding an LL(1) parser in an event loop that gives you a new
event every time a block is read off the disk.
Event loops are just manual CPS transformations of coroutines -- but why do
the transformation manually instead of having your compiler do it for you?
12:51 a.m.
On Wed, Aug 04, 2021 at 06:41:08PM -0400, John Cowan wrote:
On Wed, Aug 4, 2021 at 6:02 PM Larry McVoy
<lm(a)mcvoy.com> wrote:
The counter to this is Solaris tried to allocate a thread for each 8K page
on it's way to disk. The thread stack was 16K. This model, while seen as
ever so elegant, means that 2/3rds of main memory were thread stacks.
Sometimes threads make sense, there they did not.
--
---
Larry McVoy lm at mcvoy.com http://www.mcvoy.com/lm
A computer is a state machine. Threads are
for people who can't
program
state machines.
Alan Cox
Orly? Try embedding an LL(1) parser in an event loop that gives you a new
event every time a block is read off the disk.
Event loops are just manual CPS transformations of coroutines -- but why do
the transformation manually instead of having your compiler do it for you? 
10:32 p.m.
On Wed, Aug 4, 2021 at 6:51 PM Larry McVoy <lm(a)mcvoy.com> wrote:
Sometimes threads make sense, there they did not.
Right. The problem is too often people have a hammer and make everything a
nail. Or they have an electric screwdriver and then try to use if for
everything. The trick is to teach good taste and when certain solutions
make more sense than others. Allocating a thread for every page being
sent to disk, or needing 120 threads for a simple application -- somebody
missed the boat.
Funny, used to deliver nails in kegs. Hit with a hammer - wood fastened.
Somebody develops a screw system. A 'better fastener.' Indeed it does work
great for >>some jobs<< but not all. Frankly that sucks for framing.
What do you see at any real job site. Not screws but a pneumatic hammer
and nail cartridges? Changed the deliver system from key and forcing
somebody to swing the hammer.
Lesson learned to *use the proper tool *or update the tool if the
>technique<< is the right one.
Kids these days ... get off my lawn...
Clem
6 Aug
6 Aug
2:55 a.m.
On Wed, Aug 4, 2021 at 6:51 PM Larry McVoy <lm(a)mcvoy.com> wrote:
On Wed, Aug 04, 2021 at 06:41:08PM -0400, John Cowan
wrote:
Honestly this sounds like a misapplication of threads. It makes more sense
to me to have a small pool of workers working on a bounded queue or
something.
Sometimes threads make sense, there they did not.
It strikes me that the Unix kernel we all know and love has been
multithreaded since the early 1970s, when multiprogramming overlapped with
IO was introduced; even in something as simple as 6th Edition, every
process has a corresponding kernel thread and the fundamental mechanism to
context switch from process $n$ to process $m$ is for n to trap into the
kernel where it's running on n's kernel thread, invoke the scheduler to
perform a thread switch to m's kernel thread, and then return to m's
userspace.
Various attempts at replacing multithreaded kernels with event-driven state
machines have been proposed and tried, but in the end, I always come away
with the sense that most of the time, when used judiciously, threads are
wonderful concurrency primitives.
- Dan C.
On Wed, Aug 4, 2021 at 6:02 PM Larry McVoy
<lm(a)mcvoy.com> wrote:
new
A computer is a state machine. Threads are
for people who can't
program state machines.
Alan Cox
Orly? Try embedding an LL(1) parser in an event loop that gives you a event every time a block is read off the disk.
Event loops are just manual CPS transformations of coroutines -- but why
do
the transformation manually instead of having
your compiler do it for
you?
The counter to this is Solaris tried to allocate a thread for each 8K page
on it's way to disk. The thread stack was 16K. This model, while seen as
ever so elegant, means that 2/3rds of main memory were thread stacks.
5 Aug
5 Aug
12:13 a.m.
Lawrence Stewart <stewart(a)serissa.com> once said:
What do folks think about event-driven programming as
a substitute for
threads in UI and process control settings?
Why not both?
https://swtch.com/~rsc/talks/threads07/
https://swtch.com/~rsc/thread/cws.pdf
Cheers,
Anthony
6 Aug
6 Aug
1:02 a.m.
On Aug 4, 2021, at 2:48 PM, Lawrence Stewart <stewart(a)serissa.com> wrote:
What do folks think about event-driven programming as a substitute for threads in UI and
process control settings?
The equivalence here is with coroutines (sometimes called
green-threads), not true threads. True threads can take
advantage of multiple cores. Your event-driven program won't
benefit from multiple cores.
I wrote the service processor code for the Sicortex
Machines using libevent.a and I thought it was very lightweight and fairly easy to think
about. (This was a thing running on ucLinux on a tiny 16 MB coldfire that managed the
consoles and power supplies and temp sensors and JTAG access and booting and so forth.)
I wrote the RealNetworks media server code using an
event-driven model. It was efficient but the control flow got
quite messy as effectively my code had to do explicit
continuation passing. Given that each media stream was
independent of other streams, a thread based model was far
simpler. That is what a colleague did to test the server
code!
As a thought experiment consider implementing a Unix style
pipeline but without an OS that provides the illusion of
independent processes or threads and where each stage in the
pipeline is a separate function. Try implementing this in Go
which has support for threads vs C (but without pthreads).
I have used coroutines for hardware simulation a number of
times, the first time was in 1982 for simulating bus
transactions to see the impact of adding an ethernet
controller (5.6Mhz clock, memory access 4 clocks). The last
time was to simulate an ethernet switch with complex i/o
logic. One can think of each hardware module as a separate
state machine but smushing them into one giant state machine
would not been easy as the code was being developed and
debugging it would be very painful. One would then be tempted
to write a "preprocessor" language of some sort to hide all
this....
Tk (IIRC) has a straightforward event driven model for
UI interactions.
Meanwhile, the dropbox plugin for my Mac has 120 threads running. WTF?
Any system can be abused but the abuse-case should not be
mistaken for the normal use-case.
4:19 p.m.
Bakul Shah wrote in
<74E8349E-95A4-40C9-B429-11A4E396BE12(a)iitbombay.org>:
|On Aug 4, 2021, at 2:48 PM, Lawrence Stewart <stewart(a)serissa.com> wrote:
|>
|> What do folks think about event-driven programming as a substitute \
|> for threads in UI and process control settings?
|
|The equivalence here is with coroutines (sometimes called
|green-threads), not true threads. True threads can take
|advantage of multiple cores. Your event-driven program won't
|benefit from multiple cores.
|
|> I wrote the service processor code for the Sicortex Machines using \
|> libevent.a and I thought it was very lightweight and fairly easy \
|> to think about. (This was a thing running on ucLinux on a tiny 16 \
|> MB coldfire that managed the consoles and power supplies and temp \
|> sensors and JTAG access and booting and so forth.)
|
|I wrote the RealNetworks media server code using an
|event-driven model. It was efficient but the control flow got
|quite messy as effectively my code had to do explicit
|continuation passing. Given that each media stream was
|independent of other streams, a thread based model was far
|simpler. That is what a colleague did to test the server
|code!
Only twenty years ago but i was under the impression that i got
good (better) performance by having a single event loop object
(thread) doing the select(2) aka the OS interaction, as the driver
under the hood of an IOEvent thing, which then were emitted.
These then dispatched to worker threads, or not. It may be
different today with those highly refined mechanisms poll, epoll,
kqueue. It may anyway be different with whatever operating system
specific things, for example FreeBSD had or has some network ring
where user and kernel space could simply exchange buffers, ?, well
i remember it was written by an Italien (Professor?) living in
Pisa, in sight of the lopsided tower. And then the audioring
technology Creative Labs shipped with X-Fi ~2005.
--steffen
|
|Der Kragenbaer, The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
4:43 p.m.
On Fri, Aug 6, 2021 at 10:20 AM Steffen Nurpmeso <steffen(a)sdaoden.eu> wrote:
Bakul Shah wrote in
<74E8349E-95A4-40C9-B429-11A4E396BE12(a)iitbombay.org>:
|It was efficient but the control flow got
|quite messy as effectively my code had to do explicit
|continuation passing.
Only twenty years ago but i was under the impression that i got
good (better) performance by having a single event loop object
(thread) doing the select(2) aka the OS interaction, as the driver
under the hood of an IOEvent thing, which then were emitted.
It sounds like you are in violent agreement: you get more performance with
an event loop. So perhaps Cox's claim is true after all: threads are for
programmers who can't deal with events. And then my claim is that at a
certain not-very-high level of complexity no one can deal with events.
After all, the interface to all modern operating systems is as a coroutine:
when we call read() we give up control to the kernel, who does the read for
us and delivers it to a buffer in our space, then surrenders control back.
(How it looks from inside the kernel is another story.) Only when we
request signal handling does the kernel actually interrupt the ordered flow
within our program, but the restrictions on what a signal handler can do in
Posix are so many that it's hardly safe to do more than set a flag to be
polled later or to do a longjmp() to invoke a continuation further up the
stack.
Historic operating systems worked quite differently: they had the
equivalent of signals that meant "disk block has been read" (use the
buffer, and fast, before the OS starts to write the next block into it) and
"disk block has been written" (the buffer is available for reuse). And
programming for them was, in someone's memorable phrase, like moving dead
whales along the beach by kicking them.
5:12 p.m.
John Cowan wrote in
<CAD2gp_SH5qt2G0oyHeDYnQbjmHeejz9A8FJhCvBFOKq21aAVkg(a)mail.gmail.com>:
|On Fri, Aug 6, 2021 at 10:20 AM Steffen Nurpmeso <steffen(a)sdaoden.eu> \
|wrote:
|> Bakul Shah wrote in
|> <74E8349E-95A4-40C9-B429-11A4E396BE12(a)iitbombay.org>:
|>|It was efficient but the control flow got
|>|quite messy as effectively my code had to do explicit
|>|continuation passing.
|>
|> Only twenty years ago but i was under the impression that i got
|> good (better) performance by having a single event loop object
|> (thread) doing the select(2) aka the OS interaction, as the driver
|> under the hood of an IOEvent thing, which then were emitted.
|>
|
|It sounds like you are in violent agreement: you get more performance with
|an event loop. So perhaps Cox's claim is true after all: threads are for
|programmers who can't deal with events. And then my claim is that at a
|certain not-very-high level of complexity no one can deal with events.
Well i am just too old and lazy to really dig into the
asynchronous hype you see "everywhere". I know just for fun i did
some MacOS Audio thing around 2009/10 (when i bought the single
Apple i once wanted), nothing but a simple OggVorbis player
(because Quicktime just did not like free Audio, and because i was
interested), and those callbacks i had to deal with just drove me
insane. The Linux Jack Audio Server also does that, i think.
I do not understand. The CPU usage was just _grazy_, even though
i used the biggest buffers i could use (iirc). I mean, i could
drive audio play and recording side-by-side (with programming
work) on a Cyrix 166+ with a 3 MB/sec (30 MB/sec with caching,
there was some I/O test on Linux, forgotten the name) hard disk on
FreeBSD 4.7 or what, it was nice(1)d, but other than that, and
i think under 1 or 3 percent CPU. That is how it has to be!
And not to forget most async and maybe even many coroutine things
just internally base on threads, i think OpenSSL for example does,
so what looks so cheap and "tickless" is in effect a tremendous
monster. I hope they at least use pools. And that the pools do
not leak resources to wherever.
No, by chance, i am simple-minded and drove non-blocking event
loops, which nicely bundled I/O events and gave me timers for
free. This scaled wonderful. And for multi-core and super
efficiency it scales easily, too: either by having worker threads
(or sandboxed, isolated processes), picked out of a pool, doing
work as it happens, or maybe, today, scaled by X different event
loops. (Like, for networking, having one accepting new
connections, and several others handling them.) But this seems so
"hyper", i wonder whether any of the actually used "super
servers", say postfix, dovecot, exim, postfix, *sql, apache,
nginx, let me add lighttpd now that i said postfix, as it works so
nice, uses _so_ much scaling, in the actual everyday work.
--steffen
|
|Der Kragenbaer, The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
1200
days inactive
1202
days old
10 comments
8 participants
participants (8)
-
Anthony Martin -
Bakul Shah -
Clem Cole -
Dan Cross -
John Cowan -
Larry McVoy -
Lawrence Stewart -
Steffen Nurpmeso