15 Dec
2023
15 Dec
'23
3:48 a.m.
So Lars Brinkhoff and I were chatting about daemons:
https://gunkies.org/wiki/Talk:Daemon
and I pointed out that in addition to 'standard' daemons (e.g. the printer
spooler daemon, email daemon, etc, etc) there are some other things that are
daemon-like, but are fundamentally different in major ways (explained later
below). I dubbed them 'system processes', but I'm wondering if ayone knows
if
there is a standard term for them? (Or, failing that, if they have a
suggestion for a better name?)
Early UNIX is one of the first systems to have one (process 0, the "scheduling
(swapping)
process"), but the CACM "The UNIX Time-Sharing System" paper:
https://people.eecs.berkeley.edu/~brewer/cs262/unix.pdf
doesn't even mention it, so no guidance there. Berkeley UNIX also has one,
mentioned in "Design and Implementation of the Berkeley Virtual Memory
Extensions to the UNIX Operating System":
http://roguelife.org/~fujita/COOKIES/HISTORY/3BSD/design.pdf
where it is called the "pageout daemon".("During system initialization,
just
before the init process is created, the bootstrapping code creates process 2
which is known as the pageout daemon. It is this process that .. writ[es]
back modified pages. The process leaves its normal dormant state upon being
waken up due to the memory free list size dropping below an upper
threshold.") However, I think there are good reasons to dis-favour the term
'daemon' for them.
For one thing, typical daemons look (to the kernel) just like 'normal'
processes: their object code is kept in a file, and is loaded into the
daemon's process when it starts, using the same mechanism that 'normal'
processes use for loading their code; daemons are often started long after
the kernel itself is started, and there is usually not a special mechanism in
the kernel to start daemons (on early UNIXes, /etc/rc is run by the 'init'
process, not the kernel); daemons interact with the kernel through system
calls, just like 'ordinary' processes; the daemon's process runs in
'user'
CPU mode (using the same standard memory mapping mechanisms, just like
blah-blah).
'System processes' do none of these things: their object code is linked into
the monolithic kernel, and is thus loaded by the bootstrap; the kernel
contains special provision for starting the system process, which start as
the kernel is starting; they don't do system calls, just call kernel routines
directly; they run in kernel mode, using the same memory mapping as the
kernel itself; etc, etc.
Another important point is that system processes are highly intertwined with
the operation of the kernel; without the system process(es) operating
correctly, the operation of the system will quickly grind to a halt. The loss
of ordinary' daemons is usually not fatal; if the email daemon dies, the
system will keep running indefinitely. Not so, for the swapping process, or
the pageout daemon
Anyway, is there a standard term for these things? If not, a better name than
'system process'?
Noel
15 Dec
15 Dec
4:06 a.m.
I remember calling them kernel processes as they had no code running in user mode. Not
sure now of the year but sometime in ‘80s. Now I’d probably call them kernel threads as
they don’t have a separate address space.
On Dec 14, 2023, at 1:48 PM, jnc(a)mercury.lcs.mit.edu
wrote:
So Lars Brinkhoff and I were chatting about daemons:
https://gunkies.org/wiki/Talk:Daemon
and I pointed out that in addition to 'standard' daemons (e.g. the printer
spooler daemon, email daemon, etc, etc) there are some other things that are
daemon-like, but are fundamentally different in major ways (explained later
below). I dubbed them 'system processes', but I'm wondering if ayone knows
if
there is a standard term for them? (Or, failing that, if they have a
suggestion for a better name?)
Early UNIX is one of the first systems to have one (process 0, the "scheduling
(swapping)
process"), but the CACM "The UNIX Time-Sharing System" paper:
https://people.eecs.berkeley.edu/~brewer/cs262/unix.pdf
doesn't even mention it, so no guidance there. Berkeley UNIX also has one,
mentioned in "Design and Implementation of the Berkeley Virtual Memory
Extensions to the UNIX Operating System":
http://roguelife.org/~fujita/COOKIES/HISTORY/3BSD/design.pdf
where it is called the "pageout daemon".("During system initialization,
just
before the init process is created, the bootstrapping code creates process 2
which is known as the pageout daemon. It is this process that .. writ[es]
back modified pages. The process leaves its normal dormant state upon being
waken up due to the memory free list size dropping below an upper
threshold.") However, I think there are good reasons to dis-favour the term
'daemon' for them.
For one thing, typical daemons look (to the kernel) just like 'normal'
processes: their object code is kept in a file, and is loaded into the
daemon's process when it starts, using the same mechanism that 'normal'
processes use for loading their code; daemons are often started long after
the kernel itself is started, and there is usually not a special mechanism in
the kernel to start daemons (on early UNIXes, /etc/rc is run by the 'init'
process, not the kernel); daemons interact with the kernel through system
calls, just like 'ordinary' processes; the daemon's process runs in
'user'
CPU mode (using the same standard memory mapping mechanisms, just like
blah-blah).
'System processes' do none of these things: their object code is linked into
the monolithic kernel, and is thus loaded by the bootstrap; the kernel
contains special provision for starting the system process, which start as
the kernel is starting; they don't do system calls, just call kernel routines
directly; they run in kernel mode, using the same memory mapping as the
kernel itself; etc, etc.
Another important point is that system processes are highly intertwined with
the operation of the kernel; without the system process(es) operating
correctly, the operation of the system will quickly grind to a halt. The loss
of ordinary' daemons is usually not fatal; if the email daemon dies, the
system will keep running indefinitely. Not so, for the swapping process, or
the pageout daemon
Anyway, is there a standard term for these things? If not, a better name than
'system process'?
Noel
4:12 a.m.
When shutting down, FreeBSD refers to them as:
kern/kern_shutdown.c: printf("Waiting (max %d seconds) for system process
`%s' to stop... ",
kern/kern_shutdown.c: printf("Waiting (max %d seconds) for system thread
`%s' to stop... ",
However, a number of places, including the swap daemon, still refer to
things as this daemon
or that daemon (page demon being top of the list, but there's the buf
daemon, the vmdaemon
that handles swapping (as opposed to paging), the update daemon, etc.
Warner
On Thu, Dec 14, 2023 at 3:06 PM Bakul Shah <bakul(a)iitbombay.org> wrote:
I remember calling them kernel processes as they had
no code running in
user mode. Not sure now of the year but sometime in ‘80s. Now I’d probably
call them kernel threads as they don’t have a separate address space.
On Dec 14, 2023, at 1:48 PM,
jnc(a)mercury.lcs.mit.edu wrote:
So Lars Brinkhoff and I were chatting about daemons:
https://gunkies.org/wiki/Talk:Daemon
and I pointed out that in addition to 'standard' daemons (e.g. the
printer
spooler daemon, email daemon, etc, etc) there are
some other things that
are
daemon-like, but are fundamentally different in
major ways (explained
later
below). I dubbed them 'system
processes', but I'm wondering if ayone
knows if
there is a standard term for them? (Or, failing
that, if they have a
suggestion for a better name?)
Early UNIX is one of the first systems to have one (process 0, the
"scheduling (swapping)
process"), but the CACM "The UNIX
Time-Sharing System" paper:
https://people.eecs.berkeley.edu/~brewer/cs262/unix.pdf
doesn't even mention it, so no guidance there. Berkeley UNIX also has
one,
mentioned in "Design and Implementation of
the Berkeley Virtual Memory
Extensions to the UNIX Operating System":
http://roguelife.org/~fujita/COOKIES/HISTORY/3BSD/design.pdf
where it is called the "pageout daemon".("During system initialization,
just
before the init process is created, the
bootstrapping code creates
process 2
which is known as the pageout daemon. It is this
process that .. writ[es]
back modified pages. The process leaves its normal dormant state upon
being
waken up due to the memory free list size
dropping below an upper
threshold.") However, I think there are good reasons to dis-favour the
term
'daemon' for them.
For one thing, typical daemons look (to the kernel) just like 'normal'
processes: their object code is kept in a file, and is loaded into the
daemon's process when it starts, using the same mechanism that 'normal'
processes use for loading their code; daemons are often started long
after
the kernel itself is started, and there is
usually not a special
mechanism in
the kernel to start daemons (on early UNIXes,
/etc/rc is run by the
'init'
process, not the kernel); daemons interact with
the kernel through system
calls, just like 'ordinary' processes; the daemon's process runs in
'user'
CPU mode (using the same standard memory mapping
mechanisms, just like
blah-blah).
'System processes' do none of these things: their object code is linked
into
the monolithic kernel, and is thus loaded by the
bootstrap; the kernel
contains special provision for starting the system process, which start
as
the kernel is starting; they don't do system
calls, just call kernel
routines
directly; they run in kernel mode, using the same
memory mapping as the
kernel itself; etc, etc.
Another important point is that system processes are highly intertwined
with
the operation of the kernel; without the system
process(es) operating
correctly, the operation of the system will quickly grind to a halt. The
loss
of ordinary' daemons is usually not fatal;
if the email daemon dies, the
system will keep running indefinitely. Not so, for the swapping process,
or
the pageout daemon
Anyway, is there a standard term for these things? If not, a better name
than
'system process'?
Noel
4:09 a.m.
I don't know of a standard name. We used to call the kernel processes or
kernel threads also. For instance, in the original Masscomp EFS code, we
had a handful of processes that got forked after the pager using kernel
code. Since the basic UNIX read/write from the user space scheme is
synchronous, the premade pool of kernel processes was dispatched as needed
when we listened for asynchronous remote requests for I/O. This is similar
to the fact that asynchronous devices from serial or network interfaces
need a pool of memory to stuff things into since you never know ahead of
time when it will come.
ᐧ
On Thu, Dec 14, 2023 at 4:48 PM Noel Chiappa <jnc(a)mercury.lcs.mit.edu>
wrote:
So Lars Brinkhoff and I were chatting about daemons:
https://gunkies.org/wiki/Talk:Daemon
and I pointed out that in addition to 'standard' daemons (e.g. the printer
spooler daemon, email daemon, etc, etc) there are some other things that
are
daemon-like, but are fundamentally different in major ways (explained later
below). I dubbed them 'system processes', but I'm wondering if ayone
knows
if
there is a standard term for them? (Or, failing that, if they have a
suggestion for a better name?)
Early UNIX is one of the first systems to have one (process 0, the
"scheduling (swapping)
process"), but the CACM "The UNIX Time-Sharing System" paper:
https://people.eecs.berkeley.edu/~brewer/cs262/unix.pdf
doesn't even mention it, so no guidance there. Berkeley UNIX also has one,
mentioned in "Design and Implementation of the Berkeley Virtual Memory
Extensions to the UNIX Operating System":
http://roguelife.org/~fujita/COOKIES/HISTORY/3BSD/design.pdf
where it is called the "pageout daemon".("During system initialization,
just
before the init process is created, the bootstrapping code creates process
2
which is known as the pageout daemon. It is this process that .. writ[es]
back modified pages. The process leaves its normal dormant state upon being
waken up due to the memory free list size dropping below an upper
threshold.") However, I think there are good reasons to dis-favour the term
'daemon' for them.
For one thing, typical daemons look (to the kernel) just like 'normal'
processes: their object code is kept in a file, and is loaded into the
daemon's process when it starts, using the same mechanism that 'normal'
processes use for loading their code; daemons are often started long after
the kernel itself is started, and there is usually not a special mechanism
in
the kernel to start daemons (on early UNIXes, /etc/rc is run by the 'init'
process, not the kernel); daemons interact with the kernel through system
calls, just like 'ordinary' processes; the daemon's process runs in
'user'
CPU mode (using the same standard memory mapping mechanisms, just like
blah-blah).
'System processes' do none of these things: their object code is linked
into
the monolithic kernel, and is thus loaded by the bootstrap; the kernel
contains special provision for starting the system process, which start as
the kernel is starting; they don't do system calls, just call kernel
routines
directly; they run in kernel mode, using the same memory mapping as the
kernel itself; etc, etc.
Another important point is that system processes are highly intertwined
with
the operation of the kernel; without the system process(es) operating
correctly, the operation of the system will quickly grind to a halt. The
loss
of ordinary' daemons is usually not fatal; if the email daemon dies, the
system will keep running indefinitely. Not so, for the swapping process, or
the pageout daemon
Anyway, is there a standard term for these things? If not, a better name
than
'system process'?
Noel
8:20 p.m.
On Thu, Dec 14, 2023 at 5:17 PM Clem Cole <clemc(a)ccc.com> wrote:
I don't know of a standard name. We used to
call the kernel processes or kernel threads also.
I've heard all combinations of (system|kernel) (thread|task|process),
all of which mean more or less the same thing: something the kernel
can schedule and run that doesn't have a userspace component, reusing
the basic concurrency primitives in the kernel for its own internal
purposes. I'm not sure I've heard "kernel daemon" before, but
intuitively I'd lump it into the same category unless I was told
otherwise (as Noel mentioned, of course Berkeley had the "pageout
daemon" which ran only in the kernel).
For instance, in the original Masscomp EFS code, we
had a handful of processes that got forked after the pager using kernel code. Since the
basic UNIX read/write from the user space scheme is synchronous, the premade pool of
kernel processes was dispatched as needed when we listened for asynchronous remote
requests for I/O. This is similar to the fact that asynchronous devices from serial or
network interfaces need a pool of memory to stuff things into since you never know ahead
of time when it will come.
ᐧ
I remember reading a paper on the design of NFS (it may have been the
BSD paper) and there was a note about how the NFS server process ran
mostly in the kernel; user code created it, but pretty much all it did
was invoke a system call that implemented the server. That was kind of
neat.
- Dan C.
On Thu, Dec 14, 2023 at 4:48 PM Noel Chiappa
<jnc(a)mercury.lcs.mit.edu> wrote:
> So Lars Brinkhoff and I were chatting about daemons:
>
> https://gunkies.org/wiki/Talk:Daemon
>
> and I pointed out that in addition to 'standard' daemons (e.g. the printer
> spooler daemon, email daemon, etc, etc) there are some other things that are
> daemon-like, but are fundamentally different in major ways (explained later
> below). I dubbed them 'system processes', but I'm wondering if ayone
knows if
> there is a standard term for them? (Or, failing that, if they have a
> suggestion for a better name?)
>
>
> Early UNIX is one of the first systems to have one (process 0, the "scheduling
(swapping)
> process"), but the CACM "The UNIX Time-Sharing System" paper:
>
> https://people.eecs.berkeley.edu/~brewer/cs262/unix.pdf
>
> doesn't even mention it, so no guidance there. Berkeley UNIX also has one,
> mentioned in "Design and Implementation of the Berkeley Virtual Memory
> Extensions to the UNIX Operating System":
>
> http://roguelife.org/~fujita/COOKIES/HISTORY/3BSD/design.pdf
>
> where it is called the "pageout daemon".("During system
initialization, just
> before the init process is created, the bootstrapping code creates process 2
> which is known as the pageout daemon. It is this process that .. writ[es]
> back modified pages. The process leaves its normal dormant state upon being
> waken up due to the memory free list size dropping below an upper
> threshold.") However, I think there are good reasons to dis-favour the term
> 'daemon' for them.
>
>
> For one thing, typical daemons look (to the kernel) just like 'normal'
> processes: their object code is kept in a file, and is loaded into the
> daemon's process when it starts, using the same mechanism that
'normal'
> processes use for loading their code; daemons are often started long after
> the kernel itself is started, and there is usually not a special mechanism in
> the kernel to start daemons (on early UNIXes, /etc/rc is run by the 'init'
> process, not the kernel); daemons interact with the kernel through system
> calls, just like 'ordinary' processes; the daemon's process runs in
'user'
> CPU mode (using the same standard memory mapping mechanisms, just like
> blah-blah).
>
> 'System processes' do none of these things: their object code is linked
into
> the monolithic kernel, and is thus loaded by the bootstrap; the kernel
> contains special provision for starting the system process, which start as
> the kernel is starting; they don't do system calls, just call kernel routines
> directly; they run in kernel mode, using the same memory mapping as the
> kernel itself; etc, etc.
>
> Another important point is that system processes are highly intertwined with
> the operation of the kernel; without the system process(es) operating
> correctly, the operation of the system will quickly grind to a halt. The loss
> of ordinary' daemons is usually not fatal; if the email daemon dies, the
> system will keep running indefinitely. Not so, for the swapping process, or
> the pageout daemon
>
>
> Anyway, is there a standard term for these things? If not, a better name than
> 'system process'?
>
> Noel
10:25 p.m.
On Fri, Dec 15, 2023 at 7:21 AM Dan Cross <crossd(a)gmail.com> wrote:
I recall discussions with the kernel people at Solbourne who were bringing
up
SunOS 4.0 on Solbourne hardware about this. nfsd was little more than an N
way
fork followed by the system call. It provided a process context to sleep
in, which
couldn't be created in the kernel at the time. I've not gone to the
available SunOS
sources to confirm this is what's going on.
I'd thought, though, that this was the second nfsd implementation. The
first one
would decode the requests off the wire and schedule the I/O. It was only
when
there were issues with this approach that it moved into the kernel. This
was mostly
a context switch thing, but as more security measures were added to the
system
that root couldn't bypass, NFS needed to move into the kernel so it could
bypass
them. See getfh and similar system calls. I'm not sure how much of this was
done in SunOS, I'm only familiar with the post 4.4BSD work...
Warner
On Thu, Dec 14, 2023 at 5:17 PM Clem Cole
<clemc(a)ccc.com> wrote:
FreeBSD (and likely others) have extended this to allow kernel
threads that we loosely call daemons as well. FreeBSD has APIs
for creating a kernel-only threads and processes...
I don't know of a standard name. We used
to call the kernel processes
or kernel threads also.
I've heard all combinations of (system|kernel) (thread|task|process),
all of which mean more or less the same thing: something the kernel
can schedule and run that doesn't have a userspace component, reusing
the basic concurrency primitives in the kernel for its own internal
purposes. I'm not sure I've heard "kernel daemon" before, but
intuitively I'd lump it into the same category unless I was told
otherwise (as Noel mentioned, of course Berkeley had the "pageout
daemon" which ran only in the kernel).
For instance,
in the original Masscomp EFS code, we had a handful of
processes that got forked
after the pager using kernel code. Since the
basic UNIX read/write from the user space scheme is synchronous, the
premade pool of kernel processes was dispatched as needed when we listened
for asynchronous remote requests for I/O. This is similar to the fact that
asynchronous devices from serial or network interfaces need a pool of
memory to stuff things into since you never know ahead of time when it will
come.
ᐧ
I remember reading a paper on the design of NFS (it may have been the
BSD paper) and there was a note about how the NFS server process ran
mostly in the kernel; user code created it, but pretty much all it did
was invoke a system call that implemented the server. That was kind of
neat. - Dan C.
On Thu, Dec 14, 2023 at 4:48 PM Noel Chiappa
<jnc(a)mercury.lcs.mit.edu>
wrote:
> So Lars Brinkhoff and I were chatting about
daemons:
>
> https://gunkies.org/wiki/Talk:Daemon
>
> and I pointed out that in addition to 'standard' daemons (e.g. the
printer
> spooler daemon, email daemon, etc, etc) there
are some other things
that are
> daemon-like, but are fundamentally different
in major ways (explained
later
> below). I dubbed them 'system
processes', but I'm wondering if ayone
knows if
> there is a standard term for them? (Or,
failing that, if they have a
> suggestion for a better name?)
>
>
> Early UNIX is one of the first systems to have one (process 0, the
"scheduling (swapping)
> process"), but the CACM "The UNIX
Time-Sharing System" paper:
>
> https://people.eecs.berkeley.edu/~brewer/cs262/unix.pdf
>
> doesn't even mention it, so no guidance there. Berkeley UNIX also has
one,
> mentioned in "Design and Implementation
of the Berkeley Virtual Memory
> Extensions to the UNIX Operating System":
>
> http://roguelife.org/~fujita/COOKIES/HISTORY/3BSD/design.pdf
>
> where it is called the "pageout daemon".("During system
initialization,
just
> before the init process is created, the
bootstrapping code creates
process 2
> which is known as the pageout daemon. It is
this process that ..
writ[es]
> back modified pages. The process leaves its
normal dormant state upon
being
> waken up due to the memory free list size
dropping below an upper
> threshold.") However, I think there are good reasons to dis-favour the
term
> 'daemon' for them.
>
>
> For one thing, typical daemons look (to the kernel) just like 'normal'
> processes: their object code is kept in a file, and is loaded into the
> daemon's process when it starts, using the same mechanism that
'normal'
> processes use for loading their code; daemons are often started long
after
> the kernel itself is started, and there is
usually not a special
mechanism in
> the kernel to start daemons (on early UNIXes,
/etc/rc is run by the
'init'
> process, not the kernel); daemons interact
with the kernel through
system
> calls, just like 'ordinary'
processes; the daemon's process runs in
'user'
> CPU mode (using the same standard memory
mapping mechanisms, just like
> blah-blah).
>
> 'System processes' do none of these things: their object code is linked
into
> the monolithic kernel, and is thus loaded by
the bootstrap; the kernel
> contains special provision for starting the system process, which start
as
> the kernel is starting; they don't do
system calls, just call kernel
routines
> directly; they run in kernel mode, using the
same memory mapping as the
> kernel itself; etc, etc.
>
> Another important point is that system processes are highly intertwined
with
> the operation of the kernel; without the
system process(es) operating
> correctly, the operation of the system will quickly grind to a halt.
The loss
> of ordinary' daemons is usually not
fatal; if the email daemon dies, the
> system will keep running indefinitely. Not so, for the swapping
process, or
> the pageout daemon
>
>
> Anyway, is there a standard term for these things? If not, a better
name than
> 'system process'?
>
> Noel
11:13 p.m.
On Dec 15, 2023, at 6:21 AM, Dan Cross <crossd(a)gmail.com> wrote:
I remember reading a paper on the design of NFS (it may have been the
BSD paper) and there was a note about how the NFS server process ran
mostly in the kernel; user code created it, but pretty much all it did
was invoke a system call that implemented the server. That was kind of
neat.
At Valid Logic Systems I prototyped a relatively simple network filesystem.
Here there was no user code. There was one “agent” kernel thread per remote
system accessing local filesystem + a few more. The agent thread acted on
behalf of a remote system and maintained a session as long as at least one
local file/dir was referenced from that system. There were complications as it
was not a stateless design. I had to add code to detect when the remote
server/client died or rebooted and return ENXIO / clear out old state.
12:24 p.m.
For the record, daring to stray outside Unix, ITS originally had a
"system job", and later split off a "core job". Both running in
monitor
mode. Job means the same thing as process here, and monitor same as
kernel.
--- >8 --- cut and stop reading here --- >8 --- off topic --- >8 ---
In ITS terminology, "demon" means a (user space) process that is started
on demand by the system. This could be due to an external event (say, a
network connection), or another process indicating a need for a service.
The demon may go away right after it has done its job, or linger around
for a while.
There's a separate term "dragon" which means a continuously running
background process.
16 Dec
16 Dec
12:30 a.m.
On 2023-12-15 01:24, Lars Brinkhoff wrote:
For the record, daring to stray outside Unix, ITS
originally had a
"system job", and later split off a "core job". Both running in
monitor
mode. Job means the same thing as process here, and monitor same as
kernel.
--- >8 --- cut and stop reading here --- >8 --- off topic --- >8 ---
From https://www.tuhs.org/cgi-bin/mailman/listinfo/coff
"The Computer Old Farts Forum provides a place for people to discuss the
history of computers and their future."
It seems that you are well within bounds.
S.
373
days inactive
374
days old
8 comments
7 participants
participants (7)
-
Bakul Shah -
Clem Cole -
Dan Cross -
jnc＠mercury.lcs.mit.edu -
Lars Brinkhoff -
Stuff Received -
Warner Losh