27 Feb
2017
27 Feb
'17
1:16 a.m.
I ported GNU Emacs to the Celerity product line mostly because most of the programmers
there wanted it over vi. Not me, I’m a vi guy.
I remember that GNU Emacs launched the first time and then dumped itself out as a core
file. Each subsequent launch would then ‘undump’ itself back into memory. All this because
launching emacs the first time required compiling all that lisp code.
Does anyone else remember this?
David
27 Feb
27 Feb
1:28 a.m.
On Sun, Feb 26, 2017 at 9:16 PM, David <david(a)kdbarto.org> wrote:
I ported GNU Emacs to the Celerity product line mostly
because most of the
programmers there wanted it over vi. Not me, I’m a vi guy.
I remember that GNU Emacs launched the first time and then dumped itself
out as a core file. Each subsequent launch would then ‘undump’ itself back
into memory. All this because launching emacs the first time required
compiling all that lisp code.
It still does that, doesn't it?
https://lwn.net/Articles/673724/
--
Mantas Mikulėnas <grawity(a)gmail.com>
12:41 p.m.
On 26 Feb 2017, at 19:16, David
<david(a)kdbarto.org> wrote:
I remember that GNU Emacs launched the first time and then dumped itself out as a core
file. Each subsequent launch would then ‘undump’ itself back into memory. All this because
launching emacs the first time required compiling all that lisp code.
It still works like that. Indeed that's the conventional way that Lisp systems tend
to work for delivering applications: you compile and load your code into a running image
and then dump that image in a form where it can be reloaded quickly. The dumped image is
either directly executable or is loaded by some small bootstrap loader which might also
provide some low-level support (but might not: all that can be in the dumped image). What
you call these dumped images depends on your culture: they might be 'worlds',
'bands' or 'sysouts' among other things.
(There are often also compiled files which are generally called fasl files, though in
emacs they are elc files.)
1:19 p.m.
Tim Bradshaw wrote:
Emacs came from ITS, and many Lisps derive from Maclisp which also came
from ITS. In ITS, it was common for applications to be dumped into a
loadable core image, even if they were written in assembly language.
David wrote:
I remember that GNU Emacs launched the first time and then dumped
itself out as a core file. Each subsequent launch would then ‘undump’
itself back into memory. All this because launching emacs the first
time required compiling all that lisp code.
It still works like that. Indeed
that's the conventional way that
Lisp systems tend to work for delivering applications 
1:26 p.m.
On Mon, Feb 27, 2017 at 12:19 AM, Lars Brinkhoff <lars(a)nocrew.org> wrote:
Tim Bradshaw wrote:
Emacs came from ITS, and many Lisps derive from Maclisp which also came
from ITS. In ITS, it was common for applications to be dumped into a
loadable core image, even if they were written in assembly language.
Unix systems are retiring sbrk(2), so emacs is breaking on those
systems. Trouble is, sbrk is kinda hard to implement on systems that
allocate memory for processes from multiple pools and other crazy
things. So now Emacs has no way of knowing where the upper limit was
so it can't start allocating with its own custom allocator...
At least GNU Emacs...
Warner
David
wrote:
I remember that GNU Emacs launched the first time and then dumped
itself out as a core file. Each subsequent launch would then ‘undump’
itself back into memory. All this because launching emacs the first
time required compiling all that lisp code.
It still works like that. Indeed
that's the conventional way that
Lisp systems tend to work for delivering applications 
2:12 p.m.
I've been having a bit of trouble with /bin/sh (Bourne's original one)
for the same reason. I described my projects in more detail earlier,
but in a nutshell I have intercepted open() and other syscalls, and I
have implemented replacements so that old code like /bin/sh doesn't
have to change, it sees a pretty BSD-like system. So one thing I had
to do was check if the thing being opened is a directory, and if so I
vacuum up the contents using the modern readdir() supplied in glibc or
whatever, and then write a tempfile in a format which the old style
readdir() understands. Then I return a handle to the tempfile instead
of the directory. Trouble is, this makes a non-stdio-using program be
stdio-using, in the worst case it's a non-stdio-using program that has
its own malloc() based on sbrk()... so we get another malloc()
happening in the middle, I temporarily fixed this by redirecting the
modern system's malloc() into the ancient system's malloc() but this
is a very non desirable solution. As another possibility I was
thinking of changing the ancient system's sbrk() into realloc() and
implementing a routine to relocate the heap, it obviously would have
to understand everything on the heap and everything that can point
into it. It's a real mess. But ultimately if I could get this right, I
could implement a lightweight system with no memory manager where all
processes share the malloc().
cheers, Nick
On Mon, Feb 27, 2017 at 6:26 PM, Warner Losh <imp(a)bsdimp.com> wrote:
On Mon, Feb 27, 2017 at 12:19 AM, Lars Brinkhoff
<lars(a)nocrew.org> wrote:
Tim Bradshaw wrote:
Emacs came from ITS, and many Lisps derive from Maclisp which also came
from ITS. In ITS, it was common for applications to be dumped into a
loadable core image, even if they were written in assembly language.
Unix systems are retiring sbrk(2), so emacs is breaking on those
systems. Trouble is, sbrk is kinda hard to implement on systems that
allocate memory for processes from multiple pools and other crazy
things. So now Emacs has no way of knowing where the upper limit was
so it can't start allocating with its own custom allocator...
At least GNU Emacs...
Warner David
wrote:
I remember that GNU Emacs launched the first time and then dumped
itself out as a core file. Each subsequent launch would then ‘undump’
itself back into memory. All this because launching emacs the first
time required compiling all that lisp code.
It still works like that. Indeed
that's the conventional way that
Lisp systems tend to work for delivering applications 
8:33 p.m.
On Mon, Feb 27, 2017 at 07:12:09PM +1100, Nick Downing wrote:
I've been having a bit of trouble with /bin/sh
(Bourne's original one)
for the same reason.
[snip]
Trouble is, this makes a non-stdio-using program be
stdio-using, in the worst case it's a non-stdio-using program that has
its own malloc() based on sbrk()... so we get another malloc()
happening in the middle, I temporarily fixed this by redirecting the
modern system's malloc() into the ancient system's malloc() but this
is a very non desirable solution. As another possibility I was
thinking of changing the ancient system's sbrk() into realloc() and
implementing a routine to relocate the heap, it obviously would have
to understand everything on the heap and everything that can point
into it.
How about applying Geoff Collyer's change to the shell memory management
routine available here:
http://www.collyer.net/who/geoff/stak.port.c
DF
8:50 p.m.
Thanks for the stak.c it's a good idea. I'm not sure about the stack
freeing routine at the end of the file as it seems to rely on a bit of
pointer voodoo but I think I could rationalize all that.
cheers, Nick
On Feb 28, 2017 1:33 AM, "Derek Fawcus"
<dfawcus+lists-tuhs(a)employees.org>
wrote:
On Mon, Feb 27, 2017 at 07:12:09PM +1100, Nick Downing
wrote:
I've been having a bit of trouble with
/bin/sh (Bourne's original one)
for the same reason.
[snip]
Trouble is, this makes a non-stdio-using program
be
stdio-using, in the worst case it's a non-stdio-using program that has
its own malloc() based on sbrk()... so we get another malloc()
happening in the middle, I temporarily fixed this by redirecting the
modern system's malloc() into the ancient system's malloc() but this
is a very non desirable solution. As another possibility I was
thinking of changing the ancient system's sbrk() into realloc() and
implementing a routine to relocate the heap, it obviously would have
to understand everything on the heap and everything that can point
into it.
How about applying Geoff Collyer's change to the shell memory management
routine available here:
http://www.collyer.net/who/geoff/stak.port.c
DF
9:43 p.m.
On Tue, Feb 28, 2017 at 01:50:55a.m. +1100, Nick Downing wrote:
Thanks for the stak.c it's a good idea. I'm
not sure about the stack
freeing routine at the end of the file as it seems to rely on a bit of
pointer voodoo but I think I could rationalize all that.
Have a look at the shorter/parent url, he also has a paper describing what
he's done, a port of the v7 shell (de-ALGOL'ed), and a few small enhancements.
DF
10:43 p.m.
Derek Fawcus <dfawcus+lists-tuhs(a)employees.org> wrote:
How about applying Geoff Collyer's change to the
shell memory management
routine available here:
http://www.collyer.net/who/geoff/stak.port.c
Depends on what shell you are talking about.
The code named by you only works with a very old Bourne Shell that can be
retrieved from the server of Geoff Collyer.
If you are interested in the recent Bourne Shell (SVr4 + Solaris changes), you
better use my Bourne Shell sources that can be found inside the schily-tools:
http://sourceforge.net/projects/schilytools/files/
The code from above will not work in a recent Bourne Shell without changes in
both, Geoff Collyer's stak.c and the rest of the Bourne Shell.
Jörg
--
EMail:joerg@schily.net (home) Jörg Schilling D-13353 Berlin
joerg.schilling(a)fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
URL: http://cdrecord.org/private/ http://sourceforge.net/projects/schilytools/files/
6:59 p.m.
On 27 Feb 2017, at 07:19, Lars Brinkhoff <lars(a)nocrew.org> wrote:
Emacs came from ITS, and many Lisps derive from Maclisp which also came
from ITS. In ITS, it was common for applications to be dumped into a
loadable core image, even if they were written in assembly language.
I think the history of this must go back beyond ITS. InterLisp did the same thing, and
it's a west-coast Lisp so probably not influenced by ITS (and it probably predates
ITS in its beginnings). So either the idea was just pervasive (which is at least
plausible I think) or it has its roots somewhere further back: I wonder what Lisp 1.5
did?
Oddly I once learned something important about TCP because of this. I had a Xerox
D-machine (an 1186, which was a Daybreak I think), which I needed to move somewhere. I
was logged into it and running a Lisp image with lots of state, so I did whatever
incantation was needed to write all the dirty pages to disk halt Lisp, turned the machine
off and carried it wherever it needed to go, plugged power and network back in and
restarted the image. Unbeknownst to me I had a couple of telnet windows open to Unix
machines: when I turned the machine back on they were not only still open but they still
had live sessions in them.
Well, it seems obvious now, but that was the moment when I understood how TCP worked.
--tim
4:35 p.m.
David <david(a)kdbarto.org> wrote:
I ported GNU Emacs to the Celerity product line mostly
because most of the programmers there wanted it over vi. Not me, I???m a vi guy.
I remember that GNU Emacs launched the first time and then dumped itself out as a core
file. Each subsequent launch would then ???undump??? itself back into memory. All this
because launching emacs the first time required compiling all that lisp code.
The file "sendmail.fc" is just the dump of the heap from sendmail.
This method existed since a long time.
BTW: undump(1) has been announced on a Sun User Group in 1987, but the
next year, SunOS-4.0 came out and made things much harder to implement.
I did never see an updated undump(1) source that would be able to deal with
SunOS-4.0 and it's shared libraries. Does it exist?
Jörg
--
EMail:joerg@schily.net (home) Jörg Schilling D-13353 Berlin
joerg.schilling(a)fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
URL: http://cdrecord.org/private/ http://sourceforge.net/projects/schilytools/files/
9:13 p.m.
Joerg Schilling <schily(a)schily.net> wrote:
BTW: undump(1) has been announced on a Sun User Group in 1987, but the
next year, SunOS-4.0 came out and made things much harder to implement.
I did never see an updated undump(1) source that would be able to deal with
SunOS-4.0 and it's shared libraries. Does it exist?
Emacs uses a thing called "unexec", or rather several things, one for each
OS, with varying complexity and fearsomeness.
The SunOS one was deleted in 2008:
http://git.savannah.gnu.org/cgit/emacs.git/commit/?id=2a5cb2584f9ca171ad431…
The Solaris one is the easiest, because dldump() is part of the OS:
http://git.savannah.gnu.org/cgit/emacs.git/tree/src/unexsol.c?id=4daca38d5c…
Of course, the most popular systems (generic ELF, Windows, and Mac OS)
have the most complicated implementations of unexec.
Tony.
--
f.anthony.n.finch <dot(a)dotat.at> http://dotat.at/ - I xn--zr8h punycode
Hebrides, Bailey: Cyclonic 7 to severe gale 9, occasionally storm 10 later.
Very rough or high, occasionally very high. Rain or showers. Good,
occasionally poor.
2854
days inactive
2855
days old
12 comments
10 participants
participants (10)
-
David -
Derek Fawcus -
Lars Brinkhoff -
Mantas Mikulėnas -
Nick Downing -
schily＠schily.net -
tfb＠tfeb.org -
Tim Bradshaw
-
Tony Finch -
Warner Losh