21 Nov
2020
21 Nov
'20
3:32 a.m.
Hello All,
I know I have asked this before, but I am curious about any new replies or
insight. How did package management start? Were sites keeping track of
packages installed in a flat file that you could grep (as god intended)
somewhere, or were upgrades and additions simply done without significant
announcement? At what point did someone decide, 'Hey, we need to have a
central way to track additional software"?
I know of DEC's setld and SGI's inst in the latter half of the '80s. What
was the mechanism before that?
-Henry
21 Nov
21 Nov
6:55 a.m.
See The SPMS Software Project Management System
documented in the new/spms for 4.3BSD
http://www.retro11.de/ouxr/43bsd/usr/src/new/spms/doc/
(I couldn't find link to this at TUHS.)
I don't know about it but maybe that will help
11:50 p.m.
Things were pretty much ad hoc. Commercial software likely came
as tar/cpio tapes to install however the vendor wanted. Free software
was from USENET in source code, so again, however people wanted.
The AT&T Unix PC (7300 / 3B1) in the late 80s had a file format
for installing software from floppy and tracked what was installed,
but that was unique to it.
Package managers as we know them today really became a big thing
with Linux. Redhat's RPM was one of the earliest.
My two cents; I'm sure others remember it differently.
Arnold
Henry Bent <henry.r.bent(a)gmail.com> wrote:
Hello All,
I know I have asked this before, but I am curious about any new replies or
insight. How did package management start? Were sites keeping track of
packages installed in a flat file that you could grep (as god intended)
somewhere, or were upgrades and additions simply done without significant
announcement? At what point did someone decide, 'Hey, we need to have a
central way to track additional software"?
I know of DEC's setld and SGI's inst in the latter half of the '80s.
What
was the mechanism before that?
-Henry
22 Nov
22 Nov
5:30 a.m.
Hello!
I, myself normally run Slackware Linux. It uses package management in
the form of compressed tar files, and a flat file store of the names.
It also has a tool which when run will show the user what's there, and
what they do if need be. In fact Slackware predates Red Hat by about
four years. (Pat and his CS professor introduced themselves to one
much earlier one, which was SLS. Neither liked it, and the Prof was
convinced that Pat could do better.)
-----
Gregg C Levine gregg.drwho8(a)gmail.com
"This signature fought the Time Wars, time and again."
On Sat, Nov 21, 2020 at 1:54 PM <arnold(a)skeeve.com> wrote:
Things were pretty much ad hoc. Commercial software likely came
as tar/cpio tapes to install however the vendor wanted. Free software
was from USENET in source code, so again, however people wanted.
The AT&T Unix PC (7300 / 3B1) in the late 80s had a file format
for installing software from floppy and tracked what was installed,
but that was unique to it.
Package managers as we know them today really became a big thing
with Linux. Redhat's RPM was one of the earliest.
My two cents; I'm sure others remember it differently.
Arnold
Henry Bent <henry.r.bent(a)gmail.com> wrote:
Hello All,
I know I have asked this before, but I am curious about any new replies or
insight. How did package management start? Were sites keeping track of
packages installed in a flat file that you could grep (as god intended)
somewhere, or were upgrades and additions simply done without significant
announcement? At what point did someone decide, 'Hey, we need to have a
central way to track additional software"?
I know of DEC's setld and SGI's inst in the latter half of the '80s.
What
was the mechanism before that?
-Henry
7:17 a.m.
1) No intention to slight debian in any way.
2) dpkg was definitely an improvement over FreeBSds ports scheme. But... In
fact freebsd did have a pkg system for ports before that --- which was
basically similar to 1983 SysIII scheme
3) also as I understand (and larry feel free to correct me here as a better
chronicler of things Linux than I) but I believe that the big thing rpm
added was the DB like DEC's setld and system Sun had used which us what I
was refering too.
Pls remember that I was trying to chronicle the basic ideas and some of the
motivation which is what Henry asked. And that the original driver was to
support ISVs installs. So I was trying to explain the history of what we
did at the time.
The be fair one of the more vocal people in the early 80s was Heinz who
occasionally add color here. I remember Heinz trying to push us to an ABI
and not stop at an API.
Today most of the ISVs have abandoned Unix except for the Mac. Msft and the
phones have taken that. And the package mngr has been replaced by the app
store which has.much great use than any of the current Unix packaging
schemes. Funny how the profit motive drove that.
Working for one of the few ISVS that do package SW for Unix we basically
support two schemes. Apple Mac installs and RPM because that is were the
primary customer base has been. I'd not about goodness or being better or
being first. It's economic (Larry and I bemoan this a lot).
So pls don't take it as a comment about anything other than trying to
answer as much of the early history as I could.
Heinz, Jon, Larry you all lived this on the commercial side. Care to add
anything?
Clem
On Sat, Nov 21, 2020 at 6:31 PM Gregg Levine <gregg.drwho8(a)gmail.com> wrote:
Hello!
I, myself normally run Slackware Linux. It uses package management in
the form of compressed tar files, and a flat file store of the names.
It also has a tool which when run will show the user what's there, and
what they do if need be. In fact Slackware predates Red Hat by about
four years. (Pat and his CS professor introduced themselves to one
much earlier one, which was SLS. Neither liked it, and the Prof was
convinced that Pat could do better.)
-----
Gregg C Levine gregg.drwho8(a)gmail.com
"This signature fought the Time Wars, time and again."
On Sat, Nov 21, 2020 at 1:54 PM <arnold(a)skeeve.com> wrote:
--
Sent from a handheld expect more typos than usual
Things were pretty much ad hoc. Commercial software likely came
as tar/cpio tapes to install however the vendor wanted. Free software
was from USENET in source code, so again, however people wanted.
The AT&T Unix PC (7300 / 3B1) in the late 80s had a file format
for installing software from floppy and tracked what was installed,
but that was unique to it.
Package managers as we know them today really became a big thing
with Linux. Redhat's RPM was one of the earliest.
My two cents; I'm sure others remember it differently.
Arnold
Henry Bent <henry.r.bent(a)gmail.com> wrote:
> Hello All,
>
> I know I have asked this before, but I am curious about any new
replies or
> insight. How did package management start?
Were sites keeping track
of
> packages installed in a flat file that you
could grep (as god intended)
> somewhere, or were upgrades and additions simply done without
significant
> announcement? At what point did someone
decide, 'Hey, we need to have
a
> central way to track additional
software"?
>
> I know of DEC's setld and SGI's inst in the latter half of the '80s.
What
was the
mechanism before that?
-Henry
7:39 a.m.
On Sat, Nov 21, 2020, 6:19 PM Clem Cole <clemc(a)ccc.com> wrote:
1) No intention to slight debian in any way.
2) dpkg was definitely an improvement over FreeBSds ports scheme. But...
In fact freebsd did have a pkg system for ports before that --- which was
basically similar to 1983 SysIII scheme
FreeBSD's ports/pkg system did keep track of what was installed on the
system. There was a database in /var/db so pkg_delete could remove things
and pkg_which to know what pkg a given file belonged to.
It was first-ish, but there was some package system for the early linux
root disks. I think this is how SLS started, but I might be misremembering.
But despite being early, and being ported to other BSDs, it sucked at
upgrading for 20-odd years until it was completely rewritten.... latter day
pkg is so much better, though its repo management has been a little weak
relative to the professional efforts in the linux world.
/usr/ports none the less was ground breaking because it handled both the
local patching, the build depends and the packaging under one umbrella.
It's been on the whole a good thing and has reinvented itself several times
over the years.
When I was managing SunOS systems it seemed like everyone rolled their own.
There was nothing like VMSINSTALL...
Warner
3) also as I understand (and larry feel free to correct me here as a better
chronicler of things Linux than I) but I believe that
the big thing rpm
added was the DB like DEC's setld and system Sun had used which us what I
was refering too.
Pls remember that I was trying to chronicle the basic ideas and some of
the motivation which is what Henry asked. And that the original driver
was to support ISVs installs. So I was trying to explain the history of
what we did at the time.
The be fair one of the more vocal people in the early 80s was Heinz who
occasionally add color here. I remember Heinz trying to push us to an ABI
and not stop at an API.
Today most of the ISVs have abandoned Unix except for the Mac. Msft and
the phones have taken that. And the package mngr has been replaced by the
app store which has.much great use than any of the current Unix packaging
schemes. Funny how the profit motive drove that.
Working for one of the few ISVS that do package SW for Unix we basically
support two schemes. Apple Mac installs and RPM because that is were the
primary customer base has been. I'd not about goodness or being better or
being first. It's economic (Larry and I bemoan this a lot).
So pls don't take it as a comment about anything other than trying to
answer as much of the early history as I could.
Heinz, Jon, Larry you all lived this on the commercial side. Care to add
anything?
Clem
On Sat, Nov 21, 2020 at 6:31 PM Gregg Levine <gregg.drwho8(a)gmail.com>
wrote:
Hello!
I, myself normally run Slackware Linux. It uses package management in
the form of compressed tar files, and a flat file store of the names.
It also has a tool which when run will show the user what's there, and
what they do if need be. In fact Slackware predates Red Hat by about
four years. (Pat and his CS professor introduced themselves to one
much earlier one, which was SLS. Neither liked it, and the Prof was
convinced that Pat could do better.)
-----
Gregg C Levine gregg.drwho8(a)gmail.com
"This signature fought the Time Wars, time and again."
On Sat, Nov 21, 2020 at 1:54 PM <arnold(a)skeeve.com> wrote:
--
Sent from a handheld expect more typos than usual
Things were pretty much ad hoc. Commercial software likely came
as tar/cpio tapes to install however the vendor wanted. Free software
was from USENET in source code, so again, however people wanted.
The AT&T Unix PC (7300 / 3B1) in the late 80s had a file format
for installing software from floppy and tracked what was installed,
but that was unique to it.
Package managers as we know them today really became a big thing
with Linux. Redhat's RPM was one of the earliest.
My two cents; I'm sure others remember it differently.
Arnold
Henry Bent <henry.r.bent(a)gmail.com> wrote:
> Hello All,
>
> I know I have asked this before, but I am curious about any new
replies or
> insight. How did package management start?
Were sites keeping track
of
> packages installed in a flat file that you
could grep (as god
intended)
> somewhere, or were upgrades and additions
simply done without
significant
> announcement? At what point did someone
decide, 'Hey, we need to
have a
> central way to track additional
software"?
>
> I know of DEC's setld and SGI's inst in the latter half of the '80s.
What
was the
mechanism before that?
-Henry
24 Nov
24 Nov
1:35 p.m.
"When I was managing SunOS systems it seemed like everyone rolled their
own..."
Yep, IIRC, tarball or cpio tape, no tracking or update support. Lucky if
the ISV install script asked where to install it.
SunOS filesystem layout was thoughtfully designed though, when diskless &
diskfull systems were introduced supporting multiple architectures (2.5?
3.x?): CPU-architecture-specific and architecture-independent mount points,
directories for Sun, ISV and local apps, etc (/usr /opt /usr/local and
their variations), read-only /usr support (link writeables into /var).
Though, mostly just Sun used this flexibility/complexity, few ISVs: they
generally wanted their installs to be consistent across HP-UX, MIPS
RISC/os, Pyramid dualPort DC/OSx, Sequent (Dynix?), SunOS, (maybe-AIX??,)
etc; which made sense from a support training point of view.
Beyond ./configure; make; make install which I'd count as build but barely
packaging, I don't recall any packaging until Solaris pkgadd et al?
Unfortunately with pkgadd came patchadd & friends.
They did their level best to cross-patch random binaries and muddy the
patch/package interdependency-waters as much as humanly possible.
Partly as a result, the early OS/patch/firmware support matrices for
FibreChannel were horrible.
I'll probably have nightmares about that tonight...
On Sun, 22 Nov 2020, 09:40 Warner Losh, <imp(a)bsdimp.com> wrote:
On Sat, Nov 21, 2020, 6:19 PM Clem Cole <clemc(a)ccc.com> wrote:
1) No intention to slight debian in any way.
2) dpkg was definitely an improvement over FreeBSds ports scheme. But...
In fact freebsd did have a pkg system for ports before that --- which was
basically similar to 1983 SysIII scheme
FreeBSD's ports/pkg system did keep track of what was installed on the
system. There was a database in /var/db so pkg_delete could remove things
and pkg_which to know what pkg a given file belonged to.
It was first-ish, but there was some package system for the early linux
root disks. I think this is how SLS started, but I might be misremembering.
But despite being early, and being ported to other BSDs, it sucked at
upgrading for 20-odd years until it was completely rewritten.... latter day
pkg is so much better, though its repo management has been a little weak
relative to the professional efforts in the linux world.
/usr/ports none the less was ground breaking because it handled both the
local patching, the build depends and the packaging under one umbrella.
It's been on the whole a good thing and has reinvented itself several times
over the years.
When I was managing SunOS systems it seemed like everyone rolled their
own. There was nothing like VMSINSTALL...
Warner
3) also as I understand (and larry feel free to correct me here as a
better chronicler of things Linux than I) but I
believe that the big thing
rpm added was the DB like DEC's setld and system Sun had used which us what
I was refering too.
Pls remember that I was trying to chronicle the basic ideas and some of
the motivation which is what Henry asked. And that the original driver
was to support ISVs installs. So I was trying to explain the history of
what we did at the time.
The be fair one of the more vocal people in the early 80s was Heinz who
occasionally add color here. I remember Heinz trying to push us to an ABI
and not stop at an API.
Today most of the ISVs have abandoned Unix except for the Mac. Msft and
the phones have taken that. And the package mngr has been replaced by the
app store which has.much great use than any of the current Unix packaging
schemes. Funny how the profit motive drove that.
Working for one of the few ISVS that do package SW for Unix we basically
support two schemes. Apple Mac installs and RPM because that is were the
primary customer base has been. I'd not about goodness or being better or
being first. It's economic (Larry and I bemoan this a lot).
So pls don't take it as a comment about anything other than trying to
answer as much of the early history as I could.
Heinz, Jon, Larry you all lived this on the commercial side. Care to
add anything?
Clem
On Sat, Nov 21, 2020 at 6:31 PM Gregg Levine <gregg.drwho8(a)gmail.com>
wrote:
Hello!
I, myself normally run Slackware Linux. It uses package management in
the form of compressed tar files, and a flat file store of the names.
It also has a tool which when run will show the user what's there, and
what they do if need be. In fact Slackware predates Red Hat by about
four years. (Pat and his CS professor introduced themselves to one
much earlier one, which was SLS. Neither liked it, and the Prof was
convinced that Pat could do better.)
-----
Gregg C Levine gregg.drwho8(a)gmail.com
"This signature fought the Time Wars, time and again."
On Sat, Nov 21, 2020 at 1:54 PM <arnold(a)skeeve.com> wrote:
--
Sent from a handheld expect more typos than usual
Things were pretty much ad hoc. Commercial software likely came
as tar/cpio tapes to install however the vendor wanted. Free software
was from USENET in source code, so again, however people wanted.
The AT&T Unix PC (7300 / 3B1) in the late 80s had a file format
for installing software from floppy and tracked what was installed,
but that was unique to it.
Package managers as we know them today really became a big thing
with Linux. Redhat's RPM was one of the earliest.
My two cents; I'm sure others remember it differently.
Arnold
Henry Bent <henry.r.bent(a)gmail.com> wrote:
> Hello All,
>
> I know I have asked this before, but I am curious about any new
replies or
> insight. How did package management start?
Were sites keeping
track of
> packages installed in a flat file that you
could grep (as god
intended)
> somewhere, or were upgrades and additions
simply done without
significant
> announcement? At what point did someone
decide, 'Hey, we need to
have a
> central way to track additional
software"?
>
> I know of DEC's setld and SGI's inst in the latter half of the
'80s. What
> was the mechanism before that?
>
> -Henry
22 Nov
22 Nov
4:23 a.m.
On Fri, Nov 20, 2020 at 4:33 PM Henry Bent <henry.r.bent(a)gmail.com> wrote:
I know I have asked this before, but I am curious
about any new replies or
insight. How did package management start?
Really good question. I thought the PKG_ADD we had on Masscomp in '83 was
grabbed from PWB 3.0. Unfortunately, Warren's stuff does not include, but
that is known to missing things (like SCCS which was first distributed as
part of PWB 1.0 and every version after).
So here is what I remember ...
When we did the '85 /usr/group standard, one of the things we argued about
was how would an ISV >>deliver<< a binary *I.e.* 'interchange'
between two
systems to use the TOPS-10/TOPS-20 terminology (which is actually what we
were using since most of us were familiar with same). By the time we got
to IEEE P1003, the whole reason USTAR was created was to solve that - which
begat the famous Tar Wars of the Research (TAR format) *vs*. CPIO (AT&T)
types [USTAR was a compromise and as I have said previously, was picked due
to the code in Ken's original implementation of the header CKSUM so it had
an unintended extension mechanism and as ASCII - cpio was binary in those
days AND could not be extended so older readers could at least read a new
tape).
The 'install' was left to each ISV and the assumption had been you would
use a USTAR tape (and eventually the PAX program) to read the bits, but
each ISV did their own 'installer.' The idea of keeping a system-wide DB
on what was installed was still in the future. PWB 3.0/System III
PKG_ADD was primitive, but my memory is it was the first attempt. I do
remember it was on a number of System III based systems but was very much
tied to installing the AT&T supplied SW - which I suspect was leftover from
the AT&T external maneuver of trying to supply everything and was difficult
to use by ISVs and I don't remember many doing so.
As you point out, the first commercial UNIX I remember that really tried to
solve it, was Ultrix which had something for both their own use and for
their ISV's (setld) - which frankly sucked and I personally hated and
railed against. But to DEC's credit, it was there. It was modeled after a
similar tool for VMS. Truth is, for a while it was the best. The biggest
thing that setld did (which in practice it did poorly) was trying to keep a
DB of what you installed so that an admin could type a command and see what
had been loaded, and when and also what licenses were installed to run
purchased software. Basically, it was driven by field service and SW
licensing.
When FreeBSD 1.0 came out, the big thing Jordan Hubbard did (and was much
better than Linux installs for a long time) was work on install >>for a new
system<<. He also created the idea of 'packages' which were all of the
thousands of UNIX tools that people had ported to FreeBSD and could
optionally be installed. I think it really was the first of the same name
and most of the features we know. By today's measure, again it was crude,
my memory is that unlike setld, since it was not managing licenses, he
didn't think to add a DB/log of what was being installed. He did not try
to solve the 'update' problem when a new version of FreeBSD was released
BTW. Basically, you needed to do a new install.
Roll forward a couple of years and Linux eventually picked up Jordan's
basic installer framework which vastly improved the out-of-box for some of
the Linux distros. But the important thing that RH did beyond FreeBSD was
to create RPM, which added a setld like DB to the scheme, not for licenses,
but so that you could easily do updates, add options, etc. They combined
Jordans install ideas and packages ideas, which was cool for a system where
you got/get everything from the distro.
The truth is, none of the Research UNIX, FreeBSD nor Linux really put the
effort that DEC, Masscomp, Sun, IBM, HP did in how to update a system.
*i.e.* I'm currently running version 10.13.5 and I want to get to 10.14.2
-- what needs to be installed and how will it affect already installed and
running ISV codes. [ IMO Microsoft is the worst and Apple is not much
better].
Linux is a weird one. Because of the 'open source' thinking, the idea of
keeping old binaries running is not the high order bit. DEC, IBM, HP,
Masscomp, and to some extent SUN and SGI, because they had a market for
commercial SW, have tried to keep old binaries going.
So ... now we have apt-get - which for what it is, works pretty well but,
it still does not solve a problem someone like my firm has that sells
commercial SW. FWIW: Since I actually wrote the spec for it inside
Intel, I can tell you what the design/goal/direction to tell the install
teams in that my employer distributes using RPM and >>is suppose<< to work
unmodified with an RPM-based install (*i.e.* be 'socially compliant' to the
norms of a more commercial-like Linux site). The >>idea<< is that the RPMs
are supposed to be able to automatically converted to Yum and a few other
formats (check the specs here for each tool, however -- this is not a
warranty from me - YMMV -- just telling what I >>personal<< scream at the
team when I discover they did not test properly as sometimes they do break
that - which can cause big issues when trying to install on a
supercomputer). The >>idea<< is that the current generation of package
tools, like setld of yesterday, will allow the admin to what's running on
the local system.
Clem
5:24 a.m.
At 2020-11-21T17:23:45-0500, Clem Cole wrote:
Roll forward a couple of years and Linux eventually
picked up Jordan's
basic installer framework which vastly improved the out-of-box for
some of the Linux distros. But the important thing that RH did beyond
FreeBSD was to create RPM, which added a setld like DB to the scheme,
not for licenses, but so that you could easily do updates, add
options, etc. They combined Jordans install ideas and packages ideas,
which was cool for a system where you got/get everything from the
distro.
The complete lack of mention of dpkg and the Debian package format is an
error in your narrative. According to rpm.org, the "first commit" to
the rpm package management software was on November 27, 1995[1].
By this time, dpkg had already been around for over a year; you can find
Ian Jackson's release announcement of dpkg 0.93.63 in July 1995[2], and
dpkg's own "ChangeLog.old" file in its source tree documents its history
back to August 1994.
So ... now we have apt-get - which for what it is,
works pretty well
but, it still does not solve a problem someone like my firm has that
sells commercial SW.
It is worth noting that apt also originated in Debian, largely developed
by Jason Gunthorpe but originally uploaded by Scott Ellis in April
1998[3]. Despite apt's popularity and obvious technical advantages in
upgrade management (a cycle-breaking dependency analyzer)--it drew
grudging admiration even from many in the community who abhorred
uttering the words "Debian", "GNU/Linux", or both--and a deliberately
package-format-agnostic architecture, RPM-based distributions resisted
adopting it for years until Conectiva, a commercial distribution from
Brazil, wrote the requisite back-end for rpm support (apt-rpm)[4].
FWIW: Since I actually wrote the spec for it inside
Intel, I can tell
you what the design/goal/direction to tell the install teams in that
my employer distributes using RPM and >>is suppose<< to work
unmodified with an RPM-based install (*i.e.* be 'socially compliant'
to the norms of a more commercial-like Linux site). The >>idea<< is
that the RPMs are supposed to be able to automatically converted to
Yum and a few other formats (check the specs here for each tool,
however -- this is not a warranty from me - YMMV -- just telling what
I >>personal<< scream at the team when I discover they did not test
properly as sometimes they do break that - which can cause big issues
when trying to install on a supercomputer).
These norms tend to be distribution-specific. Even where technology is
the same, the norms that produce integration can differ. Little about
Unix kernels prescribes any particular filesystem hierarchy, for
instance.
It has often been observed that what quality the Debian system enjoys is
less due to its technological advantages--though IMO these are clear in
package format (deb), source package format (dsc) and administration
tools--but in Debian's culture of writing prescriptions for a consistent
system configuration in its Policy Manual[5], of maintaining automated
checking tools for compliance with those prescriptions[6][7], and of
being willing to gate releases on the lack of such compliance. The last
used to be a point of emphatic derision by rival distributions, most of
which were funded by venture capital and thus motivated to emphasize
cadence over technical quality, the former property being more easily
measured by non-specialists, deep-pocketed and otherwise.
Regards,
Branden
[1] https://rpm.org/timeline.html
[2] https://lists.debian.org/debian-devel/1995/07/msg00009.html
[3] https://lists.debian.org/debian-devel-changes/1998/04/msg00140.html
[4] https://lwn.net/Articles/30728/
[5] https://www.debian.org/doc/debian-policy/
[6] https://lintian.debian.org/
[7] https://piuparts.debian.org/
1489
days inactive
1493
days old
8 comments
8 participants
participants (8)
-
arnold＠skeeve.com -
Clem Cole -
G. Branden Robinson -
Gregg Levine -
Henry Bent -
Jeremy C. Reed -
Stuart Remphrey -
Warner Losh