19 Sep
2023
19 Sep
'23
4:31 a.m.
Last december Matt brought up xmodem and recently I needed it for an almost identical use
case. Studied it a bit and in the context of Unix, it is an interesting piece of software
history.
Although xmodem originated on CP/M in 1977, it seems to have arrived on Unix soon
thereafter. The first Unix implementation seems to have gone by the name of “rbsb” and
must have originated when V7 was prevalent: it uses alarm() to simulate non-blocking I/O.
Over the course of the 80’s ymodem and zmodem were added and the package became lrzsz; the
source continued to have a very V7-ish feel to it at least to the early 90’s. In the
mid-90’s it was converted to ansi-C and had some other modernization, but it looks like it
would still have run on SysIII (apart from the ansi).The last update to the upstream
source package appears to be from 1998, i.e. 25 years ago.
It is still packaged today for FreeBSD and major Linux distros. On Unix it must be one the
oldest code bases still in regular use, with the 1980 source still recognizable in its
current incarnation. Any other contenders come to mind?
I had always associated x/y/zmodem with CP/M and MSDOS, not so much with Unix. Last
December Clem already pointed out that it was popular for file exchange in the Unix scene
as well, along with several other similar tools. Also, the ymodem approach to file
metadata is very unix oriented, suggesting it originated on Unix or at least that Unix
users were an important user demographic. Yet, I could find little trace of x/y/zmodem in
the TUHS Unix Tree. The search tool finds it in 2.11BSD, in Minix 1.5 and 2.0 and in V10.
Kermit is in those as well, and in 4.3BSD and 4.4BSD on top.
Maybe these programs were commonly pulled from a bulletin board and hence not on
distribution media. Not sure how that would be bootstrapped, although the 2.11BSD files
for x/y/zmodem include “minirb.c”, a 175 line implementation to receive files using the
ymodem protocol. It is possible that this was transferred as plain text as a first step,
or even retyped.
Any recollections?
19 Sep
19 Sep
5:07 a.m.
On Mon, Sep 18, 2023 at 6:31 PM Paul Ruizendaal <pnr(a)planet.nl> wrote:
Last december Matt brought up xmodem and recently I
needed it for an almost identical use case. Studied it a bit and in the context of Unix,
it is an interesting piece of software history.
Although xmodem originated on CP/M in 1977, it seems to have arrived on Unix soon
thereafter. The first Unix implementation seems to have gone by the name of “rbsb” and
must have originated when V7 was prevalent: it uses alarm() to simulate non-blocking I/O.
Over the course of the 80’s ymodem and zmodem were added and the package became lrzsz; the
source continued to have a very V7-ish feel to it at least to the early 90’s. In the
mid-90’s it was converted to ansi-C and had some other modernization, but it looks like it
would still have run on SysIII (apart from the ansi).The last update to the upstream
source package appears to be from 1998, i.e. 25 years ago.
It is still packaged today for FreeBSD and major Linux distros. On Unix it must be one
the oldest code bases still in regular use, with the 1980 source still recognizable in its
current incarnation. Any other contenders come to mind?
I had always associated x/y/zmodem with CP/M and MSDOS, not so much with Unix. Last
December Clem already pointed out that it was popular for file exchange in the Unix scene
as well, along with several other similar tools. Also, the ymodem approach to file
metadata is very unix oriented, suggesting it originated on Unix or at least that Unix
users were an important user demographic. Yet, I could find little trace of x/y/zmodem in
the TUHS Unix Tree. The search tool finds it in 2.11BSD, in Minix 1.5 and 2.0 and in V10.
Kermit is in those as well, and in 4.3BSD and 4.4BSD on top.
Maybe these programs were commonly pulled from a bulletin board and hence not on
distribution media. Not sure how that would be bootstrapped, although the 2.11BSD files
for x/y/zmodem include “minirb.c”, a 175 line implementation to receive files using the
ymodem protocol. It is possible that this was transferred as plain text as a first step,
or even retyped.
Any recollections?
xmodem was an outgrowth of Christensen's MODEM.ASM for CP/M, and is
almost criminally simple: a small block of data coupled with a little
bit of header information, wait for an acknowledgement, and repeat.
Metadata was small (a start byte, a block number, inverse block
number, and a single checksum byte), but there was no total byte
count, so it was assumed that the final block would be padded with a
throwaway character.
Chuck Forsberg did YMODEM and ZMODEM; YMODEM is sort of a
super-XMODEM: it adds a 16-bit CRC, increases the block size, and adds
a "0 block" with some metadata (total file size and file name); thus,
it no longer needed a padding byte removed from the final packet.
Otherwise, it retains most of the overall structure of XMODEM.
ZMODEM was, as I understood it, designed for transfers across telenet,
which was pretty reliable; instead of the highly synchronous
send/wait-for-ack cycle of xmodem and ymodem, zmodem relies on error
detection and correction and is basically a streaming protocol: a
packet in a sliding window could be NAK'ed, thus rewinding the
transfer, but otherwise it basically just sends data until done.
lrzsz came kind of later; I remember Forsberg's rzsz package on Unix
back in the day, and then there was a GNU reimplementation. For a
while, the Omen Technologies BBS was listed in the BSD phone numbers
files.
xmodem lives on in a lot of embedded applications because of its
overall simplicity. We used it to bootstrap kernels onto Oxide
computers, for example, while we were doing active development.
- Dan C.
8:04 a.m.
On Mon, Sep 18, 2023 at 07:07:25PM -0400, Dan Cross wrote:
xmodem lives on in a lot of embedded applications because of its
overall simplicity. We used it to bootstrap kernels onto Oxide
computers, for example, while we were doing active development.
Today, KDE's Konsole terminal window has a "ZModem upload" function,
which I've used for sending a file up to some system where the
administrator on the remote machine has disabled scp and sftp for
random security $REASONS.
I could compress the file and use uuencode/uudecode, but for larger
files where cut and paste isn't terribly convenient, using zmodem can
be one of the simpler and more effective.
- Ted
3:47 p.m.
I just wanted to mention, that you can also get zmodem support for iTerm2 if you are on
mac.
In fact i use it a lot.
See here:
https://github.com/robberphex/iTerm2-zmodem
Best regards,
Michael
On 19. Sep 2023, at 04:04, Theodore Ts'o
<tytso(a)mit.edu> wrote:
On Mon, Sep 18, 2023 at 07:07:25PM -0400, Dan Cross wrote:
xmodem lives on in a lot of embedded applications because of its
overall simplicity. We used it to bootstrap kernels onto Oxide
computers, for example, while we were doing active development.
Today, KDE's Konsole terminal window has a "ZModem upload" function,
which I've used for sending a file up to some system where the
administrator on the remote machine has disabled scp and sftp for
random security $REASONS.
I could compress the file and use uuencode/uudecode, but for larger
files where cut and paste isn't terribly convenient, using zmodem can
be one of the simpler and more effective.
- Ted 
9:58 a.m.
On Mon, Sep 18, 2023 at 7:08 PM Dan Cross <crossd(a)gmail.com> wrote:
but there was no total byte
count, so it was assumed that the final block would be
padded with a
throwaway character.
CP/M, like RT-11 and OS/8 before it, did not track the sizes of files in
bytes or words, only in blocks. Text files ended in ^Z and were then
padded with either NULs or more ^Zs if necessary; binary files were usually
padded with NULs.
ZMODEM was, as I understood it, designed for transfers across telenet,
which was pretty reliable; instead of the highly
synchronous
send/wait-for-ack cycle of xmodem and ymodem, zmodem relies on error
detection and correction and is basically a streaming protocol: a
packet in a sliding window could be NAK'ed, thus rewinding the
transfer, but otherwise it basically just sends data until done.
It's an analogue of TCP/IP, in fact.
8:46 a.m.
On Tue, Sep 19, 2023 at 12:31:30AM +0200, Paul Ruizendaal wrote:
I had always associated x/y/zmodem with CP/M and
MSDOS, not so much with Unix. Last December Clem already pointed out that it was popular
for file exchange in the Unix scene as well, along with several other similar tools. Also,
the ymodem approach to file metadata is very unix oriented, suggesting it originated on
Unix or at least that Unix users were an important user demographic. Yet, I could find
little trace of x/y/zmodem in the TUHS Unix Tree. The search tool finds it in 2.11BSD, in
Minix 1.5 and 2.0 and in V10. Kermit is in those as well, and in 4.3BSD and 4.4BSD on top.
have a look at
tuhs/Applications/Shoppa_Tapes/usenix878889.tar.gz
usenix87/Comm/
usenix89/Comm/
also appears in
386bsd/othersrc/public/zmodem-3.03/
where the license was changed to prohibit commercial use after
the RLE changes in April 1989
https://www.ohse.de/uwe/software/lrzsz.html
is derived from an earlier version with the license changed to GPLv2
9:42 a.m.
Chuck Forsberg wrote Zmodem as a sliding Window transfer protocol
alternative to Kermit and superkermit. It was very high performance
over slow links. He provided source code support for the protocol
driver on the server and client end, in various versions. His client
programs were Yam (yet another modem) and ProYam. You could shell out
of the client, run other programs. Other client platforms supported
parts of the protocol. Available as binaries and source when licenced.
It had a scripting language and out of the box had the zmodem protocol
which has the ability to send commands, resume transfers, traverse non
8 bit or unreliable links, remove or rename source file, deal with
conflicting file names, synchronise folders and lots of other things.
I have written many scripts to help dial up users into a unix box and
download financial research data. Scripts still work across the
internet. He made the code very portable, clean and I believe it is
open source now. Chuck is no longer with us.
SecureCRT ssh client from VanDyke.com still has native support. To
send a file from the unix host, typically sz filename and it just
arrives. Sz- u and it will remove the source file after successful
transfer, plus lots of other options. The binary was typically linked
so if you type sx it would try to do a transfer with xmodem, even
though it's the same binary.
Hope this helps.
Serge
On Tue, 19 Sept 2023 at 12:47, Jonathan Gray <jsg(a)jsg.id.au> wrote:
On Tue, Sep 19, 2023 at 12:31:30AM +0200, Paul Ruizendaal wrote:
I had always associated x/y/zmodem with CP/M and
MSDOS, not so much with Unix. Last December Clem already pointed out that it was popular
for file exchange in the Unix scene as well, along with several other similar tools. Also,
the ymodem approach to file metadata is very unix oriented, suggesting it originated on
Unix or at least that Unix users were an important user demographic. Yet, I could find
little trace of x/y/zmodem in the TUHS Unix Tree. The search tool finds it in 2.11BSD, in
Minix 1.5 and 2.0 and in V10. Kermit is in those as well, and in 4.3BSD and 4.4BSD on top.
have a look at
tuhs/Applications/Shoppa_Tapes/usenix878889.tar.gz
usenix87/Comm/
usenix89/Comm/
also appears in
386bsd/othersrc/public/zmodem-3.03/
where the license was changed to prohibit commercial use after
the RLE changes in April 1989
https://www.ohse.de/uwe/software/lrzsz.html
is derived from an earlier version with the license changed to GPLv2
12:10 p.m.
If anyone wants more zmodem/Chuck Forsberg history, a prolific usenet
poster, here are news archives going back to 1994. It gives a good
indication of his personality. I had been dealing with him for about 6
years by then... Still have the licence keys.
https://groups.google.com/g/comp.protocols.misc/c/kjQ3HWqR2Ck/m/QVrdXdbsz1EJ
Kermit discussion and comparisons,
https://groups.google.com/g/comp.dcom.modems/c/GuEzWARpVQY/m/nZX5QdmY7xwJ
On Tue, 19 Sept 2023 at 13:42, Serge Burjak <sburjak(a)systech.com.au> wrote:
Chuck Forsberg wrote Zmodem as a sliding Window transfer protocol
alternative to Kermit and superkermit. It was very high performance
over slow links. He provided source code support for the protocol
driver on the server and client end, in various versions. His client
programs were Yam (yet another modem) and ProYam. You could shell out
of the client, run other programs. Other client platforms supported
parts of the protocol. Available as binaries and source when licenced.
It had a scripting language and out of the box had the zmodem protocol
which has the ability to send commands, resume transfers, traverse non
8 bit or unreliable links, remove or rename source file, deal with
conflicting file names, synchronise folders and lots of other things.
I have written many scripts to help dial up users into a unix box and
download financial research data. Scripts still work across the
internet. He made the code very portable, clean and I believe it is
open source now. Chuck is no longer with us.
SecureCRT ssh client from VanDyke.com still has native support. To
send a file from the unix host, typically sz filename and it just
arrives. Sz- u and it will remove the source file after successful
transfer, plus lots of other options. The binary was typically linked
so if you type sx it would try to do a transfer with xmodem, even
though it's the same binary.
Hope this helps.
Serge
On Tue, 19 Sept 2023 at 12:47, Jonathan Gray <jsg(a)jsg.id.au> wrote:
>
> On Tue, Sep 19, 2023 at 12:31:30AM +0200, Paul Ruizendaal wrote:
> > I had always associated x/y/zmodem with CP/M and MSDOS, not so much with Unix.
Last December Clem already pointed out that it was popular for file exchange in the Unix
scene as well, along with several other similar tools. Also, the ymodem approach to file
metadata is very unix oriented, suggesting it originated on Unix or at least that Unix
users were an important user demographic. Yet, I could find little trace of x/y/zmodem in
the TUHS Unix Tree. The search tool finds it in 2.11BSD, in Minix 1.5 and 2.0 and in V10.
Kermit is in those as well, and in 4.3BSD and 4.4BSD on top.
>
> have a look at
> tuhs/Applications/Shoppa_Tapes/usenix878889.tar.gz
> usenix87/Comm/
> usenix89/Comm/
>
> also appears in
> 386bsd/othersrc/public/zmodem-3.03/
> where the license was changed to prohibit commercial use after
> the RLE changes in April 1989
>
> https://www.ohse.de/uwe/software/lrzsz.html
> is derived from an earlier version with the license changed to GPLv2
12:37 p.m.
On Tue, 19 Sep 2023, Serge Burjak wrote:
If anyone wants more zmodem/Chuck Forsberg history, a
prolific usenet
poster, here are news archives going back to 1994. It gives a good
indication of his personality. I had been dealing with him for about 6
years by then... Still have the licence keys.
https://groups.google.com/g/comp.protocols.misc/c/kjQ3HWqR2Ck/m/QVrdXdbsz1EJ
I remember him well; thanks!
Kermit discussion and comparisons,
https://groups.google.com/g/comp.dcom.modems/c/GuEzWARpVQY/m/nZX5QdmY7xwJ
I was wondering when someone was going to mention Kermit...
-- Dave, with a 1200/75 software modem at the time
244
days inactive
245
days old
8 comments
8 participants
participants (8)
-
Dan Cross -
Dave Horsfall -
John Cowan -
Jonathan Gray -
Michael Stiller -
Paul Ruizendaal -
Serge Burjak -
Theodore Ts'o