16 May
2024
16 May
'24
9:34 a.m.
Every so often I want to compare files on remote machines, but all I can
do is to fetch them first (usually into /tmp); I'd like to do something
like:
rdiff host1:file1 host2:file2
Breathes there such a beast? I see that Penguin/OS has already taken
"rdiff" which doesn't seem to do what I want.
Think of it as an extension to the Unix philosophy of "Everything looks
like a file"...
-- Dave
16 May
16 May
9:51 a.m.
Maybe
diff -u <(ssh host1 cat file1) <(ssh host2 cat file2)
?
That could be put into a shell script that does the approriate
text manipulations on the original $1 and $2.
HTH,
Arnold
Dave Horsfall <dave(a)horsfall.org> wrote:
Every so often I want to compare files on remote
machines, but all I can
do is to fetch them first (usually into /tmp); I'd like to do something
like:
rdiff host1:file1 host2:file2
Breathes there such a beast? I see that Penguin/OS has already taken
"rdiff" which doesn't seem to do what I want.
Think of it as an extension to the Unix philosophy of "Everything looks
like a file"...
-- Dave
10:33 a.m.
Hi,
I've set ‘mail-followup-to: coff(a)tuhs.xn--org-to0a.
> Every so often I want to compare files on remote
machines, but all
> I can do is to fetch them first (usually into /tmp); I'd like to do
> something like:
>
> rdiff host1:file1 host2:file2
>
> Breathes there such a beast?
No, nor should there. It would be slain less it beget rcmp, rcomm,
rpaste, ...
> Think of it as an extension to the Unix
philosophy of "Everything
> looks like a file"...
Then make remote files look local as far as their access is concerned.
Ideally at the system-call level. Less ideal, at libc.a.
Maybe
diff -u <(ssh host1 cat file1) <(ssh host2 cat file2)
This is annoyingly noisy if the remote SSH server has sshd_config(5)'s
‘Banner’ set which spews the contents of a file before authentication,
e.g. the pointless
This computer system is the property of ...
Disconnect NOW if you have not been expressly authorised to use this
system. Unauthorised use is a criminal offence under the Computer
Misuse Act 1990.
Communications on or through ...uk's computer systems may be
monitored or recorded to secure effective system operation and for
other lawful purposes.
It appears on stderr so doesn't upset the diff but does clutter.
And discarding stderr is too sloppy.
--
Cheers, Ralph.
11:59 a.m.
Sshfs
G
On Thu, 16 May 2024, 5:34 pm Ralph Corderoy, <ralph(a)inputplus.co.uk> wrote:
Hi,
I've set ‘mail-followup-to: coff(a)tuhs.xn--org-to0a.
> Every so often I want to compare files on
remote machines, but all
> I can do is to fetch them first (usually into /tmp); I'd like to do
> something like:
>
> rdiff host1:file1 host2:file2
>
> Breathes there such a beast?
No, nor should there. It would be slain less it beget rcmp, rcomm,
rpaste, ...
> Think of it as an extension to the Unix
philosophy of "Everything
> looks like a file"...
Then make remote files look local as far as their access is concerned.
Ideally at the system-call level. Less ideal, at libc.a.
Maybe
diff -u <(ssh host1 cat file1) <(ssh host2 cat file2)
This is annoyingly noisy if the remote SSH server has sshd_config(5)'s
‘Banner’ set which spews the contents of a file before authentication,
e.g. the pointless
This computer system is the property of ...
Disconnect NOW if you have not been expressly authorised to use this
system. Unauthorised use is a criminal offence under the Computer
Misuse Act 1990.
Communications on or through ...uk's computer systems may be
monitored or recorded to secure effective system operation and for
other lawful purposes.
It appears on stderr so doesn't upset the diff but does clutter.
And discarding stderr is too sloppy.
--
Cheers, Ralph.
12:01 p.m.
Ralph Corderoy <ralph(a)inputplus.co.uk> wrote:
All true, I didn't think about that.
Maybe
diff -u <(ssh host1 cat file1) <(ssh host2 cat file2)
This is annoyingly noisy if the remote SSH server has sshd_config(5)'s
‘Banner’ set which spews the contents of a file before authentication,
e.g. the pointless
[....]
It appears on stderr so doesn't upset the diff but does clutter.
And discarding stderr is too sloppy.
But the author of a personal script knows his/her remote machines
and can decide if
diff -u <(ssh host1 cat file1 2>/dev/null) <(ssh host2 cat file2
2>/dev/null)
is appropriate or not.
My main point was that the problem is easily solved with a
few lines of shell, so no need for a utility, especially one
written in C or some other compiled language.
Thanks,
Arnold
8:08 p.m.
" The 9import tool allows an arbitrary file on a remote system, with the
capability of running the Plan 9 exportfs(4) service, to be imported into
the local name space. Usually file is a directory, so the complete file
tree under the directory is made available."
https://9fans.github.io/plan9port/man/man4/9import.html
9import host1 / /tmp/host1
9import host2 /tmp/host2
diff /tmp/host1/a/b/c /tmp/host2/a/b/c
(or whatever command you want that works with files. No need for stuff like
'rdiff' etc.)
stuff you take for granted on some systems ...
I have the plan 9 cpu command working (written in Go) and I think it's time
I get import working more widely, it's just too useful.
On Thu, May 16, 2024 at 2:01 AM <arnold(a)skeeve.com> wrote:
Ralph Corderoy <ralph(a)inputplus.co.uk> wrote:
All true, I didn't think about that.
Maybe
diff -u <(ssh host1 cat file1) <(ssh host2 cat file2)
This is annoyingly noisy if the remote SSH server has sshd_config(5)'s
‘Banner’ set which spews the contents of a file before authentication,
e.g. the pointless
[....]
It appears on stderr so doesn't upset the diff but does clutter. And discarding stderr is too sloppy.
But the author of a personal script knows his/her remote machines
and can decide if
diff -u <(ssh host1 cat file1 2>/dev/null) <(ssh host2 cat file2
2>/dev/null)
is appropriate or not.
My main point was that the problem is easily solved with a
few lines of shell, so no need for a utility, especially one
written in C or some other compiled language.
Thanks,
Arnold
9:51 p.m.
To add to Ron's post, Plan 9's cpu exports the origination's namespace to
the destination; by convention it is mounted on /mnt/term at destination.
host1% cpu -h host2
host2% diff file2 /mnt/term/usr/me/file1
On Thu, May 16, 2024 at 10:09 AM ron minnich <rminnich(a)gmail.com> wrote:
" The 9import tool allows an arbitrary file on a
remote system, with the
capability of running the Plan 9 exportfs(4) service, to be imported into
the local name space. Usually file is a directory, so the complete file
tree under the directory is made available."
https://9fans.github.io/plan9port/man/man4/9import.html
9import host1 / /tmp/host1
9import host2 /tmp/host2
diff /tmp/host1/a/b/c /tmp/host2/a/b/c
(or whatever command you want that works with files. No need for stuff
like 'rdiff' etc.)
stuff you take for granted on some systems ...
I have the plan 9 cpu command working (written in Go) and I think it's
time I get import working more widely, it's just too useful.
On Thu, May 16, 2024 at 2:01 AM <arnold(a)skeeve.com> wrote:
Ralph Corderoy <ralph(a)inputplus.co.uk>
wrote:
All true, I didn't think about that.
Maybe
diff -u <(ssh host1 cat file1) <(ssh host2 cat file2)
This is annoyingly noisy if the remote SSH server has sshd_config(5)'s
‘Banner’ set which spews the contents of a file before authentication,
e.g. the pointless
[....]
It appears on stderr so doesn't upset the diff but does clutter. And discarding stderr is too sloppy.
But the author of a personal script knows his/her remote machines
and can decide if
diff -u <(ssh host1 cat file1 2>/dev/null) <(ssh host2 cat file2
2>/dev/null)
is appropriate or not.
My main point was that the problem is easily solved with a
few lines of shell, so no need for a utility, especially one
written in C or some other compiled language.
Thanks,
Arnold
10:51 p.m.
If I recall correctly, there is a combination of flags to rsync that will
generate a report on a file, a set of files, or a set of directories to
tell if they are different.
I seem to recall DPK or RCT doing something clever with rsync and cksum to
get this sort of result without having to stream a lot of data across the
long-haul network back in the day.
Best,
Marc
=====
nygeek.net
mindthegapdialogs.com/home <https://www.mindthegapdialogs.com/home>
On Thu, May 16, 2024 at 2:51 PM Skip Tavakkolian <fariborz.t(a)gmail.com>
wrote:
To add to Ron's post, Plan 9's cpu exports
the origination's namespace to
the destination; by convention it is mounted on /mnt/term at destination.
host1% cpu -h host2
host2% diff file2 /mnt/term/usr/me/file1
On Thu, May 16, 2024 at 10:09 AM ron minnich <rminnich(a)gmail.com> wrote:
> " The 9import tool allows an arbitrary file on a remote system, with the
> capability of running the Plan 9 exportfs(4) service, to be imported into
> the local name space. Usually file is a directory, so the complete file
> tree under the directory is made available."
> https://9fans.github.io/plan9port/man/man4/9import.html
>
> 9import host1 / /tmp/host1
> 9import host2 /tmp/host2
> diff /tmp/host1/a/b/c /tmp/host2/a/b/c
> (or whatever command you want that works with files. No need for stuff
> like 'rdiff' etc.)
>
> stuff you take for granted on some systems ...
>
> I have the plan 9 cpu command working (written in Go) and I think it's
> time I get import working more widely, it's just too useful.
>
> On Thu, May 16, 2024 at 2:01 AM <arnold(a)skeeve.com> wrote:
>
>> Ralph Corderoy <ralph(a)inputplus.co.uk> wrote:
>>
>> > > Maybe
>> > >
>> > > diff -u <(ssh host1 cat file1) <(ssh host2 cat file2)
>> >
>> > This is annoyingly noisy if the remote SSH server has sshd_config(5)'s
>> > ‘Banner’ set which spews the contents of a file before authentication,
>> > e.g. the pointless
>> >
>> > [....]
>> >
>> > It appears on stderr so doesn't upset the diff but does clutter.
>>
>> All true, I didn't think about that.
>>
>> > And discarding stderr is too sloppy.
>>
>> But the author of a personal script knows his/her remote machines
>> and can decide if
>>
>> diff -u <(ssh host1 cat file1 2>/dev/null) <(ssh host2 cat file2
>> 2>/dev/null)
>>
>> is appropriate or not.
>>
>> My main point was that the problem is easily solved with a
>> few lines of shell, so no need for a utility, especially one
>> written in C or some other compiled language.
>>
>> Thanks,
>>
>> Arnold
>>
>
10:53 p.m.
On Thu, May 16, 2024 at 04:34:54PM +1000, Dave Horsfall wrote:
Every so often I want to compare files on remote
machines, but all I can
do is to fetch them first (usually into /tmp); I'd like to do something
like:
rdiff host1:file1 host2:file2
Breathes there such a beast? I see that Penguin/OS has already taken
"rdiff" which doesn't seem to do what I want.
rdiff is something which someone on the internet had created, as part
of the librsync package[1]. Thia isn't considered part of the core
package (for example, Debian consideres it as an "optional" package)
but rather something which various distributions have packaged for the
convenience for their users.
[1] https://librsync.github.io/
So if this is considered part of Penguin/OS, would we also consider
"nethack" or X11 part of BSD 4.3, since it was available and often
would be commonly installed on BSD 4.3 systems? Or are all packages
which are in FreeBSD's ports "part of FreeBSD"? Or all packages in
MacPorts part of MacOS?
In any case, the way I'd suggest that you do this that works as an
extention to the Unix philosohy of "Everything looks like a file" is
to use FUSE:
sshfs host1:/ ~/mnt/host1
sshfs host2:/ ~/mnt/host2
diff ~/mnt/host1/file1 ~/mnt/host2/file2
Cheers,
- Ted
20 Jun
20 Jun
2:20 a.m.
On Thu, May 16, 2024, at 15:53, Theodore Ts'o wrote:
On Thu, May 16, 2024 at 04:34:54PM +1000, Dave
Horsfall wrote:
> Every so often I want to compare files on remote machines, but all I can
> do is to fetch them first (usually into /tmp); I'd like to do something
> like:
>
> rdiff host1:file1 host2:file2
>
> Breathes there such a beast? I see that Penguin/OS has already taken
> "rdiff" which doesn't seem to do what I want.
[...]
In any case, the way I'd suggest that you do this
that works as an
extention to the Unix philosohy of "Everything looks like a file" is
to use FUSE:
sshfs host1:/ ~/mnt/host1
sshfs host2:/ ~/mnt/host2
diff ~/mnt/host1/file1 ~/mnt/host2/file2
Cheers,
- Ted
Mentioning this since I just came across it today: Eric S. Raymond
(ESR) has a 'netdiff' tool available here:
http://www.catb.org/~esr/netdiff/
https://gitlab.com/esr/netdiff
<quote>
Perform diff across network links.
Takes two arguments which may be host:path pairs in the style of
ssh/scp, make temporary local copies as needed, applies
diff(1). All options are passed to diff(1).
</quote>
Usage:
netdiff [diff-options] [host1:]path1 [host2:]path2
It's 56 lines of POSIX shell script that invokes 'scp' behind the
scenes and creates temp files that are fed to 'diff', with some file
name post-processing.
Compared to the one line of shell script mentioned by Arnold:
diff -u <(ssh host1 cat file1) <(ssh host2 cat file2)
the main benefits I see to 'netdiff' in its current form are that it
encapsulates the mechanism behind a descriptive name that can be
found in the file system, and it has a man page. So it can be found,
used and/or studied by those with larval shell fu.
However, the comments in the 'netdiff' source suggest openness to
supporting different transport mechanisms, which (if added) might
allow using the same command line interface regardless of the
underlying transport being used (scp, ssh, rsync, curl, wget,
whatever), possibly with different transports for each of the
requested files.
Of course, that would not offer a superior user experience to the
'9import', 'sshfs', and NFS approaches also mentioned.
But if there were an 'rf' ("remote file") tool that did just the
transport abstraction portion, then tools such as 'netdiff' could
use it internally, or folks could use it directly:
diff -u <(rf user@host1:/path/to/file1) <(rf user@host2:/path/to/file2)
The presumption here is that the user has control over some local
configuration that maps user@host pairs to transports (maybe
defaulting to 'ssh' invoking 'cat' on the remote host).
Maybe somebody who needs more than 'ssh' would find value in such a
thing...
-Al
--
a l a n d. s a l e w s k i
ads(a)salewski.email
salewski(a)att.net
https://github.com/salewski
11
days inactive
45
days old
9 comments
9 participants
participants (9)
-
Alan D. Salewski -
arnold＠skeeve.com -
Dave Horsfall -
George Michaelson -
Marc Donner -
Ralph Corderoy -
ron minnich -
Skip Tavakkolian -
Theodore Ts'o