27 Sep
2017
27 Sep
'17
12:26 a.m.
On Tue, 26 Sep 2017 11:39:56 -0600 Warner Losh <imp(a)bsdimp.com> wrote:
Warner Losh writes:
On Tue, Sep 26, 2017 at 11:34 AM, Bakul Shah <bakul(a)bitblocks.com> wrote:
I have no practical experience with either so no idea which
can be done better. Just an instinct that lower level
protocols should be kept simple.
Ethernet is not a reliable protocol and I'd rather let an
higher level protocol deal with end to end reliability. And
while AOE uses ATA commands, attached disks can be SAS or ATA
or whatever the controller can handle. The congestion should
probably be dealt with at the ethernet switch level. Since
each protocol message has a tag, the driver should keep track
of what it has not received and retry after timeout.
In a previous job I got pretty familiar with ieee 802.1ad and
802.1ah (QinQ and MACinMAC) and they get complex when they add
all the other stuff some of which probably belongs at layer 3.
FC itself is a transport protocol for SCSI etc. so FCoE adds
yet another layer. (And I suppose you can then send ethernet
frames over fiber!)
You probably know Brantley Coile did ata over
Ethernet. AOE is a simple
protocol so the client side driver is simple. He ran plan9 on his
controller. One per rack. Basically you have a storage area network. Adding
disks becomes very easy. His company is back in business if you want to buy
some!
AOE is probably a better idea than FCoE, fiber channel over Ethernet, with
its requirement for a reliable transport. On the other hand, may be there
is need for encrypted channels all the way to disks.
AoE: All the quality you'd expect from low end ATA drives couple with all
the reliability of low-end consumer grade ethernet. What could go wrong?
AoE really punted on all the reliability stuff, so if you have any kind of
congested network it's a total disaster. Plus, who doesn't want to access
storage via an arcane protocol designed so you could latch registers
effective, but not so you can encode requests so.
Sorry for the rant, but I've had to deal with too much low-end gear
recently and it makes me grumpy and AoE back in the early 2000's was a
total S***show of low-end, poor quality implementations....