Mailing List Archive

> smaller packets than they otherwise could. Some hosts get
> noticeably higher performance when they are able to use FDDI-
> sized packets compared to Ethernet-sized packets, and restricting
> the packet size to 1500 bytes will put a limit on the maximum

Some hard figures on this would be interesting. Ie, % of packets
with > 1500 MTU, % performance degradation if fragmented, etc. I
suspect that other backbone design issues (like congestion)
dominate any fragmentation issue.

I'm not sure a few people trying to get a little extra throughput
should dictate the design of a NAP (unless they want to pay for it).

- - - - - - - - - - - - - - - - -

> > smaller packets than they otherwise could. Some hosts get
> > noticeably higher performance when they are able to use FDDI-
> > sized packets compared to Ethernet-sized packets, and restricting
> > the packet size to 1500 bytes will put a limit on the maximum
>
> Some hard figures on this would be interesting. Ie, % of packets
> with > 1500 MTU, % performance degradation if fragmented, etc. I
> suspect that other backbone design issues (like congestion)
> dominate any fragmentation issue.

I'm not in a position to give you that figure, sorry (I couldn't
even if I would).

Do however note that this is not so much an issue about (IP)
fragmentation happening in routers as it is about efficiency of
host interfaces, since most of these guys use (or should use,
anyway) path MTU discovery. (It is conceivable that this
argument also has weakened over time.)

> I'm not sure a few people trying to get a little extra throughput
> should dictate the design of a NAP (unless they want to pay for it).

In some instances that would be a fair trade-off. But do note
that the "packet size" issue is a side issue -- the main argument
for using FDDI-style switches with "backpressure" via token
stealing is appropriate amounts and handling of buffering.

- Havard
- - - - - - - - - - - - - - - - -

A new stdaddr draft is forthcoming shortly. This other conversation is of
interest, though. We're arguing about 100Mb/s interconnect technology as
if we all planned to keep using it for some significant period. That's not
so. Two years from now it'll be 622Mb/s or it'll be a dead concept. PMTUD
matters, and TCP MSS (therefore IP MTU) matters because it will dictate the
frame rate to the routers and the end hosts. Bytes are cheap, frames cost.
- - - - - - - - - - - - - - - - -

>
>A new stdaddr draft is forthcoming shortly. This other conversation is of
>interest, though. We're arguing about 100Mb/s interconnect technology as
>if we all planned to keep using it for some significant period. That's not
>so. Two years from now it'll be 622Mb/s or it'll be a dead concept. PMTUD
>matters, and TCP MSS (therefore IP MTU) matters because it will dictate the
>frame rate to the routers and the end hosts. Bytes are cheap, frames cost.

Paul, with all due respect, there is a lot of apparent need and
demand for 100mbps interconnects, one could argue
that 100 100mbps interconnects are better than 3 or 4 622mbps
interconnects. I think 100 interconnects are at least as likely
in 2 years as 4 622mbps interconnects.
Since most of the customers connecting up to these interconnects
will be by devices with MTUs of 1500 (frame, T-1, switched 10 and
100mbps interconnects), and all of the network below these interconnected
sites are 1500 MTU and smaller. I know of maybe 100 sites
around the globle that can currently send FDDI MTUs in the 4000
range.a By contrast good esitmates show about 10 million
dialup users connected via termial servers which have
ethernet interfaces.

In terms of total users FDDI users would be well below .001%.
In terms of total revenues FDDI users would be well below 1%.

Basied on these numbers, my recommendations to NAP/MAE builders
would be to build based on 100mbps switched ethernet, with
the option to interconnect FDDI. To make the FDDI worth
while there would have to be at least two networks attached
that had routers co-located (or virtually co-located), with
DS-3 rate connections, and those two networks would each
need at least one DS-3 rate customer each. Assuming
this is the case the upgrade to FDDI is a relatively
simple thing.

Comparing the cost of 100mbps switched ethernet, to
the 100 mbps switched FDDI, there is about 30% to 50%
difference per port on the switch, and greater than that
on the router. Prices on switched ethernet are dropping
much faster than those of FDDI, because the price point
is below the level at which a technology is only used where
speed is the only factor.

Speaking of interfaces and speed, I see Cisco now has a OC-3
packet interface that will do raw packets via HDLC or PPP
at OC-3 rates via SONET. That seems much nicer than
the ATM AIP card. You could build a fairly nice backbone
with a dual attached mesh of OC-3 routers.

--
Jeremy Porter, Freeside Communications, Inc. jerry@fc.net
PO BOX 80315 Austin, Tx 78708 | 1-800-968-8750 | 512-339-6094
http://www.fc.net
- - - - - - - - - - - - - - - - -

Before you count-out switched FDDI, note that there are other FDDI
switches in the world these days (with all deference to the Gigaswith's
pioneering efforts, in whose hands our butts are held). They look to
have good performance and the cost-per-port is rather lower. (Getting
to be second and third, etc, is often a considerable advantage.)

It's too bad there isn't an "official" way to run larger MTU (eg, FDDI)
with 100baseTX running point-to-point Full-Duplex. Some of the chips
even have control bits ignore MTU-exceeded conditions. The packet
length limits are required only for Ethernet "cable mode" operations
where collissions must be detected. In point-to-point Full-Duplex
service, this is obviously a non-issue and an FDDI MTU would work just
fine.

So if there was an "official" (ie, interoperagble) large-packet mode
and somebody built a 100baseTX switch that could handle jumbograms,
then I think that Full-Duplexed Switched 100baseTX *would* be the
medium of choice for many, many tasks - moderate-sized exchange points
being one of them. Heck, I'd use it for interior wiring in my
superhubs.

But there are customers for whom end-to-end MTU is a buying issue.
Bogus or not, it is a very real issue.

-mo
- - - - - - - - - - - - - - - - -

> But there are customers for whom end-to-end MTU is a buying issue.
> Bogus or not, it is a very real issue.

They exist, but are there enough of them to make it work the extra
money to accomodate them? Nobody seems able to say how real the issue
is (ie, hard usage and performance numbers).

While larger MTUs are a good thing, they aren't very common and as
100BT gets more popular, I expect them to become even less common. I
guess the router/switch manufacturers will just have to be prepared
to handle high packet rates.

Or maybe we need a "reverse fragmentation" protocol. Packets to the
same destination could be lumped together and routed/switched as a
single packet.

- - - - - - - - - - - - - - - - -

In message <199605040558.AAA03102@freeside.fc.net>, Jeremy Porter writes:
>
> Paul, with all due respect, there is a lot of apparent need and
> demand for 100mbps interconnects, one could argue
> that 100 100mbps interconnects are better than 3 or 4 622mbps
> interconnects. I think 100 interconnects are at least as likely
> in 2 years as 4 622mbps interconnects.

That brings up another scaling problem. If one AS is attached to 100
or so interconnects, that's 100 border routers. That's not so bad
with one backbone (ie: national provider in the US). If there are 2
or more (there are and will be more than 2), and each reaches X% of
the 100 or so interconnects (where X% is 100% minus epsilon, say 95%
for example), then they will announce the routes they hear from one
interconnect to all other 100 or so minus 1 interconnects. The other
backbones will hear the same set of routes 100 or so times.

If we continue to do shortest path out, that's 100 paths (I'll drop
the "or so") to each prefix. That means we need to fix the "MED
latching" feature and use MED. With any good IBGP implementation, if
MEDs are different the routers with less preferred routes with
withhold their announcements. If so, we have another slight problem.
If the primary route is withdrawn, most or all of those less preferred
routes will get announces and gradually slosh around until the next
best route is installed everywhere and all others are again
suppressed.

This is not impossible for a router to handle, just a lot of route
flap to deal with and coding had better be very carefully done. There
is existance proof that suggests that handling high levels of route
flap can be done less than perfectly in ways that can promote
sustained instability.

The problem is that 100 interconnects will amplify route flap problems
by a factor of 100. Just a heads up for router vendors - just expect
to see high route flap loads and deal with it. Stability does not
require infinite CPU power, just algorithms which don't choke when the
load gets high, but rather converge at whatever rate they can sustain.

I suspect we will see both. A lot more interconnects and a lot faster
interconnects.

Curtis

- - - - - - - - - - - - - - - - -