Mailing List Archive

Hans-Werner,

> This all is probably true if the objective is to provide the best
> possible IP service. If you apply other requirements, like integration
> of the current telephony system, or, more accurately, evolve the
> telephony system, in the telephony company mindset, to something that
> also supports data, video, and things like that, things look
> different.

I can't disagree with this but I think it misses the point, which
is an issue which should be orthogonal to ATM. I think it may very
well be the case that it is not possible to build a really large,
reliable Internet without really large IP routers. I.e. it is not
ATM that is the problem, it is this picture of a stinking big ATM
switch milking hoards of teeny little routers the may not provide
a scalable Internet. If you want to build a big Internet you may
very well need big routers, it matters little whether they're
plugged into an ATM switch or directly into a TDM (with the proviso
that the router interfaces to plug into the former are going to
be a technology generation or two more complex than the latter, no
matter what you do).

We know we want to build a big Internet. Yet somehow the big routers
one may absolutely require to do this became (a) uninteresting as
topics of research and development, (b) not well-understood as a necessity
because of the big-switch-little-router picture, even though this may
have no relationship to working reality, and (c) even if none of the
above, the big-router development may still be constrained by the view
that a big router is useless unless it is equipped with big ATM interfaces,
even though the latter may turn out to be harder to build than the
former and we're really getting the cart before the horse. So the end
result is, no big routers, even though there is apparently little about
ATM which eliminates the need for big routers if you want a big Internet.
And all ATM has really managed to deliver so far is obfuscation of the
issues, putting us in a holding pattern where we're just waiting to see
what breaks but not doing that much about it.

So I can't deny that the integration thing is attactive (I personally
don't dislike ATM either, to tell the truth), but I really do think
that somehow the priorities got all mixed up somewhere along the way.
Instead of primarily worrying about advancing the state of the art
for each of the services people actually want to buy, using whatever
delivery technology was mature enough to accomodate this reliably,
we've somehow instead made the integration the holy grail and have
just forgotten to even care about whether there's anything left worth
integrating by the time we find it.

> Then again, so far I see little activity in the context of
> service integration. More ATM as a level-2 replacement for data
> networking. Which brings us back to your comments, as in such an
> environment the benefit is more marginal (e.g., ATM may still have
> multiple service qualities before it is being implemented in an IP(+)
> substrate). Oh well. If there were just concerted goals.

I would suggest that the reason you don't see a lot of activity in
the context of service integration is that (at least at the phone
company I know a little about) the guys that run the voice network are
about the most pragmatic people in the place, which means they make
sure their vendors deliver the big iron switches when they need them,
and that they've got enough fiber in the ground to connect them together,
so that even if it turns out that ATM isn't as great a thing as it was
thought it might be, you still won't get a lot of fast busy signals. I
wish one could say the same about the Internet.

Dennis Ferguson

> ... If you find that you prefer raw bandwidth over optical fiber
> and lots of cisco interfaces, fine, but if you like network level switching,
> shared network bandwidth and the price for that, buy switched services.

I find that the telco-supplied fast packet service we use inside CIX goes down
several times a day and that about once every 60 hours it is necessary to call
the switch technicians and have them power down my line module (which takes
two minutes and is usually their prelude to swapping out a card) and then
powering it back up (which takes two more minutes for diagnostics) before I
can again exchange packets with anybody.

I misspoke last time I flamed about this. There's probably very little that's
intrinsically wrong (technically speaking) with these fastpacket data services.
But the fact is, they are still so new to the telcos that they are not as
bottom-line reliable as the things telcos have been providing for much longer,
like raw bit pipes.

So for example, if my three-carrier T3 buildout won't pass bits, I can make a
four-party conference call with a pile of semi-experienced knuckle dragging
switch techs and in a few hours over a few days I can cause someone to isolate
some problem, fix it, and then we all move on to the next problem. With the
fast packet services (SMDS and ATM, at least), when it doesn't work it takes
a hell of a lot longer to fix and sometimes it doesn't get fixed at all.

F-R, as I said earlier, seems to be the exception to this. ATM may be on the
verge of getting better with the new generation of switches Kent spoke of.
But up 'til now it's been better for me to buy the kind of bandwidth that the
knuckledraggers down at my local telco know how to fix, and then put all the
switching intelligence into a box I can lay my own hands on.

Your mileage may vary.

I'll hazard flames and put my oar in...

[.various incisive comments about the disparate evolution of network
connectivity hardware and network routing hardware ommitted]

There seems to be some consensus, at least for some individuals/organizations,
that bigger/faster/better router hardware is an essential, since the dairy barn
approach currently in favor amongst the data comm equipment manufacturers may
not prove scalable.

However, the discussion so far has focussed on what won't work rather than
what specific features etc. are needed. In the cause of keeping pace with the
expansion (as well as a partisan reason or two ;-), what sort of beasts should
this bigger, faster, better generation of routers look like? If fast packet
services and co-location on a LAN milking machine won't cut it, then what's the
thinking as to what *is* needed to do the job?

Clueing the various equipment manufacturers in as to what direction the
hardware should be evolving in is probably at least worth the effort (not to
imply that individual efforts have not been underway for some time, but a
collective voice often carries more weight). Unfortunately, this goes beyond
simply getting a single company in touch with reality because of the plethora
of pieces involved in building backbone and delivery services. However, this is
probably one of the few "right groups of people" to be formulating these
requirements and then bashing the various vendors over the head with them ;-).

A fast .02 on my part would be that something in the way of non-blocking
backplane design needs to be _advanced_ in order to be able to scale current
router technology up. This to start tackling the traffic aggregation
problems... Distibuted switching and multiple transport busses seem
like steps in the right direction architecturally, but what are the
topological and traffic flow realities that instantiation of these methods
should be grounded in? Internet and Enterprise networking are markedly
different, and at least until recently the focus of the manufacturers has been
on the Enterprise...

Also, a quick-and-dirty point-to-point delivery mechanism scalable along with
the SONET hierarchy wouldn't hurt either, since as has been pointed out the
fast-packet overhead for multiple and multi- point signalling (or, worse,
data/voice/video integration) becomes an exacerbating bandwidth tax in the
traditional point-to-point circuits used to build Internet backbones and ISP
networks. This would also address Paul Vixie's all-too-true issues about the
ability of the carriers to support the services in an efficient and reliable
manner. If the transport is something they are familiar with (T3 being a good
example, and ATM of course standing as a bad one... ;), then the reliability of
the Net as a whole improves as more ISP's become able to use "supportable
carrier transports". Of course, the trick is to get the supportability of T3
with the speed of OC-48... ;-) (not that there are any routers that could
*handle* OC-48... arghhh... ;-) :-p)

I'll be happy to assist in the bashing in my day job ;-), but since I must in
conscience agree with Dennis wholeheartedly on the subject of ATM and it's
orthogonal relationship to current service delivery needs, I'm definitely
speaking solely as myself here. Curiosity got the better of me, tho, and
I'd like to know what other thoughts there are along these lines...

Cheers,
Paul

Paul "Corwin" Frommeyer
Work Internet Engineer, CCIE Play
ISP Systems Engineer Network Sorcerer At Large
Cisco Systems, Inc. Paul's Fone Company
pfrommey@cisco.com corwin@palas.com
*** Speaking solely for myself unless otherwise noted ***

On Sun, 21 Jan 1996 15:28:43 -0800 (PST) Hans-Werner Braun wrote:
> This all is probably true if the objective is to provide the best
> possible IP service. If you apply other requirements, like integration
> of the current telephony system, or, more accurately, evolve the
> telephony system, in the telephony company mindset, to something that
> also supports data, video, and things like that, things look
> different. Then again, so far I see little activity in the context of
> service integration. More ATM as a level-2 replacement for data
> networking. Which brings us back to your comments, as in such an
> environment the benefit is more marginal (e.g., ATM may still have
> multiple service qualities before it is being implemented in an IP(+)
> substrate). Oh well. If there were just concerted goals.

Telephony system, Telephony system... Oh yeah, I remember! That was
that system they had back in the twentieth century for carrying
primitive voice communications. I even think they used to run data
across it in the days before the cable companies dominated.

later,
fletcher

While we are on the subject of delay-bandwidth buffering in ATM switches.
Does anyone know where I can get a router that has adequate buffering
for those pesky little 1/4 Kbyte average size packets that keep
floating around the net :).

-joe

PS. Also does someone have numbers for the amount of
buffering in a DEC gigaswitch, and information on their buffer
managment (i.e. variable length buffers vs fixed length buffers).

At 04:54 AM 1/20/96 -0700, Dave Siegel wrote:

>> >> This breaks down when the connection speed goes from DS-3 to OC-3.
>> >
>> >LAN speeds can keep up with WAN speeds at OC-3, just use ATM.
>> >
>> >---
>> >Stephen Balbach "Driving the Internet to Work"
>> >VP, ClarkNet due to the high volume of mail I receive please quote
>>
>> You are, obviously, kidding.
>> --
>> Paul Ferguson || ||
>
>He's obviously talking about some kind of LAN quite unlike Ethernet. ;-)
>
>Dave
>

Obviously. ,-)

- paul

--
Paul Ferguson || ||
Consulting Engineering || ||
Reston, Virginia USA |||| ||||
tel: +1.703.716.9538 ..:||||||:..:||||||:..
e-mail: pferguso@cisco.com c i s c o S y s t e m s

> To: nanog@merit.edu
> Subject: Re: value of co-location
> Date: Mon, 22 Jan 1996 09:29:06 -0500
> From: "Joseph Lawrence" <lawrence@mci.net>
> [...]
> PS. Also does someone have numbers for the amount of
> buffering in a DEC gigaswitch, and information on their buffer
> managment (i.e. variable length buffers vs fixed length buffers).

DEC has a nice technical report about the flow control mechanism used in
the GigaSwitch. Your DEC salesperson should be able to get you a copy,
and probably some other interesting stuff, (make your salesperson earn
his or her living...).

In some sense, you may be asking the wrong question. I think the
GigaSwitch flow control mechanism pushes the buffering requirements back
to the FDDI stations (i.e., routers). Consult the DEC report for the
real answer.

If necessary, I can dig up the document number.

-tjs

| Bottom line: about 270,000 pps per port, 14 microsec.
| forwarding latency AND superior reliability. The choice
| for NAP designers everywhere :)-

Bilal, I think you missed a word.

"Successful" seems to have been omitted between "choice" and "for".

Some NAP designers opted for packet shredders, and might
even be getting some thousands of pps total traffic (so they
claim, but then they seem to count very local (i.e.,
cross-town) ATM connectivity as "NAP" traffic), as opposed to
the low tens of thousands of packets per second *per port*,
with much of that being traffic between sites with about 30
times the delay * bandwidth buffering requirements.

Of course, the fact that the switched FDDI exchange points
have proven to be more reliable in practice than the ATM
exchange points have -- even with a fraction of the load --
tends to do nothing to diminish the religious fervour of
the people who assert that ATM NAPs are the ultimate single
answer to the needs of the Internet.

I wonder sometimes if their brains were cellified and passed
through an ATM NAP...

Sean.

> this bigger, faster, better generation of routers look like? If fast packet
> services and co-location on a LAN milking machine won't cut it, then what's
> the thinking as to what *is* needed to do the job?

There are three ways to approach this question.

(1) Bet on ATM. If ATM succeeds, we will be able to buy an arbitrary sized
pipe into a network about whose nature and extent we know and care naught,
and we will be able to send packets to any destination in the universe without
paying special attention to the instantaneous measured bandwidth or delay.

(2) Imagine a router that used a backplane sort of like the one in the GIGA
switches only on a larger scale; the goal is to come up with a router the size
of a 5ESS with similar capacity (and which is installable and maintainable by
telco knuckledraggers). Get Bellcore or some other consortia-based lab with
near-infinitely deep pockets to invest a near-infinite amount of money building
it. Wait for the telcos who own worldwide fibre plants to buy 'em.

(3) Sign an NDA and get Vadim to tell you what he's up to at Ncubed.

| the goal is to come up with a router the size of a 5ESS
| with similar capacity ^^^^^^^^^^^^^^^^^^
^^^^^^^
greater bigger than

Just as an aside, the number of switched minutes for
PDN/VPN dialup data traffic of all sorts (IP and non-IP,
Internet and private network) through one particularly
data-oriented IXC is widely expected to exceed the
total number of switched minutes for voice traffic
by the middle of summer, this year.

It would astonish me to learn that a number of other IXCs
and some LECs have not made strikingly similar predictions,
give or take some weeks or months.

| (and which is installable and maintainable by telco knuckledraggers)

You got that part right.

Sean.

At 05:11 PM 1/20/96 -0500, Vadim Antonov wrote:
>George H. Clapp <clapp@bellcore.com> wrote:
>
>>The reasons for co-location that have been mentioned so far include...
>
>>- zero-mile circuits
>
>That means expansion of that "circuit" is free.

What is meant by free expansion of the circuit?


>>'Zero-mile circuit' isn't clear to me because, regardless of the technology
>>used for the NAP, it's still necessary to purchase a circuit from your site
>>to the NAP.
>
>The difference is between the clearline circuit and circuit with 40%
>protocol overhead. Some big players are carriers themselves and
>want to use _own_ circuits whereever possible.

What isn't mentioned is the advantage of data-link layer multiplexing that
the Fast Packet Services offer. Rather than bringing in multiple circuits
and consuming multiple ports on your router, it's possible to carry traffic
from customers as well as traffic to the NAP on that same interface. That's
a fundamental point which is driving much of the deployment of the Fast
Packet Services.

George Clapp
voice: 201-829-4610
fax: 201-829-2504
page: 800-980-1298
email: clapp@bellcore.com

That Vix guy wrote:
] (3) Sign an NDA and get Vadim to tell you what he's up to at Ncubed.

I signed the NDA, but I didn't spell my name right, so it's not
legally binding. Basically, anyone who invests over 10k gets a
seat on the space ship he's building to fly to the planet Zorgo,
where we will all pilfer advanced technology, and bring it back to
the planet earth to make us all rich.

Can I still have a seat, Vadim?

-alan

^-- getting frustrated at hearing about this exquisite piece of
technological concept, w/out being in the know.

On Mon, 22 Jan 1996, Sean Doran wrote:

> | Bottom line: about 270,000 pps per port, 14 microsec.

... how about 4 microseconds latency and 15 million cells/sec?
(for the same price)

Mike

(nsc ers does that I guess)


> | forwarding latency AND superior reliability. The choice
> | for NAP designers everywhere :)-
>
> Bilal, I think you missed a word.
>
> "Successful" seems to have been omitted between "choice" and "for".
>
> Some NAP designers opted for packet shredders, and might
> even be getting some thousands of pps total traffic (so they
> claim, but then they seem to count very local (i.e.,
> cross-town) ATM connectivity as "NAP" traffic), as opposed to
> the low tens of thousands of packets per second *per port*,
> with much of that being traffic between sites with about 30
> times the delay * bandwidth buffering requirements.
>
> Of course, the fact that the switched FDDI exchange points
> have proven to be more reliable in practice than the ATM
> exchange points have -- even with a fraction of the load --
> tends to do nothing to diminish the religious fervour of
> the people who assert that ATM NAPs are the ultimate single
> answer to the needs of the Internet.
>
> I wonder sometimes if their brains were cellified and passed
> through an ATM NAP...
>
> Sean.
>

----------------------------------------------------------
IDT
Michael F. Nittmann ---------
Senior Network Architect \ /
(201) 928 1000 xt 500 -------
(201) 928 1888 FAX \ /
mn@tremere.ios.com ---
V
IOS

> I signed the NDA, but I didn't spell my name right, so it's not
> legally binding. Basically, anyone who invests over 10k gets a
> seat on the space ship he's building to fly to the planet Zorgo,
> where we will all pilfer advanced technology, and bring it back to
> the planet earth to make us all rich.

> Can I still have a seat, Vadim?

Sure, but since you've broken NDA you won't get seat in the first
class, so we'll use a regular microwave oven to thaw (and bake)
you on arrival. You see, that's better than sacrificing guiltless
bovines.

--vadim

> PS. Also does someone have numbers for the amount of
> buffering in a DEC gigaswitch, and information on their buffer
> managment (i.e. variable length buffers vs fixed length buffers).

The Gigaswitch SCP card has (at last count) 16MB of DRAM for buffering
packets (and other stuff) that get forwarded by the crossbar. The FGL
cards (FDDI line cards) have 1MB of DRAM.

There's a Digital Technical journal article available on-line that
describes the Gigaswitch in detail. The URL is

http://www.digital.com/info/hpc/sc95/sygiga-dtj.html

Buffer management is discussed in the "Design Issues" section:

http://www.digital.com/.i/info/hpc/sc95/sygiga-dtj-design.html

Module details (like types and quantities of memory) are listed in the
(hm) "Module Details" section:

http://www.digital.com/.i/info/hpc/sc95/sygiga-dtj-module.html

We have a Gigaswitch/FDDI at the Palo Alto interconnect, if you'd like
to have a look at one.

Stephen
- -----
Stephen Stuart stuart@pa.dec.com
Network Systems Laboratory
Digital Equipment Corporation

In message <96Jan23.111533-0000_est.20608+185@chops.icp.net>, Sean Doran writes
:
>
> | If ATM is being used in VBR without the V mode, essentially providing
> | point to point connections between routers at above DS3 rate, then
> | there is no need for complex reassembly or any form of congestion
> | control. That may turn out to be the way ATM is used by ISPs.
>
> Ok, so in order to do VBR without the V mode, one effectively
> has to either *really* trust one's ATM network provider not
> to have any congestion which could lead to cell loss as we
> have seen pretty clearly that even very minimal, occasional
> cell loss is deadly -- or one has to do the UUPSI hack:
> purchase clear-channel L0 circuits and run ATM as a L1
> protocol on top of it.
>
> That is, the only ATM switches are the ones at the end points
> and owned by the NSP.

That's not true. VBR without the V means setting PCR and SCR tot the
same value and reserving a VC and traffic shaping your output so as to
never exceed that value. This would allow you to nail up a X Mb/s
pipe (where DS1 << X << DS3 might be cost effective, or DS3 < X << OC3
might also be cost effective). The is no opportunity for multiplexing
gain if numerous telco customers do this (ie: are forced to do this if
ATM buffering and congestion mechanisms are not improved), but does
provide lossless service (unless something isn't working).

The UUPSI hack works just as well if you need more than DS3 and either
can't get ATM service from your provider or VBR minus V service is not
cost effective relative to OC3 SONET. In that case PPP over SONET
would be a technically better choice if you can get it. I won't
comment in public on the likelyhood of this.

> It is my prediction that by this spring, the only reasons why
> anybody would want to run ATM for connecting pieces of the
> Internet would involve unavailability of SONET/SDH from one's
> carrier(s), the desire to save money on point-to-point
> circuits by letting one's carrier(s) sell one VCs over some
> unknown fabric rather than real circuits over their
> transmissions infrastructure, an already-installed ATM
> fabric, or general insanity.

I mostly agree if you constrain "everybody" in the statement above to
mean the major ISPs. I think you still may be missing the point that
VBR without the V is supposed to be lossless, not like UBR (currently
highly lossy under congestion by design :) or ABR (high vapor
preasure, as of yet unproven, but still might just work).

ATM doesn't solve everything, but it also isn't comkpletely useless.
As George Clapp points out, some people are using ATM successfully.
IMO just don't believe everything you read or hear about ATM without
thinking about how it will work in your application (not you Sean,
anyone's application, whether ISP or LAN or otherwise).

Curtis

ps - Regarding Kent's comments. There are other routers on the
horizon (beside Netstar and Vadim's deep secret and Ipsilon's "we're
not saying what it is").