Mailing List Archive

> Are there other topics you'd like to hear about?

How about

Analysis of Actual End to End Performance accross the NAPs/MAEs

To be given by each operator?

(as opposed to "everything is wonderful because we dropped no packets across
ten feet of level-2 wire")

randy
- - - - - - - - - - - - - - - - -

> Are there other topics you'd like to hear about?

How about

Analysis of Actual End to End Performance accross the NAPs/MAEs

To be given by each operator?

(as opposed to "everything is wonderful because we dropped no packets
across ten feet of level-2 wire")

An excellent topic, to be sure, but how do you propose that this be measured?

About the only thing NAP operators can directly gather and report upon is the
"ten feet of level-2 wire".

And before anyone color me a NAP apologist (I'm certainly not, as some
interconnect operators can probably attest), this is a serious question,
not an excuse for any poor performance that _may_ have been observed at
interconnects.

--Vince
- - - - - - - - - - - - - - - - -

>> Analysis of Actual End to End Performance accross the NAPs/MAEs
> An excellent topic, to be sure, but how do you propose that this be
> measured?

And there's a subject for a NANOG panel in itself.

I can think of some interesting experiments that would involve cooperation
of multiple peers. But there are folk far better based in measurment than I
who might suggest some fun stuff. Guy, Steve, ..., this is your cue.

randy
- - - - - - - - - - - - - - - - -

From various discussions at the FARNET meeting, I am sure there are more
folks participating in what Merit is doing in this area. Perhaps they
can share more about that at NANOG.

--
Stan | Academ Consulting Services |internet: sob@academ.com
Olan | For more info on academ, see this |uucp: {mcsun|amdahl}!academ!sob
Barber | URL- http://www.academ.com/academ |Opinions expressed are only mine.
- - - - - - - - - - - - - - - - -

Hi,

it's possible that I'm off base here, but...

> How about
>
> Analysis of Actual End to End Performance accross the NAPs/MAEs
>
> To be given by each operator?
>
> An excellent topic, to be sure, but how do you propose that
> this be measured?
>
> About the only thing NAP operators can directly gather and
> report upon is the "ten feet of level-2 wire".

Somehow I think Randy meant "connected network service operator"
rather than "NAP operator" when he said "operator". Isn't the
problem in some cases more with overloaded access circuits *into*
the NAP rather than the NAP interconnect medium itself?

How to measure? Well, I must admit that I do not have much of a
suggestion there. Pinging your own NAP router usually will not
reveal much regarding the quality of service delivered over the
access circuit, as the NAP router can fool around with where it
places it's self-originating packets in the output queue on the
access circuit.

- Havard
- - - - - - - - - - - - - - - - -

>From: Susan R. Harris[SMTP:srh@merit.edu]
>Sent: Friday, August 30, 1996 7:09 AM
>
>Greetings - Here's what we've lined up so far for the Oct. 24-25 NANOG
>agenda:
>
> - vBNS Update
> - Future router requirements
> - Multihoming pros and cons
> - Economics of Internet resource allocation


I believe there would be significant interest in a debate on the relative
merits of Peering and Transit oriented connectivity strategies. The vast
majority of operators rely at least partially on transit arrangements with
'upstream' providers, with many hoping to eventually transition (npi) to a
robust set of peering relationships. There are many technical and business
issues surrounding this topic, which is evolving rapidly over time as the
NAP environment changes. A deep exploration of the issues involving
presenters from a variety of perspectives would benefit everyone,
regardless of their chosen Approach.
--
Jim Browning


- - - - - - - - - - - - - - - - -

OK, you can color me a NAP apologist. As a NAP operator I don't have the
resources at my disposal to conduct the suggested tests. (A couple of DEC
Alpha or other capable workstations attached to the NAP media would be a
nice asset.) However, as a L2 network operator I would be VERY interested
in learning of my customers' perception of the service provided.

Steve


At 12:51 8.30.96, Vince Fuller wrote:
> > Are there other topics you'd like to hear about?
>
> How about
>
> Analysis of Actual End to End Performance accross the NAPs/MAEs
>
> To be given by each operator?
>
> (as opposed to "everything is wonderful because we dropped no packets
> across ten feet of level-2 wire")
>
>An excellent topic, to be sure, but how do you propose that this be measured?
>
>About the only thing NAP operators can directly gather and report upon is the
>"ten feet of level-2 wire".
>
>And before anyone color me a NAP apologist (I'm certainly not, as some
>interconnect operators can probably attest), this is a serious question,
>not an excuse for any poor performance that _may_ have been observed at
>interconnects.
>
> --Vince


********************************************************************************

Phone: 1.816.854.2113
Fax: 1.816.854.2201

Pager: 1.888.366.7890, PIN 398.6644

Alpha Page via Internet: 3986644@pagenet.net

********************************************************************************




- - - - - - - - - - - - - - - - -

On Fri, 30 Aug 1996, Steve Schnell, Sprint Corporation wrote:

> OK, you can color me a NAP apologist. As a NAP operator I don't have the
> resources at my disposal to conduct the suggested tests. (A couple of DEC
> Alpha or other capable workstations attached to the NAP media would be a
> nice asset.) However, as a L2 network operator I would be VERY interested
> in learning of my customers' perception of the service provided.

What would it take to set up a couple of Alphas, routers and zero-mile
T1's in a portable testing kit to go around from NAP to NAP and run tests?
I'm thinking of something like the mobile air-quality testing labs that
park in one location for a month, run tests, and then move on. Such
machines could run a full suite of tests including FTP'ing large files,
downloading complex web pages (i.e. multiple images) as well as the lower
level things like ping tests.

This kind of test would provide useful numbers that customers can
understand as well as point out problem areas that a NAP operator might
need to investigate. And with co-operative NAP peers, the same test kit
could be used with T1's out to various locations that feed into the NAP so
as to run the same tests across the peer's routers and lines.

Would this kind of testing reveal any useful information that could not be
gotten from examining router stats?

Michael Dillon - ISP & Internet Consulting
Memra Software Inc. - Fax: +1-604-546-3049
http://www.memra.com - E-mail: michael@memra.com

- - - - - - - - - - - - - - - - -

Randy,

I'm not sure we can provide the type of quantitative data your looking
for but we can discuss/present qualitative measures we are taking to
improve interconnectivity (i.e. more bandwidth).

Jim


On Fri, 30 Aug 1996, Randy Bush wrote:

> Date: Fri, 30 Aug 96 11:37 PDT
> From: Randy Bush <randy@psg.com>
> To: "Susan R. Harris" <srh@merit.edu>
> Cc: nanog@merit.edu
> Subject: Re: Agenda for next NANOG
>
> > Are there other topics you'd like to hear about?
>
> How about
>
> Analysis of Actual End to End Performance accross the NAPs/MAEs
>
> To be given by each operator?
>
> (as opposed to "everything is wonderful because we dropped no packets across
> ten feet of level-2 wire")
>
> randy
>
- - - - - - - - - - - - - - - - -

Hi Jim,

> I'm not sure we can provide the type of quantitative data your looking
> for but we can discuss/present qualitative measures we are taking to
> improve interconnectivity (i.e. more bandwidth).

The problem is that NANOG presentations always say how good it is and how
much better it is about to be. Not to pick on NAP ops, the NSPa and ISPs
have as well. We all do it. But the end customers are saying how bad it
is, and that it is getting worse.

Maybe it is time to get real metrics and real measurements of the different
pieces, so we can see where things are fine, where we have current problems,
and where problems could be in the future.

No, this is not an instant job. No, it will not be an easy job. But with
end users whining, and ill-informed press sensationalizing, it's getting
hard to sit through NANOG presentations where everybody says how wonderful
it all is.

As a community of operators, it would be cool to deal with real operational
metrics, not marketing glossies. And if we have to shut the press out, or
go to a more 'safe' forum, then that's cool too.

We need more Vern Paxson and less rose colored glasses.

randy
- - - - - - - - - - - - - - - - -

> Date: Fri, 30 Aug 1996 15:59:23 -0700 (PDT)
> From: Michael Dillon <michael@memra.com>
> To: nanog@merit.edu
> Subject: Re: Agenda for next NANOG
> [...]
> What would it take to set up a couple of Alphas, routers and zero-mile
> T1's in a portable testing kit to go around from NAP to NAP and run tests?
> [...]
> Such
> machines could run a full suite of tests including FTP'ing large files,
> downloading complex web pages (i.e. multiple images) as well as the lower
> level things like ping tests.
>
> This kind of test would provide useful numbers that customers can
> understand as well as point out problem areas that a NAP operator might
> need to investigate. And with co-operative NAP peers, the same test kit
> could be used with T1's out to various locations that feed into the NAP so
> as to run the same tests across the peer's routers and lines.
>
> Would this kind of testing reveal any useful information that could not be
> gotten from examining router stats?

Actually, it would be nice if this "portable test kit" were actually an
optional board for a router. You could, for example, stick the "portable
test kit" board in a router and then test at will.

We also tried to explain to our ATM tester vendors that we wanted ttcp
implemented in the ATM tester. Some understood, some didn't...

-tjs
- - - - - - - - - - - - - - - - -

On Fri, 30 Aug 1996, Randy Bush wrote:

> Maybe it is time to get real metrics and real measurements of the different
> pieces, so we can see where things are fine, where we have current problems,
> and where problems could be in the future.

This goes beyond just public interconnects. If you are one level removed from
NSP level, the status and performance of private interconnects between various
NSPs become a subject of great deal of interest.

Here, the general dearth of information about performance and status gets even
worse than it is with public exchange points.

I wonder if this is something involved parties (MCI, Sprint, UUNet and ANS)
are willing to talk about, given that I've noticed some degradation on
performance across some of these private interconnects already.. (and it's
only been couple of months since they came online!)

I understand that these are private arrangements between involved parties, but
given their importance, it would be a great service to the community if the
involved parties could share some information.

-dorian


- - - - - - - - - - - - - - - - -

In message <199608310000.TAA17929@uh.msc.edu>, Tim Salo writes:
>
> Actually, it would be nice if this "portable test kit" were actually an
> optional board for a router. You could, for example, stick the "portable
> test kit" board in a router and then test at will.

Any of the ISPs losing packets like crazy need only look at the stats
on their routers.

We saw some creative lying at the last NANOG. I heard the statement
"we have no loss on our links". No kidding. Now look on the routers.
That is where congestion loss occurs and that is where loss due
tolimitations in the router implementation will be recorded. The
providers that are losing a lot of poackets can very easily look at
these stats but they aren't about to put them on a viewgraph and bring
them to NANOG.

> We also tried to explain to our ATM tester vendors that we wanted ttcp
> implemented in the ATM tester. Some understood, some didn't...
>
> -tjs

This is an interesting problem and actually close to solvable at the
IP level. I'm sure you are familiar with PPP LQM. Take out the PPP
part and keep the LQM packet format and local storage. Keep one LQM
struct per ARP entry on a bcast or nbma interface (such as an ATM
NAP). Count packets used by each ARP entry and update SNMP and the
LQM entry as you would in PPP LQM. Occasionally (once a second is
fine) send an LQM packet summarizing what has been sent.

When you make two successive queries from the command interface one
either side, you cat check the differences and get an exact (accurate
to the packet) count of the packets sent during that period and the
loss during that period.

Then you just have to drive the network with whatever load you think
appropriate.

My understanding is that the ATM NAPs are not experiencing any level 2
loss at all so this is really not an issue for the NAPs. This clearly
would be useful for ISPs using ATM or any switched service that
doesn't absolutely guarentee no loss (ie: != VBR-V).

Curtis

- - - - - - - - - - - - - - - - -

On Fri, 30 Aug 1996, Curtis Villamizar wrote:

> Any of the ISPs losing packets like crazy need only look at the stats
> on their routers.

Maybe these ISP's don't know that there are stats on their routers or that
there is packet loss there. As an NSP you would be doing your customers a
service by telling them about such things. It doesn't help to call these
people clueless idiots and suggest that they have no business running an
ISP even if it is true.

And even if an ISP monitors their router stats and sees peak traffic at
around 60% of a T1 they could still have congestion problems elsewhere in
their network such as their Ethernet or lines to POP's. All of this stuff
contributes to public perceptions of a slow Internet and somebody has to
take the first step to do end-to-end testing and point out where these
trouble spots really are.


Michael Dillon - ISP & Internet Consulting
Memra Software Inc. - Fax: +1-604-546-3049
http://www.memra.com - E-mail: michael@memra.com

- - - - - - - - - - - - - - - - -

In message <Pine.SOL.3.95.960830201535.15619K-100000@nic.hq.cic.net>, "Dorian R
. Kim" writes:
> On Fri, 30 Aug 1996, Randy Bush wrote:
>
> > Maybe it is time to get real metrics and real measurements of the different
> > pieces, so we can see where things are fine, where we have current problems
> ,
> > and where problems could be in the future.
>
> This goes beyond just public interconnects. If you are one level removed from
> NSP level, the status and performance of private interconnects between variou
> s
> NSPs become a subject of great deal of interest.
>
> Here, the general dearth of information about performance and status gets eve
> n
> worse than it is with public exchange points.
>
> I wonder if this is something involved parties (MCI, Sprint, UUNet and ANS)
> are willing to talk about, given that I've noticed some degradation on
> performance across some of these private interconnects already.. (and it's
> only been couple of months since they came online!)
>
> I understand that these are private arrangements between involved parties, bu
> t
> given their importance, it would be a great service to the community if the
> involved parties could share some information.
>
> -dorian


Dorian,

I think your best bet is to run NPD. btw- Did you look at the NetNow
stats while they were still up?

Someone from Merit? Why were these stats removed?

Curtis
- - - - - - - - - - - - - - - - -

Curtis,

> I think your best bet is to run NPD.

Huh? That looks at routing [in]stability, not goodput. Is this an
intentional red herring?

randy
- - - - - - - - - - - - - - - - -

......... Curtis Villamizar is rumored to have said:
]
]
] ... btw- Did you look at the NetNow
] stats while they were still up?
]
] Someone from Merit? Why were these stats removed?
]
] Curtis
]

Maybe Robert Metcalfe was using them as resource material?

;-)

-alan
- - - - - - - - - - - - - - - - -

In message <Pine.BSI.3.93.960830205734.25896N-100000@sidhe.memra.com>, Michael
Dillon writes:

> And even if an ISP monitors their router stats and sees peak traffic at
> around 60% of a T1 they could still have congestion problems elsewhere in
> their network such as their Ethernet or lines to POP's. All of this stuff
> contributes to public perceptions of a slow Internet and somebody has to
> take the first step to do end-to-end testing and point out where these
> trouble spots really are.

I agree with you completely on this. I was talking about loss in T3
(and one OC3 that isn't all OC3) backbones. A congested serial link
will drop packets but if it is the only bottleneck TCP will usually
not push anything through the congested link twice. (Fast retransmit
and fast recovery just retransmit what never went through the
bottleneck in the first place). When you can't get anything anywhere
near 28.8 on your 28.8, you get upset (or you don't get 56k on your
56k or T1 on your T1s).

Curtis

ps- I've copied as much as 600 MB and got about 160 KB/sec (1/2 T1)
from Ann Arbor through the Elmsford T1 at mid day. That's reasonable
performance. Not everyone can do that on their provider's backbone.
Try it sometime. (Might want to go with less than 600 MB). I didn't
tweak the code and open the TCP window or it might have been more (but
folks in Elmsford might be slightly unhappy, even though it was TCP).
- - - - - - - - - - - - - - - - -

> > I think your best bet is to run NPD.
>
> Huh? That looks at routing [in]stability, not goodput. Is this an
> intentional red herring?

While the routing study I presented at the Feb NANOG was based on measurements
made using NPD, the daemon can also be used for taking TCP data. I'm right
now analyzing a bunch of such and hope to have some results to present at
the early '97 NANOG.

That said, NPD (1) does *not* provide any *analysis* itself, just measurement,
and (2) is far from industrial strength. So it's not something that someone
can use right now for doing any sort of standardized measurement.

Matt Mathis, Jamshid Mahdavi (both of PSC) & I are hoping to get funded to
build an industrial strength version of NPD, along with the key pieces of
the infrastructure for using it on a widespread basis, and analysis tools
for automating measurements. If this comes through, then with luck we'll
have something to present about it at the '97 NANOG, too.

Vern
- - - - - - - - - - - - - - - - -

>> Huh? That looks at routing [in]stability, not goodput.
>
> While the routing study I presented at the Feb NANOG was based on
> measurements made using NPD, the daemon can also be used for taking TCP
> data. I'm right now analyzing a bunch of such and hope to have some
> results to present at the early '97 NANOG.

Apologies to you and Curtis. Time to go to sleep.

randy
- - - - - - - - - - - - - - - - -

In message <m0uwhqD-0007zqC@rip.psg.com>, Randy Bush writes:
> Curtis,
>
> > I think your best bet is to run NPD.
>
> Huh? That looks at routing [in]stability, not goodput. Is this an
> intentional red herring?
>
> randy

No it wasn't an intentional red herring.

I thought NPD did more than that, since Vern made other measurements.

I also feel the NetNow stuff is valuable.

Curtis
- - - - - - - - - - - - - - - - -

At 01:11 PM 8/30/96 PDT, Randy Bush wrote:

>>> Analysis of Actual End to End Performance accross the NAPs/MAEs
>> An excellent topic, to be sure, but how do you propose that this be
>> measured?
>
>And there's a subject for a NANOG panel in itself.
>
>I can think of some interesting experiments that would involve cooperation
>of multiple peers. But there are folk far better based in measurment than I
>who might suggest some fun stuff. Guy, Steve, ..., this is your cue.
>
>randy
>

... And it might even be nice for someone who is active in the
IETF Benchmarking Methodology WG (bmwg) to participate.

[ref: http://www.ietf.org/html.charters/bmwg-charter.html]

- paul

- - - - - - - - - - - - - - - - -

Randy,
There is a cluster of closely-related variants on the issue
you raise:
[] to what extent does a given exchange point (NAP/MAE/etc) constrain
the performance that a user sees (in what a user thinks of as end-
to-end). For example, an FTP could flow at 800 kb/s for a given
pair of users, except that MAE-north is congested, so the FTP can
only flow at 400 kb/s.
[] to what extent does a given exchange point constrain the performance
that a provider sees (in what a provider thinks of as end-to-end).
For example, a given pair of backbones could sustain 20 Mb/s over a
private interconnect with acceptable packet loss, but can only sustain
10 Mb/s over the Altoona NAP.
From different perspectives, each of these notions of end-to-end has
meaning. Would you want to consider one or the other or both in the
panel?
-- Guy

At 01:11 PM 8/30/96 PDT, Randy Bush wrote:
>>> Analysis of Actual End to End Performance accross the NAPs/MAEs
>> An excellent topic, to be sure, but how do you propose that this be
>> measured?
>
>And there's a subject for a NANOG panel in itself.
>
>I can think of some interesting experiments that would involve cooperation
>of multiple peers. But there are folk far better based in measurment than I
>who might suggest some fun stuff. Guy, Steve, ..., this is your cue.
>
>randy
>

- - - - - - - - - - - - - - - - -

Paul,
The relevant BMWG work here is within the IPPM effort. The email
list is
ippm@advanced.org
with the unsurprising
ippm-request@advanced.org
to join/unjoin.
-- Guy

At 01:29 PM 8/31/96 -0400, Paul Ferguson wrote:
>At 01:11 PM 8/30/96 PDT, Randy Bush wrote:
>
>>>> Analysis of Actual End to End Performance accross the NAPs/MAEs
>>> An excellent topic, to be sure, but how do you propose that this be
>>> measured?
>>
>>And there's a subject for a NANOG panel in itself.
>>
>>I can think of some interesting experiments that would involve cooperation
>>of multiple peers. But there are folk far better based in measurment than I
>>who might suggest some fun stuff. Guy, Steve, ..., this is your cue.
>>
>>randy
>>
>
>... And it might even be nice for someone who is active in the
>IETF Benchmarking Methodology WG (bmwg) to participate.
>
>[ref: http://www.ietf.org/html.charters/bmwg-charter.html]
>
>- paul
>

- - - - - - - - - - - - - - - - -