Mailing List Archive

On Fri, 3 Nov 1995, Hans-Werner Braun wrote:

> I think Dave has the right idea here. Given the lousy overall network
> performance that I (and others) are often seeing for months from
> varieties of service providers, I think the service providers should be
> forced to provide rebates. I frequently have 10% packet losses to get
> from where I am to the Bay area (via New York). And my service
> provider (CERFnet) is telling me that their service provider (Sprint)
> is not even answering to their trouble reports.

Well, I think the problem is that providers lock people in 1 and 2 year
contracts so if people get bad service they are stuck. That is why I only
have month to month service, if people don't like there T1 then they can
quit. I think we need more providers to do that, I know of a lot users
that are on providers that want to switch, but have 8 months left on
their contract.

Nathan Stratton CEO, NetRail, Inc. Your Gateway to the World!
---------------------------------------------------------------------------
Phone (703)524-4800 NetRail, Inc.
Fax (703)534-5033 2007 N. 15 St. Suite 5
Email sales@netrail.net Arlington, Va. 22201
WWW http://www.netrail.net/ Access: (703) 524-4802 guest
---------------------------------------------------------------------------

Good point. If there is no service model and penalties for bad
performance, at least there should be a possibility for short time
horizons.

I am really curious how service providers believe that, say, 12-24
months from now customers will still just put up with the current
situation. If I were them, I would plan for survival in that time
frame and get the right things under way *now*, and before someone
else changes things instead.

>On Fri, 3 Nov 1995, Hans-Werner Braun wrote:
>
>> I think Dave has the right idea here. Given the lousy overall network
>> performance that I (and others) are often seeing for months from
>> varieties of service providers, I think the service providers should be
>> forced to provide rebates. I frequently have 10% packet losses to get
>> from where I am to the Bay area (via New York). And my service
>> provider (CERFnet) is telling me that their service provider (Sprint)
>> is not even answering to their trouble reports.
>
>Well, I think the problem is that providers lock people in 1 and 2 year
>contracts so if people get bad service they are stuck. That is why I only
>have month to month service, if people don't like there T1 then they can
>quit. I think we need more providers to do that, I know of a lot users
>that are on providers that want to switch, but have 8 months left on
>their contract.
>
>Nathan Stratton CEO, NetRail, Inc. Your Gateway to the World!
>---------------------------------------------------------------------------
>Phone (703)524-4800 NetRail, Inc.
>Fax (703)534-5033 2007 N. 15 St. Suite 5
>Email sales@netrail.net Arlington, Va. 22201
>WWW http://www.netrail.net/ Access: (703) 524-4802 guest
>---------------------------------------------------------------------------
>
>

I have heard this argument very many times (and used it myself, and
sure understand it at a technical level). It is a very network-centric
argument serving a specific service provider. As a customer I don't
buy it. You guys need to take a customer centric approach and make the
customers happy. You sell yourselves and get money as services providers
for the *Internet* not your *local environment*. If you only sell
XYZnet services, not problem, but please then do not advertise you
provide Internet services just because you are marginally connected to
the rest of the world with no clue about how to make the NANOG and
global system work.

In the end this will be a market driven by customers, not service
providers. Customers will make the rules and will determine whether a
service is good or lousy. Look for other examples. If my power outlets
would regularly drop to 50 volts (or zero) I would get quite irritated.
"Power" is sold on the open market between service providers. I would
not accept an argument from a local service provider that my power is
always dropping because some service provider 2,000 miles away is
screwing up. I would consider that to be their problem to watch out for
such things and to coordinate it right. Same with phones. I don't care
what region you are in, but if I would call you, above the 99th
percentile, as long as you are close to your phone and pick up, my call
will get through and work without significant service degradations
*despite* the fact that there are at least three service prodivers
(local, long distance, local) involved. Why? Because power and phone
companies have their shit together on that stuff, and coordinate and
cooperate because they do understand they are all in the same boat. Of
course, obviously it is not a fairly new and anarchic environment
there, but has grown quite well into coordination. Are you guys up to
it, or would you need regulation do it for you? Y'know, life can be
easier if your parents tell you what to do, if you cannot get your act
together yourself. Less stress, too. Less flexibility as well, perhaps,
but methinks we have to choose some optimization function here, and buy
off on the cost.

In the end, this is not a sandbox for having a good time. People today
*depend* on their network connection, and that it works is importenat
to them. You *have* to go beyond just thinking locally.


>On Sat, 4 Nov 1995, Nathan Stratton wrote:
>
>> On Fri, 3 Nov 1995, Hans-Werner Braun wrote:
>>
>> > I think Dave has the right idea here. Given the lousy overall network
>> > performance that I (and others) are often seeing for months from
>> > varieties of service providers, I think the service providers should be
>> > forced to provide rebates. I frequently have 10% packet losses to get
>> > from where I am to the Bay area (via New York). And my service
>> > provider (CERFnet) is telling me that their service provider (Sprint)
>> > is not even answering to their trouble reports.
>>
>> Well, I think the problem is that providers lock people in 1 and 2 year
>> contracts so if people get bad service they are stuck. That is why I only
>> have month to month service, if people don't like there T1 then they can
>> quit. I think we need more providers to do that, I know of a lot users
>> that are on providers that want to switch, but have 8 months left on
>> their contract.
>
>Some comments from my side, which are, however, not official comments of
>IDT Internet Services:
>
>Internet works the way that a provider can only guarantee a standard of
>quality within the perimeter of the provider's networks. E.g. guarantee a
>customer that he is a certain number of hops away to the meetpoints, or
>hand out a latency matrix between POPs, guarantee that the packet loss is
>under a certain margin, and of course, guarantee a certain percentage of
>uptime.
>
>There is no way to talk about end to end connectivity quality assurance.
>I have at all times at least one customer raging about how bad we are
>reachable, and that his partners on other networks can only get to us
>with such and such a delay/packet loss/unavailability.
>
>The understanding must be that a provider can ony control the own
>network. How traffic is routed outside is mostly uncontrollable, because
>most people do not run route servers yet that would take a policy from
>the radb (I have no special policy in btw, but will do this as soon as we
>run our route servers).
>
>>From this I would say it is a very different issue, if a provider has
>quality of service problems within the own network: e.g. a NOC not
>answering, sluggish links, packet loss above a normal tolerable level.
>I would say, that in
>such cases any contract can be terminated, and sued. But, of course a
>customer must be certain that the problems are caused within his
>provider's network. That's where most people are not sure, of course,
>even most 'consultants' have in reality no clue how to check something
>like this.
>
>Mike
>
>>
>> Nathan Stratton CEO, NetRail, Inc. Your Gateway to the World!
>> ---------------------------------------------------------------------------
>> Phone (703)524-4800 NetRail, Inc.
>> Fax (703)534-5033 2007 N. 15 St. Suite 5
>> Email sales@netrail.net Arlington, Va. 22201
>> WWW http://www.netrail.net/ Access: (703) 524-4802 guest
>> ---------------------------------------------------------------------------
>>
>>
>
>----------------------------------------------------------
> IDT
>Michael F. Nittmann ---------
>Senior Network Architect \ /
>(201) 928 1000 xt 500 -------
>(201) 928 1888 FAX \ /
>mn@ios.com ---
> V
> IOS
>
>

On Sat, 4 Nov 1995, Nathan Stratton wrote:

> On Fri, 3 Nov 1995, Hans-Werner Braun wrote:
>
> > I think Dave has the right idea here. Given the lousy overall network
> > performance that I (and others) are often seeing for months from
> > varieties of service providers, I think the service providers should be
> > forced to provide rebates. I frequently have 10% packet losses to get
> > from where I am to the Bay area (via New York). And my service
> > provider (CERFnet) is telling me that their service provider (Sprint)
> > is not even answering to their trouble reports.
>
> Well, I think the problem is that providers lock people in 1 and 2 year
> contracts so if people get bad service they are stuck. That is why I only
> have month to month service, if people don't like there T1 then they can
> quit. I think we need more providers to do that, I know of a lot users
> that are on providers that want to switch, but have 8 months left on
> their contract.

Some comments from my side, which are, however, not official comments of
IDT Internet Services:

Internet works the way that a provider can only guarantee a standard of
quality within the perimeter of the provider's networks. E.g. guarantee a
customer that he is a certain number of hops away to the meetpoints, or
hand out a latency matrix between POPs, guarantee that the packet loss is
under a certain margin, and of course, guarantee a certain percentage of
uptime.

There is no way to talk about end to end connectivity quality assurance.
I have at all times at least one customer raging about how bad we are
reachable, and that his partners on other networks can only get to us
with such and such a delay/packet loss/unavailability.

The understanding must be that a provider can ony control the own
network. How traffic is routed outside is mostly uncontrollable, because
most people do not run route servers yet that would take a policy from
the radb (I have no special policy in btw, but will do this as soon as we
run our route servers).

From this I would say it is a very different issue, if a provider has
quality of service problems within the own network: e.g. a NOC not
answering, sluggish links, packet loss above a normal tolerable level.
I would say, that in
such cases any contract can be terminated, and sued. But, of course a
customer must be certain that the problems are caused within his
provider's network. That's where most people are not sure, of course,
even most 'consultants' have in reality no clue how to check something
like this.

Mike

>
> Nathan Stratton CEO, NetRail, Inc. Your Gateway to the World!
> ---------------------------------------------------------------------------
> Phone (703)524-4800 NetRail, Inc.
> Fax (703)534-5033 2007 N. 15 St. Suite 5
> Email sales@netrail.net Arlington, Va. 22201
> WWW http://www.netrail.net/ Access: (703) 524-4802 guest
> ---------------------------------------------------------------------------
>
>

----------------------------------------------------------
IDT
Michael F. Nittmann ---------
Senior Network Architect \ /
(201) 928 1000 xt 500 -------
(201) 928 1888 FAX \ /
mn@ios.com ---
V
IOS

Hans Werner - presumably frustrated about the quality of his internet
service writes:

Are you guys up to it, or would you need regulation do it for
you? Y'know, life can be easier if your parents tell you what
to do, if you cannot get your act together yourself. Less stress, too.
Less flexibility as well, perhaps, but methinks we have to choose some
optimization function here, and buy off on the cost.

In the end, this is not a sandbox for having a good time.
People today *depend* on their network connection, and that it works
is important to them. You *have* to go beyond just thinking locally.

Very relevant questions. Let me try to nudge the discussion into a
slightly different direction. When people started talking to me earlier
in the summer about internet business models, these issues were the
number one argument.

The chain of reasoning went something like: you can't have a mature
internet business until you have guaranteed quality of service metrics.
You can't have guaranteed quality of service metrics until you have
settlements. It seems to me that Hans Werner's complaint above fits
squarely into this mold of thought. If I am wrong I am sure he'll tell me.

Could we talk a bit about where these conclusions may lead us?

Settlements, as I understand them, might be traffic based or route
based. If I use more of someone else's resources than he uses of mine, I
owe him a payment. Perhaps the payment might be for gigabytes delivered
or routes advertised. The payment process then would cascade from the
largest down to the smallest providers as those lower in the food chain
extracted money from those beneath them to pay the demands of those above
them.

The outcome of such a process would be significant. Traffic based
settlemnts would add substantial accounting costs to doing business.
Route charging I'd guess would add less. Again however would not the
billing components of this play into the hands of the RBOCS and five
largest IXCs who have the mainframes and software systems to handle it
most easily?

But if the big boys started it, how far would it or could it go before
shattering the internet? I predict on the basis of what regional ISPs
who have more dial up than leased line accounts tell me that they would
disconnect from the big boys rather than accept the strangle hold of
settlements. The result - viola - a fractured, balkanized internet.

How are you going to guarantee the standards of service? PUC
certification in each state? Just what the RBOCs would love. Everyone
has to open a 7 by 24 NOC or you can't be in the business? Cisco 7000
routers become minimum gear? I know. Everyone has to meet standards set
by the CIX in order to play. Seriously though - how are you going to do
it? What will the rules be and how will they be made?

Lets assume that this were done in the most benign possible way. The
only way smaller isps could stay in business would be to pass the
increased costs on to their customers. And then your friendly South
dakota Internet service, would be like the nationals - $20 a month for 15
hours and $2 and hour for each additional hour. Not to mention the fact
that the costs of entry would be catapulted so high as to exclude new
start ups lacking sufficient outside financing. As Compu$erve remarked
in July, the cost model of $20 a month for unlimited service is a
suicidal one for this industry. Yeah undoubtedly if you have the
overhead costs of an Hr Block or RBOC to satisfy.

Let me frame the question starkly. To get the reliability that Hans
Werner wants to what extent must we consolidate the industry and raise
prices? Killing in the process what makes the internet a magnet of
attraction for some millions of users? **OR** is there any alternative
of letting a few build an inustrial strength network for those who demand
it, an allowing a less costly more fault tolerant network to survive? If
so how might this be done? Comments from policy makers for the big five
would be welcome.


********************************************************************
Gordon Cook, Editor & Publisher Subscript.: Individ-ascii $85
The COOK Report on Internet Non Profit. $150
431 Greenway Ave, Ewing, NJ 08618 Small Corp & Gov't $200
(609) 882-2572 Corporate $350
Internet: cook@cookreport.com Corporate. Site Lic $650
Newly expanded COOK Report Web Pages http://pobox.com/cook/
********************************************************************

I will not go into a point by point rebuttal here, even though I
generally do not subscribe to your arguments. I am not planning on
"winning" here, I just want to get the issues on the table and evaluate
the solution space. Just let me ask you, as a customer who fairly
frequently experiences 10% packet loss between major Internet locations
across major service providers (no mom and pop shops in the middle or
at the end points), how would you suggest I deal with that? My
complaints to the involved service providers have typically gotten
unanswered by the national service provider, and saying "we can't do
anything about it except letting our national service provider know" by
my regional service provider. There is no quality control at the
inter-ISP level. I want to see that fixed. I don't care how, I do know
that the current situation is intolerable. I believe that this is prime
NANOG (and IEPG) business. NSF and the feds are out, with the NSFNET
backbone dismantling, and the kitchen you asked for to cook (ahem!) in
is all yours to get your tailfeathers burned in all by yourself.

> There is no way to talk about end to end connectivity quality assurance.
> The understanding must be that a provider can ony control the own
> network. How traffic is routed outside is mostly uncontrollable, because

You say you can control your own network, but I suspect much of it is
based on telco supplied lines where you don't directly control the
reliability. This is why companies have contracts. Within certain
parameters (and for a price), they allow you to control the actions of
other companies (and their networks).

>I think you're confusing "quality" with "locked into contract." If
>you have quality, being locked in with fixed rates and service is a good
>thing. If you don't have quality it's a bad thing.
>
>Try real hard not to blame the provider, but the idiot who signed a contract
>that doesn't have performance guarantees. We [ACES Research] offer our
>clients contracts, and if they want to pay for performance guarantees, those
>too. None of them whine, so I assume they're happy and not upset. Some of
>them have locked rates till '98 that are just too good to mention.

This is a reasonable and appropriate response. The problem, I guess, is
that the environment is young and people do not know what is reasonable
to ask for in terms of service metrics, and service providers may not
know what is reasonable to offer. Do you (or others) have suggestions
about those, that could be used to address many of the concerns? We do
not have to fix things 100%, I guess, but it would be nice to get to
some high degree, rather than retaining status quo. Can we use them in
the inter-ISP realm?

On Sat, 4 Nov 1995, Hans-Werner Braun wrote:

> I have heard this argument very many times (and used it myself, and
> sure understand it at a technical level). It is a very network-centric
> argument serving a specific service provider. As a customer I don't
> buy it. You guys need to take a customer centric approach and make the
> customers happy. You sell yourselves and get money as services providers
> for the *Internet* not your *local environment*. If you only sell

no, we sell internet access. Your argument would require carmakers to
provide for sufficiently scenic driving environment ;-)


> XYZnet services, not problem, but please then do not advertise you
> provide Internet services just because you are marginally connected to

we provide internet access services. Same with the phone: nobody
guarantees you that you get through to china, isn't it?

> the rest of the world with no clue about how to make the NANOG and
> global system work.
>
> In the end this will be a market driven by customers, not service
> providers. Customers will make the rules and will determine whether a
> service is good or lousy. Look for other examples. If my power outlets
> would regularly drop to 50 volts (or zero) I would get quite irritated.
>
this is not a good comparison: we talk reachability here, analogy would
be phone networks.
Power: there is no geographical difference: 110Volts from Kansas look
exactly the same as 110 Volts from Wisconsin.

"Power" is sold on the open market between service providers. I would
> not accept an argument from a local service provider that my power is
> always dropping because some service provider 2,000 miles away is
> screwing up. I would consider that to be their problem to watch out for


the service agreement here is not to reach that location, but to have
power. and, btw: you do not get compensated for outages at all: if I must
throwh away my freezer contents, they do not pay me for it. That's
exactly like Internet access: if you cannot reach MIT because their link
is down, we won't pay you your money back, and won't come up for any
damages because you could not deliver a document or else.



> such things and to coordinate it right. Same with phones. I don't care
> what region you are in, but if I would call you, above the 99th
> percentile, as long as you are close to your phone and pick up, my call
> will get through and work without significant service degradations
> *despite* the fact that there are at least three service prodivers

wrong: this week I triet to reach Chicago several times and got "sorry,
your call cannot be completed at this time ...... try later". Can I now
go to the phone company and say: "I did not reach my business partner to
stop a deal and lost money, compensate me?".. I won't even try.

> (local, long distance, local) involved. Why? Because power and phone
> companies have their shit together on that stuff, and coordinate and

two weeks ago one of these nice transformers on the poles exploded about
1.5 miles from here, sending a loud boom, and a fantastic power surge
that burnt out a light bulb (my surge protector held). Don't even ask
them to pay your stereo if it fries in such an incident.


> cooperate because they do understand they are all in the same boat. Of
> course, obviously it is not a fairly new and anarchic environment
> there, but has grown quite well into coordination. Are you guys up to
> it, or would you need regulation do it for you? Y'know, life can be

Regulation would make it better:I could sit back and point to a regulation.

> easier if your parents tell you what to do, if you cannot get your act
> together yourself. Less stress, too. Less flexibility as well, perhaps,
^^^^^^^^^^^^^^^^

see two lines above re regulations ;-)

> but methinks we have to choose some optimization function here, and buy
> off on the cost.
>
> In the end, this is not a sandbox for having a good time. People today

HOwever, I don't hate my job that much...

> *depend* on their network connection, and that it works is importenat
> to them. You *have* to go beyond just thinking locally.
>

We do: we offer redundant dual homing and configure even your ospf or BGP
for you. Even if you chose a different provider.

>
> >On Sat, 4 Nov 1995, Nathan Stratton wrote:
> >
> >> On Fri, 3 Nov 1995, Hans-Werner Braun wrote:
> >>
> >> > I think Dave has the right idea here. Given the lousy overall network
> >> > performance that I (and others) are often seeing for months from
> >> > varieties of service providers, I think the service providers should be
> >> > forced to provide rebates. I frequently have 10% packet losses to get
> >> > from where I am to the Bay area (via New York). And my service
> >> > provider (CERFnet) is telling me that their service provider (Sprint)
> >> > is not even answering to their trouble reports.
> >>
> >> Well, I think the problem is that providers lock people in 1 and 2 year
> >> contracts so if people get bad service they are stuck. That is why I only
> >> have month to month service, if people don't like there T1 then they can
> >> quit. I think we need more providers to do that, I know of a lot users
> >> that are on providers that want to switch, but have 8 months left on
> >> their contract.
> >
> >Some comments from my side, which are, however, not official comments of
> >IDT Internet Services:
> >
> >Internet works the way that a provider can only guarantee a standard of
> >quality within the perimeter of the provider's networks. E.g. guarantee a
> >customer that he is a certain number of hops away to the meetpoints, or
> >hand out a latency matrix between POPs, guarantee that the packet loss is
> >under a certain margin, and of course, guarantee a certain percentage of
> >uptime.
> >
> >There is no way to talk about end to end connectivity quality assurance.
> >I have at all times at least one customer raging about how bad we are
> >reachable, and that his partners on other networks can only get to us
> >with such and such a delay/packet loss/unavailability.
> >
> >The understanding must be that a provider can ony control the own
> >network. How traffic is routed outside is mostly uncontrollable, because
> >most people do not run route servers yet that would take a policy from
> >the radb (I have no special policy in btw, but will do this as soon as we
> >run our route servers).
> >
> >>From this I would say it is a very different issue, if a provider has
> >quality of service problems within the own network: e.g. a NOC not
> >answering, sluggish links, packet loss above a normal tolerable level.
> >I would say, that in
> >such cases any contract can be terminated, and sued. But, of course a
> >customer must be certain that the problems are caused within his
> >provider's network. That's where most people are not sure, of course,
> >even most 'consultants' have in reality no clue how to check something
> >like this.
> >
> >Mike
> >
> >>
> >> Nathan Stratton CEO, NetRail, Inc. Your Gateway to the World!
> >> ---------------------------------------------------------------------------
> >> Phone (703)524-4800 NetRail, Inc.
> >> Fax (703)534-5033 2007 N. 15 St. Suite 5
> >> Email sales@netrail.net Arlington, Va. 22201
> >> WWW http://www.netrail.net/ Access: (703) 524-4802 guest
> >> ---------------------------------------------------------------------------
> >>
> >>
> >
> >----------------------------------------------------------
> > IDT
> >Michael F. Nittmann ---------
> >Senior Network Architect \ /
> >(201) 928 1000 xt 500 -------
> >(201) 928 1888 FAX \ /
> >mn@ios.com ---
> > V
> > IOS
> >
> >
>
>

----------------------------------------------------------
IDT
Michael F. Nittmann ---------
Senior Network Architect \ /
(201) 928 1000 xt 500 -------
(201) 928 1888 FAX \ /
mn@ios.com ---
V
IOS

Nathan writes:
>Well, I think the problem is that providers lock people in 1 and 2 year
>contracts so if people get bad service they are stuck. That is why I only
>have month to month service, if people don't like there T1 then they can
>quit. I think we need more providers to do that, I know of a lot users
>that are on providers that want to switch, but have 8 months left on
>their contract.

I think you're confusing "quality" with "locked into contract." If
you have quality, being locked in with fixed rates and service is a good
thing. If you don't have quality it's a bad thing.

Try real hard not to blame the provider, but the idiot who signed a contract
that doesn't have performance guarantees. We [ACES Research] offer our
clients contracts, and if they want to pay for performance guarantees, those
too. None of them whine, so I assume they're happy and not upset. Some of
them have locked rates till '98 that are just too good to mention.


>Nathan Stratton CEO, NetRail, Inc. Your Gateway to the World!

Ehud

--
Ehud Gavron (EG76)
gavron@Hearts.ACES.COM

I second hwb's observation that ISPs must take more responsibility
for the overall QOS of the Internet infrastructure. IMHO, building
business profits from providing IP services and not funding the
development of routing protocols that provide Policy Based Routing
and QOS hooks does not make good long term business.

If end users, for example, could choose how their datagrams travel
based on reliability and routing policy, then end-users would benefit
greatly by enhancced IP services and all providers would benefit.

The current, post NSF funding, days appears to resemble this paradigm:
IP service providers take advantage of work by the NSF, IETF and code
developers to create a commercial business. IP service providers
(not all, of course, but most) make good profits and do not help fund
(again not all, but most) the development of enhanced features in the
IP development saga.

Just sending packets "up stream" and then telling the end-user
they do have any responsibility with "up stream networks" does
nothing to improve the reliability of the Internet nor does it
enhance the product offering.

NANOG would appear to be an excellent vehicle for developing a strategic
plan, as well as a funding strategy, to improve, enhance, and forward
the development of these services. The IEFT is, for the most part,
a volunteer organization. Where does all that CIX money go, BTW?

Regards,

Tim

--
+--------------------------------------------------------------------------+
| Tim Bass | #include<campfire.h> |
| Principal Network Systems Engineer | for(beer=100;beer>1;beer++){ |
| The Silk Road Group, Ltd. | take_one_down(); |
| | pass_it_around(); |
| http://www.silkroad.com/ | } |
| | back_to_work(); /*never reached */ |
+--------------------------------------------------------------------------+

> power. and, btw: you do not get compensated for outages at all: if I must
> throwh away my freezer contents, they do not pay me for it. That's
> exactly like Internet access: if you cannot reach MIT because their link

If you are willing to pay for it, you can get what you want regarding
power. There are companies that will gladly sell you dual diesel
generators and companies that will sell you insurance in case the
main power and both generators fail.

If there are enough people who want and are willing to pay for < .001%
packet loss, providers can and eventually will provide it. Good
measurement systems are a necessary first step. Priority based
packet routing will also help.

At 2:48 PM 11/4/95, Michael Dillon wrote:
>On Sat, 4 Nov 1995, Hans-Werner Braun wrote:
>> Just let me ask you, as a customer who fairly
>> frequently experiences 10% packet loss between major Internet locations
>> across major service providers (no mom and pop shops in the middle or
>> at the end points), how would you suggest I deal with that?
>
>Uh... Ignore it?
>10% packet loss is quite within the normal range of parameters for a
>packet switching network such as the Internet.

Sorry Michael, wrong answer. 1% packet loss is intolerable. 10% packet
loss is all but useless for serious work.

On Sat, 4 Nov 1995, Hans-Werner Braun wrote:

> I will not go into a point by point rebuttal here, even though I
> generally do not subscribe to your arguments. I am not planning on
> "winning" here, I just want to get the issues on the table and evaluate
> the solution space. Just let me ask you, as a customer who fairly
> frequently experiences 10% packet loss between major Internet locations
> across major service providers (no mom and pop shops in the middle or
> at the end points), how would you suggest I deal with that?

Uh... Ignore it?
10% packet loss is quite within the normal range of parameters for a
packet switching network such as the Internet. If you want 0% packet loss,
you can lease your own private point-to-point lines.

> my regional service provider. There is no quality control at the
> inter-ISP level. I want to see that fixed.

But nothing is broken. There is no inter-ISP level. ISP's buy access to
the global network from and NSP and resell those access rights to you.
Instead of millions of inter-ISP relationships, there is only one (or a
few for multi-homed ISP's) relationship to negotiate and to manage.

> I don't care how, I do know
> that the current situation is intolerable.

Well why didn't you say so in the first place! We thought you *DID* care
how. Since you don't care how the problem is solved you will be happy to
know that SPRINT and MCI et al. will be pleased to provide you with the
performance guarantees that you require. Just get out your checkbook and
call them on Monday morning. Make sure you tell them that price is no object.

> I believe that this is prime
> NANOG (and IEPG) business. NSF and the feds are out, with the NSFNET
> backbone dismantling,

The NSF backbone is long gone (5 months or so), and the national
backbones (note plural) at the core of the Internet are fast evolving
into international webworks of fibre. As the networks grow and the
infrastructure is deployed there are lots of pains. Live with them. In 10
years it will be over. Think of it like putting up with construction in
your house and mud where the front yard should be. Once the construction
is finished you will soon forget about it as you enjoy your new home.


Michael Dillon Voice: +1-604-546-8022
Memra Software Inc. Fax: +1-604-542-4130
http://www.memra.com E-mail: michael@memra.com

The current method should probably be: locate where the packet loss
occurs, forward it to your provider, your provider forwards it to his,
and so on.
Yes, there is no formal mechanism to my knowledge, but should we really
make one?

If we start to formalize then we have later to formalize the meetpoint
relationships (who is responsible for packet loss on a meetpoint network?).

A better point of pressure: chose a provider with shorter internal paths.

On the Internet anyone can kick the tires of any network by sending
probing data accross it. You get latency, hop count, and throughput of
the smallest link in your path.

The job to find this out is a consultant's job. The consultant knows
about the Internet topology, and how to quickly find out if a provider
will be ok for the wanted connectivity, or not.
Scopes of connectivity can be global reachability (trading off maybe
hopcounts and comparing on offered routes and latency), short paths (for
virtual private networks or national connectivity), points of presence in
your business locations.

If everyone would chose, then the now big networks, that offer an
impressive display of capital investment (called excessive hopcount by
some people with negative attitude ;-) ), will probably redisign their
routing (or why do some connections backtrack via both coasts and then to
the next metropole 200 miles away?).

I would say, that market pressure is the best regulatory agent: pinches
directly into the providers pocketbook. I accept this and design for
customers, and have customer needs and quality issues in mind when the
network grows (which it does currently tremendously).

Instead of organizational reglementation, I would put the current path of
Internet routing forward: with current tools it is possible to include
routing decisions of networks around a provider in that providers routing
topology. It is just a matter of time to implement it.

Mike



On Sat, 4 Nov 1995, Hans-Werner Braun wrote:

> I will not go into a point by point rebuttal here, even though I
> generally do not subscribe to your arguments. I am not planning on
> "winning" here, I just want to get the issues on the table and evaluate
> the solution space. Just let me ask you, as a customer who fairly
> frequently experiences 10% packet loss between major Internet locations
> across major service providers (no mom and pop shops in the middle or
> at the end points), how would you suggest I deal with that? My
> complaints to the involved service providers have typically gotten
> unanswered by the national service provider, and saying "we can't do
> anything about it except letting our national service provider know" by
> my regional service provider. There is no quality control at the
> inter-ISP level. I want to see that fixed. I don't care how, I do know
> that the current situation is intolerable. I believe that this is prime
> NANOG (and IEPG) business. NSF and the feds are out, with the NSFNET
> backbone dismantling, and the kitchen you asked for to cook (ahem!) in
> is all yours to get your tailfeathers burned in all by yourself.
>

----------------------------------------------------------
IDT
Michael F. Nittmann ---------
Senior Network Architect \ /
(201) 928 1000 xt 500 -------
(201) 928 1888 FAX \ /
mn@ios.com ---
V
IOS

On Sat, 4 Nov 1995, Hans-Werner Braun wrote:

> >I think you're confusing "quality" with "locked into contract." If
> >you have quality, being locked in with fixed rates and service is a good
> >thing. If you don't have quality it's a bad thing.
> >
> >Try real hard not to blame the provider, but the idiot who signed a contract
> >that doesn't have performance guarantees. We [ACES Research] offer our
> >clients contracts, and if they want to pay for performance guarantees, those
> >too. None of them whine, so I assume they're happy and not upset. Some of

.... just saw this: how does ACES then guarantee, e.g., that abc.cd.com
is reachable whitn certain quality parameters, when that site is
overseas? Of course, if a customer wants it, I can always guarantee
overseas reachability by buying a line, signing up abc.cd.com on the
other side on my network, and have that used strictly by that customer.
Redundancy too: other side around of the blue ball. No problem.

But are we talking about this here?

Mike




> >them have locked rates till '98 that are just too good to mention.
>
> This is a reasonable and appropriate response. The problem, I guess, is
> that the environment is young and people do not know what is reasonable
> to ask for in terms of service metrics, and service providers may not
> know what is reasonable to offer. Do you (or others) have suggestions
> about those, that could be used to address many of the concerns? We do
> not have to fix things 100%, I guess, but it would be nice to get to
> some high degree, rather than retaining status quo. Can we use them in
> the inter-ISP realm?
>

----------------------------------------------------------
IDT
Michael F. Nittmann ---------
Senior Network Architect \ /
(201) 928 1000 xt 500 -------
(201) 928 1888 FAX \ /
mn@ios.com ---
V
IOS

>On Sat, 4 Nov 1995, Hans-Werner Braun wrote:

>> >Try real hard not to blame the provider, but the idiot who signed a contract
>> >that doesn't have performance guarantees. We [ACES Research] offer our
>> >clients contracts, and if they want to pay for performance guarantees, those
>> >too. None of them whine, so I assume they're happy and not upset. Some of

>.... just saw this: how does ACES then guarantee, e.g., that abc.cd.com
>is reachable whitn certain quality parameters, when that site is
>overseas? Of course, if a customer wants it, I can always guarantee

Eh? Our customers understand what they're buying is THEIR connectivity,
not someone else's. What we guarantee to those who pay for it is as follows:
1. We will deliver their packets to the next hop 100% of the time.
2. We will have no unannoucned outages. If we do, they are free
of the contract and don't have to pay either.
3. We will contact other providers between us and an unreachable
or packet-lossy destinations on their behalf
4. We will handle all appropriate technical matters.

Think of this like you do from the phone company. You're buying connectivity
to the network. If someone on the remote end has a busy signal, what you've
bought is the RIGHT to find that out by having a connection go as far as
possible.

If the connection never gets to your local CO, then you have a poor service
provider. If your service provider is too busy making graphical web pages
and resting on their laurels while their routers go down, then you have a
poor service provider. If your service provider is ignoring a 10% packet
loss, then you have a poor service provider, etc... etc...

>But are we talking about this here?

>Mike

Ehud

--
Ehud Gavron (EG76)
gavron@Hearts.ACES.COM

>Date: Sat, 4 Nov 1995 12:48:36 -0800 (PST)
>From: Michael Dillon <michael@memra.com>
>Subject: Re: links on the blink (fwd)

>Uh... Ignore it?
>10% packet loss is quite within the normal range of parameters for a
>packet switching network such as the Internet.

I must disagree here. 10% packet loss within national backbones is a
problem to be fixed as soon as possible. It is not something to be
tolerated. An examination of some of the interconnect points will find
providers talking across media that is far past saturation, and is
at capacity.

The good news is that from my point of view, these things are being
addressed. Not as quickly as everyone would like (including me), but
it's happening.

>> my regional service provider. There is no quality control at the
>> inter-ISP level. I want to see that fixed.
>
>But nothing is broken. There is no inter-ISP level. ISP's buy access to
>the global network from and NSP and resell those access rights to you.
>Instead of millions of inter-ISP relationships, there is only one (or a
>few for multi-homed ISP's) relationship to negotiate and to manage.

Assuming Hans-Werner meant inter-NSP, I must differ with both of
you.. There *is* quality control at the inter-NSP level. But there are
things broken, as I said. There are also a bunch of folks working night
and day to make sure it works as well as it does, and a bunch more
trying to make it get better.

Speaking only for myself...

RobS

> 10% packet loss is quite within the normal range of parameters for a
> packet switching network such as the Internet.

Well, I don't think I will need to be reminded not to buy Internet service
from your organization.

Here is a bit of context.

At home I am a beta site for the PSI Internet service via cable TV. I
generally use a computer at Harvard University to read my mail. Although
the Harvard network is about 1.5 miles away from my house the sparse
distribution of connections between Internet service providers and the
current routing policies mean that packets from my house to Harvard take 20
hops and go through San Francisco and packets from Harvard to my house,
(also 20 hops) go through Washington DC.

Here are the results of some reliability tests I just ran.

from my house to San Francisco via PSI Net:

209 packets transmitted, 209 packets received, 0% packet loss
round-trip min/avg/max = 99.996/268.978/983.334 ms

from Harvard to Washington DC via BBN Planet and MCI

204 packets transmitted, 204 packets received, 0% packet loss
round-trip (ms) min/avg/max = 16/46/249

from my house to Harvard via PSI, MAE-west, MCI, BBN Planet and back via
BBN Planet, MCI, MAE-east and PSI.

505 packets transmitted, 497 packets received, 1% packet loss
round-trip min/avg/max = 116.702/192.69/3000 ms

(Note this is being tested at a time when Harvard's service provider (BBN
planet) is busily working on one of the links in its backbone because they
do not think the quality is high enough.)

I consider it a problem when the loss exceeds 1% through this long path -
as do the people who run the networks that my traffic passes through. The
normal loss through this path is less than 1% and, much of the time it is 0.

> But nothing is broken. There is no inter-ISP level.

It is so much easier to just say it is the other guy's problem.
Hans-Werner suggests that most phone companies do not take this attitude,
lucky for some of us, not all Internet "service" providers do.

Scott

(just in case you might think I'm off on some other world, there are other
paths I could have used which would have shown far higher loss, those paths
would have gone through service providers who apparently feel the way that
the person quoted here does or who are unable to make the investment in
their infrastructure warranted by the level of traffic their customers
would like to exchange.)

"take the contracts away from MFS...."

uh, the last I knew there was no "contract" that MFS has from anyone
(the gubmint types??) that can be taken away. I haven't taken the time
to sort out who pays for what at the other NAPs.

The Gigaswitch at MAE-EAST is smoking along just fine.
There *are* ports onto MAE-EAST with lower-performance than dedicated
Gigaswitch ports, but that is a function of what some clients want to
buy. To a good first approximation, MFS sells what the clients want.
Same with MAE-WEST as well. That is why some attachees have a port on
a shared Ethernet off the FDDI and others have (possibly multiple)
dediated FDDI ports on the Gigaswitch.

Remember that MAE-EAST existed over a year before NAPs were created.

As for additional exchange points, this is trivially obvious and
will almost certainly come to pass in some time frame, I hope sooner
rather than later.

What is less obvious is who runs them, especially if one has second
thoughts about the majority of the exchanges not be run by one
company, as well as insist they be implemented with non-experimental
technology and operated by people with clues about telco-grade service
quality. It would also be nice if the exchange operator were not
a competitor of the subscribers, but all of these desires may
over-constrain the problem.

One would think there is a nice little business in running exchange
points around the country. Get co-lo space on a contract that will
let you sub-let, put in a nice FDDI Gigaswitch, heavy-duty power
with generator backup, good remote hands, plenty of DS3 entrance
facilities (ideally on OC12 or faster sonet for growth!), monitoring
and statistics gathering facilities, out-of-band management network,
and a nice NOC somewhere to coordinate with all your customers.

Put locations in major metro areas and go to town. You may need
some up-front cash, but hey, this Internet thing is hot, so you
can probably find investors (big grin).

So how much do you have to charge subscribers to make this work?

-mo

>My opinion of the complainers is that they are swell headed bigots with
>their minds full of the 21st century who fail to realize that the level
>of today's network technology is about the same level of automotive
>technology when Model T Fords first started to roll off the assembly lines.

You don't get it, do you? Ross Veach and I have been working in this
environment for many years. I was a PI for the NSFNET backbone, and
Ross was one of our major collaborators. I forgive you your newbie
mentality, but please forgive us that after bringing the Internet over
the last ten years from a network research environment to a state that
was attractive enough to private industry to get it delivered on a
silver plate, that we would at least like the service qualities we
worked so hard on achieving. My hope was that you guys would do better
than we did, and provide a ubiquitous high quality network (we did high
quality, but not ubiquitous). Are you going to get to work and fix it,
or continue to screw it up and whine around about the heat in the
kitchen? Your choice.

> Yes, there is no formal mechanism to my knowledge, but should we really
> make one?

There was a document written by Paul A. Flahery called "NSFNET Trouble
Reporting Standards". It was published in August of 1994 as an outgrowth
of the NSFNET RAC. It is not a formal standard, but might be a place to
start work towards some mechanism to effectively deal with trouble reporting
and resolution.

I don't know if a copy of this document is available on-line or not. Perhaps
someone else in the groups knows.

--
Stan | Academ Consulting Services |internet: sob@academ.com
Olan | For more info on academ, see this |uucp: {mcsun|amdahl}!academ!sob
Barber | URL- http://www.academ.com/academ |Opinions expressed are only mine.