Mailing List Archive: BGP Routing problem

BGP Routing problem

ant at sharenet

Jun 5, 1997, 7:39 AM

Post #1 of 11 (416 views)

Hi

We are a ISP in South Africa and have a problem we are trying to
figure out rather urgently, without much success so far.

We are currently multi-homed during the day and tripple homed at
night. (our peaks are at night and we get cheap bandwidth from
another ISP who has their peak in the day). Internic has refused to
give us our own block of addresses which would have made thins much
simpler. (despite repeated attempts, and we now have 7 /24's from
various places and badly need more).

Basicaly we (AS6180) have been announcing all our adresses including
196.25.116.0, 196.25.117.0 and 196.25.203.0 via AS 3741 at night,
and this gets switched of during the day. The problem seems to relate
to the fact that these addresses are part of AS 5713's CIDR
196.25.0.0/16 and we thought we could get away with announcing them
like this. It did work fine for over a week, but now today we have a
problem.

MAE-East (see example at end of message) , the Sprintnap and who
knows who else still have entries for these via AS3741 but they are
showing as received-only, and no best path. Yet via MAE West they are
fine.

We are going to stop announcing these via AS3741 at night, but its
been 9 hours now since they were not announced and these
received-only entries are still sitting there blocking these routes
from it seems about a third of the internet.

Can any one offer any comments on why they are still there, what
received-only means and how we can get if fixed?

Many thanks
Regards
Anthony Walker

MAE-East Looking Glass Results
Query: bgp
Addr: 196.25.203.0
BGP routing table entry for 196.25.203.0/24, version 5006458
Paths: (1 available, no best path, advertised over IBGP)
1673 1239 4005 3741 6180, (received-only)
192.41.177.141 from 192.41.177.141 (140.223.57.217)
Origin IGP, external

Re: BGP Routing problem [ In reply to ]

danny at genuity

Jun 5, 1997, 8:21 AM

Post #2 of 11 (427 views)

i believe (received-only) means that bgp soft-reconfiguration is configured
for the peer and since some policy is denying the route, it won't be used ..
unless a policy change occurs.

here's what we're currently hearing from you folks...

peer1.sjc1#sh ip bgp re 6180
BGP table version is 2830358, local router ID is 207.240.24.10
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Weight Path
* i196.6.121.0 192.41.177.248 10 80 0 701 5713 6180 i
* i 198.32.176.2 10 80 0 701 5713 6180 i
*> 198.32.184.42 10 80 0 701 5713 6180 i
* i196.28.5.0 192.41.177.248 10 80 0 701 5713 6180 i
* i 198.32.176.2 10 80 0 701 5713 6180 i
*> 198.32.184.42 10 80 0 701 5713 6180 i
h 206.49.228.0 198.32.184.42 10 80 0 701 5713 6180 i
* i 192.41.177.243 10 80 0 4000 6187 6180 i
* i 192.157.69.80 10 80 0 4000 6187 6180 i
*> 198.32.136.94 10 80 0 4000 6187 6180 i
* i206.49.229.0 192.41.177.243 10 80 0 4000 6187 6180 i
* i 192.157.69.80 10 80 0 4000 6187 6180 i
*> 198.32.136.94 10 80 0 4000 6187 6180 i

and it appears as though routing via alternet is working for the networks you
mentioned.

-danny

> Hi
>
> We are a ISP in South Africa and have a problem we are trying to
> figure out rather urgently, without much success so far.
>
> We are currently multi-homed during the day and tripple homed at
> night. (our peaks are at night and we get cheap bandwidth from
> another ISP who has their peak in the day). Internic has refused to
> give us our own block of addresses which would have made thins much
> simpler. (despite repeated attempts, and we now have 7 /24's from
> various places and badly need more).
>
> Basicaly we (AS6180) have been announcing all our adresses including
> 196.25.116.0, 196.25.117.0 and 196.25.203.0 via AS 3741 at night,
> and this gets switched of during the day. The problem seems to relate
> to the fact that these addresses are part of AS 5713's CIDR
> 196.25.0.0/16 and we thought we could get away with announcing them
> like this. It did work fine for over a week, but now today we have a
> problem.
>
> MAE-East (see example at end of message) , the Sprintnap and who
> knows who else still have entries for these via AS3741 but they are
> showing as received-only, and no best path. Yet via MAE West they are
> fine.
>
> We are going to stop announcing these via AS3741 at night, but its
> been 9 hours now since they were not announced and these
> received-only entries are still sitting there blocking these routes
> from it seems about a third of the internet.
>
> Can any one offer any comments on why they are still there, what
> received-only means and how we can get if fixed?
>
> Many thanks
> Regards
> Anthony Walker
>
>
> MAE-East Looking Glass Results
> Query: bgp
> Addr: 196.25.203.0
> BGP routing table entry for 196.25.203.0/24, version 5006458
> Paths: (1 available, no best path, advertised over IBGP)
> 1673 1239 4005 3741 6180, (received-only)
> 192.41.177.141 from 192.41.177.141 (140.223.57.217)
> Origin IGP, external

Re: BGP Routing problem [ In reply to ]

Jun 5, 1997, 12:02 PM

Post #3 of 11 (420 views)

We have seen upon occasion where a dampened route will not be replaced by
a route of the same specificity for about 20-45 minutes. While we didn't
care for this behaviour, I guess it was somewhat predictable.

What had us concerned was we have seen each of our three providers black
hole us on at least one occasion (sometimes more) when their backbone or
core flapped. It could be described as appearing as if their borders did
not receive the information to withdraw our advertisements. Looking Glass
showed us "history" and "received only" entries, but no active
replacements... similar to the symptoms that you are describing.
Sprint's NOC reported hearing active advertisements from our "down"
providerm but I heard this second-hand. Killing our BGP session with the
offending provider after the problem started would not fix the problem,
our routes continued to be preferred to someone we no longer had a bgp
session with. This situation would usually fix itself within 45 minutes.

While this doesn't happen often, it has the affect that having two other
providers is useless if the third is erronously advertising us. Is there
a failure scenerio where borders will continue to advertise AS's due to
loss of connectivity with their interior routers?

Bil Herd
InterActive

On Thu, 5 Jun 1997, Anthony Walker wrote:
> We are currently multi-homed during the day and tripple homed at
> night.
> Basicaly we (AS6180) have been announcing all our adresses including
> 196.25.116.0, 196.25.117.0 and 196.25.203.0 via AS 3741 at night,
> and this gets switched of during the day. The problem seems to relate
> to the fact that these addresses are part of AS 5713's CIDR
> 196.25.0.0/16 and we thought we could get away with announcing them
> like this. It did work fine for over a week, but now today we have a
> problem.
>
> MAE-East (see example at end of message) , the Sprintnap and who
> knows who else still have entries for these via AS3741 but they are
> showing as received-only, and no best path. Yet via MAE West they are
> fine.
>
> We are going to stop announcing these via AS3741 at night, but its
> been 9 hours now since they were not announced and these
> received-only entries are still sitting there blocking these routes
> from it seems about a third of the internet.
> MAE-East Looking Glass Results
> Query: bgp
> Addr: 196.25.203.0
> BGP routing table entry for 196.25.203.0/24, version 5006458
> Paths: (1 available, no best path, advertised over IBGP)
> 1673 1239 4005 3741 6180, (received-only)
> 192.41.177.141 from 192.41.177.141 (140.223.57.217)
> Origin IGP, external
>

Re: BGP Routing problem [ In reply to ]

Jun 6, 1997, 12:32 AM

Post #4 of 11 (419 views)

Anthony Walker said that the following was seen while the BGP session
between AS6180 and AS3741 was down:

> MAE-East Looking Glass Results
> Query: bgp
> Addr: 196.25.203.0
> BGP routing table entry for 196.25.203.0/24, version 5006458
> Paths: (1 available, no best path, advertised over IBGP)
> 1673 1239 4005 3741 6180, (received-only)
> 192.41.177.141 from 192.41.177.141 (140.223.57.217)
> Origin IGP, external

AFAIK, "received-only" means that DIGEX heard that route from AS1673
but did not install it, presumably because it failed to pass a filter.

The question is, why did that route not get withdrawn properly when the
BGP session between AS6189 and AS3741 was terminated? This looks very
much like the cisco bug that bit Sprint during the AS7007 incident a month
or so ago.

Hey, Ravi, I thought you fixed that bug? Perhaps AS4005 is not running
the fixed code. Perhaps there's another bug.

--apb (Alan Barrett)

Re: BGP Routing problem [ In reply to ]

danny at genuity

Jun 6, 1997, 6:17 AM

Post #5 of 11 (415 views)

> Hey, Ravi, I thought you fixed that bug? Perhaps AS4005 is not running
> the fixed code. Perhaps there's another bug.

it's not 'another' bug. (received only) means that soft-reconfiguration is
configured for the peer and some policy denied the route so it's not being
used - yet.

had he did a "sh ip route x.x.x.x" rather than "sh ip bgp x.x.x.x" he would
have seen the less specific entry being used.

-danny

Re: BGP Routing problem [ In reply to ]

Jun 6, 1997, 6:49 AM

Post #6 of 11 (419 views)

> > Hey, Ravi, I thought you fixed that bug? Perhaps AS4005 is not running
> > the fixed code. Perhaps there's another bug.
>
> it's not 'another' bug. (received only) means that soft-reconfiguration is
> configured for the peer and some policy denied the route so it's not being
> used - yet.

Yes, I know what "received only" means in this context. It means that
AS1673 was announcing the route to DIGEX, but DIGEX was filtering it.

That's independent of the issue I am talking about, which is that the BGP
sesson between AS3741 and AS6180 had been shut down for several hours, the
route in question was no longer present in AS3741, but the route still
appeared (with path "1673 1239 4005 3741 6180") at DIGEX's MAE-East
looking glass. The route should have been withdrawn completely from the
entire Internet when the BGP session between AS6180 and AS3741 was shut
down, but the route was not properly withdrawn. DIGEX should not have
seen any trace of the route (except perhaps a history entry for flap
damping purposes).

> had he did a "sh ip route x.x.x.x" rather than "sh ip bgp x.x.x.x" he would
> have seen the less specific entry being used.

Yes.

--apb (Alan Barrett)

Re: BGP Routing problem [ In reply to ]

Jun 6, 1997, 6:58 AM

Post #7 of 11 (419 views)

On Fri, 6 Jun 1997, Danny McPherson wrote:

>
> > Hey, Ravi, I thought you fixed that bug? Perhaps AS4005 is not running
> > the fixed code. Perhaps there's another bug.
>
> it's not 'another' bug. (received only) means that soft-reconfiguration is
> configured for the peer and some policy denied the route so it's not being
> used - yet.

But that doesn't answer the question of why it is still appearing many
hours after it has stopped being advertised; we have seen this happen too.
Even though it doesn't hurt anything as is because it isn't being used,
but what happens if the policy changes so it was being used? Would it
magically process a withdrawal that presumably happened some time ago, or
would it actually use the bogus route?

Re: BGP Routing problem [ In reply to ]

Jun 6, 1997, 7:22 AM

Post #8 of 11 (416 views)

===== Alan Barrett previously wrote: ====
>
> That's independent of the issue I am talking about, which is that the BGP
> sesson between AS3741 and AS6180 had been shut down for several hours, the
> route in question was no longer present in AS3741, but the route still
> appeared (with path "1673 1239 4005 3741 6180") at DIGEX's MAE-East
> looking glass. The route should have been withdrawn completely from the
> entire Internet when the BGP session between AS6180 and AS3741 was shut
> down, but the route was not properly withdrawn. DIGEX should not have
> seen any trace of the route (except perhaps a history entry for flap
> damping purposes).
>

It would not hurt to check a few other looking glasses. Such as,

route-views.oregon-ix.net or
route-server.cerf.net

to see if the path is really there. We have found out that some of these
looking glasses sometimes keep the routes much longer, at one time
several hours after we see routes disppeared in our routers.

Just one observation.

Jun

Re: BGP Routing problem [ In reply to ]

Jun 6, 1997, 7:37 AM

Post #9 of 11 (414 views)

> to see if the path is really there. We have found out that some of these
> looking glasses sometimes keep the routes much longer, at one time
> several hours after we see routes disppeared in our routers.

AFAIK nitrous *is* one of "our routers" (well DIGEX's) - it's
done using Cisco autocommands.

Alex Bligh
Xara Networks

Re: BGP Routing problem [ In reply to ]

Jun 6, 1997, 8:01 AM

Post #10 of 11 (414 views)

> >
>
> It would not hurt to check a few other looking glasses. Such as,
>
> route-views.oregon-ix.net or
> route-server.cerf.net
>

always a good idea to check multiple sources...

> to see if the path is really there. We have found out that some of these
> looking glasses sometimes keep the routes much longer, at one time
> several hours after we see routes disppeared in our routers.
>

do you just see the received only routes are you seeing "normal"
routes as well?

Ed

Ps while I cache some responses..sh ip route and sh ip bgp are not on
the list

Re: BGP Routing problem [ In reply to ]

Jun 6, 1997, 10:13 AM

Post #11 of 11 (416 views)

===== Ed Kern previously wrote: ====
>
> > to see if the path is really there. We have found out that some of these
> > looking glasses sometimes keep the routes much longer, at one time
> > several hours after we see routes disppeared in our routers.
>
> do you just see the received only routes are you seeing "normal"
> routes as well?
>

Ed,

That's a good question. I think they were all received-only
routes. And it normally happens at Sprintnap looking glass.

Jun