Mailing List Archive

I’m not aware of any IXPs using Quagga as a route server.

But you should definitely plan on two route servers. Each IXP participant should create a session to each route server. The route servers should not be connected to each other.

If the IXP is for peering only, it is unlikely you will have millions of routes. Probably 100K to 200K for a large IXP.

Again, if this is peering, you probably don’t need outbound route filtering. Peers will want the entire list of peer routes. But if they want to filter some by community, you should probably just ask the peers to do that on their side. You definitely want some sort of inbound route filtering. Hopefully with data from a trustworthy IRR.


Tom



> On Jan 17, 2018, at 6:08 AM, Pardeep <pardeep@vayugroup.net> wrote:
>
> Hi All,
>
> We are designing IXP L3 Design to one of our customer as below, where all ISP routers will have one eBGP peering with Quagga Route-Server. As we would like to take your suggestions if you have and we have some queries below. Please respond, It would be helpful for us and your help will be appreciated.
>
> Queries;
>
> 1. Will Quagga server be able to handle BGP millions of routes in its routing table?
> 2. Where we will perform the route filtering on Route-Server only?
> 3. Does Quagga support Outbond route-filtering? Please share with us any link.
> 4. How we will link-up route-server with IXP Manager?
>
> We got a lot of help from you folks and we are very thankful for that. Provide you support.
> Thanks..
>
> Design>>>>>>
>
> If you have any queries, please let us know.
>
> <image003.jpg>
> _______________________________________________
> Quagga-users mailing list
> Quagga-users@lists.quagga.net <mailto:Quagga-users@lists.quagga.net>
> https://lists.quagga.net/mailman/listinfo/quagga-users <https://lists.quagga.net/mailman/listinfo/quagga-users>

On Wed, 17 Jan 2018, Pardeep wrote:

> Queries;
>
> 1. Will Quagga server be able to handle BGP millions of routes in its
> routing table?

With few peers, yes. In an IXP with many peers, likely not - well, most
of the time it will, as long as the peers are stable, but peer's going
down will cause havoc (if they advertise non-trivial numbers of routes
esp.). This is why the other respondent doesn't know of many IXes using
Quagga.

Attempts have been made to address this, by adding even more buffering
of works (on top of the buffering I added years ago with the workqueue),
by adding threading (either trying to fully multi-thread bgpd, or trying
to just thread off non-UPDATE peer messages like KEEPALIVE).

The former perhaps buys some latency here and there, but ultimately it
adds more overheads, more CPU, more memory usage, and makes bgpd
throughput slower again - which actually can make the problem worse, by
making bgpd scale even worse. It also adds complexity, and bugs, and
makes the code harder to maintain. Which the threading approach also
does, but even more so.

Killing one of the good properties of Quagga bgpd (it scales up well
memory and CPU wise, route-wise - better than BIRD, ignoring the poor
"significant peer down" problem), to add dubious fixes for the peer-down
performance problem, is probably not the right long-term approach.

BIRD proves the buffering approaches in Quagga were, ultimately, largely
dumb. BIRD (and OpenBGPd) also proves you don't need threads everywhere.
If you need layers and layers of buffering, and threads per peer, to
handle BGP, you're just doing something wrong and have screwed up.

The lessons from BIRD should be heeded. ;)

I was thinking about that a while ago, and there may actually some very
simple fixes to the peer-down problem. Least some significant
improvements.

Hoping to try those out sometime. Just need to find some elusive spare
time.

----

At the protocol level, there is also the issue that standard BGP simply
isn't very good at the job an IXP wants it to do: Act as a light-weight
central 'hub'.

Stock BGP intrinsically is a distributed shortest-path algorithm. The
very semantics of its UPDATE messages inherently implement an iterative,
distributed computation of a matrix of costs over a graph.

Moving a lump of data quickly from one neighbour to another simply is
_not_ what a BGP-4 UPDATE message is designed to be good at.

There have been attempts to try bend BGP UPDATEs to be good at moving
data, but they (the 2 I'm thinking of anyway) retain significant
elements of the distributed path computation aspects of BGP UPDATE, for
compatibility / minimum change reasons. Hence they will continue to suck
for efficiently moving data from one place to another (cause just that's
intrinsically at odds with doing an iterative, distributed path
computation at the same time with the same messages).

We could actually add a Quagga specific extension though..

Again, just needs someone with time. ;)

regards,
--
Paul Jakma | paul@jakma.org | @pjakma | Key ID: 0xD86BF79464A2FF6A
Fortune:
Freedom is what you do with what's been done to you.
-- Jean-Paul Sartre
_______________________________________________
Quagga-users mailing list
Quagga-users@lists.quagga.net
https://lists.quagga.net/mailman/listinfo/quagga-users

On Thu, 18 Jan 2018, Paul Jakma wrote:

> moving data from one place to another (cause just that's intrinsically at
> odds with doing an iterative, distributed path computation at the same time
> with the same messages).

And by "intrinsically" I mean that finding a way to do the path
computation in a non-iterative way, that was still amenable to
distributed computation (i.e. not just doing the same or similar
iterative computation on each host in a non-distributed way) would imply
a huge step forward in one or both of linear algebra / matrix
multiplication or graph theory, with wide applications, and probably
resulting in an ACM award. ;)

regards,
--
Paul Jakma | paul@jakma.org | @pjakma | Key ID: 0xD86BF79464A2FF6A
Fortune:
halon system went off and killed the operators.
_______________________________________________
Quagga-users mailing list
Quagga-users@lists.quagga.net
https://lists.quagga.net/mailman/listinfo/quagga-users

Thanks Paul for your comments

Thanks
Pardeep

-----Original Message-----
From: Paul Jakma [mailto:paul@jakma.org]
Sent: Thursday, January 18, 2018 4:36 PM
To: Pardeep <pardeep@vayugroup.net>
Cc: Quagga Users <quagga-users@lists.quagga.net>
Subject: Re: [quagga-users 14908] IXP Design with Quagga Server

On Wed, 17 Jan 2018, Pardeep wrote:

> Queries;
>
> 1. Will Quagga server be able to handle BGP millions of routes in its
> routing table?

With few peers, yes. In an IXP with many peers, likely not - well, most of
the time it will, as long as the peers are stable, but peer's going down
will cause havoc (if they advertise non-trivial numbers of routes esp.).
This is why the other respondent doesn't know of many IXes using Quagga.

Attempts have been made to address this, by adding even more buffering of
works (on top of the buffering I added years ago with the workqueue), by
adding threading (either trying to fully multi-thread bgpd, or trying to
just thread off non-UPDATE peer messages like KEEPALIVE).

The former perhaps buys some latency here and there, but ultimately it adds
more overheads, more CPU, more memory usage, and makes bgpd throughput
slower again - which actually can make the problem worse, by making bgpd
scale even worse. It also adds complexity, and bugs, and makes the code
harder to maintain. Which the threading approach also does, but even more
so.

Killing one of the good properties of Quagga bgpd (it scales up well memory
and CPU wise, route-wise - better than BIRD, ignoring the poor "significant
peer down" problem), to add dubious fixes for the peer-down performance
problem, is probably not the right long-term approach.

BIRD proves the buffering approaches in Quagga were, ultimately, largely
dumb. BIRD (and OpenBGPd) also proves you don't need threads everywhere.
If you need layers and layers of buffering, and threads per peer, to handle
BGP, you're just doing something wrong and have screwed up.

The lessons from BIRD should be heeded. ;)

I was thinking about that a while ago, and there may actually some very
simple fixes to the peer-down problem. Least some significant improvements.

Hoping to try those out sometime. Just need to find some elusive spare time.

----

At the protocol level, there is also the issue that standard BGP simply
isn't very good at the job an IXP wants it to do: Act as a light-weight
central 'hub'.

Stock BGP intrinsically is a distributed shortest-path algorithm. The very
semantics of its UPDATE messages inherently implement an iterative,
distributed computation of a matrix of costs over a graph.

Moving a lump of data quickly from one neighbour to another simply is _not_
what a BGP-4 UPDATE message is designed to be good at.

There have been attempts to try bend BGP UPDATEs to be good at moving data,
but they (the 2 I'm thinking of anyway) retain significant elements of the
distributed path computation aspects of BGP UPDATE, for compatibility /
minimum change reasons. Hence they will continue to suck for efficiently
moving data from one place to another (cause just that's intrinsically at
odds with doing an iterative, distributed path computation at the same time
with the same messages).

We could actually add a Quagga specific extension though..

Again, just needs someone with time. ;)

regards,
--
Paul Jakma | paul@jakma.org | @pjakma | Key ID: 0xD86BF79464A2FF6A
Fortune:
Freedom is what you do with what's been done to you.
-- Jean-Paul Sartre

_______________________________________________
Quagga-users mailing list
Quagga-users@lists.quagga.net
https://lists.quagga.net/mailman/listinfo/quagga-users