Mailing List Archive

On Thu, 6 May 2004, Andrew J. Schorr wrote:

> Then I flush the address from the interface. ip monitor shows the
> following netlink messages:
>
> Deleted 7: eth1.4 inet 10.147.33.197/27 brd 10.147.33.223 scope global eth1.4
> Deleted broadcast 10.147.33.223 dev eth1.4 table local proto kernel scope link src 10.147.33.197
> Deleted broadcast 10.147.33.192 dev eth1.4 table local proto kernel scope link src 10.147.33.197
> Deleted local 10.147.33.197 dev eth1.4 table local proto kernel scope host src 10.147.33.197


> So the netlink delete messages include everything except the first
> "scope link" route.

Hmmm:

# ip add del 10.0.0.1/24 broadcast 10.0.0.255 dev dummy0

gives me:

Deleted 4: dummy0 inet 10.0.0.1/24 brd 10.0.0.255 scope global dummy0
Deleted 10.0.0.0/24 dev dummy0 proto kernel scope link src 10.0.0.1
Deleted broadcast 10.0.0.255 dev dummy0 table local proto kernel scope link src 10.0.0.1
Deleted broadcast 10.0.0.0 dev dummy0 table local proto kernel scope link src 10.0.0.1
Deleted local 10.0.0.1 dev dummy0 table local proto kernel scope host src 10.0.0.1

> I guess the kernel assumes that the deletion is implicit in the
> deletion of the address. But zebra seems to hold on to that route.

I guess it is, and we could potentially scan the routing table and
remove the corresponding connected route on an interface address
removal, instead of on the netlink event.

> This is a problem because zebra may then ignore other routes from
> OSPF that are actually now the best route, but zebra doesn't
> realize it.
>
> Any thoughts? Is zebra wrong? Or is netlink wrong?

No. Normally netlink tells us. The driver which handles eth1.4
(vlan?) potentially is to blame.

> Thanks, Andy

regards,
--
Paul Jakma paul@clubi.ie paul@jakma.org Key ID: 64A2FF6A
warning: do not ever send email to spam@dishone.st
Fortune:
Q: Why is it that the more accuracy you demand from an interpolation
function, the more expensive it becomes to compute?
A: That's the Law of Spline Demand.

Paul,

> No. Normally netlink tells us. The driver which handles eth1.4
> (vlan?) potentially is to blame.

You are 100% correct, this issue seems to be related to the use
of vlans on that interface. When no vlans are involved, the netlink
message to delete the link route does seem to be sent.

> I guess it is, and we could potentially scan the routing table and
> remove the corresponding connected route on an interface address
> removal, instead of on the netlink event.

I think that would be perhaps more robust and paranoid than assuming
that the kernel will send the proper netlink event, but this problem
does seem to be vlan-related. So either the vlan stuff should
be fixed, or zebra should be patched to be more paranoid and delete
all the routes as soon as the interface address goes away (or
both).

If people are interested, I could take a stab at a zebra patch to
do this. In the meantime, I will post a question on the vlan mailing
list.

-Andy

On Thu, May 06, 2004 at 03:37:26PM -0400, Andrew J. Schorr wrote:
> > No. Normally netlink tells us. The driver which handles eth1.4
> > (vlan?) potentially is to blame.
>
> You are 100% correct, this issue seems to be related to the use
> of vlans on that interface. When no vlans are involved, the netlink
> message to delete the link route does seem to be sent.

Actually, I spoke too soon, I am seeing exactly the same problem
on a non-VLAN ethernet interface (eth0). So I don't think we
can blame it on vlans. I agree that the behavior in the example with
dummy0 looks ideal, but ethernet interfaces seem not to behave so nicely,
perhaps because the dummy0 device wasn't really being used. It seems to me
like something is holding open a reference to the ethernet interface
that is stopping that route from being deleted, but that's just a wild
guess...

So the question is how to proceed: try to chase down the kernel netlink
issue here, or just try to make quagga/zebra more robust so that
it doesn't care?

-Andy

On Thu, 6 May 2004, Andrew J. Schorr wrote:

> I think that would be perhaps more robust and paranoid than
> assuming that the kernel will send the proper netlink event, but
> this problem does seem to be vlan-related. So either the vlan
> stuff should be fixed, or zebra should be patched to be more
> paranoid and delete all the routes as soon as the interface address
> goes away (or both).

Netlink is unreliable, so removing connected routes when we receive
the address removal, instead of waiting for the removal event, might
also work. (though, unreliability of netlink is a more general
problem).

The only problem would be, what do we do about the route removal
event? we'd end up looking through the route_table twice for every
address removal. (at address removal, then for the route delete
event).

> If people are interested, I could take a stab at a zebra patch to
> do this. In the meantime, I will post a question on the vlan
> mailing list.

Getting VLAN fixed would probably be easiest.

> -Andy

regards,
--
Paul Jakma paul@clubi.ie paul@jakma.org Key ID: 64A2FF6A
warning: do not ever send email to spam@dishone.st
Fortune:
the printer thinks its a router.

Andrew J. Schorr wrote:
> On Thu, May 06, 2004 at 03:37:26PM -0400, Andrew J. Schorr wrote:
> > > No. Normally netlink tells us. The driver which handles eth1.4
> > > (vlan?) potentially is to blame.
> >
> > You are 100% correct, this issue seems to be related to the use
> > of vlans on that interface. When no vlans are involved, the
> > netlink message to delete the link route does seem to be sent.
>
> Actually, I spoke too soon, I am seeing exactly the same problem
> on a non-VLAN ethernet interface (eth0). So I don't think we
> can blame it on vlans. I agree that the behavior in the example
> with dummy0 looks ideal, but ethernet interfaces seem not to behave
> so nicely, perhaps because the dummy0 device wasn't really being
> used. It seems to me like something is holding open a reference to
> the ethernet interface that is stopping that route from being
> deleted, but that's just a wild guess...

How do you get this kernel route at all?

t3# conf t
t3(config)# interface eth1.118
t3(config-if)# ip address 10.123.123.1/30
t3(config-if)# do show ip route 10.123.123.0/24 longer-prefixes
Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF,
I - ISIS, B - BGP, > - selected route, * - FIB route

t3(config-if)# do start-shell
t3:/home/hasso# ip addr add 10.123.123.5/30 dev eth1.118
t3:/home/hasso# exit
t3(config-if)# do show ip route 10.123.123.0/24 longer-prefixes
Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF,
I - ISIS, B - BGP, > - selected route, * - FIB route

t3(config-if)#

I'm using 2.4.26 at the moment.

> So the question is how to proceed: try to chase down the kernel
> netlink issue here, or just try to make quagga/zebra more robust so
> that it doesn't care?

Netlink handling in Quagga has serious issues anyway and I would be
more than glad if someone will take care about fixing it. See
http://lists.quagga.net/pipermail/quagga-dev/2004-April/001018.html


--
Hasso Tepper
Elion Enterprises Ltd.
WAN administrator

Hasso Tepper wrote:

> How do you get this kernel route at all?

Good question indeed... this appears to be the root cause for this
problem. And I have some doubts as for previous arguments brought
earlier in this thread:

- normally, you should not see two auto-generated routes per configured
IP address, as in the case described here (IIUC, you see one
"connected" and one "kernel" routes, in the zebra VTY that is). What
you should have, when things work as expected, is a single "connected"
route in zebra's RIB (see below).

- zebra generates the connected route internally, without relying or
waiting for the explicit route reflection from the kernel, as an
outcome of the IP address event. See uses of
zebra/connected.c/connected_up_ipv4(), as those accompany any event of
address change (either user/VTY or kernel originated).

- same holds for removal of the "connected" route upon IP address flush.
And, indeed, in the description earlier this thread it appears that
the problem lies with the removal of the other route, that denoted as
"kernel" (see below).

- how doesn't zebra normally generate a duplicate route, when it gets
the kernel-originated connected route reflection? Normally, the kernel
installs those routes with RTPROT_KERNEL protocol identifier (you can
see this with iproute2 as well), and zebra's netlink layer ignores
such routes (see zebra/rt_netlink.c/netlink_route_table() and
netlink_route_change()). So, there's no problem.

(Note: one needs to make a delicate observation when talking about route
types in zebra, as there are two separate sets of "protocol" or "types"
in this case -- there's the kernel notation, which uses RTPROT_KERNEL,
RTPROT_STATIC, RTPROT_ZEBRA, etc, and there's zebra's internal notation,
which uses ZEBRA_ROUTE_KERNEL, ZEBRA_ROUTE_CONNECT, ZEBRA_ROUTE_STATIC,
etc. This way, all kernel originated routes, except those filtered by
rt_netlink.c, are denoted as ZEBRA_ROUTE_KERNEL in zebra, and all zebra
originated routes -- be them static or dynamic -- are denoted as
RTPROT_ZEBRA in the kernel...)

My conclusions so far are: (a) zebra's modeling of connected routes is
consistent, and it works in the common case; (b) netlink messages are
consistent to that extent (again, in the common case); and so (c) it all
should work.

So, what doesn't work here? My guess: your connected route in the kernel
is not denoted as RTPROT_KERNEL, for some reason. And so, zebra's
netlink doesn't filter it, and it finds its way into the RIB as a
"kernel" route. I don't really understand why it isn't flushed when you
remove the address, and it should be further investigated, but I can
assume that there's some pathological coincidence that made this route
come up in your kernel tables in the first place, that might explain
this as well. Could you please dump your 'ip route list table main'
output, before this whole scenario begins? (that is, prior to any
problematic address removal)

> > So the question is how to proceed: try to chase down the kernel
> > netlink issue here, or just try to make quagga/zebra more robust so
> > that it doesn't care?
>
> Netlink handling in Quagga has serious issues anyway and I would be
> more than glad if someone will take care about fixing it. See
> http://lists.quagga.net/pipermail/quagga-dev/2004-April/001018.html

Changing netlink behavior based on that instance of a problem seems a
little hasty to me... I must say that, although zebra's netlink layer
does suffer from some inherent design problems (like dealing with
asynchronous ack messages, socket overflows, and so on) it is well
written to handle the normal mode of operation, as is evident to be in
this case. I believe we should first eliminate the suspect of an
inconsistent kernel routing table configuration that caused this.

Gilad

Gilad Arnold wrote:
> Hasso Tepper wrote:
> > Netlink handling in Quagga has serious issues anyway and I would
> > be more than glad if someone will take care about fixing it. See
> > http://lists.quagga.net/pipermail/quagga-dev/2004-April/001018.h
> >tml
>
> Changing netlink behavior based on that instance of a problem seems
> a little hasty to me... I must say that, although zebra's netlink
> layer does suffer from some inherent design problems (like dealing
> with asynchronous ack messages, socket overflows, and so on) it is
> well written to handle the normal mode of operation, as is evident
> to be in this case. I believe we should first eliminate the suspect
> of an inconsistent kernel routing table configuration that caused
> this.

I'm not saying that we should change netlink behavior. As I can't
reproduce "kernel routes remain in rib" bug, I'm not sure that we
should fix something because of that at all ;). But unability to
handle socket overflows and so on is critical IMHO. It just doesn't
scale. User can run into serious problems (rib and kernel table out
of sync) any time without notice and there is no easy workaround.
It's very annoying for me as well, but unfortunately fixing this is
beyond my knowledge at the moment.

--
Hasso Tepper
Elion Enterprises Ltd.
WAN administrator

On Thu, 6 May 2004, Andrew J. Schorr wrote:

> Hi,
>
> I am seeing the following problem with zebra (quagga 0.96.4 pulled
> from CVS on 20040427). It occurs under linux 2.4.22 and
> under linux 2.6.4.
>
> If I remove the IPV4 address from an ethernet interface, using this command:
>
> ip addr flush dev eth1.4
>
> then the zebra routing table is not being updated properly.
>

The quick answer is:

Don't use ip route tools to add/del addresses or routes. Use ZEBRA. It
is very good at doing this and I've been doing this in production forever
with both v4 and v6 addresses.

conf t
int [insert your interface here]
ip add x.x.x.x/y [secondary]
ipv6 add xx:xx:xx::xx/yy

...you get the point.

The long answer is: Sounds like more of an issue with netlink than with
Zebra. Write a patch, submit it and hope it is adopted into the kernel.

John

John Fraizer wrote:
> Don't use ip route tools to add/del addresses or routes. Use
> ZEBRA. It is very good at doing this and I've been doing this in
> production forever with both v4 and v6 addresses.

You don't use many interfaces probably. With 15 interfaces you will
get netlink buffer overrun probably. I'm using Quagga on routers with
more than hundred interfaces (vlans) and using zebra daemon for
assigning ip addresses is certainly not option for me. See
[quagga-dev 1019] for detailed description.

--
Hasso Tepper
Elion Enterprises Ltd.
WAN administrator

Thanks for your responses.

On Fri, May 07, 2004 at 01:22:52PM +0300, Gilad Arnold wrote:
> Hasso Tepper wrote:
>
> > How do you get this kernel route at all?
>
> Good question indeed... this appears to be the root cause for this
> problem.
>
> So, what doesn't work here? My guess: your connected route in the kernel
> is not denoted as RTPROT_KERNEL, for some reason. And so, zebra's
> netlink doesn't filter it, and it finds its way into the RIB as a
> "kernel" route. I don't really understand why it isn't flushed when you
> remove the address, and it should be further investigated, but I can
> assume that there's some pathological coincidence that made this route
> come up in your kernel tables in the first place, that might explain
> this as well. Could you please dump your 'ip route list table main'
> output, before this whole scenario begins? (that is, prior to any
> problematic address removal)
>

You guys are correct. I did some further investigation, and the problem
is that the Red Hat ifup script that is bringing up the interface
contains this code:

# Add a route for the subnet. Replace any existing route.
if [ "${ISALIAS}" = no -a "${NETMASK}" != "255.255.255.255" ]; then
ip route replace ${NETWORK}/${PREFIX} ${SRC} ${WINDOW:+window $WINDOW} dev ${REALDEVICE}
fi

This is where the problematic route comes from (it actually replaces an
almost identical route that was automatically generated by the kernel when
the address was assigned and that did have the proto kernel flag).

If I try to avoid using ifup by configuring the interface instead
inside zebra.conf, it seems not to work with VLANs. In order to
create the VLAN device, one must use the /sbin/vconfig command
(e.g. /sbin/vconfig add eth0 11). If I run that command outside
of zebra, then zebra can configure the interface. But zebra itself
does not contain the logic to create the VLAN device (which is
actually quite trivial, just an ioctl SIOCSIFVLAN request).

So it seems to me that there are basically two ways to solve this problem:
either patch ifup to include "proto kernel" in that ip route replace command,
or enhance quagga/zebra to create VLAN interfaces properly
(by adding support for the IOS "encapsulation dot1Q <vlan #>" interface
subcommand).

But there seem to be reports of other problems with netlink when configuring
all the interfaces inside quagga/zebra, so patching ifup seems like the best
bet right now. In the long run, though, I guess we should add the
encapsulation dot1Q interface subcommand to zebra.

Thanks,
Andy

Andrew J. Schorr wrote:
> If I try to avoid using ifup by configuring the interface instead
> inside zebra.conf, it seems not to work with VLANs. In order to
> create the VLAN device, one must use the /sbin/vconfig command
> (e.g. /sbin/vconfig add eth0 11). If I run that command outside
> of zebra, then zebra can configure the interface. But zebra itself
> does not contain the logic to create the VLAN device (which is
> actually quite trivial, just an ioctl SIOCSIFVLAN request).

It isn't so trivial actually. About year ago I wrote patch for myself
to handle vlans in zebra daemon, but I faced several issues and it
never became production ready. And btw, logic of handling vlans in
*BSD platform is quite different. Don't know about Solaris.

> So it seems to me that there are basically two ways to solve this
> problem: either patch ifup to include "proto kernel" in that ip
> route replace command, or enhance quagga/zebra to create VLAN
> interfaces properly (by adding support for the IOS "encapsulation
> dot1Q <vlan #>" interface subcommand).

<vlan #> doesn't make sense in Linux.

> But there seem to be reports of other problems with netlink when
> configuring all the interfaces inside quagga/zebra, so patching
> ifup seems like the best bet right now. In the long run, though, I
> guess we should add the encapsulation dot1Q interface subcommand to
> zebra.

I doubt that it will be high priority. Fixing init scripts is good
idea anyway.

--
Hasso Tepper
Elion Enterprises Ltd.
WAN administrator

On Fri, May 07, 2004 at 11:52:44PM +0300, Hasso Tepper wrote:
> Andrew J. Schorr wrote:
> > If I try to avoid using ifup by configuring the interface instead
> > inside zebra.conf, it seems not to work with VLANs. In order to
> > create the VLAN device, one must use the /sbin/vconfig command
> > (e.g. /sbin/vconfig add eth0 11). If I run that command outside
> > of zebra, then zebra can configure the interface. But zebra itself
> > does not contain the logic to create the VLAN device (which is
> > actually quite trivial, just an ioctl SIOCSIFVLAN request).
>
> It isn't so trivial actually. About year ago I wrote patch for myself
> to handle vlans in zebra daemon, but I faced several issues and it
> never became production ready. And btw, logic of handling vlans in
> *BSD platform is quite different. Don't know about Solaris.

Right, I'm not surprised that other O/S's handle it differently,
but I guess that the proper autoconf magic should let us add
vlan support for each O/S as we understand how to do it.
It seems to me that we should shoot for implementing the
"encapsulation dot1Q <vlan #>" interface subcommand", unless
people are content to configure their interfaces outside of zebra.
I don't know about other shops, but we are using VLANs more and more here...

> <vlan #> doesn't make sense in Linux.

I'm not sure what you mean by that, but I don't think the concept
of the VLAN id number is an O/S issue, the VLAN id is a function of
the 802.1Q spec's creating a 12-bit field in the ethernet frame header
to contain the VLAN number.

> Fixing init scripts is good idea anyway.

Yes, that is what we are doing (patching /sbin/ifup).

-Andy

Gilad Arnold wrote:
> Hasso Tepper wrote:
> > How do you get this kernel route at all?
>
> Good question indeed... this appears to be the root cause for this
> problem. And I have some doubts as for previous arguments brought
> earlier in this thread:
>
> - normally, you should not see two auto-generated routes per
> configured IP address, as in the case described here (IIUC, you see
> one "connected" and one "kernel" routes, in the zebra VTY that is).
> What you should have, when things work as expected, is a single
> "connected" route in zebra's RIB (see below).

While it's certainly case with IPv4, I just discovered that every
interafce up generates two kernel routes as well when IPv6 is used -
fe80::/64 and ff00::/8. So if last event was interafce up, you see
them, if down, then you don't. Sounds like bug.


--
Hasso Tepper
Elion Enterprises Ltd.
WAN administrator

(Apologies for my late response)

Andrew J. Schorr wrote:

>>It isn't so trivial actually. About year ago I wrote patch for myself
>>to handle vlans in zebra daemon, but I faced several issues and it
>>never became production ready. And btw, logic of handling vlans in
>>*BSD platform is quite different. Don't know about Solaris.
>
> Right, I'm not surprised that other O/S's handle it differently,
> but I guess that the proper autoconf magic should let us add
> vlan support for each O/S as we understand how to do it.
> It seems to me that we should shoot for implementing the
> "encapsulation dot1Q <vlan #>" interface subcommand", unless
> people are content to configure their interfaces outside of zebra.
> I don't know about other shops, but we are using VLANs more and more here...

I'm with Hasso on this, simply because zebra does not deal with layer-2
matters (and VLAN encapsulation is definitely such). On the other hand,
it should well manage assigning IP addresses to hundreds of "interfaces"
(be them VLAN or else), and according to Hasso this isn't possible --
I'd say that this should be fixed indeed (somehow, sometime...).

One thing I can tell about zebra: IIRC, the code if flexible enough so
you can actually configure IP addresses for an interface which doesn't
exist in the underlying kernel by the time you configure the addresses
-- zebra will create an abstract internal interface object, and assign
the addresses to it (although they will have no effect on the system,
obviously). As soon as that interface is attached to the kernel (and
zebra gets a netlink reflection) zebra "wakes up" all addresses that
were previously associated with it. In other words, you can save address
configuration in zebra.conf, but add the actual VLAN through ifconfig,
without worrying about which comes first. Worth a try, I think...

>>Fixing init scripts is good idea anyway.
>
> Yes, that is what we are doing (patching /sbin/ifup).

Should solve your problem, but I wonder -- why is it there in the first
place?

Gilad

Gilad Arnold wrote:
> Andrew J. Schorr wrote:
> >>It isn't so trivial actually. About year ago I wrote patch for
> >> myself to handle vlans in zebra daemon, but I faced several
> >> issues and it never became production ready. And btw, logic of
> >> handling vlans in *BSD platform is quite different. Don't know
> >> about Solaris.
> >
> > Right, I'm not surprised that other O/S's handle it differently,
> > but I guess that the proper autoconf magic should let us add
> > vlan support for each O/S as we understand how to do it.
> > It seems to me that we should shoot for implementing the
> > "encapsulation dot1Q <vlan #>" interface subcommand", unless
> > people are content to configure their interfaces outside of
> > zebra. I don't know about other shops, but we are using VLANs
> > more and more here...
>
> I'm with Hasso on this, simply because zebra does not deal with
> layer-2 matters (and VLAN encapsulation is definitely such).

Yes. It's not hard to implement it in plaform specific way and if
someone does it, please put it up so other users can apply patch, use
it etc. But it will be damn hard to implement it to support all
platforms and it's even harder to make it bulletproof. Zebra just
isn't meant to deal with creating/destroying l2 interfaces.

> On the
> other hand, it should well manage assigning IP addresses to
> hundreds of "interfaces" (be them VLAN or else), and according to
> Hasso this isn't possible -- I'd say that this should be fixed
> indeed (somehow, sometime...).

Yes, pleeeeease :). I just don't have needed knowledge about netlink
and zebra daemon internals to do it myself. Of course, I have ability
to learn, but it will take a lot of time and I'd spend my time to
fix/implement things I already know quite well. ;)

In my VERY humble opinion there are some steps that should be done:

1) Thread to resync data with kernel - ie. reads interface/routing
tables from kernel; makes needed changes - adds data (addresses,
interfaces, routes) to zebra structures if needed, adds
routes/addresses to kernel if needed etc.
2) Detect overrun in netlink socket (it actually does it already),
switch socket at once into nonblocking mode and schedule resync
thread.

That's receiveing part, but that's critical. Sending is simpler as
messages to the kernel can be queued.

> One thing I can tell about zebra: IIRC, the code if flexible enough
> so you can actually configure IP addresses for an interface which
> doesn't exist in the underlying kernel by the time you configure
> the addresses -- zebra will create an abstract internal interface
> object, and assign the addresses to it (although they will have no
> effect on the system, obviously). As soon as that interface is
> attached to the kernel (and zebra gets a netlink reflection) zebra
> "wakes up" all addresses that were previously associated with it.
> In other words, you can save address configuration in zebra.conf,
> but add the actual VLAN through ifconfig, without worrying about
> which comes first. Worth a try, I think...

Yes, it works very well if you don't have many interfaces. I'm using
zebra to store IPv6 addresses. For users who have problems with
buffer overruns in netlink socket and don't want to change
rmem_default (it will affect all created sockets), I made small patch
to set netlink socket receive buffer size from command line. But this
is hack and I don't intend to commit it. Note, that for needed effect
rmem_max should be changed anyway.

http://hasso.linux.ee/quagga/pending-patches/ht-20040512-netlink-rcvbuf.patch

> >>Fixing init scripts is good idea anyway.
> >
> > Yes, that is what we are doing (patching /sbin/ifup).
>
> Should solve your problem, but I wonder -- why is it there in the
> first place?

Init scripts in distributions make strange things. In Debian there is
different problem with init scripts. For reference my experience to
other users who might hit this as well.

I discovered that init scripts in Debian use ifconfig and assigning ip
addresses with ifconfig results heavy netlink storm. Attached file
contains output from "ip monitor" when I executed only one command -
"ifconfig eth0.100 192.168.0.1 netmask 255.255.255.252 broadcast
192.168.0.3 up". This will confuse ospfd if you redistribute
connected (that's how I discovered it), but this is bug in ospfd of
course and fixing this is in my TODO list. So, Debian init scripts in
current form are unusable as well :(. As workaround I used this kind
of constructs in /etc/network/interfaces (manual method exists only
in unstable):

iface eth0.100 inet manual
vlan_raw_device eth0
up ip link set dev eth0.100 up
up ip addr add 192.168.0.1/30 dev eth0.100


--
Hasso Tepper
Elion Enterprises Ltd.
WAN administrator

On Wed, May 12, 2004 at 12:57:54PM +0300, Gilad Arnold wrote:
> As soon as that interface is attached to the kernel (and
> zebra gets a netlink reflection) zebra "wakes up" all addresses that
> were previously associated with it. In other words, you can save address
> configuration in zebra.conf, but add the actual VLAN through ifconfig,
> without worrying about which comes first. Worth a try, I think...

Perhaps, but that does not appeal to me. If I have to create the VLAN
outside of zebra, I'm just as happy to assign the address in the
same place (e.g. using Red Hat's ifup script). From a system administration
perspective, it's much more maintainable to have everything in the same
place (either ifup or zebra, but not both).

In any case, I don't think zebra is capable of handling addressing
for PtP links (I don't think it lets you assign the peer address), so
it seems to me that zebra is not really capable of managing the
interface addresses at this time. Plus, the netlink problems that Hasso
mentions seem to make it clear that the interfaces are best configured
outside of zebra.

> >Yes, that is what we are doing (patching /sbin/ifup).
>
> Should solve your problem, but I wonder -- why is it there in the first
> place?

I'm not sure, but I think the idea is to allow setting the TCP window.
Here's the command:

ip route replace ${NETWORK}/${PREFIX} ${SRC} ${WINDOW:+window $WINDOW} dev ${REALDEVICE}

We just patched it by adding "proto kernel" to the end.

-Andy