Mailing List Archive

On Wed, 14 Jan 2004, Greg Troxel wrote:

> Not on linux, its IP_PKTINFO, slightly different.
>
> I suppose we should have an abstraction layer to hide this then.

Quite probably.

> That's too bad, since it can be wrong and is more subject to
> spoofing.

Well, that's why OSPF MD5 auth exists :)

regards,
--
Paul Jakma paul@clubi.ie paul@jakma.org Key ID: 64A2FF6A
warning: do not ever send email to spam@dishone.st
Fortune:
I don't want to live on in my work, I want to live on in my apartment.
-- Woody Allen

Paul Jakma wrote:

> I had this discussion with Gilad a long time ago. I argued that
> perhaps zebra should do detection of duplicate and included subnets
> and set the secondary flag accordingly. Discussion was inconclusive
> (though in the case of linux, kernel sets secondary flag, and its
> reflected on).

Not accurate: indeed rt_netlink.c sets the secondary flag for addresses
that are not configured by zebra, but internally configured addresses
carry a secondary attribute the obeys the user statement (ie, if the
user said 'ip address <addr>/<plen> secondary' then it's secondary, and
otherwise non-secondary, from zebra's pov).


> What about included subnets? Eg,
>
> eth0:
> 1.1.1.0/24
> 1.1.1.x/25
> 1.1.1.y/32

What about them? They don't imply any forwarding ambiguity, since the
longest prefix match wins, so what's the problem?


> ie, in the case of linux, removing the primary address removes the
> DUP (or secondary) addresses too.

Yeah, and IMO it makes sense and doesn't cause any trouble for zebra, as
this flushing of addresses is well captured via netlink reflections.

Gilad

Paul Jakma wrote:

>>Paul tells me that it will not send packets on ZEBRA_IFA_SECONDARY
>>networks, which strikes me as behavior in the same spirit?
>
> Yep, however, ZEBRA_IFA_SECONDARY at present is only set on netlink
> platforms (ie linux).

Not true, for addresses configured from within zebra's VTY, it is set
for any address which the user has stated to be 'secondary' in the
command line. I think that's one basic inconsistency with zebra's
handling of the secondary attribute, and some decision should be taken
here, and my personal beliefs about this say that

#1, secondary in VTY is redundant and is there for historical (ie,
Cisco) reasons; nonetheless, although attempting to affect address
properties in the kernel, it's totally useless in this regard

#2, a true secondary policy should be kernel driven; this however,
requires a more sophisticated address redistribution process, such that
waits for a kernel reflection (to see what attributes the kernel has
attached to an address) and respond accordingly.

#3, even if #2 above is not currently implemented, it may be a good
practice for the protocols to determine their own "overlapping
avoidance" policy, without counting on zebra to mark secondaries for
them -- healthy and modular. That is, if secondaries are marked, let's
use those flags, otherwise we can do without it (because it means that
zebra/kernel don't really care for primary/secondary hierarchy).

Gilad

Paul Jakma wrote:

>>like it might well be the right thing, but I'd like to see the
>>semantics for ZEBRA_IFA_SECONDARY documented.
>
> At the moment it reflects the netlink IFA_F_SECONDARY flag.

Inaccurate, see my previous posts.

Gilad

Paul Jakma wrote:

> One other thing we do need to consider is 'included' subnets - linux
> allows this and there is no flag to distinguish such.

Obviously, because it does not imply any forwarding ambiguity (see
previous replies).


>> eth0:
>> 1.1.1.0/24
>> 1.1.1.x/25
>> 1.1.1.y/32
>>
>>I'd say we should look at all the routing protocols which deal with
>>this, and see what def would be most useful for them. I consider
>>the above example pathological;
>
> So do i. I would consider it admin error.

Why's that? It is perfectly valid from IP forwarding pov...


> Another example is:
>
> eth0:
> 1.1.1.x/24
> eth1:
> 1.1.1.y/25
>
> where the two interfaces are on completely seperate networks, if must
> associate an address (eg source addr of a packet) with an interface
> and that address fals within 1.1.1.y/25 which interface do you pick?

The one whose implied connected route holds to be the longest prefix
match for the destination: given that x=1 and y=129, if your destination
is 5 then your longest prefix match is this connected route

1.1.1.0/24 via eth0 src 1.1.1.1

and if it's 150 then your longest prefix match is this connected route

1.1.1.128/25 via eth1 src 1.1.1.129

Note that, y may even imply a connected route with overlaps with x
itself, eg y=5; then again, the kernel follows the same forwarding
decision scheme as demonstrated above.


>>having two addrs in the same prefix, having overlapping nonequal
>>prefixes to me is a sign of a confused network design.
>
> yep.

Clearly, a reasonable sysadmin wouldn't set those addresses in the first
place, however they seem perfectly valid to me.

Gilad

Greg Troxel wrote:

> So yes, I think that we can define a sensible general behavior.
>
> I had one addr, and added an alias. ifconfig showed the new one as an
> alias. I deleted the first, and added it back as an alias. Now the
> original address shows as an alias.

I didn't quite get it -- can you list the commands and corresponding
ifconfig output for this sequence?


> NetBSD does this, I think. ifconfig loops over the addrs, and
> addresses which are not equal to the 'primary' addr (what SIOCGIFADDR
> returns) are printed with the word alias.

So, an 'alias' in NetBSD (or *BSD?) is any address past the first
address configured for an interface? If so, then it's a rougher scheme
than Linux, which allows finer chains in the sense that only addresses
implying the very same subnet are primary/secondary related.

Is this a correct prognosis?


> So zebra would have to keep track of such designations. This is
> probably hairy, but it is pretty easy to write an invariant checker
> and call it often.

It should only be called upon address change reflection from the kernel;
on the other hand, if BSD has a netlink interface, then any such address
change should be reported either way (namely, if a non-alias is deleted
and some alias is promoted for non-alias, then those two changes should
be reported back to zebra, aren't they?)


> - for kernels which set such flags, do these kernel's properly notify
> zebra of events relating to change in status of duplicate/secondary
> addresses? eg, if the primary address is removed, what does the
> kernel do for secondary addresses and does it inform zebra
> appropriately?
>
> On Linux, the secondary address seems to vanish too.

And it seems to distribute the corresponding sequence of netlink
messages to listeners (haven't really checked on that, but it seems
obvious).


> Do BSD kernels have IFA_F_SECONDARY? :)
>
> no

But they have alias labels, I understand from your saying? (should be
installing a BSD soon... ;->)


> Some do, but it isn't clear that this is a service that the kernel
> needs to provide. AFAICT, the only thing the kernel does with the
> flag is delete the secondary addrs when the primary is deleted.

No, it's original intend is to break ambiguity when assigning a source
address to an outbound packet. Flushing the chain of secondaries in
Linux is just one choice out of many (eg, promoting some secondary to be
a primary, as I understand your NetBSD does).


> Is it an error (on Linux) to have two addrs in the same prefix with
> neither marked secondary?

It's impossible, as the second address of this kind would be
automatically marked secondary by the kernel. Try it out.


> One other thing we do need to consider is 'included' subnets - linux
> allows this and there is no flag to distinguish such.
>
> Right, so we would need another flag that says that this address is on
> a subnet that is included in another (larger) subnet. If anyone
> cares, and I'm not clear.

I don't think it's required, nested addresses/subnets do not involve any
forwarding ambiguity, IMO (see previous posting).


> So do i. I would consider it admin error. Another example is:
>
> eth0:
> 1.1.1.x/24
> eth1:
> 1.1.1.y/25
>
> This is also a broken setup.

Haa? I don't think so (see previous posting). It may be "illegal" in the
eyes of a Cisco user, but it is perfectly valid from IP forwarding pov.


> where the two interfaces are on completely seperate networks, if must
> associate an address (eg source addr of a packet) with an interface
> and that address fals within 1.1.1.y/25 which interface do you pick?
>
> Do you mean use the source addr to find the interface? One shouldn't
> do that, but instead set SO_RECVIF so that the kernel will tell you
> the interface that the packet came in on.

Okay, that's the correct answer to Paul's question (I answered a
different question that I figured he was referring to... sorry).


> ospfd gets very confused by such setups, it'd be nice if it could
> detect such cases. However it is still a case of admin error - dont
> do that.
>
> One could add code to zebra to intersect all the configured prefixes
> and if a, b are found s.t. a != b and intersect(a,b) then issue a
> warning.

Again, I don't see a reason to do so.

Gilad

Paul Jakma wrote:

>>To find the source address of the packet, it can use IP_RECVIF,
>>right?
>
> Not on linux, its IP_PKTINFO, slightly different. Iirc, ospfd uses
> source address of incoming packets to lookup appropriate ospf
> interface struct.

So apparently that's another problem with ospfd... ;-> Now seriously,
why should the daemon assume any restriction on the address scheme of
it's underlying kernel? Why not use some solid, guaranteed way to
conclude the inbound interface? Clearly, it should be embedded into the
kernel layer API, but that's not too much of a problem, IMO.

Gilad

On Thu, 15 Jan 2004, Gilad Arnold wrote:

> Not accurate: indeed rt_netlink.c sets the secondary flag for
> addresses that are not configured by zebra, but internally
> configured addresses carry a secondary attribute the obeys the user
> statement (ie, if the user said 'ip address <addr>/<plen>
> secondary' then it's secondary, and otherwise non-secondary, from
> zebra's pov).

right yes, they can be manually specified. fair enough.

> What about them? They don't imply any forwarding ambiguity, since
> the longest prefix match wins, so what's the problem?

Not a forwarding ambiguity, of course. But for connected subnets its
ambigious. Routers dont just forward packets, they also communicate
(eg for routing protocols :) ).

If you have overlapping connected subnets and you wish to send a
packet to an address within the overlap, where does it go? You argue
to the most specific prefix subnet, but that host could be on the
other subnet. Anyway, such a case is admin error.

> Yeah, and IMO it makes sense and doesn't cause any trouble for
> zebra, as this flushing of addresses is well captured via netlink
> reflections.

Yes, it makes sense.

> Gilad

regards,
--
Paul Jakma paul@clubi.ie paul@jakma.org Key ID: 64A2FF6A
warning: do not ever send email to spam@dishone.st
Fortune:
Atlanta makes it against the law to tie a giraffe to a telephone pole
or street lamp.

On Thu, 15 Jan 2004, Gilad Arnold wrote:

> > At the moment it reflects the netlink IFA_F_SECONDARY flag.
>
> Inaccurate, see my previous posts.

It means either netlink set IFA_F_SECONDARY or the admin manually set
the flag.

> Gilad

regards,
--
Paul Jakma paul@clubi.ie paul@jakma.org Key ID: 64A2FF6A
warning: do not ever send email to spam@dishone.st
Fortune:
If built in great numbers, motels will be used for nothing but illegal
purposes.
-- J. Edgar Hoover

Paul Jakma wrote:

> Not a forwarding ambiguity, of course. But for connected subnets its
> ambigious. Routers dont just forward packets, they also communicate
> (eg for routing protocols :) ).

What's the difference to that matter?...


> If you have overlapping connected subnets and you wish to send a
> packet to an address within the overlap, where does it go? You argue
> to the most specific prefix subnet, but that host could be on the
> other subnet. Anyway, such a case is admin error.

Clearly if the destination resides in the nested portion of the larger
subnet, then someone made a mistake. But someone can also make a mistake
by adding a default route to to unknown nexthop, or keep an interface in
shutdown status, and so on. Errors happen, but the model remains the
same ;-> (and has to be as generalized as possible yet consistent)

Gilad

Paul Jakma wrote:

> It means either netlink set IFA_F_SECONDARY or the admin manually set
> the flag.

And in the latter case, there's absolutely no correlation between
zebra's secondary flag and that of the kernel.

Gilad

On Thu, 15 Jan 2004, Gilad Arnold wrote:

> Obviously, because it does not imply any forwarding ambiguity (see
> previous replies).

Right, but it does imply a connected subnet ambiguity. Overlapping
routes are normal and healthy, overlapping interface addresses are
not (other than as secondary interfaces).

> Why's that? It is perfectly valid from IP forwarding pov...

absolutely.

> The one whose implied connected route holds to be the longest prefix
> match for the destination: given that x=1 and y=129, if your destination
> is 5 then your longest prefix match is this connected route
>
> 1.1.1.0/24 via eth0 src 1.1.1.1
>
> and if it's 150 then your longest prefix match is this connected route
>
> 1.1.1.128/25 via eth1 src 1.1.1.129
>
> Note that, y may even imply a connected route with overlaps with x
> itself, eg y=5; then again, the kernel follows the same forwarding
> decision scheme as demonstrated above.

Yes, i agree, as i had stated, i consider it admin error.

> Clearly, a reasonable sysadmin wouldn't set those addresses in the
> first place, however they seem perfectly valid to me.

WRT to reasonble admin: quite clearly, yes. Wrt valid, technically
yes, but as you admin, not reasonable. And having admined a network
where a previous admin had decided to number one network using a /25
from an in use /24, i know funny things happen. Also, admins
sometimes make mistakes or sometimes they just dont realise all the
ins and outs (they can be a hard worked bunch often), it'd be nice to
give them a hand and have ospfd do the right thing rather than have
them scratch their heads wondering why ospfd misbehaves so and
eventually figure out its cause of that host address they added (or
their HA software added) - if they ever figure it out.

The problem is both OSPF and RIP expect to work on basis of subnets.
And I think if its reasonable that they work out what connected nets
are secondary and which are included, then its reasonable for zebra
to do it (do it one place, rather than seperately in every daemon
surely?).

I argued this before, to keep a master table of connected prefixes
and mark addresses, but you didnt seem agreeable to the idea iirc.
The flags can be purely informational, for use by the daemons at
their discretion (eg, as ospfd currently uses).

After the last discussion, i made ospfd check for SECONDARY and not
enable ospf on those addresses (and made sure to tell a certain user
to have his HA software configure 'failover' addresses with the same
netmask as his 'primary' address to ensure the kernel set secondary).
It'd be nice if there was also a INCLUDED flag, so ospfd could do the
same. And it would help ripd too, it appears.

> Gilad

regards,
--
Paul Jakma paul@clubi.ie paul@jakma.org Key ID: 64A2FF6A
warning: do not ever send email to spam@dishone.st
Fortune:
Dreams are free, but you get soaked on the connect time.

On Thu, 15 Jan 2004, Gilad Arnold wrote:

> What's the difference to that matter?...

The OSPF and RIP RFCs like to think of connected subnets? :)

> keep an interface in shutdown status, and so on. Errors happen, but
> the model remains the same ;-> (and has to be as generalized as
> possible yet consistent)

Agreed. See my previous email for more obtuse discussion.
Specifically manual setting of the flag doesnt help where external
software adds addresses (HA software was what brought up previous
ospfd problem (heartbeat and amir's SRRD)).

Detecting secondary addresses and setting a flag thus would be
consistent.

Detecting included subnets and setting a flag thus would be
consistent.

What daemons might make of the flags is down to them.

Is that not consistent?

> Gilad

regards,
--
Paul Jakma paul@clubi.ie paul@jakma.org Key ID: 64A2FF6A
warning: do not ever send email to spam@dishone.st
Fortune:
To understand a program you must become both the machine and the program.

On Thu, 15 Jan 2004, Gilad Arnold wrote:

> And in the latter case, there's absolutely no correlation between
> zebra's secondary flag and that of the kernel.

Yep. But then again, I do not argue that people should not have
enough rope to hang themselves :)

> Gilad

regards,
--
Paul Jakma paul@clubi.ie paul@jakma.org Key ID: 64A2FF6A
warning: do not ever send email to spam@dishone.st
Fortune:
Biology grows on you.

Paul Jakma wrote:

> Detecting secondary addresses and setting a flag thus would be
> consistent.

Yes, as long as it goes hand-in-hand with the kernel's concept of
secondary. Generally speaking, I believe that secondary (the Linux way,
or the Cisco way which is probably equivalent to BSD's aliases) means
that the router should not use that address as a source address, but
rather the primary one. I presume that routing protocols should better
be following this paradigm, so as not to confuse neighbors. And so, I
conclude that such link-oriented protocols (ripd, ospfd) should
implement bundling of secondary network prefixes into the same abstract
"interface" object indexed by the corresponding primary address (thus
not advertising more than once for each such bundle). The point is, that
this "bundle" is different from OS to OS, as Linux probably allows more
delicate bundling that others, for example.


> Detecting included subnets and setting a flag thus would be
> consistent.

Consistent it is, but how would you benefit from this flag? How does it
promote correct behavior of routing protocols (or else)?


> What daemons might make of the flags is down to them.

Don't agree: since secondary is aimed to solve a forwarding problem, and
thus induces constraints on the way routers can use their IP addresses
as source address for outbound packets, routing protocols should follow
this paradigm quite strictly, IMO (that is, given that they have an
accurate notion of what's primary and what isn't with their addresses).
In any case, they should be able to comply with the most general known
scheme (probably the Linux one) with respect to this "bundling" feature
I mentioned above.


> Is that not consistent?

Depends on your scope... ;-> (see above)

Gilad

Paul Jakma wrote:

>>And in the latter case, there's absolutely no correlation between
>>zebra's secondary flag and that of the kernel.
>
> Yep. But then again, I do not argue that people should not have
> enough rope to hang themselves :)

If 'ip addr <addr>/<plen> secondary' is completely useless other than
allowing sysadmins to hang themselves, then I'd say it should be gone.

Gilad

Paul Jakma wrote:

> WRT to reasonble admin: quite clearly, yes. Wrt valid, technically
> yes, but as you admin, not reasonable. And having admined a network
> where a previous admin had decided to number one network using a /25
> from an in use /24, i know funny things happen. Also, admins
> sometimes make mistakes or sometimes they just dont realise all the
> ins and outs (they can be a hard worked bunch often), it'd be nice to
> give them a hand and have ospfd do the right thing rather than have
> them scratch their heads wondering why ospfd misbehaves so and
> eventually figure out its cause of that host address they added (or
> their HA software added) - if they ever figure it out.

Isn't you who mentioned the rope's length a few minutes ago?... ;->


> The problem is both OSPF and RIP expect to work on basis of subnets.
> And I think if its reasonable that they work out what connected nets
> are secondary and which are included, then its reasonable for zebra
> to do it (do it one place, rather than seperately in every daemon
> surely?).

Secondary I agree, as long as it sticks to the OS's notion of secondary.
Included -- well, what exactly would you expect your protocol to do
given that it notices that some connected network is in fact "included"?
If it isn't supposed to advertise (into) it, your saying, wouldn't that
cause those clueless admins to raise an eyebrow trying to figure out why
the hell ospfd doesn't work for that subnet? I say it's crooked either
way, and I still don't see the benefit in concluding "included" network.

Let alone, that computing "included" may be problematic: suppose there
are tens of /24 addresses on a router (no overlapping or so), but
there's a single /16 which contains them all. Clearly, of all them are
marked "included"; however, when that /16 is deleted (the admin has
finally discovered why ospfd helped him by not advertising on his /24
networks), zebra should do some exhaustive work in changing that
attribute, distributing to all protocols, and those in turn should
change their whole configuration by opening new abstract interfaces for
each subnet, state machine, etc... I don't really seem to see the
utility in this.


> After the last discussion, i made ospfd check for SECONDARY and not
> enable ospf on those addresses (and made sure to tell a certain user
> to have his HA software configure 'failover' addresses with the same
> netmask as his 'primary' address to ensure the kernel set secondary).
> It'd be nice if there was also a INCLUDED flag, so ospfd could do the
> same. And it would help ripd too, it appears.

Obviously, you're more pragmatic than I am... ;->

However, I'm looking for a consistent solution, and I believe it should
be as generalized as possible: to this extent I'm not really convinced
that marking "included" is of any benefit.

Gilad

On Thu, 15 Jan 2004, Gilad Arnold wrote:

> If 'ip addr <addr>/<plen> secondary' is completely useless other
> than allowing sysadmins to hang themselves, then I'd say it should
> be gone.

No, its useful to force the secondary flag.

> Gilad

regards,
--
Paul Jakma paul@clubi.ie paul@jakma.org Key ID: 64A2FF6A
warning: do not ever send email to spam@dishone.st
Fortune:
linux: because a PC is a terrible thing to waste
(ksh@cis.ufl.edu put this on Tshirts in '93)

Here is a trace of doing some alias configuration.

2#> ifconfig ex0
ex0: flags=8822<BROADCAST,NOTRAILERS,SIMPLEX,MULTICAST> mtu 1500
capabilities=7<IP4CSUM,TCP4CSUM,UDP4CSUM>
enabled=0<>
address: 00:50:da:[redacted]
media: Ethernet autoselect
status: no carrier
3#> ifconfig ex0 10.0.0.1/24
4#> ifconfig ex0
ex0: flags=8863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST> mtu 1500
capabilities=7<IP4CSUM,TCP4CSUM,UDP4CSUM>
enabled=0<>
address: 00:50:da:[redacted]
media: Ethernet autoselect (none)
status: no carrier
inet 10.0.0.1 netmask 0xffffff00 broadcast 10.0.0.255
inet6 fe80::[redacted]%ex0 prefixlen 64 scopeid 0x5
5#> ifconfig ex0 10.0.0.2/24 alias
6#> ifconfig ex0
ex0: flags=8863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST> mtu 1500
capabilities=7<IP4CSUM,TCP4CSUM,UDP4CSUM>
enabled=0<>
address: 00:50:da:[redacted]
media: Ethernet autoselect (none)
status: no carrier
inet 10.0.0.1 netmask 0xffffff00 broadcast 10.0.0.255
inet alias 10.0.0.2 netmask 0xffffff00 broadcast 10.0.0.255
inet6 fe80::[redacted]%ex0 prefixlen 64 scopeid 0x5

now we have 2 addrs

7#> ifconfig ex0 10.0.0.1/24 delete
8#> ifconfig ex0
ex0: flags=8863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST> mtu 1500
capabilities=7<IP4CSUM,TCP4CSUM,UDP4CSUM>
enabled=0<>
address: 00:50:da:[redacted]
media: Ethernet autoselect (none)
status: no carrier
inet 10.0.0.2 netmask 0xffffff00 broadcast 10.0.0.255
inet6 fe80::[redacted]%ex0 prefixlen 64 scopeid 0x5

alias no longer shows up, since .2 is now the first v4 addr.

9#> ifconfig ex0 10.0.0.2/24 alias
10#> ifconfig ex0
ex0: flags=8863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST> mtu 1500
capabilities=7<IP4CSUM,TCP4CSUM,UDP4CSUM>
enabled=0<>
address: 00:50:da:[redacted]
media: Ethernet autoselect (none)
status: no carrier
inet 10.0.0.2 netmask 0xffffff00 broadcast 10.0.0.255
inet6 fe80::[redacted]%ex0 prefixlen 64 scopeid 0x5

I feebed typing; had I typed 1, there would have been a second inet
line with .1 and alias.

11#> ifconfig ex0 10.0.0.3/16
12#> ifconfig ex0
ex0: flags=8863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST> mtu 1500
capabilities=7<IP4CSUM,TCP4CSUM,UDP4CSUM>
enabled=0<>
address: 00:50:da:[redacted]
media: Ethernet autoselect (none)
status: no carrier
inet 10.0.0.3 netmask 0xffff0000 broadcast 10.0.255.255
inet6 fe80::[redacted]%ex0 prefixlen 64 scopeid 0x5

Without alias, the address is replaced.


So, an 'alias' in NetBSD (or *BSD?) is any address past the first
address configured for an interface? If so, then it's a rougher scheme
than Linux, which allows finer chains in the sense that only addresses
implying the very same subnet are primary/secondary related.

Right, except that I'd say simpler rather than rougher; there is
simply no concept of related enforced by the kernel.

In NetBSD (and I think the others) that interfaces simply may have
more than one address (within any given address family). The notion
of alias is IPv4 only, and is purely for interacting with the user in
order to distinguish between the following two requests:

# basic setup for both requests
ifconfig le0 inet 10.0.0.1/24

# request A:
ifconfig le0 inet 10.0.0.2/24
# now le0 has just 10.0.0.2/24 (or any other addr; the netmask of
# the second address is used of course but the replacement happens
# regardless of its value).

# request B:
ifconfig le0 inet 10.0.0.3/24 alias
# now le0 has two addresses. The second one is second because
# addresses are put at the end of the list.

I am pretty sure that in case B the first address will be used for
source selection.

When you do 'ifconfig le0', the second address (and succeeding) are
printed as alias, but that's because ifconfig has logic to get the
first address and prints the word alias on all addresses that aren't
the same. There is no alias flag in the kernel. This means that
alias is printed regardless of whether the subnets match.

I found a local machine with a non-matching subnet on an interface.
After anonymizing, this is what it looks like:

le0: flags=8863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST> mtu 1500
address: 08:00:20:[redacted]
media: Ethernet autoselect (10baseT)
status: active
inet 10.0.1.1 netmask 0xffffff00 broadcast 10.0.1.255
inet alias 10.0.2.5 netmask 0xfffffff8 broadcast 10.0.2.7
inet6 fe80::[redacted]%le0 prefixlen 64 scopeid 0x1

This is a broken setup; the 10.0.2.0/29 block is for vhosts and there
is no reason to expect to find say .4 on the same ethernet; the point
of the scheme is to distribute all of those addresses to their
servers.
Really this address should be configured as a /32, and I think best
practices call for it to be on lo0 too. (This way only packets that
are routed to the box are answered, and we are very sure that it will
not be used for a source address of a new sendto()/connect().

Note that this is all for IPv4. On IPv6 the default behavior is to
add a new address, not replace. If you want to replace, it is delete
and add (or add and delete). This is because in IPv6 it is normal to
have more than one address, esp. since there is always a link-local
address.

So yes, this lacks the ability of Linux to configure primary and
secondary addresses in arbitrary order, and to then have source
selection use the primary one. It also doesn't auto-delete chained
addresses. In practice, people put the primary address first in
/etc/ifconfig.le0, and then the others, and the lack of this feature
does not cause pain that I have either noticed or heard about.

Also, if you want a 'virtual host' type address that isn't on the
subnet that your main interface is on, typically you put it

Does Linux have the same secondary notion for IPv6?

Greg Troxel wrote:

> Here is a trace of doing some alias configuration.

Thanks for that, demonstrates well what you've explained previsouly.


> So, an 'alias' in NetBSD (or *BSD?) is any address past the first
> address configured for an interface? If so, then it's a rougher scheme
> than Linux, which allows finer chains in the sense that only addresses
> implying the very same subnet are primary/secondary related.
>
> Right, except that I'd say simpler rather than rougher; there is
> simply no concept of related enforced by the kernel.

When I said rougher, I referred to the implication on source address
selection: it seems that a single address is used to access all
connected networks on a given interface. For instance, assume your
interface has 10.0.0.1/24 and 10.0.1.1/24 (alias), following this scheme
the former will be used to access neighbor 10.0.1.5/24, although the
natural choice, and the one taken in Linux kernels, would be 10.0.0.1/24.


> I am pretty sure that in case B the first address will be used for
> source selection.
>
> When you do 'ifconfig le0', the second address (and succeeding) are
> printed as alias, but that's because ifconfig has logic to get the
> first address and prints the word alias on all addresses that aren't
> the same. There is no alias flag in the kernel. This means that
> alias is printed regardless of whether the subnets match.

You're saying that 'alias' is just an ifconfig notation, but I doubt
that: assume a multihomed interface with 10.0.0.1/24, 10.0.1.1/24
(alias) and 10.0.2.1/24 (alias); what happens when you issue 'ifconfig
ifname 10.0.3.1/24'? According to your explanation, 10.0.3.1/24 would
replace 10.0.0.1/24 as the primary (non-alias) address, while keeping
the other two intact. However, if you claim that alias is just a user
syntactic sugar, how does ifconfig actually remove the old address and
install the new one such that it is the *first* in the list of IPv4
addresses? If I accept your saying that 'alias' has no meaning other
than "not first", then how can ifconfig set an address to be "first"
(meaning, non-alias)?


> This is a broken setup; the 10.0.2.0/29 block is for vhosts and there
> is no reason to expect to find say .4 on the same ethernet; the point
> of the scheme is to distribute all of those addresses to their
> servers.
> Really this address should be configured as a /32, and I think best
> practices call for it to be on lo0 too. (This way only packets that
> are routed to the box are answered, and we are very sure that it will
> not be used for a source address of a new sendto()/connect().

I'm afraid I didn't quite understand the moral of this example -- ?


> Note that this is all for IPv4. On IPv6 the default behavior is to
> add a new address, not replace. If you want to replace, it is delete
> and add (or add and delete). This is because in IPv6 it is normal to
> have more than one address, esp. since there is always a link-local
> address.

Or the other way round: in IPv4, it is legacy to have a single address,
and so legacy-derived schemes use all kind of tricks (like aliases)... ;->


> So yes, this lacks the ability of Linux to configure primary and
> secondary addresses in arbitrary order, and to then have source
> selection use the primary one. It also doesn't auto-delete chained
> addresses. In practice, people put the primary address first in
> /etc/ifconfig.le0, and then the others, and the lack of this feature
> does not cause pain that I have either noticed or heard about.

On the other hand, a simple extension to Linux's ioctl/netlink can imply
different fallback policy for a deleted primary address, like promoting
the first (FIFO order) secondary in the chain.


> Does Linux have the same secondary notion for IPv6?

Didn't ever check, however I believe it does (otherwise can't tell how
source address ambiguity problem is resolved).

Gilad

When I said rougher, I referred to the implication on source address
selection: it seems that a single address is used to access all
connected networks on a given interface. For instance, assume your
interface has 10.0.0.1/24 and 10.0.1.1/24 (alias), following this scheme
the former will be used to access neighbor 10.0.1.5/24, although the
natural choice, and the one taken in Linux kernels, would be 10.0.0.1/24.

Sorry, I wasn't thinking about that case. The source address is
indeed chosen as the first one matching the route that led to the
network.

e.g., I did

# add alias that is unrelated to the 'real' prefix
ifconfig ex0 192.168.100.210 alias

leading to that showing up second/alias on ifconfig, and then

ping 192.168.100.x

10:45:10.650652 arp who-has 192.168.100.x tell 192.168.100.210
10:45:10.651745 arp reply 192.168.100.x is-at [redacted]
10:45:11.659325 192.168.100.210 > 192.168.100.x: icmp: echo request
10:45:11.660388 192.168.100.x > 192.168.100.210: icmp: echo reply

ssh 192.168.100.x

10:46:51.654199 192.168.100.210.59867 > 192.168.100.x.22: S 642066818:642066818(0) win 16384 <mss 1460,nop,wscale 0,nop,nop,timestamp 12 0>
10:46:51.655281 192.168.100.x.22 > 192.168.100.210.59867: R 0:0(0) ack 642066819 win 0

You're saying that 'alias' is just an ifconfig notation, but I doubt
that: assume a multihomed interface with 10.0.0.1/24, 10.0.1.1/24
(alias) and 10.0.2.1/24 (alias); what happens when you issue 'ifconfig
ifname 10.0.3.1/24'? According to your explanation, 10.0.3.1/24 would
replace 10.0.0.1/24 as the primary (non-alias) address, while keeping
the other two intact. However, if you claim that alias is just a user
syntactic sugar, how does ifconfig actually remove the old address and
install the new one such that it is the *first* in the list of IPv4
addresses?

No, I've read the code. It really is in sbin/ifconfig.c.

I did the experiment you described, and got this:

inet 10.0.1.1 netmask 0xffffff00 broadcast 10.0.1.255
inet alias 10.0.2.1 netmask 0xffffff00 broadcast 10.0.2.255
inet alias 10.0.3.1 netmask 0xffffff00 broadcast 10.0.3.255

So it deleted the first one and then added one.
The new one showed up at the end.

This is arguably odd behavior.
But, either you do manual remove and add with 'alias', or you just
have one.

If I accept your saying that 'alias' has no meaning other
than "not first", then how can ifconfig set an address to be "first"
(meaning, non-alias)?

Well, it could use some mythical add-first ioctl, or remove all and
readd, or something. Right now it doesn't.

> This is a broken setup; the 10.0.2.0/29 block is for vhosts and there
> is no reason to expect to find say .4 on the same ethernet; the point
> of the scheme is to distribute all of those addresses to their
> servers.
> Really this address should be configured as a /32, and I think best
> practices call for it to be on lo0 too. (This way only packets that
> are routed to the box are answered, and we are very sure that it will
> not be used for a source address of a new sendto()/connect().

I'm afraid I didn't quite understand the moral of this example -- ?

With the (real) setup I described, attempts to connect to 10.0.2.4/29
may fail, since that address isn't necessarily on a host on that LAN,
since the setup is to assign the /29 to virtual use, assign individual
addressses (e.g. www.ir.bbn.com) to specific hosts, and then inject
/32 routes to the 'real' hosts for each virtual address.
So it should be set up not to assume that the other virtual addrs are
reachable directly, because they might not be.

Or the other way round: in IPv4, it is legacy to have a single address,
and so legacy-derived schemes use all kind of tricks (like aliases)... ;->

Sure. The natural thing would be to drop the 'alias' flag and the
delete-on-new-addr behavior of ifconfig.

On the other hand, a simple extension to Linux's ioctl/netlink can imply
different fallback policy for a deleted primary address, like promoting
the first (FIFO order) secondary in the chain.

Sounds fine, but there's always the 'does the kernel need to do this,
or can ifconfig' debate (that isn't useful to have).

Didn't ever check, however I believe it does (otherwise can't tell how
source address ambiguity problem is resolved).

BSD resolves it without flags. Look at the route that got us here,
find a matching prefix, and take the first one. Arguably this is less
flexible/configurable, but it basically works.

actually v6 source selection is much more complicated, since if you
have an interface with only link-local addrs (eg. gif in some cases,
ppp), locally-originated packets going over the ppp interface need to
get a global addr from someplace, and an addr (from another interface)
is chosen.

I've been distracted with some other things and I haven't been
keeping up with this thread carefully, so please bear with me
if I'm stating something that you guys have already figured out.

But, understanding the kernel's data structs and ioctls for bsd
will make the whole alias thing easier to understand. The kernel
has something pretty similar to zebra's data structs:


struct ifnet

|
`--> ifaddr --> ifaddr --> ifaddr etc.

the first ifaddr is usually the AF_LINK for the interface,
and then (assuming the interface has only ipv4) the ifaddr for the
primary ipv4 address, followed by the aliases.

If the user (e.g., ifconfig) does a SIOCSIFADDR, the primary ipv4 address
is added (modified, if it already exists). If ifconfig does a
SIOCAIFADDR, the aliases are added. The SIFADDR ioctl expects an
ifreq data, the AIFADDR expects an in_aliasreq struct

Does that straighten things out wrt *bsd? Or was I talking about
something else than what's being discussed? :-(

--Sowmini

Does that straighten things out wrt *bsd? Or was I talking about
something else than what's being discussed? :-(

Not redundant, and your diagram is quite helpful.

But on NetBSD 1.6.1, ifconfig always does SIOCAIFADDR, and it does a
SIOCDIFADDR first if the alias keyword is not given.
But I think the SIOCSIFADDR ioctl works as you describe.

On Thu, 15 Jan 2004, Greg Troxel wrote:

> Does Linux have the same secondary notion for IPv6?

Doesnt appear to:

# ip -6 ad sho dev eth0
7: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
inet6 fe80::260:97ff:fe54:1ec9/64 scope link
inet6 2001:770:105:1:260:97ff:fe54:1ec9/64 scope global
# ip -6 addr add 2001:770:105:1::22/64 dev eth0
# ip -6 ad sho dev eth0
7: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
inet6 2001:770:105:1::22/64 scope global tentative
inet6 fe80::260:97ff:fe54:1ec9/64 scope link
inet6 2001:770:105:1:260:97ff:fe54:1ec9/64 scope global

that's Fedora 2.4.22-1.2140.nptl kernel, 2.6.0 also does not set the
secondary flag.

--paulj