Mailing List Archive

Strange...

The interface handling works as expected when I tried ife.
So, it's looking like the problem is, as I thought earlier, in
Release( entry *VE ) function.

My C programing skill is really rustic, so, ehm.., I might need some
help debuging it.

Let's see:

static void Release( entry *VE )
{
int ic, n=0;
struct interface idown, *nif;

for(nif=&VE->pseudo_if; n<MAX_DEP_IF; nif = &(VE->extra_ifs[n++])) {
if(nif->ipaddr.s_addr == 0) break;


/*
The releasing of the interfaces happen in a loop.
Is it intentionally written to drop all interfaces?
It's looking like it's going through all interfaces and dropping them
one by one util it gets to the last interface or until it gets to the
interface with ip_addr = 0.
Why doesn't it drop just one?
*/



memcpy(&idown, nif, sizeof(struct interface));
idown.ipaddr.s_addr = htonl(idown.ipaddr.s_addr);
idown.netmask.s_addr = htonl(idown.netmask.s_addr);
idown.bcast.s_addr = htonl(idown.bcast.s_addr);
ic = if_down(&idown);
if(ic) {
const char *em = if_error();
if(em && strlen(em)) {
Alarm(PRINT, "%s", if_error());
}
} else {
char buffer[16];
snprintf(buffer, 16, inet_ntoa(idown.ipaddr));
Alarm(PRINT, "DOWN: %s:%s/%s",
idown.ifname,buffer,inet_ntoa(idown.netmask));
}
}
My.num_allocated--;
}




On Fri, 2003-06-06 at 23:03, Theo Schlossnagle wrote:
>
> On Friday, Jun 6, 2003, at 16:08 US/Eastern, Ezra Nugroho wrote:
> > At that time wackamole at first declares that
> > "INTERFACE ETH0:1 64.255.110.80 DOWN"
> >
> > All the above makes sense, wackamole has balanced the VIP assignments
> > accross two servers. However, apparently when wackamole brought eth0:1
> > on first, it accidentally took eth0:2 down without telling spread.
> > Wackamole things that all VIP are acquired, but they are not!
> >
> > Could it be a bug in the Release( entry *VE ) function, particularly
> > for
> > RHN 7.3?
>
> I've seen that problem on Linux before. It has to do with Linux's
> screwy interface system.

> snipped...

Ezra Nugroho wrote:
> Strange...
>
> The interface handling works as expected when I tried ife.
> So, it's looking like the problem is, as I thought earlier, in
> Release( entry *VE ) function.
>
> My C programing skill is really rustic, so, ehm.., I might need some
> help debuging it.
> Let's see:
>
> static void Release( entry *VE )
> {
> int ic, n=0;
> struct interface idown, *nif;
>
> for(nif=&VE->pseudo_if; n<MAX_DEP_IF; nif = &(VE->extra_ifs[n++])) {
> if(nif->ipaddr.s_addr == 0) break;


It is supposed to drop all the interfaces in the VE. VE != VIP. VE is a set
of virtual IP addresses. You should have two VEs, each with on VIP specified.

You should only have one VIP in each of your VEs using your configuration.

wackamole is designed to work on routers too:

Prefer None
VirtualInterfaces {
{ fxp2:10.77.52.1/32 fxp1:192.168.52.1/32 fxp0:192.168.0.1/32 }
}

The above conf shows a sine VE with one psuedo_if and two extra_ifs. If the
VE drops, it need to drop all three of those IPs and if the VE is "brought up"
it needs to bring all three IPs up. (note the extra set of braces).

--
Theo Schlossnagle
Principal Consultant
OmniTI Computer Consulting, Inc. -- http://www.omniti.com/
Phone: +1 410 872 4910 x201 Fax: +1 410 872 4911
1024D/82844984/95FD 30F1 489E 4613 F22E 491A 7E88 364C 8284 4984
2047R/33131B65/71 F7 95 64 49 76 5D BA 3D 90 B9 9F BE 27 24 E7

Thanks for the quick response.

I quickly found out that that was not the problem.
I have also verified that there was no bug in the main code leading to
the call of Release ( entry *VE)

Interface handling in ife.c works, but in wackamole.c doesn't.
The only thing I can think of right now is the interface presentations
that are passed to if_down are different.

req that is passed to if_down in ife.c is much simpler than ifdown in
wackamole.c. Remeber ife call for releasing interface is only 'ife -d
ip_address', no netmask, no broadcast.

Is there any C function call to do variable dump? I am not familiar with
gdb yet...
Is there something you can pass my way about 'struct interface'.
I don't know what the structure is, and I don't know where I can find
it.

I am darn close. I'll get there with a little nudge :).





On Mon, 2003-06-09 at 14:37, Theo E. Schlossnagle wrote:
> Ezra Nugroho wrote:
> > Strange...
> >
> > The interface handling works as expected when I tried ife.
> > So, it's looking like the problem is, as I thought earlier, in
> > Release( entry *VE ) function.
> >
> > My C programing skill is really rustic, so, ehm.., I might need some
> > help debuging it.
> > Let's see:
> >
> > static void Release( entry *VE )
> > {
> > int ic, n=0;
> > struct interface idown, *nif;
> >
> > for(nif=&VE->pseudo_if; n<MAX_DEP_IF; nif = &(VE->extra_ifs[n++])) {
> > if(nif->ipaddr.s_addr == 0) break;
>
>
> It is supposed to drop all the interfaces in the VE. VE != VIP. VE is a set
> of virtual IP addresses. You should have two VEs, each with on VIP specified.
>
> You should only have one VIP in each of your VEs using your configuration.
>
> wackamole is designed to work on routers too:
>
> Prefer None
> VirtualInterfaces {
> { fxp2:10.77.52.1/32 fxp1:192.168.52.1/32 fxp0:192.168.0.1/32 }
> }
>
> The above conf shows a sine VE with one psuedo_if and two extra_ifs. If the
> VE drops, it need to drop all three of those IPs and if the VE is "brought up"
> it needs to bring all three IPs up. (note the extra set of braces).
>
> --
> Theo Schlossnagle
> Principal Consultant
> OmniTI Computer Consulting, Inc. -- http://www.omniti.com/
> Phone: +1 410 872 4910 x201 Fax: +1 410 872 4911
> 1024D/82844984/95FD 30F1 489E 4613 F22E 491A 7E88 364C 8284 4984
> 2047R/33131B65/71 F7 95 64 49 76 5D BA 3D 90 B9 9F BE 27 24 E7

Theo and others,


Here is what I concluded after hours of playing arround with the code.

As you said earlier, the ioctl call seems to be somewhat screwy.
However it works perfectly from ife.
I noticed that interface that was passed to if_down from ife is much
simpler, it only has the ip address.
I have changed the interface passed by wackamole to if_down to be as
simple as the one in ife, but it didn't fix it.

ioctl seems to require only the interface name to pass flags to in:

(ife-sockpacket.c)
strcpy(ifr.ifr_name, ifs[i].ifname);
ifr.ifr_flags &= ~(IFF_RUNNING | IFF_UP);
ioctl(_if_sock, SIOCSIFFLAGS, (char *)&ifr)

There must be something with the calls above in wackamole that breaks
it. Maybe it's a bug in ioctl if it is called by a daemon. Maybe
something else...
I might try implementing wackamole on a different linux with different
glibc to see if that solves it.

Has anyone ever gotten wackamole to work with more than 1 VIP in linux?
Please tell me what distro, what kernel and what glibc.
I would appreciate it much.


I saw another implementation of if_down in ife-bpf.c file.
The ioctl call is different:
ioctl(_if_sock, SIOCDIFADDR, &toup)

Is it not working? What was it (originally) for?


Now, the simple (,somewhat stuppid) solution to this problem could be
simply acquiring ips that was accidentally lost.
So wackamole on the server to get up first would work like this:

Acquire all ips
if other boxes to come up
release ips that are requested by other box
(accidentally give up too many)
compare actual interfaces with what the box should have retain.
acquire those lost ips.

Is that acceptable?
I don't like it too much because some IPs are still lost for some time.
However that's better than nothing.


Thanks,