Mailing List Archive

On 20 Mar 2009, at 17:02, Theo Schlossnagle wrote:
>
>>> b) wackamole doesn't seem to do ARP announcements for more than 2 IP
>>> addresses in a group { eth0:ip1 eth0:ip2 eth0:ip3 ... }. Doing
>>> tcpdump I can see only two of IPs being announced on the new MAC.
>
> I must have missed this message originally. I haven't seen that
> behaviour, but also have not looked for it. The code that does this
> stuff is pretty isolated, so it should be easy to track down if that
> is indeed a repeatable behaviour.
>
>>> Are these known bugs? Any clues on what needs attention here?
>
>
> Nope. Nope. But, I'll be happy to play the role of remote debugger
> for you.

Thanks! what would you suggest I look at (code or diagnostics) and
what would
you like to see?

Kind Regards,
Mark Blackman
Exonetric

>
>
> --
> Theo Schlossnagle
> Esoteric Curio -- http://lethargy.org/
> OmniTI Computer Consulting, Inc. -- http://omniti.com/
>

_______________________________________________
wackamole-users mailing list
wackamole-users@lists.backhand.org
http://lists.backhand.org/mailman/listinfo/wackamole-users

Thanks. During start-up I agree it's doing the right thing. I'm not so
sure
I agree it does the right thing when a 'wackatrl -f' is issued on one
of the machines.

That sequence includes a wackatrl -f / wackatrl -s (on s2 I think).
Does your reading
indicated the failover case results in the "right" grat. arp packets?

I'll reread my playback to spell out what I believe is missing.

- Mark

On 23 Mar 2009, at 15:50, Theo Schlossnagle wrote:

> Looks right to me. You are grat-arping from every IP to every IP on
> your network...
>
> 82.138.255.16/28
> 82.138.254.16/28
>
> There are 16 address - 2 (net/bcast) ... so 14 people on each
> network that need to be notified that the IPs are owned by new
> machines. You have 4 IPs configured on each box.... 3 are on one
> subnet and 1 is on the other... So, you should see 3*14 + 1*14
> gratuitous arp responses from each machine. Your tcpdump looks
> fine. If you look inside the payload, you'll see all the
> combinations represented in the source and target addresses.
>
>
> On Mar 22, 2009, at 6:53 PM, Mark Blackman wrote:
>
>> Ok, i've attached the correct config, wackamole debug output and
>> tcpdump of arp-only
>> packets.
>>
>> The original config was the result of paring down the 4 IPs per
>> group to a single IP
>> as a test.
>>
>> - Mark
>>
>> <wackamole-diag-correct.tgz>
>>
>>
>> On 22 Mar 2009, at 22:25, Mark Blackman wrote:
>>
>>>
>>> On 20 Mar 2009, at 17:18, Theo Schlossnagle wrote:
>>>
>>>> config file, and an arp-only tcpdump on every interface for the
>>>> same time period and the relevant syslog output.
>>>>
>>>
>>> Attached as requested. Sent only to yourself as it include client
>>> set-up details, but feel
>>> free to respond on the list.
>>>
>>> Two machines, s1 and s2 both on the following 3 subnets
>>>
>>> 10.10.10.0/24
>>> 82.138.255.16/28
>>> 82.138.254.16/28
>>>
>>> the spread daemon is configured to use the 10.10.10.0/24 subnet IPs
>>>
>>> the machines take responsibility for two groups of 4 IPs as
>>> the configuration file should indicate. With luck I've merely
>>> screwed up the configuration.
>>>
>>> I've attached the tcpdump output (arp) from each machine, suitable
>>> for tcpdump -r playback, both configuration files (hopefully
>>> identical)
>>> and finally output from both wackamole daemons with '-d' specified.
>>>
>>> I ran wacktrl -f / wackatrl -s sequence while the daemons were
>>> running
>>> as an example.
>>>
>>> Cheers,
>>> Mark Blackman
>>> Exonetric
>>>
>>>
>>>
>>> <wackamole-diag.tgz>
>>>
>>>
>>>
>>
>
> --
> Theo Schlossnagle
> Principal/CEO
> OmniTI Computer Consulting, Inc.
> Web Applications & Internet Architectures
> w: http://omniti.com p: +1.443.325.1357 x201 f: +1.410.872.4911
>
>
>
>

_______________________________________________
wackamole-users mailing list
wackamole-users@lists.backhand.org
http://lists.backhand.org/mailman/listinfo/wackamole-users

-----------------------------------------------------------
First, regarding the duplication issue :

tcpdump -e -r s1-tcpdump-arp.dat -n | fgrep 82.138.254.24

shows that the IP 82.138.254.24 associated with the "first" group of
IPs,
gets sent in an grat. arp packet 32 times within a period of as
follows.

22:46:52.656125 00:e0:81:41:3e:4c > ff:ff:ff:ff:ff:ff, ethertype ARP
(0x0806), length 60: arp reply 82.138.254.24 is-at 00:e0:81:41:3e:4c

i.e. same originating MAC and always to the broadcast address (if my
reading is correct).
That doesn't seem right to me.
-----------------------------------------------------------

Second, regarding the missing grat. arp packets:

I've filtered out the 'arp reply' packets and then just selected the
source MAC,
dst MAC and the announced IP. I also took out the IP addresses not
being managed
by wackamole itself. Those totals aren't what I would expect if all
members of a
VIF group are treated equally.


I "failed" and "re-enabled" the s2 node with 'wackatrl -f/s' once or
twice during the collection
period, so I'd expect the second group of 4 IPs to show up the most as
they flip
back and forth between hosts. MAC ending '4c' represents host s1 and
MAC ending
'96' represents host s2.

It's possible I didn't wait long enough between disable and re-enable
to see
the 3rd and 4th IPs to be re-announced though. So I need to redo this
and wait
longer between the wackatrl state changes.

for the second VIF group:

82.138.255.18: host s1 sent 96 identical grat. arp packets announcing
itself for that IP (broadcast)
82.138.254.25: host s1 sent 32 identical grat. arp packets announcing
itself for that IP (broadcast)
82.138.255.25: host s1 sent 0 grat. arp packets
82.138.254.27: host s1 sent 1 grat. arp packets announcing itself for
that IP (unicast, to router I think)

for the first VIF group:

82.138.255.17: host s1 sent 64 identical grat. arp packets announcing
itself for that IP (broadcast)
82.138.254.24: host s1 sent 32 identical grat. arp packets announcing
itself for that IP (broadcast)
82.138.255.24: host s1 sent 0 grat. arp packets
82.138.254.26: host s1 sent 0 grat. arp packets.


markimac% tcpdump -e -r s1-tcpdump-arp.dat -n | awk '/arp reply/
{print $2 "-- " $4 " -- " $12}' | sort -n | uniq -c | sort -k 5 -nr
reading from file s1-tcpdump-arp.dat, link-type EN10MB (Ethernet)
96 00:e0:81:41:3e:4c-- ff:ff:ff:ff:ff:ff, -- 82.138.255.18
64 00:e0:81:41:3e:96-- ff:ff:ff:ff:ff:ff, -- 82.138.255.17
64 00:e0:81:41:3e:4c-- ff:ff:ff:ff:ff:ff, -- 82.138.255.17
32 00:e0:81:41:3e:96-- ff:ff:ff:ff:ff:ff, -- 82.138.255.18
32 00:e0:81:41:3e:96-- ff:ff:ff:ff:ff:ff, -- 82.138.254.25
32 00:e0:81:41:3e:4c-- ff:ff:ff:ff:ff:ff, -- 82.138.254.24
1 00:e0:81:41:3e:4c-- 00:02:17:e0:40:1c, -- 82.138.254.27

markimac% tcpdump -e -r s2-tcpdump-arp.dat -n | awk '/arp reply/
{print $2 "-- " $4 " -- " $12}' | sort -n | uniq -c | sort -k 5 -nr
reading from file s2-tcpdump-arp.dat, link-type EN10MB (Ethernet)
96 00:e0:81:41:3e:4c-- ff:ff:ff:ff:ff:ff, -- 82.138.255.18
64 00:e0:81:41:3e:96-- ff:ff:ff:ff:ff:ff, -- 82.138.255.18
64 00:e0:81:41:3e:96-- ff:ff:ff:ff:ff:ff, -- 82.138.255.17
64 00:e0:81:41:3e:4c-- ff:ff:ff:ff:ff:ff, -- 82.138.255.17
32 00:e0:81:41:3e:96-- ff:ff:ff:ff:ff:ff, -- 82.138.254.25
32 00:e0:81:41:3e:4c-- ff:ff:ff:ff:ff:ff, -- 82.138.254.24

Regards,
Mark
On 23 Mar 2009, at 15:50, Theo Schlossnagle wrote:

> Looks right to me. You are grat-arping from every IP to every IP on
> your network...
>
> 82.138.255.16/28
> 82.138.254.16/28
>
> There are 16 address - 2 (net/bcast) ... so 14 people on each
> network that need to be notified that the IPs are owned by new
> machines. You have 4 IPs configured on each box.... 3 are on one
> subnet and 1 is on the other... So, you should see 3*14 + 1*14
> gratuitous arp responses from each machine. Your tcpdump looks
> fine. If you look inside the payload, you'll see all the
> combinations represented in the source and target addresses.
>
>
> On Mar 22, 2009, at 6:53 PM, Mark Blackman wrote:
>
>> Ok, i've attached the correct config, wackamole debug output and
>> tcpdump of arp-only
>> packets.
>>
>> The original config was the result of paring down the 4 IPs per
>> group to a single IP
>> as a test.
>>
>> - Mark
>>
>> <wackamole-diag-correct.tgz>
>>
>>
>> On 22 Mar 2009, at 22:25, Mark Blackman wrote:
>>
>>>
>>> On 20 Mar 2009, at 17:18, Theo Schlossnagle wrote:
>>>
>>>> config file, and an arp-only tcpdump on every interface for the
>>>> same time period and the relevant syslog output.
>>>>
>>>
>>> Attached as requested. Sent only to yourself as it include client
>>> set-up details, but feel
>>> free to respond on the list.
>>>
>>> Two machines, s1 and s2 both on the following 3 subnets
>>>
>>> 10.10.10.0/24
>>> 82.138.255.16/28
>>> 82.138.254.16/28
>>>
>>> the spread daemon is configured to use the 10.10.10.0/24 subnet IPs
>>>
>>> the machines take responsibility for two groups of 4 IPs as
>>> the configuration file should indicate. With luck I've merely
>>> screwed up the configuration.
>>>
>>> I've attached the tcpdump output (arp) from each machine, suitable
>>> for tcpdump -r playback, both configuration files (hopefully
>>> identical)
>>> and finally output from both wackamole daemons with '-d' specified.
>>>
>>> I ran wacktrl -f / wackatrl -s sequence while the daemons were
>>> running
>>> as an example.
>>>
>>> Cheers,
>>> Mark Blackman
>>> Exonetric
>>>
>>>
>>>
>>> <wackamole-diag.tgz>
>>>
>>>
>>>
>>
>
> --
> Theo Schlossnagle
> Principal/CEO
> OmniTI Computer Consulting, Inc.
> Web Applications & Internet Architectures
> w: http://omniti.com p: +1.443.325.1357 x201 f: +1.410.872.4911
>
>
>
>

_______________________________________________
wackamole-users mailing list
wackamole-users@lists.backhand.org
http://lists.backhand.org/mailman/listinfo/wackamole-users

On 20 Mar 2009, at 10:24, Mark Blackman wrote:

> Hi,
>
> I'm writing to report that
>
> a) wackamole failover events seem to lead to an excess of identical
> ARP announcements (using wackamole 2.1.4)

This seems to be the usual case for subnet notifications when the
destination MAC address is unknown and the broadcast MAC is used
for each IP notification, possibly redundant, but deliberate anyway.

>
>
> b) wackamole doesn't seem to do ARP announcements for more than 2 IP
> addresses in a group { eth0:ip1 eth0:ip2 eth0:ip3 ... }. Doing
> tcpdump I can see only two of IPs being announced on the new MAC.

To me, this appears to be logic bug where the current (2.1.4) code
only does as many IPs as it has 'Notify' entries (completely orthogonal
and no relationship to number of IPs), so I've patched the code against
2.1.4 and wackamole seems to now do grat. arp replies for every IP
in the relevant VirtualInterfaces grouping. One other person reports
that this does the right thing for him as well, so I've attached the
patchset.

It merely moves the "completed notifications" counter into the per-
managed-IP
structure so that the correct orthogonality is preserved. In any case,
would be interested to know what anyone else finds. The suggested
patching
sequence for those unfamiliar with 'patch' would be something like.

tar -zxvpf wackamole-2.1.4.tar.gz
cd wackamole-2.1.4
patch < ../wackamole-2.1.4-patches.txt

Kind Regards,
Mark Blackman
Exonetric