Mailing List Archive

[lvs-users] IPVS - Piranha - Syncdaemon
Hi,

I hope this is the right place for this question because I tried other
places and no response from people so this is my last resort I guess.

I'm looking for someone with experience with Syncdaemon IPVS and the
piranha package.
I'm trying to build a loadbalancer with session information synchronized
between the hosts.

The problem is that I don't see the session information on the backup node
(server-b) with the command ipvsadm -Lcn

The config:
--- LVS.CF ---
serial_no = 19
primary = <IP server-a>
service = lvs
backup_active = 1
backup = <IP server-b>
heartbeat = 1
heartbeat_port = 539
keepalive = 6
deadtime = 18
network = direct
debug_level = NONE
monitor_links = 1
syncdaemon = 1
syncd_iface = eth0
syncd_id = 1
virtual http {
active = 1
address = <VIP_IP> eth0:1
vip_nmask = 255.255.255.255
port = 80
send = "GET / HTTP/1.0\r\n\r\n"
expect = "HTTP"
use_regex = 0
load_monitor = none
scheduler = rr
protocol = tcp
timeout = 6
reentry = 15
quiesce_server = 0
persistent = 120
server server-a {
address = <IP server-a>
active = 1
weight = 1
}
server server-b {
address = <IP server-b>
active = 1
weight = 1
}
}

--- LVS.CF ---

On the backup node i see with netstat -tulpen:
udp 0 0 224.0.0.81:8848 0.0.0.0:*

>From the documentation I know it's multicast udp.

A TCP dump shows me that server-a is sending packages:

[root@server-a ~]# tcpdump -ni eth0 host 224.0.0.81
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
16:24:29.726138 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
16:24:35.726112 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
16:24:36.726189 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
16:24:37.726128 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100

[root@server-b ~]# tcpdump -ni eth0 host 224.0.0.81
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
16:24:30.177808 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
16:24:36.177817 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
16:24:37.177847 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
16:24:38.177760 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100

Server-b get the traffic on it's interface but does nothing with it.
ipvsadm -Lcn on server-a show a list with information but on server-b
nothing but documentation states that it should and there is a kernel
thread running called [ipvs_syncbackup].

A strace of the pulse process on server-b shows only the polling of server-a.

So what am I missing here? I hope somebody has a clue.

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
Re: [lvs-users] IPVS - Piranha - Syncdaemon [ In reply to ]
On Mon, Mar 04, 2013 at 02:59:56PM +0100, maillist@toerental.nl wrote:
> Hi,
>
> I hope this is the right place for this question because I tried other
> places and no response from people so this is my last resort I guess.
>
> I'm looking for someone with experience with Syncdaemon IPVS and the
> piranha package.
> I'm trying to build a loadbalancer with session information synchronized
> between the hosts.
>
> The problem is that I don't see the session information on the backup node
> (server-b) with the command ipvsadm -Lcn
>
> The config:
> --- LVS.CF ---
> serial_no = 19
> primary = <IP server-a>
> service = lvs
> backup_active = 1
> backup = <IP server-b>
> heartbeat = 1
> heartbeat_port = 539
> keepalive = 6
> deadtime = 18
> network = direct
> debug_level = NONE
> monitor_links = 1
> syncdaemon = 1
> syncd_iface = eth0
> syncd_id = 1
> virtual http {
> active = 1
> address = <VIP_IP> eth0:1
> vip_nmask = 255.255.255.255
> port = 80
> send = "GET / HTTP/1.0\r\n\r\n"
> expect = "HTTP"
> use_regex = 0
> load_monitor = none
> scheduler = rr
> protocol = tcp
> timeout = 6
> reentry = 15
> quiesce_server = 0
> persistent = 120
> server server-a {
> address = <IP server-a>
> active = 1
> weight = 1
> }
> server server-b {
> address = <IP server-b>
> active = 1
> weight = 1
> }
> }
>
> --- LVS.CF ---
>
> On the backup node i see with netstat -tulpen:
> udp 0 0 224.0.0.81:8848 0.0.0.0:*
>
> >From the documentation I know it's multicast udp.
>
> A TCP dump shows me that server-a is sending packages:
>
> [root@server-a ~]# tcpdump -ni eth0 host 224.0.0.81
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
> 16:24:29.726138 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
> 16:24:35.726112 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
> 16:24:36.726189 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
> 16:24:37.726128 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
>
> [root@server-b ~]# tcpdump -ni eth0 host 224.0.0.81
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
> 16:24:30.177808 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
> 16:24:36.177817 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
> 16:24:37.177847 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
> 16:24:38.177760 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
>
> Server-b get the traffic on it's interface but does nothing with it.
> ipvsadm -Lcn on server-a show a list with information but on server-b
> nothing but documentation states that it should and there is a kernel
> thread running called [ipvs_syncbackup].
>
> A strace of the pulse process on server-b shows only the polling of server-a.
>
> So what am I missing here? I hope somebody has a clue.

What version of piranha are you using? Also, please run 'ivpsadm
--list --daemon' on each director to verify the sync daemon is running
as expected.

Ryan

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
Re: [lvs-users] IPVS - Piranha - Syncdaemon [ In reply to ]
> On Mon, Mar 04, 2013 at 02:59:56PM +0100, maillist@toerental.nl wrote:
>> Hi,
>>
>> I hope this is the right place for this question because I tried other
>> places and no response from people so this is my last resort I guess.
>>
>> I'm looking for someone with experience with Syncdaemon IPVS and the
>> piranha package.
>> I'm trying to build a loadbalancer with session information synchronized
>> between the hosts.
>>
>> The problem is that I don't see the session information on the backup
>> node
>> (server-b) with the command ipvsadm -Lcn
>>
>> The config:
>> --- LVS.CF ---
>> serial_no = 19
>> primary = <IP server-a>
>> service = lvs
>> backup_active = 1
>> backup = <IP server-b>
>> heartbeat = 1
>> heartbeat_port = 539
>> keepalive = 6
>> deadtime = 18
>> network = direct
>> debug_level = NONE
>> monitor_links = 1
>> syncdaemon = 1
>> syncd_iface = eth0
>> syncd_id = 1
>> virtual http {
>> active = 1
>> address = <VIP_IP> eth0:1
>> vip_nmask = 255.255.255.255
>> port = 80
>> send = "GET / HTTP/1.0\r\n\r\n"
>> expect = "HTTP"
>> use_regex = 0
>> load_monitor = none
>> scheduler = rr
>> protocol = tcp
>> timeout = 6
>> reentry = 15
>> quiesce_server = 0
>> persistent = 120
>> server server-a {
>> address = <IP server-a>
>> active = 1
>> weight = 1
>> }
>> server server-b {
>> address = <IP server-b>
>> active = 1
>> weight = 1
>> }
>> }
>>
>> --- LVS.CF ---
>>
>> On the backup node i see with netstat -tulpen:
>> udp 0 0 224.0.0.81:8848 0.0.0.0:*
>>
>> >From the documentation I know it's multicast udp.
>>
>> A TCP dump shows me that server-a is sending packages:
>>
>> [root@server-a ~]# tcpdump -ni eth0 host 224.0.0.81
>> tcpdump: verbose output suppressed, use -v or -vv for full protocol
>> decode
>> listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
>> 16:24:29.726138 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
>> 16:24:35.726112 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
>> 16:24:36.726189 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
>> 16:24:37.726128 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
>>
>> [root@server-b ~]# tcpdump -ni eth0 host 224.0.0.81
>> tcpdump: verbose output suppressed, use -v or -vv for full protocol
>> decode
>> listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
>> 16:24:30.177808 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
>> 16:24:36.177817 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
>> 16:24:37.177847 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
>> 16:24:38.177760 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
>>
>> Server-b get the traffic on it's interface but does nothing with it.
>> ipvsadm -Lcn on server-a show a list with information but on server-b
>> nothing but documentation states that it should and there is a kernel
>> thread running called [ipvs_syncbackup].
>>
>> A strace of the pulse process on server-b shows only the polling of
>> server-a.
>>
>> So what am I missing here? I hope somebody has a clue.
>
> What version of piranha are you using? Also, please run 'ivpsadm
> --list --daemon' on each director to verify the sync daemon is running
> as expected.
>
> Ryan
>

The version of piranha is: piranha-0.8.5-19.el6.x86_64

The command "ipvsadm --list --daemon" does not return anything?

Does I mis a running daemon? Which daemon would that be?



_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
Re: [lvs-users] IPVS - Piranha - Syncdaemon [ In reply to ]
On Tue, Mar 05, 2013 at 05:11:30PM +0100, maillist@toerental.nl wrote:
> > On Mon, Mar 04, 2013 at 02:59:56PM +0100, maillist@toerental.nl wrote:
> >> Hi,
> >>
> >> I hope this is the right place for this question because I tried other
> >> places and no response from people so this is my last resort I guess.
> >>
> >> I'm looking for someone with experience with Syncdaemon IPVS and the
> >> piranha package.
> >> I'm trying to build a loadbalancer with session information synchronized
> >> between the hosts.
> >>
> >> The problem is that I don't see the session information on the backup
> >> node
> >> (server-b) with the command ipvsadm -Lcn
> >>
> >> The config:
> >> --- LVS.CF ---
> >> serial_no = 19
> >> primary = <IP server-a>
> >> service = lvs
> >> backup_active = 1
> >> backup = <IP server-b>
> >> heartbeat = 1
> >> heartbeat_port = 539
> >> keepalive = 6
> >> deadtime = 18
> >> network = direct
> >> debug_level = NONE
> >> monitor_links = 1
> >> syncdaemon = 1
> >> syncd_iface = eth0
> >> syncd_id = 1
> >> virtual http {
> >> active = 1
> >> address = <VIP_IP> eth0:1
> >> vip_nmask = 255.255.255.255
> >> port = 80
> >> send = "GET / HTTP/1.0\r\n\r\n"
> >> expect = "HTTP"
> >> use_regex = 0
> >> load_monitor = none
> >> scheduler = rr
> >> protocol = tcp
> >> timeout = 6
> >> reentry = 15
> >> quiesce_server = 0
> >> persistent = 120
> >> server server-a {
> >> address = <IP server-a>
> >> active = 1
> >> weight = 1
> >> }
> >> server server-b {
> >> address = <IP server-b>
> >> active = 1
> >> weight = 1
> >> }
> >> }
> >>
> >> --- LVS.CF ---
> >>
> >> On the backup node i see with netstat -tulpen:
> >> udp 0 0 224.0.0.81:8848 0.0.0.0:*
> >>
> >> >From the documentation I know it's multicast udp.
> >>
> >> A TCP dump shows me that server-a is sending packages:
> >>
> >> [root@server-a ~]# tcpdump -ni eth0 host 224.0.0.81
> >> tcpdump: verbose output suppressed, use -v or -vv for full protocol
> >> decode
> >> listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
> >> 16:24:29.726138 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
> >> 16:24:35.726112 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
> >> 16:24:36.726189 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
> >> 16:24:37.726128 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
> >>
> >> [root@server-b ~]# tcpdump -ni eth0 host 224.0.0.81
> >> tcpdump: verbose output suppressed, use -v or -vv for full protocol
> >> decode
> >> listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
> >> 16:24:30.177808 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
> >> 16:24:36.177817 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
> >> 16:24:37.177847 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
> >> 16:24:38.177760 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
> >>
> >> Server-b get the traffic on it's interface but does nothing with it.
> >> ipvsadm -Lcn on server-a show a list with information but on server-b
> >> nothing but documentation states that it should and there is a kernel
> >> thread running called [ipvs_syncbackup].
> >>
> >> A strace of the pulse process on server-b shows only the polling of
> >> server-a.
> >>
> >> So what am I missing here? I hope somebody has a clue.
> >
> > What version of piranha are you using? Also, please run 'ivpsadm
> > --list --daemon' on each director to verify the sync daemon is running
> > as expected.
> >
> > Ryan
> >
>
> The version of piranha is: piranha-0.8.5-19.el6.x86_64

This version should work, but I have fixed a few bugs in piranha
related to the sync daemon. I'd update if possible.

> The command "ipvsadm --list --daemon" does not return anything?
>
> Does I mis a running daemon? Which daemon would that be?

You might be hitting a bug in ipvsadm. I reported a bug where 'ipvsadm
--list --daemon' where it would not show a running sync daemon in
backup state. I'm not sure if it was every fixed upstream, but I did
fix in the RHEL. Check your version of ipvsadm. The
ipvsadm-1.25-10.el6 package should have the fix.

Ryan


_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
Re: [lvs-users] IPVS - Piranha - Syncdaemon [ In reply to ]
> On Tue, Mar 05, 2013 at 05:11:30PM +0100, maillist@toerental.nl wrote:
>> > On Mon, Mar 04, 2013 at 02:59:56PM +0100, maillist@toerental.nl wrote:
>> >> Hi,
>> >>
>> >> I hope this is the right place for this question because I tried
>> other
>> >> places and no response from people so this is my last resort I guess.
>> >>
>> >> I'm looking for someone with experience with Syncdaemon IPVS and the
>> >> piranha package.
>> >> I'm trying to build a loadbalancer with session information
>> synchronized
>> >> between the hosts.
>> >>
>> >> The problem is that I don't see the session information on the backup
>> >> node
>> >> (server-b) with the command ipvsadm -Lcn
>> >>
>> >> The config:
>> >> --- LVS.CF ---
>> >> serial_no = 19
>> >> primary = <IP server-a>
>> >> service = lvs
>> >> backup_active = 1
>> >> backup = <IP server-b>
>> >> heartbeat = 1
>> >> heartbeat_port = 539
>> >> keepalive = 6
>> >> deadtime = 18
>> >> network = direct
>> >> debug_level = NONE
>> >> monitor_links = 1
>> >> syncdaemon = 1
>> >> syncd_iface = eth0
>> >> syncd_id = 1
>> >> virtual http {
>> >> active = 1
>> >> address = <VIP_IP> eth0:1
>> >> vip_nmask = 255.255.255.255
>> >> port = 80
>> >> send = "GET / HTTP/1.0\r\n\r\n"
>> >> expect = "HTTP"
>> >> use_regex = 0
>> >> load_monitor = none
>> >> scheduler = rr
>> >> protocol = tcp
>> >> timeout = 6
>> >> reentry = 15
>> >> quiesce_server = 0
>> >> persistent = 120
>> >> server server-a {
>> >> address = <IP server-a>
>> >> active = 1
>> >> weight = 1
>> >> }
>> >> server server-b {
>> >> address = <IP server-b>
>> >> active = 1
>> >> weight = 1
>> >> }
>> >> }
>> >>
>> >> --- LVS.CF ---
>> >>
>> >> On the backup node i see with netstat -tulpen:
>> >> udp 0 0 224.0.0.81:8848 0.0.0.0:*
>> >>
>> >> >From the documentation I know it's multicast udp.
>> >>
>> >> A TCP dump shows me that server-a is sending packages:
>> >>
>> >> [root@server-a ~]# tcpdump -ni eth0 host 224.0.0.81
>> >> tcpdump: verbose output suppressed, use -v or -vv for full protocol
>> >> decode
>> >> listening on eth0, link-type EN10MB (Ethernet), capture size 65535
>> bytes
>> >> 16:24:29.726138 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
>> >> 16:24:35.726112 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
>> >> 16:24:36.726189 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
>> >> 16:24:37.726128 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
>> >>
>> >> [root@server-b ~]# tcpdump -ni eth0 host 224.0.0.81
>> >> tcpdump: verbose output suppressed, use -v or -vv for full protocol
>> >> decode
>> >> listening on eth0, link-type EN10MB (Ethernet), capture size 65535
>> bytes
>> >> 16:24:30.177808 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
>> >> 16:24:36.177817 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
>> >> 16:24:37.177847 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
>> >> 16:24:38.177760 IP server-a.54366 > 224.0.0.81.8848: UDP, length 100
>> >>
>> >> Server-b get the traffic on it's interface but does nothing with it.
>> >> ipvsadm -Lcn on server-a show a list with information but on server-b
>> >> nothing but documentation states that it should and there is a kernel
>> >> thread running called [ipvs_syncbackup].
>> >>
>> >> A strace of the pulse process on server-b shows only the polling of
>> >> server-a.
>> >>
>> >> So what am I missing here? I hope somebody has a clue.
>> >
>> > What version of piranha are you using? Also, please run 'ivpsadm
>> > --list --daemon' on each director to verify the sync daemon is running
>> > as expected.
>> >
>> > Ryan
>> >
>>
>> The version of piranha is: piranha-0.8.5-19.el6.x86_64
>
> This version should work, but I have fixed a few bugs in piranha
> related to the sync daemon. I'd update if possible.
>
>> The command "ipvsadm --list --daemon" does not return anything?
>>
>> Does I mis a running daemon? Which daemon would that be?
>
> You might be hitting a bug in ipvsadm. I reported a bug where 'ipvsadm
> --list --daemon' where it would not show a running sync daemon in
> backup state. I'm not sure if it was every fixed upstream, but I did
> fix in the RHEL. Check your version of ipvsadm. The
> ipvsadm-1.25-10.el6 package should have the fix.
>
> Ryan
>

I have that version running: ipvsadm-1.25-10.el6.x86_64
So that's not the case I guess.

Is the syncdaemon a kernel thread?

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users