Mailing List Archive

[lvs-users] Problem with ldirectord Bug?
Hi all.

I'm having a problem with ldirectord that I can not solve, I comment my situation:


The problem is in the ipvsadm only show a one real IP, when i configured two now, show the ldirector.cf:


root@ldap1:/home/mamartin# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP ldap-maqueta.cica.es:ldap wrr
-> ldap2.cica.es:ldap Route 4 0 0
TCP [[fc00:1::3]]:ldap wrr
-> [[fc00:1::2]]:ldap Route 4 0 0

root@ldap1:/home/mamartin# cat /etc/ldirectord.cf
checktimeout=10
checkinterval=2
autoreload=yes
logfile="/var/log/ldirectord.log"
quiescent=yes

#ipv4
virtual=192.168.1.10:389
real=192.168.1.11:389 gate 4
real=192.168.1.12:389 gate 4
scheduler=wrr
protocol=tcp
checktype=on
#ipv6
virtual6=[[fc00:1::3]]:389
real6=[[fc00:1::1]]:389 gate 4
real6=[[fc00:1::2]]:389 gate 4
scheduler=wrr
protocol=tcp
checkport=389
checktype=on


and in the logs I see nothing clear:

root@ldap1:/home/mamartin# ldirectord -d /etc/ldirectord.cf start
DEBUG2: Running system(/sbin/ipvsadm -a -t 192.168.1.10:389 -r 192.168.1.11:389 -g -w 0)
Running system(/sbin/ipvsadm -a -t 192.168.1.10:389 -r 192.168.1.11:389 -g -w 0)
DEBUG2: Quiescent real server: 192.168.1.11:389 (192.168.1.10:389) (Weight set to 0)
Quiescent real server: 192.168.1.11:389 (192.168.1.10:389) (Weight set to 0)
DEBUG2: Disabled real server=on:tcp:192.168.1.11:389:::4:gate:\/: (virtual=tcp:192.168.1.10:389)
DEBUG2: Running system(/sbin/ipvsadm -a -t 192.168.1.10:389 -r 192.168.1.12:389 -g -w 0)
Running system(/sbin/ipvsadm -a -t 192.168.1.10:389 -r 192.168.1.12:389 -g -w 0)
DEBUG2: Quiescent real server: 192.168.1.12:389 (192.168.1.10:389) (Weight set to 0)
Quiescent real server: 192.168.1.12:389 (192.168.1.10:389) (Weight set to 0)
DEBUG2: Disabled real server=on:tcp:192.168.1.12:389:::4:gate:\/: (virtual=tcp:192.168.1.10:389)
DEBUG2: Checking on: Real servers are added without any checks
DEBUG2: Resetting soft failure count: 192.168.1.12:389 (tcp:192.168.1.10:389)
Resetting soft failure count: 192.168.1.12:389 (tcp:192.168.1.10:389)
DEBUG2: Running system(/sbin/ipvsadm -a -t 192.168.1.10:389 -r 192.168.1.12:389 -g -w 4)
Running system(/sbin/ipvsadm -a -t 192.168.1.10:389 -r 192.168.1.12:389 -g -w 4)
Destination already exists




root@ldap1:/home/mamartin# cat /var/log/ldirectord.log
[Tue Jun 3 09:39:29 2014|ldirectord.cf|19266] Quiescent real server: 192.168.1.11:389 (192.168.1.10:389) (Weight set to 0)
[Tue Jun 3 09:39:29 2014|ldirectord.cf|19266] Quiescent real server: 192.168.1.12:389 (192.168.1.10:389) (Weight set to 0)
[Tue Jun 3 09:39:29 2014|ldirectord.cf|19266] Resetting soft failure count: 192.168.1.12:389 (tcp:192.168.1.10:389)
[Tue Jun 3 09:39:29 2014|ldirectord.cf|19266] system(/sbin/ipvsadm -a -t 192.168.1.10:389 -r 192.168.1.12:389 -g -w 4) failed:
[Tue Jun 3 09:39:29 2014|ldirectord.cf|19266] Added real server: 192.168.1.12:389 (192.168.1.10:389) (Weight set to 4)
[Tue Jun 3 09:39:29 2014|ldirectord.cf|19266] Resetting soft failure count: 192.168.1.11:389 (tcp:192.168.1.10:389)
[Tue Jun 3 09:39:29 2014|ldirectord.cf|19266] Restored real server: 192.168.1.11:389 (192.168.1.10:389) (Weight set to 4)
[Tue Jun 3 09:39:29 2014|ldirectord.cf|19266] Resetting soft failure count: [[fc00:1::2]]:389 (tcp:[[fc00:1::3]]:389)
[Tue Jun 3 09:39:29 2014|ldirectord.cf|19266] system(/sbin/ipvsadm -a -t [[fc00:1::3]]:389 -r [[fc00:1::2]]:389 -g -w 4) failed:
[Tue Jun 3 09:39:29 2014|ldirectord.cf|19266] Added real server: [[fc00:1::2]]:389 ([[fc00:1::3]]:389) (Weight set to 4)
[Tue Jun 3 09:39:29 2014|ldirectord.cf|19266] Resetting soft failure count: [[fc00:1::1]]:389 (tcp:[[fc00:1::3]]:389)
[Tue Jun 3 09:39:29 2014|ldirectord.cf|19266] Restored real server: [[fc00:1::1]]:389 ([[fc00:1::3]]:389) (Weight set to 4)


do not know if this is a bug or a configuration error, can anyone help?

Regards.

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
Re: [lvs-users] Problem with ldirectord Bug? [ In reply to ]
I've seen the same thing, It looks like ipvsadm is not displaying the
details from lvs correctly.
However looking at a packet capture you can see it is sending traffic to
all servers.


[root@ip-10-0-1-72]# ipvsadm -A -t 10.0.1.72:808 -s rr
[root@ip-10-0-1-72]# ipvsadm -a -t 10.0.1.72:808 -r 10.0.1.73:808 -m
[root@ip-10-0-1-72]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.1.72:808 rr
-> 10.0.1.73:808 Masq 1 0 0
[root@ip-10-0-1-72]# ipvsadm -a -t 10.0.1.72:808 -r 10.0.1.74:808 -m
[root@ip-10-0-1-72]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.1.72:808 rr
-> 10.0.1.73:808 Masq 1 0 0
[root@ip-10-0-1-72]# ipvsadm -v
ipvsadm v1.26 2008/5/15 (compiled with popt and IPVS v1.2.1)

This was using ec2 with amazons kernel 3.10.40-50.136.amzn1.i686.



On 4 June 2014 10:38, Migue Martin <miguegt@hotmail.es> wrote:

> Hi all.
>
> I'm having a problem with ldirectord that I can not solve, I comment my
> situation:
>
>
> The problem is in the ipvsadm only show a one real IP, when i configured
> two now, show the ldirector.cf:
>
>
> root@ldap1:/home/mamartin# ipvsadm
> IP Virtual Server version 1.2.1 (size=4096)
> Prot LocalAddress:Port Scheduler Flags
> -> RemoteAddress:Port Forward Weight ActiveConn InActConn
> TCP ldap-maqueta.cica.es:ldap wrr
> -> ldap2.cica.es:ldap Route 4 0 0
> TCP [[fc00:1::3]]:ldap wrr
> -> [[fc00:1::2]]:ldap Route 4 0 0
>
> root@ldap1:/home/mamartin# cat /etc/ldirectord.cf
> checktimeout=10
> checkinterval=2
> autoreload=yes
> logfile="/var/log/ldirectord.log"
> quiescent=yes
>
> #ipv4
> virtual=192.168.1.10:389
> real=192.168.1.11:389 gate 4
> real=192.168.1.12:389 gate 4
> scheduler=wrr
> protocol=tcp
> checktype=on
> #ipv6
> virtual6=[[fc00:1::3]]:389
> real6=[[fc00:1::1]]:389 gate 4
> real6=[[fc00:1::2]]:389 gate 4
> scheduler=wrr
> protocol=tcp
> checkport=389
> checktype=on
>
>
> and in the logs I see nothing clear:
>
> root@ldap1:/home/mamartin# ldirectord -d /etc/ldirectord.cf start
> DEBUG2: Running system(/sbin/ipvsadm -a -t 192.168.1.10:389 -r
> 192.168.1.11:389 -g -w 0)
> Running system(/sbin/ipvsadm -a -t 192.168.1.10:389 -r 192.168.1.11:389
> -g -w 0)
> DEBUG2: Quiescent real server: 192.168.1.11:389 (192.168.1.10:389)
> (Weight set to 0)
> Quiescent real server: 192.168.1.11:389 (192.168.1.10:389) (Weight set to
> 0)
> DEBUG2: Disabled real server=on:tcp:192.168.1.11:389:::4:gate:\/:
> (virtual=tcp:192.168.1.10:389)
> DEBUG2: Running system(/sbin/ipvsadm -a -t 192.168.1.10:389 -r
> 192.168.1.12:389 -g -w 0)
> Running system(/sbin/ipvsadm -a -t 192.168.1.10:389 -r 192.168.1.12:389
> -g -w 0)
> DEBUG2: Quiescent real server: 192.168.1.12:389 (192.168.1.10:389)
> (Weight set to 0)
> Quiescent real server: 192.168.1.12:389 (192.168.1.10:389) (Weight set to
> 0)
> DEBUG2: Disabled real server=on:tcp:192.168.1.12:389:::4:gate:\/:
> (virtual=tcp:192.168.1.10:389)
> DEBUG2: Checking on: Real servers are added without any checks
> DEBUG2: Resetting soft failure count: 192.168.1.12:389 (tcp:
> 192.168.1.10:389)
> Resetting soft failure count: 192.168.1.12:389 (tcp:192.168.1.10:389)
> DEBUG2: Running system(/sbin/ipvsadm -a -t 192.168.1.10:389 -r
> 192.168.1.12:389 -g -w 4)
> Running system(/sbin/ipvsadm -a -t 192.168.1.10:389 -r 192.168.1.12:389
> -g -w 4)
> Destination already exists
>
>
>
>
> root@ldap1:/home/mamartin# cat /var/log/ldirectord.log
> [Tue Jun 3 09:39:29 2014|ldirectord.cf|19266] Quiescent real server:
> 192.168.1.11:389 (192.168.1.10:389) (Weight set to 0)
> [Tue Jun 3 09:39:29 2014|ldirectord.cf|19266] Quiescent real server:
> 192.168.1.12:389 (192.168.1.10:389) (Weight set to 0)
> [Tue Jun 3 09:39:29 2014|ldirectord.cf|19266] Resetting soft failure
> count: 192.168.1.12:389 (tcp:192.168.1.10:389)
> [Tue Jun 3 09:39:29 2014|ldirectord.cf|19266] system(/sbin/ipvsadm -a -t
> 192.168.1.10:389 -r 192.168.1.12:389 -g -w 4) failed:
> [Tue Jun 3 09:39:29 2014|ldirectord.cf|19266] Added real server:
> 192.168.1.12:389 (192.168.1.10:389) (Weight set to 4)
> [Tue Jun 3 09:39:29 2014|ldirectord.cf|19266] Resetting soft failure
> count: 192.168.1.11:389 (tcp:192.168.1.10:389)
> [Tue Jun 3 09:39:29 2014|ldirectord.cf|19266] Restored real server:
> 192.168.1.11:389 (192.168.1.10:389) (Weight set to 4)
> [Tue Jun 3 09:39:29 2014|ldirectord.cf|19266] Resetting soft failure
> count: [[fc00:1::2]]:389 (tcp:[[fc00:1::3]]:389)
> [Tue Jun 3 09:39:29 2014|ldirectord.cf|19266] system(/sbin/ipvsadm -a -t
> [[fc00:1::3]]:389 -r [[fc00:1::2]]:389 -g -w 4) failed:
> [Tue Jun 3 09:39:29 2014|ldirectord.cf|19266] Added real server:
> [[fc00:1::2]]:389 ([[fc00:1::3]]:389) (Weight set to 4)
> [Tue Jun 3 09:39:29 2014|ldirectord.cf|19266] Resetting soft failure
> count: [[fc00:1::1]]:389 (tcp:[[fc00:1::3]]:389)
> [Tue Jun 3 09:39:29 2014|ldirectord.cf|19266] Restored real server:
> [[fc00:1::1]]:389 ([[fc00:1::3]]:389) (Weight set to 4)
>
>
> do not know if this is a bug or a configuration error, can anyone help?
>
> Regards.
>
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> Send requests to lvs-users-request@LinuxVirtualServer.org
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
>



--
LOADBALANCER.ORG LTD.
www.loadbalancer.org
support@loadbalancer.org
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
Re: [lvs-users] Problem with ldirectord Bug? [ In reply to ]
Hello,

On Thu, 5 Jun 2014, Ben Cabot wrote:

> I've seen the same thing, It looks like ipvsadm is not displaying the
> details from lvs correctly.
> However looking at a packet capture you can see it is sending traffic to
> all servers.
>
>
> [root@ip-10-0-1-72]# ipvsadm -A -t 10.0.1.72:808 -s rr
> [root@ip-10-0-1-72]# ipvsadm -a -t 10.0.1.72:808 -r 10.0.1.73:808 -m
> [root@ip-10-0-1-72]# ipvsadm -Ln
> IP Virtual Server version 1.2.1 (size=4096)
> Prot LocalAddress:Port Scheduler Flags
> -> RemoteAddress:Port Forward Weight ActiveConn InActConn
> TCP 10.0.1.72:808 rr
> -> 10.0.1.73:808 Masq 1 0 0
> [root@ip-10-0-1-72]# ipvsadm -a -t 10.0.1.72:808 -r 10.0.1.74:808 -m
> [root@ip-10-0-1-72]# ipvsadm -Ln
> IP Virtual Server version 1.2.1 (size=4096)
> Prot LocalAddress:Port Scheduler Flags
> -> RemoteAddress:Port Forward Weight ActiveConn InActConn
> TCP 10.0.1.72:808 rr
> -> 10.0.1.73:808 Masq 1 0 0

Do you see the same with this command?:
cat /proc/net/ip_vs

> [root@ip-10-0-1-72]# ipvsadm -v
> ipvsadm v1.26 2008/5/15 (compiled with popt and IPVS v1.2.1)
>
> This was using ec2 with amazons kernel 3.10.40-50.136.amzn1.i686.
>
>
>
> On 4 June 2014 10:38, Migue Martin <miguegt@hotmail.es> wrote:
>
> > Hi all.
> >
> > I'm having a problem with ldirectord that I can not solve, I comment my
> > situation:
> >
> >
> > The problem is in the ipvsadm only show a one real IP, when i configured
> > two now, show the ldirector.cf:
> >
> >
> > root@ldap1:/home/mamartin# ipvsadm
> > IP Virtual Server version 1.2.1 (size=4096)
> > Prot LocalAddress:Port Scheduler Flags
> > -> RemoteAddress:Port Forward Weight ActiveConn InActConn
> > TCP ldap-maqueta.cica.es:ldap wrr
> > -> ldap2.cica.es:ldap Route 4 0 0
> > TCP [[fc00:1::3]]:ldap wrr
> > -> [[fc00:1::2]]:ldap Route 4 0 0

Regards

--
Julian Anastasov <ja@ssi.bg>

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
Re: [lvs-users] Problem with ldirectord Bug? [ In reply to ]
Julian,

Running cat /proc/net/ip_vs gives the correct output.

IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 0A000148:0328 rr
-> 0A00014A:0328 Masq 1 0 0
-> 0A000149:0328 Masq 1 0 0

Regards


On 6 June 2014 08:16, Julian Anastasov <ja@ssi.bg> wrote:

>
> Hello,
>
> On Thu, 5 Jun 2014, Ben Cabot wrote:
>
> > I've seen the same thing, It looks like ipvsadm is not displaying the
> > details from lvs correctly.
> > However looking at a packet capture you can see it is sending traffic to
> > all servers.
> >
> >
> > [root@ip-10-0-1-72]# ipvsadm -A -t 10.0.1.72:808 -s rr
> > [root@ip-10-0-1-72]# ipvsadm -a -t 10.0.1.72:808 -r 10.0.1.73:808 -m
> > [root@ip-10-0-1-72]# ipvsadm -Ln
> > IP Virtual Server version 1.2.1 (size=4096)
> > Prot LocalAddress:Port Scheduler Flags
> > -> RemoteAddress:Port Forward Weight ActiveConn InActConn
> > TCP 10.0.1.72:808 rr
> > -> 10.0.1.73:808 Masq 1 0 0
> > [root@ip-10-0-1-72]# ipvsadm -a -t 10.0.1.72:808 -r 10.0.1.74:808 -m
> > [root@ip-10-0-1-72]# ipvsadm -Ln
> > IP Virtual Server version 1.2.1 (size=4096)
> > Prot LocalAddress:Port Scheduler Flags
> > -> RemoteAddress:Port Forward Weight ActiveConn InActConn
> > TCP 10.0.1.72:808 rr
> > -> 10.0.1.73:808 Masq 1 0 0
>
> Do you see the same with this command?:
> cat /proc/net/ip_vs
>
> > [root@ip-10-0-1-72]# ipvsadm -v
> > ipvsadm v1.26 2008/5/15 (compiled with popt and IPVS v1.2.1)
> >
> > This was using ec2 with amazons kernel 3.10.40-50.136.amzn1.i686.
> >
> >
> >
> > On 4 June 2014 10:38, Migue Martin <miguegt@hotmail.es> wrote:
> >
> > > Hi all.
> > >
> > > I'm having a problem with ldirectord that I can not solve, I comment my
> > > situation:
> > >
> > >
> > > The problem is in the ipvsadm only show a one real IP, when i
> configured
> > > two now, show the ldirector.cf:
> > >
> > >
> > > root@ldap1:/home/mamartin# ipvsadm
> > > IP Virtual Server version 1.2.1 (size=4096)
> > > Prot LocalAddress:Port Scheduler Flags
> > > -> RemoteAddress:Port Forward Weight ActiveConn InActConn
> > > TCP ldap-maqueta.cica.es:ldap wrr
> > > -> ldap2.cica.es:ldap Route 4 0 0
> > > TCP [[fc00:1::3]]:ldap wrr
> > > -> [[fc00:1::2]]:ldap Route 4 0 0
>
> Regards
>
> --
> Julian Anastasov <ja@ssi.bg>
>



--
LOADBALANCER.ORG LTD.
www.loadbalancer.org
support@loadbalancer.org
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
Re: [lvs-users] Problem with ldirectord Bug? [ In reply to ]
Hello,

On Fri, 6 Jun 2014, Ben Cabot wrote:

> Julian,
>
> Running cat /proc/net/ip_vs gives the correct output.

So, it is a problem with the utility. Can someone
check by downloading and compiling latest ipvsadm:

git clone git://git.kernel.org/pub/scm/utils/kernel/ipvsadm/ipvsadm.git

libipvs/libipvs.c has a function ipvs_get_dests() that
gets the destinations for service, not sure what can
go wrong there, may be some of the latest fixes can help.
It can be easily debugged in gdb if the latest version
has the same problem.

> IP Virtual Server version 1.2.1 (size=4096)
> Prot LocalAddress:Port Scheduler Flags
> -> RemoteAddress:Port Forward Weight ActiveConn InActConn
> TCP 0A000148:0328 rr
> -> 0A00014A:0328 Masq 1 0 0
> -> 0A000149:0328 Masq 1 0 0
>
> Regards

Regards

--
Julian Anastasov <ja@ssi.bg>

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
Re: [lvs-users] Problem with ldirectord Bug? [ In reply to ]
Thanks Julian. I have just tested the latest ipvsadm and it works as
expected.

Regards


On 6 June 2014 21:14, Julian Anastasov <ja@ssi.bg> wrote:

>
> Hello,
>
> On Fri, 6 Jun 2014, Ben Cabot wrote:
>
> > Julian,
> >
> > Running cat /proc/net/ip_vs gives the correct output.
>
> So, it is a problem with the utility. Can someone
> check by downloading and compiling latest ipvsadm:
>
> git clone git://git.kernel.org/pub/scm/utils/kernel/ipvsadm/ipvsadm.git
>
> libipvs/libipvs.c has a function ipvs_get_dests() that
> gets the destinations for service, not sure what can
> go wrong there, may be some of the latest fixes can help.
> It can be easily debugged in gdb if the latest version
> has the same problem.
>
> > IP Virtual Server version 1.2.1 (size=4096)
> > Prot LocalAddress:Port Scheduler Flags
> > -> RemoteAddress:Port Forward Weight ActiveConn InActConn
> > TCP 0A000148:0328 rr
> > -> 0A00014A:0328 Masq 1 0 0
> > -> 0A000149:0328 Masq 1 0 0
> >
> > Regards
>
> Regards
>
> --
> Julian Anastasov <ja@ssi.bg>
>



--
LOADBALANCER.ORG LTD.
www.loadbalancer.org
support@loadbalancer.org
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users