Mailing List Archive: [lvs-users] lvs not routing to real

[lvs-users] lvs not routing to real

Sep 23, 2017, 9:46 AM

Post #1 of 5 (817 views)

In AWS, I am trying to get the keepalived/ipvs working and see that the
load balancer/director does not forward the requests to my real server

ubuntu@ip-172-31-6-2:~$ cat /etc/keepalived/keepalived.conf
global_defs {
notification_email {
test@test.com
}
}

virtual_server 13.57.18.235 1935 {
delay_loop 30
lb_algo rr
lb_kind DR
persistence_timeout 50
protocol TCP
real_server 172.31.12.15 1935 {
TCP_CHECK {
connect_port 1935
connect_timeout 3
}
}
}

vrrp_instance v1 {
state MASTER
interface eth0
lvs_sync_daemon_interface eth0
virtual_router_id 51
priority 150
advert_int 1
virtual_ipaddress {
13.57.18.235
}
}

ubuntu@ip-172-31-6-2:~$ sudo ipvsadm -L -n
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 13.57.18.235:1935 rr persistent 50
-> 172.31.12.15:1935 Route 1 0 0

loopback on real

ubuntu@ip-172-31-12-15:~$ ifconfig lo:0
lo:0 Link encap:Local Loopback
inet addr:13.57.18.235 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:65536 Metric:1

When i run tcpdump on both the loadbalancer and real see that tcp reset
happens on the loadbalancer after sync and reset happens after the
handshake on the real

tcpdump on loadbalancer

03:50:35.405489 IP (tos 0x0, ttl 51, id 59005, offset 0, flags [DF],
proto TCP (6), length 64)
c-98-207-84-68.hsd1.ca.comcast.net.51279 >
ip-172-31-6-2.us-west-1.compute.internal.1935: Flags [S], cksum 0x4e32
(correct), seq 2162214371, win 65535, options [mss 1460,nop,wscale
5,nop,nop,TS val 373641902 ecr 0,sackOK,eol], length 0
E..@.}@.3...b.TD.....O..............N2.............
.ER.........
03:50:35.405531 IP (tos 0x0, ttl 64, id 46103, offset 0, flags [DF],
proto TCP (6), length 40)
ip-172-31-6-2.us-west-1.compute.internal.1935 >
c-98-207-84-68.hsd1.ca.comcast.net.51279: Flags [R.], cksum 0x2ff8
(correct), seq 0, ack 1, win 0, length 0
E..(..@.@.......b.TD...O........P.../...

tcpdump on real

ip-172-31-6-2.us-west-1.compute.internal.44230 >
ip-172-31-12-15.us-west-1.compute.internal.1935: Flags [S], cksum 0x9ef2
(correct), seq 2759878464, win 26883, options [mss 8961,sackOK,TS val
2324983234 ecr 0,nop,wscale 7], length 0
E..<.v@.@.?...............g@......i.......#....
..m.........
03:51:52.362958 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto
TCP (6), length 60)
ip-172-31-12-15.us-west-1.compute.internal.1935 >
ip-172-31-6-2.us-west-1.compute.internal.44230: Flags [S.], cksum 0x6a7e
(incorrect -> 0x7528), seq 527723804, ack 2759878465, win 26847, options
[mss 8961,sackOK,TS val 156996593 ecr 2324983234,nop,wscale 7], length 0
E..<..@.@..l.............tm...gA..h.j~....#....
[....m.....
ip-172-31-6-2.us-west-1.compute.internal.44230 >
ip-172-31-12-15.us-west-1.compute.internal.1935: Flags [.], cksum 0x294e
(correct), seq 1, ack 1, win 211, options [nop,nop,TS val 2324983234 ecr
156996593], length 0
E..4.w@.@.?...............gA.tm.....)N.....
..m. [...
03:51:52.363391 IP (tos 0x0, ttl 64, id 36984, offset 0, flags [DF],
proto TCP (6), length 52)
ip-172-31-6-2.us-west-1.compute.internal.44230 >
ip-172-31-12-15.us-west-1.compute.internal.1935: Flags [R.], cksum 0x294a
(correct), seq 1, ack 1, win 211, options [nop,nop,TS val 2324983234 ecr
156996593], length 0
E..4.x@.@.?...............gA.tm.....)J.....
..m. [..

Let me know if i am missing anything here
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

Re: [lvs-users] lvs not routing to real [ In reply to ]

bseenu at gmail

Sep 23, 2017, 5:20 PM

Post #2 of 5 (817 views)

Permalink

my bad, the reset on real server is happening as the part of the tcp check
and not from load balancer

So the lvs director replies with rst,ack as a reply to syn packet

-srini

On Sat, Sep 23, 2017 at 9:46 AM, Srinivas Boga <bseenu@gmail.com> wrote:

> In AWS, I am trying to get the keepalived/ipvs working and see that the
> load balancer/director does not forward the requests to my real server
>
>
> ubuntu@ip-172-31-6-2:~$ cat /etc/keepalived/keepalived.conf
> global_defs {
> notification_email {
> test@test.com
> }
> }
>
> virtual_server 13.57.18.235 1935 {
> delay_loop 30
> lb_algo rr
> lb_kind DR
> persistence_timeout 50
> protocol TCP
> real_server 172.31.12.15 1935 {
> TCP_CHECK {
> connect_port 1935
> connect_timeout 3
> }
> }
> }
>
> vrrp_instance v1 {
> state MASTER
> interface eth0
> lvs_sync_daemon_interface eth0
> virtual_router_id 51
> priority 150
> advert_int 1
> virtual_ipaddress {
> 13.57.18.235
> }
> }
>
> ubuntu@ip-172-31-6-2:~$ sudo ipvsadm -L -n
> IP Virtual Server version 1.2.1 (size=4096)
> Prot LocalAddress:Port Scheduler Flags
> -> RemoteAddress:Port Forward Weight ActiveConn InActConn
> TCP 13.57.18.235:1935 rr persistent 50
> -> 172.31.12.15:1935 Route 1 0 0
>
> loopback on real
>
> ubuntu@ip-172-31-12-15:~$ ifconfig lo:0
> lo:0 Link encap:Local Loopback
> inet addr:13.57.18.235 Mask:255.0.0.0
> UP LOOPBACK RUNNING MTU:65536 Metric:1
>
> When i run tcpdump on both the loadbalancer and real see that tcp reset
> happens on the loadbalancer after sync and reset happens after the
> handshake on the real
>
> tcpdump on loadbalancer
>
> 03:50:35.405489 IP (tos 0x0, ttl 51, id 59005, offset 0, flags [DF],
> proto TCP (6), length 64)
> c-98-207-84-68.hsd1.ca.comcast.net.51279 >
> ip-172-31-6-2.us-west-1.compute.internal.1935: Flags [S], cksum 0x4e32
> (correct), seq 2162214371 <(216)%20221-4371>, win 65535, options [mss
> 1460,nop,wscale 5,nop,nop,TS val 373641902 ecr 0,sackOK,eol], length 0
> E..@.}@.3...b.TD.....O..............N2.............
> .ER.........
> 03:50:35.405531 IP (tos 0x0, ttl 64, id 46103, offset 0, flags [DF],
> proto TCP (6), length 40)
> ip-172-31-6-2.us-west-1.compute.internal.1935 >
> c-98-207-84-68.hsd1.ca.comcast.net.51279: Flags [R.], cksum 0x2ff8
> (correct), seq 0, ack 1, win 0, length 0
> E..(..@.@.......b.TD...O........P.../...
>
> tcpdump on real
>
> ip-172-31-6-2.us-west-1.compute.internal.44230 >
> ip-172-31-12-15.us-west-1.compute.internal.1935: Flags [S], cksum 0x9ef2
> (correct), seq 2759878464, win 26883, options [mss 8961,sackOK,TS val
> 2324983234 ecr 0,nop,wscale 7], length 0
> E..<.v@.@.?...............g@......i.......#....
> ..m.........
> 03:51:52.362958 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto
> TCP (6), length 60)
> ip-172-31-12-15.us-west-1.compute.internal.1935 >
> ip-172-31-6-2.us-west-1.compute.internal.44230: Flags [S.], cksum 0x6a7e
> (incorrect -> 0x7528), seq 527723804, ack 2759878465, win 26847, options
> [mss 8961,sackOK,TS val 156996593 ecr 2324983234,nop,wscale 7], length 0
> E..<..@.@..l.............tm...gA..h.j~....#....
> [....m.....
> ip-172-31-6-2.us-west-1.compute.internal.44230 >
> ip-172-31-12-15.us-west-1.compute.internal.1935: Flags [.], cksum 0x294e
> (correct), seq 1, ack 1, win 211, options [nop,nop,TS val 2324983234 ecr
> 156996593], length 0
> E..4.w@.@.?...............gA.tm.....)N.....
> ..m. [...
> 03:51:52.363391 IP (tos 0x0, ttl 64, id 36984, offset 0, flags [DF],
> proto TCP (6), length 52)
> ip-172-31-6-2.us-west-1.compute.internal.44230 >
> ip-172-31-12-15.us-west-1.compute.internal.1935: Flags [R.], cksum 0x294a
> (correct), seq 1, ack 1, win 211, options [nop,nop,TS val 2324983234 ecr
> 156996593], length 0
> E..4.x@.@.?...............gA.tm.....)J.....
> ..m. [..
>
> Let me know if i am missing anything here
>
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

Re: [lvs-users] lvs not routing to real [ In reply to ]

aaron at loadbalancer

Sep 26, 2017, 6:55 AM

Post #3 of 5 (817 views)

Permalink

Hi Srini,

Have you disabled the source and destination checks on both the real
servers and the load balancers?
Aaron West

Loadbalancer.org

www.loadbalancer.org
+1 888 867 9504 / +44 (0)330 380 1064
aaron@loadbalancer.org

LEAVE A REVIEW | DEPLOYMENT GUIDES | BLOG

On 24 September 2017 at 01:20, Srinivas Boga <bseenu@gmail.com> wrote:
> my bad, the reset on real server is happening as the part of the tcp check
> and not from load balancer
>
> So the lvs director replies with rst,ack as a reply to syn packet
>
> -srini
>
> On Sat, Sep 23, 2017 at 9:46 AM, Srinivas Boga <bseenu@gmail.com> wrote:
>
>> In AWS, I am trying to get the keepalived/ipvs working and see that the
>> load balancer/director does not forward the requests to my real server
>>
>>
>> ubuntu@ip-172-31-6-2:~$ cat /etc/keepalived/keepalived.conf
>> global_defs {
>> notification_email {
>> test@test.com
>> }
>> }
>>
>> virtual_server 13.57.18.235 1935 {
>> delay_loop 30
>> lb_algo rr
>> lb_kind DR
>> persistence_timeout 50
>> protocol TCP
>> real_server 172.31.12.15 1935 {
>> TCP_CHECK {
>> connect_port 1935
>> connect_timeout 3
>> }
>> }
>> }
>>
>> vrrp_instance v1 {
>> state MASTER
>> interface eth0
>> lvs_sync_daemon_interface eth0
>> virtual_router_id 51
>> priority 150
>> advert_int 1
>> virtual_ipaddress {
>> 13.57.18.235
>> }
>> }
>>
>> ubuntu@ip-172-31-6-2:~$ sudo ipvsadm -L -n
>> IP Virtual Server version 1.2.1 (size=4096)
>> Prot LocalAddress:Port Scheduler Flags
>> -> RemoteAddress:Port Forward Weight ActiveConn InActConn
>> TCP 13.57.18.235:1935 rr persistent 50
>> -> 172.31.12.15:1935 Route 1 0 0
>>
>> loopback on real
>>
>> ubuntu@ip-172-31-12-15:~$ ifconfig lo:0
>> lo:0 Link encap:Local Loopback
>> inet addr:13.57.18.235 Mask:255.0.0.0
>> UP LOOPBACK RUNNING MTU:65536 Metric:1
>>
>> When i run tcpdump on both the loadbalancer and real see that tcp reset
>> happens on the loadbalancer after sync and reset happens after the
>> handshake on the real
>>
>> tcpdump on loadbalancer
>>
>> 03:50:35.405489 IP (tos 0x0, ttl 51, id 59005, offset 0, flags [DF],
>> proto TCP (6), length 64)
>> c-98-207-84-68.hsd1.ca.comcast.net.51279 >
>> ip-172-31-6-2.us-west-1.compute.internal.1935: Flags [S], cksum 0x4e32
>> (correct), seq 2162214371 <(216)%20221-4371>, win 65535, options [mss
>> 1460,nop,wscale 5,nop,nop,TS val 373641902 ecr 0,sackOK,eol], length 0
>> E..@.}@.3...b.TD.....O..............N2.............
>> .ER.........
>> 03:50:35.405531 IP (tos 0x0, ttl 64, id 46103, offset 0, flags [DF],
>> proto TCP (6), length 40)
>> ip-172-31-6-2.us-west-1.compute.internal.1935 >
>> c-98-207-84-68.hsd1.ca.comcast.net.51279: Flags [R.], cksum 0x2ff8
>> (correct), seq 0, ack 1, win 0, length 0
>> E..(..@.@.......b.TD...O........P.../...
>>
>> tcpdump on real
>>
>> ip-172-31-6-2.us-west-1.compute.internal.44230 >
>> ip-172-31-12-15.us-west-1.compute.internal.1935: Flags [S], cksum 0x9ef2
>> (correct), seq 2759878464, win 26883, options [mss 8961,sackOK,TS val
>> 2324983234 ecr 0,nop,wscale 7], length 0
>> E..<.v@.@.?...............g@......i.......#....
>> ..m.........
>> 03:51:52.362958 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto
>> TCP (6), length 60)
>> ip-172-31-12-15.us-west-1.compute.internal.1935 >
>> ip-172-31-6-2.us-west-1.compute.internal.44230: Flags [S.], cksum 0x6a7e
>> (incorrect -> 0x7528), seq 527723804, ack 2759878465, win 26847, options
>> [mss 8961,sackOK,TS val 156996593 ecr 2324983234,nop,wscale 7], length 0
>> E..<..@.@..l.............tm...gA..h.j~....#....
>> [....m.....
>> ip-172-31-6-2.us-west-1.compute.internal.44230 >
>> ip-172-31-12-15.us-west-1.compute.internal.1935: Flags [.], cksum 0x294e
>> (correct), seq 1, ack 1, win 211, options [nop,nop,TS val 2324983234 ecr
>> 156996593], length 0
>> E..4.w@.@.?...............gA.tm.....)N.....
>> ..m. [...
>> 03:51:52.363391 IP (tos 0x0, ttl 64, id 36984, offset 0, flags [DF],
>> proto TCP (6), length 52)
>> ip-172-31-6-2.us-west-1.compute.internal.44230 >
>> ip-172-31-12-15.us-west-1.compute.internal.1935: Flags [R.], cksum 0x294a
>> (correct), seq 1, ack 1, win 211, options [nop,nop,TS val 2324983234 ecr
>> 156996593], length 0
>> E..4.x@.@.?...............gA.tm.....)J.....
>> ..m. [..
>>
>> Let me know if i am missing anything here
>>
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> Send requests to lvs-users-request@LinuxVirtualServer.org
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

Re: [lvs-users] lvs not routing to real [ In reply to ]

aaron at loadbalancer

Sep 26, 2017, 7:02 AM

Post #4 of 5 (817 views)

Permalink

Srini,

I just noticed that you are trying LVS-DR, can I ask where the clients
are? If they are external you will likely find packets get dropped at
Amazon's edge.

Amazon seem to employ anti-spoofing at their edge, one way to prove
this is to set the default gateway of the real server to be the load
balancer(as you would for NAT mode). Just by making it route back via
the instance Amazon knows to own the VIP will make it work in my
experience.

Thus you may as well use a single arm NAT implementation when
supporting external clients, however, if the client is in the same VPC
it should work so can still be useful for some deployments.

Aaron West

Loadbalancer.org

www.loadbalancer.org
+1 888 867 9504 / +44 (0)330 380 1064
aaron@loadbalancer.org

LEAVE A REVIEW | DEPLOYMENT GUIDES | BLOG

On 26 September 2017 at 14:55, Aaron West <aaron@loadbalancer.org> wrote:
> Hi Srini,
>
> Have you disabled the source and destination checks on both the real
> servers and the load balancers?
> Aaron West
>
> Loadbalancer.org
>
> www.loadbalancer.org
> +1 888 867 9504 / +44 (0)330 380 1064
> aaron@loadbalancer.org
>
> LEAVE A REVIEW | DEPLOYMENT GUIDES | BLOG
>
>
> On 24 September 2017 at 01:20, Srinivas Boga <bseenu@gmail.com> wrote:
>> my bad, the reset on real server is happening as the part of the tcp check
>> and not from load balancer
>>
>> So the lvs director replies with rst,ack as a reply to syn packet
>>
>> -srini
>>
>> On Sat, Sep 23, 2017 at 9:46 AM, Srinivas Boga <bseenu@gmail.com> wrote:
>>
>>> In AWS, I am trying to get the keepalived/ipvs working and see that the
>>> load balancer/director does not forward the requests to my real server
>>>
>>>
>>> ubuntu@ip-172-31-6-2:~$ cat /etc/keepalived/keepalived.conf
>>> global_defs {
>>> notification_email {
>>> test@test.com
>>> }
>>> }
>>>
>>> virtual_server 13.57.18.235 1935 {
>>> delay_loop 30
>>> lb_algo rr
>>> lb_kind DR
>>> persistence_timeout 50
>>> protocol TCP
>>> real_server 172.31.12.15 1935 {
>>> TCP_CHECK {
>>> connect_port 1935
>>> connect_timeout 3
>>> }
>>> }
>>> }
>>>
>>> vrrp_instance v1 {
>>> state MASTER
>>> interface eth0
>>> lvs_sync_daemon_interface eth0
>>> virtual_router_id 51
>>> priority 150
>>> advert_int 1
>>> virtual_ipaddress {
>>> 13.57.18.235
>>> }
>>> }
>>>
>>> ubuntu@ip-172-31-6-2:~$ sudo ipvsadm -L -n
>>> IP Virtual Server version 1.2.1 (size=4096)
>>> Prot LocalAddress:Port Scheduler Flags
>>> -> RemoteAddress:Port Forward Weight ActiveConn InActConn
>>> TCP 13.57.18.235:1935 rr persistent 50
>>> -> 172.31.12.15:1935 Route 1 0 0
>>>
>>> loopback on real
>>>
>>> ubuntu@ip-172-31-12-15:~$ ifconfig lo:0
>>> lo:0 Link encap:Local Loopback
>>> inet addr:13.57.18.235 Mask:255.0.0.0
>>> UP LOOPBACK RUNNING MTU:65536 Metric:1
>>>
>>> When i run tcpdump on both the loadbalancer and real see that tcp reset
>>> happens on the loadbalancer after sync and reset happens after the
>>> handshake on the real
>>>
>>> tcpdump on loadbalancer
>>>
>>> 03:50:35.405489 IP (tos 0x0, ttl 51, id 59005, offset 0, flags [DF],
>>> proto TCP (6), length 64)
>>> c-98-207-84-68.hsd1.ca.comcast.net.51279 >
>>> ip-172-31-6-2.us-west-1.compute.internal.1935: Flags [S], cksum 0x4e32
>>> (correct), seq 2162214371 <(216)%20221-4371>, win 65535, options [mss
>>> 1460,nop,wscale 5,nop,nop,TS val 373641902 ecr 0,sackOK,eol], length 0
>>> E..@.}@.3...b.TD.....O..............N2.............
>>> .ER.........
>>> 03:50:35.405531 IP (tos 0x0, ttl 64, id 46103, offset 0, flags [DF],
>>> proto TCP (6), length 40)
>>> ip-172-31-6-2.us-west-1.compute.internal.1935 >
>>> c-98-207-84-68.hsd1.ca.comcast.net.51279: Flags [R.], cksum 0x2ff8
>>> (correct), seq 0, ack 1, win 0, length 0
>>> E..(..@.@.......b.TD...O........P.../...
>>>
>>> tcpdump on real
>>>
>>> ip-172-31-6-2.us-west-1.compute.internal.44230 >
>>> ip-172-31-12-15.us-west-1.compute.internal.1935: Flags [S], cksum 0x9ef2
>>> (correct), seq 2759878464, win 26883, options [mss 8961,sackOK,TS val
>>> 2324983234 ecr 0,nop,wscale 7], length 0
>>> E..<.v@.@.?...............g@......i.......#....
>>> ..m.........
>>> 03:51:52.362958 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto
>>> TCP (6), length 60)
>>> ip-172-31-12-15.us-west-1.compute.internal.1935 >
>>> ip-172-31-6-2.us-west-1.compute.internal.44230: Flags [S.], cksum 0x6a7e
>>> (incorrect -> 0x7528), seq 527723804, ack 2759878465, win 26847, options
>>> [mss 8961,sackOK,TS val 156996593 ecr 2324983234,nop,wscale 7], length 0
>>> E..<..@.@..l.............tm...gA..h.j~....#....
>>> [....m.....
>>> ip-172-31-6-2.us-west-1.compute.internal.44230 >
>>> ip-172-31-12-15.us-west-1.compute.internal.1935: Flags [.], cksum 0x294e
>>> (correct), seq 1, ack 1, win 211, options [nop,nop,TS val 2324983234 ecr
>>> 156996593], length 0
>>> E..4.w@.@.?...............gA.tm.....)N.....
>>> ..m. [...
>>> 03:51:52.363391 IP (tos 0x0, ttl 64, id 36984, offset 0, flags [DF],
>>> proto TCP (6), length 52)
>>> ip-172-31-6-2.us-west-1.compute.internal.44230 >
>>> ip-172-31-12-15.us-west-1.compute.internal.1935: Flags [R.], cksum 0x294a
>>> (correct), seq 1, ack 1, win 211, options [nop,nop,TS val 2324983234 ecr
>>> 156996593], length 0
>>> E..4.x@.@.?...............gA.tm.....)J.....
>>> ..m. [..
>>>
>>> Let me know if i am missing anything here
>>>
>> _______________________________________________
>> Please read the documentation before posting - it's available at:
>> http://www.linuxvirtualserver.org/
>>
>> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
>> Send requests to lvs-users-request@LinuxVirtualServer.org
>> or go to http://lists.graemef.net/mailman/listinfo/lvs-users

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

Re: [lvs-users] lvs not routing to real [ In reply to ]

thannyd at earthlink

Sep 26, 2017, 7:11 AM

Post #5 of 5 (817 views)

Permalink

What do you mean the director replies? You have that resource configured
with direct routing, which means the LVS director has nothing to do with
replying to the client. The real server is responsible for replying
directly back to the client.

Do you have routing set up correctly? You're using an internal IP
address to refer to the real server, while the virtual IP is an external
one. Why doesn't the tcpdump info show the virtual IP? Should you be
using NAT instead of DR?

On 9/23/2017 20:20, Srinivas Boga wrote:
> my bad, the reset on real server is happening as the part of the tcp check
> and not from load balancer
>
> So the lvs director replies with rst,ack as a reply to syn packet
>
> -srini
>
> On Sat, Sep 23, 2017 at 9:46 AM, Srinivas Boga <bseenu@gmail.com> wrote:
>
>> In AWS, I am trying to get the keepalived/ipvs working and see that the
>> load balancer/director does not forward the requests to my real server
>>
>>
>> ubuntu@ip-172-31-6-2:~$ cat /etc/keepalived/keepalived.conf
>> global_defs {
>> notification_email {
>> test@test.com
>> }
>> }
>>
>> virtual_server 13.57.18.235 1935 {
>> delay_loop 30
>> lb_algo rr
>> lb_kind DR
>> persistence_timeout 50
>> protocol TCP
>> real_server 172.31.12.15 1935 {
>> TCP_CHECK {
>> connect_port 1935
>> connect_timeout 3
>> }
>> }
>> }
>>
>> vrrp_instance v1 {
>> state MASTER
>> interface eth0
>> lvs_sync_daemon_interface eth0
>> virtual_router_id 51
>> priority 150
>> advert_int 1
>> virtual_ipaddress {
>> 13.57.18.235
>> }
>> }
>>
>> ubuntu@ip-172-31-6-2:~$ sudo ipvsadm -L -n
>> IP Virtual Server version 1.2.1 (size=4096)
>> Prot LocalAddress:Port Scheduler Flags
>> -> RemoteAddress:Port Forward Weight ActiveConn InActConn
>> TCP 13.57.18.235:1935 rr persistent 50
>> -> 172.31.12.15:1935 Route 1 0 0
>>
>> loopback on real
>>
>> ubuntu@ip-172-31-12-15:~$ ifconfig lo:0
>> lo:0 Link encap:Local Loopback
>> inet addr:13.57.18.235 Mask:255.0.0.0
>> UP LOOPBACK RUNNING MTU:65536 Metric:1
>>
>> When i run tcpdump on both the loadbalancer and real see that tcp reset
>> happens on the loadbalancer after sync and reset happens after the
>> handshake on the real
>>
>> tcpdump on loadbalancer
>>
>> 03:50:35.405489 IP (tos 0x0, ttl 51, id 59005, offset 0, flags [DF],
>> proto TCP (6), length 64)
>> c-98-207-84-68.hsd1.ca.comcast.net.51279 >
>> ip-172-31-6-2.us-west-1.compute.internal.1935: Flags [S], cksum 0x4e32
>> (correct), seq 2162214371 <(216)%20221-4371>, win 65535, options [mss
>> 1460,nop,wscale 5,nop,nop,TS val 373641902 ecr 0,sackOK,eol], length 0
>> E..@.}@.3...b.TD.....O..............N2.............
>> .ER.........
>> 03:50:35.405531 IP (tos 0x0, ttl 64, id 46103, offset 0, flags [DF],
>> proto TCP (6), length 40)
>> ip-172-31-6-2.us-west-1.compute.internal.1935 >
>> c-98-207-84-68.hsd1.ca.comcast.net.51279: Flags [R.], cksum 0x2ff8
>> (correct), seq 0, ack 1, win 0, length 0
>> E..(..@.@.......b.TD...O........P.../...
>>
>> tcpdump on real
>>
>> ip-172-31-6-2.us-west-1.compute.internal.44230 >
>> ip-172-31-12-15.us-west-1.compute.internal.1935: Flags [S], cksum 0x9ef2
>> (correct), seq 2759878464, win 26883, options [mss 8961,sackOK,TS val
>> 2324983234 ecr 0,nop,wscale 7], length 0
>> E..<.v@.@.?...............g@......i.......#....
>> ..m.........
>> 03:51:52.362958 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto
>> TCP (6), length 60)
>> ip-172-31-12-15.us-west-1.compute.internal.1935 >
>> ip-172-31-6-2.us-west-1.compute.internal.44230: Flags [S.], cksum 0x6a7e
>> (incorrect -> 0x7528), seq 527723804, ack 2759878465, win 26847, options
>> [mss 8961,sackOK,TS val 156996593 ecr 2324983234,nop,wscale 7], length 0
>> E..<..@.@..l.............tm...gA..h.j~....#....
>> [....m.....
>> ip-172-31-6-2.us-west-1.compute.internal.44230 >
>> ip-172-31-12-15.us-west-1.compute.internal.1935: Flags [.], cksum 0x294e
>> (correct), seq 1, ack 1, win 211, options [nop,nop,TS val 2324983234 ecr
>> 156996593], length 0
>> E..4.w@.@.?...............gA.tm.....)N.....
>> ..m. [...
>> 03:51:52.363391 IP (tos 0x0, ttl 64, id 36984, offset 0, flags [DF],
>> proto TCP (6), length 52)
>> ip-172-31-6-2.us-west-1.compute.internal.44230 >
>> ip-172-31-12-15.us-west-1.compute.internal.1935: Flags [R.], cksum 0x294a
>> (correct), seq 1, ack 1, win 211, options [nop,nop,TS val 2324983234 ecr
>> 156996593], length 0
>> E..4.x@.@.?...............gA.tm.....)J.....
>> ..m. [..
>>
>> Let me know if i am missing anything here
>>
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> Send requests to lvs-users-request@LinuxVirtualServer.org
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users