Mailing List Archive

[lvs-users] cant get passive ftp working through nat for clustered ftp hosts.
hey folks, Ive read the howtows for a few days now and cant seem to get this to work.. Ive got a
centos box running lvs and 2 backend ftp servers running vsftpd. the backend servers are
set to use passive ports 50000-60000
my server running lvs, has 2 nics, one on the innernets and one private. the two ftp servers are on the
private network. the 2 ftp servers have
pasv_address=x.x.x.x
where x.x.x.x is the outside (internet facing) ip address of my lvs server..
my lvs server is doing the following
export realip=(outside ip address of my server)
ipvsadm -A -t $realip:21 -s wrr
ipvsadm -a -t $realip:21 -r 10.1.6.11 -m
ipvsadm -a -t $realip:21 -r 10.1.6.12 -m

10.1.6.11=vsftp server 1
10.1.6.12=vsftp server 2

sooo from a host on the outside, I can connect to my lvs server's outside ip address on port 21
and if Im using active mode ftp, I can list directories and see files and whatnot..
If I use passive mode, it just hangs..

on the lvs server, I have
[jason@host1 ~]$ lsmod | grep ftp
nf_nat_ftp 3507 0
nf_conntrack_ftp 12913 1 nf_nat_ftp
nf_nat 23316 3 nf_nat_ftp,ipt_MASQUERADE,iptable_nat
ip_vs_ftp 3738 2
ip_vs 125694 7 ip_vs_ftp,ip_vs_wrr,ip_vs_wlc
nf_conntrack 80422 8 nf_nat_ftp,nf_conntrack_ftp,ipt_MASQUERADE,iptable_nat,nf_nat,nf_conntrack_ipv4,nf_conntrack_ipv6,xt_state


but when I sniff the ftp connection on host1 (lvs server), I see the following:
20:21:41.928714 IP myclienthost.org.44588 > mylvsserver.58374: Flags [S], seq 3921685969, win 14600, options [mss 1460,sackOK,TS val 3671275663 ecr 0,nop,wscale 6], length 0
20:21:43.928811 IP myclienthost.org.44588 > mylvsserver.58374: Flags [S], seq 3921685969, win 14600, options [mss 1460,sackOK,TS val 3671277663 ecr 0,nop,wscale 6], length 0

which looks correct for the most part, but I seem to be misisng the "config" that passes along the passive ftp connections
from the lvs server to the back end servers.

i tried the
iptables -t mangle -A PREROUTING -p tcp -d lvsoutsideaddress/32 --dport 21 -j MARK --set-mark 21
iptables -t mangle -A PREROUTING -p tcp -d lvsoutsideaddress/32 --dport 50000:60000 -j MARK --set-mark 21
this seems like it wouldnt work anyway, because its just setting marks on the traffic, dont you need
some other config to DO something with the marked traffic?


regards,
Jason


_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
Re: [lvs-users] cant get passive ftp working through nat for clustered ftp hosts. [ In reply to ]
Hi,

I use proftpd, but i guess there are some similar things.
I has also the same ipvs setup and has also only active ftp working...

Perhaps, to make work the passive mode, we should add a permanent
listener on passive ports on the centos box, with something like

ipvsadm -A -t $realip:50000-60000 -s wrr
ipvsadm -a -t $realip:50000-60000 -r 10.1.6.11 -m
ipvsadm -a -t $realip:50000-60000 -r 10.1.6.12 -m

firewall-cmd --zone=public --add-port=50000-60000/udp --permanent

First, it is not working (multiple ports are not allowed) but i also prefer,
for security reasons to have not so many ports opened on my server.

In active mode, the ip_vs_ftp do all the job via conntrack :

conntrack -L | grep dport=21

In this case, the only needed setup to make it work is to allow FTP
traffic :

firewall-cmd --zone=public --add-port=21/udp --permanent

and setup via ipvsadm the loadbalancing.

So, do you really need the ftp passive mode ?
--
Ivan

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
Re: [lvs-users] cant get passive ftp working through nat for clustered ftp hosts. [ In reply to ]
yes, I need to get passive ftp working as well so I dont have to explain
the difference between active and passive to all our customers.

regards,
Jason


On 06/14/2015 09:01 PM, jason@monsterjam.org wrote:
> hey folks, Ive read the howtows for a few days now and cant seem to get this to work.. Ive got a
> centos box running lvs and 2 backend ftp servers running vsftpd. the backend servers are
> set to use passive ports 50000-60000
> my server running lvs, has 2 nics, one on the innernets and one private. the two ftp servers are on the
> private network. the 2 ftp servers have
> pasv_address=x.x.x.x
> where x.x.x.x is the outside (internet facing) ip address of my lvs server..
> my lvs server is doing the following
> export realip=(outside ip address of my server)
> ipvsadm -A -t $realip:21 -s wrr
> ipvsadm -a -t $realip:21 -r 10.1.6.11 -m
> ipvsadm -a -t $realip:21 -r 10.1.6.12 -m
>
> 10.1.6.11=vsftp server 1
> 10.1.6.12=vsftp server 2
>
> sooo from a host on the outside, I can connect to my lvs server's outside ip address on port 21
> and if Im using active mode ftp, I can list directories and see files and whatnot..
> If I use passive mode, it just hangs..
>
> on the lvs server, I have
> [jason@host1 ~]$ lsmod | grep ftp
> nf_nat_ftp 3507 0
> nf_conntrack_ftp 12913 1 nf_nat_ftp
> nf_nat 23316 3 nf_nat_ftp,ipt_MASQUERADE,iptable_nat
> ip_vs_ftp 3738 2
> ip_vs 125694 7 ip_vs_ftp,ip_vs_wrr,ip_vs_wlc
> nf_conntrack 80422 8 nf_nat_ftp,nf_conntrack_ftp,ipt_MASQUERADE,iptable_nat,nf_nat,nf_conntrack_ipv4,nf_conntrack_ipv6,xt_state
>
>
> but when I sniff the ftp connection on host1 (lvs server), I see the following:
> 20:21:41.928714 IP myclienthost.org.44588 > mylvsserver.58374: Flags [S], seq 3921685969, win 14600, options [mss 1460,sackOK,TS val 3671275663 ecr 0,nop,wscale 6], length 0
> 20:21:43.928811 IP myclienthost.org.44588 > mylvsserver.58374: Flags [S], seq 3921685969, win 14600, options [mss 1460,sackOK,TS val 3671277663 ecr 0,nop,wscale 6], length 0
>
> which looks correct for the most part, but I seem to be misisng the "config" that passes along the passive ftp connections
> from the lvs server to the back end servers.
>
> i tried the
> iptables -t mangle -A PREROUTING -p tcp -d lvsoutsideaddress/32 --dport 21 -j MARK --set-mark 21
> iptables -t mangle -A PREROUTING -p tcp -d lvsoutsideaddress/32 --dport 50000:60000 -j MARK --set-mark 21
> this seems like it wouldnt work anyway, because its just setting marks on the traffic, dont you need
> some other config to DO something with the marked traffic?
>
>
> regards,
> Jason
>
>
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> Send requests to lvs-users-request@LinuxVirtualServer.org
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
>

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
Re: [lvs-users] cant get passive ftp working through nat for clustered ftp hosts. [ In reply to ]
Hello,

On Sun, 14 Jun 2015, jason@monsterjam.org wrote:

> hey folks, Ive read the howtows for a few days now and cant seem to get this to work.. Ive got a
> centos box running lvs and 2 backend ftp servers running vsftpd. the backend servers are
> set to use passive ports 50000-60000
> my server running lvs, has 2 nics, one on the innernets and one private. the two ftp servers are on the
> private network. the 2 ftp servers have
> pasv_address=x.x.x.x
> where x.x.x.x is the outside (internet facing) ip address of my lvs server..
> my lvs server is doing the following
> export realip=(outside ip address of my server)
> ipvsadm -A -t $realip:21 -s wrr
> ipvsadm -a -t $realip:21 -r 10.1.6.11 -m
> ipvsadm -a -t $realip:21 -r 10.1.6.12 -m
>
> 10.1.6.11=vsftp server 1
> 10.1.6.12=vsftp server 2
>
> sooo from a host on the outside, I can connect to my lvs server's outside ip address on port 21
> and if Im using active mode ftp, I can list directories and see files and whatnot..
> If I use passive mode, it just hangs..

Check if INPUT firewall rules allow the passive data
traffic. For example, such rules may help:

# Accept FTP DATA (related) and FTP CONTROL (established) traffic:
iptables -A INPUT -p tcp -d $VIP -m state --state RELATED,ESTABLISHED -j ACCEPT
# Accept FTP CONTROL:
iptables -A INPUT -p tcp -d $VIP --dport 21 -m state --state NEW -j ACCEPT

> but when I sniff the ftp connection on host1 (lvs server), I see the following:
> 20:21:41.928714 IP myclienthost.org.44588 > mylvsserver.58374: Flags [S], seq 3921685969, win 14600, options [mss 1460,sackOK,TS val 3671275663 ecr 0,nop,wscale 6], length 0
> 20:21:43.928811 IP myclienthost.org.44588 > mylvsserver.58374: Flags [S], seq 3921685969, win 14600, options [mss 1460,sackOK,TS val 3671277663 ecr 0,nop,wscale 6], length 0

No ACK packets? May be SYNs are dropped by firewall?

Regards

--
Julian Anastasov <ja@ssi.bg>

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
Re: [lvs-users] cant get passive ftp working through nat for clustered ftp hosts. [ In reply to ]
> > hey folks,

Hello Jason,

> > export realip=(outside ip address of my server)
> > ipvsadm -A -t $realip:21 -s wrr
> > ipvsadm -a -t $realip:21 -r 10.1.6.11 -m
> > ipvsadm -a -t $realip:21 -r 10.1.6.12 -m
> >
> > 10.1.6.11=vsftp server 1
> > 10.1.6.12=vsftp server 2

That setup reads as an plan for LVS-NAT ... but with incompleted LVS-NAT rules.
You need to cover ALL configured vsftpd server PORTS on the LVS balancer nodes - otherwise you'll never see an SYNC nor an ACK :)

with LVS-DR the vsftpd can give direct answers , on LVS-NAT you have to take care of the configured redirected ports very carefully.

You probably also need the ip_vs_ftp kernel modules..

# lsmod | grep ftp
ip_vs_ftp 6731 0
nf_nat 16229 1 ip_vs_ftp
ip_vs 157311 6 ip_vs_ftp,ip_vs_rr


so it shuold be i.e :
>
> > > ipvsadm -A -t $realip:20-21 -s wrr
> > > ipvsadm -a -t $realip:20-21 -r 10.1.6.11 -m
> > > ipvsadm -a -t $realip:20-21 -r 10.1.6.12 -m
>
> > > ipvsadm -A -t $realip:50000-60000 -s wrr
> > > ipvsadm -a -t $realip:50000-60000 -r 10.1.6.11 -m
> > > ipvsadm -a -t $realip:50000-60000 -r 10.1.6.12 -m
>


if still issuies , then install wireshark on lvs , that will extactly tell you what been happen on your LVS Nodes network traffiik.


For passv vsftpd you must configure like this : http://splatdot.com/running-vsftpd-behind-a-nat-firewall/
and let the ports 20/21 + passvive ftp ports traffic pass your lvs-nat setup.

if you want passiv sftpd you need as well some other standard ports for that , i.e 115 .

> > Check if INPUT firewall rules allow the passive data
> > traffic. For example, such rules may help:
> >
> > # Accept FTP DATA (related) and FTP CONTROL (established) traffic:
> > iptables -A INPUT -p tcp -d $VIP -m state --state RELATED,ESTABLISHED -j ACCEPT
> > # Accept FTP CONTROL:
> > iptables -A INPUT -p tcp -d $VIP --dport 21 -m state --state NEW -j ACCEPT



I whuold suggest to use i.e keepalived to manage your LVS-NAT FTP port Rules and the IP failover .
..take care of port & NAT routing as Junian suggested.
- https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/3/html/Cluster_Administration/s1-lvs-ftp.html
- http://keepalived.org/LVS-NAT-Keepalived-HOWTO.html


Hope this helps.


--
Mit freundlichen Grüßen / Best Regards

Horst Venzke ; PGP NET : 1024G/082F2E6D ; http://www.remsnet.de - 1995 - 2015 - 20 Jahre Linux/Unix Support.

Legal Notice: This transmittal and/or attachments may be privileged or confidential. It is intended solely for the addressee named above. Any review, dissemination, or copying is strictly prohibited. If you received this transmittal in error, please notify us immediately by reply and immediately delete this message and all
Re: [lvs-users] cant get passive ftp working through nat for clustered ftp hosts. [ In reply to ]
ok thanks, ill try that out.

Jason

On Sun, Jun 28, 2015 at 12:35:49PM +0200, Horst.venzke@remsnet.de , Horst Venzke ,Remsnet Consulting und Internet Services LTD, Niederlassung Deutschland wrote:
> > > hey folks,
>
> Hello Jason,
>
>
> That setup reads as an plan for LVS-NAT ... but with incompleted LVS-NAT rules.
> You need to cover ALL configured vsftpd server PORTS on the LVS balancer nodes - otherwise you'll never see an SYNC nor an ACK :)
>
> with LVS-DR the vsftpd can give direct answers , on LVS-NAT you have to take care of the configured redirected ports very carefully.
>
> You probably also need the ip_vs_ftp kernel modules..
>
> # lsmod | grep ftp
> ip_vs_ftp 6731 0
> nf_nat 16229 1 ip_vs_ftp
> ip_vs 157311 6 ip_vs_ftp,ip_vs_rr
>
>
> so it shuold be i.e :
> >
> > > > ipvsadm -A -t $realip:20-21 -s wrr
> > > > ipvsadm -a -t $realip:20-21 -r 10.1.6.11 -m
> > > > ipvsadm -a -t $realip:20-21 -r 10.1.6.12 -m
> >
> > > > ipvsadm -A -t $realip:50000-60000 -s wrr
> > > > ipvsadm -a -t $realip:50000-60000 -r 10.1.6.11 -m
> > > > ipvsadm -a -t $realip:50000-60000 -r 10.1.6.12 -m
> >
>
>
> if still issuies , then install wireshark on lvs , that will extactly tell you want been happen on your LVS Nodes network traffiik.
>
>
> For passv vsftpd you must configure like this : http://splatdot.com/running-vsftpd-behind-a-nat-firewall/
> and let the ports 20/21 + passvive ftp ports traffic pass your lvs-nat setup.
>
> if you want passiv sftpd you need as well some other standard ports for that , i.e 115 .
>
> > > Check if INPUT firewall rules allow the passive data
> > > traffic. For example, such rules may help:
> > >
> > > # Accept FTP DATA (related) and FTP CONTROL (established) traffic:
> > > iptables -A INPUT -p tcp -d $VIP -m state --state RELATED,ESTABLISHED -j ACCEPT
> > > # Accept FTP CONTROL:
> > > iptables -A INPUT -p tcp -d $VIP --dport 21 -m state --state NEW -j ACCEPT
>
>
>
> I whuold suggest to use i.e keepalived to manage your LVS-NAT FTP port Rules and the IP failover .
> ..take care of port & NAT routing !!
> - https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/3/html/Cluster_Administration/s1-lvs-ftp.html
> - http://keepalived.org/LVS-NAT-Keepalived-HOWTO.html
>
> Keepalive can also handel the required virtual gateway IP on the private network well.
>
>
> Hope this helps.
>
>
> --
> Mit freundlichen Grüßen / Best Regards
>
> Horst Venzke ; PGP NET : 1024G/082F2E6D ; http://www.remsnet.de - 1995 - 2015 - 20 Jahre Linux/Unix Support.
>
> Legal Notice: This transmittal and/or attachments may be privileged or confidential. It is intended solely for the addressee named above. Any review, dissemination, or copying is strictly prohibited. If you received this transmittal in error, please notify us immediately by reply and immediately delete this message and all





_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users