Advanced
Mailing List Archive

Mailing List Archive

  1. Home >
  2. Linux Virtual Server>
  3. Users
[lvs-users] IPVSadm and passive FTP bug on CentOS 7/Debian 8, works fine on CentOS 5
mischa.verheul at sentia
Dec 2, 2016, 4:28 AM
Post #1 of 2 (1252 views)
Permalink
Hello,

I'm trying to set up an IPVS NAT loadbalance solution with failover through
keepalived and sync through "ipvsadm --start-daemon" on CentOS 7.
Protocols such as HTTP work as expected, but FTP in passive mode isn't
working for the FTP-data connection. It stops after issueing a list or get
command.

After I couldn't find out what was possibly wrong with my configuration
after following documentation, I set up two basic IPVS setups for
trial&error:
Client -> IPVSmachine -> FTPserver
192.168.2.6 2.100/157.254 192.168.157.10/GW
192.168.157.254

CentOS 5: uname -r 2.6.18-398.el5
CentOS 7: uname -r 3.10.0-327.36.3.el7.x86_64

Configuration on both IPVS machines is identical:
ipvsadm -A -t 192.168.2.100:21 -s rr
ipvsadm -a -t 192.168.2.100:21 -r 192.168.157.10:21 -m
net.ipv4.ip_forward=1
modprobe ip_vs_ftp
no iptables rules

[root@cent5 ~]# lsmod | grep -i ftp
Module Size Used by
ip_vs_ftp 39109 1
ip_vs 122241 5 ip_vs_ftp,ip_vs_rr


[root@cent7 ~]# lsmod | grep -i ftp
Module Size Used by
ip_vs_ftp 13079 0
nf_nat 26146 1 ip_vs_ftp
ip_vs 140944 5 ip_vs_rr,ip_vs_ftp

CentOS 5, with ip_vs_ftp loaded, FTP in passive mode works as expected. The
module correctly translates the IP from the real server to that of the IPVS
machine in the "227 Entering Passive Mode (x,x,x,x,.,.)" line.
For some reason I can't figure out, this doesn't happen on the latest
CentOS 7 version (same results on the latest Debian 8).

The IPVS machine on CentOS 7 never translates the IP in the "227 Entering
Passive Mode" line and never sends it back to the client.

Does anyone know what is causing this bug to occur and/or how to solve it?
Perhaps it isn't a bug and something between versions changed and I haven't
been able to find out?

Below are the capture dumps for the client->IPVS machine->FTP server. The
first dumps are for version 5, where it works, the second dump are the
dumps for version 7.

CENTOS5 SETUP:
------------------------------------------------------------------------------------
CLIENT
No. Time Source Destination Protocol
Length Info
71 7.283164 192.168.2.6 192.168.2.100
FTP 74 Request: PASV
73 7.284789 192.168.2.100 192.168.2.6
FTP 119 Response: 227 Entering Passive Mode (192,168,2,100,185,100).
80 7.286195 192.168.2.6 192.168.2.100
FTP 74 Request: LIST
81 7.287193 192.168.2.100 192.168.2.6
FTP 107 Response: 150 Here comes the directory listing.
82 7.287456 192.168.2.100 192.168.2.6
FTP-DATA 339 FTP Data: 271 bytes
89 7.288480 192.168.2.100 192.168.2.6
FTP 92 Response: 226 Directory send OK.
107 9.630096 192.168.2.6 192.168.2.100
FTP 74 Request: QUIT
109 9.634286 192.168.2.100 192.168.2.6
FTP 82 Response: 221 Goodbye.


IPVS CENTOS5
No. Time Source Destination Protocol
Length Info
39 8.234350 192.168.2.6 192.168.2.100
FTP 74 Request: PASV
40 8.234390 192.168.2.6 192.168.157.10 FTP
74 Request: PASV
41 8.235294 192.168.157.10 192.168.2.6 FTP
121 Response: 227 Entering Passive Mode (192,168,157,10,185,100).
42 8.235336 192.168.2.100 192.168.2.6
FTP 119 Response: 227 Entering Passive Mode (192,168,2,100,185,100).
51 8.236985 192.168.2.6 192.168.2.100
FTP 74 Request: LIST
52 8.237031 192.168.2.6 192.168.157.10 FTP
74 Request: LIST
53 8.237753 192.168.157.10 192.168.2.6 FTP
107 Response: 150 Here comes the directory listing.
54 8.237770 192.168.2.100 192.168.2.6
FTP 107 Response: 150 Here comes the directory listing.
55 8.237995 192.168.157.10 192.168.2.6 FTP-DATA
339 FTP Data: 271 bytes
56 8.238022 192.168.2.100 192.168.2.6
FTP-DATA 339 FTP Data: 271 bytes
65 8.238914 192.168.157.10 192.168.2.6 FTP
92 Response: 226 Directory send OK.
66 8.238937 192.168.2.100 192.168.2.6
FTP 92 Response: 226 Directory send OK.
69 10.581285 192.168.2.6 192.168.2.100
FTP 74 Request: QUIT
70 10.581332 192.168.2.6 192.168.157.10 FTP
74 Request: QUIT
71 10.583614 192.168.157.10 192.168.2.6 FTP
82 Response: 221 Goodbye.
72 10.583730 192.168.2.100 192.168.2.6
FTP 82 Response: 221 Goodbye.

FTPSERVER
No. Time Source Destination Protocol
Length Info
19 6.190654 192.168.2.6 192.168.157.10 FTP
74 Request: PASV
20 6.191207 192.168.157.10 192.168.2.6 FTP
121 Response: 227 Entering Passive Mode (192,168,157,10,185,100).
25 6.193210 192.168.2.6 192.168.157.10 FTP
74 Request: LIST
26 6.193646 192.168.157.10 192.168.2.6 FTP
107 Response: 150 Here comes the directory listing.
27 6.193958 192.168.157.10 192.168.2.6 FTP-DATA
339 FTP Data: 271 bytes
32 6.194930 192.168.157.10 192.168.2.6 FTP
92 Response: 226 Directory send OK.
34 8.537715 192.168.2.6 192.168.157.10 FTP
74 Request: QUIT
35 8.537959 192.168.157.10 192.168.2.6 FTP
82 Response: 221 Goodbye.

CENTOS7 SETUP:
------------------------------------------------------------------------------------
CLIENT
No. Time Source Destination Protocol
Length Info
156 12.323031 192.168.2.100 192.168.2.6
FTP 99 Response: 200 Switching to Binary mode.
158 12.323454 192.168.2.6 192.168.2.100
FTP 74 Request: PASV

IPVS CENTOS7
No. Time Source Destination Protocol
Length Info
58 13.679705 192.168.2.6 192.168.2.100
FTP 74 Request: PASV
59 13.679760 192.168.2.6 192.168.157.10 FTP
74 Request: PASV
60 13.680715 192.168.157.10 192.168.2.6 FTP
119 Response: 227 Entering Passive Mode (192,168,157,10,44,65).
62 13.883872 192.168.2.6 192.168.157.10 FTP
74 [TCP Spurious Retransmission] Request: PASV

FTPSERVER
No. Time Source Destination Protocol
Length Info
27 9.044371 192.168.2.6 192.168.157.10 FTP
74 Request: PASV
28 9.044819 192.168.157.10 192.168.2.6 FTP
119 Response: 227 Entering Passive Mode (192,168,157,10,44,65).
29 9.248510 192.168.2.6 192.168.157.10 FTP
74 [TCP Spurious Retransmission] Request: PASV
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
Re: [lvs-users] IPVSadm and passive FTP bug on CentOS 7/Debian 8, works fine on CentOS 5 [ In reply to ]
ja at ssi
Dec 5, 2016, 12:46 AM
Post #2 of 2 (1240 views)
Permalink
Hello,

On Fri, 2 Dec 2016, Mischa Verheul wrote:

> Hello,
>
> I'm trying to set up an IPVS NAT loadbalance solution with failover through
> keepalived and sync through "ipvsadm --start-daemon" on CentOS 7.
> Protocols such as HTTP work as expected, but FTP in passive mode isn't
> working for the FTP-data connection. It stops after issueing a list or get
> command.
>
> After I couldn't find out what was possibly wrong with my configuration
> after following documentation, I set up two basic IPVS setups for
> trial&error:
> Client -> IPVSmachine -> FTPserver
> 192.168.2.6 2.100/157.254 192.168.157.10/GW
> 192.168.157.254
>
> CentOS 5: uname -r 2.6.18-398.el5
> CentOS 7: uname -r 3.10.0-327.36.3.el7.x86_64
>
> Configuration on both IPVS machines is identical:
> ipvsadm -A -t 192.168.2.100:21 -s rr
> ipvsadm -a -t 192.168.2.100:21 -r 192.168.157.10:21 -m
> net.ipv4.ip_forward=1
> modprobe ip_vs_ftp
> no iptables rules
>
> [root@cent5 ~]# lsmod | grep -i ftp
> Module Size Used by
> ip_vs_ftp 39109 1
> ip_vs 122241 5 ip_vs_ftp,ip_vs_rr
>
>
> [root@cent7 ~]# lsmod | grep -i ftp
> Module Size Used by
> ip_vs_ftp 13079 0
> nf_nat 26146 1 ip_vs_ftp
> ip_vs 140944 5 ip_vs_rr,ip_vs_ftp

For the new kernels more modules are needed:

# 2.6.22+
modprobe nf_conntrack_ftp
modprobe nf_nat_ftp

# For ip_vs_ftp (NAT mode) to properly adjust TCP SEQ numbers:
echo 1 > /proc/sys/net/ipv4/netfilter/ip_conntrack_tcp_be_liberal
# or
echo 1 > /proc/sys/net/netfilter/nf_conntrack_tcp_be_liberal

# 2.6.36+, ip_vs_ftp (NAT mode) to properly adjust TCP SEQ numbers:
modprobe iptables_nat

You can find more info (a bit outdated, up to 2.6.36) in
HOWTO.txt here:

http://ja.ssi.bg/nfct/

i.e. you can check the example with iptables rules
in case you have some filtering applied.

Regards

--
Julian Anastasov <ja@ssi.bg>

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

©2024 GT.net. A Gossamer Threads company.
5th Floor, 455 Granville St., Vancouver, BC V6C 1T1, Canada | Legal