Mailing List Archive

[lvs-users] FTP data port connection not closing?
Hi,

I've set up LVS-DR - via ldirectord, fired up by heartbeat - for my
cluster and I've got it serving up HTTP / HTTPS just fine. So I moved
onto FTP and it seems to work, except that when uploading files, it gets
stuck at 100% on FileZilla (but also the same on other FTP clients and
via the command line), and then timeouts.

The interesting thing is that if I abort and then check the remote
directory, the file is actually there.

Either in its entirety or it's a few bytes / kilobytes short (and,
often, that appears to be a power of two short - that is, if the file
was 36,000 bytes then the file on the server is 32,769 bytes). This
looks to me like a disk caching thing - as the connection isn't closed,
the file isn't closed and it's not always fully writing back all the data.

The file on the server is also, up to the point where it cuts off,
perfectly correct when I test with a text file to be able to see that,
yes, the data is being transferred and transferred correctly.

With FTP, if I understand things correctly, the data connection itself
is used as end-of-file. When the file has been transferred, the client
just closes the connection and this signals that all the data has been
transferred. And, from what I see happening, this is what's seemingly
not happening here.

The data connection is remaining open, but no more data is being sent
and the server sits just there waiting for an EOF that never comes.
Until it times out. But, other than that, everything else appears to be
functioning correctly - including the file transfers themselves, as data
is reaching the server because the files are there.

In ldirectord, I've got this for the FTP:

--- >8 ---

virtual=21
service = ftp
scheduler = wlc
protocol = fwm
checktype = connect
real = [hostname of FTP server] gate

--- 8< ---

I'm using firewall marks, simply because passive FTP uses a lot of
ports. Currently, I've only got a single FTP server in the cluster, so
really all LVS is doing is passing the packets on (I might well up the
number of FTP servers in the cluster later, but currently I'm focusing
on just getting it working first, then I'll expand later). This is
LVS-DR, so the realserver replies directly to the client. But the client
only knows the external VIP of the cluster and sends replies back, so
the passive ports are firewall marked to be sent to the FTP server too.

I'm adding the firewall marks to the packets by adding these rules to
"before.rules" in UFW:

--- >8 ---

*mangle
:PREROUTING - [0:0]

-A PREROUTING -p tcp -d 192.168.0.99/32 --dport 21 -j MARK --set-mark 21
-A PREROUTING -p tcp -d 192.168.0.99/32 --dport 20000:21000 -j MARK
--set-mark 21

COMMIT

--- 8< ---

Where "192.168.0.99" is the VIP of the cluster.

I've configured my FTP server to report the external IP address of the
cluster and to restrict itself to the passive ports 20000-21000. I know
this works correctly, as I can see that the "entering PASV
(x,x,x,x,p,p)" response has the right IP and is always within the
passive port range.

Indeed, everything otherwise seems to function correctly. The data
connection is being made and the files are being sent (and in terms of
downloads, directory listing, deleting files, TLS authentication and all
the rest of it, this all works 100%). But as the data connection appears
not to close, then it just gets stuck at 100% until it times out with an
error message.

I've also, of course, tried connecting directly to the FTP server on the
LAN - without LVS being involved - and everything works 100%.

What seems to be happening is that LVS isn't passing onto the realserver
the fact that the data connection has closed. But with FTP, it is
necessary for this to be sent, as that's how EOF is signalled to the server.

Please help, if you can. Everything else with the server is good, so
it's just this little glitch holding everything up.

Thanks.

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
Re: [lvs-users] FTP data port connection not closing? [ In reply to ]
Hello,

On Wed, 23 Aug 2017, Owain Jones wrote:

> Hi,
>
> I've set up LVS-DR - via ldirectord, fired up by heartbeat - for my
> cluster and I've got it serving up HTTP / HTTPS just fine. So I moved
> onto FTP and it seems to work, except that when uploading files, it gets
> stuck at 100% on FileZilla (but also the same on other FTP clients and
> via the command line), and then timeouts.
>
> The interesting thing is that if I abort and then check the remote
> directory, the file is actually there.
>
> Either in its entirety or it's a few bytes / kilobytes short (and,
> often, that appears to be a power of two short - that is, if the file
> was 36,000 bytes then the file on the server is 32,769 bytes). This
> looks to me like a disk caching thing - as the connection isn't closed,
> the file isn't closed and it's not always fully writing back all the data.
>
> The file on the server is also, up to the point where it cuts off,
> perfectly correct when I test with a text file to be able to see that,
> yes, the data is being transferred and transferred correctly.
>
> With FTP, if I understand things correctly, the data connection itself
> is used as end-of-file. When the file has been transferred, the client
> just closes the connection and this signals that all the data has been
> transferred. And, from what I see happening, this is what's seemingly
> not happening here.
>
> The data connection is remaining open, but no more data is being sent
> and the server sits just there waiting for an EOF that never comes.
> Until it times out. But, other than that, everything else appears to be
> functioning correctly - including the file transfers themselves, as data
> is reaching the server because the files are there.
>
> In ldirectord, I've got this for the FTP:
>
> --- >8 ---
>
> virtual=21
> service = ftp
> scheduler = wlc
> protocol = fwm
> checktype = connect
> real = [hostname of FTP server] gate
>
> --- 8< ---
>
> I'm using firewall marks, simply because passive FTP uses a lot of
> ports. Currently, I've only got a single FTP server in the cluster, so
> really all LVS is doing is passing the packets on (I might well up the
> number of FTP servers in the cluster later, but currently I'm focusing
> on just getting it working first, then I'll expand later). This is

When you add more real servers you will need to use -p to
enable persistence, ipvsadm man page explains this for FTP.

> LVS-DR, so the realserver replies directly to the client. But the client
> only knows the external VIP of the cluster and sends replies back, so
> the passive ports are firewall marked to be sent to the FTP server too.
>
> I'm adding the firewall marks to the packets by adding these rules to
> "before.rules" in UFW:
>
> --- >8 ---
>
> *mangle
> :PREROUTING - [0:0]
>
> -A PREROUTING -p tcp -d 192.168.0.99/32 --dport 21 -j MARK --set-mark 21
> -A PREROUTING -p tcp -d 192.168.0.99/32 --dport 20000:21000 -j MARK
> --set-mark 21
>
> COMMIT
>
> --- 8< ---
>
> Where "192.168.0.99" is the VIP of the cluster.
>
> I've configured my FTP server to report the external IP address of the
> cluster and to restrict itself to the passive ports 20000-21000. I know
> this works correctly, as I can see that the "entering PASV
> (x,x,x,x,p,p)" response has the right IP and is always within the
> passive port range.
>
> Indeed, everything otherwise seems to function correctly. The data
> connection is being made and the files are being sent (and in terms of
> downloads, directory listing, deleting files, TLS authentication and all
> the rest of it, this all works 100%). But as the data connection appears
> not to close, then it just gets stuck at 100% until it times out with an
> error message.
>
> I've also, of course, tried connecting directly to the FTP server on the
> LAN - without LVS being involved - and everything works 100%.
>
> What seems to be happening is that LVS isn't passing onto the realserver
> the fact that the data connection has closed. But with FTP, it is
> necessary for this to be sent, as that's how EOF is signalled to the server.

What shows 'ipvsadm -Lnc' when connection stucks ? What is
shown for the command (:21) and data (20xxx data port) connection?

IPVS can see only the packets from client, so any
FIN bit we see is the client's wish to half-close the TCP
connection. While there is existing connection entry, IPVS
should not stop the traffic. But if the transfer is very long
and short TCP EST timeout is used (ipvsadm --set TCP ...)
the command connection can expire. If persistence is used
this can not happen because the data connections bump a
reference count that keeps the command connection.

In any case, tcpdump -lnnnv 'host CLIENT_IP' output
on director would be useful, even if only for the last packets
before connections stucks. You can run it both on
incoming (from client) and outgoing (to real server)
interface, so that we can see if some packets are not
forwarded.

Also, what is the kernel version?: uname -a

> Please help, if you can. Everything else with the server is good, so
> it's just this little glitch holding everything up.
>
> Thanks.

Regards

--
Julian Anastasov <ja@ssi.bg>

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
Re: [lvs-users] FTP data port connection not closing? [ In reply to ]
On 23/08/2017 16:32, Julian Anastasov wrote:
> When you add more real servers you will need to use -p to enable
> persistence, ipvsadm man page explains this for FTP.
Yeah, I have persistence enabled on the HTTP / HTTPS and other services,
but I didn't bother here simply because there's only a single FTP server
at the moment.

> What shows 'ipvsadm -Lnc' when connection stucks ? What is shown for
> the command (:21) and data (20xxx data port) connection? IPVS can see
> only the packets from client, so any FIN bit we see is the client's
> wish to half-close the TCP connection. While there is existing
> connection entry, IPVS should not stop the traffic. But if the
> transfer is very long and short TCP EST timeout is used (ipvsadm --set
> TCP ...) the command connection can expire. If persistence is used
> this can not happen because the data connections bump a reference
> count that keeps the command connection. In any case, tcpdump -lnnnv
> 'host CLIENT_IP' output on director would be useful, even if only for
> the last packets before connections stucks. You can run it both on
> incoming (from client) and outgoing (to real server) interface, so
> that we can see if some packets are not forwarded. Also, what is the
> kernel version?: uname -a

It's kernel version 4.4.0-92-generic on Ubuntu Server 16.04.

(And in "/etc/modules", I've got "bonding" - as all the inter-server
links are bonded for HA purposes - and the whole list of ip_vs modules,
which "lsmod" confirms are loaded.)

I've tried "watch ipvsadm -Lnc" as it got stuck and everything looks
normal. An "ESTABLISHED" TCP connection from the router to port 21 of
the virtual and then routed to the destination of port 21 on the FTP
server. And the same thing with another TCP connection but with a port
in my passive port range. What I'd expect to see.

I did try using tshark on the director, filtering for the VIP and ports
20000 to 21000 (my passive port range). When it gets stuck, what I'm
seeing is:

192.68.0.1 [router IP] -> 192.168.0.99 [VIP] TCP 85 [TCP
Retransmission] -> 20103 [ FIN, PSH, ACK ]

And this is repeating over and over.

So what I'm interpreting this as being is the client signalling to the
VIP that it wants to close the connection (FIN) and, being TCP, there
should be an ACKnowledgement from the server that it received this. But
as that ACK ain't coming, then it's repeatedly retransmitting the
passive port "close connection" to a server that isn't responding.

And, looking on the FTP server, when it's stuck, I'm seeing this
repeating over and over:

192.168.0.99 (VIP) -> 192.168.0.1 (router) TCP 85 [TCP Retransmission]
20197 -> 51311 [ FIN, PSH, ACK ]

So the server is getting it and trying to respond, but it's not getting
back to the client.

Hmm, on the realserver, I've got a dummy interface with the VIP and, to
combat the ARP problem, have used the following rules in my
/etc/network/interfaces:

---- 8< ----

auto dummy0
iface dummy0 inet static
address 192.168.0.99
netmask 255.255.255.0
pre-up arptables -F
pre-up arptables -A INPUT -d 192.168.0.99 -j DROP
pre-up arptables -A OUTPUT -s 192.168.0.99 -j mangle
--mangle-ip-s [external IP address]

---- >8 ----

So that any ARP requests for the VIP are ignored by the realserver and
if it ARPs the VIP itself then it's changed to the external IP address
to redirect it to the right entry point into the cluster.

I preferred to use "arptables" to explicitly control the ARP problem. It
just felt less messy to do it that way to me.

Looking at these results, I think LVS is actually delivering the close
connection to the FTP server but the response is not getting back to the
client. So my configuration of "dummy0" that carries the VIP is possibly
wrong somehow?

Regards,
Owain

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
Re: [lvs-users] FTP data port connection not closing? [ In reply to ]
Hello,

On Thu, 24 Aug 2017, Owain Jones wrote:

> I've tried "watch ipvsadm -Lnc" as it got stuck and everything looks normal.
> An "ESTABLISHED" TCP connection from the router to port 21 of the virtual and
> then routed to the destination of port 21 on the FTP server. And the same
> thing with another TCP connection but with a port in my passive port range.
> What I'd expect to see.
>
> I did try using tshark on the director, filtering for the VIP and ports 20000
> to 21000 (my passive port range). When it gets stuck, what I'm seeing is:
>
> 192.68.0.1 [router IP] -> 192.168.0.99 [VIP] TCP 85 [TCP Retransmission] ->
> 20103 [ FIN, PSH, ACK ]
>
> And this is repeating over and over.
>
> So what I'm interpreting this as being is the client signalling to the VIP
> that it wants to close the connection (FIN) and, being TCP, there should be an
> ACKnowledgement from the server that it received this. But as that ACK ain't
> coming, then it's repeatedly retransmitting the passive port "close
> connection" to a server that isn't responding.
>
> And, looking on the FTP server, when it's stuck, I'm seeing this repeating
> over and over:
>
> 192.168.0.99 (VIP) -> 192.168.0.1 (router) TCP 85 [TCP Retransmission] 20197
> -> 51311 [ FIN, PSH, ACK ]
>
> So the server is getting it and trying to respond, but it's not getting back
> to the client.
>
> Hmm, on the realserver, I've got a dummy interface with the VIP and, to combat
> the ARP problem, have used the following rules in my /etc/network/interfaces:
>
> ---- 8< ----
>
> auto dummy0
> iface dummy0 inet static
> address 192.168.0.99
> netmask 255.255.255.0
> pre-up arptables -F
> pre-up arptables -A INPUT -d 192.168.0.99 -j DROP
> pre-up arptables -A OUTPUT -s 192.168.0.99 -j mangle
> --mangle-ip-s [external IP address]
>
> ---- >8 ----
>
> So that any ARP requests for the VIP are ignored by the realserver and if it
> ARPs the VIP itself then it's changed to the external IP address to redirect
> it to the right entry point into the cluster.
>
> I preferred to use "arptables" to explicitly control the ARP problem. It just
> felt less messy to do it that way to me.
>
> Looking at these results, I think LVS is actually delivering the close
> connection to the FTP server but the response is not getting back to the
> client. So my configuration of "dummy0" that carries the VIP is possibly wrong
> somehow?

ARP filtering settings should be ok. They should not be a
reason connection to stop every time just before the end of file.
May be you can check for checksum errors, the -v in tcpdump -lnnnv
should report if real server receives corrupted packets from
real server. Sometimes bad-csum is reported for transmitted
packets, eg. when their checksum is to-be-calculated by chip.
This should be ignored. So, just make sure the packets received in
director (by client) and real server (by director) have valid csum.

You can also check if there is conntrack entry in
the real server, its state. I don't expect transfer to stop
just because there is a FIN bit, the retransmitted packets
still have the chance to fill the gaps.

Regards

--
Julian Anastasov <ja@ssi.bg>

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
Re: [lvs-users] FTP data port connection not closing? [ In reply to ]
On 24/08/2017 12:14, Julian Anastasov wrote:

> ARP filtering settings should be ok. They should not be a
> reason connection to stop every time just before the end of file.
> May be you can check for checksum errors, the -v in tcpdump -lnnnv
> should report if real server receives corrupted packets from
> real server. Sometimes bad-csum is reported for transmitted
> packets, eg. when their checksum is to-be-calculated by chip.
> This should be ignored. So, just make sure the packets received in
> director (by client) and real server (by director) have valid csum.
>
Okay, I used tcpdump with -v, filtering on the VIP in the passive port
range (20000-21000).

On the director, all the packets have correct checksums.

But, on the realserver, from the VIP to the router IP, the checksums are
all coming up as incorrect.

What does this mean?

Regards,
Owain

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
Re: [lvs-users] FTP data port connection not closing? [ In reply to ]
As an addendum, I just used "ethtool" in order to turn off NIC-based
checksumming on the bonded interface and then tried tcpdump again on the
FTP real server. This time, all the checksums came back as "correct".

So the checksums are correct. I was suspicious that all of the outgoing
packets had bad checksums - particularly seeing as port 21 also had
this, yet the FTP commands besides uploading work fine - so I thought
I'd remove the possibility of it being NIC-based checksumming causing it
and see if there's any packets left as "incorrect" afterwards ("grep
incorrect").

There isn't. All the checksums are fine.

Regards,
Owain

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
Re: [lvs-users] FTP data port connection not closing? [ In reply to ]
Hi,

The packets seem to be dying at the router. As I can see the packets
being received on the director and the response packets being sent from
the real server.

One thing I'm thinking of, that I failed to mention earlier, is that the
router does NAT. I've placed the VIP in the DMZ, so the director should
be receiving all external packets directly. But the actual machines
themselves are in the router's LAN and being NAT'ed.

As I'm using LVS-DR, then the only thing that should be being changed in
the incoming packet is the MAC address, yes? But then, when the real
server responds, it'll have a different MAC address to the incoming
packet because it's actually a physically different machine.

So my thought is, could this MAC address mismatch be possibly confusing
the router's NATting?

I guess I could test it by rewriting the MAC address on outgoing packets
from the real server to have the MAC of the director, so that, from the
router's perspective, the LVS is entirely transparent.

Though surely, that said, the source MAC address on outgoing packets
shouldn't really matter, I'd have thought.

Regards,
Owain

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
Re: [lvs-users] FTP data port connection not closing? [ In reply to ]
Okay, now we're getting somewhere interesting.

Looking at the pure-ftpd verbose log, it shows a connection from
192.168.0.1 (the router). Then 192.168.0.1 logs in - so it's
user@192.168.0.1 - and it issues a few commands - "opts", "pbsz",
"prot", "pwd" - then it says:

(?@192.168.0.100) New connection from 192.168.0.100
(?@192.168.0.100) Logout

And it has this a few times in a row.

The thing is, 192.168.0.100 is the DIP. 192.168.0.99 is the VIP.
192.168.0.1 is the router IP (which is NAT'ing for the client, so
effectively the CIP).

What seems to be happening is that it's correctly connecting to the FTP
server initially, but then it's wrongly connecting as the director with
the DIP half-way through.

This is probably, based on when it's happening, when the client is
making a connection to the passive port range. But, the thing is, for
that even be reaching the FTP server, the firewall marks must be
working, as that's what LVS is using to redirect to the FTP server.

But it hasn't got the correct IP address.

Why's this happening? And how would I fix it?

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
Re: [lvs-users] FTP data port connection not closing? [ In reply to ]
No, wait, sorry.

The connections from 192.168.0.100 are just the LVS healthchecks,
ensuring that the real server is still alive.

Ignore that last post.


_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
Re: [lvs-users] FTP data port connection not closing? [ In reply to ]
Hello,

On Tue, 29 Aug 2017, Owain Jones wrote:

> Hi,
>
> The packets seem to be dying at the router. As I can see the packets being
> received on the director and the response packets being sent from the real
> server.
>
> One thing I'm thinking of, that I failed to mention earlier, is that the
> router does NAT. I've placed the VIP in the DMZ, so the director should be
> receiving all external packets directly. But the actual machines themselves
> are in the router's LAN and being NAT'ed.
>
> As I'm using LVS-DR, then the only thing that should be being changed in the
> incoming packet is the MAC address, yes? But then, when the real server
> responds, it'll have a different MAC address to the incoming packet because
> it's actually a physically different machine.
>
> So my thought is, could this MAC address mismatch be possibly confusing the
> router's NATting?

The MAC usually does not play. You can also check the state
of conntrack entries in router, if possible. But to be sure that it
is not the router, you can start client connection from some box
on the LAN, then the real server will talk directly with this
client box.

> I guess I could test it by rewriting the MAC address on outgoing packets from
> the real server to have the MAC of the director, so that, from the router's
> perspective, the LVS is entirely transparent.

Almost, Linux 4.10+ decrements the IP TTL field for all
forwarding methods including DR.

> Though surely, that said, the source MAC address on outgoing packets shouldn't
> really matter, I'd have thought.

Yep

Regards

--
Julian Anastasov <ja@ssi.bg>

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
Re: [lvs-users] FTP data port connection not closing? [ In reply to ]
On 29/08/2017 19:53, Julian Anastasov wrote:
> Hello,
>
> On Tue, 29 Aug 2017, Owain Jones wrote:
>
>> Hi,
>>
>> The packets seem to be dying at the router. As I can see the packets being
>> received on the director and the response packets being sent from the real
>> server.
>>
>> One thing I'm thinking of, that I failed to mention earlier, is that the
>> router does NAT. I've placed the VIP in the DMZ, so the director should be
>> receiving all external packets directly. But the actual machines themselves
>> are in the router's LAN and being NAT'ed.
>>
>> As I'm using LVS-DR, then the only thing that should be being changed in the
>> incoming packet is the MAC address, yes? But then, when the real server
>> responds, it'll have a different MAC address to the incoming packet because
>> it's actually a physically different machine.
>>
>> So my thought is, could this MAC address mismatch be possibly confusing the
>> router's NATting?
> The MAC usually does not play. You can also check the state
> of conntrack entries in router, if possible.
Not possible, I'm afraid.

The router is an ISP-provided thing which doesn't provide access to that
sort of thing.

> But to be sure that it
> is not the router, you can start client connection from some box
> on the LAN, then the real server will talk directly with this
> client box.
The FTP server is configured to report the external IP address.

But if I change that to the internal IP address and connect to it from a
client box within the LAN, then everything works perfectly. No issues.
So the FTP server itself works.

And using tcpdump to watch the packets, things are failing when the real
server is trying to communicate back to the client, to ACK the
half-close and close the connection (and, behaviourally, that fits
because data is being sent to the server, it's just that the connection
is not closing - so, eventually, it times out and sometimes a few bytes
are missing at the end of the file, but I'm thinking that's just caching
and if the connection did close then it would write back the whole lot
on receipt of EOF, it's only shortened because I'm having to abort the
connection manually).

>> I guess I could test it by rewriting the MAC address on outgoing packets from
>> the real server to have the MAC of the director, so that, from the router's
>> perspective, the LVS is entirely transparent.
> Almost, Linux 4.10+ decrements the IP TTL field for all
> forwarding methods including DR.
>
>> Though surely, that said, the source MAC address on outgoing packets shouldn't
>> really matter, I'd have thought.
> Yep
Yeah, I thought that the source MAC address really ought to do nothing.
But the only reason I'm thinking of this is because things start failing
when the real server is passing its response to the client's FIN packet
on the passive port.

The router "owns" the LAN, by the way, and is providing the NAT. So my
thought was that, though I can't access it, it's got connection tracking
and maybe, as the source MAC coming out of the real server is different
to the dest MAC that came in on the VIP, that the router's conntrack is
not recognising the response as being connected to the request.
Confusing it.

But that's just a guess. I honestly wish I had some clue why this was
happening.

Regards,
Owain

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
Re: [lvs-users] FTP data port connection not closing? [ In reply to ]
Hello,

On Wed, 30 Aug 2017, Owain Jones wrote:

> > But to be sure that it
> > is not the router, you can start client connection from some box
> > on the LAN, then the real server will talk directly with this
> > client box.
> The FTP server is configured to report the external IP address.
>
> But if I change that to the internal IP address and connect to it from a
> client box within the LAN, then everything works perfectly. No issues. So the
> FTP server itself works.

The idea is to test connection from LAN client
via IPVS director, not directly from client to FTP server.

Regards

--
Julian Anastasov <ja@ssi.bg>

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users