Mailing List Archive

Debugging network problems
Hello!

My network was just changed from a vanilla ADSL connection to direct
ftth. There is now a network connector with a 100MB/s entry, which gets
routed to a Buffalo Broad station.

I'm having some troubles and my debugging so far has not been
successful, so I'm hoping some more experienced hands can give me some
advice.


First of all, my previous setup was working exactly as I wanted.
Essentially, when making the switch to the new network, on my
firewall/proxy machine, I just did:

adsl-stop (to stop the pppoe daemon)
ifconfig eth0 new.ip.address up
route add default gw ip.address.of.broad.station

Then in my iptables, I changed:

-A POSTROUTING -o ppp0 -j MASQUERADE

to

-A POSTROUTING -o eth0 -j MASQUERADE


Here's what's happening now...

Generally, I can connect to the outside world, and the outside world can
connect to me. By this, I mean that each of the local machines behind my
proxy can connect.

However, the connections back to my own URL are sporadic. In other
words, sometimes I can connect, sometimes I can't. Assuming my domain is
my.company.com, when I try to connect to my.company.com from within my
network, sometimes I can, sometimes I can't, but I have not at all
figured out a pattern.

When this happens, domain names are being resolved, but I get
"Connection timed out" errors.

I guess I first need to check to see if I can't get out, or I can't get
back in.


Any advice as to how/where I can look for the cause would be greatly
appreciated! I suspect it may have something to do with NAT, but I'm not
experienced at debugging this stuff.


Thanks so much!!!

David
Re: Debugging network problems [ In reply to ]
David Leangen wrote:
> Hello!
>
> My network was just changed from a vanilla ADSL connection to direct
> ftth. There is now a network connector with a 100MB/s entry, which gets
> routed to a Buffalo Broad station.
>
> I'm having some troubles and my debugging so far has not been
> successful, so I'm hoping some more experienced hands can give me some
> advice.
>
>
> First of all, my previous setup was working exactly as I wanted.
> Essentially, when making the switch to the new network, on my
> firewall/proxy machine, I just did:
>
> adsl-stop (to stop the pppoe daemon)
> ifconfig eth0 new.ip.address up
> route add default gw ip.address.of.broad.station
>
> Then in my iptables, I changed:
>
> -A POSTROUTING -o ppp0 -j MASQUERADE
>
> to
>
> -A POSTROUTING -o eth0 -j MASQUERADE
>
>
> Here's what's happening now...
>
> Generally, I can connect to the outside world, and the outside world can
> connect to me. By this, I mean that each of the local machines behind my
> proxy can connect.
>
> However, the connections back to my own URL are sporadic. In other
> words, sometimes I can connect, sometimes I can't. Assuming my domain is
> my.company.com, when I try to connect to my.company.com from within my
> network, sometimes I can, sometimes I can't, but I have not at all
> figured out a pattern.
>
> When this happens, domain names are being resolved, but I get
> "Connection timed out" errors.
>
> I guess I first need to check to see if I can't get out, or I can't get
> back in.
>

Sounds like an PMTUD issue. Do you allow all ESTABLISHED packets in, not
just tcp?

M4
Re: Debugging network problems [ In reply to ]
Thank you, Martijn,

My reply inline.


> > Generally, I can connect to the outside world, and the outside world can
> > connect to me. By this, I mean that each of the local machines behind my
> > proxy can connect.
> >
> > However, the connections back to my own URL are sporadic. In other
> > words, sometimes I can connect, sometimes I can't. Assuming my domain is
> > my.company.com, when I try to connect to my.company.com from within my
> > network, sometimes I can, sometimes I can't, but I have not at all
> > figured out a pattern.
> >
> > When this happens, domain names are being resolved, but I get
> > "Connection timed out" errors.
> >

> Sounds like an PMTUD issue. Do you allow all ESTABLISHED packets in, not
> just tcp?

Yes, I'm letting all packets in:

-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT


This is my iptables file (below).

Maybe somebody can spot the problem?


Cheers,
David



*mangle
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
-A PREROUTING -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG
FIN,PSH,URG -j DROP
-A PREROUTING -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -j
DROP
-A PREROUTING -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -j DROP
-A PREROUTING -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -j DROP
-A PREROUTING -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG
FIN,PSH,URG -j DROP
-A PREROUTING -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -j
DROP
-A PREROUTING -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -j DROP
-A PREROUTING -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -j DROP
-A PREROUTING -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG
FIN,PSH,URG -j DROP
-A PREROUTING -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -j
DROP
-A PREROUTING -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -j DROP
-A PREROUTING -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -j DROP
-A PREROUTING -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG
FIN,PSH,URG -j DROP
-A PREROUTING -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -j
DROP
-A PREROUTING -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -j DROP
-A PREROUTING -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -j DROP
COMMIT

*nat
:PREROUTING ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A PREROUTING -p tcp --dport 5433 -j DNAT --to 192.168.2.10:5432
-A PREROUTING -p udp --dport 5433 -j DNAT --to 192.168.2.10:5432
-A PREROUTING -p tcp --dport 5434 -j DNAT --to 192.168.2.11:5432
-A PREROUTING -p udp --dport 5434 -j DNAT --to 192.168.2.11:5432
-A POSTROUTING -d 192.168.2.10 -s 192.168.0.0/255.255.0.0 -p tcp -j SNAT
--to 192.168.11.100
-A PREROUTING -p tcp -s 202.238.89.88 --dport 50000:50100 -j DNAT --to
192.168.2.5
-A POSTROUTING -o eth0 -j MASQUERADE

COMMIT
*filter
:INPUT DROP [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:BLACKLIST - [0:0]
:LOG_ACCEPT - [0:0]
:LOG_DROP - [0:0]
:icmp_packets - [0:0]
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p tcp -m tcp --dport 20 -j LOG_ACCEPT
-A INPUT -p tcp -m tcp --dport 21 -j LOG_ACCEPT
-A INPUT -p tcp -m tcp --dport 22 -j LOG_ACCEPT
-A INPUT -p tcp -m tcp --dport 25 -j LOG_ACCEPT
-A INPUT -p udp -m udp --dport 53 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 80 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 110 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 143 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 443 -j LOG_ACCEPT
-A INPUT -p tcp -m tcp --dport 993 -j ACCEPT
-A INPUT -p udp -m udp -s 192.168.0.0/255.255.0.0 --dport 10080 -j
ACCEPT
# The following line is for FTP passive ports
-A INPUT -p tcp -m tcp --dport 55000:55500 -j LOG_ACCEPT
-A INPUT -s 127.0.0.1 -j ACCEPT
-A INPUT -p icmp -j icmp_packets
-A INPUT -j LOG_DROP
-A OUTPUT -p tcp -m tcp --dport 50000:50100 -j LOG_ACCEPT
-A OUTPUT -p udp -m udp --dport 700:710 -j LOG_ACCEPT
-A BLACKLIST -j LOG_DROP
-A LOG_ACCEPT -j LOG --log-prefix "[IPTABLES ACCEPT] : "
--log-tcp-options --log-ip-options
-A LOG_ACCEPT -j ACCEPT
-A LOG_DROP -j LOG --log-prefix "[IPTABLES DROP] : " --log-tcp-options
--log-ip-options
-A LOG_DROP -j DROP
-A icmp_packets -p icmp -m icmp --icmp-type 0 -j ACCEPT
-A icmp_packets -s 192.168.0.0/255.255.255.0 -p icmp -m icmp --icmp-type
8 -j ACCEPT
-A icmp_packets -s 192.168.1.0/255.255.255.0 -p icmp -m icmp --icmp-type
8 -j ACCEPT
-A icmp_packets -s 192.168.2.0/255.255.255.0 -p icmp -m icmp --icmp-type
8 -j ACCEPT
-A icmp_packets -p icmp -m icmp --icmp-type 8 -j ACCEPT
-A icmp_packets -p icmp -m icmp --icmp-type 3 -j ACCEPT
-A icmp_packets -p icmp -m icmp --icmp-type 11 -j ACCEPT
COMMIT
Re: Debugging network problems [ In reply to ]
Some more info:

One of my major issues is during svn operations. In the middle of an
operation such svn up, the update starts ok, then at some point, I can
no longer connect to my server.

Each time, it stops at a different file, so that also doesn't tell me
anything about packet sizes or whatever, since I am unable to see any
pattern in all of this.


Any ideas would be greatly appreciated before I lose the little hair I
have left.

:-)




On Fri, 2007-08-31 at 16:43 +0900, David Leangen wrote:
> Thank you, Martijn,
>
> My reply inline.
>
>
> > > Generally, I can connect to the outside world, and the outside world can
> > > connect to me. By this, I mean that each of the local machines behind my
> > > proxy can connect.
> > >
> > > However, the connections back to my own URL are sporadic. In other
> > > words, sometimes I can connect, sometimes I can't. Assuming my domain is
> > > my.company.com, when I try to connect to my.company.com from within my
> > > network, sometimes I can, sometimes I can't, but I have not at all
> > > figured out a pattern.
> > >
> > > When this happens, domain names are being resolved, but I get
> > > "Connection timed out" errors.
> > >
>
> > Sounds like an PMTUD issue. Do you allow all ESTABLISHED packets in, not
> > just tcp?
>
> Yes, I'm letting all packets in:
>
> -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
>
>
> This is my iptables file (below).
>
> Maybe somebody can spot the problem?
>
>
> Cheers,
> David
>
>
>
> *mangle
> :PREROUTING ACCEPT [0:0]
> :INPUT ACCEPT [0:0]
> :FORWARD ACCEPT [0:0]
> :OUTPUT ACCEPT [0:0]
> :POSTROUTING ACCEPT [0:0]
> -A PREROUTING -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG
> FIN,PSH,URG -j DROP
> -A PREROUTING -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -j
> DROP
> -A PREROUTING -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -j DROP
> -A PREROUTING -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -j DROP
> -A PREROUTING -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG
> FIN,PSH,URG -j DROP
> -A PREROUTING -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -j
> DROP
> -A PREROUTING -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -j DROP
> -A PREROUTING -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -j DROP
> -A PREROUTING -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG
> FIN,PSH,URG -j DROP
> -A PREROUTING -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -j
> DROP
> -A PREROUTING -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -j DROP
> -A PREROUTING -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -j DROP
> -A PREROUTING -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG
> FIN,PSH,URG -j DROP
> -A PREROUTING -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -j
> DROP
> -A PREROUTING -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -j DROP
> -A PREROUTING -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -j DROP
> COMMIT
>
> *nat
> :PREROUTING ACCEPT [0:0]
> :POSTROUTING ACCEPT [0:0]
> :OUTPUT ACCEPT [0:0]
> -A PREROUTING -p tcp --dport 5433 -j DNAT --to 192.168.2.10:5432
> -A PREROUTING -p udp --dport 5433 -j DNAT --to 192.168.2.10:5432
> -A PREROUTING -p tcp --dport 5434 -j DNAT --to 192.168.2.11:5432
> -A PREROUTING -p udp --dport 5434 -j DNAT --to 192.168.2.11:5432
> -A POSTROUTING -d 192.168.2.10 -s 192.168.0.0/255.255.0.0 -p tcp -j SNAT
> --to 192.168.11.100
> -A PREROUTING -p tcp -s 202.238.89.88 --dport 50000:50100 -j DNAT --to
> 192.168.2.5
> -A POSTROUTING -o eth0 -j MASQUERADE
>
> COMMIT
> *filter
> :INPUT DROP [0:0]
> :FORWARD ACCEPT [0:0]
> :OUTPUT ACCEPT [0:0]
> :BLACKLIST - [0:0]
> :LOG_ACCEPT - [0:0]
> :LOG_DROP - [0:0]
> :icmp_packets - [0:0]
> -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A INPUT -p tcp -m tcp --dport 20 -j LOG_ACCEPT
> -A INPUT -p tcp -m tcp --dport 21 -j LOG_ACCEPT
> -A INPUT -p tcp -m tcp --dport 22 -j LOG_ACCEPT
> -A INPUT -p tcp -m tcp --dport 25 -j LOG_ACCEPT
> -A INPUT -p udp -m udp --dport 53 -j ACCEPT
> -A INPUT -p tcp -m tcp --dport 80 -j ACCEPT
> -A INPUT -p tcp -m tcp --dport 110 -j ACCEPT
> -A INPUT -p tcp -m tcp --dport 143 -j ACCEPT
> -A INPUT -p tcp -m tcp --dport 443 -j LOG_ACCEPT
> -A INPUT -p tcp -m tcp --dport 993 -j ACCEPT
> -A INPUT -p udp -m udp -s 192.168.0.0/255.255.0.0 --dport 10080 -j
> ACCEPT
> # The following line is for FTP passive ports
> -A INPUT -p tcp -m tcp --dport 55000:55500 -j LOG_ACCEPT
> -A INPUT -s 127.0.0.1 -j ACCEPT
> -A INPUT -p icmp -j icmp_packets
> -A INPUT -j LOG_DROP
> -A OUTPUT -p tcp -m tcp --dport 50000:50100 -j LOG_ACCEPT
> -A OUTPUT -p udp -m udp --dport 700:710 -j LOG_ACCEPT
> -A BLACKLIST -j LOG_DROP
> -A LOG_ACCEPT -j LOG --log-prefix "[IPTABLES ACCEPT] : "
> --log-tcp-options --log-ip-options
> -A LOG_ACCEPT -j ACCEPT
> -A LOG_DROP -j LOG --log-prefix "[IPTABLES DROP] : " --log-tcp-options
> --log-ip-options
> -A LOG_DROP -j DROP
> -A icmp_packets -p icmp -m icmp --icmp-type 0 -j ACCEPT
> -A icmp_packets -s 192.168.0.0/255.255.255.0 -p icmp -m icmp --icmp-type
> 8 -j ACCEPT
> -A icmp_packets -s 192.168.1.0/255.255.255.0 -p icmp -m icmp --icmp-type
> 8 -j ACCEPT
> -A icmp_packets -s 192.168.2.0/255.255.255.0 -p icmp -m icmp --icmp-type
> 8 -j ACCEPT
> -A icmp_packets -p icmp -m icmp --icmp-type 8 -j ACCEPT
> -A icmp_packets -p icmp -m icmp --icmp-type 3 -j ACCEPT
> -A icmp_packets -p icmp -m icmp --icmp-type 11 -j ACCEPT
> COMMIT
>
>
>