Mailing List Archive

Hello,

>From our side we have seen lots of IPv4 traffic from sources originated from AS15169 (UDP port 443) .We are using netflow to identify the traffic.

Michalis


----------------------------------------------------------------------

Message: 1
Date: Thu, 04 Jun 2015 10:30:11 +0300
From: Yannis Nikolopoulos <dez@otenet.gr>
Subject: IPv6 QUIC traffic
To: ipv6-ops@lists.cluenet.de
Message-ID: <556FFE83.1010907@otenet.gr>
Content-Type: text/plain; charset=utf-8; format=flowed

Hello all,

we've been seeing IPv6 UDP traffic on port 443 coming from Google servers being dropped on our ACLs. We realise that this is probably just
(QUIC-enabled) chrome browsers accessing google content.

Isn't it strange that no such IPv4 traffic exists?

cheers,
Yannis



------------------------------

_______________________________________________
ipv6-ops mailing list
ipv6-ops@lists.cluenet.de
http://lists.cluenet.de/mailman/listinfo/ipv6-ops


End of ipv6-ops Digest, Vol 123, Issue 1
****************************************

Yannis Nikolopoulos schreef op 4-6-2015 om 9:30:
> Hello all,
>
> we've been seeing IPv6 UDP traffic on port 443 coming from Google
> servers being dropped on our ACLs. We realise that this is probably just
> (QUIC-enabled) chrome browsers accessing google content.
>
> Isn't it strange that no such IPv4 traffic exists?

Yes it does, I just added firewall rules for it. I was puzzled for a
moment today too by the UDP traffic.

Regards,
Seth

> Yes it does, I just added firewall rules for it. I was puzzled for a
> moment today too by the UDP traffic.

Permitting or denying? (always logging, I assume)

On 06/04/2015 01:08 PM, michalis.bersimis@hq.cyta.gr wrote:
> Hello,
>
>>From our side we have seen lots of IPv4 traffic from sources originated from AS15169 (UDP port 443) .We are using netflow to identify the traffic.

You're right,

We can also see the relevant IPv4/IPv6 traffic via Netflow, as it enters
our network, It's just weird that we don't see logs for the denied
matched IPv4 packets further down the network, as we do for IPv6...


> Michalis
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Thu, 04 Jun 2015 10:30:11 +0300
> From: Yannis Nikolopoulos <dez@otenet.gr>
> Subject: IPv6 QUIC traffic
> To: ipv6-ops@lists.cluenet.de
> Message-ID: <556FFE83.1010907@otenet.gr>
> Content-Type: text/plain; charset=utf-8; format=flowed
>
> Hello all,
>
> we've been seeing IPv6 UDP traffic on port 443 coming from Google servers being dropped on our ACLs. We realise that this is probably just
> (QUIC-enabled) chrome browsers accessing google content.
>
> Isn't it strange that no such IPv4 traffic exists?
>
> cheers,
> Yannis
>
>
>
> ------------------------------
>
> _______________________________________________
> ipv6-ops mailing list
> ipv6-ops@lists.cluenet.de
> http://lists.cluenet.de/mailman/listinfo/ipv6-ops
>
>
> End of ipv6-ops Digest, Vol 123, Issue 1
> ****************************************
>

On 06/04/2015 04:00 PM, Yannis Nikolopoulos wrote:
> On 06/04/2015 01:08 PM, michalis.bersimis@hq.cyta.gr wrote:
>>> From our side we have seen lots of IPv4 traffic from sources
>>> originated from AS15169 (UDP port 443) .We are using netflow to
>>> identify the traffic.
>
> You're right,
>
> We can also see the relevant IPv4/IPv6 traffic via Netflow, as it enters
> our network, It's just weird that we don't see logs for the denied
> matched IPv4 packets further down the network, as we do for IPv6...

Why are you blocking QUIC traffic anyway?

-dominik

On Thu, Jun 4, 2015 at 8:03 AM, Dominik Bay <db@rrbone.net> wrote:

> On 06/04/2015 04:00 PM, Yannis Nikolopoulos wrote:
> > On 06/04/2015 01:08 PM, michalis.bersimis@hq.cyta.gr wrote:
> >>> From our side we have seen lots of IPv4 traffic from sources
> >>> originated from AS15169 (UDP port 443) .We are using netflow to
> >>> identify the traffic.
> >
> > You're right,
> >
> > We can also see the relevant IPv4/IPv6 traffic via Netflow, as it enters
> > our network, It's just weird that we don't see logs for the denied
> > matched IPv4 packets further down the network, as we do for IPv6...
>
> Why are you blocking QUIC traffic anyway?
>
> -dominik
>

This is an IPv6 list, and there is no reason to block quic on IPv6.

But, i definitely rate limit UDP ipv4 since that vast majority of UDP is
DDoS traffic... I have suggested to the QUIC folks to use a new protocol
number so they are not guilty by association with other UDP service... but
they think broken CPE are a bigger problem. I think DDoS from UDP (NTP,
DNS, SSDP, CHARGEN, SNMP ...) is a bigger problem....

I guess Google will just have to deal with QUIC not working on IPv4 in my
network. Or they can use a new L4 protocol number and use the proven fall
back strategy they already have with TCP to apply here as well...

CB

On Thu, Jun 4, 2015 at 8:18 AM, Ca By <cb.list6@gmail.com> wrote:
>
> On Thu, Jun 4, 2015 at 8:03 AM, Dominik Bay <db@rrbone.net> wrote:
>
>> On 06/04/2015 04:00 PM, Yannis Nikolopoulos wrote:
>> > On 06/04/2015 01:08 PM, michalis.bersimis@hq.cyta.gr wrote:
>> >>> From our side we have seen lots of IPv4 traffic from sources
>> >>> originated from AS15169 (UDP port 443) .We are using netflow to
>> >>> identify the traffic.
>> >
>> > You're right,
>> >
>> > We can also see the relevant IPv4/IPv6 traffic via Netflow, as it enters
>> > our network, It's just weird that we don't see logs for the denied
>> > matched IPv4 packets further down the network, as we do for IPv6...
>>
>
Random guess: All your traffic to Google is over IPv6, and that's why you
don't see IPv4?

Why are you blocking QUIC traffic anyway?
>>
>
> This is an IPv6 list, and there is no reason to block quic on IPv6.
>
> But, i definitely rate limit UDP ipv4 since that vast majority of UDP is
> DDoS traffic... I have suggested to the QUIC folks to use a new protocol
> number so they are not guilty by association with other UDP service... but
> they think broken CPE are a bigger problem. I think DDoS from UDP (NTP,
> DNS, SSDP, CHARGEN, SNMP ...) is a bigger problem....
>

You don't need to block all UDP to filter DDoS traffic. Rate-limiting
traffic from the specific ports you mentioned (123, 53, 1900, 19, 161) is
sufficient. Given QUIC traffic always uses a high-numbered ephemeral port,
there's little risk of impact to it if you rate-limit only those ports
commonly used for amplification.

I guess Google will just have to deal with QUIC not working on IPv4 in my
> network. Or they can use a new L4 protocol number and use the proven fall
> back strategy they already have with TCP to apply here as well...
>

I'm not sure your users will agree with your choice, especially as more
sites start to support QUIC, but I suppose they can always move to another
provider.

Damian

On Thu, Jun 4, 2015 at 10:28 AM, Damian Menscher <damian@google.com> wrote:

> On Thu, Jun 4, 2015 at 8:18 AM, Ca By <cb.list6@gmail.com> wrote:
>>
>> On Thu, Jun 4, 2015 at 8:03 AM, Dominik Bay <db@rrbone.net> wrote:
>>
>>> On 06/04/2015 04:00 PM, Yannis Nikolopoulos wrote:
>>> > On 06/04/2015 01:08 PM, michalis.bersimis@hq.cyta.gr wrote:
>>> >>> From our side we have seen lots of IPv4 traffic from sources
>>> >>> originated from AS15169 (UDP port 443) .We are using netflow to
>>> >>> identify the traffic.
>>> >
>>> > You're right,
>>> >
>>> > We can also see the relevant IPv4/IPv6 traffic via Netflow, as it
>>> enters
>>> > our network, It's just weird that we don't see logs for the denied
>>> > matched IPv4 packets further down the network, as we do for IPv6...
>>>
>>
> Random guess: All your traffic to Google is over IPv6, and that's why you
> don't see IPv4?
>
> Why are you blocking QUIC traffic anyway?
>>>
>>
>> This is an IPv6 list, and there is no reason to block quic on IPv6.
>>
>> But, i definitely rate limit UDP ipv4 since that vast majority of UDP is
>> DDoS traffic... I have suggested to the QUIC folks to use a new protocol
>> number so they are not guilty by association with other UDP service... but
>> they think broken CPE are a bigger problem. I think DDoS from UDP (NTP,
>> DNS, SSDP, CHARGEN, SNMP ...) is a bigger problem....
>>
>
> You don't need to block all UDP to filter DDoS traffic. Rate-limiting
> traffic from the specific ports you mentioned (123, 53, 1900, 19, 161) is
> sufficient. Given QUIC traffic always uses a high-numbered ephemeral port,
> there's little risk of impact to it if you rate-limit only those ports
> commonly used for amplification.
>
>
UDP 80 and 443 are very commonly associated with DDoS in my experience. I
think it is common used as a reflection source port.


> I guess Google will just have to deal with QUIC not working on IPv4 in my
>> network. Or they can use a new L4 protocol number and use the proven fall
>> back strategy they already have with TCP to apply here as well...
>>
>
> I'm not sure your users will agree with your choice, especially as more
> sites start to support QUIC, but I suppose they can always move to another
> provider.
>
>
Cute. And helpful. QUIC can just fall back. Or QUIC can use a new L4
protocol. That's why we have protocol numbers. I provided my feedback to
the QUIC team.


> Damian
>

> On Jun 4, 2015, at 1:28 PM, Damian Menscher <damian@google.com> wrote:
>
> You don't need to block all UDP to filter DDoS traffic. Rate-limiting traffic from the specific ports you mentioned (123, 53, 1900, 19, 161) is sufficient. Given QUIC traffic always uses a high-numbered ephemeral port, there's little risk of impact to it if you rate-limit only those ports commonly used for amplification.

Not really since ~46% of DNS amplifiers respond with non udp-53 port.

http://openresolverproject.org/breakdown.cgi

Last week it was 9.7m hosts.

- Jared

hi,

On Thu, Jun 04, 2015 at 04:03:14PM +0100, Dominik Bay wrote:
> Why are you blocking QUIC traffic anyway?

Because traditionally, UDP/80 was badness from stupid blackhats...

(We don't, but I can very well understand why UDP on "traditional tcp ports"
would get blocked, or strictly rate-limited...)

Gert Doering
-- NetMaster
--
have you enabled IPv6 on something today...?

SpaceNet AG Vorstand: Sebastian v. Bomhard
Joseph-Dollinger-Bogen 14 Aufsichtsratsvors.: A. Grundner-Culemann
D-80807 Muenchen HRB: 136055 (AG Muenchen)
Tel: +49 (0)89/32356-444 USt-IdNr.: DE813185279

On Thu, Jun 4, 2015 at 11:37 AM, Gert Doering <gert@space.net> wrote:

> hi,
>
> On Thu, Jun 04, 2015 at 04:03:14PM +0100, Dominik Bay wrote:
> > Why are you blocking QUIC traffic anyway?
>
> Because traditionally, UDP/80 was badness from stupid blackhats...
>
> (We don't, but I can very well understand why UDP on "traditional tcp
> ports"
> would get blocked, or strictly rate-limited...)
>
>
FYI, the QUIC people have been informed 2x that UDP is not safe and
operators will rate limit it.

https://groups.google.com/a/chromium.org/forum/#!searchin/proto-quic/UDP/proto-quic/09L5YD2u5xU/EsZgXHJq0o4J

https://groups.google.com/a/chromium.org/forum/#!searchin/proto-quic/UDP/proto-quic/1tN6j-dErw0/3c7bEHQm2gMJ





> Gert Doering
> -- NetMaster
> --
> have you enabled IPv6 on something today...?
>
> SpaceNet AG Vorstand: Sebastian v. Bomhard
> Joseph-Dollinger-Bogen 14 Aufsichtsratsvors.: A. Grundner-Culemann
> D-80807 Muenchen HRB: 136055 (AG Muenchen)
> Tel: +49 (0)89/32356-444 USt-IdNr.: DE813185279
>

On 06/04/2015 05:13 PM, Seth Mos wrote:
> Yannis Nikolopoulos schreef op 4-6-2015 om 9:30:
>> Hello all,
>>
>> we've been seeing IPv6 UDP traffic on port 443 coming from Google
>> servers being dropped on our ACLs. We realise that this is probably just
>> (QUIC-enabled) chrome browsers accessing google content.
>>
>> Isn't it strange that no such IPv4 traffic exists?
> Yes it does, I just added firewall rules for it. I was puzzled for a
> moment today too by the UDP traffic.

yep, a typo in the pattern matching vanished all IPv4 entries.

> Regards,
> Seth

On 06/04/2015 06:03 PM, Dominik Bay wrote:
> On 06/04/2015 04:00 PM, Yannis Nikolopoulos wrote:
>> On 06/04/2015 01:08 PM, michalis.bersimis@hq.cyta.gr wrote:
>>>> From our side we have seen lots of IPv4 traffic from sources
>>>> originated from AS15169 (UDP port 443) .We are using netflow to
>>>> identify the traffic.
>> You're right,
>>
>> We can also see the relevant IPv4/IPv6 traffic via Netflow, as it enters
>> our network, It's just weird that we don't see logs for the denied
>> matched IPv4 packets further down the network, as we do for IPv6...
> Why are you blocking QUIC traffic anyway?

we were not specifically blocking QUIC traffic, we're just filtering UDP
traffic for our secure LANs.
BTW, we're also limiting UDP traffic in general

> -dominik

On 2015-06-04 20:48, Ca By wrote:
> FYI, the QUIC people have been informed 2x that UDP is not safe and
> operators will rate limit it.
>
> https://groups.google.com/a/chromium.org/forum/#!searchin/proto-quic/UDP/proto-quic/09L5YD2u5xU/EsZgXHJq0o4J
> https://groups.google.com/a/chromium.org/forum/#!searchin/proto-quic/UDP/proto-quic/1tN6j-dErw0/3c7bEHQm2gMJ

Then evidence for this should turn up in the telemetry, you'd think.
(Which in turn means overblocking or throttling in every network you
control right now would convey your message better than "this might
happen", when it's not visible.)

At this point adding new L4 protocols won't work either because of
firewalls, router ACLs, CPEs and throttling. So it'd be out of the
frying pan into the fire.

Given that there is Happy Eyeballs for this and there is a probably not
too unreasonable fallback with HTTP/2, can't we just see how this plays
out? ;-)

Kind regards
Philipp Kern

On Thu, 4 Jun 2015, Philipp Kern wrote:

> Given that there is Happy Eyeballs for this and there is a probably not
> too unreasonable fallback with HTTP/2, can't we just see how this plays
> out? ;-)

Happy Eyeballs doesn't solve things working badly (=ratelimiting),
unfortunately, neither does it help in detecting PMTU blackholing.

At least there is TCP PMTU blackhole detection in modern TCP, I couldn't
find any reference to this for QUIC in my 10 second google search.

--
Mikael Abrahamsson email: swmike@swm.pp.se

On Fri, Jun 5, 2015 at 2:23 PM, Mikael Abrahamsson <swmike@swm.pp.se> wrote:

> At least there is TCP PMTU blackhole detection in modern TCP, I couldn't
> find any reference to this for QUIC in my 10 second google search.


IIRC QUIC starts the connection off by sending maximum-sized packets so it
won't suffer from MTU blackholes unless the MTU changes during the
connection.

On 2015-06-05 07:23, Mikael Abrahamsson wrote:
> On Thu, 4 Jun 2015, Philipp Kern wrote:
>> Given that there is Happy Eyeballs for this and there is a probably
>> not too unreasonable fallback with HTTP/2, can't we just see how this
>> plays out? ;-)
> Happy Eyeballs doesn't solve things working badly (=ratelimiting),
> unfortunately, neither does it help in detecting PMTU blackholing.

What's the bucket for ratelimiting? (Given that it's also a memory
question how many buckets can be opened for this.) Because if enough
people use QUIC, ratelimiting should be across all users and hence HE
should work? Of course it would not if ratelimiting is per (srcip,
dstip) tuple or something in which case it would only fail after the
initial ramping up phase.

> At least there is TCP PMTU blackhole detection in modern TCP, I
> couldn't find any reference to this for QUIC in my 10 second google
> search.

What Lorenzo said.

Kind regards
Philipp Kern

On 04/06/2015 18:51, Ca By wrote:
>
> UDP 80 and 443 are very commonly associated with DDoS in my experience.
> I think it is common used as a reflection source port.

Sadly true. We see this a lot.

Not saying I agree with blocking it, but UDP 80/443 is deeply suspicious
traffic in my experience.