Mailing List Archive

Reflection DDoS last week (was: syn flood attacks from NL-based netblocks)
Peace,

Here's to confirm that the pattern reported before in NANOG was indeed a
reflection DDoS attack. On Sunday, it also hit our customer, here's the
report:

https://www.prnewswire.com/news-releases/root-cause-analysis-and-incident-report-on-the-august-ddos-attack-300905405.html

tl;dr: basically that was a rather massive reflected SYN/ACK carpet bombing
against several datacenter prefixes (no particular target was identified).

--
Töma

On Sat, Aug 17, 2019, 1:06 AM Jim Shankland <nanog@shankland.org> wrote:

> Greetings,
>
> I'm seeing slow-motion (a few per second, per IP/port pair) syn flood
> attacks ostensibly originating from 3 NL-based IP blocks: 88.208.0.0/18
> , 5.11.80.0/21, and 78.140.128.0/18 ("ostensibly" because ... syn flood,
> and BCP 38 not yet fully adopted).
>
> Why is this syn flood different from all other syn floods? Well ...
>
> 1. Rate seems too slow to do any actual damage (is anybody really
> bothered by a few bad SYN packets per second per service, at this
> point?); but
>
> 2. IPs/port combinations with actual open services are being targeted
> (I'm seeing ports 22, 443, and 53, just at a glance, to specific IPs
> with those services running), implying somebody checked for open
> services first;
>
> 3. I'm seeing this in at least 2 locations, to addresses in different,
> completely unrelated ASes, implying it may be pretty widespread.
>
> Is anybody else seeing the same thing? Any thoughts on what's going on?
> Or should I just be ignoring this and getting on with the weekend?
>
> Jim
>
Re: Reflection DDoS last week (was: syn flood attacks from NL-based netblocks) [ In reply to ]
Thanks for following up, and for publishing two bits of key data:
- This was part of a larger attack campaign that included CLDAP
amplification
- The SYN/ACK amplification resulted in 208Mpps (or more)

Some additional questions, if you're able to answer them (off-list is fine
if there are things that can't be shared broadly):
- How large was the CLDAP amplification attack? What was the packet rate
of the initial fragments?
- The post suggested that the 208Mpps saturated some links. Did it cause
other problems as well?
- Was the attack referred to law enforcement?
- Were any transit providers asked to trace the source of the spoofing to
either stop the attack or facilitate the law enforcement investigation?

Damian

On Wed, Aug 21, 2019 at 12:44 PM Töma Gavrichenkov <ximaera@gmail.com>
wrote:

> Peace,
>
> Here's to confirm that the pattern reported before in NANOG was indeed a
> reflection DDoS attack. On Sunday, it also hit our customer, here's the
> report:
>
>
> https://www.prnewswire.com/news-releases/root-cause-analysis-and-incident-report-on-the-august-ddos-attack-300905405.html
>
> tl;dr: basically that was a rather massive reflected SYN/ACK carpet
> bombing against several datacenter prefixes (no particular target was
> identified).
>
> --
> Töma
>
> On Sat, Aug 17, 2019, 1:06 AM Jim Shankland <nanog@shankland.org> wrote:
>
>> Greetings,
>>
>> I'm seeing slow-motion (a few per second, per IP/port pair) syn flood
>> attacks ostensibly originating from 3 NL-based IP blocks: 88.208.0.0/18
>> , 5.11.80.0/21, and 78.140.128.0/18 ("ostensibly" because ... syn flood,
>> and BCP 38 not yet fully adopted).
>>
>> Why is this syn flood different from all other syn floods? Well ...
>>
>> 1. Rate seems too slow to do any actual damage (is anybody really
>> bothered by a few bad SYN packets per second per service, at this
>> point?); but
>>
>> 2. IPs/port combinations with actual open services are being targeted
>> (I'm seeing ports 22, 443, and 53, just at a glance, to specific IPs
>> with those services running), implying somebody checked for open
>> services first;
>>
>> 3. I'm seeing this in at least 2 locations, to addresses in different,
>> completely unrelated ASes, implying it may be pretty widespread.
>>
>> Is anybody else seeing the same thing? Any thoughts on what's going on?
>> Or should I just be ignoring this and getting on with the weekend?
>>
>> Jim
>>
>
Re: Reflection DDoS last week (was: syn flood attacks from NL-based netblocks) [ In reply to ]
Peace,

On Thu, Aug 22, 2019 at 12:17 AM Damian Menscher <damian@google.com> wrote:
> Some additional questions, if you're able to answer them (off-list is fine if there are things that can't be shared broadly):
> - Was the attack referred to law enforcement?

It is being referred to now. This would most probably get going under
the jurisdiction of the Netherlands. Whether the latter would be able
to address it properly or not remains to be seen, but honestly I'm not
quite optimistic here.

> - Were any transit providers asked to trace the
> source of the spoofing to either stop the attack
> or facilitate the law enforcement investigation?

No.
Initially we were busy setting up the game and pushing the upstreams
to accept our new customer prefix advertisements a.s.a.p.
Afterwards we were too busy trying to understand why some of the
upstreams didn't work as expected (that part was mentioned in the
report).

Hence, tracing the source was not deemed a high priority task.

--
Töma
Re: Reflection DDoS last week (was: syn flood attacks from NL-based netblocks) [ In reply to ]
Töma, thanks for this interesting update. The best defense against this
type of DDoS attacks seems idd to be relaying to
sufficiently-large-bandwidth cloud/CDN, and filtering TCP traffic (received
not from the relay). Such relaying should be done well - smart attacks may
still be possible for `naive' relaying.
--
Amir



On Wed, Aug 21, 2019 at 3:46 PM Töma Gavrichenkov <ximaera@gmail.com> wrote:

> Peace,
>
> Here's to confirm that the pattern reported before in NANOG was indeed a
> reflection DDoS attack. On Sunday, it also hit our customer, here's the
> report:
>
>
> https://www.prnewswire.com/news-releases/root-cause-analysis-and-incident-report-on-the-august-ddos-attack-300905405.html
>
> tl;dr: basically that was a rather massive reflected SYN/ACK carpet
> bombing against several datacenter prefixes (no particular target was
> identified).
>
> --
> Töma
>
> On Sat, Aug 17, 2019, 1:06 AM Jim Shankland <nanog@shankland.org> wrote:
>
>> Greetings,
>>
>> I'm seeing slow-motion (a few per second, per IP/port pair) syn flood
>> attacks ostensibly originating from 3 NL-based IP blocks: 88.208.0.0/18
>> , 5.11.80.0/21, and 78.140.128.0/18 ("ostensibly" because ... syn flood,
>> and BCP 38 not yet fully adopted).
>>
>> Why is this syn flood different from all other syn floods? Well ...
>>
>> 1. Rate seems too slow to do any actual damage (is anybody really
>> bothered by a few bad SYN packets per second per service, at this
>> point?); but
>>
>> 2. IPs/port combinations with actual open services are being targeted
>> (I'm seeing ports 22, 443, and 53, just at a glance, to specific IPs
>> with those services running), implying somebody checked for open
>> services first;
>>
>> 3. I'm seeing this in at least 2 locations, to addresses in different,
>> completely unrelated ASes, implying it may be pretty widespread.
>>
>> Is anybody else seeing the same thing? Any thoughts on what's going on?
>> Or should I just be ignoring this and getting on with the weekend?
>>
>> Jim
>>
>
Re: Reflection DDoS last week (was: syn flood attacks from NL-based netblocks) [ In reply to ]
On Wed, Aug 21, 2019 at 3:21 PM Töma Gavrichenkov <ximaera@gmail.com> wrote:

> On Thu, Aug 22, 2019 at 12:17 AM Damian Menscher <damian@google.com>
> wrote:
> > Some additional questions, if you're able to answer them (off-list is
> fine if there are things that can't be shared broadly):
> > - Was the attack referred to law enforcement?
>
> It is being referred to now. This would most probably get going under
> the jurisdiction of the Netherlands.
>

Deeper analysis and discussion indicates there were several victims: we saw
brief attacks targeting some of our cloud customers with syn-ack peaks
above 125 Mpps; another provider reported seeing 275Mpps sustained. So
presumably there are a few law enforcement investigations under way, in
various jurisdictions.

> - Were any transit providers asked to trace the
> > source of the spoofing to either stop the attack
> > or facilitate the law enforcement investigation?
>
> No.... tracing the source was not deemed a high priority task.
>

Fair enough. I just didn't want to duplicate effort.

The source of the spoofing has been traced. The responsible hosting
provider has kicked off their problem customer, and is exploring the
necessary filtering to prevent a recurrence.

If anyone sees more of this style of attack please send up a flare so the
community knows to track down the new source.

Damian