Mailing List Archive: [Wikimedia-l] Update on Wikimedia site and system outages

[Wikimedia-l] Update on Wikimedia site and system outages

Sep 7, 2019, 4:25 PM

Post #1 of 9 (927 views)

Hello everyone,

By now you are likely aware that the Wikimedia sites suffered from a
relatively significant botnet driven DDOS attack on September 6th, taking
them offline in several countries throughout the day. This primarily
affected Wikipedia access in Europe and the Middle East. We posted a short
update of the event on our website.[1]

I would like to thank everyone who stepped up to support the restoration of
our projects, including the fast reporting of community members throughout
the world and our security and engineering teams who worked long hours to
address many complex issues surrounding the attack and our response—the
Site Reliability Engineering team in particular.

The Wikimedia Foundation leadership team is proud to work with such
talented and dedicated staff and supporters.

Yours,
Heather

1.
https://wikimediafoundation.org/news/2019/09/07/malicious-attack-on-wikipedia-what-we-know-and-what-were-doing/

"Today, Wikipedia was hit with a malicious attack that has taken it offline
in several countries for intermittent periods. The attack is ongoing and
our Site Reliability Engineering team is working hard to stop it and
restore access to the site.

As one of the world’s most popular sites, Wikipedia sometimes attracts “bad
faith” actors. Along with the rest of the web, we operate in an
increasingly sophisticated and complex environment where threats are
continuously evolving. Because of this, the Wikimedia communities and
Wikimedia Foundation have created dedicated systems and staff to regularly
monitor and address risks. If a problem occurs, we learn, we improve, and
we prepare to be better for next time.

We condemn these sorts of attacks. They’re not just about taking Wikipedia
offline. Takedown attacks threaten everyone’s fundamental rights to freely
access and share information. We in the Wikimedia movement and Foundation
are committed to protecting these rights for everyone.

Right now, we’re continuing to work to restore access wherever you might be
reading Wikipedia in the world. We’ll keep you posted."

--

Heather Walls (she/her)

Chief Creative Officer
Wikimedia Foundation
https://wikimediafoundation.org
_______________________________________________
Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l
New messages to: Wikimedia-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>

Re: [Wikimedia-l] Update on Wikimedia site and system outages [ In reply to ]

rhinosf1 at gmail

Sep 8, 2019, 7:40 AM

Post #2 of 9 (925 views)

Permalink

Well done everyone involved!

On Sun, 8 Sep 2019 at 00:26, Heather Walls <hwalls@wikimedia.org> wrote:

> Hello everyone,
>
> By now you are likely aware that the Wikimedia sites suffered from a
> relatively significant botnet driven DDOS attack on September 6th, taking
> them offline in several countries throughout the day. This primarily
> affected Wikipedia access in Europe and the Middle East. We posted a short
> update of the event on our website.[1]
>
> I would like to thank everyone who stepped up to support the restoration of
> our projects, including the fast reporting of community members throughout
> the world and our security and engineering teams who worked long hours to
> address many complex issues surrounding the attack and our response—the
> Site Reliability Engineering team in particular.
>
> The Wikimedia Foundation leadership team is proud to work with such
> talented and dedicated staff and supporters.
>
> Yours,
> Heather
>
>
> 1.
>
> https://wikimediafoundation.org/news/2019/09/07/malicious-attack-on-wikipedia-what-we-know-and-what-were-doing/
>
> "Today, Wikipedia was hit with a malicious attack that has taken it offline
> in several countries for intermittent periods. The attack is ongoing and
> our Site Reliability Engineering team is working hard to stop it and
> restore access to the site.
>
> As one of the world’s most popular sites, Wikipedia sometimes attracts “bad
> faith” actors. Along with the rest of the web, we operate in an
> increasingly sophisticated and complex environment where threats are
> continuously evolving. Because of this, the Wikimedia communities and
> Wikimedia Foundation have created dedicated systems and staff to regularly
> monitor and address risks. If a problem occurs, we learn, we improve, and
> we prepare to be better for next time.
>
> We condemn these sorts of attacks. They’re not just about taking Wikipedia
> offline. Takedown attacks threaten everyone’s fundamental rights to freely
> access and share information. We in the Wikimedia movement and Foundation
> are committed to protecting these rights for everyone.
>
> Right now, we’re continuing to work to restore access wherever you might be
> reading Wikipedia in the world. We’ll keep you posted."
>
>
> --
>
> Heather Walls (she/her)
>
> Chief Creative Officer
> Wikimedia Foundation
> https://wikimediafoundation.org
> _______________________________________________
> Wikimedia-l mailing list, guidelines at:
> https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
> https://meta.wikimedia.org/wiki/Wikimedia-l
> New messages to: Wikimedia-l@lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>

--
RhinosF1
Miraheze Volunteer
_______________________________________________
Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l
New messages to: Wikimedia-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>

Re: [Wikimedia-l] Update on Wikimedia site and system outages [ In reply to ]

hwalls at wikimedia

Sep 19, 2019, 4:29 PM

Post #3 of 9 (921 views)

Permalink

Hello again,

As a follow-up to my last note on the September 6th DDoS attack, we wanted
to provide you with an update. There have been no further attacks in the
last week and our sites are now running normally. Our SRE team is
continuing to monitor the situation.

Based on what we learned in this attack, our security and engineering teams
are researching and putting together plans for more protection of our
infrastructure to address any potential attacks in the future.

We appreciate everyone’s support, particularly the folks on the SRE team,
in helping to restore access.

Yours,
Heather

On Sat, Sep 7, 2019 at 4:25 PM Heather Walls <hwalls@wikimedia.org> wrote:

> Hello everyone,
>
> By now you are likely aware that the Wikimedia sites suffered from a
> relatively significant botnet driven DDOS attack on September 6th, taking
> them offline in several countries throughout the day. This primarily
> affected Wikipedia access in Europe and the Middle East. We posted a short
> update of the event on our website.[1]
>
> I would like to thank everyone who stepped up to support the restoration
> of our projects, including the fast reporting of community members
> throughout the world and our security and engineering teams who worked long
> hours to address many complex issues surrounding the attack and our
> response—the Site Reliability Engineering team in particular.
>
> The Wikimedia Foundation leadership team is proud to work with such
> talented and dedicated staff and supporters.
>
> Yours,
> Heather
>
>
> 1.
> https://wikimediafoundation.org/news/2019/09/07/malicious-attack-on-wikipedia-what-we-know-and-what-were-doing/
>
> "Today, Wikipedia was hit with a malicious attack that has taken it
> offline in several countries for intermittent periods. The attack is
> ongoing and our Site Reliability Engineering team is working hard to stop
> it and restore access to the site.
>
> As one of the world’s most popular sites, Wikipedia sometimes attracts
> “bad faith” actors. Along with the rest of the web, we operate in an
> increasingly sophisticated and complex environment where threats are
> continuously evolving. Because of this, the Wikimedia communities and
> Wikimedia Foundation have created dedicated systems and staff to regularly
> monitor and address risks. If a problem occurs, we learn, we improve, and
> we prepare to be better for next time.
>
> We condemn these sorts of attacks. They’re not just about taking Wikipedia
> offline. Takedown attacks threaten everyone’s fundamental rights to freely
> access and share information. We in the Wikimedia movement and Foundation
> are committed to protecting these rights for everyone.
>
> Right now, we’re continuing to work to restore access wherever you might
> be reading Wikipedia in the world. We’ll keep you posted."
>
>
> --
>
> Heather Walls (she/her)
>
> Chief Creative Officer
> Wikimedia Foundation
> https://wikimediafoundation.org
>
>
>

--

Heather Walls (she/her)

Chief Creative Officer

Wikimedia Foundation <https://wikimediafoundation.org/>
_______________________________________________
Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l
New messages to: Wikimedia-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>

Re: [Wikimedia-l] Update on Wikimedia site and system outages [ In reply to ]

rhinosf1 at gmail

Oct 20, 2019, 2:55 PM

Post #4 of 9 (914 views)

Permalink

Any idea of when an incident report may come out?

RhinosF1
Wikimedia User & Incident Reporter

On Fri, 20 Sep 2019 at 00:29, Heather Walls <hwalls@wikimedia.org> wrote:

> Hello again,
>
> As a follow-up to my last note on the September 6th DDoS attack, we wanted
> to provide you with an update. There have been no further attacks in the
> last week and our sites are now running normally. Our SRE team is
> continuing to monitor the situation.
>
> Based on what we learned in this attack, our security and engineering teams
> are researching and putting together plans for more protection of our
> infrastructure to address any potential attacks in the future.
>
> We appreciate everyone’s support, particularly the folks on the SRE team,
> in helping to restore access.
>
> Yours,
> Heather
>
> On Sat, Sep 7, 2019 at 4:25 PM Heather Walls <hwalls@wikimedia.org> wrote:
>
> > Hello everyone,
> >
> > By now you are likely aware that the Wikimedia sites suffered from a
> > relatively significant botnet driven DDOS attack on September 6th, taking
> > them offline in several countries throughout the day. This primarily
> > affected Wikipedia access in Europe and the Middle East. We posted a
> short
> > update of the event on our website.[1]
> >
> > I would like to thank everyone who stepped up to support the restoration
> > of our projects, including the fast reporting of community members
> > throughout the world and our security and engineering teams who worked
> long
> > hours to address many complex issues surrounding the attack and our
> > response—the Site Reliability Engineering team in particular.
> >
> > The Wikimedia Foundation leadership team is proud to work with such
> > talented and dedicated staff and supporters.
> >
> > Yours,
> > Heather
> >
> >
> > 1.
> >
> https://wikimediafoundation.org/news/2019/09/07/malicious-attack-on-wikipedia-what-we-know-and-what-were-doing/
> >
> > "Today, Wikipedia was hit with a malicious attack that has taken it
> > offline in several countries for intermittent periods. The attack is
> > ongoing and our Site Reliability Engineering team is working hard to stop
> > it and restore access to the site.
> >
> > As one of the world’s most popular sites, Wikipedia sometimes attracts
> > “bad faith” actors. Along with the rest of the web, we operate in an
> > increasingly sophisticated and complex environment where threats are
> > continuously evolving. Because of this, the Wikimedia communities and
> > Wikimedia Foundation have created dedicated systems and staff to
> regularly
> > monitor and address risks. If a problem occurs, we learn, we improve, and
> > we prepare to be better for next time.
> >
> > We condemn these sorts of attacks. They’re not just about taking
> Wikipedia
> > offline. Takedown attacks threaten everyone’s fundamental rights to
> freely
> > access and share information. We in the Wikimedia movement and Foundation
> > are committed to protecting these rights for everyone.
> >
> > Right now, we’re continuing to work to restore access wherever you might
> > be reading Wikipedia in the world. We’ll keep you posted."
> >
> >
> > --
> >
> > Heather Walls (she/her)
> >
> > Chief Creative Officer
> > Wikimedia Foundation
> > https://wikimediafoundation.org
> >
> >
> >
>
> --
>
> Heather Walls (she/her)
>
> Chief Creative Officer
>
> Wikimedia Foundation <https://wikimediafoundation.org/>
> _______________________________________________
> Wikimedia-l mailing list, guidelines at:
> https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
> https://meta.wikimedia.org/wiki/Wikimedia-l
> New messages to: Wikimedia-l@lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>
_______________________________________________
Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l
New messages to: Wikimedia-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>

Re: [Wikimedia-l] Update on Wikimedia site and system outages [ In reply to ]

benjaminikuta at gmail

Oct 21, 2019, 1:49 PM

Post #5 of 9 (914 views)

Permalink

I am also curious about this.

> On Oct 20, 2019, at 2:55 PM, RhinosF1 - <rhinosf1@gmail.com> wrote:
>
> Any idea of when an incident report may come out?
>
> RhinosF1
> Wikimedia User & Incident Reporter
>
> On Fri, 20 Sep 2019 at 00:29, Heather Walls <hwalls@wikimedia.org> wrote:
>
>> Hello again,
>>
>> As a follow-up to my last note on the September 6th DDoS attack, we wanted
>> to provide you with an update. There have been no further attacks in the
>> last week and our sites are now running normally. Our SRE team is
>> continuing to monitor the situation.
>>
>> Based on what we learned in this attack, our security and engineering teams
>> are researching and putting together plans for more protection of our
>> infrastructure to address any potential attacks in the future.
>>
>> We appreciate everyone’s support, particularly the folks on the SRE team,
>> in helping to restore access.
>>
>> Yours,
>> Heather
>>
>> On Sat, Sep 7, 2019 at 4:25 PM Heather Walls <hwalls@wikimedia.org> wrote:
>>
>>> Hello everyone,
>>>
>>> By now you are likely aware that the Wikimedia sites suffered from a
>>> relatively significant botnet driven DDOS attack on September 6th, taking
>>> them offline in several countries throughout the day. This primarily
>>> affected Wikipedia access in Europe and the Middle East. We posted a
>> short
>>> update of the event on our website.[1]
>>>
>>> I would like to thank everyone who stepped up to support the restoration
>>> of our projects, including the fast reporting of community members
>>> throughout the world and our security and engineering teams who worked
>> long
>>> hours to address many complex issues surrounding the attack and our
>>> response—the Site Reliability Engineering team in particular.
>>>
>>> The Wikimedia Foundation leadership team is proud to work with such
>>> talented and dedicated staff and supporters.
>>>
>>> Yours,
>>> Heather
>>>
>>>
>>> 1.
>>>
>> https://wikimediafoundation.org/news/2019/09/07/malicious-attack-on-wikipedia-what-we-know-and-what-were-doing/
>>>
>>> "Today, Wikipedia was hit with a malicious attack that has taken it
>>> offline in several countries for intermittent periods. The attack is
>>> ongoing and our Site Reliability Engineering team is working hard to stop
>>> it and restore access to the site.
>>>
>>> As one of the world’s most popular sites, Wikipedia sometimes attracts
>>> “bad faith” actors. Along with the rest of the web, we operate in an
>>> increasingly sophisticated and complex environment where threats are
>>> continuously evolving. Because of this, the Wikimedia communities and
>>> Wikimedia Foundation have created dedicated systems and staff to
>> regularly
>>> monitor and address risks. If a problem occurs, we learn, we improve, and
>>> we prepare to be better for next time.
>>>
>>> We condemn these sorts of attacks. They’re not just about taking
>> Wikipedia
>>> offline. Takedown attacks threaten everyone’s fundamental rights to
>> freely
>>> access and share information. We in the Wikimedia movement and Foundation
>>> are committed to protecting these rights for everyone.
>>>
>>> Right now, we’re continuing to work to restore access wherever you might
>>> be reading Wikipedia in the world. We’ll keep you posted."
>>>
>>>
>>> --
>>>
>>> Heather Walls (she/her)
>>>
>>> Chief Creative Officer
>>> Wikimedia Foundation
>>> https://wikimediafoundation.org
>>>
>>>
>>>
>>
>> --
>>
>> Heather Walls (she/her)
>>
>> Chief Creative Officer
>>
>> Wikimedia Foundation <https://wikimediafoundation.org/>
>> _______________________________________________
>> Wikimedia-l mailing list, guidelines at:
>> https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
>> https://meta.wikimedia.org/wiki/Wikimedia-l
>> New messages to: Wikimedia-l@lists.wikimedia.org
>> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
>> <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>
> _______________________________________________
> Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l
> New messages to: Wikimedia-l@lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>

_______________________________________________
Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l
New messages to: Wikimedia-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>

Re: [Wikimedia-l] Update on Wikimedia site and system outages [ In reply to ]

hwalls at wikimedia

Oct 27, 2019, 8:20 PM

Post #6 of 9 (889 views)

Permalink

Hi everyone,

We hope to have something for you in the next few weeks.

Regards,
Heather

On Mon, Oct 21, 2019 at 2:13 PM Benjamin Ikuta <benjaminikuta@gmail.com>
wrote:

>
>
> I am also curious about this.
>
>
>
> > On Oct 20, 2019, at 2:55 PM, RhinosF1 - <rhinosf1@gmail.com> wrote:
> >
> > Any idea of when an incident report may come out?
> >
> > RhinosF1
> > Wikimedia User & Incident Reporter
> >
> > On Fri, 20 Sep 2019 at 00:29, Heather Walls <hwalls@wikimedia.org>
> wrote:
> >
> >> Hello again,
> >>
> >> As a follow-up to my last note on the September 6th DDoS attack, we
> wanted
> >> to provide you with an update. There have been no further attacks in the
> >> last week and our sites are now running normally. Our SRE team is
> >> continuing to monitor the situation.
> >>
> >> Based on what we learned in this attack, our security and engineering
> teams
> >> are researching and putting together plans for more protection of our
> >> infrastructure to address any potential attacks in the future.
> >>
> >> We appreciate everyone’s support, particularly the folks on the SRE
> team,
> >> in helping to restore access.
> >>
> >> Yours,
> >> Heather
> >>
> >> On Sat, Sep 7, 2019 at 4:25 PM Heather Walls <hwalls@wikimedia.org>
> wrote:
> >>
> >>> Hello everyone,
> >>>
> >>> By now you are likely aware that the Wikimedia sites suffered from a
> >>> relatively significant botnet driven DDOS attack on September 6th,
> taking
> >>> them offline in several countries throughout the day. This primarily
> >>> affected Wikipedia access in Europe and the Middle East. We posted a
> >> short
> >>> update of the event on our website.[1]
> >>>
> >>> I would like to thank everyone who stepped up to support the
> restoration
> >>> of our projects, including the fast reporting of community members
> >>> throughout the world and our security and engineering teams who worked
> >> long
> >>> hours to address many complex issues surrounding the attack and our
> >>> response—the Site Reliability Engineering team in particular.
> >>>
> >>> The Wikimedia Foundation leadership team is proud to work with such
> >>> talented and dedicated staff and supporters.
> >>>
> >>> Yours,
> >>> Heather
> >>>
> >>>
> >>> 1.
> >>>
> >>
> https://wikimediafoundation.org/news/2019/09/07/malicious-attack-on-wikipedia-what-we-know-and-what-were-doing/
> >>>
> >>> "Today, Wikipedia was hit with a malicious attack that has taken it
> >>> offline in several countries for intermittent periods. The attack is
> >>> ongoing and our Site Reliability Engineering team is working hard to
> stop
> >>> it and restore access to the site.
> >>>
> >>> As one of the world’s most popular sites, Wikipedia sometimes attracts
> >>> “bad faith” actors. Along with the rest of the web, we operate in an
> >>> increasingly sophisticated and complex environment where threats are
> >>> continuously evolving. Because of this, the Wikimedia communities and
> >>> Wikimedia Foundation have created dedicated systems and staff to
> >> regularly
> >>> monitor and address risks. If a problem occurs, we learn, we improve,
> and
> >>> we prepare to be better for next time.
> >>>
> >>> We condemn these sorts of attacks. They’re not just about taking
> >> Wikipedia
> >>> offline. Takedown attacks threaten everyone’s fundamental rights to
> >> freely
> >>> access and share information. We in the Wikimedia movement and
> Foundation
> >>> are committed to protecting these rights for everyone.
> >>>
> >>> Right now, we’re continuing to work to restore access wherever you
> might
> >>> be reading Wikipedia in the world. We’ll keep you posted."
> >>>
> >>>
> >>> --
> >>>
> >>> Heather Walls (she/her)
> >>>
> >>> Chief Creative Officer
> >>> Wikimedia Foundation
> >>> https://wikimediafoundation.org
> >>>
> >>>
> >>>
> >>
> >> --
> >>
> >> Heather Walls (she/her)
> >>
> >> Chief Creative Officer
> >>
> >> Wikimedia Foundation <https://wikimediafoundation.org/>
> >> _______________________________________________
> >> Wikimedia-l mailing list, guidelines at:
> >> https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
> >> https://meta.wikimedia.org/wiki/Wikimedia-l
> >> New messages to: Wikimedia-l@lists.wikimedia.org
> >> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> >> <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>
> > _______________________________________________
> > Wikimedia-l mailing list, guidelines at:
> https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
> https://meta.wikimedia.org/wiki/Wikimedia-l
> > New messages to: Wikimedia-l@lists.wikimedia.org
> > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>
>
>
> _______________________________________________
> Wikimedia-l mailing list, guidelines at:
> https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
> https://meta.wikimedia.org/wiki/Wikimedia-l
> New messages to: Wikimedia-l@lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>

--

Heather Walls (she/her)

Chief Creative Officer

Wikimedia Foundation <https://wikimediafoundation.org/>
_______________________________________________
Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l
New messages to: Wikimedia-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>

Re: [Wikimedia-l] Update on Wikimedia site and system outages [ In reply to ]

haebwiki at gmail

Dec 8, 2019, 8:29 PM

Post #7 of 9 (873 views)

Permalink

Hi Benjamin and RhinosF1,

in the meantime, you could also check out the coverage in the September
edition of the Signpost:
https://en.wikipedia.org/wiki/Wikipedia:Wikipedia_Signpost/2019-09-30/In_the_media

(I'm sure however that the Foundation's incident report
<https://wikitech.wikimedia.org/wiki/Incident_documentation> will be more
detailed and informative, in particular regarding any significant changes
made to the traffic setup of Wikimedia sites and support obtained from
third-party companies.)

Regards, HaeB

On Sun, Oct 27, 2019 at 8:20 PM Heather Walls <hwalls@wikimedia.org> wrote:

> Hi everyone,
>
> We hope to have something for you in the next few weeks.
>
> Regards,
> Heather
>
>
> On Mon, Oct 21, 2019 at 2:13 PM Benjamin Ikuta <benjaminikuta@gmail.com>
> wrote:
>
> >
> >
> > I am also curious about this.
> >
> >
> >
> > > On Oct 20, 2019, at 2:55 PM, RhinosF1 - <rhinosf1@gmail.com> wrote:
> > >
> > > Any idea of when an incident report may come out?
> > >
> > > RhinosF1
> > > Wikimedia User & Incident Reporter
> > >
> > > On Fri, 20 Sep 2019 at 00:29, Heather Walls <hwalls@wikimedia.org>
> > wrote:
> > >
> > >> Hello again,
> > >>
> > >> As a follow-up to my last note on the September 6th DDoS attack, we
> > wanted
> > >> to provide you with an update. There have been no further attacks in
> the
> > >> last week and our sites are now running normally. Our SRE team is
> > >> continuing to monitor the situation.
> > >>
> > >> Based on what we learned in this attack, our security and engineering
> > teams
> > >> are researching and putting together plans for more protection of our
> > >> infrastructure to address any potential attacks in the future.
> > >>
> > >> We appreciate everyone’s support, particularly the folks on the SRE
> > team,
> > >> in helping to restore access.
> > >>
> > >> Yours,
> > >> Heather
> > >>
> > >> On Sat, Sep 7, 2019 at 4:25 PM Heather Walls <hwalls@wikimedia.org>
> > wrote:
> > >>
> > >>> Hello everyone,
> > >>>
> > >>> By now you are likely aware that the Wikimedia sites suffered from a
> > >>> relatively significant botnet driven DDOS attack on September 6th,
> > taking
> > >>> them offline in several countries throughout the day. This primarily
> > >>> affected Wikipedia access in Europe and the Middle East. We posted a
> > >> short
> > >>> update of the event on our website.[1]
> > >>>
> > >>> I would like to thank everyone who stepped up to support the
> > restoration
> > >>> of our projects, including the fast reporting of community members
> > >>> throughout the world and our security and engineering teams who
> worked
> > >> long
> > >>> hours to address many complex issues surrounding the attack and our
> > >>> response—the Site Reliability Engineering team in particular.
> > >>>
> > >>> The Wikimedia Foundation leadership team is proud to work with such
> > >>> talented and dedicated staff and supporters.
> > >>>
> > >>> Yours,
> > >>> Heather
> > >>>
> > >>>
> > >>> 1.
> > >>>
> > >>
> >
> https://wikimediafoundation.org/news/2019/09/07/malicious-attack-on-wikipedia-what-we-know-and-what-were-doing/
> > >>>
> > >>> "Today, Wikipedia was hit with a malicious attack that has taken it
> > >>> offline in several countries for intermittent periods. The attack is
> > >>> ongoing and our Site Reliability Engineering team is working hard to
> > stop
> > >>> it and restore access to the site.
> > >>>
> > >>> As one of the world’s most popular sites, Wikipedia sometimes
> attracts
> > >>> “bad faith” actors. Along with the rest of the web, we operate in an
> > >>> increasingly sophisticated and complex environment where threats are
> > >>> continuously evolving. Because of this, the Wikimedia communities and
> > >>> Wikimedia Foundation have created dedicated systems and staff to
> > >> regularly
> > >>> monitor and address risks. If a problem occurs, we learn, we improve,
> > and
> > >>> we prepare to be better for next time.
> > >>>
> > >>> We condemn these sorts of attacks. They’re not just about taking
> > >> Wikipedia
> > >>> offline. Takedown attacks threaten everyone’s fundamental rights to
> > >> freely
> > >>> access and share information. We in the Wikimedia movement and
> > Foundation
> > >>> are committed to protecting these rights for everyone.
> > >>>
> > >>> Right now, we’re continuing to work to restore access wherever you
> > might
> > >>> be reading Wikipedia in the world. We’ll keep you posted."
> > >>>
> > >>>
> > >>> --
> > >>>
> > >>> Heather Walls (she/her)
> > >>>
> > >>> Chief Creative Officer
> > >>> Wikimedia Foundation
> > >>> https://wikimediafoundation.org
> > >>>
> > >>>
> > >>>
> > >>
> > >> --
> > >>
> > >> Heather Walls (she/her)
> > >>
> > >> Chief Creative Officer
> > >>
> > >> Wikimedia Foundation <https://wikimediafoundation.org/>
> > >> _______________________________________________
> > >> Wikimedia-l mailing list, guidelines at:
> > >> https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
> > >> https://meta.wikimedia.org/wiki/Wikimedia-l
> > >> New messages to: Wikimedia-l@lists.wikimedia.org
> > >> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
> ,
> > >> <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>
> > > _______________________________________________
> > > Wikimedia-l mailing list, guidelines at:
> > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
> > https://meta.wikimedia.org/wiki/Wikimedia-l
> > > New messages to: Wikimedia-l@lists.wikimedia.org
> > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> > <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>
> >
> >
> > _______________________________________________
> > Wikimedia-l mailing list, guidelines at:
> > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
> > https://meta.wikimedia.org/wiki/Wikimedia-l
> > New messages to: Wikimedia-l@lists.wikimedia.org
> > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> > <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>
>
>
>
> --
>
> Heather Walls (she/her)
>
> Chief Creative Officer
>
> Wikimedia Foundation <https://wikimediafoundation.org/>
> _______________________________________________
> Wikimedia-l mailing list, guidelines at:
> https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
> https://meta.wikimedia.org/wiki/Wikimedia-l
> New messages to: Wikimedia-l@lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>
_______________________________________________
Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l
New messages to: Wikimedia-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>

Re: [Wikimedia-l] Update on Wikimedia site and system outages [ In reply to ]

jsalsman at gmail

Dec 8, 2019, 9:14 PM

Post #8 of 9 (869 views)

Permalink

Is this a good juncture at which to request that the Foundation ask
for a version of Encrypted-SNI from the CDNs which tunnels DNS?

https://twitter.com/jsalsman/status/1142172682751864832

https://twitter.com/jsalsman/status/1142940652851695616

https://twitter.com/jsalsman/status/1053786384463355905

I remember back in 2009 I had specified CDN upgrades, but the privacy
implications were considered prohibitive. In retrospect, establishing
an early relationship with the CDNs would have probably not been
particularly beneficial at the time, but they do have DDoS avoidance
worked out for the most part. There is no reason why we shouldn't ask
for an effective VPN by default from them.

On Sun, Dec 8, 2019 at 8:30 PM Tilman Bayer <haebwiki@gmail.com> wrote:
>
> Hi Benjamin and RhinosF1,
>
> in the meantime, you could also check out the coverage in the September
> edition of the Signpost:
> https://en.wikipedia.org/wiki/Wikipedia:Wikipedia_Signpost/2019-09-30/In_the_media
>
>
> (I'm sure however that the Foundation's incident report
> <https://wikitech.wikimedia.org/wiki/Incident_documentation> will be more
> detailed and informative, in particular regarding any significant changes
> made to the traffic setup of Wikimedia sites and support obtained from
> third-party companies.)
>
> Regards, HaeB
>
> On Sun, Oct 27, 2019 at 8:20 PM Heather Walls <hwalls@wikimedia.org> wrote:
>
> > Hi everyone,
> >
> > We hope to have something for you in the next few weeks.
> >
> > Regards,
> > Heather
> >
> >
> > On Mon, Oct 21, 2019 at 2:13 PM Benjamin Ikuta <benjaminikuta@gmail.com>
> > wrote:
> >
> > >
> > >
> > > I am also curious about this.
> > >
> > >
> > >
> > > > On Oct 20, 2019, at 2:55 PM, RhinosF1 - <rhinosf1@gmail.com> wrote:
> > > >
> > > > Any idea of when an incident report may come out?
> > > >
> > > > RhinosF1
> > > > Wikimedia User & Incident Reporter
> > > >
> > > > On Fri, 20 Sep 2019 at 00:29, Heather Walls <hwalls@wikimedia.org>
> > > wrote:
> > > >
> > > >> Hello again,
> > > >>
> > > >> As a follow-up to my last note on the September 6th DDoS attack, we
> > > wanted
> > > >> to provide you with an update. There have been no further attacks in
> > the
> > > >> last week and our sites are now running normally. Our SRE team is
> > > >> continuing to monitor the situation.
> > > >>
> > > >> Based on what we learned in this attack, our security and engineering
> > > teams
> > > >> are researching and putting together plans for more protection of our
> > > >> infrastructure to address any potential attacks in the future.
> > > >>
> > > >> We appreciate everyone’s support, particularly the folks on the SRE
> > > team,
> > > >> in helping to restore access.
> > > >>
> > > >> Yours,
> > > >> Heather
> > > >>
> > > >> On Sat, Sep 7, 2019 at 4:25 PM Heather Walls <hwalls@wikimedia.org>
> > > wrote:
> > > >>
> > > >>> Hello everyone,
> > > >>>
> > > >>> By now you are likely aware that the Wikimedia sites suffered from a
> > > >>> relatively significant botnet driven DDOS attack on September 6th,
> > > taking
> > > >>> them offline in several countries throughout the day. This primarily
> > > >>> affected Wikipedia access in Europe and the Middle East. We posted a
> > > >> short
> > > >>> update of the event on our website.[1]
> > > >>>
> > > >>> I would like to thank everyone who stepped up to support the
> > > restoration
> > > >>> of our projects, including the fast reporting of community members
> > > >>> throughout the world and our security and engineering teams who
> > worked
> > > >> long
> > > >>> hours to address many complex issues surrounding the attack and our
> > > >>> response—the Site Reliability Engineering team in particular.
> > > >>>
> > > >>> The Wikimedia Foundation leadership team is proud to work with such
> > > >>> talented and dedicated staff and supporters.
> > > >>>
> > > >>> Yours,
> > > >>> Heather
> > > >>>
> > > >>>
> > > >>> 1.
> > > >>>
> > > >>
> > >
> > https://wikimediafoundation.org/news/2019/09/07/malicious-attack-on-wikipedia-what-we-know-and-what-were-doing/
> > > >>>
> > > >>> "Today, Wikipedia was hit with a malicious attack that has taken it
> > > >>> offline in several countries for intermittent periods. The attack is
> > > >>> ongoing and our Site Reliability Engineering team is working hard to
> > > stop
> > > >>> it and restore access to the site.
> > > >>>
> > > >>> As one of the world’s most popular sites, Wikipedia sometimes
> > attracts
> > > >>> “bad faith” actors. Along with the rest of the web, we operate in an
> > > >>> increasingly sophisticated and complex environment where threats are
> > > >>> continuously evolving. Because of this, the Wikimedia communities and
> > > >>> Wikimedia Foundation have created dedicated systems and staff to
> > > >> regularly
> > > >>> monitor and address risks. If a problem occurs, we learn, we improve,
> > > and
> > > >>> we prepare to be better for next time.
> > > >>>
> > > >>> We condemn these sorts of attacks. They’re not just about taking
> > > >> Wikipedia
> > > >>> offline. Takedown attacks threaten everyone’s fundamental rights to
> > > >> freely
> > > >>> access and share information. We in the Wikimedia movement and
> > > Foundation
> > > >>> are committed to protecting these rights for everyone.
> > > >>>
> > > >>> Right now, we’re continuing to work to restore access wherever you
> > > might
> > > >>> be reading Wikipedia in the world. We’ll keep you posted."
> > > >>>
> > > >>>
> > > >>> --
> > > >>>
> > > >>> Heather Walls (she/her)
> > > >>>
> > > >>> Chief Creative Officer
> > > >>> Wikimedia Foundation
> > > >>> https://wikimediafoundation.org
> > > >>>
> > > >>>
> > > >>>
> > > >>
> > > >> --
> > > >>
> > > >> Heather Walls (she/her)
> > > >>
> > > >> Chief Creative Officer
> > > >>
> > > >> Wikimedia Foundation <https://wikimediafoundation.org/>
> > > >> _______________________________________________
> > > >> Wikimedia-l mailing list, guidelines at:
> > > >> https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
> > > >> https://meta.wikimedia.org/wiki/Wikimedia-l
> > > >> New messages to: Wikimedia-l@lists.wikimedia.org
> > > >> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
> > ,
> > > >> <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>
> > > > _______________________________________________
> > > > Wikimedia-l mailing list, guidelines at:
> > > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
> > > https://meta.wikimedia.org/wiki/Wikimedia-l
> > > > New messages to: Wikimedia-l@lists.wikimedia.org
> > > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> > > <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>
> > >
> > >
> > > _______________________________________________
> > > Wikimedia-l mailing list, guidelines at:
> > > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
> > > https://meta.wikimedia.org/wiki/Wikimedia-l
> > > New messages to: Wikimedia-l@lists.wikimedia.org
> > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> > > <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>
> >
> >
> >
> > --
> >
> > Heather Walls (she/her)
> >
> > Chief Creative Officer
> >
> > Wikimedia Foundation <https://wikimediafoundation.org/>
> > _______________________________________________
> > Wikimedia-l mailing list, guidelines at:
> > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
> > https://meta.wikimedia.org/wiki/Wikimedia-l
> > New messages to: Wikimedia-l@lists.wikimedia.org
> > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> > <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>
> _______________________________________________
> Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l
> New messages to: Wikimedia-l@lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>

_______________________________________________
Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l
New messages to: Wikimedia-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>

Re: [Wikimedia-l] Update on Wikimedia site and system outages [ In reply to ]

wiki.pine at gmail

Feb 28, 2020, 12:35 PM

Post #9 of 9 (767 views)

Permalink

Hello,

A gentle reminder that four months have passed since the last update
to this thread from WMF regarding the September DDoS incident. When
will an incident report be published, or has a decision been made not
to publish a report?

Thanks,
Pine
( https://meta.wikimedia.org/wiki/User:Pine )

On Mon, Oct 28, 2019 at 3:20 AM Heather Walls <hwalls@wikimedia.org> wrote:
>
> Hi everyone,
>
> We hope to have something for you in the next few weeks.
>
> Regards,
> Heather
>
>
> On Mon, Oct 21, 2019 at 2:13 PM Benjamin Ikuta <benjaminikuta@gmail.com>
> wrote:
>
> >
> >
> > I am also curious about this.
> >
> >
> >
> > > On Oct 20, 2019, at 2:55 PM, RhinosF1 - <rhinosf1@gmail.com> wrote:
> > >
> > > Any idea of when an incident report may come out?
> > >
> > > RhinosF1
> > > Wikimedia User & Incident Reporter
> > >
> > > On Fri, 20 Sep 2019 at 00:29, Heather Walls <hwalls@wikimedia.org>
> > wrote:
> > >
> > >> Hello again,
> > >>
> > >> As a follow-up to my last note on the September 6th DDoS attack, we
> > wanted
> > >> to provide you with an update. There have been no further attacks in the
> > >> last week and our sites are now running normally. Our SRE team is
> > >> continuing to monitor the situation.
> > >>
> > >> Based on what we learned in this attack, our security and engineering
> > teams
> > >> are researching and putting together plans for more protection of our
> > >> infrastructure to address any potential attacks in the future.
> > >>
> > >> We appreciate everyone’s support, particularly the folks on the SRE
> > team,
> > >> in helping to restore access.
> > >>
> > >> Yours,
> > >> Heather
> > >>
> > >> On Sat, Sep 7, 2019 at 4:25 PM Heather Walls <hwalls@wikimedia.org>
> > wrote:
> > >>
> > >>> Hello everyone,
> > >>>
> > >>> By now you are likely aware that the Wikimedia sites suffered from a
> > >>> relatively significant botnet driven DDOS attack on September 6th,
> > taking
> > >>> them offline in several countries throughout the day. This primarily
> > >>> affected Wikipedia access in Europe and the Middle East. We posted a
> > >> short
> > >>> update of the event on our website.[1]
> > >>>
> > >>> I would like to thank everyone who stepped up to support the
> > restoration
> > >>> of our projects, including the fast reporting of community members
> > >>> throughout the world and our security and engineering teams who worked
> > >> long
> > >>> hours to address many complex issues surrounding the attack and our
> > >>> response—the Site Reliability Engineering team in particular.
> > >>>
> > >>> The Wikimedia Foundation leadership team is proud to work with such
> > >>> talented and dedicated staff and supporters.
> > >>>
> > >>> Yours,
> > >>> Heather
> > >>>
> > >>>
> > >>> 1.
> > >>>
> > >>
> > https://wikimediafoundation.org/news/2019/09/07/malicious-attack-on-wikipedia-what-we-know-and-what-were-doing/
> > >>>
> > >>> "Today, Wikipedia was hit with a malicious attack that has taken it
> > >>> offline in several countries for intermittent periods. The attack is
> > >>> ongoing and our Site Reliability Engineering team is working hard to
> > stop
> > >>> it and restore access to the site.
> > >>>
> > >>> As one of the world’s most popular sites, Wikipedia sometimes attracts
> > >>> “bad faith” actors. Along with the rest of the web, we operate in an
> > >>> increasingly sophisticated and complex environment where threats are
> > >>> continuously evolving. Because of this, the Wikimedia communities and
> > >>> Wikimedia Foundation have created dedicated systems and staff to
> > >> regularly
> > >>> monitor and address risks. If a problem occurs, we learn, we improve,
> > and
> > >>> we prepare to be better for next time.
> > >>>
> > >>> We condemn these sorts of attacks. They’re not just about taking
> > >> Wikipedia
> > >>> offline. Takedown attacks threaten everyone’s fundamental rights to
> > >> freely
> > >>> access and share information. We in the Wikimedia movement and
> > Foundation
> > >>> are committed to protecting these rights for everyone.
> > >>>
> > >>> Right now, we’re continuing to work to restore access wherever you
> > might
> > >>> be reading Wikipedia in the world. We’ll keep you posted."
> > >>>
> > >>>
> > >>> --
> > >>>
> > >>> Heather Walls (she/her)
> > >>>
> > >>> Chief Creative Officer
> > >>> Wikimedia Foundation
> > >>> https://wikimediafoundation.org
> > >>>
> > >>>
> > >>>
> > >>
> > >> --
> > >>
> > >> Heather Walls (she/her)
> > >>
> > >> Chief Creative Officer
> > >>
> > >> Wikimedia Foundation <https://wikimediafoundation.org/>
> > >> _______________________________________________
> > >> Wikimedia-l mailing list, guidelines at:
> > >> https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
> > >> https://meta.wikimedia.org/wiki/Wikimedia-l
> > >> New messages to: Wikimedia-l@lists.wikimedia.org
> > >> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> > >> <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>
> > > _______________________________________________
> > > Wikimedia-l mailing list, guidelines at:
> > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
> > https://meta.wikimedia.org/wiki/Wikimedia-l
> > > New messages to: Wikimedia-l@lists.wikimedia.org
> > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> > <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>
> >
> >
> > _______________________________________________
> > Wikimedia-l mailing list, guidelines at:
> > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
> > https://meta.wikimedia.org/wiki/Wikimedia-l
> > New messages to: Wikimedia-l@lists.wikimedia.org
> > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> > <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>
>
>
>
> --
>
> Heather Walls (she/her)
>
> Chief Creative Officer
>
> Wikimedia Foundation <https://wikimediafoundation.org/>
> _______________________________________________
> Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l
> New messages to: Wikimedia-l@lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>

_______________________________________________
Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l
New messages to: Wikimedia-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>