Mailing List Archive

Facebook
Its been around 30 minutes. Is this the same result from less than 20 hours
ago?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20100923/0bb9d965/attachment.html>
Facebook [ In reply to ]
Facebook appears to be having some issues. Unreachable from multiple
providers in this neck of the woods, unreachable from
downforeveryoneorjustme.com, but reachable via AT&T mobile. I'm sure
customers are going to start bitching soon if they can't get to their
Farmville.

--
William S. Duncanson
caesar at starkreality.com
Facebook [ In reply to ]
Yes, it is down. Let's get back to work!

http://downforeveryoneorjustme.com/facebook.com

Josh Luthman
Office: 937-552-2340
Direct: 937-552-2343
1100 Wayne St
Suite 1337
Troy, OH 45373



On Thu, Sep 23, 2010 at 3:21 PM, jon <relicwr at gmail.com> wrote:
> Its been around 30 minutes. Is this the same result from less than 20 hours
> ago?
>
> _______________________________________________
> Outages mailing list
> Outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages
>
>
Facebook [ In reply to ]
On 9/23/2010 12:28, Josh Luthman wrote:
> Yes, it is down. Let's get back to work!
>
> http://downforeveryoneorjustme.com/facebook.com
>


I didn't even notice since I *was* working. ;)

~Seth
Facebook [ In reply to ]
> Its been around 30 minutes. Is this the same result from less than 20 hours ago?
>
Seems that is the case.

Via Facebook:

> API Latency Issues
> Today, September 23 at 3:07pm
> We are currently experiencing latency issues with the API, and we are actively investigating. We will provide an update when either the issue is resolved or we have an ETA for resolution.

http://developers.facebook.com/live_status/

-Matt Dodd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20100923/c32bc547/attachment.html>
Facebook [ In reply to ]
---- Original Message -----
From: "jon" <relicwr@gmail.com>

> Its been around 30 minutes. Is this the same result from less than 20 hours
ago?

Based on Pete Cashmore's comments at Mashable, I think not, but we'll see.
It's just come back up and is slowly rolling out, both vertically and
horizontally.

Cheers,
-- jra
--
Jay R. Ashworth Baylink jra at baylink.com
Designer The Things I Think RFC 2100
Ashworth & Associates http://baylink.pitas.com '87 e24
St Petersburg FL USA http://photo.imageinc.us +1 727 647 1274

Start a man a fire, and he'll be warm all night.
Set a man on fire, and he'll be warm for the rest of his life.
Facebook [ In reply to ]
--- josh at imaginenetworksllc.com wrote:
From: Josh Luthman <josh@imaginenetworksllc.com>

Yes, it is down. Let's get back to work!
----------------------------------------------



Ewwww, that's dripping with manager-itis. Quick, wipe it off! ;-)

scott












































----------------------
-----------------------
----------------------
Facebook [ In reply to ]
---- Original Message -----
From: "Seth Mattinen" <sethm@rollernet.us>
To: outages at outages.org
Sent: Thursday, September 23, 2010 3:29:41 PM
Subject: Re: [outages] Facebook

> On 9/23/2010 12:28, Josh Luthman wrote:
> > Yes, it is down. Let's get back to work!
> > http://downforeveryoneorjustme.com/facebook.com

> I didn't even notice since I *was* working. ;)

Just to provide the customary reminder: for some of us, "taking calls about
why customers can't get to Facebook" *is* working.

Cheers,
-- jr 'but being snarky *is* fun, ain't it?' a

--
Jay R. Ashworth Baylink jra at baylink.com
Designer The Things I Think RFC 2100
Ashworth & Associates http://baylink.pitas.com '87 e24
St Petersburg FL USA http://photo.imageinc.us +1 727 647 1274

Start a man a fire, and he'll be warm all night.
Set a man on fire, and he'll be warm for the rest of his life.
Facebook [ In reply to ]
Maybe Facebook uses XO for connectivity, and thats why they are down!

Just had a through an XO dig in there....

-John

On Sep 23, 2010, at 3:40 PM, Jay R. Ashworth wrote:

> ---- Original Message -----
> From: "Seth Mattinen" <sethm at rollernet.us>
> To: outages at outages.org
> Sent: Thursday, September 23, 2010 3:29:41 PM
> Subject: Re: [outages] Facebook
>
>> On 9/23/2010 12:28, Josh Luthman wrote:
>>> Yes, it is down. Let's get back to work!
>>> http://downforeveryoneorjustme.com/facebook.com
>
>> I didn't even notice since I *was* working. ;)
>
> Just to provide the customary reminder: for some of us, "taking
> calls about
> why customers can't get to Facebook" *is* working.
>
> Cheers,
> -- jr 'but being snarky *is* fun, ain't it?' a
>
> --
> Jay R. Ashworth Baylink jra at baylink.com
> Designer The Things I
> Think RFC 2100
> Ashworth & Associates http://
> baylink.pitas.com '87 e24
> St Petersburg FL USA http://photo.imageinc.us +1
> 727 647 1274
>
> Start a man a fire, and he'll be warm all night.
> Set a man on fire, and he'll be warm for the rest of his life.
> _______________________________________________
> Outages mailing list
> Outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages
Facebook [ In reply to ]
On Thu, 23 Sep 2010, Josh Luthman wrote:

> Yes, it is down. Let's get back to work!

Funny, facebook being down == cant work for some of us. ;)



--
david raistrick http://www.netmeister.org/news/learn2quote.html
drais at icantclick.org http://www.expita.com/nomime.html
Facebook [ In reply to ]
Once upon a time, jon <relicwr at gmail.com> said:
> Its been around 30 minutes. Is this the same result from less than 20 hours
> ago?

Interesting, www.facebook.com switched to an Akamai IP for me at about
19:30 UTC.
--
Chris Adams <cmadams at hiwaay.net>
Systems and Network Administrator - HiWAAY Internet Services
I don't speak for anybody but myself - that's enough trouble.
Facebook [ In reply to ]
Seeing different things from different places, hasn't fully propagated yet.

dig www.facebook.com @8.8.8.8 soa |grep -P '[0-9]{10}'
g.akamai.net. 471 IN SOA n0g.akamai.net. hostmaster.akamai.com. 1285274188 1000 1000 1000 1800

dig www.facebook.com @4.2.2.1 soa |grep -P '[0-9]{10}'
x.akamai.net. 900 IN SOA n0x.akamai.net. hostmaster.akamai.com. 1285274726 1000 1000 1000 1800


So 8.8.8.8 is definitely the older version, and it shows their IP:
dig www.facebook.com @8.8.8.8 +short
69.63.189.11

And 4.2.2.1 is fresher and appears to show a switch to akamai:
dig www.facebook.com @4.2.2.1 +short
wwws.ak.facebook.com.edgesuite.net.
a1254.x.akamai.net.
66.160.206.18
65.49.92.113

Somebody dun goofed.

-wil

On Sep 23, 2010, at 1:18 PM, Chris Adams wrote:

> Once upon a time, jon <relicwr at gmail.com> said:
>> Its been around 30 minutes. Is this the same result from less than 20 hours
>> ago?
>
> Interesting, www.facebook.com switched to an Akamai IP for me at about
> 19:30 UTC.
> --
> Chris Adams <cmadams at hiwaay.net>
> Systems and Network Administrator - HiWAAY Internet Services
> I don't speak for anybody but myself - that's enough trouble.
> _______________________________________________
> Outages mailing list
> Outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages
Facebook [ In reply to ]
On 9/23/2010 15:18, Chris Adams wrote:
> Once upon a time, jon <relicwr at gmail.com> said:
>> Its been around 30 minutes. Is this the same result from less than 20 hours
>> ago?
>
> Interesting, www.facebook.com switched to an Akamai IP for me at about
> 19:30 UTC.

I thought FB always ran throuhg Akamai.

I'm now getting
quote
Service Unavailable - DNS failure
The server is temporarily unable to service your request. Please try
again later.

Reference #11.8397a746.1285274927.3ef1e4f0
unquote.

I think that is an Akamai error, isn't it?

The browser address is: http://www.facebook.com/home.php?#!/?ref=home




--
Somebody should have said:
A democracy is two wolves and a lamb voting on what to have for dinner.

Freedom under a constitutional republic is a well armed lamb contesting
the vote.

Requiescas in pace o email
Ex turpi causa non oritur actio
Eppure si rinfresca

ICBM Targeting Information: http://tinyurl.com/4sqczs
http://tinyurl.com/7tp8ml
Facebook [ In reply to ]
http://developers.facebook.com/live_status


-----Original Message-----
From: outages-bounces@outages.org [mailto:outages-bounces@outages.org] On Behalf Of Larry Sheldon
Sent: Thursday, September 23, 2010 1:54 PM
To: outages at outages.org
Subject: Re: [outages] Facebook

On 9/23/2010 15:18, Chris Adams wrote:
> Once upon a time, jon <relicwr at gmail.com> said:
>> Its been around 30 minutes. Is this the same result from less than 20
>> hours ago?
>
> Interesting, www.facebook.com switched to an Akamai IP for me at about
> 19:30 UTC.

I thought FB always ran throuhg Akamai.

I'm now getting
quote
Service Unavailable - DNS failure
The server is temporarily unable to service your request. Please try again later.

Reference #11.8397a746.1285274927.3ef1e4f0 unquote.

I think that is an Akamai error, isn't it?

The browser address is: http://www.facebook.com/home.php?#!/?ref=home




--
Somebody should have said:
A democracy is two wolves and a lamb voting on what to have for dinner.

Freedom under a constitutional republic is a well armed lamb contesting the vote.

Requiescas in pace o email
Ex turpi causa non oritur actio
Eppure si rinfresca

ICBM Targeting Information: http://tinyurl.com/4sqczs http://tinyurl.com/7tp8ml


_______________________________________________
Outages mailing list
Outages at outages.org
https://puck.nether.net/mailman/listinfo/outages
Facebook [ In reply to ]
On 9/23/2010 15:53, Larry Sheldon wrote:
> On 9/23/2010 15:18, Chris Adams wrote:
>> Once upon a time, jon <relicwr at gmail.com> said:
>>> Its been around 30 minutes. Is this the same result from less than 20 hours
>>> ago?
>>
>> Interesting, www.facebook.com switched to an Akamai IP for me at about
>> 19:30 UTC.
>
> I thought FB always ran throuhg Akamai.
>
> I'm now getting
> quote
> Service Unavailable - DNS failure
> The server is temporarily unable to service your request. Please try
> again later.
>
> Reference #11.8397a746.1285274927.3ef1e4f0
> unquote.
>
> I think that is an Akamai error, isn't it?
>
> The browser address is: http://www.facebook.com/home.php?#!/?ref=home

Daughter in KCMO sez it (FB) was down, now up to her.

Still down to me.

Trace www.facebook.com goes to
a-70-183-191-122.deploy.akamaitechnologies.com [
70.183.191.122]


--
Somebody should have said:
A democracy is two wolves and a lamb voting on what to have for dinner.

Freedom under a constitutional republic is a well armed lamb contesting
the vote.

Requiescas in pace o email
Ex turpi causa non oritur actio
Eppure si rinfresca

ICBM Targeting Information: http://tinyurl.com/4sqczs
http://tinyurl.com/7tp8ml
Facebook [ In reply to ]
Once upon a time, Larry Sheldon <LarrySheldon at cox.net> said:
> On 9/23/2010 15:18, Chris Adams wrote:
> > Interesting, www.facebook.com switched to an Akamai IP for me at about
> > 19:30 UTC.
>
> I thought FB always ran throuhg Akamai.

Well, it changed from an external host to our local Akamai servers. Now
that it is back up, it is back to external servers.

Looks like this might have been an Akamai problem then.
--
Chris Adams <cmadams at hiwaay.net>
Systems and Network Administrator - HiWAAY Internet Services
I don't speak for anybody but myself - that's enough trouble.
Facebook [ In reply to ]
Works for me 5:16PM EST

C:\Users\jluthman>dig facebook.com

; <<>> DiG 9.3.2 <<>> facebook.com
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1929
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;facebook.com. IN A

;; ANSWER SECTION:
facebook.com. 3166 IN A 69.63.189.11
facebook.com. 3166 IN A 69.63.189.16
facebook.com. 3166 IN A 69.63.181.12

;; Query time: 4 msec
;; SERVER: 192.168.21.1#53(192.168.21.1)
;; WHEN: Thu Sep 23 17:16:12 2010
;; MSG SIZE rcvd: 78

Josh Luthman
Office: 937-552-2340
Direct: 937-552-2343
1100 Wayne St
Suite 1337
Troy, OH 45373



On Thu, Sep 23, 2010 at 4:59 PM, Larry Sheldon <LarrySheldon at cox.net> wrote:
> On 9/23/2010 15:53, Larry Sheldon wrote:
>> On 9/23/2010 15:18, Chris Adams wrote:
>>> Once upon a time, jon <relicwr at gmail.com> said:
>>>> Its been around 30 minutes. Is this the same result from less than 20 hours
>>>> ago?
>>>
>>> Interesting, www.facebook.com switched to an Akamai IP for me at about
>>> 19:30 UTC.
>>
>> I thought FB always ran throuhg Akamai.
>>
>> I'm now getting
>> quote
>> Service Unavailable - DNS failure
>> The server is temporarily unable to service your request. Please try
>> again later.
>>
>> Reference #11.8397a746.1285274927.3ef1e4f0
>> unquote.
>>
>> I think that is an Akamai error, isn't it?
>>
>> The browser address is: ?http://www.facebook.com/home.php?#!/?ref=home
>
> Daughter in KCMO sez it (FB) was down, now up to her.
>
> Still down to me.
>
> Trace www.facebook.com goes to
> a-70-183-191-122.deploy.akamaitechnologies.com [
> 70.183.191.122]
>
>
> --
> Somebody should have said:
> A democracy is two wolves and a lamb voting on what to have for dinner.
>
> Freedom under a constitutional republic is a well armed lamb contesting
> the vote.
>
> Requiescas in pace o email
> Ex turpi causa non oritur actio
> Eppure si rinfresca
>
> ICBM Targeting Information: ?http://tinyurl.com/4sqczs
> http://tinyurl.com/7tp8ml
>
>
> _______________________________________________
> Outages mailing list
> Outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages
>
Facebook [ In reply to ]
The length of the outage could actually affect Sept productivity. Scary how
many need their Farmville.
On Sep 23, 2010 4:09 PM, "david raistrick" <drais at icantclick.org> wrote:
> On Thu, 23 Sep 2010, Josh Luthman wrote:
>
>> Yes, it is down. Let's get back to work!
>
> Funny, facebook being down == cant work for some of us. ;)
>
>
>
> --
> david raistrick http://www.netmeister.org/news/learn2quote.html
> drais at icantclick.org http://www.expita.com/nomime.html
>
> _______________________________________________
> Outages mailing list
> Outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20100923/429fe9f2/attachment.html>
Facebook [ In reply to ]
DItto as of 2 min ago:

<<>> DiG 9.6.0-APPLE-P2 <<>> facebook.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 22921
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;facebook.com. IN A

;; ANSWER SECTION:
facebook.com. 3427 IN A 69.63.189.16
facebook.com. 3427 IN A 69.63.189.11
facebook.com. 3427 IN A 69.63.181.12

;; Query time: 106 msec
;; SERVER: 208.67.220.220#53(208.67.220.220)
;; WHEN: Thu Sep 23 17:33:42 2010
;; MSG SIZE rcvd: 78

~jason

On Thu, Sep 23, 2010 at 5:16 PM, Josh Luthman
<josh at imaginenetworksllc.com>wrote:

> Works for me 5:16PM EST
>
> C:\Users\jluthman>dig facebook.com
>
> ; <<>> DiG 9.3.2 <<>> facebook.com
> ;; global options: printcmd
> ;; Got answer:
> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1929
> ;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0
>
> ;; QUESTION SECTION:
> ;facebook.com. IN A
>
> ;; ANSWER SECTION:
> facebook.com. 3166 IN A 69.63.189.11
> facebook.com. 3166 IN A 69.63.189.16
> facebook.com. 3166 IN A 69.63.181.12
>
> ;; Query time: 4 msec
> ;; SERVER: 192.168.21.1#53(192.168.21.1)
> ;; WHEN: Thu Sep 23 17:16:12 2010
> ;; MSG SIZE rcvd: 78
>
> Josh Luthman
> Office: 937-552-2340
> Direct: 937-552-2343
> 1100 Wayne St
> Suite 1337
> Troy, OH 45373
>
>
>
> On Thu, Sep 23, 2010 at 4:59 PM, Larry Sheldon <LarrySheldon at cox.net>
> wrote:
> > On 9/23/2010 15:53, Larry Sheldon wrote:
> >> On 9/23/2010 15:18, Chris Adams wrote:
> >>> Once upon a time, jon <relicwr at gmail.com> said:
> >>>> Its been around 30 minutes. Is this the same result from less than 20
> hours
> >>>> ago?
> >>>
> >>> Interesting, www.facebook.com switched to an Akamai IP for me at about
> >>> 19:30 UTC.
> >>
> >> I thought FB always ran throuhg Akamai.
> >>
> >> I'm now getting
> >> quote
> >> Service Unavailable - DNS failure
> >> The server is temporarily unable to service your request. Please try
> >> again later.
> >>
> >> Reference #11.8397a746.1285274927.3ef1e4f0
> >> unquote.
> >>
> >> I think that is an Akamai error, isn't it?
> >>
> >> The browser address is: http://www.facebook.com/home.php?#!/?ref=home
> >
> > Daughter in KCMO sez it (FB) was down, now up to her.
> >
> > Still down to me.
> >
> > Trace www.facebook.com goes to
> > a-70-183-191-122.deploy.akamaitechnologies.com [
> > 70.183.191.122]
> >
> >
> > --
> > Somebody should have said:
> > A democracy is two wolves and a lamb voting on what to have for dinner.
> >
> > Freedom under a constitutional republic is a well armed lamb contesting
> > the vote.
> >
> > Requiescas in pace o email
> > Ex turpi causa non oritur actio
> > Eppure si rinfresca
> >
> > ICBM Targeting Information: http://tinyurl.com/4sqczs
> > http://tinyurl.com/7tp8ml
> >
> >
> > _______________________________________________
> > Outages mailing list
> > Outages at outages.org
> > https://puck.nether.net/mailman/listinfo/outages
> >
>
> _______________________________________________
> Outages mailing list
> Outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20100923/a572a6a1/attachment-0001.html>
Facebook [ In reply to ]
Its worth noting that facebook.com and www.facebook.com are handled
differently. facebook.com is a limited set of RR IPs that redirect to www.
www is a gslb'ed domain.

On Thu, Sep 23, 2010 at 2:34 PM, Jason L. Sparks <jlsparks at gmail.com> wrote:

> DItto as of 2 min ago:
>
> <<>> DiG 9.6.0-APPLE-P2 <<>> facebook.com
> ;; global options: +cmd
> ;; Got answer:
> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 22921
> ;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0
>
> ;; QUESTION SECTION:
> ;facebook.com. IN A
>
> ;; ANSWER SECTION:
> facebook.com. 3427 IN A 69.63.189.16
> facebook.com. 3427 IN A 69.63.189.11
> facebook.com. 3427 IN A 69.63.181.12
>
> ;; Query time: 106 msec
> ;; SERVER: 208.67.220.220#53(208.67.220.220)
> ;; WHEN: Thu Sep 23 17:33:42 2010
> ;; MSG SIZE rcvd: 78
>
> ~jason
>
> On Thu, Sep 23, 2010 at 5:16 PM, Josh Luthman <josh at imaginenetworksllc.com
> > wrote:
>
>> Works for me 5:16PM EST
>>
>> C:\Users\jluthman>dig facebook.com
>>
>> ; <<>> DiG 9.3.2 <<>> facebook.com
>> ;; global options: printcmd
>> ;; Got answer:
>> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1929
>> ;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0
>>
>> ;; QUESTION SECTION:
>> ;facebook.com. IN A
>>
>> ;; ANSWER SECTION:
>> facebook.com. 3166 IN A 69.63.189.11
>> facebook.com. 3166 IN A 69.63.189.16
>> facebook.com. 3166 IN A 69.63.181.12
>>
>> ;; Query time: 4 msec
>> ;; SERVER: 192.168.21.1#53(192.168.21.1)
>> ;; WHEN: Thu Sep 23 17:16:12 2010
>> ;; MSG SIZE rcvd: 78
>>
>> Josh Luthman
>> Office: 937-552-2340
>> Direct: 937-552-2343
>> 1100 Wayne St
>> Suite 1337
>> Troy, OH 45373
>>
>>
>>
>> On Thu, Sep 23, 2010 at 4:59 PM, Larry Sheldon <LarrySheldon at cox.net>
>> wrote:
>> > On 9/23/2010 15:53, Larry Sheldon wrote:
>> >> On 9/23/2010 15:18, Chris Adams wrote:
>> >>> Once upon a time, jon <relicwr at gmail.com> said:
>> >>>> Its been around 30 minutes. Is this the same result from less than 20
>> hours
>> >>>> ago?
>> >>>
>> >>> Interesting, www.facebook.com switched to an Akamai IP for me at
>> about
>> >>> 19:30 UTC.
>> >>
>> >> I thought FB always ran throuhg Akamai.
>> >>
>> >> I'm now getting
>> >> quote
>> >> Service Unavailable - DNS failure
>> >> The server is temporarily unable to service your request. Please try
>> >> again later.
>> >>
>> >> Reference #11.8397a746.1285274927.3ef1e4f0
>> >> unquote.
>> >>
>> >> I think that is an Akamai error, isn't it?
>> >>
>> >> The browser address is: http://www.facebook.com/home.php?#!/?ref=home
>> >
>> > Daughter in KCMO sez it (FB) was down, now up to her.
>> >
>> > Still down to me.
>> >
>> > Trace www.facebook.com goes to
>> > a-70-183-191-122.deploy.akamaitechnologies.com [
>> > 70.183.191.122]
>> >
>> >
>> > --
>> > Somebody should have said:
>> > A democracy is two wolves and a lamb voting on what to have for dinner.
>> >
>> > Freedom under a constitutional republic is a well armed lamb contesting
>> > the vote.
>> >
>> > Requiescas in pace o email
>> > Ex turpi causa non oritur actio
>> > Eppure si rinfresca
>> >
>> > ICBM Targeting Information: http://tinyurl.com/4sqczs
>> > http://tinyurl.com/7tp8ml
>> >
>> >
>> > _______________________________________________
>> > Outages mailing list
>> > Outages at outages.org
>> > https://puck.nether.net/mailman/listinfo/outages
>> >
>>
>> _______________________________________________
>> Outages mailing list
>> Outages at outages.org
>> https://puck.nether.net/mailman/listinfo/outages
>>
>
>
> _______________________________________________
> Outages mailing list
> Outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20100923/3417962a/attachment.html>
Facebook [ In reply to ]
Also up for me in Central Oregon:

root at oem-desktop:~# dig facebook.com

; <<>> DiG 9.5.1-P2.1 <<>> facebook.com
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 55496
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 5, ADDITIONAL: 5

;; QUESTION SECTION:
;facebook.com. IN A

;; ANSWER SECTION:
facebook.com. 890 IN A 69.63.189.11
facebook.com. 890 IN A 69.63.189.16
facebook.com. 890 IN A 69.63.181.12

;; AUTHORITY SECTION:
facebook.com. 130293 IN NS ns1.facebook.com.
facebook.com. 130293 IN NS ns2.facebook.com.
facebook.com. 130293 IN NS ns4.facebook.com.
facebook.com. 130293 IN NS ns3.facebook.com.
facebook.com. 130293 IN NS ns5.facebook.com.

;; ADDITIONAL SECTION:
ns1.facebook.com. 1436 IN A 204.74.66.132
ns2.facebook.com. 1436 IN A 204.74.67.132
ns3.facebook.com. 1436 IN A 69.63.178.21
ns4.facebook.com. 1436 IN A 69.63.186.49
ns5.facebook.com. 1436 IN A 69.63.176.200

;; Query time: 8 msec
;; SERVER: 216.228.160.6#53(216.228.160.6)
;; WHEN: Thu Sep 23 14:46:44 2010
;; MSG SIZE rcvd: 248

Plus, haven't heard any gripes from over the cube walls, all day.

Dave Owens

On Thu, Sep 23, 2010 at 2:16 PM, Josh Luthman
<josh at imaginenetworksllc.com>wrote:

> Works for me 5:16PM EST
>
> C:\Users\jluthman>dig facebook.com
>
> ; <<>> DiG 9.3.2 <<>> facebook.com
> ;; global options: printcmd
> ;; Got answer:
> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1929
> ;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0
>
> ;; QUESTION SECTION:
> ;facebook.com. IN A
>
> ;; ANSWER SECTION:
> facebook.com. 3166 IN A 69.63.189.11
> facebook.com. 3166 IN A 69.63.189.16
> facebook.com. 3166 IN A 69.63.181.12
>
> ;; Query time: 4 msec
> ;; SERVER: 192.168.21.1#53(192.168.21.1)
> ;; WHEN: Thu Sep 23 17:16:12 2010
> ;; MSG SIZE rcvd: 78
>
> Josh Luthman
> Office: 937-552-2340
> Direct: 937-552-2343
> 1100 Wayne St
> Suite 1337
> Troy, OH 45373
>
>
>
> On Thu, Sep 23, 2010 at 4:59 PM, Larry Sheldon <LarrySheldon at cox.net>
> wrote:
> > On 9/23/2010 15:53, Larry Sheldon wrote:
> >> On 9/23/2010 15:18, Chris Adams wrote:
> >>> Once upon a time, jon <relicwr at gmail.com> said:
> >>>> Its been around 30 minutes. Is this the same result from less than 20
> hours
> >>>> ago?
> >>>
> >>> Interesting, www.facebook.com switched to an Akamai IP for me at about
> >>> 19:30 UTC.
> >>
> >> I thought FB always ran throuhg Akamai.
> >>
> >> I'm now getting
> >> quote
> >> Service Unavailable - DNS failure
> >> The server is temporarily unable to service your request. Please try
> >> again later.
> >>
> >> Reference #11.8397a746.1285274927.3ef1e4f0
> >> unquote.
> >>
> >> I think that is an Akamai error, isn't it?
> >>
> >> The browser address is: http://www.facebook.com/home.php?#!/?ref=home
> >
> > Daughter in KCMO sez it (FB) was down, now up to her.
> >
> > Still down to me.
> >
> > Trace www.facebook.com goes to
> > a-70-183-191-122.deploy.akamaitechnologies.com [
> > 70.183.191.122]
> >
> >
> > --
> > Somebody should have said:
> > A democracy is two wolves and a lamb voting on what to have for dinner.
> >
> > Freedom under a constitutional republic is a well armed lamb contesting
> > the vote.
> >
> > Requiescas in pace o email
> > Ex turpi causa non oritur actio
> > Eppure si rinfresca
> >
> > ICBM Targeting Information: http://tinyurl.com/4sqczs
> > http://tinyurl.com/7tp8ml
> >
> >
> > _______________________________________________
> > Outages mailing list
> > Outages at outages.org
> > https://puck.nether.net/mailman/listinfo/outages
> >
>
> _______________________________________________
> Outages mailing list
> Outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20100923/44581ee7/attachment-0001.html>
Facebook [ In reply to ]
Details of the outage case are at
http://www.facebook.com/notes/facebook-engineering/more-details-on-todays-outage/431441338919

In short, code problem which required them "turn off" the site to resolve.
Nothing to do with Akamai/etc.

Scott.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20100923/5ede9b23/attachment.html>
Facebook [ In reply to ]
Yeah all is well. Thanks for everything.

Thread closed.
On Sep 23, 2010 11:22 PM, "L.T. Easterly" <corqpub at gmail.com> wrote:
> Up in Central FL (Brighthouse)
>
> On Thu, Sep 23, 2010 at 3:21 PM, jon <relicwr at gmail.com> wrote:
>
>> Its been around 30 minutes. Is this the same result from less than 20
hours
>> ago?
>>
>> _______________________________________________
>> Outages mailing list
>> Outages at outages.org
>> https://puck.nether.net/mailman/listinfo/outages
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20100923/af3bd5d7/attachment.html>
Re: Facebook [ In reply to ]
Can you be a bit more specific? "Works for me."

cquinn@quinntel ~ % dig facebook.com 5344 15:14:37 Mon 12-10-2012

; <<>> DiG 9.9.1-P2 <<>> facebook.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 63691
;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 2, ADDITIONAL: 3

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;facebook.com. IN A

;; ANSWER SECTION:
facebook.com. 7200 IN A 66.220.152.16
facebook.com. 7200 IN A 69.171.224.32
facebook.com. 7200 IN A 173.252.100.16
facebook.com. 7200 IN A 69.171.229.16
facebook.com. 7200 IN A 173.252.101.16
facebook.com. 7200 IN A 66.220.158.16

;; AUTHORITY SECTION:
facebook.com. 139086 IN NS a.ns.facebook.com.
facebook.com. 139086 IN NS b.ns.facebook.com.

;; ADDITIONAL SECTION:
a.ns.facebook.com. 139086 IN A 69.171.239.12
b.ns.facebook.com. 139086 IN A 69.171.255.12

;; Query time: 50 msec
;; SERVER: 10.201.1.103#53(10.201.1.103)
;; WHEN: Mon Dec 10 15:14:40 2012
;; MSG SIZE rcvd: 204


On Dec 10, 2012, at 3:12 PM, Richard Mahoney <richard.mahoney@tracesmart.co.uk> wrote:

> Seeing DNS issues for Facebook here.
> Anyone else?
>
> Kind regards
>
> Richard Mahoney, CEH
> Systems Administrator
>
> Tracesmart
> T 029 2067 8534 M 07714 486543 E richard.mahoney@tracesmart.co.uk
> www.tracesmartcorporate.co.uk www.traceiq.co.uk
>
> Global Reach Dunleavy Drive Cardiff CF11 0SN
>
> Follow us on Twitter
>
> ISO/IEC 27001 CERTIFICATE: GB 10/81945
> We are proud to sponsor missingpeople.org.uk
>
> This email and any attachments are confidential to Tracesmart Ltd and are solely for use by the intended recipient. If you are not the intended recipient you must not disclose, copy or distribute its contents to any other person nor make use of its contents in any way. If you have received this email in error please forward a copy to info@tracesmart.co.uk and remove it from your system.This email and any attachments have been scanned for the presence of computer viruses. Neither Tracesmart Ltd nor the sender accepts any responsibility for computer viruses once this email has been transmitted. The content of this message may contain personal views, which are not the views of Tracesmart Ltd, unless specifically stated. Tracesmart may monitor email traffic data and also the content of email for the purposes of security and staff training.Tracesmart Ltd is a company registered in England & Wales with company registration number 3827062 whose registered office is at Global Reach, Dunleavy Drive, Cardiff CF11 0SN. Our Data Protection Number is Z708281X and our Consumer Credit Licence Number is 565961.
>
> _______________________________________________
> Outages mailing list
> Outages@outages.org
> https://puck.nether.net/mailman/listinfo/outages
Re: Facebook [ In reply to ]
It's out in Thailand and the UK

On 11/12/2012 06:12, Richard Mahoney wrote:
>
> Seeing DNS issues for Facebook here.
>
> Anyone else?
>
> Kind regards
>
> *Richard Mahoney, CEH *
> Systems Administrator
>
> *Tracesmart*
> *T* 029 2067 8534 *M* 07714 486543 *E*richard.mahoney@tracesmart.co.uk
> <mailto:richard.mahoney@tracesmart.co.uk>
> www.tracesmartcorporate.co.uk <http://www.tracesmartcorporate.co.uk>
> www.traceiq.co.uk <http://www.traceiq.co.uk>
>
> Global Reach Dunleavy Drive Cardiff CF11 0SN
>
> Follow us on Twitter <http://www.twitter.com/tracesmart>
>
> *ISO/IEC 27001* CERTIFICATE: GB 10/81945
> We are proud to sponsor missingpeople.org.uk
> <http://www.missingpeople.org.uk>
>
> This email and any attachments are confidential to Tracesmart Ltd and
> are solely for use by the intended recipient. If you are not the
> intended recipient you must not disclose, copy or distribute its
> contents to any other person nor make use of its contents in any way.
> If you have received this email in error please forward a copy
> to info@tracesmart.co.uk and remove it from your system.This email and
> any attachments have been scanned for the presence of computer
> viruses. Neither Tracesmart Ltd nor the sender accepts any
> responsibility for computer viruses once this email has been
> transmitted. The content of this message may contain personal views,
> which are not the views of Tracesmart Ltd, unless specifically stated.
> Tracesmart may monitor email traffic data and also the content of
> email for the purposes of security and staff training.Tracesmart Ltd
> is a company registered in England & Wales with company registration
> number 3827062 whose registered office is at Global Reach, Dunleavy
> Drive, Cardiff CF11 0SN. Our Data Protection Number is Z708281X and
> our Consumer Credit Licence Number is 565961.
>
>
>
> _______________________________________________
> Outages mailing list
> Outages@outages.org
> https://puck.nether.net/mailman/listinfo/outages
Re: Facebook [ In reply to ]
On Mon, 10 Dec 2012, Richard Mahoney wrote:

> Seeing DNS issues for Facebook here.

haven't tried to nail it down (been nailing other coffins) but we
definitely saw facebook related issues over the past hour. dev teams were
crying about facebook timeouts and delays as well.



--
david raistrick http://www.netmeister.org/news/learn2quote.html
drais@icantclick.org ascii ribbon campaign - stop html mail
http://www.asciiribbon.org/



_______________________________________________
Outages mailing list
Outages@outages.org
https://puck.nether.net/mailman/listinfo/outages
Re: Facebook [ In reply to ]
I’m not seeing any issues with Facebook here.

Regards,
Steve

From: outages-bounces@outages.org [mailto:outages-bounces@outages.org] On Behalf Of Richard Mahoney
Sent: Monday, December 10, 2012 3:12 PM
To: outages@outages.org
Subject: [outages] Facebook

Seeing DNS issues for Facebook here.
Anyone else?

Kind regards

Richard Mahoney, CEH
Systems Administrator
Tracesmart
T 029 2067 8534 M 07714 486543 E richard.mahoney@tracesmart.co.uk<mailto:richard.mahoney@tracesmart.co.uk>
www.tracesmartcorporate.co.uk<http://www.tracesmartcorporate.co.uk> www.traceiq.co.uk<http://www.traceiq.co.uk>
Global Reach Dunleavy Drive Cardiff CF11 0SN
Follow us on Twitter<http://www.twitter.com/tracesmart>
ISO/IEC 27001 CERTIFICATE: GB 10/81945
We are proud to sponsor missingpeople.org.uk<http://www.missingpeople.org.uk>
This email and any attachments are confidential to Tracesmart Ltd and are solely for use by the intended recipient. If you are not the intended recipient you must not disclose, copy or distribute its contents to any other person nor make use of its contents in any way. If you have received this email in error please forward a copy to info@tracesmart.co.uk<mailto:info@tracesmart.co.uk> and remove it from your system.This email and any attachments have been scanned for the presence of computer viruses. Neither Tracesmart Ltd nor the sender accepts any responsibility for computer viruses once this email has been transmitted. The content of this message may contain personal views, which are not the views of Tracesmart Ltd, unless specifically stated. Tracesmart may monitor email traffic data and also the content of email for the purposes of security and staff training.Tracesmart Ltd is a company registered in England & Wales with company registration number 3827062 whose registered office is at Global Reach, Dunleavy Drive, Cardiff CF11 0SN. Our Data Protection Number is Z708281X and our Consumer Credit Licence Number is 565961.


________________________________
This message contains confidential information and is intended only for the individual named. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. Any opinions presented in this email are solely those of the author and do not necessarily represent those of the company. E-mail transmission cannot be guaranteed to be secure or error–free; the sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e–mail transmission.
Re: Facebook [ In reply to ]
Facebook.com is not resolving from quite a few DNS servers. On Verizon FiOS here, and it is failing. Google's DNS seems to resolve it fine, and my Verizon wireless phone seems to have no trouble accessing it via the app or the mobile web page. There are quite a number of folks on Twitter noting that Facebook is unreachable for them as well.


On Dec 10, 2012, at 6:15 PM, Corey Quinn <corey@sequestered.net> wrote:

> Can you be a bit more specific? "Works for me."
>
> cquinn@quinntel ~ % dig facebook.com 5344 15:14:37 Mon 12-10-2012
>
> ; <<>> DiG 9.9.1-P2 <<>> facebook.com
> ;; global options: +cmd
> ;; Got answer:
> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 63691
> ;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 2, ADDITIONAL: 3
>
> ;; OPT PSEUDOSECTION:
> ; EDNS: version: 0, flags:; udp: 4096
> ;; QUESTION SECTION:
> ;facebook.com. IN A
>
> ;; ANSWER SECTION:
> facebook.com. 7200 IN A 66.220.152.16
> facebook.com. 7200 IN A 69.171.224.32
> facebook.com. 7200 IN A 173.252.100.16
> facebook.com. 7200 IN A 69.171.229.16
> facebook.com. 7200 IN A 173.252.101.16
> facebook.com. 7200 IN A 66.220.158.16
>
> ;; AUTHORITY SECTION:
> facebook.com. 139086 IN NS a.ns.facebook.com.
> facebook.com. 139086 IN NS b.ns.facebook.com.
>
> ;; ADDITIONAL SECTION:
> a.ns.facebook.com. 139086 IN A 69.171.239.12
> b.ns.facebook.com. 139086 IN A 69.171.255.12
>
> ;; Query time: 50 msec
> ;; SERVER: 10.201.1.103#53(10.201.1.103)
> ;; WHEN: Mon Dec 10 15:14:40 2012
> ;; MSG SIZE rcvd: 204
>
>
> On Dec 10, 2012, at 3:12 PM, Richard Mahoney <richard.mahoney@tracesmart.co.uk> wrote:
>
>> Seeing DNS issues for Facebook here.
>> Anyone else?
>>
>> Kind regards
>>
>> Richard Mahoney, CEH
>> Systems Administrator
>>
>> Tracesmart
>> T 029 2067 8534 M 07714 486543 E richard.mahoney@tracesmart.co.uk
>> www.tracesmartcorporate.co.uk www.traceiq.co.uk
>>
>> Global Reach Dunleavy Drive Cardiff CF11 0SN
>>
>> Follow us on Twitter
>>
>> ISO/IEC 27001 CERTIFICATE: GB 10/81945
>> We are proud to sponsor missingpeople.org.uk
>>
>> This email and any attachments are confidential to Tracesmart Ltd and are solely for use by the intended recipient. If you are not the intended recipient you must not disclose, copy or distribute its contents to any other person nor make use of its contents in any way. If you have received this email in error please forward a copy to info@tracesmart.co.uk and remove it from your system.This email and any attachments have been scanned for the presence of computer viruses. Neither Tracesmart Ltd nor the sender accepts any responsibility for computer viruses once this email has been transmitted. The content of this message may contain personal views, which are not the views of Tracesmart Ltd, unless specifically stated. Tracesmart may monitor email traffic data and also the content of email for the purposes of security and staff training.Tracesmart Ltd is a company registered in England & Wales with company registration number 3827062 whose registered office is at Global Reach, Dunleavy Drive, Cardiff CF11 0SN. Our Data Protection Number is Z708281X and our Consumer Credit Licence Number is 565961.
>>
>> _______________________________________________
>> Outages mailing list
>> Outages@outages.org
>> https://puck.nether.net/mailman/listinfo/outages
>
> _______________________________________________
> Outages mailing list
> Outages@outages.org
> https://puck.nether.net/mailman/listinfo/outages
Re: Facebook [ In reply to ]
See:

http://thenextweb.com/facebook/2012/12/11/facebook-encountering-dns-issues-making-it-unavailable-for-some-users/


- ferg

On Mon, Dec 10, 2012 at 3:12 PM, Richard Mahoney
<richard.mahoney@tracesmart.co.uk> wrote:

> Seeing DNS issues for Facebook here.
>
> Anyone else?
>
>
>
> Kind regards
>
> Richard Mahoney, CEH
> Systems Administrator
>
> Tracesmart
> T 029 2067 8534 M 07714 486543 E richard.mahoney@tracesmart.co.uk
> www.tracesmartcorporate.co.uk www.traceiq.co.uk
>
> Global Reach Dunleavy Drive Cardiff CF11 0SN
>
> Follow us on Twitter
>
> ISO/IEC 27001 CERTIFICATE: GB 10/81945
> We are proud to sponsor missingpeople.org.uk
>
> This email and any attachments are confidential to Tracesmart Ltd and are
> solely for use by the intended recipient. If you are not the intended
> recipient you must not disclose, copy or distribute its contents to any
> other person nor make use of its contents in any way. If you have received
> this email in error please forward a copy to info@tracesmart.co.uk and
> remove it from your system.This email and any attachments have been scanned
> for the presence of computer viruses. Neither Tracesmart Ltd nor the sender
> accepts any responsibility for computer viruses once this email has been
> transmitted. The content of this message may contain personal views, which
> are not the views of Tracesmart Ltd, unless specifically stated. Tracesmart
> may monitor email traffic data and also the content of email for the
> purposes of security and staff training.Tracesmart Ltd is a company
> registered in England & Wales with company registration number 3827062 whose
> registered office is at Global Reach, Dunleavy Drive, Cardiff CF11 0SN. Our
> Data Protection Number is Z708281X and our Consumer Credit Licence Number is
> 565961.
>
>
>
>
> _______________________________________________
> Outages mailing list
> Outages@outages.org
> https://puck.nether.net/mailman/listinfo/outages
>



--
"Fergie", a.k.a. Paul Ferguson
fergdawgster(at)gmail.com
_______________________________________________
Outages mailing list
Outages@outages.org
https://puck.nether.net/mailman/listinfo/outages
Re: Facebook [ In reply to ]
Seems to be resolving again now on Virgin Media (UK). Guess it was just a hiccup.

PS C:\Windows\system32> nslookup www.facebook.com
Server: (removed)
Address: (removed)

Non-authoritative answer:
Name: star.facebook.com
Addresses: 2a03:2880:2110:9f02:face:b00c:0:4
69.171.247.20
Aliases: www.facebook.com

Kind regards

Richard Mahoney, CEH
Systems Administrator
Tracesmart
T 029 2067 8534 M 07714 486543 E richard.mahoney@tracesmart.co.uk<mailto:richard.mahoney@tracesmart.co.uk>
www.tracesmartcorporate.co.uk<http://www.tracesmartcorporate.co.uk> www.traceiq.co.uk<http://www.traceiq.co.uk>
Global Reach Dunleavy Drive Cardiff CF11 0SN
Follow us on Twitter<http://www.twitter.com/tracesmart>
ISO/IEC 27001 CERTIFICATE: GB 10/81945
We are proud to sponsor missingpeople.org.uk<http://www.missingpeople.org.uk>
This email and any attachments are confidential to Tracesmart Ltd and are solely for use by the intended recipient. If you are not the intended recipient you must not disclose, copy or distribute its contents to any other person nor make use of its contents in any way. If you have received this email in error please forward a copy to info@tracesmart.co.uk and remove it from your system.This email and any attachments have been scanned for the presence of computer viruses. Neither Tracesmart Ltd nor the sender accepts any responsibility for computer viruses once this email has been transmitted. The content of this message may contain personal views, which are not the views of Tracesmart Ltd, unless specifically stated. Tracesmart may monitor email traffic data and also the content of email for the purposes of security and staff training.Tracesmart Ltd is a company registered in England & Wales with company registration number 3827062 whose registered office is at Global Reach, Dunleavy Drive, Cardiff CF11 0SN. Our Data Protection Number is Z708281X and our Consumer Credit Licence Number is 565961.

From: outages-bounces@outages.org [mailto:outages-bounces@outages.org] On Behalf Of Corey Quinn
Sent: 10 December 2012 23:15
To: outages@outages.org
Subject: Re: [outages] Facebook

Can you be a bit more specific? "Works for me."

cquinn@quinntel ~ % dig facebook.com<http://facebook.com> 5344 15:14:37 Mon 12-10-2012

; <<>> DiG 9.9.1-P2 <<>> facebook.com<http://facebook.com>
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 63691
;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 2, ADDITIONAL: 3

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;facebook.com<http://facebook.com>. IN A

;; ANSWER SECTION:
facebook.com<http://facebook.com>. 7200 IN A 66.220.152.16
facebook.com<http://facebook.com>. 7200 IN A 69.171.224.32
facebook.com<http://facebook.com>. 7200 IN A 173.252.100.16
facebook.com<http://facebook.com>. 7200 IN A 69.171.229.16
facebook.com<http://facebook.com>. 7200 IN A 173.252.101.16
facebook.com<http://facebook.com>. 7200 IN A 66.220.158.16

;; AUTHORITY SECTION:
facebook.com<http://facebook.com>. 139086 IN NS a.ns.facebook.com<http://a.ns.facebook.com>.
facebook.com<http://facebook.com>. 139086 IN NS b.ns.facebook.com<http://b.ns.facebook.com>.

;; ADDITIONAL SECTION:
a.ns.facebook.com<http://a.ns.facebook.com>. 139086 IN A 69.171.239.12
b.ns.facebook.com<http://b.ns.facebook.com>. 139086 IN A 69.171.255.12

;; Query time: 50 msec
;; SERVER: 10.201.1.103#53(10.201.1.103)
;; WHEN: Mon Dec 10 15:14:40 2012
;; MSG SIZE rcvd: 204


On Dec 10, 2012, at 3:12 PM, Richard Mahoney <richard.mahoney@tracesmart.co.uk<mailto:richard.mahoney@tracesmart.co.uk>> wrote:


Seeing DNS issues for Facebook here.
Anyone else?

Kind regards

Richard Mahoney, CEH
Systems Administrator
Tracesmart
T 029 2067 8534 M 07714 486543 E richard.mahoney@tracesmart.co.uk<mailto:richard.mahoney@tracesmart.co.uk>
www.tracesmartcorporate.co.uk<http://www.tracesmartcorporate.co.uk> www.traceiq.co.uk<http://www.traceiq.co.uk>
Global Reach Dunleavy Drive Cardiff CF11 0SN
Follow us on Twitter<http://www.twitter.com/tracesmart>
ISO/IEC 27001 CERTIFICATE: GB 10/81945
We are proud to sponsor missingpeople.org.uk<http://www.missingpeople.org.uk>
This email and any attachments are confidential to Tracesmart Ltd and are solely for use by the intended recipient. If you are not the intended recipient you must not disclose, copy or distribute its contents to any other person nor make use of its contents in any way. If you have received this email in error please forward a copy to info@tracesmart.co.uk<mailto:info@tracesmart.co.uk> and remove it from your system.This email and any attachments have been scanned for the presence of computer viruses. Neither Tracesmart Ltd nor the sender accepts any responsibility for computer viruses once this email has been transmitted. The content of this message may contain personal views, which are not the views of Tracesmart Ltd, unless specifically stated. Tracesmart may monitor email traffic data and also the content of email for the purposes of security and staff training.Tracesmart Ltd is a company registered in England & Wales with company registration number 3827062 whose registered office is at Global Reach, Dunleavy Drive, Cardiff CF11 0SN. Our Data Protection Number is Z708281X and our Consumer Credit Licence Number is 565961.

_______________________________________________
Outages mailing list
Outages@outages.org<mailto:Outages@outages.org>
https://puck.nether.net/mailman/listinfo/outages
Re: Facebook [ In reply to ]
Interesting it was working fine but now it’s not.
This seems to coincide with this:

http://www.washingtonpost.com/blogs/worldviews/wp/2012/12/10/facebook-shuts-down-taliban-recruiting-account/

Lots of reports on Yahoo Answers as well

http://answers.yahoo.com/question/index?qid=20121210150803AAJpcFF

Regards,
Steve

From: outages-bounces@outages.org [mailto:outages-bounces@outages.org] On Behalf Of Richard Mahoney
Sent: Monday, December 10, 2012 3:12 PM
To: outages@outages.org
Subject: [outages] Facebook

Seeing DNS issues for Facebook here.
Anyone else?

Kind regards

Richard Mahoney, CEH
Systems Administrator
Tracesmart
T 029 2067 8534 M 07714 486543 E richard.mahoney@tracesmart.co.uk<mailto:richard.mahoney@tracesmart.co.uk>
www.tracesmartcorporate.co.uk<http://www.tracesmartcorporate.co.uk> www.traceiq.co.uk<http://www.traceiq.co.uk>
Global Reach Dunleavy Drive Cardiff CF11 0SN
Follow us on Twitter<http://www.twitter.com/tracesmart>
ISO/IEC 27001 CERTIFICATE: GB 10/81945
We are proud to sponsor missingpeople.org.uk<http://www.missingpeople.org.uk>
This email and any attachments are confidential to Tracesmart Ltd and are solely for use by the intended recipient. If you are not the intended recipient you must not disclose, copy or distribute its contents to any other person nor make use of its contents in any way. If you have received this email in error please forward a copy to info@tracesmart.co.uk<mailto:info@tracesmart.co.uk> and remove it from your system.This email and any attachments have been scanned for the presence of computer viruses. Neither Tracesmart Ltd nor the sender accepts any responsibility for computer viruses once this email has been transmitted. The content of this message may contain personal views, which are not the views of Tracesmart Ltd, unless specifically stated. Tracesmart may monitor email traffic data and also the content of email for the purposes of security and staff training.Tracesmart Ltd is a company registered in England & Wales with company registration number 3827062 whose registered office is at Global Reach, Dunleavy Drive, Cardiff CF11 0SN. Our Data Protection Number is Z708281X and our Consumer Credit Licence Number is 565961.


________________________________
This message contains confidential information and is intended only for the individual named. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. Any opinions presented in this email are solely those of the author and do not necessarily represent those of the company. E-mail transmission cannot be guaranteed to be secure or error–free; the sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e–mail transmission.
Re: Facebook [ In reply to ]
Still broke here. Silly CNAMEs.

~ > nslookup
> server a.ns.facebook.com
Default server: a.ns.facebook.com
Address: 69.171.239.12#53

> www.facebook.com
Server:         a.ns.facebook.com
Address:        69.171.239.12#53
www.facebook.com        canonical name = star.facebook.com.


> star.facebook.com
Server:         a.ns.facebook.com
Address:        69.171.239.12#53

Non-authoritative answer:
*** Can't find star.facebook.com: No answer


________________________________
From: Richard Mahoney <richard.mahoney@tracesmart.co.uk>
To: Corey Quinn <corey@sequestered.net>; "outages@outages.org" <outages@outages.org>
Sent: Monday, December 10, 2012 6:21 PM
Subject: Re: [outages] Facebook



Seems to be resolving again now on Virgin Media (UK). Guess it was just a hiccup.
 
PS C:\Windows\system32> nslookup www.facebook.com
Server:  (removed)
Address:  (removed)
 
Non-authoritative answer:
Name:    star.facebook.com
Addresses:  2a03:2880:2110:9f02:face:b00c:0:4
          69.171.247.20
Aliases:  www.facebook.com
 
Kind regards
 
Richard Mahoney, CEH 
Systems Administrator
Tracesmart
T 029 2067 8534    M 07714 486543    E richard.mahoney@tracesmart.co.uk
www.tracesmartcorporate.co.uk    www.traceiq.co.uk
Global Reach  Dunleavy Drive  Cardiff  CF11 0SN
Follow us on Twitter
ISO/IEC 27001 CERTIFICATE: GB 10/81945
We are proud to sponsor missingpeople.org.uk
This email and any attachments are confidential to Tracesmart Ltd and are solely for use by the intended recipient. If you are not the intended recipient you must not disclose, copy or distribute its contents to any other person nor make use of its contents in any way. If you have received this email in error please forward a copy to info@tracesmart.co.uk and remove it from your system.This email and any attachments have been scanned for the presence of computer viruses. Neither Tracesmart Ltd nor the sender accepts any responsibility for computer viruses once this email has been transmitted. The content of this message may contain personal views, which are not the views of Tracesmart Ltd, unless specifically stated. Tracesmart may monitor email traffic data and also the content of email for the purposes of security and staff training.Tracesmart Ltd is a company registered in England & Wales with company registration number 3827062 whose registered
office is at Global Reach, Dunleavy Drive, Cardiff CF11 0SN.  Our Data Protection Number is Z708281X and our Consumer Credit Licence Number is 565961.
 
From:outages-bounces@outages.org [mailto:outages-bounces@outages.org] On Behalf Of Corey Quinn
Sent: 10 December 2012 23:15
To: outages@outages.org
Subject: Re: [outages] Facebook
 
Can you be a bit more specific?  "Works for me."
 
cquinn@quinntel ~ % dig facebook.com               5344 15:14:37 Mon 12-10-2012
 
; <<>> DiG 9.9.1-P2 <<>> facebook.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 63691
;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 2, ADDITIONAL: 3
 
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;facebook.com.                                    IN        A
 
;; ANSWER SECTION:
facebook.com.             7200    IN        A         66.220.152.16
facebook.com.             7200    IN        A         69.171.224.32
facebook.com.             7200    IN        A         173.252.100.16
facebook.com.             7200    IN        A         69.171.229.16
facebook.com.             7200    IN        A         173.252.101.16
facebook.com.             7200    IN        A         66.220.158.16
 
;; AUTHORITY SECTION:
facebook.com.             139086            IN        NS       a.ns.facebook.com.
facebook.com.             139086            IN        NS       b.ns.facebook.com.
 
;; ADDITIONAL SECTION:
a.ns.facebook.com.      139086            IN        A         69.171.239.12
b.ns.facebook.com.      139086            IN        A         69.171.255.12
 
;; Query time: 50 msec
;; SERVER: 10.201.1.103#53(10.201.1.103)
;; WHEN: Mon Dec 10 15:14:40 2012
;; MSG SIZE  rcvd: 204
 
 
On Dec 10, 2012, at 3:12 PM, Richard Mahoney <richard.mahoney@tracesmart.co.uk> wrote:


Seeing DNS issues for Facebook here.
Anyone else?
 
Kind regards
 
Richard Mahoney, CEH 
Systems Administrator
Tracesmart
T 029 2067 8534    M 07714 486543    E richard.mahoney@tracesmart.co.uk
www.tracesmartcorporate.co.uk    www.traceiq.co.uk
Global Reach  Dunleavy Drive  Cardiff  CF11 0SN
Follow us on Twitter
ISO/IEC 27001 CERTIFICATE: GB 10/81945
We are proud to sponsor missingpeople.org.uk
This email and any attachments are confidential to Tracesmart Ltd and are solely for use by the intended recipient. If you are not the intended recipient you must not disclose, copy or distribute its contents to any other person nor make use of its contents in any way. If you have received this email in error please forward a copy to info@tracesmart.co.uk and remove it from your system.This email and any attachments have been scanned for the presence of computer viruses. Neither Tracesmart Ltd nor the sender accepts any responsibility for computer viruses once this email has been transmitted. The content of this message may contain personal views, which are not the views of Tracesmart Ltd, unless specifically stated. Tracesmart may monitor email traffic data and also the content of email for the purposes of security and staff training.Tracesmart Ltd is a company registered in England & Wales with company registration number 3827062 whose registered
office is at Global Reach, Dunleavy Drive, Cardiff CF11 0SN.  Our Data Protection Number is Z708281X and our Consumer Credit Licence Number is 565961.
 
_______________________________________________
Outages mailing list
Outages@outages.org
https://puck.nether.net/mailman/listinfo/outages
 
_______________________________________________
Outages mailing list
Outages@outages.org
https://puck.nether.net/mailman/listinfo/outages
Re: Facebook [ In reply to ]
http://techcrunch.com/2012/12/10/gmail-experiences-a-widespread-outage-most-users-affected/

Lonnie Bozeman
Operations Manager

CORESITE
(NYSE: COR)
900 N. Alameda St. Suite 200
Los Angeles, CA 90012 USA
+1 213.327.1204 | Office
+1 213.598.1739 | Cell
+1 213.327.1289 | eFax

Lonnie.Bozeman@CoreSite.com<mailto:Lonnie.Bozeman@CoreSite.com>| www.CoreSite.com<https://webmail.coresite.com/exchweb/bin/redir.asp?URL=http://www.CoreSite.com>

From: outages-bounces@outages.org [mailto:outages-bounces@outages.org] On Behalf Of Thomas Mackintosh Jr
Sent: Monday, December 10, 2012 3:19 PM
To: outages@outages.org
Subject: Re: [outages] Facebook

Facebook.com<http://Facebook.com> is not resolving from quite a few DNS servers. On Verizon FiOS here, and it is failing. Google's DNS seems to resolve it fine, and my Verizon wireless phone seems to have no trouble accessing it via the app or the mobile web page. There are quite a number of folks on Twitter noting that Facebook is unreachable for them as well.


On Dec 10, 2012, at 6:15 PM, Corey Quinn <corey@sequestered.net<mailto:corey@sequestered.net>> wrote:


Can you be a bit more specific? "Works for me."

cquinn@quinntel ~ % dig facebook.com<http://facebook.com/> 5344 15:14:37 Mon 12-10-2012

; <<>> DiG 9.9.1-P2 <<>> facebook.com<http://facebook.com/>
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 63691
;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 2, ADDITIONAL: 3

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;facebook.com<http://facebook.com/>. IN A

;; ANSWER SECTION:
facebook.com<http://facebook.com/>. 7200 IN A 66.220.152.16
facebook.com<http://facebook.com/>. 7200 IN A 69.171.224.32
facebook.com<http://facebook.com/>. 7200 IN A 173.252.100.16
facebook.com<http://facebook.com/>. 7200 IN A 69.171.229.16
facebook.com<http://facebook.com/>. 7200 IN A 173.252.101.16
facebook.com<http://facebook.com/>. 7200 IN A 66.220.158.16

;; AUTHORITY SECTION:
facebook.com<http://facebook.com/>. 139086 IN NS a.ns.facebook.com<http://a.ns.facebook.com/>.
facebook.com<http://facebook.com/>. 139086 IN NS b.ns.facebook.com<http://b.ns.facebook.com/>.

;; ADDITIONAL SECTION:
a.ns.facebook.com<http://a.ns.facebook.com/>. 139086 IN A 69.171.239.12
b.ns.facebook.com<http://b.ns.facebook.com/>. 139086 IN A 69.171.255.12

;; Query time: 50 msec
;; SERVER: 10.201.1.103#53(10.201.1.103)
;; WHEN: Mon Dec 10 15:14:40 2012
;; MSG SIZE rcvd: 204


On Dec 10, 2012, at 3:12 PM, Richard Mahoney <richard.mahoney@tracesmart.co.uk<mailto:richard.mahoney@tracesmart.co.uk>> wrote:


Seeing DNS issues for Facebook here.
Anyone else?

Kind regards

Richard Mahoney, CEH
Systems Administrator
Tracesmart
T 029 2067 8534 M 07714 486543 E richard.mahoney@tracesmart.co.uk<mailto:richard.mahoney@tracesmart.co.uk>
www.tracesmartcorporate.co.uk<http://www.tracesmartcorporate.co.uk/> www.traceiq.co.uk<http://www.traceiq.co.uk/>
Global Reach Dunleavy Drive Cardiff CF11 0SN
Follow us on Twitter<http://www.twitter.com/tracesmart>
ISO/IEC 27001 CERTIFICATE: GB 10/81945
We are proud to sponsor missingpeople.org.uk<http://www.missingpeople.org.uk/>
This email and any attachments are confidential to Tracesmart Ltd and are solely for use by the intended recipient. If you are not the intended recipient you must not disclose, copy or distribute its contents to any other person nor make use of its contents in any way. If you have received this email in error please forward a copy to info@tracesmart.co.uk<mailto:info@tracesmart.co.uk> and remove it from your system.This email and any attachments have been scanned for the presence of computer viruses. Neither Tracesmart Ltd nor the sender accepts any responsibility for computer viruses once this email has been transmitted. The content of this message may contain personal views, which are not the views of Tracesmart Ltd, unless specifically stated. Tracesmart may monitor email traffic data and also the content of email for the purposes of security and staff training.Tracesmart Ltd is a company registered in England & Wales with company registration number 3827062 whose registered office is at Global Reach, Dunleavy Drive, Cardiff CF11 0SN. Our Data Protection Number is Z708281X and our Consumer Credit Licence Number is 565961.

_______________________________________________
Outages mailing list
Outages@outages.org<mailto:Outages@outages.org>
https://puck.nether.net/mailman/listinfo/outages

_______________________________________________
Outages mailing list
Outages@outages.org<mailto:Outages@outages.org>
https://puck.nether.net/mailman/listinfo/outages
Re: Facebook [ In reply to ]
On 11/12/12 10:18 AM, "Thomas Mackintosh Jr" <twmjr@twmjr.com<mailto:twmjr@twmjr.com>> wrote:

Facebook.com<http://Facebook.com> is not resolving from quite a few DNS servers. On Verizon FiOS here, and it is failing. Google's DNS seems to resolve it fine, and my Verizon wireless phone seems to have no trouble accessing it via the app or the mobile web page. There are quite a number of folks on Twitter noting that Facebook is unreachable for them as well.

http://developers.facebook.com/live_status also shows a large sudden and atypical drop-off ("Improvement") in Average API response time commencing approx 2:30pm (TZ unknown). Graph attached for anyone not able to resolve developers.facebook.com – speculated load reduction arising from traffic loss?
--
Aaron Wigley
Re: Facebook [ In reply to ]
The Google outage this morning and the Facebook outage this afternoon are
unrelated.

Damian

On Mon, Dec 10, 2012 at 3:24 PM, Lonnie Bozeman <Lonnie.Bozeman@coresite.com
> wrote:

>
> http://techcrunch.com/2012/12/10/gmail-experiences-a-widespread-outage-most-users-affected/
> ****
>
> ** **
>
> *Lonnie Bozeman*****
>
> *Operations Manager*****
>
> ****
>
> *C**ORE**S**ITE** *
>
> (NYSE: COR)****
>
> 900 N. Alameda St. Suite 200****
>
> Los Angeles, CA 90012 USA****
>
> +1 213.327.1204 | Office****
>
> +1 213.598.1739 | Cell****
>
> +1 213.327.1289 | eFax ****
>
> ****
>
> *Lonnie.Bozeman@CoreSite.com*| www.CoreSite.com<https://webmail.coresite.com/exchweb/bin/redir.asp?URL=http://www.CoreSite.com>
> ****
>
> ** **
>
> *From:* outages-bounces@outages.org [mailto:outages-bounces@outages.org] *On
> Behalf Of *Thomas Mackintosh Jr
> *Sent:* Monday, December 10, 2012 3:19 PM
>
> *To:* outages@outages.org
> *Subject:* Re: [outages] Facebook****
>
> ** **
>
> Facebook.com is not resolving from quite a few DNS servers. On Verizon
> FiOS here, and it is failing. Google's DNS seems to resolve it fine, and my
> Verizon wireless phone seems to have no trouble accessing it via the app or
> the mobile web page. There are quite a number of folks on Twitter noting
> that Facebook is unreachable for them as well.****
>
> ** **
>
> ** **
>
> On Dec 10, 2012, at 6:15 PM, Corey Quinn <corey@sequestered.net> wrote:***
> *
>
>
>
> ****
>
> Can you be a bit more specific? "Works for me."****
>
> ** **
>
> cquinn@quinntel ~ % dig facebook.com 5344 15:14:37 Mon
> 12-10-2012****
>
> ** **
>
> ; <<>> DiG 9.9.1-P2 <<>> facebook.com****
>
> ;; global options: +cmd****
>
> ;; Got answer:****
>
> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 63691****
>
> ;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 2, ADDITIONAL: 3****
>
> ** **
>
> ;; OPT PSEUDOSECTION:****
>
> ; EDNS: version: 0, flags:; udp: 4096****
>
> ;; QUESTION SECTION:****
>
> ;facebook.com. IN A****
>
> ** **
>
> ;; ANSWER SECTION:****
>
> facebook.com. 7200 IN A 66.220.152.16****
>
> facebook.com. 7200 IN A 69.171.224.32****
>
> facebook.com. 7200 IN A 173.252.100.16****
>
> facebook.com. 7200 IN A 69.171.229.16****
>
> facebook.com. 7200 IN A 173.252.101.16****
>
> facebook.com. 7200 IN A 66.220.158.16****
>
> ** **
>
> ;; AUTHORITY SECTION:****
>
> facebook.com. 139086 IN NS
> a.ns.facebook.com.****
>
> facebook.com. 139086 IN NS
> b.ns.facebook.com.****
>
> ** **
>
> ;; ADDITIONAL SECTION:****
>
> a.ns.facebook.com. 139086 IN A 69.171.239.12
> ****
>
> b.ns.facebook.com. 139086 IN A 69.171.255.12
> ****
>
> ** **
>
> ;; Query time: 50 msec****
>
> ;; SERVER: 10.201.1.103#53(10.201.1.103)****
>
> ;; WHEN: Mon Dec 10 15:14:40 2012****
>
> ;; MSG SIZE rcvd: 204****
>
> ** **
>
> ** **
>
> On Dec 10, 2012, at 3:12 PM, Richard Mahoney <
> richard.mahoney@tracesmart.co.uk> wrote:****
>
>
>
> ****
>
> Seeing DNS issues for Facebook here.****
>
> Anyone else?****
>
> ****
>
> Kind regards
>
> *Richard Mahoney, CEH *
> Systems Administrator****
>
> *Tracesmart*
> *T* 029 2067 8534 *M* 07714 486543 *E*
> richard.mahoney@tracesmart.co.uk
> www.tracesmartcorporate.co.uk www.traceiq.co.uk****
>
> Global Reach Dunleavy Drive Cardiff CF11 0SN****
>
> Follow us on Twitter <http://www.twitter.com/tracesmart>****
>
> *ISO/IEC 27001* CERTIFICATE: GB 10/81945
> We are proud to sponsor missingpeople.org.uk<http://www.missingpeople.org.uk/>
> ****
>
> This email and any attachments are confidential to Tracesmart Ltd and are
> solely for use by the intended recipient. If you are not the intended
> recipient you must not disclose, copy or distribute its contents to any
> other person nor make use of its contents in any way. If you have received
> this email in error please forward a copy to info@tracesmart.co.uk and
> remove it from your system.This email and any attachments have been scanned
> for the presence of computer viruses. Neither Tracesmart Ltd nor the sender
> accepts any responsibility for computer viruses once this email has been
> transmitted. The content of this message may contain personal views, which
> are not the views of Tracesmart Ltd, unless specifically stated. Tracesmart
> may monitor email traffic data and also the content of email for the
> purposes of security and staff training.Tracesmart Ltd is a company
> registered in England & Wales with company registration number 3827062
> whose registered office is at Global Reach, Dunleavy Drive, Cardiff CF11
> 0SN. Our Data Protection Number is Z708281X and our Consumer Credit
> Licence Number is 565961.****
>
> ****
>
> _______________________________________________
> Outages mailing list
> Outages@outages.org
> https://puck.nether.net/mailman/listinfo/outages****
>
> ** **
>
> _______________________________________________
> Outages mailing list
> Outages@outages.org
> https://puck.nether.net/mailman/listinfo/outages****
>
> ** **
>
> _______________________________________________
> Outages mailing list
> Outages@outages.org
> https://puck.nether.net/mailman/listinfo/outages
>
>
Re: Facebook [ In reply to ]
I could have provided dig +trace output but this is shorter and reads
easier.

It looks like records get looked up as follows (and I'm excluding the
root server lookups, i.e. . --> .com --> facebook.com):

facebook.com. 147814 IN NS b.ns.facebook.com.
facebook.com. 147814 IN NS a.ns.facebook.com.

And the A records:

a.ns.facebook.com. 172573 IN A 69.171.239.12
b.ns.facebook.com. 172573 IN A 69.171.255.12

The SOA for facebook.com (domain itself) hasn't been changed since
2012/12/07 (if SOA serial is truly kept in lines with the YYYYMMDD
model).

69.171.239.12 when queried for any records for www.facebook.com
results in a CNAME response to star.facebook.com. It's probably named
"star" to indicate asterisk (*):

www.facebook.com. 338 IN CNAME star.facebook.com.
star.facebook.com. 1238 IN NS glb2.facebook.com.
star.facebook.com. 1238 IN NS glb1.facebook.com.

And the A records:

glb1.facebook.com. 3038 IN A 69.171.239.10
glb2.facebook.com. 3038 IN A 69.171.255.10

glb obviously stands for "global load balancer", though I have no idea
what device they use (F5s, Citrix Netscalers, Alteons (god forbid), or
something home-grown).

Given the below analysis from Terry, it looks to me like:

a) one or both of their load balancers may have been overloaded briefly
and did not respond to DNS queries (or possibly something at layer 2
or layer 3 was affecting this)
b) one or more of the nameservers *behind* glb[12].facebook.com were
overloaded or broken in some way, or layer 2/3 was responsible for
breakage (between glbs and nameservers)

The only people who know for certain are -- yup -- the Facebook folks.

And naturally this is me doing my testing from a single source, so its
possible they use anycast to distribute some of their load, in which
case the above analysis (despite speculative) is still correct, except
what actual devices/networks are involved would be different.

You're welcome. :-)

--
| Jeremy Chadwick jdc@koitsu.org |
| UNIX Systems Administrator http://jdc.koitsu.org/ |
| Mountain View, CA, US |
| Making life hard for others since 1977. PGP 4BD6C0CB |

On Mon, Dec 10, 2012 at 03:24:28PM -0800, Terry wrote:
> Still broke here. Silly CNAMEs.
>
> ~ > nslookup
> > server a.ns.facebook.com
> Default server: a.ns.facebook.com
> Address: 69.171.239.12#53
>
> > www.facebook.com
> Server: ? ? ? ? a.ns.facebook.com
> Address: ? ? ? ?69.171.239.12#53
> www.facebook.com ? ? ? ?canonical name = star.facebook.com.
>
>
> > star.facebook.com
> Server: ? ? ? ? a.ns.facebook.com
> Address: ? ? ? ?69.171.239.12#53
>
> Non-authoritative answer:
> *** Can't find star.facebook.com: No answer
>
>
> ________________________________
> From: Richard Mahoney <richard.mahoney@tracesmart.co.uk>
> To: Corey Quinn <corey@sequestered.net>; "outages@outages.org" <outages@outages.org>
> Sent: Monday, December 10, 2012 6:21 PM
> Subject: Re: [outages] Facebook
>
>
>
> Seems to be resolving again now on Virgin Media (UK). Guess it was just a hiccup.
> ?
> PS C:\Windows\system32> nslookup www.facebook.com
> Server:? (removed)
> Address:? (removed)
> ?
> Non-authoritative answer:
> Name:??? star.facebook.com
> Addresses:? 2a03:2880:2110:9f02:face:b00c:0:4
> ????????? 69.171.247.20
> Aliases:? www.facebook.com
> ?
> Kind regards
> ?
> Richard Mahoney, CEH?
> Systems Administrator
> Tracesmart
> T?029 2067 8534????M?07714 486543????E?richard.mahoney@tracesmart.co.uk
> www.tracesmartcorporate.co.uk????www.traceiq.co.uk
> Global Reach ?Dunleavy Drive ?Cardiff ?CF11 0SN
> Follow us on?Twitter
> ISO/IEC 27001?CERTIFICATE: GB 10/81945
> We are proud to sponsor?missingpeople.org.uk
> This email and any attachments are confidential to Tracesmart Ltd and are solely for use by the intended recipient. If you are not the intended recipient you must not disclose, copy or distribute its contents to any other person nor make use of its contents in any way. If you have received this email in error please forward a copy to?info@tracesmart.co.uk?and remove it from your system.This email and any attachments have been scanned for the presence of computer viruses. Neither Tracesmart Ltd nor the sender accepts any responsibility for computer viruses once this email has been transmitted. The content of this message may contain personal views, which are not the views of Tracesmart Ltd, unless specifically stated. Tracesmart may monitor email traffic data and also the content of email for the purposes of security and staff training.Tracesmart Ltd is a company registered in England & Wales with company registration number 3827062 whose registered
> office is at Global Reach, Dunleavy Drive, Cardiff CF11 0SN. ?Our Data Protection Number is Z708281X and our Consumer Credit Licence Number is 565961.
> ?
> From:outages-bounces@outages.org [mailto:outages-bounces@outages.org] On Behalf Of Corey Quinn
> Sent: 10 December 2012 23:15
> To: outages@outages.org
> Subject: Re: [outages] Facebook
> ?
> Can you be a bit more specific? ?"Works for me."
> ?
> cquinn@quinntel ~ % dig facebook.com ? ? ? ? ? ? ? 5344 15:14:37 Mon 12-10-2012
> ?
> ; <<>> DiG 9.9.1-P2 <<>> facebook.com
> ;; global options: +cmd
> ;; Got answer:
> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 63691
> ;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 2, ADDITIONAL: 3
> ?
> ;; OPT PSEUDOSECTION:
> ; EDNS: version: 0, flags:; udp: 4096
> ;; QUESTION SECTION:
> ;facebook.com.??????????????????????????????????? IN??????? A
> ?
> ;; ANSWER SECTION:
> facebook.com.???????????? 7200??? IN??????? A???????? 66.220.152.16
> facebook.com.???????????? 7200??? IN??????? A???????? 69.171.224.32
> facebook.com.???????????? 7200??? IN??????? A???????? 173.252.100.16
> facebook.com.???????????? 7200??? IN??????? A???????? 69.171.229.16
> facebook.com.???????????? 7200??? IN??????? A???????? 173.252.101.16
> facebook.com.???????????? 7200??? IN??????? A???????? 66.220.158.16
> ?
> ;; AUTHORITY SECTION:
> facebook.com.???????????? 139086??????????? IN??????? NS?????? a.ns.facebook.com.
> facebook.com.???????????? 139086??????????? IN??????? NS?????? b.ns.facebook.com.
> ?
> ;; ADDITIONAL SECTION:
> a.ns.facebook.com.????? 139086??????????? IN??????? A???????? 69.171.239.12
> b.ns.facebook.com.????? 139086??????????? IN??????? A???????? 69.171.255.12
> ?
> ;; Query time: 50 msec
> ;; SERVER: 10.201.1.103#53(10.201.1.103)
> ;; WHEN: Mon Dec 10 15:14:40 2012
> ;; MSG SIZE ?rcvd: 204
> ?
> ?
> On Dec 10, 2012, at 3:12 PM, Richard Mahoney <richard.mahoney@tracesmart.co.uk> wrote:
>
>
> Seeing DNS issues for Facebook here.
> Anyone else?
> ?
> Kind regards
> ?
> Richard Mahoney, CEH?
> Systems Administrator
> Tracesmart
> T?029 2067 8534????M?07714 486543????E?richard.mahoney@tracesmart.co.uk
> www.tracesmartcorporate.co.uk????www.traceiq.co.uk
> Global Reach ?Dunleavy Drive ?Cardiff ?CF11 0SN
> Follow us on?Twitter
> ISO/IEC 27001?CERTIFICATE: GB 10/81945
> We are proud to sponsor?missingpeople.org.uk
> This email and any attachments are confidential to Tracesmart Ltd and are solely for use by the intended recipient. If you are not the intended recipient you must not disclose, copy or distribute its contents to any other person nor make use of its contents in any way. If you have received this email in error please forward a copy to?info@tracesmart.co.uk?and remove it from your system.This email and any attachments have been scanned for the presence of computer viruses. Neither Tracesmart Ltd nor the sender accepts any responsibility for computer viruses once this email has been transmitted. The content of this message may contain personal views, which are not the views of Tracesmart Ltd, unless specifically stated. Tracesmart may monitor email traffic data and also the content of email for the purposes of security and staff training.Tracesmart Ltd is a company registered in England & Wales with company registration number 3827062 whose registered
> office is at Global Reach, Dunleavy Drive, Cardiff CF11 0SN. ?Our Data Protection Number is Z708281X and our Consumer Credit Licence Number is 565961.
> ?
> _______________________________________________
> Outages mailing list
> Outages@outages.org
> https://puck.nether.net/mailman/listinfo/outages
> ?
> _______________________________________________
> Outages mailing list
> Outages@outages.org
> https://puck.nether.net/mailman/listinfo/outages

> _______________________________________________
> Outages mailing list
> Outages@outages.org
> https://puck.nether.net/mailman/listinfo/outages

_______________________________________________
Outages mailing list
Outages@outages.org
https://puck.nether.net/mailman/listinfo/outages
Re: Facebook [ In reply to ]
Yeah, I think my result was a red herring. a.ns.facebook.com and b.ns.facebook.com still can't resolve the A record for star.facebook.com, despite things seemingly being back to normal now. The NS record is what's key and by the time I looked at it, it was fixed.

Why some people feel the need to get so clever with DNS is beyond me. How about just resolving the A records directly from the facebook.com NS servers, instead of via a CNAME to another group of DNS servers? Would that be so difficult? Then you're shocked when there's an outage.


________________________________
From: Jeremy Chadwick <jdc@koitsu.org>
To: Terry <t0psecret@yahoo.com>
Cc: Richard Mahoney <richard.mahoney@tracesmart.co.uk>; Corey Quinn <corey@sequestered.net>; "outages@outages.org" <outages@outages.org>
Sent: Monday, December 10, 2012 6:40 PM
Subject: Re: [outages] Facebook

I could have provided dig +trace output but this is shorter and reads
easier.

It looks like records get looked up as follows (and I'm excluding the
root server lookups, i.e. . --> .com --> facebook.com):

facebook.com.          147814  IN      NS      b.ns.facebook.com.
facebook.com.          147814  IN      NS      a.ns.facebook.com.

And the A records:

a.ns.facebook.com.      172573  IN      A      69.171.239.12
b.ns.facebook.com.      172573  IN      A      69.171.255.12

The SOA for facebook.com (domain itself) hasn't been changed since
2012/12/07 (if SOA serial is truly kept in lines with the YYYYMMDD
model).

69.171.239.12 when queried for any records for www.facebook.com
results in a CNAME response to star.facebook.com.  It's probably named
"star" to indicate asterisk (*):

www.facebook.com.      338    IN      CNAME  star.facebook.com.
star.facebook.com.      1238    IN      NS      glb2.facebook.com.
star.facebook.com.      1238    IN      NS      glb1.facebook.com.

And the A records:

glb1.facebook.com.      3038    IN      A      69.171.239.10
glb2.facebook.com.      3038    IN      A      69.171.255.10

glb obviously stands for "global load balancer", though I have no idea
what device they use (F5s, Citrix Netscalers, Alteons (god forbid), or
something home-grown).

Given the below analysis from Terry, it looks to me like:

a) one or both of their load balancers may have been overloaded briefly
  and did not respond to DNS queries (or possibly something at layer 2
  or layer 3 was affecting this)
b) one or more of the nameservers *behind* glb[12].facebook.com were
  overloaded or broken in some way, or layer 2/3 was responsible for
  breakage (between glbs and nameservers)

The only people who know for certain are -- yup -- the Facebook folks.

And naturally this is me doing my testing from a single source, so its
possible they use anycast to distribute some of their load, in which
case the above analysis (despite speculative) is still correct, except
what actual devices/networks are involved would be different.

You're welcome.  :-)

--
| Jeremy Chadwick                                  jdc@koitsu.org |
| UNIX Systems Administrator                http://jdc.koitsu.org/ |
| Mountain View, CA, US                                            |
| Making life hard for others since 1977.            PGP 4BD6C0CB |

On Mon, Dec 10, 2012 at 03:24:28PM -0800, Terry wrote:
> Still broke here. Silly CNAMEs.
>
> ~ > nslookup
> > server a.ns.facebook.com
> Default server: a.ns.facebook.com
> Address: 69.171.239.12#53
>
> > www.facebook.com
> Server: ? ? ? ? a.ns.facebook.com
> Address: ? ? ? ?69.171.239.12#53
> www.facebook.com ? ? ? ?canonical name = star.facebook.com.
>
>
> > star.facebook.com
> Server: ? ? ? ? a.ns.facebook.com
> Address: ? ? ? ?69.171.239.12#53
>
> Non-authoritative answer:
> *** Can't find star.facebook.com: No answer
>
>
> ________________________________
>  From: Richard Mahoney <richard.mahoney@tracesmart.co.uk>
> To: Corey Quinn <corey@sequestered.net>; "outages@outages.org" <outages@outages.org>
> Sent: Monday, December 10, 2012 6:21 PM
> Subject: Re: [outages] Facebook

>

> Seems to be resolving again now on Virgin Media (UK). Guess it was just a hiccup.
> ?
> PS C:\Windows\system32> nslookup www.facebook.com
> Server:? (removed)
> Address:? (removed)
> ?
> Non-authoritative answer:
> Name:??? star.facebook.com
> Addresses:? 2a03:2880:2110:9f02:face:b00c:0:4
> ????????? 69.171.247.20
> Aliases:? www.facebook.com
> ?
> Kind regards
> ?
> Richard Mahoney, CEH?
> Systems Administrator
> Tracesmart
> T?029 2067 8534????M?07714 486543????E?richard.mahoney@tracesmart.co.uk
> www.tracesmartcorporate.co.uk????www.traceiq.co.uk
> Global Reach ?Dunleavy Drive ?Cardiff ?CF11 0SN
> Follow us on?Twitter
> ISO/IEC 27001?CERTIFICATE: GB 10/81945
> We are proud to sponsor?missingpeople.org.uk
> This email and any attachments are confidential to Tracesmart Ltd and are solely for use by the intended recipient. If you are not the intended recipient you must not disclose, copy or distribute its contents to any other person nor make use of its contents in any way. If you have received this email in error please forward a copy to?info@tracesmart.co.uk?and remove it from your system.This email and any attachments have been scanned for the presence of computer viruses. Neither Tracesmart Ltd nor the sender accepts any responsibility for computer viruses once this email has been transmitted. The content of this message may contain personal views, which are not the views of Tracesmart Ltd, unless specifically stated. Tracesmart may monitor email traffic data and also the content of email for the purposes of security and staff training.Tracesmart Ltd is a company registered in England & Wales with company registration number 3827062 whose registered
>  office is at Global Reach, Dunleavy Drive, Cardiff CF11 0SN. ?Our Data Protection Number is Z708281X and our Consumer Credit Licence Number is 565961.
> ?
> From:outages-bounces@outages.org [mailto:outages-bounces@outages.org] On Behalf Of Corey Quinn
> Sent: 10 December 2012 23:15
> To: outages@outages.org
> Subject: Re: [outages] Facebook
> ?
> Can you be a bit more specific? ?"Works for me."
> ?
> cquinn@quinntel ~ % dig facebook.com ? ? ? ? ? ? ? 5344 15:14:37 Mon 12-10-2012
> ?
> ; <<>> DiG 9.9.1-P2 <<>> facebook.com
> ;; global options: +cmd
> ;; Got answer:
> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 63691
> ;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 2, ADDITIONAL: 3
> ?
> ;; OPT PSEUDOSECTION:
> ; EDNS: version: 0, flags:; udp: 4096
> ;; QUESTION SECTION:
> ;facebook.com.??????????????????????????????????? IN??????? A
> ?
> ;; ANSWER SECTION:
> facebook.com.???????????? 7200??? IN??????? A???????? 66.220.152.16
> facebook.com.???????????? 7200??? IN??????? A???????? 69.171.224.32
> facebook.com.???????????? 7200??? IN??????? A???????? 173.252.100.16
> facebook.com.???????????? 7200??? IN??????? A???????? 69.171.229.16
> facebook.com.???????????? 7200??? IN??????? A???????? 173.252.101.16
> facebook.com.???????????? 7200??? IN??????? A???????? 66.220.158.16
> ?
> ;; AUTHORITY SECTION:
> facebook.com.???????????? 139086??????????? IN??????? NS?????? a.ns.facebook.com.
> facebook.com.???????????? 139086??????????? IN??????? NS?????? b.ns.facebook.com.
> ?
> ;; ADDITIONAL SECTION:
> a.ns.facebook.com.????? 139086??????????? IN??????? A???????? 69.171.239.12
> b.ns.facebook.com.????? 139086??????????? IN??????? A???????? 69.171.255.12
> ?
> ;; Query time: 50 msec
> ;; SERVER: 10.201.1.103#53(10.201.1.103)
> ;; WHEN: Mon Dec 10 15:14:40 2012
> ;; MSG SIZE ?rcvd: 204
> ?
> ?
> On Dec 10, 2012, at 3:12 PM, Richard Mahoney <richard.mahoney@tracesmart.co.uk> wrote:
>
>
> Seeing DNS issues for Facebook here.
> Anyone else?
> ?
> Kind regards
> ?
> Richard Mahoney, CEH?
> Systems Administrator
> Tracesmart
> T?029 2067 8534????M?07714 486543????E?richard.mahoney@tracesmart.co.uk
> www.tracesmartcorporate.co.uk????www.traceiq.co.uk
> Global Reach ?Dunleavy Drive ?Cardiff ?CF11 0SN
> Follow us on?Twitter
> ISO/IEC 27001?CERTIFICATE: GB 10/81945
> We are proud to sponsor?missingpeople.org.uk
> This email and any attachments are confidential to Tracesmart Ltd and are solely for use by the intended recipient. If you are not the intended recipient you must not disclose, copy or distribute its contents to any other person nor make use of its contents in any way. If you have received this email in error please forward a copy to?info@tracesmart.co.uk?and remove it from your system.This email and any attachments have been scanned for the presence of computer viruses. Neither Tracesmart Ltd nor the sender accepts any responsibility for computer viruses once this email has been transmitted. The content of this message may contain personal views, which are not the views of Tracesmart Ltd, unless specifically stated. Tracesmart may monitor email traffic data and also the content of email for the purposes of security and staff training.Tracesmart Ltd is a company registered in England & Wales with company registration number 3827062 whose registered
>  office is at Global Reach, Dunleavy Drive, Cardiff CF11 0SN. ?Our Data Protection Number is Z708281X and our Consumer Credit Licence Number is 565961.
> ?
> _______________________________________________
> Outages mailing list
> Outages@outages.org
> https://puck.nether.net/mailman/listinfo/outages
> ?
> _______________________________________________
> Outages mailing list
> Outages@outages.org
> https://puck.nether.net/mailman/listinfo/outages

> _______________________________________________
> Outages mailing list
> Outages@outages.org
> https://puck.nether.net/mailman/listinfo/outages
Re: Facebook [ In reply to ]
There *is* no A record for star.facebook.com -- there's only a CNAME,
which in turn induces per-record NS lookups. So this is normal.

As for your 2nd question -- this opens a very large Pandora's box when
it comes to DNS engineering and operational deployments. I can think of
quite a few reasons there should be separate load balancers for the
"main" facebook.com domain and ns.facebook.com bits, and the actual LBs
which return A records for the webservers that return web content.
Effectively what you're proposing is just to have lots of LBs handling
the DNS queries for everything, rather than segregate/delineate things
a bit more. I can assure you there are justifications for this, but
since I don't work at Facebook, I can't provide those.

What I've seen today is not that complex of a setup, though depending on
what LBs they use, I might not particularly enjoy looking at those
configurations. But what's shown here, at least if using a Citrix
NetScaler, isn't that complex.

All that said -- it is absolutely possible for them to remove use of the
star.facebook.com CNAME and just have a series of LBs respond with
appropriate A records when faced with a www.facebook.com A record query.

However, given that there's quite a lot of crapola-nonsense that falls
under the facebook.com domain (I'm talking about the equivalent of a
content delivery network, their image hosting stuff, and God knows what
else today -- I haven't used Facebook since January 2011), due to "Web
2.0" nutballs always wanting to make a mess of things ( ;-) ), I'm not
too surprised by the above.

--
| Jeremy Chadwick jdc@koitsu.org |
| UNIX Systems Administrator http://jdc.koitsu.org/ |
| Mountain View, CA, US |
| Making life hard for others since 1977. PGP 4BD6C0CB |

On Mon, Dec 10, 2012 at 03:58:31PM -0800, Terry wrote:
> Yeah, I think my result was a red herring. a.ns.facebook.com and b.ns.facebook.com still can't resolve the A record for star.facebook.com, despite things seemingly being back to normal now. The NS record is what's key and by the time I looked at it, it was fixed.
>
> Why some people feel the need to get so clever with DNS is beyond me. How about just resolving the A records directly from the facebook.com NS servers, instead of via a CNAME to another group of DNS servers? Would that be so difficult? Then you're shocked when there's an outage.
>
>
> ________________________________
> From: Jeremy Chadwick <jdc@koitsu.org>
> To: Terry <t0psecret@yahoo.com>
> Cc: Richard Mahoney <richard.mahoney@tracesmart.co.uk>; Corey Quinn <corey@sequestered.net>; "outages@outages.org" <outages@outages.org>
> Sent: Monday, December 10, 2012 6:40 PM
> Subject: Re: [outages] Facebook
>
> I could have provided dig +trace output but this is shorter and reads
> easier.
>
> It looks like records get looked up as follows (and I'm excluding the
> root server lookups, i.e. . --> .com --> facebook.com):
>
> facebook.com.? ? ? ? ? 147814? IN? ? ? NS? ? ? b.ns.facebook.com.
> facebook.com.? ? ? ? ? 147814? IN? ? ? NS? ? ? a.ns.facebook.com.
>
> And the A records:
>
> a.ns.facebook.com.? ? ? 172573? IN? ? ? A? ? ? 69.171.239.12
> b.ns.facebook.com.? ? ? 172573? IN? ? ? A? ? ? 69.171.255.12
>
> The SOA for facebook.com (domain itself) hasn't been changed since
> 2012/12/07 (if SOA serial is truly kept in lines with the YYYYMMDD
> model).
>
> 69.171.239.12 when queried for any records for www.facebook.com
> results in a CNAME response to star.facebook.com.? It's probably named
> "star" to indicate asterisk (*):
>
> www.facebook.com.? ? ? 338? ? IN? ? ? CNAME? star.facebook.com.
> star.facebook.com.? ? ? 1238? ? IN? ? ? NS? ? ? glb2.facebook.com.
> star.facebook.com.? ? ? 1238? ? IN? ? ? NS? ? ? glb1.facebook.com.
>
> And the A records:
>
> glb1.facebook.com.? ? ? 3038? ? IN? ? ? A? ? ? 69.171.239.10
> glb2.facebook.com.? ? ? 3038? ? IN? ? ? A? ? ? 69.171.255.10
>
> glb obviously stands for "global load balancer", though I have no idea
> what device they use (F5s, Citrix Netscalers, Alteons (god forbid), or
> something home-grown).
>
> Given the below analysis from Terry, it looks to me like:
>
> a) one or both of their load balancers may have been overloaded briefly
> ? and did not respond to DNS queries (or possibly something at layer 2
> ? or layer 3 was affecting this)
> b) one or more of the nameservers *behind* glb[12].facebook.com were
> ? overloaded or broken in some way, or layer 2/3 was responsible for
> ? breakage (between glbs and nameservers)
>
> The only people who know for certain are -- yup -- the Facebook folks.
>
> And naturally this is me doing my testing from a single source, so its
> possible they use anycast to distribute some of their load, in which
> case the above analysis (despite speculative) is still correct, except
> what actual devices/networks are involved would be different.
>
> You're welcome.? :-)
>
> --
> | Jeremy Chadwick? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? jdc@koitsu.org |
> | UNIX Systems Administrator? ? ? ? ? ? ? ? http://jdc.koitsu.org/ |
> | Mountain View, CA, US? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? |
> | Making life hard for others since 1977.? ? ? ? ? ? PGP 4BD6C0CB |
>
> On Mon, Dec 10, 2012 at 03:24:28PM -0800, Terry wrote:
> > Still broke here. Silly CNAMEs.
> >
> > ~ > nslookup
> > > server a.ns.facebook.com
> > Default server: a.ns.facebook.com
> > Address: 69.171.239.12#53
> >
> > > www.facebook.com
> > Server: ? ? ? ? a.ns.facebook.com
> > Address: ? ? ? ?69.171.239.12#53
> > www.facebook.com ? ? ? ?canonical name = star.facebook.com.
> >
> >
> > > star.facebook.com
> > Server: ? ? ? ? a.ns.facebook.com
> > Address: ? ? ? ?69.171.239.12#53
> >
> > Non-authoritative answer:
> > *** Can't find star.facebook.com: No answer
> >
> >
> > ________________________________
> >? From: Richard Mahoney <richard.mahoney@tracesmart.co.uk>
> > To: Corey Quinn <corey@sequestered.net>; "outages@outages.org" <outages@outages.org>
> > Sent: Monday, December 10, 2012 6:21 PM
> > Subject: Re: [outages] Facebook
> >?
> >
> >?
> > Seems to be resolving again now on Virgin Media (UK). Guess it was just a hiccup.
> > ?
> > PS C:\Windows\system32> nslookup www.facebook.com
> > Server:? (removed)
> > Address:? (removed)
> > ?
> > Non-authoritative answer:
> > Name:??? star.facebook.com
> > Addresses:? 2a03:2880:2110:9f02:face:b00c:0:4
> > ????????? 69.171.247.20
> > Aliases:? www.facebook.com
> > ?
> > Kind regards
> > ?
> > Richard Mahoney, CEH?
> > Systems Administrator
> > Tracesmart
> > T?029 2067 8534????M?07714 486543????E?richard.mahoney@tracesmart.co.uk
> > www.tracesmartcorporate.co.uk????www.traceiq.co.uk
> > Global Reach ?Dunleavy Drive ?Cardiff ?CF11 0SN
> > Follow us on?Twitter
> > ISO/IEC 27001?CERTIFICATE: GB 10/81945
> > We are proud to sponsor?missingpeople.org.uk
> > This email and any attachments are confidential to Tracesmart Ltd and are solely for use by the intended recipient. If you are not the intended recipient you must not disclose, copy or distribute its contents to any other person nor make use of its contents in any way. If you have received this email in error please forward a copy to?info@tracesmart.co.uk?and remove it from your system.This email and any attachments have been scanned for the presence of computer viruses. Neither Tracesmart Ltd nor the sender accepts any responsibility for computer viruses once this email has been transmitted. The content of this message may contain personal views, which are not the views of Tracesmart Ltd, unless specifically stated. Tracesmart may monitor email traffic data and also the content of email for the purposes of security and staff training.Tracesmart Ltd is a company registered in England & Wales with company registration number 3827062 whose registered
> >? office is at Global Reach, Dunleavy Drive, Cardiff CF11 0SN. ?Our Data Protection Number is Z708281X and our Consumer Credit Licence Number is 565961.
> > ?
> > From:outages-bounces@outages.org [mailto:outages-bounces@outages.org] On Behalf Of Corey Quinn
> > Sent: 10 December 2012 23:15
> > To: outages@outages.org
> > Subject: Re: [outages] Facebook
> > ?
> > Can you be a bit more specific? ?"Works for me."
> > ?
> > cquinn@quinntel ~ % dig facebook.com ? ? ? ? ? ? ? 5344 15:14:37 Mon 12-10-2012
> > ?
> > ; <<>> DiG 9.9.1-P2 <<>> facebook.com
> > ;; global options: +cmd
> > ;; Got answer:
> > ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 63691
> > ;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 2, ADDITIONAL: 3
> > ?
> > ;; OPT PSEUDOSECTION:
> > ; EDNS: version: 0, flags:; udp: 4096
> > ;; QUESTION SECTION:
> > ;facebook.com.??????????????????????????????????? IN??????? A
> > ?
> > ;; ANSWER SECTION:
> > facebook.com.???????????? 7200??? IN??????? A???????? 66.220.152.16
> > facebook.com.???????????? 7200??? IN??????? A???????? 69.171.224.32
> > facebook.com.???????????? 7200??? IN??????? A???????? 173.252.100.16
> > facebook.com.???????????? 7200??? IN??????? A???????? 69.171.229.16
> > facebook.com.???????????? 7200??? IN??????? A???????? 173.252.101.16
> > facebook.com.???????????? 7200??? IN??????? A???????? 66.220.158.16
> > ?
> > ;; AUTHORITY SECTION:
> > facebook.com.???????????? 139086??????????? IN??????? NS?????? a.ns.facebook.com.
> > facebook.com.???????????? 139086??????????? IN??????? NS?????? b.ns.facebook.com.
> > ?
> > ;; ADDITIONAL SECTION:
> > a.ns.facebook.com.????? 139086??????????? IN??????? A???????? 69.171.239.12
> > b.ns.facebook.com.????? 139086??????????? IN??????? A???????? 69.171.255.12
> > ?
> > ;; Query time: 50 msec
> > ;; SERVER: 10.201.1.103#53(10.201.1.103)
> > ;; WHEN: Mon Dec 10 15:14:40 2012
> > ;; MSG SIZE ?rcvd: 204
> > ?
> > ?
> > On Dec 10, 2012, at 3:12 PM, Richard Mahoney <richard.mahoney@tracesmart.co.uk> wrote:
> >
> >
> > Seeing DNS issues for Facebook here.
> > Anyone else?
> > ?
> > Kind regards
> > ?
> > Richard Mahoney, CEH?
> > Systems Administrator
> > Tracesmart
> > T?029 2067 8534????M?07714 486543????E?richard.mahoney@tracesmart.co.uk
> > www.tracesmartcorporate.co.uk????www.traceiq.co.uk
> > Global Reach ?Dunleavy Drive ?Cardiff ?CF11 0SN
> > Follow us on?Twitter
> > ISO/IEC 27001?CERTIFICATE: GB 10/81945
> > We are proud to sponsor?missingpeople.org.uk
> > This email and any attachments are confidential to Tracesmart Ltd and are solely for use by the intended recipient. If you are not the intended recipient you must not disclose, copy or distribute its contents to any other person nor make use of its contents in any way. If you have received this email in error please forward a copy to?info@tracesmart.co.uk?and remove it from your system.This email and any attachments have been scanned for the presence of computer viruses. Neither Tracesmart Ltd nor the sender accepts any responsibility for computer viruses once this email has been transmitted. The content of this message may contain personal views, which are not the views of Tracesmart Ltd, unless specifically stated. Tracesmart may monitor email traffic data and also the content of email for the purposes of security and staff training.Tracesmart Ltd is a company registered in England & Wales with company registration number 3827062 whose registered
> >? office is at Global Reach, Dunleavy Drive, Cardiff CF11 0SN. ?Our Data Protection Number is Z708281X and our Consumer Credit Licence Number is 565961.
> > ?
> > _______________________________________________
> > Outages mailing list
> > Outages@outages.org
> > https://puck.nether.net/mailman/listinfo/outages
> > ?
> > _______________________________________________
> > Outages mailing list
> > Outages@outages.org
> > https://puck.nether.net/mailman/listinfo/outages
>
> > _______________________________________________
> > Outages mailing list
> > Outages@outages.org
> > https://puck.nether.net/mailman/listinfo/outages
_______________________________________________
Outages mailing list
Outages@outages.org
https://puck.nether.net/mailman/listinfo/outages
Re: Facebook [ In reply to ]
I didn't realize this network operator outage listserv converted to a
political satire listserv...

On Mon, Sep 3, 2018, 17:02 Jon Lewis via Outages <outages@outages.org>
wrote:

> So...did Trump shut them down? :)
>
> ----------------------------------------------------------------------
> Jon Lewis, MCP :) | I route
> | therefore you are
> _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
> _______________________________________________
> Outages mailing list
> Outages@outages.org
> https://puck.nether.net/mailman/listinfo/outages
>
Re: Facebook [ In reply to ]
Seems to be worldwide, judging by the disqus comments pouring in (dozens
per minute) at downdetector.com. Went down for me in Philadelphia at ~4:45
PM EDT.

On Mon, Sep 3, 2018 at 5:03 PM Jon Lewis via Outages <outages@outages.org>
wrote:

> So...did Trump shut them down? :)
>
> ----------------------------------------------------------------------
> Jon Lewis, MCP :) | I route
> | therefore you are
> _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
> _______________________________________________
> Outages mailing list
> Outages@outages.org
> https://puck.nether.net/mailman/listinfo/outages
>
Re: Facebook [ In reply to ]
Political commentary aside, I confirm from a metoo standpoint in southern California.

Get Outlook for Android<https://aka.ms/ghei36>

________________________________
From: Outages <outages-bounces@outages.org> on behalf of Dom Colangelo via Outages <outages@outages.org>
Sent: Monday, September 3, 2018 2:08:06 PM
To: Jon Lewis
Cc: Outages@outages.org
Subject: Re: [outages] Facebook

I didn't realize this network operator outage listserv converted to a political satire listserv...

On Mon, Sep 3, 2018, 17:02 Jon Lewis via Outages <outages@outages.org<mailto:outages@outages.org>> wrote:
So...did Trump shut them down? :)

----------------------------------------------------------------------
Jon Lewis, MCP :) | I route
| therefore you are
_________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
_______________________________________________
Outages mailing list
Outages@outages.org<mailto:Outages@outages.org>
https://puck.nether.net/mailman/listinfo/outages
Re: Facebook [ In reply to ]
Can't handle some light humor? They are down though. At least from here
(both web and mobile) via Spectrum cable and AT&T wireless.

On Mon, 3 Sep 2018, Dominick Colangelo wrote:

> I didn't realize this network operator outage listserv converted to a political satire listserv. 
>
> On Mon, Sep 3, 2018, 17:02 Jon Lewis via Outages <outages@outages.org> wrote:
> So...did Trump shut them down?  :)
>
> ----------------------------------------------------------------------
>   Jon Lewis, MCP :)           |  I route
>                               |  therefore you are
> _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
> _______________________________________________
> Outages mailing list
> Outages@outages.org
> https://puck.nether.net/mailman/listinfo/outages
>
>
>

----------------------------------------------------------------------
Jon Lewis, MCP :) | I route
| therefore you are
_________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
Re: Facebook [ In reply to ]
FB is working perfectly for me in Denver on Comcast.

On Mon, Sep 3, 2018 at 3:13 PM Jon Lewis via Outages <outages@outages.org>
wrote:

> Can't handle some light humor? They are down though. At least from here
> (both web and mobile) via Spectrum cable and AT&T wireless.
>
> On Mon, 3 Sep 2018, Dominick Colangelo wrote:
>
> > I didn't realize this network operator outage listserv converted to a
> political satire listserv.
> >
> > On Mon, Sep 3, 2018, 17:02 Jon Lewis via Outages <outages@outages.org>
> wrote:
> > So...did Trump shut them down? :)
> >
> >
> ----------------------------------------------------------------------
> > Jon Lewis, MCP :) | I route
> > | therefore you are
> > _________ http://www.lewis.org/~jlewis/pgp for PGP public
> key_________
> > _______________________________________________
> > Outages mailing list
> > Outages@outages.org
> > https://puck.nether.net/mailman/listinfo/outages
> >
> >
> >
>
> ----------------------------------------------------------------------
> Jon Lewis, MCP :) | I route
> | therefore you are
> _________ http://www.lewis.org/~jlewis/pgp for PGP public
> key________________________________________________________
> Outages mailing list
> Outages@outages.org
> https://puck.nether.net/mailman/listinfo/outages
>
Re: Facebook [ In reply to ]
Out in NJ and NY

Rich





> On Sep 3, 2018, at 17:14, John Neiberger via Outages <outages@outages.org> wrote:
>
> FB is working perfectly for me in Denver on Comcast.
>
>> On Mon, Sep 3, 2018 at 3:13 PM Jon Lewis via Outages <outages@outages.org> wrote:
>> Can't handle some light humor? They are down though. At least from here
>> (both web and mobile) via Spectrum cable and AT&T wireless.
>>
>> On Mon, 3 Sep 2018, Dominick Colangelo wrote:
>>
>> > I didn't realize this network operator outage listserv converted to a political satire listserv.
>> >
>> > On Mon, Sep 3, 2018, 17:02 Jon Lewis via Outages <outages@outages.org> wrote:
>> > So...did Trump shut them down? :)
>> >
>> > ----------------------------------------------------------------------
>> > Jon Lewis, MCP :) | I route
>> > | therefore you are
>> > _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
>> > _______________________________________________
>> > Outages mailing list
>> > Outages@outages.org
>> > https://puck.nether.net/mailman/listinfo/outages
>> >
>> >
>> >
>>
>> ----------------------------------------------------------------------
>> Jon Lewis, MCP :) | I route
>> | therefore you are
>> _________ http://www.lewis.org/~jlewis/pgp for PGP public key________________________________________________________
>> Outages mailing list
>> Outages@outages.org
>> https://puck.nether.net/mailman/listinfo/outages
> _______________________________________________
> Outages mailing list
> Outages@outages.org
> https://puck.nether.net/mailman/listinfo/outages
Re: Facebook [ In reply to ]
I just hope people don't flood the 911 call centers again...

On Mon, Sep 3, 2018 at 5:15 PM John Neiberger via Outages <
outages@outages.org> wrote:

> FB is working perfectly for me in Denver on Comcast.
>
> On Mon, Sep 3, 2018 at 3:13 PM Jon Lewis via Outages <outages@outages.org>
> wrote:
>
>> Can't handle some light humor? They are down though. At least from here
>> (both web and mobile) via Spectrum cable and AT&T wireless.
>>
>> On Mon, 3 Sep 2018, Dominick Colangelo wrote:
>>
>> > I didn't realize this network operator outage listserv converted to a
>> political satire listserv.
>> >
>> > On Mon, Sep 3, 2018, 17:02 Jon Lewis via Outages <outages@outages.org>
>> wrote:
>> > So...did Trump shut them down? :)
>> >
>> >
>> ----------------------------------------------------------------------
>> > Jon Lewis, MCP :) | I route
>> > | therefore you are
>> > _________ http://www.lewis.org/~jlewis/pgp for PGP public
>> key_________
>> > _______________________________________________
>> > Outages mailing list
>> > Outages@outages.org
>> > https://puck.nether.net/mailman/listinfo/outages
>> >
>> >
>> >
>>
>> ----------------------------------------------------------------------
>> Jon Lewis, MCP :) | I route
>> | therefore you are
>> _________ http://www.lewis.org/~jlewis/pgp for PGP public
>> key________________________________________________________
>> Outages mailing list
>> Outages@outages.org
>> https://puck.nether.net/mailman/listinfo/outages
>>
> _______________________________________________
> Outages mailing list
> Outages@outages.org
> https://puck.nether.net/mailman/listinfo/outages
>


--


[image: --]
Keith I Myers
[image: http://]http://KMyers.me
Mobile : (305)-929-3475
Re: Facebook [ In reply to ]
iOS app is working for me from Verizon in Nebraska (even did a test post) - but if I attempt to log in via the mobile website page it tells me Facebook is down for maintenance.

-Ryan McGinnis
http://bigstormpicture.com
PGP: 0x5C738727
Sent with ProtonMail Secure Email

On Mon, Sep 3, 2018 at 4:09 PM, Jay Farrell via Outages <outages@outages.org> wrote:

> Seems to be worldwide, judging by the disqus comments pouring in (dozens per minute) at downdetector.com. Went down for me in Philadelphia at ~4:45 PM EDT.
>
> On Mon, Sep 3, 2018 at 5:03 PM Jon Lewis via Outages <outages@outages.org> wrote:
>
>> So...did Trump shut them down? :)
>>
>> ----------------------------------------------------------------------
>> Jon Lewis, MCP :) | I route
>> | therefore you are
>> _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
>> _______________________________________________
>> Outages mailing list
>> Outages@outages.org
>> https://puck.nether.net/mailman/listinfo/outages
Re: Facebook [ In reply to ]
Lol. Mobile app for Droid works but web page does not in Pittsburgh.

-bruce




On Mon, Sep 3, 2018 at 5:19 PM -0400, "Keith I Myers via Outages" <outages@outages.org<mailto:outages@outages.org>> wrote:

I just hope people don't flood the 911 call centers again...

On Mon, Sep 3, 2018 at 5:15 PM John Neiberger via Outages <outages@outages.org<mailto:outages@outages.org>> wrote:
FB is working perfectly for me in Denver on Comcast.

On Mon, Sep 3, 2018 at 3:13 PM Jon Lewis via Outages <outages@outages.org<mailto:outages@outages.org>> wrote:
Can't handle some light humor? They are down though. At least from here
(both web and mobile) via Spectrum cable and AT&T wireless.

On Mon, 3 Sep 2018, Dominick Colangelo wrote:

> I didn't realize this network operator outage listserv converted to a political satire listserv.
>
> On Mon, Sep 3, 2018, 17:02 Jon Lewis via Outages <outages@outages.org<mailto:outages@outages.org>> wrote:
> So...did Trump shut them down? :)
>
> ----------------------------------------------------------------------
> Jon Lewis, MCP :) | I route
> | therefore you are
> _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
> _______________________________________________
> Outages mailing list
> Outages@outages.org<mailto:Outages@outages.org>
> https://puck.nether.net/mailman/listinfo/outages
>
>
>

----------------------------------------------------------------------
Jon Lewis, MCP :) | I route
| therefore you are
_________ http://www.lewis.org/~jlewis/pgp for PGP public key________________________________________________________
Outages mailing list
Outages@outages.org<mailto:Outages@outages.org>
https://puck.nether.net/mailman/listinfo/outages
_______________________________________________
Outages mailing list
Outages@outages.org<mailto:Outages@outages.org>
https://puck.nether.net/mailman/listinfo/outages


--


Keith I Myers
http://KMyers.me
Mobile : (305)-929-3475
Re: Facebook [ In reply to ]
Web and Messenger app both down from Ohio on v6

> On Sep 3, 2018, at 5:25 PM, Bruce Freshwater via Outages <outages@outages.org> wrote:
>
> Lol. Mobile app for Droid works but web page does not in Pittsburgh.
>
> -bruce
>
>
>
>
> On Mon, Sep 3, 2018 at 5:19 PM -0400, "Keith I Myers via Outages" <outages@outages.org <mailto:outages@outages.org>> wrote:
>
> I just hope people don't flood the 911 call centers again...
>
> On Mon, Sep 3, 2018 at 5:15 PM John Neiberger via Outages <outages@outages.org <mailto:outages@outages.org>> wrote:
> FB is working perfectly for me in Denver on Comcast.
>
> On Mon, Sep 3, 2018 at 3:13 PM Jon Lewis via Outages <outages@outages.org <mailto:outages@outages.org>> wrote:
> Can't handle some light humor? They are down though. At least from here
> (both web and mobile) via Spectrum cable and AT&T wireless.
>
> On Mon, 3 Sep 2018, Dominick Colangelo wrote:
>
> > I didn't realize this network operator outage listserv converted to a political satire listserv.
> >
> > On Mon, Sep 3, 2018, 17:02 Jon Lewis via Outages <outages@outages.org <mailto:outages@outages.org>> wrote:
> > So...did Trump shut them down? :)
> >
> > ----------------------------------------------------------------------
> > Jon Lewis, MCP :) | I route
> > | therefore you are
> > _________ http://www.lewis.org/~jlewis/pgp <http://www.lewis.org/~jlewis/pgp> for PGP public key_________
> > _______________________________________________
> > Outages mailing list
> > Outages@outages.org <mailto:Outages@outages.org>
> > https://puck.nether.net/mailman/listinfo/outages <https://puck.nether.net/mailman/listinfo/outages>
> >
> >
> >
>
> ----------------------------------------------------------------------
> Jon Lewis, MCP :) | I route
> | therefore you are
> _________ http://www.lewis.org/~jlewis/pgp <http://www.lewis.org/~jlewis/pgp> for PGP public key________________________________________________________
> Outages mailing list
> Outages@outages.org <mailto:Outages@outages.org>
> https://puck.nether.net/mailman/listinfo/outages <https://puck.nether.net/mailman/listinfo/outages>
> _______________________________________________
> Outages mailing list
> Outages@outages.org <mailto:Outages@outages.org>
> https://puck.nether.net/mailman/listinfo/outages <https://puck.nether.net/mailman/listinfo/outages>
>
>
> --
>
>
> Keith I Myers
> http://KMyers.me <http://kmyers.me/>
> Mobile : (305)-929-3475
>
>
>
>
> _______________________________________________
> Outages mailing list
> Outages@outages.org
> https://puck.nether.net/mailman/listinfo/outages
Re: Facebook [ In reply to ]
Appears to be a DNS issue for both Facebook and WhatsApp.


--
Bob Harman


From: Outages <outages-bounces@outages.org> on behalf of Bruce Freshwater via Outages <outages@outages.org>
Reply-To: Bruce Freshwater <BFreshwater@sierraexperts.com>
Date: Monday, October 4, 2021 at 12:31 PM
To: "outages@outages.org" <outages@outages.org>
Subject: [outages] Facebook

-----EXTERNAL EMAIL – USE CAUTION -----

Down in Pittsburgh via Comcast.

Regards,
Bruce Freshwater

Direct: 412.722.1701
Mobile: 412.292.0282
Main: 866.707.5869
Fax: 412.774.2469
Email: BFreshwater@SierraExperts.com<mailto:BFreshwater@SierraExperts.com>
Request Help: Support Portal (SIM)<sim.sierraincidentmanager.com>

Sierra Experts [cid:image001.jpg@01D7B91B.D13C6250]
2 Robinson Plaza, Suite 300, Pittsburgh, PA 15205-1017
Our Services: Managed Services<https://urldefense.com/v3/__http:/www.sierraexperts.com/managed-services/__;!!DfQtb1M!ECR4EA04FvSZ50o40FUlww-Iod9TVvezsbM34QoCdXkmv7CQ_Zti_QaJybdxkAk$> | Development & Design<https://urldefense.com/v3/__http:/www.sierraexperts.com/development-and-design/__;!!DfQtb1M!ECR4EA04FvSZ50o40FUlww-Iod9TVvezsbM34QoCdXkmv7CQ_Zti_QaJyKz19rk$> | Telephony & VoIP<https://urldefense.com/v3/__http:/www.sierraexperts.com/telephony-and-voip/__;!!DfQtb1M!ECR4EA04FvSZ50o40FUlww-Iod9TVvezsbM34QoCdXkmv7CQ_Zti_QaJpFr5Ezo$> | Hardware & Software<https://urldefense.com/v3/__http:/www.sierraexperts.com/hardware-and-software/__;!!DfQtb1M!ECR4EA04FvSZ50o40FUlww-Iod9TVvezsbM34QoCdXkmv7CQ_Zti_QaJyXLTbvo$> | Data Center<https://urldefense.com/v3/__http:/sierradatacenters.com/__;!!DfQtb1M!ECR4EA04FvSZ50o40FUlww-Iod9TVvezsbM34QoCdXkmv7CQ_Zti_QaJav9gTiY$>
Stay Connected: Website<https://urldefense.com/v3/__http:/www.sierraexperts.com/__;!!DfQtb1M!ECR4EA04FvSZ50o40FUlww-Iod9TVvezsbM34QoCdXkmv7CQ_Zti_QaJT21rJ4E$> | Awards<https://urldefense.com/v3/__http:/www.sierraexperts.com/about-us/awards/__;!!DfQtb1M!ECR4EA04FvSZ50o40FUlww-Iod9TVvezsbM34QoCdXkmv7CQ_Zti_QaJhwlJUVY$> | Facebook<https://urldefense.com/v3/__https:/www.facebook.com/SierraExperts__;!!DfQtb1M!ECR4EA04FvSZ50o40FUlww-Iod9TVvezsbM34QoCdXkmv7CQ_Zti_QaJMcp4kU0$> | Twitter<https://urldefense.com/v3/__https:/twitter.com/sierraexperts__;!!DfQtb1M!ECR4EA04FvSZ50o40FUlww-Iod9TVvezsbM34QoCdXkmv7CQ_Zti_QaJvvyW5XM$> | LinkedIn<https://urldefense.com/v3/__https:/www.linkedin.com/company/sierraexperts__;!!DfQtb1M!ECR4EA04FvSZ50o40FUlww-Iod9TVvezsbM34QoCdXkmv7CQ_Zti_QaJqPj4Ii0$> | Blog<https://urldefense.com/v3/__http:/www.sierraexperts.com/about-us/blog/__;!!DfQtb1M!ECR4EA04FvSZ50o40FUlww-Iod9TVvezsbM34QoCdXkmv7CQ_Zti_QaJ9mBE40Y$>
[cid:image002.png@01D7B91B.D13C6250]


CAUTION: This email originated outside of the organization. DO NOT CLICK any links, open attachments or reply unless you were expecting this email and trust that the content is safe. Keep confidential data safe.
Re: Facebook [ In reply to ]
It is looking "ok" in Texas....for now:

;; Got answer:

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 12849

;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1


;; OPT PSEUDOSECTION:

; EDNS: version: 0, flags:; udp: 512

;; QUESTION SECTION:

;www.facebook.com. IN A


;; ANSWER SECTION:

www.facebook.com. 3475 IN CNAME star-mini.c10r.facebook.com.

star-mini.c10r.facebook.com. 12 IN A 157.240.25.35


;; Query time: 19 msec

;; SERVER: 8.8.8.8#53(8.8.8.8)

;; WHEN: Mon Oct 04 17:32:00 CDT 2021

;; MSG SIZE rcvd: 90


Jason Pope

On Mon, Oct 4, 2021 at 5:32 PM Matt Harris via Outages <outages@outages.org>
wrote:

> I'm getting NXDOMAINs for facebook.com and instagram.com at my home in
> Kansas City. Recursive resolver is a local system, so this isn't related to
> any ISP's recursive DNS hosts.
>
> - Matt
>
>
> Matt Harris?
> | Infrastructure Lead
> 816?256?5446
> | Direct
> Looking for help?
> *Helpdesk* <https://help.netfire.net/>
> | *Email Support* <help@netfire.net>
>
> We build customized end?to?end technology solutions powered by NetFire Cloud.
> On Mon, Oct 4, 2021 at 11:25 AM Bruce Freshwater via Outages <
> outages@outages.org> wrote:
>
>> Down in Pittsburgh via Comcast.
>>
>>
>>
>> Regards,
>>
>> *Bruce Freshwater*
>>
>>
>>
>> *Direct: *412.722.1701
>>
>> *Mobile: *412.292.0282
>>
>> *Main: *866.707.5869
>>
>> *Fax: * 412.774.2469
>>
>> *Email: * *BFreshwater@SierraExperts.com <BFreshwater@SierraExperts.com>*
>>
>> *Request Help: *Support Portal (SIM)
>> <http://sim.sierraincidentmanager.com>
>>
>>
>>
>> *Sierra Experts **[image: cid:image002.jpg@01D1C645.CF870330]*
>>
>> 2 Robinson Plaza, Suite 300, Pittsburgh, PA 15205-1017
>>
>> Our Services: Managed Services
>> <http://www.sierraexperts.com/managed-services/> | Development & Design
>> <http://www.sierraexperts.com/development-and-design/> | Telephony & VoIP
>> <http://www.sierraexperts.com/telephony-and-voip/> | Hardware & Software
>> <http://www.sierraexperts.com/hardware-and-software/> | Data Center
>> <http://sierradatacenters.com/>
>>
>> Stay Connected: Website <http://www.sierraexperts.com/> | Awards
>> <http://www.sierraexperts.com/about-us/awards/> | Facebook
>> <https://www.facebook.com/SierraExperts> | Twitter
>> <https://twitter.com/sierraexperts> | LinkedIn
>> <https://www.linkedin.com/company/sierraexperts> | Blog
>> <http://www.sierraexperts.com/about-us/blog/>
>>
>> [image: cid:image001.png@01D573C1.8B84DB80]
>>
>>
>> _______________________________________________
>> Outages mailing list
>> Outages@outages.org
>> https://puck.nether.net/mailman/listinfo/outages
>>
> _______________________________________________
> Outages mailing list
> Outages@outages.org
> https://puck.nether.net/mailman/listinfo/outages
>
Re: Facebook [ In reply to ]
Cloudflare says it's BGP
https://blog.cloudflare.com/october-2021-facebook-outage/


On Mon, Oct 4, 2021 at 4:53 PM Julian Panke via Outages <outages@outages.org>
wrote:

> If you look at the amount of SERVFAIL from around the world it seems to be
> save to say that it is a global outage:
> https://atlas.ripe.net/measurements/32421799/#probes
>
> Mit freundlichen Grüßen
>
> Julian Panke
>
> ??????? Original Message ???????
> On Monday, October 4th, 2021 at 5:54 PM, Bruce Freshwater via Outages <
> outages@outages.org> wrote:
>
> Down in Pittsburgh via Comcast.
>
>
>
> Regards,
>
> *Bruce Freshwater*
>
>
>
> *Direct: *412.722.1701
>
> *Mobile: *412.292.0282
>
> *Main: *866.707.5869
>
> *Fax: * 412.774.2469
>
> *Email: * *BFreshwater@SierraExperts.com <BFreshwater@SierraExperts.com>*
>
> *Request Help: *Support Portal (SIM)
> <http://sim.sierraincidentmanager.com>
>
>
>
> *Sierra Experts **[image: cid:image002.jpg@01D1C645.CF870330]*
>
> 2 Robinson Plaza, Suite 300, Pittsburgh, PA 15205-1017
>
> Our Services: Managed Services
> <http://www.sierraexperts.com/managed-services/> | Development & Design
> <http://www.sierraexperts.com/development-and-design/> | Telephony & VoIP
> <http://www.sierraexperts.com/telephony-and-voip/> | Hardware & Software
> <http://www.sierraexperts.com/hardware-and-software/> | Data Center
> <http://sierradatacenters.com/>
>
> Stay Connected: Website <http://www.sierraexperts.com/> | Awards
> <http://www.sierraexperts.com/about-us/awards/> | Facebook
> <https://www.facebook.com/SierraExperts> | Twitter
> <https://twitter.com/sierraexperts> | LinkedIn
> <https://www.linkedin.com/company/sierraexperts> | Blog
> <http://www.sierraexperts.com/about-us/blog/>
>
> [image: cid:image001.png@01D573C1.8B84DB80]
>
>
>
>
> _______________________________________________
> Outages mailing list
> Outages@outages.org
> https://puck.nether.net/mailman/listinfo/outages
>


--

Kurt Albershardt | Mimbres Communications, LLC | 575-342-0042
Re: Facebook [ In reply to ]
It's never DNS. It's the bonehead that messed up DNS.


-------- Original message --------
From: Ryan McGinnis via Outages <outages@outages.org>
Date: 10/4/21 6:41 PM (GMT-06:00)
To: Outages <outages@outages.org>
Subject: Re: [outages] Facebook

CAUTION: This email is from OUTSIDE our organization.
Please do not open/download any attachment or click any link unless you know it's safe.

Facebook looks like a dns issue - I get SERVFAIL from quad 9s when attempting an nslookup. Cloudflare has no records, Google dns doesn?t know, opendns makes an exaggerated shrug gesture and looks worried

Here?s a sysadmin poem:

It?s not DNS
There is no way it?s DNS
It was DNS

-Ryan McGinnis
ryan@digicana.com<mailto:ryan@digicana.com>
http://www.bigstormpicture.com<http://bigstormpicture.com>
GPG: 5C73 8727 EE58 786A 777C 4F1D B5AA 3FA3 486E D7AD


On Mon, Oct 4, 2021 at 10:54, Bruce Freshwater via Outages <outages@outages.org<mailto:outages@outages.org>> wrote:
Down in Pittsburgh via Comcast.

Regards,
Bruce Freshwater

Direct: 412.722.1701
Mobile: 412.292.0282
Main: 866.707.5869
Fax: 412.774.2469
Email: BFreshwater@SierraExperts.com<mailto:BFreshwater@SierraExperts.com>
Request Help: Support Portal (SIM)<sim.sierraincidentmanager.com>

Sierra Experts [cid:image002.jpg@01D1C645.CF870330]
2 Robinson Plaza, Suite 300, Pittsburgh, PA 15205-1017
Our Services: Managed Services<http://www.sierraexperts.com/managed-services/> | Development & Design<http://www.sierraexperts.com/development-and-design/> | Telephony & VoIP<http://www.sierraexperts.com/telephony-and-voip/> | Hardware & Software<http://www.sierraexperts.com/hardware-and-software/> | Data Center<http://sierradatacenters.com/>
Stay Connected: Website<http://www.sierraexperts.com/> | Awards<http://www.sierraexperts.com/about-us/awards/> | Facebook<https://www.facebook.com/SierraExperts> | Twitter<https://twitter.com/sierraexperts> | LinkedIn<https://www.linkedin.com/company/sierraexperts> | Blog<http://www.sierraexperts.com/about-us/blog/>
[cid:image001.png@01D573C1.8B84DB80]




Jason Baugher, Network Operations Manager
405 Emminga Road | PO Box 217 | Golden, IL 62339-0217
P:(217) 696-4411 | F:(217) 696-4811 | www.adams.net<http://www.adams.net/>
[Adams-Logo]<http://adams.net/>
________________________________
The information contained in this email message is PRIVILEGED AND CONFIDENTIAL, and is intended for the use of the addressee and no one else. If you are not the intended recipient, please do not read, distribute, reproduce or use this email message (or the attachments) and notify the sender of the mistaken transmission. Thank you.
Re: Facebook [ In reply to ]
Restored now and working here now (after 3-4 hrs of outage).
Location: India


anurag@desktop ~> dig www.facebook.com a

; <<>> DiG 9.16.1-Ubuntu <<>> www.facebook.com a
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 56238
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;www.facebook.com. IN A

;; ANSWER SECTION:
www.facebook.com. 3396 IN CNAME star-mini.c10r.facebook.com.
star-mini.c10r.facebook.com. 59 IN A 157.240.239.35

;; Query time: 12 msec
;; SERVER: 127.0.0.53#53(127.0.0.53)
;; WHEN: Tue Oct 05 05:27:56 IST 2021
;; MSG SIZE rcvd: 90

anurag@desktop ~>



anurag@desktop ~> ping -c 5 www.facebook.com
PING star-mini.c10r.facebook.com (157.240.239.35) 56(84) bytes of data.
64 bytes from edge-star-mini-shv-02-del1.facebook.com (157.240.239.35):
icmp_seq=1 ttl=57 time=6.32 ms
64 bytes from edge-star-mini-shv-02-del1.facebook.com (157.240.239.35):
icmp_seq=2 ttl=57 time=6.03 ms
64 bytes from edge-star-mini-shv-02-del1.facebook.com (157.240.239.35):
icmp_seq=3 ttl=57 time=6.31 ms
64 bytes from edge-star-mini-shv-02-del1.facebook.com (157.240.239.35):
icmp_seq=4 ttl=57 time=6.28 ms
64 bytes from edge-star-mini-shv-02-del1.facebook.com (157.240.239.35):
icmp_seq=5 ttl=57 time=6.27 ms

--- star-mini.c10r.facebook.com ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4005ms
rtt min/avg/max/mdev = 6.032/6.241/6.320/0.106 ms
anurag@desktop ~>


On Tue, Oct 5, 2021 at 4:36 AM Bob Harman - Huntersville via Outages <
outages@outages.org> wrote:

> Appears to be a DNS issue for both Facebook and WhatsApp.
>
>
>
>
>
> --
>
> Bob Harman
>
>
>
>
>
> *From: *Outages <outages-bounces@outages.org> on behalf of Bruce
> Freshwater via Outages <outages@outages.org>
> *Reply-To: *Bruce Freshwater <BFreshwater@sierraexperts.com>
> *Date: *Monday, October 4, 2021 at 12:31 PM
> *To: *"outages@outages.org" <outages@outages.org>
> *Subject: *[outages] Facebook
>
>
>
> *-----EXTERNAL EMAIL – USE CAUTION -----*
>
>
>
> Down in Pittsburgh via Comcast.
>
>
>
> Regards,
>
> *Bruce Freshwater*
>
>
>
> *Direct: *412.722.1701
>
> *Mobile: *412.292.0282
>
> *Main: *866.707.5869
>
> *Fax: * 412.774.2469
>
> *Email: * *BFreshwater@SierraExperts.com <BFreshwater@SierraExperts.com>*
>
> *Request Help: *Support Portal (SIM)
> <http://sim.sierraincidentmanager.com>
>
>
>
> *Sierra Experts *
>
> 2 Robinson Plaza, Suite 300, Pittsburgh, PA 15205-1017
>
> Our Services: Managed Services
> <https://urldefense.com/v3/__http:/www.sierraexperts.com/managed-services/__;!!DfQtb1M!ECR4EA04FvSZ50o40FUlww-Iod9TVvezsbM34QoCdXkmv7CQ_Zti_QaJybdxkAk$>
> | Development & Design
> <https://urldefense.com/v3/__http:/www.sierraexperts.com/development-and-design/__;!!DfQtb1M!ECR4EA04FvSZ50o40FUlww-Iod9TVvezsbM34QoCdXkmv7CQ_Zti_QaJyKz19rk$>
> | Telephony & VoIP
> <https://urldefense.com/v3/__http:/www.sierraexperts.com/telephony-and-voip/__;!!DfQtb1M!ECR4EA04FvSZ50o40FUlww-Iod9TVvezsbM34QoCdXkmv7CQ_Zti_QaJpFr5Ezo$>
> | Hardware & Software
> <https://urldefense.com/v3/__http:/www.sierraexperts.com/hardware-and-software/__;!!DfQtb1M!ECR4EA04FvSZ50o40FUlww-Iod9TVvezsbM34QoCdXkmv7CQ_Zti_QaJyXLTbvo$>
> | Data Center
> <https://urldefense.com/v3/__http:/sierradatacenters.com/__;!!DfQtb1M!ECR4EA04FvSZ50o40FUlww-Iod9TVvezsbM34QoCdXkmv7CQ_Zti_QaJav9gTiY$>
>
> Stay Connected: Website
> <https://urldefense.com/v3/__http:/www.sierraexperts.com/__;!!DfQtb1M!ECR4EA04FvSZ50o40FUlww-Iod9TVvezsbM34QoCdXkmv7CQ_Zti_QaJT21rJ4E$>
> | Awards
> <https://urldefense.com/v3/__http:/www.sierraexperts.com/about-us/awards/__;!!DfQtb1M!ECR4EA04FvSZ50o40FUlww-Iod9TVvezsbM34QoCdXkmv7CQ_Zti_QaJhwlJUVY$>
> | Facebook
> <https://urldefense.com/v3/__https:/www.facebook.com/SierraExperts__;!!DfQtb1M!ECR4EA04FvSZ50o40FUlww-Iod9TVvezsbM34QoCdXkmv7CQ_Zti_QaJMcp4kU0$>
> | Twitter
> <https://urldefense.com/v3/__https:/twitter.com/sierraexperts__;!!DfQtb1M!ECR4EA04FvSZ50o40FUlww-Iod9TVvezsbM34QoCdXkmv7CQ_Zti_QaJvvyW5XM$>
> | LinkedIn
> <https://urldefense.com/v3/__https:/www.linkedin.com/company/sierraexperts__;!!DfQtb1M!ECR4EA04FvSZ50o40FUlww-Iod9TVvezsbM34QoCdXkmv7CQ_Zti_QaJqPj4Ii0$>
> | Blog
> <https://urldefense.com/v3/__http:/www.sierraexperts.com/about-us/blog/__;!!DfQtb1M!ECR4EA04FvSZ50o40FUlww-Iod9TVvezsbM34QoCdXkmv7CQ_Zti_QaJ9mBE40Y$>
>
>
>
>
>
> *CAUTION:* *This email originated outside of the organization. DO NOT
> CLICK any links, open attachments or reply unless you were expecting this
> email and trust that the content is safe. Keep confidential data safe. *
>
>
> _______________________________________________
> Outages mailing list
> Outages@outages.org
> https://puck.nether.net/mailman/listinfo/outages
>


--
Anurag Bhatia
anuragbhatia.com