Mailing List Archive

911 service is out as well in the affected area


------Original Message------
From: Jay Hennigan
Sender: outages-bounces at outages.org
Cc: outages at outages.org
Subject: [outages] Verizon Southern California issues?
Sent: Jul 6, 2009 10:31

We're seeing several T1s down in Oxnard, Camarillo, Thousand Oaks CA as
well as circuits busy on POTS calls to the area.

--
Jay Hennigan - CCIE #7880 - Network Engineering - jay at impulse.net
Impulse Internet Service - http://www.impulse.net/
Your local telephone and internet company - 805 884-6323 - WB6RDV
_______________________________________________
outages mailing list
outages at outages.org
https://puck.nether.net/mailman/listinfo/outages


Sent via BlackBerry from T-Mobile

We lost our ATT circuit at 10 am. still down.

They claim 'Transport Issues'

Clients are claiming that their ATT and Sprint T1's are down.

Our Sprint circuit is doing fine.

-bill
PixelGate Networks
(Thousand Oaks)





Jay Hennigan wrote:
> We're seeing several T1s down in Oxnard, Camarillo, Thousand Oaks CA
> as well as circuits busy on POTS calls to the area.
>
> --
> Jay Hennigan - CCIE #7880 - Network Engineering - jay at impulse.net
> Impulse Internet Service - http://www.impulse.net/
> Your local telephone and internet company - 805 884-6323 - WB6RDV
> _______________________________________________
> outages mailing list
> outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages

Jay Hennigan wrote:
> We're seeing several T1s down in Oxnard, Camarillo, Thousand Oaks CA as
> well as circuits busy on POTS calls to the area.
>
Fiber cut on Dawson drive in Camarillo is the suspected cause of a major
outage phone, network, & 911 in ventura county california

Evidently 911 has been restored.

http://twitter.com/socalincidents

TSP for the win.

Chaim Rieger wrote:
> Jay Hennigan wrote:
>> We're seeing several T1s down in Oxnard, Camarillo, Thousand Oaks CA as
>> well as circuits busy on POTS calls to the area.
>>
> Fiber cut on Dawson drive in Camarillo is the suspected cause of a major
> outage phone, network, & 911 in ventura county california
> _______________________________________________
> outages mailing list
> outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages
>

3.5 hours and no restoration?

If $BIGTELCOS are going to run everything through one physical path,
then they had better be able to roll techs to a break point really fast.

Get a police escort, ride a bicycle, something.

So they rerouted PSAP traffic, but not internet service.... fascinating.
Now I want to know how they have things setup.

I used to live in Oxnard, and Verizon is the last mile dial tone
provider out there. Cell reception was very bad, so I got a land line so
the NOC could get a hold of me. We had a Verizon fiber feed to our office.

Wasn't aware that ATT and Sprint had a presence out there. Or is it
reselling Verizon loops?

Something something redundant paths, something something we don't care
about anything except maximizing profit margins.

Bah.



William Kern wrote:
> We lost our ATT circuit at 10 am. still down.
>
> They claim 'Transport Issues'
>
> Clients are claiming that their ATT and Sprint T1's are down.
>
> Our Sprint circuit is doing fine.
>
> -bill
> PixelGate Networks
> (Thousand Oaks)
>
>
>
>
>
> Jay Hennigan wrote:
>> We're seeing several T1s down in Oxnard, Camarillo, Thousand Oaks CA
>> as well as circuits busy on POTS calls to the area.
>>
>> --
>> Jay Hennigan - CCIE #7880 - Network Engineering - jay at impulse.net
>> Impulse Internet Service - http://www.impulse.net/
>> Your local telephone and internet company - 805 884-6323 - WB6RDV
>> _______________________________________________
>> outages mailing list
>> outages at outages.org
>> https://puck.nether.net/mailman/listinfo/outages
>
> _______________________________________________
> outages mailing list
> outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages
>

any updates?

Charles Wyble wrote:
> 3.5 hours and no restoration?
>
> If $BIGTELCOS are going to run everything through one physical path,
> then they had better be able to roll techs to a break point really fast.
>
> Get a police escort, ride a bicycle, something.
>
> So they rerouted PSAP traffic, but not internet service.... fascinating.
> Now I want to know how they have things setup.
>
> I used to live in Oxnard, and Verizon is the last mile dial tone
> provider out there. Cell reception was very bad, so I got a land line so
> the NOC could get a hold of me. We had a Verizon fiber feed to our office.
>
> Wasn't aware that ATT and Sprint had a presence out there. Or is it
> reselling Verizon loops?
>
> Something something redundant paths, something something we don't care
> about anything except maximizing profit margins.
>
> Bah.
>
>
>
> William Kern wrote:
>> We lost our ATT circuit at 10 am. still down.
>>
>> They claim 'Transport Issues'
>>
>> Clients are claiming that their ATT and Sprint T1's are down.
>>
>> Our Sprint circuit is doing fine.
>>
>> -bill
>> PixelGate Networks
>> (Thousand Oaks)
>>
>>
>>
>>
>>
>> Jay Hennigan wrote:
>>> We're seeing several T1s down in Oxnard, Camarillo, Thousand Oaks CA
>>> as well as circuits busy on POTS calls to the area.
>>>
>>> --
>>> Jay Hennigan - CCIE #7880 - Network Engineering - jay at impulse.net
>>> Impulse Internet Service - http://www.impulse.net/
>>> Your local telephone and internet company - 805 884-6323 - WB6RDV
>>> _______________________________________________
>>> outages mailing list
>>> outages at outages.org
>>> https://puck.nether.net/mailman/listinfo/outages
>>
>> _______________________________________________
>> outages mailing list
>> outages at outages.org
>> https://puck.nether.net/mailman/listinfo/outages
>>
> _______________________________________________
> outages mailing list
> outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages

Any updates?



Charles Wyble wrote:
> 3.5 hours and no restoration?
>
> If $BIGTELCOS are going to run everything through one physical path,
> then they had better be able to roll techs to a break point really fast.
>
> Get a police escort, ride a bicycle, something.
>
> So they rerouted PSAP traffic, but not internet service.... fascinating.
> Now I want to know how they have things setup.
>
> I used to live in Oxnard, and Verizon is the last mile dial tone
> provider out there. Cell reception was very bad, so I got a land line so
> the NOC could get a hold of me. We had a Verizon fiber feed to our office.
>
> Wasn't aware that ATT and Sprint had a presence out there. Or is it
> reselling Verizon loops?
>
> Something something redundant paths, something something we don't care
> about anything except maximizing profit margins.
>
> Bah.
>
>
>
> William Kern wrote:
>> We lost our ATT circuit at 10 am. still down.
>>
>> They claim 'Transport Issues'
>>
>> Clients are claiming that their ATT and Sprint T1's are down.
>>
>> Our Sprint circuit is doing fine.
>>
>> -bill
>> PixelGate Networks
>> (Thousand Oaks)
>>
>>
>>
>>
>>
>> Jay Hennigan wrote:
>>> We're seeing several T1s down in Oxnard, Camarillo, Thousand Oaks CA
>>> as well as circuits busy on POTS calls to the area.
>>>
>>> --
>>> Jay Hennigan - CCIE #7880 - Network Engineering - jay at impulse.net
>>> Impulse Internet Service - http://www.impulse.net/
>>> Your local telephone and internet company - 805 884-6323 - WB6RDV
>>> _______________________________________________
>>> outages mailing list
>>> outages at outages.org
>>> https://puck.nether.net/mailman/listinfo/outages
>>
>> _______________________________________________
>> outages mailing list
>> outages at outages.org
>> https://puck.nether.net/mailman/listinfo/outages
>>
> _______________________________________________
> outages mailing list
> outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages

John Martinez wrote:
> Any updates?
>
According to VZW, most if not all customers should be back in business,

will continue repairs till 7 pm (local time)

Our ATT circuits are now up.

-bill

Chaim Rieger wrote:
> John Martinez wrote:
>
>> Any updates?
>>
>>
> According to VZW, most if not all customers should be back in business,
>
> will continue repairs till 7 pm (local time)
> _______________________________________________
> outages mailing list
> outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages
>

On Mon, Jul 6, 2009 at 6:09 PM, John Martinez<jmartinez at zero11.com> wrote:
> any updates?

<N.B.>

If the reader thinks of this as a breach of netiquette, please hit delete _now_.

VERIZON CONFIDENTIAL
FLASH SUMMARY
</N.B.>

=====

VERIZON CONFIDENTIAL
FLASH SUMMARY

FLASH NUMBER 2009070600119
ORGANIZATION NSMC
SEVERITY LEVEL CATA
CRITERIA HSI - HSI outage impacting 2500 or more customers
IMPACT LOSS OF DATA AND/OR CONNECTIVITY
NETWORK HSI
SUBNETWORK ROUTER
SPECIAL SERVICES
STATE CA
CITY THOUSAND OAKS
COUNTRY USA

OUTAGE START DATE 07-06-2009
OUTAGE START TIME 17:10:00 GMT
OUTAGE END DATE 07-06-2009
OUTAGE END TIME 22:26:00 GMT
DURATION 5:16:00

EQUIPMENT N/A

CAUSE OF OUTAGE FIBER/CABLE OUTAGE - OUTSIDE
CORRECTIVE ACTION Fiber spliced.

TICKET SOURCE WHITEBOARD
TICKET NUMBER 92475
LEC/OCC TICKET
PVC/CKT Affected


WEBSITE oasis.vzbi.com

COMMENTS
07-06-2009 18:34 GMT
Thousand Oaks, CA - HSI - Informational Bridge: (517) 477-3482 pc
865-5190. NSMC reports 3500 HSI customers are down. There is a fiber
cut located 1.3 miles outside of the Camarillo Central Office on
Dawson Drive. Approximately 1000 feet of fiber has been damaged.
There is no estimated time of restoral. Number of Trouble Tickets =
107 -----------
07-06-2009 19:45 GMT
NSMC reports that there are 252 attached tickets. Construction is
currently prepping the fiber with an ETA of new fiber onsite at 1:23
PM PT.
07-06-2009 21:24 GMT
NSMC reports that there are 345 attached tickets. All fiber is onsite
and pulled through duct, splicing commencing in approximately 5
minutes.
07-06-2009 22:49 GMT
NSMC reports construction spliced the fiber. HSI service restored.
406 Attached tickets.