Mailing List Archive

Random ServerIron Crashes
Anyone ever have to deal with random switch crashes? Specifically I am dealing with ServerIronGT-E chassis devices. I have one in a HA pair that has crashed 4 times in the last 12 hours. Logs and syslogged data off device aren’t giving me anything to go on other than the usual telnet/ssh/snmp abuse. I just blocked that traffic at the edge routers, but would love to find out why it crashed instead of simply speculating.

-Rob Heilman


_______________________________________________
foundry-nsp mailing list
foundry-nsp@puck.nether.net
http://puck.nether.net/mailman/listinfo/foundry-nsp
Re: Random ServerIron Crashes [ In reply to ]
Some years ago I had similar behaviour on one of our SI-400s with it swapping
onto the secondary management card. The other SI in the HA pair didn't appear
to be affected if we made that one live so we assumed a hardware fault on the
resetting one. We spent some time swapping management and line cards in and
out but that didn't resolve it so it looked like it was the chassis/backplane.

I am afraid that I never managed to get to the bottom of it as I was migrating
to ADXs at the time so it was easier to rip and replace. Mind you we've ripped
out the ADXs now as Brocade refused to accept a bug report for the SSL related
denial of service that I discovered. So ended an 16 year engagement with
Foundry/Brocade. Way to go Brocade!


Chris

_______________________________________________
foundry-nsp mailing list
foundry-nsp@puck.nether.net
http://puck.nether.net/mailman/listinfo/foundry-nsp
Re: Random ServerIron Crashes [ In reply to ]
On most platforms you can run "dm save_area” or “dm save_area 0” to view all of the crash dump areas. If you ran "show tech” aka "show tech-support” aka “show short-tech-support” the crash dump areas are often listed at the end of the report. On some models there may even be a short benchmark run on the modules at the end of the “show tech”. The crash dumps may or may not be helpful to you(or Brocade) in determining the cause of the crashes. BTW, if you reboot manually you will clear the crash dump areas.

If you are looking for a simple cause, you might want to look for hardware anomalies like to compare the DRAM sizes on the management module(s) between your working and crashing SI’s, as well as the size of your CAM on each of the IPC/IGC’s of your modules.

--JK

> On Apr 4, 2015, at 4:23 AM, Rob Heilman <rheilman@echolabs.net> wrote:
>
> Anyone ever have to deal with random switch crashes? Specifically I am dealing with ServerIronGT-E chassis devices. I have one in a HA pair that has crashed 4 times in the last 12 hours. Logs and syslogged data off device aren’t giving me anything to go on other than the usual telnet/ssh/snmp abuse. I just blocked that traffic at the edge routers, but would love to find out why it crashed instead of simply speculating.
>
> -Rob Heilman
>
>
> _______________________________________________
> foundry-nsp mailing list
> foundry-nsp@puck.nether.net
> http://puck.nether.net/mailman/listinfo/foundry-nsp

_______________________________________________
foundry-nsp mailing list
foundry-nsp@puck.nether.net
http://puck.nether.net/mailman/listinfo/foundry-nsp
Re: Random ServerIron Crashes [ In reply to ]
Does the serial console show any output? I have our serial terminal console configured to capture that output.

Frank

-----Original Message-----
From: foundry-nsp [mailto:foundry-nsp-bounces@puck.nether.net] On Behalf Of Rob Heilman
Sent: Saturday, April 04, 2015 3:23 AM
To: foundry-nsp@puck.nether.net
Subject: [f-nsp] Random ServerIron Crashes

Anyone ever have to deal with random switch crashes? Specifically I am dealing with ServerIronGT-E chassis devices. I have one in a HA pair that has crashed 4 times in the last 12 hours. Logs and syslogged data off device aren’t giving me anything to go on other than the usual telnet/ssh/snmp abuse. I just blocked that traffic at the edge routers, but would love to find out why it crashed instead of simply speculating.

-Rob Heilman


_______________________________________________
foundry-nsp mailing list
foundry-nsp@puck.nether.net
http://puck.nether.net/mailman/listinfo/foundry-nsp

_______________________________________________
foundry-nsp mailing list
foundry-nsp@puck.nether.net
http://puck.nether.net/mailman/listinfo/foundry-nsp