Mailing List Archive

Hello,

If memory serves, this is due to multiple paths to a stack using the same SAS controller card. Best practice is of course to make sure that each of the redundant paths is on different hardware to remove that single point of failure.

IMHO it's more a warning (at least that's how I have interpreted it) than an error.

--rdp

-----Original Message-----
From: toasters-bounces@teaparty.net <toasters-bounces@teaparty.net> On Behalf Of John Stoffel
Sent: Tuesday, April 16, 2019 11:01 AM
To: Toasters <toasters@teaparty.net>
Subject: SAS Connectivity Monitor: DualPathToDiskShelf_Alert


Hi guys,

My out of support FAS3120 running 8.1.2 7-Mode has suddenly had two instances of the following message sent via autosupport notifications.

SAS Connectivity Monitor: DualPathToDiskShelf_Alert[50:05:0c:c1:02:04:db:ad] on mafs1a at Mon
Apr 15 20:04:00 PDT 2019

But I'm not seeing anything in the event or alert logs, nor do I see any problems via sysconfig, storage shelf show, etc.

I know the cables haven't been touched since I was just at the site this past weekend and the rack doors are closed and locked.

Thanks,
John

P.S. And yes, this sucker should have been upgraded two years ago, but management won't let us...
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

There was a bug about this too. As long as the ports ( a/b ) are not used to the same stack and ports (c/d) are not used on the same stack you are most of the way there.

Send the node shell output of
sadadmin expander_map

That would say a lot.

Typing errors courtesy of GBoard

Tim McCarthy, Principal Consultant

Proud Member of the NetApp ATeam

________________________________
From: toasters-bounces@teaparty.net on behalf of Payne, Richard <richard.payne@amd.com>
Sent: Tuesday, April 16, 2019 11:11 AM
To: John Stoffel; Toasters
Subject: RE: SAS Connectivity Monitor: DualPathToDiskShelf_Alert

Hello,

If memory serves, this is due to multiple paths to a stack using the same SAS controller card. Best practice is of course to make sure that each of the redundant paths is on different hardware to remove that single point of failure.

IMHO it's more a warning (at least that's how I have interpreted it) than an error.

--rdp

-----Original Message-----
From: toasters-bounces@teaparty.net <toasters-bounces@teaparty.net> On Behalf Of John Stoffel
Sent: Tuesday, April 16, 2019 11:01 AM
To: Toasters <toasters@teaparty.net>
Subject: SAS Connectivity Monitor: DualPathToDiskShelf_Alert


Hi guys,

My out of support FAS3120 running 8.1.2 7-Mode has suddenly had two instances of the following message sent via autosupport notifications.

SAS Connectivity Monitor: DualPathToDiskShelf_Alert[50:05:0c:c1:02:04:db:ad] on mafs1a at Mon
Apr 15 20:04:00 PDT 2019

But I'm not seeing anything in the event or alert logs, nor do I see any problems via sysconfig, storage shelf show, etc.

I know the cables haven't been touched since I was just at the site this past weekend and the rack doors are closed and locked.

Thanks,
John

P.S. And yes, this sucker should have been upgraded two years ago, but management won't let us...
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

Tim> There was a bug about this too. As long as the ports ( a/b ) are
Tim> not used to the same stack and ports (c/d) are not used on the
Tim> same stack you are most of the way there.

Hi Tim,
The suprising thing about these messages is that the system has been
in this configuration for the past six years, without any shelf
changes at all. I did look at the "sasadmin expander_map" output and
it's all looking good from what I see. I appended it to the end of
this email.

Tim> Send the node shell output of
Tim> sadadmin expander_map

Tim> That would say a lot.



Expanders on channel 0a:
Cannot Complete operation on channel 0a; Status Not Available.

Expanders on channel 0b:
Level 1: WWN 500a0980017df3bf, ID 0, Serial Number '
SHJ0000000159E4', Product 'DS424IOM6 ', Rev '0191', Slot A
Level 2: WWN 500a09800187e0ff, ID 1, Serial Number '
6000557869', Product 'DS224IOM6 ', Rev '0191', Slot A
Level 3: WWN 500a09800187b3ff, ID 2, Serial Number '
6000557833', Product 'DS224IOM6 ', Rev '0191', Slot A
Level 4: WWN 500a098001a1207f, ID 3, Serial Number '
6000557821', Product 'DS224IOM6 ', Rev '0191', Slot A
Level 5: WWN 500a098001a15cff, ID 4, Serial Number '
6000557819', Product 'DS224IOM6 ', Rev '0191', Slot A
Level 6: WWN 500a098001a11e7f, ID 5, Serial Number '
6000557807', Product 'DS224IOM6 ', Rev '0191', Slot A
Level 7: WWN 500a09800196fe7f, ID 6, Serial Number '
6000521559', Product 'DS224IOM6 ', Rev '0191', Slot A
Level 8: WWN 500a098001a3cfff, ID 7, Serial Number '
6000558124', Product 'DS224IOM6 ', Rev '0191', Slot A

Expanders on channel 3a:
Level 1: WWN 500a0980016f3c7f, ID 10, Serial Number '
SHJ000000015A3B', Product 'DS424IOM6 ', Rev '0191', Slot A
Level 2: WWN 500a098001976d3f, ID 11, Serial Number '
6000521420', Product 'DS224IOM6 ', Rev '0191', Slot A
Level 3: WWN 500a098001a3a2ff, ID 12, Serial Number '
6000558382', Product 'DS224IOM6 ', Rev '0191', Slot A
Level 4: WWN 500a098001a3c43f, ID 13, Serial Number '
6000557584', Product 'DS224IOM6 ', Rev '0191', Slot A
Level 5: WWN 500a098001a3b67f, ID 14, Serial Number '
6000558069', Product 'DS224IOM6 ', Rev '0191', Slot A
Level 6: WWN 500a098001a3c1ff, ID 15, Serial Number '
6000557613', Product 'DS224IOM6 ', Rev '0191', Slot A
Level 7: WWN 500a098001a3bb7f, ID 16, Serial Number '
6000558083', Product 'DS224IOM6 ', Rev '0191', Slot A

Expanders on channel 3b:
Level 1: WWN 500a098001a3cd7f, ID 7, Serial Number '
6000558124', Product 'DS224IOM6 ', Rev '0191', Slot B
Level 2: WWN 500a09800197263f, ID 6, Serial Number '
6000521559', Product 'DS224IOM6 ', Rev '0191', Slot B
Level 3: WWN 500a098001a158bf, ID 5, Serial Number '
6000557807', Product 'DS224IOM6 ', Rev '0191', Slot B
Level 4: WWN 500a098001a15aff, ID 4, Serial Number '
6000557819', Product 'DS224IOM6 ', Rev '0191', Slot B
Level 5: WWN 500a098001a104ff, ID 3, Serial Number '
6000557821', Product 'DS224IOM6 ', Rev '0191', Slot B
Level 6: WWN 500a09800187b63f, ID 2, Serial Number '
6000557833', Product 'DS224IOM6 ', Rev '0191', Slot B
Level 7: WWN 500a098001880c7f, ID 1, Serial Number '
6000557869', Product 'DS224IOM6 ', Rev '0191', Slot B
Level 8: WWN 500a0980017df53f, ID 0, Serial Number '
SHJ0000000159E4', Product 'DS424IOM6 ', Rev '0191', Slot B

Expanders on channel 3c:
Cannot Complete operation on channel 3c; Status Not Available.

Expanders on channel 3d:
Cannot Complete operation on channel 3d; Status Not Available.

Expanders on channel 4a:
Level 1: WWN 500a098001a0e8ff, ID 16, Serial Number '
6000558083', Product 'DS224IOM6 ', Rev '0191', Slot B
Level 2: WWN 500a098001a3c4ff, ID 15, Serial Number '
6000557613', Product 'DS224IOM6 ', Rev '0191', Slot B
Level 3: WWN 500a098001a3b77f, ID 14, Serial Number '
6000558069', Product 'DS224IOM6 ', Rev '0191', Slot B
Level 4: WWN 500a098001a3c23f, ID 13, Serial Number '
6000557584', Product 'DS224IOM6 ', Rev '0191', Slot B
Level 5: WWN 500a098001a3a3bf, ID 12, Serial Number '
6000558382', Product 'DS224IOM6 ', Rev '0191', Slot B
Level 6: WWN 500a098001976ebf, ID 11, Serial Number '
6000521420', Product 'DS224IOM6 ', Rev '0191', Slot B
Level 7: WWN 500a0980017f43ff, ID 10, Serial Number '
SHJ000000015A3B', Product 'DS424IOM6 ', Rev '0191', Slot B

Expanders on channel 4b:
Cannot Complete operation on channel 4b; Status Not Available.

Expanders on channel 4c:
Cannot Complete operation on channel 4c; Status Not Available.

Expanders on channel 4d:
Cannot Complete operation on channel 4d; Status Not Available.
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

So there it is...

Look at one stack, connected to 0b/3b
The other stack is connected to 3a/4a

For your best practice,

the SAS Adapter A/C ports should connect IN to the shelf on the SQR port

the SAS Adapter B/D ports should connect OUT to the shelf on the CIR port

I know there was some versions of code that complained if the IN/OUT were
both on like 3a/4a in your case.

Ideally, you should have something like:
Node 1
0a -> top shelf (DS4246, ID0), A side SQR port.
3b -> bottom shelf (DS2246, ID7), B side CIR port
3a -> top shelf (DS4246, ID10), A side SQR port.
4b -> bottom shelf (DS2246, ID16), B side CIR port

Node 2
0a -> top shelf (DS4246, ID0), B side SQR port.
3b -> bottom shelf (DS2246, ID7), A side CIR port
3a -> top shelf (DS4246, ID10), B side SQR port.
4b -> bottom shelf (DS2246, ID16), A side CIR port

The port pairs would look something like this:
0a/3b , 3a/4b, 4a/0d, 0c/3d, 3c/4d, 4c/0b

What you have will work. They are plugged into different adapters on
totally different ASICs.
If you want to the message(s) to stop, you would need to cable a bit
differently.

--tmac

*Tim McCarthy, **Principal Consultant*

*Proud Member of the #NetAppATeam <https://twitter.com/NetAppATeam>*

*I Blog at TMACsRack <https://tmacsrack.wordpress.com/>*




On Tue, Apr 16, 2019 at 2:29 PM John Stoffel <john@stoffel.org> wrote:

>
> Tim> There was a bug about this too. As long as the ports ( a/b ) are
> Tim> not used to the same stack and ports (c/d) are not used on the
> Tim> same stack you are most of the way there.
>
> Hi Tim,
> The suprising thing about these messages is that the system has been
> in this configuration for the past six years, without any shelf
> changes at all. I did look at the "sasadmin expander_map" output and
> it's all looking good from what I see. I appended it to the end of
> this email.
>
> Tim> Send the node shell output of
> Tim> sadadmin expander_map
>
> Tim> That would say a lot.
>
>
>
> Expanders on channel 0a:
> Cannot Complete operation on channel 0a; Status Not Available.
>
> Expanders on channel 0b:
> Level 1: WWN 500a0980017df3bf, ID 0, Serial Number '
> SHJ0000000159E4', Product 'DS424IOM6 ', Rev '0191', Slot A
> Level 2: WWN 500a09800187e0ff, ID 1, Serial Number '
> 6000557869', Product 'DS224IOM6 ', Rev '0191', Slot A
> Level 3: WWN 500a09800187b3ff, ID 2, Serial Number '
> 6000557833', Product 'DS224IOM6 ', Rev '0191', Slot A
> Level 4: WWN 500a098001a1207f, ID 3, Serial Number '
> 6000557821', Product 'DS224IOM6 ', Rev '0191', Slot A
> Level 5: WWN 500a098001a15cff, ID 4, Serial Number '
> 6000557819', Product 'DS224IOM6 ', Rev '0191', Slot A
> Level 6: WWN 500a098001a11e7f, ID 5, Serial Number '
> 6000557807', Product 'DS224IOM6 ', Rev '0191', Slot A
> Level 7: WWN 500a09800196fe7f, ID 6, Serial Number '
> 6000521559', Product 'DS224IOM6 ', Rev '0191', Slot A
> Level 8: WWN 500a098001a3cfff, ID 7, Serial Number '
> 6000558124', Product 'DS224IOM6 ', Rev '0191', Slot A
>
> Expanders on channel 3a:
> Level 1: WWN 500a0980016f3c7f, ID 10, Serial Number '
> SHJ000000015A3B', Product 'DS424IOM6 ', Rev '0191', Slot A
> Level 2: WWN 500a098001976d3f, ID 11, Serial Number '
> 6000521420', Product 'DS224IOM6 ', Rev '0191', Slot A
> Level 3: WWN 500a098001a3a2ff, ID 12, Serial Number '
> 6000558382', Product 'DS224IOM6 ', Rev '0191', Slot A
> Level 4: WWN 500a098001a3c43f, ID 13, Serial Number '
> 6000557584', Product 'DS224IOM6 ', Rev '0191', Slot A
> Level 5: WWN 500a098001a3b67f, ID 14, Serial Number '
> 6000558069', Product 'DS224IOM6 ', Rev '0191', Slot A
> Level 6: WWN 500a098001a3c1ff, ID 15, Serial Number '
> 6000557613', Product 'DS224IOM6 ', Rev '0191', Slot A
> Level 7: WWN 500a098001a3bb7f, ID 16, Serial Number '
> 6000558083', Product 'DS224IOM6 ', Rev '0191', Slot A
>
> Expanders on channel 3b:
> Level 1: WWN 500a098001a3cd7f, ID 7, Serial Number '
> 6000558124', Product 'DS224IOM6 ', Rev '0191', Slot B
> Level 2: WWN 500a09800197263f, ID 6, Serial Number '
> 6000521559', Product 'DS224IOM6 ', Rev '0191', Slot B
> Level 3: WWN 500a098001a158bf, ID 5, Serial Number '
> 6000557807', Product 'DS224IOM6 ', Rev '0191', Slot B
> Level 4: WWN 500a098001a15aff, ID 4, Serial Number '
> 6000557819', Product 'DS224IOM6 ', Rev '0191', Slot B
> Level 5: WWN 500a098001a104ff, ID 3, Serial Number '
> 6000557821', Product 'DS224IOM6 ', Rev '0191', Slot B
> Level 6: WWN 500a09800187b63f, ID 2, Serial Number '
> 6000557833', Product 'DS224IOM6 ', Rev '0191', Slot B
> Level 7: WWN 500a098001880c7f, ID 1, Serial Number '
> 6000557869', Product 'DS224IOM6 ', Rev '0191', Slot B
> Level 8: WWN 500a0980017df53f, ID 0, Serial Number '
> SHJ0000000159E4', Product 'DS424IOM6 ', Rev '0191', Slot B
>
> Expanders on channel 3c:
> Cannot Complete operation on channel 3c; Status Not Available.
>
> Expanders on channel 3d:
> Cannot Complete operation on channel 3d; Status Not Available.
>
> Expanders on channel 4a:
> Level 1: WWN 500a098001a0e8ff, ID 16, Serial Number '
> 6000558083', Product 'DS224IOM6 ', Rev '0191', Slot B
> Level 2: WWN 500a098001a3c4ff, ID 15, Serial Number '
> 6000557613', Product 'DS224IOM6 ', Rev '0191', Slot B
> Level 3: WWN 500a098001a3b77f, ID 14, Serial Number '
> 6000558069', Product 'DS224IOM6 ', Rev '0191', Slot B
> Level 4: WWN 500a098001a3c23f, ID 13, Serial Number '
> 6000557584', Product 'DS224IOM6 ', Rev '0191', Slot B
> Level 5: WWN 500a098001a3a3bf, ID 12, Serial Number '
> 6000558382', Product 'DS224IOM6 ', Rev '0191', Slot B
> Level 6: WWN 500a098001976ebf, ID 11, Serial Number '
> 6000521420', Product 'DS224IOM6 ', Rev '0191', Slot B
> Level 7: WWN 500a0980017f43ff, ID 10, Serial Number '
> SHJ000000015A3B', Product 'DS424IOM6 ', Rev '0191', Slot B
>
> Expanders on channel 4b:
> Cannot Complete operation on channel 4b; Status Not Available.
>
> Expanders on channel 4c:
> Cannot Complete operation on channel 4c; Status Not Available.
>
> Expanders on channel 4d:
> Cannot Complete operation on channel 4d; Status Not Available.
>

A big thanks to Jeff for talking with me offline and giving a bunch of
suggestions, and for Tim for noticing from the output of the 'sasadmin
expander_map' command that I've got some mis-wiring in my setup.
Which is strange because we haven't re-wired this system in ages, not
since we moved it across country three or four years ago. And the
warnings havent come back again either. So far. So I think I'll just
table this for now and maybe think about re-wiring it all when I'm
next on site and have the downtime.

Thanks again!
John
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters