Mailing List Archive

ssh <vserver> vol show <volume_full_name> hangs
Guys,
I've run into an interesting problem on a 4 node cDOT cluster running
8.3.2P9 today. Basically, I've setup a user account with access to a
particular VServer. It's used for creating snapshots and such.

When we do the command:

ssh <vserver> vol show <volume_full_name>

it just hangs, and doesn't show anything. We have to do a Ctrl-C to
get it back.

But if I simply do a simple: 'ssh <vserver> vol show' it all works
nice and quickly. So wierd. I can even do the show command when
there are multiple volumes using a wild card and it works.

ssh <vserver> vol show '<volume_*>'

works, as long as it matches MORE than just the single volume. Using
the -instance flag doesn't help either.

I even tried moving the admin lif to another node, thinking that one
node had some stuff ssh processes, but it didn't make a differnce.

Thanks,
John
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
RE: ssh <vserver> vol show <volume_full_name> hangs [ In reply to ]
Did you try adding -volume volname to the command?

So:

ssh <vserver> vol show -volume <volume_full_name>

-----Original Message-----
From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of John Stoffel
Sent: Thursday, September 21, 2017 3:09 PM
To: Toasters@teaparty.net
Subject: ssh <vserver> vol show <volume_full_name> hangs


Guys,
I've run into an interesting problem on a 4 node cDOT cluster running
8.3.2P9 today. Basically, I've setup a user account with access to a particular VServer. It's used for creating snapshots and such.

When we do the command:

ssh <vserver> vol show <volume_full_name>

it just hangs, and doesn't show anything. We have to do a Ctrl-C to get it back.

But if I simply do a simple: 'ssh <vserver> vol show' it all works nice and quickly. So wierd. I can even do the show command when there are multiple volumes using a wild card and it works.

ssh <vserver> vol show '<volume_*>'

works, as long as it matches MORE than just the single volume. Using the -instance flag doesn't help either.

I even tried moving the admin lif to another node, thinking that one node had some stuff ssh processes, but it didn't make a differnce.

Thanks,
John
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
RE: ssh <vserver> vol show <volume_full_name> hangs [ In reply to ]
Yup, tried that as well. Really quite strange. I suspect it's a
stuck ssh process/thread inside onTap somewhere. But who knows?

Justin> Did you try adding -volume volname to the command?
Justin> So:

Justin> ssh <vserver> vol show -volume <volume_full_name>

Justin> -----Original Message-----
Justin> From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of John Stoffel
Justin> Sent: Thursday, September 21, 2017 3:09 PM
Justin> To: Toasters@teaparty.net
Justin> Subject: ssh <vserver> vol show <volume_full_name> hangs


Justin> Guys,
Justin> I've run into an interesting problem on a 4 node cDOT cluster running
Justin> 8.3.2P9 today. Basically, I've setup a user account with access to a particular VServer. It's used for creating snapshots and such.

Justin> When we do the command:

Justin> ssh <vserver> vol show <volume_full_name>

Justin> it just hangs, and doesn't show anything. We have to do a Ctrl-C to get it back.

Justin> But if I simply do a simple: 'ssh <vserver> vol show' it all works nice and quickly. So wierd. I can even do the show command when there are multiple volumes using a wild card and it works.

Justin> ssh <vserver> vol show '<volume_*>'

Justin> works, as long as it matches MORE than just the single volume. Using the -instance flag doesn't help either.

Justin> I even tried moving the admin lif to another node, thinking that one node had some stuff ssh processes, but it didn't make a differnce.

Justin> Thanks,
Justin> John
Justin> _______________________________________________
Justin> Toasters mailing list
Justin> Toasters@teaparty.net
Justin> http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
Re: ssh <vserver> vol show <volume_full_name> hangs [ In reply to ]
"ssh -vvv ... " should output some more details from client side perspective.

On Thu, Sep 21, 2017 at 9:46 PM, John Stoffel <john@stoffel.org> wrote:
>
> Yup, tried that as well. Really quite strange. I suspect it's a
> stuck ssh process/thread inside onTap somewhere. But who knows?
>
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
Re: ssh <vserver> vol show <volume_full_name> hangs [ In reply to ]
>>>>> "Momonth" == Momonth <momonth@gmail.com> writes:

Nope, nothing really shows up beyond a timeout. I even created a new
user (and reused the RSA key) on that VServer and it too has the same
problem. It just times out eventually. I've got a ticket open with
Netapp on this, we'll see what they say.

Momonth> "ssh -vvv ... " should output some more details from client side perspective.


Momonth> On Thu, Sep 21, 2017 at 9:46 PM, John Stoffel <john@stoffel.org> wrote:
>>
>> Yup, tried that as well. Really quite strange. I suspect it's a
>> stuck ssh process/thread inside onTap somewhere. But who knows?
>>
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
RE: ssh <vserver> vol show <volume_full_name> hangs [ In reply to ]
Well, just a quick update on my 'ssh <vserver> vol show <vol>' hangs
we've been seeing under 8.3.2P9, which is basically nothing. I've
been asked if I could try a TO/GB to see if that fixes anything.

At this point, if I'm going to do that, I may as well goto 8.3.2P12 or
even jump to 9.1P7 since (amazingly!) the upgrade planning docs are
something like 9 and 13 pages for each release. Per-node of course.

Wow... the new upgrade process looks *really* good, assuming that CIFS
sessions won't get too hosed.

More when I know more.
John
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters
RE: ssh <vserver> vol show <volume_full_name> hangs [ In reply to ]
>>>>> "John" == John Stoffel <john@stoffel.org> writes:

John> Well, just a quick update on my 'ssh <vserver> vol show <vol>'
John> hangs we've been seeing under 8.3.2P9, which is basically
John> nothing. I've been asked if I could try a TO/GB to see if that
John> fixes anything.

Well, it looks like it's a bug in the Solaris Sparc 11.3 version of
SSH somewhere. The command works properly from RHEL 6.6 x86_64
systems, and older Sun Sparc systems.

And it's even a problem with OpenSSH on Solaris 11.3 as
well... dammit. I can't make it work reliably with

root@lnd01:~# ssh -V
OpenSSH_7.2p2, OpenSSL 1.0.2j 26 Sep 2016

This is after using the command:

# pkg set-mediator -I openssh -V openssh ssh

on the Solaris server to change the defaults to use a different version
of ssh... So something on Solaris sucks.

John

_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters