Mailing List Archive

Be sure on your distances.
I think you are allowed up to 300M on a fiber from filer to switch for 10G.
If you are using patch panels, that length drops.

If your NFS is only vmware datastores, just create a new datastore and
storage vmotion. Dont overthink it

If your NFS/CIFS is normal user data, Then you should be able to do a "vol
move" from the old AFF to the new AFF and then provided networking is the
same, you can also modify the home-node/port of the LIF and move it to the
A300s with nearly no disruption.

For iSCSI, if you have full multipathing going, and the networking is the
same on the old/new aff, then you could (ONE AT A TIME):
set adv
net int mod -vserv xxx -lif iscsi_lifa -status-admin down
net int mod vserv xxx -lif iscsi_lifa -home-node a300-a -home-port new-port
net int mod -vserv xxx -lif iscsi_lifa -status-admin up
set admin
WAIT 8 minutes, let multipathing stabilize. Verify at the hosts all paths
up. Do the next one.

--tmac

*Tim McCarthy, **Principal Consultant*

*Proud Member of the #NetAppATeam <https://twitter.com/NetAppATeam>*

*I Blog at TMACsRack <https://tmacsrack.wordpress.com/>*



On Tue, Oct 13, 2020 at 7:40 AM Heino Walther <hw@beardmann.dk> wrote:

> Hi there
>
>
>
> We have to migrate our existing AFF8080 which has NFSv3, ISCSI and CIFS
> data on it to a new location.
>
> The plan is to add a temp A300 in the new location, and add it to the
> AFF8080’s cluster with two cluster switches (they are sub 200M apart)
>
> ISCSI and CIFS should be fairly simple to migrate, but NFSv3 is
> IP-centric, so as we start the migration volume by volume, the data path
> for some of our vmware host will be extended somewhat…
>
> It will go via the AFF8080’s configured IP address over the cluster
> network to the A300 controller that holds the volume, and back again.
>
> Has anyone tried this with success?
>
> Of cause another way would be to migrate the VMs from the VMWare side,
> which I would like to avoid if possible, but I am a bit worried about the
> added latency over the cluster network…
>
>
>
> Should I be worried?
>
>
>
> /Heino
>
>
>
>
> _______________________________________________
> Toasters mailing list
> Toasters@teaparty.net
> https://www.teaparty.net/mailman/listinfo/toasters

The length is not the issue.

The issue is that we have 4 large datastores (20TB+) that needs to be migrated… they all have snapmirror relations attached, so I would rather not use vmotion to a new datastore as I will have to reseed all the snapmirrors and because the destination is rotating aggregates we have no cross-volume dedupe benefits. So… vmotion will only be used if everything else doesn’t work ????

The problem of cause is that we will end up in a situation where data from the NFS datastores will be pulled via one node across the intercluster links to the node that holds the active volume and back again…

I seem to remember that you can do a “vol move” and hold the cut-over process…. I was thinking of doing the initial vol move (basically snapmirror) of the larger volumes, and then do the cut-over of all the volumes and move the LIF as one big “off-hours” process ????. Just to be sure…

I think I remember issues while running NFS over the cluster interconnect… basically high latency.

CIFS should not be a problem, and also ISCSI can be moved, just by adding LIFS to the target nodes, and MPIO will take care of the rest… (this is mainly single servers with LUNs for SQL Server etc..)

But has anyone tried to run NFS datastores that uses the intercluster link? Hos severe is the latency? Would it make sense to add 4 intercluster links per node rather than two?

/Heino


Fra: tmac <tmacmd@gmail.com>
Dato: tirsdag den 13. oktober 2020 kl. 14.01
Til: Heino Walther <hw@beardmann.dk>
Cc: "toasters@teaparty.net" <toasters@teaparty.net>
Emne: Re: Migration to new cluster nodes...

Be sure on your distances.
I think you are allowed up to 300M on a fiber from filer to switch for 10G. If you are using patch panels, that length drops.

If your NFS is only vmware datastores, just create a new datastore and storage vmotion. Dont overthink it

If your NFS/CIFS is normal user data, Then you should be able to do a "vol move" from the old AFF to the new AFF and then provided networking is the same, you can also modify the home-node/port of the LIF and move it to the A300s with nearly no disruption.

For iSCSI, if you have full multipathing going, and the networking is the same on the old/new aff, then you could (ONE AT A TIME):
set adv
net int mod -vserv xxx -lif iscsi_lifa -status-admin down
net int mod vserv xxx -lif iscsi_lifa -home-node a300-a -home-port new-port
net int mod -vserv xxx -lif iscsi_lifa -status-admin up
set admin
WAIT 8 minutes, let multipathing stabilize. Verify at the hosts all paths up. Do the next one.

--tmac

Tim McCarthy, Principal Consultant

Proud Member of the #NetAppATeam<https://twitter.com/NetAppATeam>

I Blog at TMACsRack<https://tmacsrack.wordpress.com/>



On Tue, Oct 13, 2020 at 7:40 AM Heino Walther <hw@beardmann.dk<mailto:hw@beardmann.dk>> wrote:
Hi there

We have to migrate our existing AFF8080 which has NFSv3, ISCSI and CIFS data on it to a new location.
The plan is to add a temp A300 in the new location, and add it to the AFF8080’s cluster with two cluster switches (they are sub 200M apart)
ISCSI and CIFS should be fairly simple to migrate, but NFSv3 is IP-centric, so as we start the migration volume by volume, the data path for some of our vmware host will be extended somewhat…
It will go via the AFF8080’s configured IP address over the cluster network to the A300 controller that holds the volume, and back again.
Has anyone tried this with success?
Of cause another way would be to migrate the VMs from the VMWare side, which I would like to avoid if possible, but I am a bit worried about the added latency over the cluster network…

Should I be worried?

/Heino


_______________________________________________
Toasters mailing list
Toasters@teaparty.net<mailto:Toasters@teaparty.net>
https://www.teaparty.net/mailman/listinfo/toasters

I run over the intercluster quite frequently. But the only time I have ever seen a problem with latency was when the lif was hosted on a FAS and the volume was on an AFF. In those cases I will get double digit latency.

I use NFSv3 for most of my workloads.

The process you describe, using a delayed vol move, will work just fine.

Wayne

From: Toasters <toasters-bounces@teaparty.net> On Behalf Of Heino Walther
Sent: Tuesday, October 13, 2020 8:46 AM
To: tmac <tmacmd@gmail.com>
Cc: toasters@teaparty.net
Subject: Re: Migration to new cluster nodes...


CAUTION: This email is from an external source. Do not click links or open attachments unless you recognize the sender and know the content is safe.
The length is not the issue.

The issue is that we have 4 large datastores (20TB+) that needs to be migrated… they all have snapmirror relations attached, so I would rather not use vmotion to a new datastore as I will have to reseed all the snapmirrors and because the destination is rotating aggregates we have no cross-volume dedupe benefits. So… vmotion will only be used if everything else doesn’t work ????

The problem of cause is that we will end up in a situation where data from the NFS datastores will be pulled via one node across the intercluster links to the node that holds the active volume and back again…

I seem to remember that you can do a “vol move” and hold the cut-over process…. I was thinking of doing the initial vol move (basically snapmirror) of the larger volumes, and then do the cut-over of all the volumes and move the LIF as one big “off-hours” process ????. Just to be sure…

I think I remember issues while running NFS over the cluster interconnect… basically high latency.

CIFS should not be a problem, and also ISCSI can be moved, just by adding LIFS to the target nodes, and MPIO will take care of the rest… (this is mainly single servers with LUNs for SQL Server etc..)

But has anyone tried to run NFS datastores that uses the intercluster link? Hos severe is the latency? Would it make sense to add 4 intercluster links per node rather than two?

/Heino


Fra: tmac <tmacmd@gmail.com<mailto:tmacmd@gmail.com>>
Dato: tirsdag den 13. oktober 2020 kl. 14.01
Til: Heino Walther <hw@beardmann.dk<mailto:hw@beardmann.dk>>
Cc: "toasters@teaparty.net<mailto:toasters@teaparty.net>" <toasters@teaparty.net<mailto:toasters@teaparty.net>>
Emne: Re: Migration to new cluster nodes...

Be sure on your distances.
I think you are allowed up to 300M on a fiber from filer to switch for 10G. If you are using patch panels, that length drops.

If your NFS is only vmware datastores, just create a new datastore and storage vmotion. Dont overthink it

If your NFS/CIFS is normal user data, Then you should be able to do a "vol move" from the old AFF to the new AFF and then provided networking is the same, you can also modify the home-node/port of the LIF and move it to the A300s with nearly no disruption.

For iSCSI, if you have full multipathing going, and the networking is the same on the old/new aff, then you could (ONE AT A TIME):
set adv
net int mod -vserv xxx -lif iscsi_lifa -status-admin down
net int mod vserv xxx -lif iscsi_lifa -home-node a300-a -home-port new-port
net int mod -vserv xxx -lif iscsi_lifa -status-admin up
set admin
WAIT 8 minutes, let multipathing stabilize. Verify at the hosts all paths up. Do the next one.

--tmac

Tim McCarthy, Principal Consultant

Proud Member of the #NetAppATeam<https://twitter.com/NetAppATeam>

I Blog at TMACsRack<https://tmacsrack.wordpress.com/>



On Tue, Oct 13, 2020 at 7:40 AM Heino Walther <hw@beardmann.dk<mailto:hw@beardmann.dk>> wrote:
Hi there

We have to migrate our existing AFF8080 which has NFSv3, ISCSI and CIFS data on it to a new location.
The plan is to add a temp A300 in the new location, and add it to the AFF8080’s cluster with two cluster switches (they are sub 200M apart)
ISCSI and CIFS should be fairly simple to migrate, but NFSv3 is IP-centric, so as we start the migration volume by volume, the data path for some of our vmware host will be extended somewhat…
It will go via the AFF8080’s configured IP address over the cluster network to the A300 controller that holds the volume, and back again.
Has anyone tried this with success?
Of cause another way would be to migrate the VMs from the VMWare side, which I would like to avoid if possible, but I am a bit worried about the added latency over the cluster network…

Should I be worried?

/Heino


_______________________________________________
Toasters mailing list
Toasters@teaparty.net<mailto:Toasters@teaparty.net>
https://www.teaparty.net/mailman/listinfo/toasters

On 2020-10-13 10:45, Heino Walther wrote:
> But has anyone tried to run NFS datastores that uses the intercluster
> link? Hos severe is the latency? Would it make sense to add 4
> intercluster links per node rather than two?

I just did a 8040 -> 8200/A220 migration doing both methods. Some were
moved with "vol move" and some were storage vMotion. Total was about 150TB.

We noted no great increased latency on the VMs that were moved, but they
were not latency sensitive workloads, and moving from HDD.

--
Jason
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
https://www.teaparty.net/mailman/listinfo/toasters

Hi Heino,

I have used this migration scenario in setups where there was a distance of
a couple of kilometers between two datacenters and did not run into any
issues, even though this was an unsupported setup. I was doing vol moves
while data serving was also progressively using more and more cluster
interconnect bandwidth.

You will still be in a supported setup so I don’t think you will run into a
lot of issues. Some things that you can take into consideration to mitigate
risks:

- you can limit the number of concurrent vol moves by lowering the vol move
governor setting - see the TR mentioned here:
https://community.netapp.com/t5/ONTAP-Discussions/How-many-vol-move-operations-can-be-active-at-same-time/td-p/129331
- you can move the most I/O bound datastores first and then once they are
cut over move over their data serving LIF(s) to limit traffic on the
cluster backend.
- you can monitor latency in your VMware environment and abort vol moves if
you are running into issues.

Lastly, maybe not all that important but you don’t want to have your
cluster switches go down while you are in your stretched 4-node cluster
setup. In my unsupported scenario we had 2 switches on each side, 4 in
total with ISLs between them. Like I said, this was not a supported setup
bit we were confident enough to get the migration done that was without a
whole lot of hassle.

Regards,
Filip




On Tue, 13 Oct 2020 at 16:48, Heino Walther <hw@beardmann.dk> wrote:

> The length is not the issue.
>
>
>
> The issue is that we have 4 large datastores (20TB+) that needs to be
> migrated… they all have snapmirror relations attached, so I would rather
> not use vmotion to a new datastore as I will have to reseed all the
> snapmirrors and because the destination is rotating aggregates we have no
> cross-volume dedupe benefits. So… vmotion will only be used if everything
> else doesn’t work ????
>
>
>
> The problem of cause is that we will end up in a situation where data from
> the NFS datastores will be pulled via one node across the intercluster
> links to the node that holds the active volume and back again…
>
>
>
> I seem to remember that you can do a “vol move” and hold the cut-over
> process…. I was thinking of doing the initial vol move (basically
> snapmirror) of the larger volumes, and then do the cut-over of all the
> volumes and move the LIF as one big “off-hours” process ????. Just to be
> sure…
>
>
>
> I think I remember issues while running NFS over the cluster interconnect…
> basically high latency.
>
>
>
> CIFS should not be a problem, and also ISCSI can be moved, just by adding
> LIFS to the target nodes, and MPIO will take care of the rest… (this is
> mainly single servers with LUNs for SQL Server etc..)
>
>
>
> But has anyone tried to run NFS datastores that uses the intercluster
> link? Hos severe is the latency? Would it make sense to add 4
> intercluster links per node rather than two?
>
>
>
> /Heino
>
>
>
>
>
> *Fra: *tmac <tmacmd@gmail.com>
> *Dato: *tirsdag den 13. oktober 2020 kl. 14.01
> *Til: *Heino Walther <hw@beardmann.dk>
> *Cc: *"toasters@teaparty.net" <toasters@teaparty.net>
> *Emne: *Re: Migration to new cluster nodes...
>
>
>
> Be sure on your distances.
>
> I think you are allowed up to 300M on a fiber from filer to switch for
> 10G. If you are using patch panels, that length drops.
>
>
>
> If your NFS is only vmware datastores, just create a new datastore and
> storage vmotion. Dont overthink it
>
>
>
> If your NFS/CIFS is normal user data, Then you should be able to do a "vol
> move" from the old AFF to the new AFF and then provided networking is the
> same, you can also modify the home-node/port of the LIF and move it to the
> A300s with nearly no disruption.
>
>
>
> For iSCSI, if you have full multipathing going, and the networking is the
> same on the old/new aff, then you could (ONE AT A TIME):
>
> set adv
>
> net int mod -vserv xxx -lif iscsi_lifa -status-admin down
>
> net int mod vserv xxx -lif iscsi_lifa -home-node a300-a -home-port
> new-port
>
> net int mod -vserv xxx -lif iscsi_lifa -status-admin up
>
> set admin
>
> WAIT 8 minutes, let multipathing stabilize. Verify at the hosts all paths
> up. Do the next one.
>
>
>
> --tmac
>
>
>
> *Tim McCarthy, **Principal Consultant*
>
> *Proud Member of the #NetAppATeam <https://twitter.com/NetAppATeam>*
>
> *I Blog at **TMACsRack <https://tmacsrack.wordpress.com/>*
>
>
>
>
>
>
>
> On Tue, Oct 13, 2020 at 7:40 AM Heino Walther <hw@beardmann.dk> wrote:
>
> Hi there
>
>
>
> We have to migrate our existing AFF8080 which has NFSv3, ISCSI and CIFS
> data on it to a new location.
>
> The plan is to add a temp A300 in the new location, and add it to the
> AFF8080’s cluster with two cluster switches (they are sub 200M apart)
>
> ISCSI and CIFS should be fairly simple to migrate, but NFSv3 is
> IP-centric, so as we start the migration volume by volume, the data path
> for some of our vmware host will be extended somewhat…
>
> It will go via the AFF8080’s configured IP address over the cluster
> network to the A300 controller that holds the volume, and back again.
>
> Has anyone tried this with success?
>
> Of cause another way would be to migrate the VMs from the VMWare side,
> which I would like to avoid if possible, but I am a bit worried about the
> added latency over the cluster network…
>
>
>
> Should I be worried?
>
>
>
> /Heino
>
>
>
>
>
> _______________________________________________
> Toasters mailing list
> Toasters@teaparty.net
> https://www.teaparty.net/mailman/listinfo/toasters
>
> _______________________________________________
> Toasters mailing list
> Toasters@teaparty.net
> https://www.teaparty.net/mailman/listinfo/toasters

Heino> We have to migrate our existing AFF8080 which has NFSv3, ISCSI
Heino> and CIFS data on it to a new location.

How many nodes in your cluster?

Heino> The plan is to add a temp A300 in the new location, and add it
Heino> to the AFF8080’s cluster with two cluster switches (they are
Heino> sub 200M apart)

That should work, then you just failover LIFs and vol move volumes.

Heino> ISCSI and CIFS should be fairly simple to migrate, but NFSv3 is
Heino> IP-centric, so as we start the migration volume by volume, the
Heino> data path for some of our vmware host will be extended
Heino> somewhat…

CIFS would be the problem child, since moving it from one node to
another is a client outage. NFSv3 shouldn't really care.

Heino> It will go via the AFF8080’s configured IP address over the
Heino> cluster network to the A300 controller that holds the volume,
Heino> and back again.

Until you move the LIF over to the A300...

Heino> Has anyone tried this with success?

Heino> Of cause another way would be to migrate the VMs from the
Heino> VMWare side, which I would like to avoid if possible, but I am
Heino> a bit worried about the added latency over the cluster network…

I assume your real goal is to move a two node cluster without any
downtime, right? If so, adding in the A300 (with enough storage!) to
your cluster should do the trick.

Then it will jsut take time to vol move the data from one system to
another, and then once the physical move is complete, you move it back
again.

I'd probably remove the AFF8080 pair from the cluster before you shut
them down and move them. Then you just power them up, re-add to the
cluster, and vol move back.

Tedious, but very doable.

_______________________________________________
Toasters mailing list
Toasters@teaparty.net
https://www.teaparty.net/mailman/listinfo/toasters

I do not think the NetApp Inter Cluster has anything to do with Cisco...

I do use two NetApp branded cisco switches for intercluster connections (cannot remember their model) the two switches are also interconnected together, but basically you just cable at least one link into each of the switches from your nodes, instead of the two small 0.5M interconnect cables that normally interconnects two HA nodes together. (there is a procedure of doing this that netapp describes... (going from switchless to switched))

I only have two switches at hand __. Do I will have to manage.. the distance is under 200M between the two DCs so I think it will work just fine.
I might link up four links from each controller as ISL... this is possible in the AFFF8080, cannot remember if we can manage this in the A300 (we also need some data ports)

/Heino


?D. 13.10.2020 18.48 skrev "Michael Bergman" <michael.bergman@ericsson.com>:

I got curious here (I'm not very good at all at Cisco related networking
stuff), ISL is the Cisco proprietary VLAN protocol (Inter-Switch Link) and
its really old. Right? It's deprecated now and not even all Cisco switches
supports it.

You set up a 4-switch "ClusterNet" for your temp stuff there, with 2x the
redundancy HW-wise, in this unsupported stretch cluster, correct?
Why did you chose to use the old ISL?

/M

On 2020-10-13 18:10, Filip Sneppe wrote:
> Lastly, maybe not all that important but you don't want to have your cluster
> switches go down while you are in your stretched 4-node cluster setup. In my
> unsupported scenario we had 2 switches on each side, 4 in total with ISLs
> between them. Like I said, this was not a supported setup bit we were
> confident enough to get the migration done that was without a whole lot of
> hassle.

--
Michael Bergman
Sr Systems Analyst / Storage Architect michael.bergman@ericsson.com
BNEW Service Area Networks, Lab Ops Phone +46 10 7152945
Global Network & Tech, Architecture SMS/MMS +46 70 5480835
Ericsson Torshamnsg 36, 16480 Sthlm, Sweden
--
This communication is confidential. We only send and receive email on the
basis of the terms set out at www.ericsson.com/email_disclaimer


_______________________________________________
Toasters mailing list
Toasters@teaparty.net
https://www.teaparty.net/mailman/listinfo/toasters

Heino> We have to migrate our existing AFF8080 which has NFSv3, ISCSI
Heino> and CIFS data on it to a new location.

How many nodes in your cluster?

A two node AFF8080HA and a two node A300 so four nodes.
So we will migrate from the AFF8080 to the A300 (which has enough storage attached to cover the AFF8080)

Heino> The plan is to add a temp A300 in the new location, and add it
Heino> to the AFF8080’s cluster with two cluster switches (they are
Heino> sub 200M apart)

That should work, then you just failover LIFs and vol move volumes.

Yes the indications I have gotten here is that I should not be so afraid about running some data via the inter cluster link... __
And come to think of it, the issues I had in the past may have been with FAS32XX nodes and spinning disks... which may explain quite a bit __
But I will try to move the larger NFS volumes in one go, I think I can have up to 7 or 8 volume moves pending cut-over at a time... so I think this will be that way...

Heino> ISCSI and CIFS should be fairly simple to migrate, but NFSv3 is
Heino> IP-centric, so as we start the migration volume by volume, the
Heino> data path for some of our vmware host will be extended
Heino> somewhat…

CIFS would be the problem child, since moving it from one node to
another is a client outage. NFSv3 shouldn't really care.

Yeah, CIFS is a pain, but again we can do cut-overs in a service window, so should not be an issue here.

Heino> It will go via the AFF8080’s configured IP address over the
Heino> cluster network to the A300 controller that holds the volume,
Heino> and back again.

Until you move the LIF over to the A300...

Heino> Has anyone tried this with success?

Heino> Of cause another way would be to migrate the VMs from the
Heino> VMWare side, which I would like to avoid if possible, but I am
Heino> a bit worried about the added latency over the cluster network…

I assume your real goal is to move a two node cluster without any
downtime, right? If so, adding in the A300 (with enough storage!) to
your cluster should do the trick.

We have the storage required on the A300 __

Then it will jsut take time to vol move the data from one system to
another, and then once the physical move is complete, you move it back
again.

I'd probably remove the AFF8080 pair from the cluster before you shut
them down and move them. Then you just power them up, re-add to the
cluster, and vol move back.

Tedious, but very doable.

Yes it's Tedious, but I think it's faster than storage vmotion... also if you have ever tried to do vmotion on larger setups, you will quickly find out just how stupid it is... just a CD mounted or whatever can stop the process... __
Also as we have existing snapmirror relations I would prefer volume move to keep the relations intact.

/Heino




_______________________________________________
Toasters mailing list
Toasters@teaparty.net
https://www.teaparty.net/mailman/listinfo/toasters