Mailing List Archive

On Wed, Apr 27, 2022 at 04:52:24AM +0000, Eric Robinson wrote:
> I've read through the Linstor User Guide and looked at Linstor's many
> help screens, but I don't see an answer to this question. We just
> added a new physical disk to our cluster nodes. What linstor command
> adds them to existing storage pools? Or am I supposed to add them
> manually to the underlying LVM volume group outside of Linstor?

yes.

Regards, rck
_______________________________________________
Star us on GITHUB: https://github.com/LINBIT
drbd-user mailing list
drbd-user@lists.linbit.com
https://lists.linbit.com/mailman/listinfo/drbd-user

> -----Original Message-----
> From: drbd-user-bounces@lists.linbit.com <drbd-user-
> bounces@lists.linbit.com> On Behalf Of Roland Kammerer
> Sent: Wednesday, April 27, 2022 2:07 AM
> To: drbd-user@lists.linbit.com
> Subject: Re: [DRBD-user] Adding a Physical Disk to a Storage Pool
>
> On Wed, Apr 27, 2022 at 04:52:24AM +0000, Eric Robinson wrote:
> > I've read through the Linstor User Guide and looked at Linstor's many
> > help screens, but I don't see an answer to this question. We just
> > added a new physical disk to our cluster nodes. What linstor command
> > adds them to existing storage pools? Or am I supposed to add them
> > manually to the underlying LVM volume group outside of Linstor?
>
> yes.

I did that, and now vgdisplay shows that the volume group's size has increased by the expected amount, but the storage does not show in linstor. The total and free capacities on the pools have not changed.

[root@ha50a sa]# linstor physical-storage list
?????????????????????????????
? Size ? Rotational ? Nodes ?
?????????????????????????????
?????????????????????????????
[root@ha50a sa]# linstor storage-pool l
????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
? StoragePool ? Node ? Driver ? PoolName ? FreeCapacity ? TotalCapacity ? CanSnapshots ? State ? SharedName ?
????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
? DfltDisklessStorPool ? ha50a-cl ? DISKLESS ? ? ? ? False ? Ok ? ?
? DfltDisklessStorPool ? ha50b-cl ? DISKLESS ? ? ? ? False ? Ok ? ?
? DfltDisklessStorPool ? ha51a-cl ? DISKLESS ? ? ? ? False ? Ok ? ?
? DfltDisklessStorPool ? ha51b-cl ? DISKLESS ? ? ? ? False ? Ok ? ?
? DfltDisklessStorPool ? quorum01 ? DISKLESS ? ? ? ? False ? Ok ? ?
? spool0 ? ha50a-cl ? LVM ? vg0 ? 4.62 TiB ? 17.47 TiB ? False ? Ok ? ?
? spool0 ? ha50b-cl ? LVM ? vg0 ? 4.62 TiB ? 17.47 TiB ? False ? Ok ? ?
? spool1 ? ha51a-cl ? LVM ? vg0 ? 3.35 TiB ? 17.47 TiB ? False ? Ok ? ?
? spool1 ? ha51b-cl ? LVM ? vg0 ? 3.34 TiB ? 17.47 TiB ? False ? Ok ? ?
?????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????

>
> Regards, rck
> _______________________________________________
> Star us on GITHUB: https://github.com/LINBIT drbd-user mailing list drbd-
> user@lists.linbit.com https://lists.linbit.com/mailman/listinfo/drbd-user
Disclaimer : This email and any files transmitted with it are confidential and intended solely for intended recipients. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of Physician Select Management. Warning: Although Physician Select Management has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments.
_______________________________________________
Star us on GITHUB: https://github.com/LINBIT
drbd-user mailing list
drbd-user@lists.linbit.com
https://lists.linbit.com/mailman/listinfo/drbd-user

On Thu, Apr 28, 2022 at 03:00:35AM +0000, Eric Robinson wrote:
> > -----Original Message-----
> > From: drbd-user-bounces@lists.linbit.com <drbd-user-
> > bounces@lists.linbit.com> On Behalf Of Roland Kammerer
> > Sent: Wednesday, April 27, 2022 2:07 AM
> > To: drbd-user@lists.linbit.com
> > Subject: Re: [DRBD-user] Adding a Physical Disk to a Storage Pool
> >
> > On Wed, Apr 27, 2022 at 04:52:24AM +0000, Eric Robinson wrote:
> > > I've read through the Linstor User Guide and looked at Linstor's many
> > > help screens, but I don't see an answer to this question. We just
> > > added a new physical disk to our cluster nodes. What linstor command
> > > adds them to existing storage pools? Or am I supposed to add them
> > > manually to the underlying LVM volume group outside of Linstor?
> >
> > yes.
>
> I did that, and now vgdisplay shows that the volume group's size has
> increased by the expected amount, but the storage does not show in
> linstor. The total and free capacities on the pools have not changed.

not a LINSTOR dev, but AFAIK it only updates the information if it has
to (and maybe periodically?), like when it needs to create a new
resource. The easiest is to just 'systemctl restart linstor-satellite'
on that host. There should also be some reconnect command in the client
IIRC.

Regards, rck
_______________________________________________
Star us on GITHUB: https://github.com/LINBIT
drbd-user mailing list
drbd-user@lists.linbit.com
https://lists.linbit.com/mailman/listinfo/drbd-user

> -----Original Message-----
> From: drbd-user-bounces@lists.linbit.com <drbd-user-
> bounces@lists.linbit.com> On Behalf Of Roland Kammerer
> Sent: Thursday, April 28, 2022 1:41 AM
> To: drbd-user@lists.linbit.com
> Subject: Re: [DRBD-user] Adding a Physical Disk to a Storage Pool
>
> On Thu, Apr 28, 2022 at 03:00:35AM +0000, Eric Robinson wrote:
> > > -----Original Message-----
> > > From: drbd-user-bounces@lists.linbit.com <drbd-user-
> > > bounces@lists.linbit.com> On Behalf Of Roland Kammerer
> > > Sent: Wednesday, April 27, 2022 2:07 AM
> > > To: drbd-user@lists.linbit.com
> > > Subject: Re: [DRBD-user] Adding a Physical Disk to a Storage Pool
> > >
> > > On Wed, Apr 27, 2022 at 04:52:24AM +0000, Eric Robinson wrote:
> > > > I've read through the Linstor User Guide and looked at Linstor's
> > > > many help screens, but I don't see an answer to this question. We
> > > > just added a new physical disk to our cluster nodes. What linstor
> > > > command adds them to existing storage pools? Or am I supposed to
> > > > add them manually to the underlying LVM volume group outside of
> Linstor?
> > >
> > > yes.
> >
> > I did that, and now vgdisplay shows that the volume group's size has
> > increased by the expected amount, but the storage does not show in
> > linstor. The total and free capacities on the pools have not changed.
>
> not a LINSTOR dev, but AFAIK it only updates the information if it has to (and
> maybe periodically?), like when it needs to create a new resource. The
> easiest is to just 'systemctl restart linstor-satellite'
> on that host. There should also be some reconnect command in the client
> IIRC.
>

There are about 200 resources between the two nodes. 'systemctl restart linstor-satellite' does not touch the running resources or disrupt service, right?

> Regards, rck
> _______________________________________________
> Star us on GITHUB: https://github.com/LINBIT drbd-user mailing list drbd-
> user@lists.linbit.com https://lists.linbit.com/mailman/listinfo/drbd-user
Disclaimer : This email and any files transmitted with it are confidential and intended solely for intended recipients. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of Physician Select Management. Warning: Although Physician Select Management has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments.
_______________________________________________
Star us on GITHUB: https://github.com/LINBIT
drbd-user mailing list
drbd-user@lists.linbit.com
https://lists.linbit.com/mailman/listinfo/drbd-user

On Thu, Apr 28, 2022 at 02:41:53PM +0000, Eric Robinson wrote:
> There are about 200 resources between the two nodes. 'systemctl
> restart linstor-satellite' does not touch the running resources or
> disrupt service, right?

There are no interruptions. That is the lovely part when control and
data plane are separated.

Regards, rck
_______________________________________________
Star us on GITHUB: https://github.com/LINBIT
drbd-user mailing list
drbd-user@lists.linbit.com
https://lists.linbit.com/mailman/listinfo/drbd-user

> -----Original Message-----
> From: drbd-user-bounces@lists.linbit.com <drbd-user-
> bounces@lists.linbit.com> On Behalf Of Roland Kammerer
> Sent: Thursday, April 28, 2022 9:47 AM
> To: drbd-user@lists.linbit.com
> Subject: Re: [DRBD-user] Adding a Physical Disk to a Storage Pool
>
> On Thu, Apr 28, 2022 at 02:41:53PM +0000, Eric Robinson wrote:
> > There are about 200 resources between the two nodes. 'systemctl
> > restart linstor-satellite' does not touch the running resources or
> > disrupt service, right?
>
> There are no interruptions. That is the lovely part when control and data
> plane are separated.
>

Turns out there was no need. I added a small resource as a test and the storage pool numbers now show correctly. You were right, thanks!

This does raise a related question, though. I only recently read the part of the Linstor User Guide where it talks about confining failure domains by creating one storage pool for each backend device (section 1.10.1). I wish I had known that sooner. When we created our volume groups, we did it like this...

# vgcreate vg0 /dev/nvme0n1 /dev/nvme1n1 /dev/nvme2n1 /dev/nvme3n1 /dev/nvme4n1 /dev/nvme5n1

It's all one big volume group and we let Linstor provision resources wherever it wants. What is the effect If one NVME drive fails? Do all resources on the node go diskless, or just resources that happen to live entirely on the failed drive?

> Regards, rck
> _______________________________________________
> Star us on GITHUB: https://github.com/LINBIT drbd-user mailing list drbd-
> user@lists.linbit.com https://lists.linbit.com/mailman/listinfo/drbd-user
Disclaimer : This email and any files transmitted with it are confidential and intended solely for intended recipients. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of Physician Select Management. Warning: Although Physician Select Management has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments.
_______________________________________________
Star us on GITHUB: https://github.com/LINBIT
drbd-user mailing list
drbd-user@lists.linbit.com
https://lists.linbit.com/mailman/listinfo/drbd-user

> # vgcreate vg0 /dev/nvme0n1 /dev/nvme1n1 /dev/nvme2n1 /dev/nvme3n1
> /dev/nvme4n1 /dev/nvme5n1
>
> It's all one big volume group and we let Linstor provision resources
> wherever it wants. What is the effect If one NVME drive fails? Do all
> resources on the node go diskless, or just resources that happen to live
> entirely on the failed drive?
>

I am not an LVM expert, but to my understanding, LVM (by default) just uses
the devices as they come / are needed in your setup. That means, if one
device fails, those volumes that are at least partially placed on the
failed device will be in trouble (I assume IO errors when accessing them).
I am not sure how LVM behaves if a part of a volume is accessed which is on
a healthy disk... but you should be able to try such things by yourself
using loop devices as PVs.

--
Best regards,
Gabor Hernadi