Mailing List Archive

IO flow on multi-node setup
Hello,

I have a requirement to create a multi node setup with Controller, Compute and Storage nodes on different machines.

In case I create a virtual machine on the compute node, it's going to be provisioned by the Controller node and the storage is going to be provided by the Storage node. In case I require any more volumes, the same are going to be provisioned by the Storage node, and then can be attached to the VM.

I wanted to know will the IO requests from Compute to Storage node go via Controller or will these be direct? In case it's the former, how much performance impact will be there due to the added redirection?

Thanks,
Tushar

_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Re: IO flow on multi-node setup [ In reply to ]
Volume I/O won't flow through the controller. The LVM reference driver,
for example, sets up an iSCSI target on the storage node, which the
compute node then imports. The NFS driver has the compute node mount the
exported filesystem. Depending on the driver, I/O may even flow directly
between an iSCSI or fibre-channel disk array and the compute node.

What storage backend(s) are you deploying?

On 10/26/2018 6:50 PM, Tushar Tyagi wrote:
> Hello,
>
> I have a requirement to create a multi node setup with Controller, Compute and Storage nodes on different machines.
>
> In case I create a virtual machine on the compute node, it's going to be provisioned by the Controller node and the storage is going to be provided by the Storage node. In case I require any more volumes, the same are going to be provisioned by the Storage node, and then can be attached to the VM.
>
> I wanted to know will the IO requests from Compute to Storage node go via Controller or will these be direct? In case it's the former, how much performance impact will be there due to the added redirection?
>
> Thanks,
> Tushar
>
> _______________________________________________
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to : openstack@lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Re: IO flow on multi-node setup [ In reply to ]
I am using Rocky release and configuring for NVMe backend with LVM. Can you please tell me how this one might flow, or redirect me to the documentation?
I have not been able to found much documentation regarding this.

--
Tushar Tyagi
ttml@fastmail.com

On Fri, Oct 26, 2018, at 17:48, Bernd Bausch wrote:
> Volume I/O won't flow through the controller. The LVM reference driver,
> for example, sets up an iSCSI target on the storage node, which the
> compute node then imports. The NFS driver has the compute node mount the
> exported filesystem. Depending on the driver, I/O may even flow directly
> between an iSCSI or fibre-channel disk array and the compute node.
>
> What storage backend(s) are you deploying?
>
> On 10/26/2018 6:50 PM, Tushar Tyagi wrote:
> > Hello,
> >
> > I have a requirement to create a multi node setup with Controller, Compute and Storage nodes on different machines.
> >
> > In case I create a virtual machine on the compute node, it's going to be provisioned by the Controller node and the storage is going to be provided by the Storage node. In case I require any more volumes, the same are going to be provisioned by the Storage node, and then can be attached to the VM.
> >
> > I wanted to know will the IO requests from Compute to Storage node go via Controller or will these be direct? In case it's the former, how much performance impact will be there due to the added redirection?
> >
> > Thanks,
> > Tushar
> >
> > _______________________________________________
> > Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> > Post to : openstack@lists.openstack.org
> > Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>
>
> Email had 1 attachment:
> + signature.asc
> 1k (application/pgp-signature)

_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Re: IO flow on multi-node setup [ In reply to ]
You configure an LVM volume group for each Cinder backend. A CInder
volume on that backend corresponds to an LVM volume in that volume
group. The LVM volume is exposed as an iSCSI target, usually via LIO,
but I believe this is configurable. The compute node uses /iscsiadm /to
import that volume as a SCSI device and connects that device to the VM.

I don't know if this is documented anywhere, but it's easy and fun to
explore. On the storage node, use LVM commands to look into volume
groups and volumes, and the /tgtadm /command to check targets. On the
compute node, list all SCSI disks, use /iscsiadm -m session /to check
iSCSI sessions, and use /virsh domblklist /and /virsh edit /(assuming
your hypervisor is libvirt-controlled) to confirm that the iSCSI LUN is
connected to the VM.

On 10/26/2018 10:03 PM, Tushar Tyagi wrote:
> I am using Rocky release and configuring for NVMe backend with LVM. Can you please tell me how this one might flow, or redirect me to the documentation?
> I have not been able to found much documentation regarding this.
>