Mailing List Archive: Live migration of VM on 10Gb network

Hi all,

I'm trying to do some experiments about checking the speed of live
migration between two host servers.
Each server contains two NICs, one is 1Gb and the other one is 10Gb. There
are also two xenbridge on each NIC.

At first, I tested the live migration of 8GB memory VM on 1Gb and the
result was 100~120MB/s. I also tried to use only 10Gb ethernet, however,
the result was the same as around 100~120MB/s.
I expected that doing live migration by using 10Gb will be much faster, but
it was the same as 1Gb. I'm not sure it is the right result.

I searched for some information about live migration, and I saw source
codes that "xl migrate" uses "ssh". Is it the bottleneck of slow live
migration?
I'm not sure if I have wrong configuration on networking.

I used Xen 4.7, and the command was "xl migrate <domain> <host>" same as
default.

Thanks!
?

On Fri, Jan 10, 2020 at 1:37 AM Taeklim Kim <limkim4233@gmail.com> wrote:

> Hi all,
>
> I'm trying to do some experiments about checking the speed of live
> migration between two host servers.
> Each server contains two NICs, one is 1Gb and the other one is 10Gb. There
> are also two xenbridge on each NIC.
>
> At first, I tested the live migration of 8GB memory VM on 1Gb and the
> result was 100~120MB/s. I also tried to use only 10Gb ethernet, however,
> the result was the same as around 100~120MB/s.
> I expected that doing live migration by using 10Gb will be much faster,
> but it was the same as 1Gb. I'm not sure it is the right result.
>
> I searched for some information about live migration, and I saw source
> codes that "xl migrate" uses "ssh". Is it the bottleneck of slow live
> migration?
> I'm not sure if I have wrong configuration on networking.
>
> I used Xen 4.7, and the command was "xl migrate <domain> <host>" same as
> default.
>
>

I have not seen a reply on this list in a very long time so I am going to
jump in:

https://xcp-ng.org/forum/topic/2283/faster-10gbit/16?_=1578676058390

I am working a similar issue in xcp-ng. So not vanilla Xen, but I see the
same stuff that you are talking about.

The nic drivers will use cpu, I think pure PV bypasses this. I have a
research project going to try, and fix this/figure out what is going on.

The last time I messed around w/ nic speeds it was in proxmox, so not xen
but still virtualized.

I did not even know that Xen supports xl heh.