Mailing List Archive

dom0 crashes when having more than 1 vCPU
Hello!

Ubuntu 14.04, 3.13.0-141-generic #190-Ubuntu SMP Fri Jan 19 12:52:38 UTC
2018 x86_64, xen_version: 4.4.2, Server: DL380 G6 with 2x Intel E5540
(more details below)

We configured our dom0 with a single CPU:
GRUB_CMDLINE_XEN="dom0_mem=1024M,max:1024M dom0_max_vcpus=1
dom0_vcpus_pin iommu=no-intremap"

The last 5 years we did not had any single problem with Xen. Yesterday,
we had poor io-performance when using dd in the dom0 and tried if it
would be better when having more CPUs. Hence, we changed dom0_max_vcpus
to 4 vCPUs. io was much better, but today it crashed (the server was
totally unresponive, we had to power toggle it, hence, no console logs).

We saw that more than once (Ubuntu 14.04 on DL380G6 with dual CPU,
Debian 9 on DL380G7 with single CPU) that Xen locks up when we tried to
play with the number of CPUs in the dom0, or with CPU pinning.

Is there a known issue with this Xen version, this hardware, or whatever
which can cause this Xen lock ups when using multiple vCPUs?

What is the recommended setting for number of CPUs in the dom0?

Thanks
Klaus



# xl info -n
release : 3.13.0-141-generic
version : #190-Ubuntu SMP Fri Jan 19 12:52:38 UTC 2018
machine : x86_64
nr_cpus : 16
max_cpu_id : 31
nr_nodes : 2
cores_per_socket : 4
threads_per_core : 2
cpu_mhz : 2533
hw_caps :
bfebfbff:28100800:00000000:00003b00:009ce3bd:00000000:00000001:00000000
virt_caps : hvm hvm_directio
total_memory : 122869
free_memory : 104431
sharing_freed_memory : 0
sharing_used_memory : 0
outstanding_claims : 0
free_cpus : 0
cpu_topology :
cpu: core socket node
0: 0 0 0
1: 0 0 0
2: 1 0 0
3: 1 0 0
4: 2 0 0
5: 2 0 0
6: 3 0 0
7: 3 0 0
8: 0 1 1
9: 0 1 1
10: 1 1 1
11: 1 1 1
12: 2 1 1
13: 2 1 1
14: 3 1 1
15: 3 1 1
numa_info :
node: memsize memfree distances
0: 61952 56439 10,20
1: 61439 47992 20,10
xen_major : 4
xen_minor : 4
xen_extra : .2
xen_version : 4.4.2
xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32
hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler : credit
xen_pagesize : 4096
platform_params : virt_start=0xffff800000000000
xen_changeset :
xen_commandline : placeholder dom0_mem=1024M,max:1024M
dom0_max_vcpus=4 dom0_vcpus_pin iommu=no-intremap
cc_compiler : gcc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4
cc_compile_by : stefan.bader
cc_compile_domain : canonical.com
cc_compile_date : Fri Oct 13 15:48:48 UTC 2017
xend_config_format : 4

_______________________________________________
Xen-users mailing list
Xen-users@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-users
Re: dom0 crashes when having more than 1 vCPU [ In reply to ]
Hi Klaus

Op wo 8 mei 2019 om 11:52 schreef Klaus Darilion <
klaus.mailinglists@pernau.at>:

> Hello!
>
> Ubuntu 14.04, 3.13.0-141-generic #190-Ubuntu SMP Fri Jan 19 12:52:38 UTC
> 2018 x86_64, xen_version: 4.4.2, Server: DL380 G6 with 2x Intel E5540
> (more details below)
>
> We configured our dom0 with a single CPU:
> GRUB_CMDLINE_XEN="dom0_mem=1024M,max:1024M dom0_max_vcpus=1
> dom0_vcpus_pin iommu=no-intremap"
>

If the load on your server is also network related then the Dom0 needs
memory to handle network sessions. Maybe increasing the dom0_mem to 2048M
would help prevent crashes. Two cpus instead of only 1 cpu will also help
since the Dom0 must handle IO for the DomU hosts.

In addition you could also give the Domain-0 more prio for IO, so it will
not compete with DomU hosts in cpu cylces.
You can do that by increasing the sched-credits to 512 (max value) for the
Domain-0

xl sched-credit -d Domain-0 -w 512

This helped us a lot to improve overall responsiveness. The Dom0 needs to
do the real work in the end, so if it can complete its work a.s.a.p. it is
best for the performance of the DomU.



> The last 5 years we did not had any single problem with Xen. Yesterday,
> we had poor io-performance when using dd in the dom0 and tried if it
> would be better when having more CPUs. Hence, we changed dom0_max_vcpus
> to 4 vCPUs. io was much better, but today it crashed (the server was
> totally unresponive, we had to power toggle it, hence, no console logs).
>
> We saw that more than once (Ubuntu 14.04 on DL380G6 with dual CPU,
> Debian 9 on DL380G7 with single CPU) that Xen locks up when we tried to
> play with the number of CPUs in the dom0, or with CPU pinning.
>
> Is there a known issue with this Xen version, this hardware, or whatever
> which can cause this Xen lock ups when using multiple vCPUs?
>
> What is the recommended setting for number of CPUs in the dom0?
>
> Thanks
> Klaus
>


Greetings,

Roalt